Subject: Re: [ippm] Vote at IPPM session

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Fri, 7 Apr 2017 01:57:55 +0000

The new Scope+ section meets my needs,
it will be good to hear from those who
hummed against this work.

Al

> -----Original Message-----
> From: Frank Brockners (fbrockne) [mailto:[email-masked]]
> Sent: Friday, March 31, 2017 10:28 AM
> To: MORTON, ALFRED C (AL); [email-masked]; 'Brian Trammell (IETF)'
> Cc: 'IPPM Chairs'; [email-masked]
> Subject: RE: [ippm] Vote at IPPM session
> 
> Thanks for the suggestions and comments. We've posted a new revision of
> draft-brockners-inband-oam-data.
> 
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_id_draft-2Dbrockners-2Dinband-2Doam-2Ddata-
> 2D04.txt&d=DwIGaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=ootSI9_CjEColrhCFKEVPg5PZ77
> NOEMgztsEAI0kZFw&s=C4wSwZi8cUmFfYjfnUkfF7qrJTQW5h_SNJgt9GW-R-Y&e=
> includes section 3 on Scope, Applicability, and Assumptions, capturing
> the discussion we had in the WG meeting and on the list. We've also
> updated the introduction to refer to RFC7799 and classify in-situ OAM
> appropriately as hybrid, type-1 OAM.
> 
> 
> 
> For everyone's benefit, here is a quote of the new section 3:
> 
> 
> 
> 3.  Scope, Applicability, and Assumptions
> 
> 
> 
>    In-situ OAM deployment assumes a set of contraints, requirements, and
> 
>    guiding principles which are described in this section.
> 
> 
> 
>    Scope: This document defines the data fields and associated data
> 
>    types for in-situ OAM.  The in-situ OAM data field can be transported
> 
>    by a variety of transport protocols, including NSH, Segment Routing,
> 
>    VXLAN-GPE, Geneve, IPv6, or IPv4.  Encapsulation details for these
> 
>    different transport protocols are outside the scope of this document.
> 
> 
> 
>    Deployment domain (or scope) of in-situ OAM deployment: IOAM is a
> 
>    network domain focused feature, with "network domain" being a set of
> 
>    network devices or entities within a single administration.  For
> 
>    example, a network domain can include an enterprise campus using
> 
>    physical connections between devices or an overlay network using
> 
>    virtual connections / tunnels for connectivity between said devices.
> 
>    A network domain is defined by its perimiter or edge.  The operator
> 
>    of such a domain MUST put provisions in place to ensure that in-situ
> 
>    OAM data stays within the specific domain only (i.e., does not leak
> 
>    beyond the edge) and consider potential impact of IOAM to ECMP
> 
>    processing, path MTU and ICMP message handling.
> 
> 
> 
>    In-situ OAM control points: IOAM data fields are added to or removed
> 
>    from the live user traffic by the devices which form the edge of a
> 
>    domain.  Devices within an IOAM domain can update and/or add IOAM
> 
>    data-fields.  Domain edge devices can be hosts or network devices.
> 
> 
> 
>    Traffic-sets that in-situ OAM is applied to: IOAM can be deployed on
> 
>    all or only on subsets of the live user traffic.  It SHOULD be
> 
>    possible to enable in-situ OAM on a selected set of traffic (e.g.,
> 
>    per interface, based on an access control list or flow specification
> 
>    defining a specific set of traffic, etc.)  The selected set of
> 
>    traffic can also be all traffic.
> 
> 
> 
>    Encapsulation independence: Data formats for in-situ OAM SHOULD be
> 
>    defined in a transport-independent manner.  In-situ OAM applies to a
> 
>    variety of encapsulating protocols.  A definition of how IOAM data
> 
>    fields are carried by different transport protocols is outside the
> 
>    scope of this document.
> 
> 
> 
>    Layering: If several encapsulation protocols (e.g., in case of
> 
>    tunneling) are stacked on top of each other, in-situ OAM data-records
> 
>    could be present at every layer.  The behavior follows the ships-in-
> 
>    the-night model.
> 
> 
> 
>    Combination with active OAM mechanisms: In-situ OAM SHOULD be usable
> 
>    for active network probing, enabling for example a customized version
> 
>    of traceroute.  Decapsulating in-situ OAM nodes may have an ability
> 
>    to send the in-situ OAM information retrieved from the packet back to
> 
>    the source address of the packet or to the encapsulating node.
> 
> 
> 
>    Im-situ OAM implementation: The IOAM data-field definitions take the
> 
>    specifics of devices with hardware data-plane and software data-plane
> 
>    into account.
> 
> 
> 
> Thoughts/comments? With these changes, are we able to kick-off an
> adoption call?
> 
> 
> 
> Thanks, Frank
> 
> 
> 
> -----Original Message-----
> 
> From: MORTON, ALFRED C (AL) [mailto:[email-masked]]
> 
> Sent: Mittwoch, 29. März 2017 18:11
> 
> To: [email-masked]; 'Brian Trammell (IETF)' <[email-masked]>;
> Frank Brockners (fbrockne) <[email-masked]>
> 
> Cc: 'IPPM Chairs' <[email-masked]>; [email-masked]
> 
> Subject: RE: [ippm] Vote at IPPM session
> 
> 
> 
> > -----Original Message-----
> 
> > From: ippm [mailto:[email-masked]] On Behalf Of Adrian Farrel
> 
> > Sent: Wednesday, March 29, 2017 5:32 PM
> 
> > To: 'Brian Trammell (IETF)'; 'Frank Brockners (fbrockne)'
> 
> > Cc: 'IPPM Chairs'; [email-masked]
> 
> > Subject: Re: [ippm] Vote at IPPM session
> 
> >
> 
> > > > There were several questions related to scope and applicability in
> 
> > > > the WG
> 
> > > discussion – and even more recently on the list. Those can easily be
> 
> > > addressed by adding paragraph on applicability to
> 
> > > draft-brockners-inband-oam-data – there isn’t a need for a dedicated
> requirements document.
> 
> > >
> 
> > > I tend to agree with this, although "a paragraph" seems a little
> 
> > > thin to me. I think there were some valid points made in that
> 
> > > discussion about the scope of the proposal that should be addressed:
> 
> > > is IOAM meant for use tunnel-end- to-tunnel- end environment,
> 
> > > end-host-to-end-host, within a single network and/or across the
> 
> > > Internet.  What I would suggest is adding a section to the data
> 
> > > model draft on applicability and assumptions about the environment
> 
> > > -- both about the devices adding IOAM signals to traffic as well as
> 
> > > those consuming these signals from the wire and analyzing them
> 
> > > (possibly with the cooperation of devices not on the
> 
> > > wire) -- submitting a new revision, and we can run a more formal
> 
> > > adoption call on that.
> 
> >
> 
> > Although I was one of the people raising the "need" for the scope and
> 
> > requirements, I don't have a strong leading on whether this needs to
> 
> > be in a separate document on folded into the data format document. So
> 
> > Brian's proposal would work for me.
> 
> >
> 
> > Thus, I'd love to see this scoping text drafted and floated to the
> 
> > list as an email or in a revision of the data format document.
> 
> >
> 
> > Cheers,
> 
> > Adrian
> 
> [ACM]
> 
> 
> 
> +1 for adding a Scope section to
> 
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_draft-2Dbrockners-2Dinband-2Doam-2Ddata-
> 2D02&d=DwIGaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=ootSI9_CjEColrhCFKEVPg5PZ77
> NOEMgztsEAI0kZFw&s=Sii0JulSEcLyiQb7EHsPkCy-n-mk5M-4BC1qV0yJe00&e=
> 
> 
> 
> I felt that I understood the scope going into the discussion Monday and
> the applicable (restricted) domain (I did the homework, the single draft
> Frank introduced on the mailing list was [0] ).
> 
> 
> 
> Some additional questions have been raised (e.g., "where will OAM be
> inserted and removed?") and I'd like to see those items sorted-out
> before we go very far (the advantage of wider review).
> 
> 
> 
> The requirements draft only came-up when I asked some questions on the
> list. Frank mentioned that the our list discussion on equivalent
> treatment of OAM & non-OAM packets ("class C") was added to the
> requirements document, and that should move to -oam-data- too.
> 
> 
> 
> It would be good to have a version of the scope to review ASAP.
> 
> 
> 
> thanks and regards,
> 
> Al
> 
> 
> 
> [0] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_draft-2Dbrockners-2Dinband-2Doam-2Ddata-
> 2D02&d=DwIGaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=ootSI9_CjEColrhCFKEVPg5PZ77
> NOEMgztsEAI0kZFw&s=Sii0JulSEcLyiQb7EHsPkCy-n-mk5M-4BC1qV0yJe00&e=
> 
> 
> 
> 
> 
> 
> 
> >
> 
> > _______________________________________________
> 
> > ippm mailing list
> 
> > [email-masked]
> 
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 
> > 3A__www.ietf.org_mailman_listinfo_ippm&d=DwIGaQ&c=LFYZ-
> 
> > o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=GaqucztK57fTj3vyYcMnHG9YK
> 
> > L1 tVkASQ5C3Kh63ooA&s=MesM6BFZyEkVPqsGq2Ho4LKS_A4ntf5X5hL0Sx0Rsdc&e=

---

From: Greg Mirsky <[email-masked]>
Date: Thu, 6 Apr 2017 19:02:48 -0700

Hi Frank,
thank you for the most expedient update. You've stated:

Devices within an IOAM domain can update and/or add IOAM data-fields.

Then, correct if I misunderstand, transient nodes in the domain in fact
allowed to add data thus enlarging the packet. How the edge would control
the packet size?

Regards,
Greg

On Fri, Mar 31, 2017 at 7:27 AM, Frank Brockners (fbrockne) <
[email-masked]> wrote:

> https://www.ietf.org/id/draft-brockners-inband-oam-data-04.txt includes
> section 3 on Scope, Applicability, and Assumptions, capturing the
> discussion we had in the WG meeting and on the list. We've also updated the
> introduction to refer to RFC7799 and classify in-situ OAM appropriately as
> hybrid, type-1 OAM.
>
> Thoughts/comments? With these changes, are we able to kick-off an adoption
> call?
>
> Thanks, Frank
>
> https://tools.ietf.org/html/draft-brockners-inband-oam-data-02
>
> inserted and removed?") and I'd like to see those items sorted-out before
> we go very far (the advantage of wider review).
>
> list. Frank mentioned that the our list discussion on equivalent treatment
> of OAM & non-OAM packets ("class C") was added to the requirements
> document, and that should move to -oam-data- too.
>
> [0] https://tools.ietf.org/html/draft-brockners-inband-oam-data-02
>
>
>
> >
> https://www.ietf.org/mailman/listinfo/ippm
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Fri, 7 Apr 2017 03:01:00 +0000

Greg,

> On Apr 6, 2017, at 10:02 PM, Greg Mirsky <[email-masked]> wrote:
> 
> Hi Frank,
> Then, correct if I misunderstand, transient nodes in the domain in fact allowed to add data thus enlarging the packet. How the edge would control the packet size?

You missed the prior sentence that reads:

  and consider potential impact of IOAM to ECMP
  processing, path MTU and ICMP message handling.

The Scope and Applicability section should not be muddied and overly complicated with solutions. For reference, the -requirements document talks about this topic in much more detail.

Thanks,

— Carlos.

> 
> Regards,
> Greg
> 
> On Fri, Mar 31, 2017 at 7:27 AM, Frank Brockners (fbrockne) <[email-masked]> wrote:
> Thanks for the suggestions and comments. We've posted a new revision of draft-brockners-inband-oam-data.
> https://www.ietf.org/id/draft-brockners-inband-oam-data-04.txt includes section 3 on Scope, Applicability, and Assumptions, capturing the discussion we had in the WG meeting and on the list. We've also updated the introduction to refer to RFC7799 and classify in-situ OAM appropriately as hybrid, type-1 OAM.
> 
> Thoughts/comments? With these changes, are we able to kick-off an adoption call?
> 
> Thanks, Frank
> 
> To: [email-masked]; 'Brian Trammell (IETF)' <[email-masked]>; Frank Brockners (fbrockne) <[email-masked]>
> > > draft-brockners-inband-oam-data – there isn’t a need for a dedicated requirements document.
> > >
> I felt that I understood the scope going into the discussion Monday and the applicable (restricted) domain (I did the homework, the single draft Frank introduced on the mailing list was [0] ).
> 
> Some additional questions have been raised (e.g., "where will OAM be inserted and removed?") and I'd like to see those items sorted-out before we go very far (the advantage of wider review).
> 
> The requirements draft only came-up when I asked some questions on the list. Frank mentioned that the our list discussion on equivalent treatment of OAM & non-OAM packets ("class C") was added to the requirements document, and that should move to -oam-data- too.
>

---

From: "Shwetha Bhandari (shwethab)" <[email-masked]>
Date: Fri, 7 Apr 2017 03:04:51 +0000

Greg,

> transient nodes in the domain in fact allowed to add data thus enlarging the packet. How the edge would control the packet size?

IOAM defined “Incremental Trace Option” [1] allows for transient nodes to add the trace data there by increasing the packet size. The maximum size the packet can grow to is controlled by “Maximum Length” field within this option.

Maximum Length:  8-bit unsigned integer.  This field specifies the
      maximum length of the node data list in octets.  Given that the
      sender knows the minimum path MTU, the sender can set the maximum
      of node data bytes allowed before exceeding the MTU.  Thus, a
      simple comparison between "Opt data Len" and "Max Length" allows
      to decide whether or not data could be added.

Thanks,

Shwetha

[1] https://www.ietf.org/id/draft-brockners-inband-oam-data-04.txt: Section 4.1.2

From: ippm <[email-masked]> on behalf of Greg Mirsky <[email-masked]>
Date: Friday, April 7, 2017 at 7:32 AM
To: "Frank Brockners (fbrockne)" <[email-masked]>
Cc: IPPM Chairs <[email-masked]>, "MORTON, ALFRED C (AL)" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [ippm] Vote at IPPM session

Hi Frank,

Regards,
Greg

On Fri, Mar 31, 2017 at 7:27 AM, Frank Brockners (fbrockne) <[email-masked]<mailto:[email-masked]>> wrote:
Thanks for the suggestions and comments. We've posted a new revision of draft-brockners-inband-oam-data.
https://www.ietf.org/id/draft-brockners-inband-oam-data-04.txt includes section 3 on Scope, Applicability, and Assumptions, capturing the discussion we had in the WG meeting and on the list. We've also updated the introduction to refer to RFC7799 and classify in-situ OAM appropriately as hybrid, type-1 OAM.

Thanks, Frank

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Mittwoch, 29. März 2017 18:11
To: [email-masked]<mailto:[email-masked]>; 'Brian Trammell (IETF)' <[email-masked]<mailto:[email-masked]>>; Frank Brockners (fbrockne) <[email-masked]<mailto:[email-masked]>>
Cc: 'IPPM Chairs' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: RE: [ippm] Vote at IPPM session

> From: ippm [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Adrian Farrel
> Cc: 'IPPM Chairs'; [email-masked]<mailto:[email-masked]>
[ACM]

thanks and regards,
Al

> [email-masked]<mailto:[email-masked]>
_______________________________________________
ippm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ippm

---

From: "Adrian Farrel" <[email-masked]>
Date: Fri, 7 Apr 2017 16:43:57 +0100

Hi Frank,

Thanks for writing this text. It is a good clarification of the intended scope and, I think, will help the ADs who seemed in Chicago to think the intended scope was more limited.

The sentence I have some concern over is:

>    The operator

To tell an operator to put provisions in place to protect themselves against a protocol is a bit delicate.

Maybe you could make this less sensitive by explaining:

- Which provisions are things like packet filters that MUST be configured 
   and where they have to be configured. Are they packet drop filters?
   Can we assume that if iOAM tries to escape from a domain then there
   is a misbehaving/misconfigured node somewhere?

- What "considering the potential impact" really means. I think that as
   worded it means "think about it and if you are worried, don't use iOAM"
   which is probably not the effect you wanted to achieve :-)  But otherwise,
   what is the operator supposed to do? Worry? Configure something? Tune
   iOAM in some way?

I am also not sure about the full extent of the functional scope here.
Consider a packet going from A to Z.
It enters the iOAM-aware domain at B and exits at Y.
You have certainly defined that I can run iOAM from B to Y.
I cannot quite tell whether you intend that iOAM is only used if B is a point of encapsulation. In other words, is B...Y a tunnel, or can iOAM be applied at B to the native packet that is flowing A...Z without further encapsulation?
Consider some transit nodes C and X within the iOAM-aware domain. I also can't tell whether C can be the initiation point for iOAM and X the termination point. If so, can this be done without requiring additional encapsulation, or does C...X have to be a tunnel within the domain (effectively creating a layer)?

I am aware that there is an overlap between scope and solution in the answers to my questions. I am not trying to dig into the solutions, but I am concerned that without understanding the extent of the scope we will either develop solutions that don't fit the intended scope, or we will develop solutions that, when applied to the full scope, cause alarming things to happen.

Cheers,
Adrian

> Sent: 06 April 2017 18:09
> To: Adrian Farrel ([email-masked]) ([email-masked])
> Subject: FW: [ippm] Vote at IPPM session
> 
> Hi Adrian,
> 
> thanks for your suggestions and continuous drive to include a scope section into
> draft-brockners-inband-oam-data. Does the new section 3 (see below) meet
> what you had in mind?
> 
> Thanks, Frank
> 
> From: ippm [mailto:[email-masked]] On Behalf Of Frank Brockners
> (fbrockne)
> Sent: Freitag, 31. März 2017 07:28
> To: MORTON, ALFRED C (AL) <[email-masked]>; [email-masked]; 'Brian
> Trammell (IETF)' <[email-masked]>
> section 3 on Scope, Applicability, and Assumptions, capturing the discussion we
> had in the WG meeting and on the list. We've also updated the introduction to
> refer to RFC7799 and classify in-situ OAM appropriately as hybrid, type-1 OAM.
> 
> To: [email-masked]; 'Brian Trammell (IETF)' <[email-masked]>; Frank
> Brockners (fbrockne) <[email-masked]>
> I felt that I understood the scope going into the discussion Monday and the
> applicable (restricted) domain (I did the homework, the single draft Frank
> introduced on the mailing list was [0] ).
> 
> Some additional questions have been raised (e.g., "where will OAM be inserted
> and removed?") and I'd like to see those items sorted-out before we go very far
> (the advantage of wider review).
> 
> The requirements draft only came-up when I asked some questions on the list.
> Frank mentioned that the our list discussion on equivalent treatment of OAM &
> non-OAM packets ("class C") was added to the requirements document, and that
> should move to -oam-data- too.
> 
> > L1
> tVkASQ5C3Kh63ooA&s=MesM6BFZyEkVPqsGq2Ho4LKS_A4ntf5X5hL0Sx0Rsdc&e
> =

---

From: Sarah B <[email-masked]>
Date: Fri, 7 Apr 2017 14:41:23 -0700

Hi Frank,
	Thanks for the timely update.
	At first glance, reading through the below, I very much appreciate the specific text outlining explicitly where the IOAM is expected to be limited to; that there is a limit at all, and that it's domain specific (picking up on the comment in the IPPM session to call this out). Thanks. This seems to make sense. There is a section below that gives me pause: "<snip>The operator of such a domain MUST put provisions in place to ensure that in-situ OAM data stays within the specific domain only (i.e., does not leak beyond the edge) and consider potential impact of IOAM to ECMP processing, path MTU and ICMP message handling." I'm sure every operator or implementation will have the best laid plans in mind, but if there's a misconfigured or otherwise malicious configuration wherein the IOAM does indeed leak outside of the domain, what then? Instead of simply asking the user to not allow this, provide proactive advice on options to alleviate or remediate this possibility. Would you also consider either making the "and consider potential impact" a separate explicit MUST or SHOULD (or otherwise) requirement, with a little more clarity around what you mean for potential impact?

Thanks
Sarah

> On Mar 31, 2017, at 7:27 AM, Frank Brockners (fbrockne) <[email-masked]> wrote:
>

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Mon, 10 Apr 2017 16:29:32 +0000

Hi Sarah,

thanks - both you and Adrian highlighted the need to provide additional considerations for what the impact of IOAM could be to ECMP processing, path MTU and ICMP message handling. While the detailed considerations for the impact are transport protocol dependent and as such beyond the scope of the document that discusses the data-fields and data-types for IOAM, I agree that it does make sense to provide some further clarification and at least mention a couple of example cases. 

How about: "The operator of such a domain MUST put provisions in place to ensure that in-situ OAM data stays within the specific domain only (i.e., does not leak beyond the edge) using for example packet filtering methods. The operator SHOULD consider potential operational impact of IOAM to mechanisms such as ECMP processing (e.g. load-balancing schemes based on packet length could be impacted by the increased packet size due to IOAM), path MTU (i.e. ensure that the MTU of all links within a domain is sufficiently large to support the increased packet size due to IOAM) and ICMP message handling (i.e. in case of a native IPv6 transport, IOAM support for ICMPv6 Echo Request/Reply could desired which would translate into ICMPv6 extensions to enable IOAM data fields to be copied from an Echo Request message to an Echo Reply message)."

Thoughts?

Thanks, Frank

-----Original Message-----
From: Sarah B [mailto:[email-masked]] 
Sent: Freitag, 7. April 2017 23:41
To: Frank Brockners (fbrockne) <[email-masked]>
Cc: ALFRED MORTON <[email-masked]>; [email-masked]; Brian Trammell (IETF) <[email-masked]>; IPPM Chairs <[email-masked]>; [email-masked]
Subject: Re: [ippm] Vote at IPPM session

Hi Frank,

Thanks
Sarah

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Mon, 10 Apr 2017 16:48:20 +0000

Hi Adrian,

thanks for your comments. Please see inline ("...FB").

-----Original Message-----
From: Adrian Farrel [mailto:[email-masked]] 
Sent: Freitag, 7. April 2017 17:44
To: Frank Brockners (fbrockne) <[email-masked]>
Cc: [email-masked]
Subject: RE: [ippm] Vote at IPPM session

Hi Frank,

...FB: Sarah mentioned a similar concern (see my earlier email).  And while the detailed considerations for the impact are transport protocol dependent and as such beyond the scope of the document at hand that discusses the data-fields and data-types for IOAM, I agree that it does make sense to provide some further clarification and at least mention a couple of example cases. 

...FB: What "IOAM data-fields encapsulation" translates to is really transport protocol dependent, i.e. whether it is a "tunnel" will dependent on the transport protocol used. As such, it would need to be dealt with in the I-Ds that specify the transport. draft-brockners-inband-oam-transport-03 lists a couple of ideas, which we could, but don't have to leverage. 

...FB: That would indeed lead to a layered IOAM deployment, i.e. you have your Domain1 which is (B to Y) and then have a second Domain2 within Domain1 which is (X to C). draft-brockners-inband-oam-data-04 already mentions that IOAM could be layered. The specifics on insertion/removal of IOAM data-fields would again be defined in an I-D which deals with encap for a specific transport protocol.

...FB: Agreed. There is some need to have a "mental model" in mind how the IOAM data-fields would be carried, even though we would not be able to tackle the specifics on that in the I-D on IOAM data-fields. This was the reason why we originally published draft-brockners-inband-oam-transport - it was not to preempt any standardization effort but to just provide ideas/clarification how the data-fields would actually be inserted and carried.

Cheers, Frank 

Cheers,
Adrian

> thanks for your suggestions and continuous drive to include a scope 
> section into draft-brockners-inband-oam-data. Does the new section 3 
> (see below) meet what you had in mind?
> 
> Thanks, Frank
> 
> To: MORTON, ALFRED C (AL) <[email-masked]>; [email-masked]; 
> 'Brian Trammell (IETF)' <[email-masked]>
> Thanks for the suggestions and comments. We've posted a new revision 
> of draft- brockners-inband-oam-data.
> https://www.ietf.org/id/draft-brockners-inband-oam-data-04.txt 
> updated the introduction to refer to RFC7799 and classify in-situ OAM appropriately as hybrid, type-1 OAM.
> 
> > > > There were several questions related to scope and applicability 
> > > > in the WG
> > > discussion – and even more recently on the list. Those can easily 
> > > be addressed by adding paragraph on applicability to 
> > > -- both about the devices adding IOAM signals to traffic as well 
> > > as those consuming these signals from the wire and analyzing them 
> > Although I was one of the people raising the "need" for the scope 
> > and requirements, I don't have a strong leading on whether this 
> > needs to be in a separate document on folded into the data format 
> > document. So Brian's proposal would work for me.
> >
> I felt that I understood the scope going into the discussion Monday 
> and the applicable (restricted) domain (I did the homework, the single 
> draft Frank introduced on the mailing list was [0] ).
> 
> Frank mentioned that the our list discussion on equivalent treatment

---

From: Sarah B <[email-masked]>
Date: Wed, 12 Apr 2017 08:20:46 -0700

Hi Frank,

> How about: "The operator of such a domain MUST put provisions in place to ensure that in-situ OAM data stays within the specific domain only (i.e., does not leak beyond the edge) using for example…

	I think this is a good start; thanks for putting the thought and effort into it. It makes me a bit uneasy, because there's no way to ensure it doesn't leak out, but it is what it is. I do appreciate that you have a MUST, and still offered an example on how to mitigate the leaks; thanks for taking the feedback into consideration. I also like that you've pulled out the potential operational impact as a SHOULD, and in a separate sentence; this makes sense to me. Finally, I think it's crystal clear that this is domain-specific and limited; that alleviates my objection from the in-room discussion. :)

Thanks
Sarah

---

From: "Adrian Farrel" <[email-masked]>
Date: Sat, 15 Apr 2017 10:15:39 +0100

Hi Frank, all,

Thanks for proposing some text.

> How about: "The operator of such a domain MUST put provisions in place to
> ensure that in-situ OAM data stays within the specific domain only (i.e., does
not
> leak beyond the edge) using for example packet filtering methods. The operator
> SHOULD consider potential operational impact of IOAM to mechanisms such as
> ECMP processing (e.g. load-balancing schemes based on packet length could be
> impacted by the increased packet size due to IOAM), path MTU (i.e. ensure that
> the MTU of all links within a domain is sufficiently large to support the
increased
> packet size due to IOAM) and ICMP message handling (i.e. in case of a native
IPv6
> transport, IOAM support for ICMPv6 Echo Request/Reply could desired which
> would translate into ICMPv6 extensions to enable IOAM data fields to be copied
> from an Echo Request message to an Echo Reply message)."

Like Sarah, I think this is a start.

I remain disappointed that it is the operators' responsibility to ensure various
things, and not a feature of the protocol or implementation.

But perhaps this text serves as a warning to the operators about using
implementations of iOAM and so will suffice.

I think the use of 2119 capitalisation is meaningless in this paragraph, and
that the "SHOULD" in the second sentence should in any case be a "must".

Thanks,
Adrian

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Tue, 18 Apr 2017 16:53:36 +0000

Hi Adrian,

thanks - please see inline below...

-----Original Message-----
From: Adrian Farrel [mailto:[email-masked]] 
Sent: Samstag, 15. April 2017 11:16
To: Frank Brockners (fbrockne) <[email-masked]>
Cc: 'ALFRED MORTON' <[email-masked]>; 'Brian Trammell (IETF)' <[email-masked]>; 'IPPM Chairs' <[email-masked]>; [email-masked]; 'Sarah B' <[email-masked]>
Subject: RE: [ippm] Vote at IPPM session

Hi Frank, all,

> How about: "The operator of such a domain MUST put provisions in place 
> to ensure that in-situ OAM data stays within the specific domain only 
> (i.e., does
not
> leak beyond the edge) using for example packet filtering methods. The 
> operator SHOULD consider potential operational impact of IOAM to 
> mechanisms such as ECMP processing (e.g. load-balancing schemes based 
> on packet length could be impacted by the increased packet size due to 
> IOAM), path MTU (i.e. ensure that the MTU of all links within a domain 
> is sufficiently large to support the
increased
IPv6
> transport, IOAM support for ICMPv6 Echo Request/Reply could desired 
> which would translate into ICMPv6 extensions to enable IOAM data 
> fields to be copied from an Echo Request message to an Echo Reply message)."

I remain disappointed that it is the operators' responsibility to ensure various things, and not a feature of the protocol or implementation.

But perhaps this text serves as a warning to the operators about using implementations of iOAM and so will suffice.

...FB: The current document is just the starting point. Let's hope that the WG can improve the current approach further and/or suggest improved text.

I think the use of 2119 capitalisation is meaningless in this paragraph, and that the "SHOULD" in the second sentence should in any case be a "must".

...FB: Thanks. Will change/correct in the next revision (along with a set of typos that I introduced when crafting the section)

Thanks, Frank

Thanks,
Adrian

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Tue, 18 Apr 2017 21:25:43 +0200

hi Frank, all,

Speaking as an individual.

First, thanks for the scope section... this is all much more concrete now.

So, as I understand it, iOAM is designed first and foremost to be a single-network (in the sense of "coherent administrative domain") protocol data model with a binding to a variety of "carrier" protocols (the draft speaks of "transports" in the RTG-area meaning of the term, not the TSV-area one, so let's overload "carrier" here instead), some of which are also explicitly single-network, some of which less so...

I tend to share the concerns of those who've expressed discomfort with the "warning label approach" to ensuring that iOAM data stays single-network, but I'm not sure it's a problem *for the data model*. I don't necessarily read this "MUST NOT leave the network" as an additional responsibility of operators, but rather on the designers of carrier protocols for iOAM data. Now, some of the carrier protocols (e.g. IPv6 extension headers) provide no such protection or support for providing that protection, so in that case it falls to the implementor of the iOAM solution to provide it... but this is rather a matter to be handled on a per carrier protocol basis, isn't it?

Cheers,

Brian

> On 18 Apr 2017, at 18:53, Frank Brockners (fbrockne) <[email-masked]> wrote:
> 
> Hi Adrian,
>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 18 Apr 2017 20:04:22 +0000

All,
one alternative to consider, below.
Al

> From: Brian Trammell (IETF) [mailto:[email-masked]]
> Sent: Tuesday, April 18, 2017 3:26 PM
> To: Frank Brockners (fbrockne)
> Cc: [email-masked]; MORTON, ALFRED C (AL); IPPM Chairs;
> [email-masked]; Sarah B
> So, as I understand it, iOAM is designed first and foremost to be a
> single-network (in the sense of "coherent administrative domain")
> protocol data model with a binding to a variety of "carrier" protocols
> (the draft speaks of "transports" in the RTG-area meaning of the term,
> not the TSV-area one, so let's overload "carrier" here instead), some of
> which are also explicitly single-network, some of which less so...
> 
> I tend to share the concerns of those who've expressed discomfort with
> the "warning label approach" to ensuring that iOAM data stays single-
> network, but I'm not sure it's a problem *for the data model*. I don't
> necessarily read this "MUST NOT leave the network" as an additional
> responsibility of operators, but rather on the designers of carrier
> protocols for iOAM data. Now, some of the carrier protocols (e.g. IPv6
> extension headers) provide no such protection or support for providing
> that protection, so in that case it falls to the implementor of the iOAM
> solution to provide it... but this is rather a matter to be handled on a
> per carrier protocol basis, isn't it?
> 
> Cheers,
> Brian
[ACM] 

IOM, there is a general message to convey for all future protocol 
development. Something like "sufficient provisions should be made
to restrict the iOAM traffic to the intended domain."

We could provide guidance to interconnecting network operators,
as well, effectively allowing them to discard traffic containing 
unexpected iOAM data. This would be applicable to 
non-iOAM network operators, but might be more important for
interconnecting operators who have established their own iOAM domain.

This is similar to the approach we used for BMWG test traffic:
there are v4 and v6 address spaces dedicated for isolated test 
environments, and any packet with testing addresses observed on 
the Internet may be discarded.

regards,
Al

> > Cc: 'ALFRED MORTON' <[email-masked]>; 'Brian Trammell (IETF)'
> <[email-masked]>; 'IPPM Chairs' <[email-masked]>; [email-masked];
> 'Sarah B' <[email-masked]>
> > I remain disappointed that it is the operators' responsibility to
> ensure various things, and not a feature of the protocol or
> implementation.
> >
> > ...FB: The current document is just the starting point. Let's hope
> that the WG can improve the current approach further and/or suggest
> improved text.
> >
> > I think the use of 2119 capitalisation is meaningless in this
> paragraph, and that the "SHOULD" in the second sentence should in any
> case be a "must".
> >
> > ...FB: Thanks. Will change/correct in the next revision (along with a
> set of typos that I introduced when crafting the section)
> >
> > Thanks, Frank
> >
> > Thanks,
> > Adrian

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 19 Apr 2017 02:10:59 +0000

Hi Al,

I am not clear about some of your suggestions.
Please see inline.

Thanks,
Tianran

> From: ippm [mailto:[email-masked]] On Behalf Of MORTON, ALFRED C
> (AL)
> Sent: Wednesday, April 19, 2017 4:04 AM
> To: Brian Trammell (IETF); Frank Brockners (fbrockne)
> Cc: IPPM Chairs; [email-masked]
> > not the TSV-area one, so let's overload "carrier" here instead), some
> > of which are also explicitly single-network, some of which less so...
> >
> > that protection, so in that case it falls to the implementor of the
> > iOAM solution to provide it... but this is rather a matter to be
> > handled on a per carrier protocol basis, isn't it?
> >
> > Cheers,
> > Brian
> [ACM]
> 
> IOM, there is a general message to convey for all future protocol development.
> Something like "sufficient provisions should be made to restrict the iOAM
> traffic to the intended domain."
> 
> We could provide guidance to interconnecting network operators, as well,
> effectively allowing them to discard traffic containing unexpected iOAM
> data. 

"Discard" the traffic? This might be available for active measurement with additional testing packets. But this iOAM is in-situ within the data packet. The discard will harm the user traffic. Or bypassing the iOAM instruction/header be better?

>This would be applicable to non-iOAM network operators, but might
> be more important for interconnecting operators who have established their
> own iOAM domain.
> 
> there are v4 and v6 address spaces dedicated for isolated test environments,
> and any packet with testing addresses observed on the Internet may be
> discarded.
> 
> regards,
> Al
> 
> >
> >
> >
> > <[email-masked]>; 'IPPM Chairs' <[email-masked]>;
> > [email-masked]; 'Sarah B' <[email-masked]>
> > >> to ensure that in-situ OAM data stays within the specific domain
> > >> only (i.e., does
> > > not
> > >> leak beyond the edge) using for example packet filtering methods.
> > >> The operator SHOULD consider potential operational impact of IOAM
> > >> to mechanisms such as ECMP processing (e.g. load-balancing schemes
> > >> based on packet length could be impacted by the increased packet
> > >> size due
> > to

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 19 Apr 2017 02:16:53 +0000

Quick reply in-line,

> From: Tianran Zhou [mailto:[email-masked]]
> Sent: Tuesday, April 18, 2017 10:11 PM
> To: MORTON, ALFRED C (AL); Brian Trammell (IETF); Frank Brockners
> (fbrockne)
> > > First, thanks for the scope section... this is all much more
> concrete
> > > now.
> > >
> development.
> "Discard" the traffic? This might be available for active measurement
> with additional testing packets. But this iOAM is in-situ within the
> data packet. The discard will harm the user traffic. Or bypassing the
> iOAM instruction/header be better?
[ACM] 
Yes, discard.

The traffic was not properly processed at the domain boundary 
(the iOAM data should have been processed and removed).
Traffic that unintentionally escapes a network may have 
undesirable effects (harm) on downstream networks,
especially if others are planning to add iOAM data.

Al

> environments,
> > > >> packet size due to IOAM) and ICMP message handling (i.e. in case
> of
> > > >> a native
> > > > IPv6
> > > > ...FB: Thanks. Will change/correct in the next revision (along
> with
> > > > a
> 3A__www.ietf.org_mailman_listinfo_ippm&d=DwIFAg&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=8cY2uEv8ohR5Vo7ht_h5e44At-
> tNd7YjYG-2Zqv08rg&s=azHzymcSqyhfRrLm7_QX2YbyttdWrjvr2-Y268j9wWI&e=

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 19 Apr 2017 03:09:17 +0000

Thanks Al. Got you.
Then I think the inter-domain/operator should be complex while discard is one of  many negotiable policies.
For example, one operator(access) may want to transit another network(core).

> Sent: Wednesday, April 19, 2017 10:17 AM
> To: Tianran Zhou; Brian Trammell (IETF); Frank Brockners (fbrockne)
> > > From: ippm [mailto:[email-masked]] On Behalf Of MORTON,
> > > ALFRED
> > C
> > > (AL)
> > > > So, as I understand it, iOAM is designed first and foremost to be
> > > > a single-network (in the sense of "coherent administrative
> > > > domain") protocol data model with a binding to a variety of "carrier"
> > protocols
> > > > necessarily read this "MUST NOT leave the network" as an
> > > > additional responsibility of operators, but rather on the
> > > > designers of carrier protocols for iOAM data. Now, some of the carrier
> protocols (e.g.
> > IPv6
> > > > that protection, so in that case it falls to the implementor of
> > > > the iOAM solution to provide it... but this is rather a matter to
> > > > be handled on a per carrier protocol basis, isn't it?
> > > >
> > > > Cheers,
> > > > Brian
> > > [ACM]
> > >
> The traffic was not properly processed at the domain boundary (the iOAM
> data should have been processed and removed).
> Traffic that unintentionally escapes a network may have undesirable effects
> (harm) on downstream networks, especially if others are planning to add
> iOAM data.
> 
> Al
> 
> >
> > >This would be applicable to non-iOAM network operators, but might  be
> > >more important for interconnecting operators who have established
> > their
> > > own iOAM domain.
> > >
> > > and any packet with testing addresses observed on the Internet may
> > > be discarded.
> > >
> > > regards,
> > > Al
> > >
> > > >
> > > >
> > > >
> > > > >> How about: "The operator of such a domain MUST put provisions
> > > > >> in
> > > > place
> > > > >> to ensure that in-situ OAM data stays within the specific
> > > > >> domain only (i.e., does
> > > > > not
> > > > >> The operator SHOULD consider potential operational impact of
> > > > >> IOAM to mechanisms such as ECMP processing (e.g. load-balancing
> > schemes
> > > > >> based on packet length could be impacted by the increased
> > > > >> packet size due
> > > > to
> > > > >> which would translate into ICMPv6 extensions to enable IOAM
> > > > >> data fields to be copied from an Echo Request message to an
> > > > >> Echo Reply
> > > > message)."
> > > > >
> > > > that the WG can improve the current approach further and/or
> > > > suggest improved text.
> > > > >
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=8cY2uEv8ohR5Vo7ht_h5e44At
> > - tNd7YjYG-2Zqv08rg&s=azHzymcSqyhfRrLm7_QX2YbyttdWrjvr2-Y268j9wWI&e=

---

From: <[email-masked]>
Date: Wed, 19 Apr 2017 07:14:45 +0000

Hi Al,

is a non-iOAM network operator able to detect standard ipv6 packets with iOAM extensions, if the receiving operators equipment is configured to support standard ipv6 protocol only? The pre-condition here is "non-iOAM domain" at the receiving side. My point is, if a domain isn't interested in supporting the iOAM extensions, is it obliged to operate iOAM aware equipment to detect undesired traffic at network boundaries?

I'm not sure, whether this is an IPPM discussion. Is there any other IPPM protocol or packet content requiring a discard of traffic at a domain boundary?

Regards,

Ruediger

 [ACM] 

IOM, there is a general message to convey for all future protocol development. Something like "sufficient provisions should be made to restrict the iOAM traffic to the intended domain."

We could provide guidance to interconnecting network operators, as well, effectively allowing them to discard traffic containing unexpected iOAM data. This would be applicable to non-iOAM network operators, but might be more important for interconnecting operators who have established their own iOAM domain.

This is similar to the approach we used for BMWG test traffic:
there are v4 and v6 address spaces dedicated for isolated test environments, and any packet with testing addresses observed on the Internet may be discarded.

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Wed, 19 Apr 2017 12:24:21 +0000

Just echo'ing and agreeing to what was said below and earlier: IMHO the statement of making it the responsibility of the operator to avoid IOAM data fields to be carried beyond the IOAM domain should really be the gateway of last resort. One should rather try to design the encapsulation of IOAM data into the various carrier protocols in a way that additional provisions to avoid leaking of IOAM data don't necessarily need to be put in by an operator. Like what Brian and Ruediger said, this is obviously beyond IPPM documents on IOAM data fields but would need to be tackled by carrier protocol specific specifications. For the text in the IOAM data-draft we probably want to call out the fact that carrier protocol design should consider this requirement. How about:

"Designers of carrier protocols for IOAM designers must consider to put mechanisms in place to ensure that in-situ OAM data stays within an IOAM domain. In addition, the operator of such a domain is expected to put provisions in place to ensure that IOAM data does not leak beyond the edge of an IOAM domain, e.g. using for example packet filtering methods. The operator should consider potential operational impact of IOAM to mechanisms such as ECMP processing (e.g. load-balancing schemes based on packet length could be impacted by the increased packet size due to IOAM), path MTU (i.e. ensure that the MTU of all links within a domain is sufficiently large to support the increased packet size due to IOAM) and ICMP message handling (i.e. in case of a native IPv6 transport, IOAM support for ICMPv6 Echo Request/Reply could desired which would translate into ICMPv6 extensions to enable IOAM data fields to be copied from an Echo Request message to an Echo Reply message)."

Thoughts?

Thanks, Frank

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Mittwoch, 19. April 2017 09:15
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]; Frank Brockners (fbrockne) <[email-masked]>
Subject: AW: [ippm] Vote at IPPM session

Hi Al,

Regards,

Ruediger

 [ACM]

---

From: "Bert Wijnen (IETF)" <[email-masked]>
Date: Wed, 19 Apr 2017 17:51:04 +0200

Inline

On 19/04/2017 14:24, Frank Brockners (fbrockne) wrote:
> Just echo'ing and agreeing to what was said below and earlier: IMHO the statement of making it the responsibility of the operator to avoid IOAM data fields to be carried beyond the IOAM domain should really be the gateway of last resort. One should rather try to design the encapsulation of IOAM data into the various carrier protocols in a way that additional provisions to avoid leaking of IOAM data don't necessarily need to be put in by an operator.
Agreed. No need to put extra burden on the operators if we can avoid it.

I do wonder though.... were we not thinking of also having IOAM end to end? Maybe that was some
other proposal and I am mixed up.

>   Like what Brian and Ruediger said, this is obviously beyond IPPM documents on IOAM data fields but would need to be tackled by carrier protocol specific specifications. For the text in the IOAM data-draft we probably want to call out the fact that carrier protocol design should consider this requirement. How about:
>
> "Designers of carrier protocols for IOAM designers must consider to put mechanisms in
2 times "designers". I suggest to remove the 2nd occurance.
> place to ensure that in-situ OAM data stays within an IOAM domain. In addition, the operator of such a domain is expected to put provisions in place to ensure that IOAM data does not leak beyond the edge of an IOAM domain, e.g. using for example packet filtering methods. The operator should consider potential operational impact of IOAM to mechanisms such as ECMP processing (e.g. load-balancing schemes based on packet length could be impacted by the increased packet size due to IOAM), path MTU (i.e. ensure that the MTU of all links within a domain is sufficiently large to support the increased packet size due to IOAM) and ICMP message handling (i.e. in case of a native IPv6 transport, IOAM support for ICMPv6 Echo Request/Reply could desired which would translate into ICMPv6 extensions to enable IOAM data fields to be copied from an Echo Request message to an Echo Reply message)."
>
> Thoughts?
see above, Bert

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 19 Apr 2017 20:02:21 +0000

Hi Rüdiger,
short reply in-line,
Al

> Sent: Wednesday, April 19, 2017 3:15 AM
> To: MORTON, ALFRED C (AL)
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> is a non-iOAM network operator able to detect standard ipv6 packets with
> iOAM extensions, if the receiving operators equipment is configured to
> support standard ipv6 protocol only? The pre-condition here is "non-iOAM
> domain" at the receiving side. My point is, if a domain isn't interested
> in supporting the iOAM extensions, is it obliged to operate iOAM aware
> equipment to detect undesired traffic at network boundaries?
[ACM] 
No, an operator can let the traffic flow if they want.
However, if operators find that their network is 
affected by traffic with iOAM data from another domain,
they would be justified to discard that traffic
(as long as there is a clear domain limit expressed to 
iOAM protocol designers of the future, as Frank has
suggested).

This is the same ability afforded operators by declaring
BMWG address space for isolated testing-only, or the
various private network address spaces. Plenty of 
Net10 traffic escapes, and operators are not obliged
to drop it, but they certainly can.

> 
> I'm not sure, whether this is an IPPM discussion. Is there any other
> IPPM protocol or packet content requiring a discard of traffic at a
> domain boundary?
> 
> Regards,
> 
> Ruediger
> 
> 
>  [ACM]
> 
> development. Something like "sufficient provisions should be made to
> restrict the iOAM traffic to the intended domain."
> 
> data. This would be applicable to non-iOAM network operators, but might
> be more important for interconnecting operators who have established
> their own iOAM domain.
> 
> environments, and any packet with testing addresses observed on the
> Internet may be discarded.
>

---

From: <[email-masked]>
Date: Thu, 20 Apr 2017 07:22:43 +0000

Hi Al,

IOAM parameters are not the same as private address space. Routers operate IP addresses and an access list doesn't require detection of a type of parameter which isn't supported. The access list controls whether values of a known parameter are ok. IOAM requires detection of an unknown type of parameter at a domain boundary. A behavior often requested in that case is transfer, not discard. 
An interconnection router likely operates an access list to protect the own domain, but interconnection routers are no firewalls (be liberal in what you accept....).

There are two scenarios which may worry:

- IOAM-domain passes traffic to non-IOAM domain 
  or IOAM domain respectively and IOAM parameters 
  may still be present.

- IOAM-domain passes traffic to non-IOAM domain 
  which passes packets containing IOAM unmodified 
  to another IOAM domain.

To me, the latter is the less desirable one. 

Regards, Rüdiger

> is a non-iOAM network operator able to detect standard ipv6 packets 
> with iOAM extensions, if the receiving operators equipment is 
> configured to support standard ipv6 protocol only? The pre-condition 
> here is "non-iOAM domain" at the receiving side. My point is, if a 
> domain isn't interested in supporting the iOAM extensions, is it 
> obliged to operate iOAM aware equipment to detect undesired traffic at network boundaries?

[ACM]
No, an operator can let the traffic flow if they want.
However, if operators find that their network is affected by traffic with iOAM data from another domain, they would be justified to discard that traffic (as long as there is a clear domain limit expressed to iOAM protocol designers of the future, as Frank has suggested).

[ACM]
This is the same ability afforded operators by declaring BMWG address space for isolated testing-only, or the various private network address spaces. Plenty of Net10 traffic escapes, and operators are not obliged to drop it, but they certainly can.

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 20 Apr 2017 09:54:40 +0100

>   or IOAM domain respectively and IOAM parameters

This is the one where the non-iOAM router gets a packet containing iOAM data and
misbehaves in some way...
- crash
- mis-forward (including broken ECMP)
- black-hole

>   which passes packets containing IOAM unmodified

This is the one where the downstream domain is caused to execute behavior under
the control of the upstream domain.

It is clear (to me) that in both cases the downstream domain will want to
protect itself (and so, to some extent, Frank is right that the operator of the
downstream domain will want to take precautions), but my concern is to make it
simple to take those precautions, build them into the protocols themselves, and
allow innocent legacy networks to continue undisturbed in the presence of iOAM
in other networks.

I understand that to some extent all of this discussion goes with the
encapsulation/use-cases and not with the fundamental generic encoding of iOAM
data. However, I think that the question applies to all applications of iOAM and
so clear statements in the base document are necessary.

Adrian

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Sun, 23 Apr 2017 11:44:12 +0000

Hi,

This is a very useful discussion — although on the tail-end seems to be diverging from the original goal. I suggest that we should probably not take this discussion into all inter-domain permutations.

In other words, is there some case that the original text does not cover? The text includes “mechanisms in place to ensure that in-situ OAM data stays within an IOAM domain” and “provisions in place to ensure that IOAM data does not leak beyond the edge of an IOAM domain”.

The idea is that an IOAM Domain should take care of considerations because of e.g., increased packet sizes within its domain, and prevent leakage.

In many deployments, the latter (i.e., "ensuring that iOAM data stays within the domain, and does not leak beyond the exit edge”) would be a natural characteristic of the encapsulation. For example, when using IOAM with NSH, an IOAM domain would be an SFC domain, and thus removing NSH encap will also ensure IOAM does not leak. Same with VxLAN-GPE, etc.

But net-net, I’d recommend we define the IOAM domain behavior and not all other possible views.

Thanks!

— Carlos.

> On Apr 19, 2017, at 4:02 PM, MORTON, ALFRED C (AL) <[email-masked]> wrote:
> 
> Hi Rüdiger,
> short reply in-line,
> Al
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Sun, 23 Apr 2017 11:53:53 +0000

Hi Adrian,

> On Apr 20, 2017, at 4:54 AM, Adrian Farrel <[email-masked]> wrote:
> 

I’d imagine blame for software bugs triggered by legal packets is outside of scope of what a specification can own. :-)

That said, running (non-crashing) code is important and hence IOAM in Bits-n-bytes was useful. An opportunity for the Hackathon as well!

> so clear statements in the base document are necessary.

Yes. Agreed. Some of this is indeed encap-specific, but the requirement and consideration is broadly applicable.

The -data draft will include Frank’s proposal and review comments on it.

Thanks!

Carlos.

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 26 Apr 2017 02:22:10 -0700

Hi Adrian, et. al,
I strongly believe that having agreed upon list of requirements that answer
questions What? and Where? rather than How? is the most beneficial and not
only for the discussion of applicability of in-situ OAM. What needs to be
solved, measured that is not ameasureable by already existing OAM methods?
I think that if we can compile such list, then it will be more clear what
value may be added by developing new hybrid OAM method.

Regards,
Greg

On Wed, Mar 29, 2017 at 2:32 PM, Adrian Farrel <[email-masked]> wrote:

> > > There were several questions related to scope and applicability in the
> WG
> addressed by
> > adding paragraph on applicability to draft-brockners-inband-oam-data –
> there
> > isn’t a need for a dedicated requirements document.
> >
> > I tend to agree with this, although "a paragraph" seems a little thin to
> me. I think
> > there were some valid points made in that discussion about the scope of
> the
> > proposal that should be addressed: is IOAM meant for use
> tunnel-end-to-tunnel-
> > end environment,  end-host-to-end-host, within a single network and/or
> across
> > the Internet.  What I would suggest is adding a section to the data
> model draft on
> > applicability and assumptions about the environment -- both about the
> devices
> > adding IOAM signals to traffic as well as those consuming these signals
> from the
> > wire and analyzing them (possibly with the cooperation of devices not on
> the
> adoption call on
> > that.
> requirements, I don't have a strong leading on whether this needs to be in
> a separate document on folded into the data format document. So Brian's
> proposal would work for me.
>
> Thus, I'd love to see this scoping text drafted and floated to the list as
> an email or in a revision of the data format document.
>
> Cheers,
> Adrian
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 26 Apr 2017 13:13:34 +0000

Dear Greg,

Something like https://tools.ietf.org/html/draft-brockners-inband-oam-requirements-03?

For the record, my opinion: perfecting requirements is perhaps not the best use of WG cycles. However, placing requirements as a serialized requisite before progressing protocol work is certainly harmful in this case.

Thanks,

-- Carlos.

From: ippm <[email-masked]> on behalf of Greg Mirsky <[email-masked]>
Date: Wednesday, April 26, 2017 at 5:22 AM
To: Adrian Farrel <[email-masked]>
Cc: IPPM Chairs <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [ippm] Vote at IPPM session

Hi Adrian, et. al,
I strongly believe that having agreed upon list of requirements that answer questions What? and Where? rather than How? is the most beneficial and not only for the discussion of applicability of in-situ OAM. What needs to be solved, measured that is not ameasureable by already existing OAM methods? I think that if we can compile such list, then it will be more clear what value may be added by developing new hybrid OAM method.

Regards,
Greg

On Wed, Mar 29, 2017 at 2:32 PM, Adrian Farrel <[email-masked]<mailto:[email-masked]>> wrote:
> > There were several questions related to scope and applicability in the WG
> discussion – and even more recently on the list. Those can easily be addressed by
> adding paragraph on applicability to draft-brockners-inband-oam-data – there
> I tend to agree with this, although "a paragraph" seems a little thin to me. I think
> there were some valid points made in that discussion about the scope of the
> proposal that should be addressed: is IOAM meant for use tunnel-end-to-tunnel-
> end environment,  end-host-to-end-host, within a single network and/or across
> the Internet.  What I would suggest is adding a section to the data model draft on
> applicability and assumptions about the environment -- both about the devices
> adding IOAM signals to traffic as well as those consuming these signals from the
> wire and analyzing them (possibly with the cooperation of devices not on the
> wire) -- submitting a new revision, and we can run a more formal adoption call on
> that.

Although I was one of the people raising the "need" for the scope and requirements, I don't have a strong leading on whether this needs to be in a separate document on folded into the data format document. So Brian's proposal would work for me.

Thus, I'd love to see this scoping text drafted and floated to the list as an email or in a revision of the data format document.

Cheers,
Adrian

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 26 Apr 2017 06:22:12 -0700

Dear Carlos,
thank you for the reference.
I've never called for perfecting but for rough consensus on what problem we
need to address. How, IMHO, is secondary. And that is what may be not
working in this discussion - we've been presented answer to how do
measurement before we have reached an agreement that there's a problem to
be solved.

Regards,
Greg

On Wed, Apr 26, 2017 at 6:13 AM, Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> For the record, my opinion: perfecting requirements is perhaps not the
> best use of WG cycles. However, placing requirements as a serialized
> requisite before progressing protocol work is certainly harmful in this
> case.
>
>
>
> Thanks,
>
>
>
> -- Carlos.
>
>
>
> *From: *ippm <[email-masked]> on behalf of Greg Mirsky <
> [email-masked]>
> *Date: *Wednesday, April 26, 2017 at 5:22 AM
> *To: *Adrian Farrel <[email-masked]>
> *Cc: *IPPM Chairs <[email-masked]>, "[email-masked]" <[email-masked]>
> *Subject: *Re: [ippm] Vote at IPPM session
>
>
>
> Hi Adrian, et. al,
>
> I strongly believe that having agreed upon list of requirements that
> answer questions What? and Where? rather than How? is the most beneficial
> and not only for the discussion of applicability of in-situ OAM. What needs
> to be solved, measured that is not ameasureable by already existing OAM
> methods? I think that if we can compile such list, then it will be more
> clear what value may be added by developing new hybrid OAM method.
>
>
>
> Regards,
>
> Greg
>
>
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 26 Apr 2017 14:21:23 +0000

Greg,

I agree that a shared understanding and alignment on the problem space is critically important. My understanding is that we have already been doing that extensively, including the work on potential charter text discussed in Opsawg. That was, for example, a discussion solely about the problem to be solved.

What concerns me is the strict serialization and waterfall you seem to be advocating for.

— Carlos.

On Apr 26, 2017, at 9:22 AM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:

Dear Carlos,
thank you for the reference.
I've never called for perfecting but for rough consensus on what problem we need to address. How, IMHO, is secondary. And that is what may be not working in this discussion - we've been presented answer to how do measurement before we have reached an agreement that there's a problem to be solved.

Regards,
Greg

On Wed, Apr 26, 2017 at 6:13 AM, Carlos Pignataro (cpignata) <[email-masked]<mailto:[email-masked]>> wrote:
Dear Greg,

Thanks,

-- Carlos.

From: ippm <[email-masked]<mailto:[email-masked]>> on behalf of Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, April 26, 2017 at 5:22 AM
To: Adrian Farrel <[email-masked]<mailto:[email-masked]>>
Cc: IPPM Chairs <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [ippm] Vote at IPPM session

Hi Adrian, et. al,

Regards,
Greg

Cheers,
Adrian

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 26 Apr 2017 07:47:59 -0700

Dear Carlos,
thank you for the reference to the discussion in opsawg after the meeting
in Seoul. After reading through I was left with expectation of the followup
but I couldn't find any update since last January. I believe it will be
beneficial for all interested to have joint discussion thread.

Regards,
Greg

On Wed, Apr 26, 2017 at 7:21 AM, Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> Greg,
>
> I agree that a shared understanding and alignment on the problem space is
> critically important. My understanding is that we have already been doing
> that extensively, including the work on potential charter text discussed in
> Opsawg. That was, for example, a discussion solely about the problem to be
> solved.
>
> On Apr 26, 2017, at 9:22 AM, Greg Mirsky <[email-masked]> wrote:
>
> Dear Carlos,
> I've never called for perfecting but for rough consensus on what problem
> we need to address. How, IMHO, is secondary. And that is what may be not
>> Something like https://tools.ietf.org/html/dr
>> aft-brockners-inband-oam-requirements-03?
>>
>>
>>
>> Thus, I'd love to see this scoping text drafted and floated to the list
>> as an email or in a revision of the data format document.
>>
>> Cheers,
>> Adrian
>>

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 27 Apr 2017 12:05:00 +0100

Carlos is right, but we got to this discussion because after Frank spoke in Chicago there were a number of scoping questions that remained unclear with possible misunderstandings and assumptions.
 
It should be clear from the email exchanges on this list so far that the type of solution we develop will be considerably dependent on the scope of the problem.
 
I don't believe we (well, not me anyway) are calling for a detailed requirements document. Not even a problem statement document. But we do need a clear scoping statement.
 
As it happens, Frank has done some good work to address this with his new text, and I think we are close to done.
 
The only issue might be one of language. Saying "requirements" conveys different meanings to different people.
 
Probably the best way to ensure Greg's concerns are answered is for Greg to ask specific targeted questions (such as, but not specifically, "Do you intend this technology to be used in kitchen appliances?"). Then Frank can suggest answers, others can chime in for consensus, and the result can be briefly captured in the document.
 
Ciao,
Adrian
 
From: Carlos Pignataro (cpignata) [mailto:[email-masked]] 
Sent: 26 April 2017 15:21
To: Greg Mirsky
Cc: Adrian Farrel; IPPM Chairs; [email-masked]
Subject: Re: [ippm] Vote at IPPM session
 
Greg, 

 
— Carlos.

Dear Carlos, 
 
Regards,
Greg
 
On Wed, Apr 26, 2017 at 6:13 AM, Carlos Pignataro (cpignata) <[email-masked]> wrote:
Dear Greg,

 
Thanks,
 
-- Carlos.

Hi Adrian, et. al, 
 
Regards,
Greg

Cheers,
Adrian

---

From: <[email-masked]>
Date: Thu, 27 Apr 2017 13:56:08 +0000

Hi Carlos,

Let me pick up Frank's proposal:

OLD
"Designers of carrier protocols for IOAM designers must consider to put mechanisms in place to ensure that in-situ OAM data stays within an IOAM domain.."

NEW
"Designers of carrier protocols for IOAM designers must specify mechanisms to ensure that in-situ OAM data stays within an IOAM domain.."

I don't object that the operator is involved. But the operators task to me should be to turn an "in-situ OAM configuration knob" to solve the problem. This knob must be delivered by the protocol developers. "Consider" to me as a non-native speaker sounds like "think about and if it's getting complex, doing nothing is a valid option". 

As mentioned earlier, relying on a firewall to me is an option I'd like to avoid. 

Regards,

Ruediger

-----Ursprüngliche Nachricht-----
Von: Carlos Pignataro (cpignata) [mailto:[email-masked]] 
Gesendet: Sonntag, 23. April 2017 13:44
An: Al Morton
Cc: Geib, Rüdiger; [email-masked]; [email-masked]
Betreff: Re: [ippm] Vote at IPPM session

Hi,

Thanks!

— Carlos.

> However, if operators find that their network is affected by traffic 
> with iOAM data from another domain, they would be justified to discard 
> that traffic (as long as there is a clear domain limit expressed to 
> iOAM protocol designers of the future, as Frank has suggested).
> 
> This is the same ability afforded operators by declaring BMWG address 
> space for isolated testing-only, or the various private network 
> address spaces. Plenty of Net10 traffic escapes, and operators are not 
> obliged to drop it, but they certainly can.
> 
>> 
>> We could provide guidance to interconnecting network operators, as 
>> well, effectively allowing them to discard traffic containing 
>> unexpected iOAM data. This would be applicable to non-iOAM network 
>> operators, but might be more important for interconnecting operators 
>> who have established their own iOAM domain.
>>

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 27 Apr 2017 16:43:52 +0100

That's nice Rüdiger,

Although too many "designers" in the text.

Adrian

> From: ippm [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: 27 April 2017 14:56
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]
> "Designers of carrier protocols for IOAM designers must consider to put
> mechanisms in place to ensure that in-situ OAM data stays within an IOAM
> domain.."
> 
> NEW
> "Designers of carrier protocols for IOAM designers must specify mechanisms to
> ensure that in-situ OAM data stays within an IOAM domain.."
> 
> I don't object that the operator is involved. But the operators task to me should
> be to turn an "in-situ OAM configuration knob" to solve the problem. This knob
> must be delivered by the protocol developers. "Consider" to me as a non-native
> speaker sounds like "think about and if it's getting complex, doing nothing is a
> valid option".
> 
> This is a very useful discussion — although on the tail-end seems to be diverging
> from the original goal. I suggest that we should probably not take this discussion
> into all inter-domain permutations.
> 
> In other words, is there some case that the original text does not cover? The text
> includes “mechanisms in place to ensure that in-situ OAM data stays within an
> IOAM domain” and “provisions in place to ensure that IOAM data does not leak
> beyond the edge of an IOAM domain”.
> 
> The idea is that an IOAM Domain should take care of considerations because of
> e.g., increased packet sizes within its domain, and prevent leakage.
> 
> In many deployments, the latter (i.e., "ensuring that iOAM data stays within the
> domain, and does not leak beyond the exit edge”) would be a natural
> characteristic of the encapsulation. For example, when using IOAM with NSH, an
> IOAM domain would be an SFC domain, and thus removing NSH encap will also
> ensure IOAM does not leak. Same with VxLAN-GPE, etc.
>

---

From: Greg Mirsky <[email-masked]>
Date: Thu, 27 Apr 2017 09:59:38 -0700

Hi Adrian,
per my understanding WG adoption call is to answer three basic questions:

   - is the document reasonably readable;
   - does it solve outstanding operational problem, i.e. yet unaddressed or
   operationally burdensome scenario;
   - is the document technically sound.

I certainly say 'yes' to #1. I cannot say 'yes' neither on # 2, nor on #3:

   - On #2. It appears to me that we're discussing what iOAM does and what
   it should not do even though the stated scope of the document is the data
   types, not the mechanism of the protocol itself. Does that indicate that
   the scope is not yet clearly defined? And if the mechanism of in-situ OAM
   is in scope of the document, then I cannot find clear statement of the
   problem(s) it solves or was intended to solve. If anyone can help me with
   the answer, would be much obliged.
   - On #3. References to non-existent "out-of-band OAM" don't make the
   document technically sound, rather contrary to that. As the allusion that
   active OAM is inherently out-of-band and thus active OAM methods are not
   suitable to address needs of network operators.

Regards,
Greg

On Thu, Apr 27, 2017 at 4:05 AM, Adrian Farrel <[email-masked]> wrote:

> Carlos is right, but we got to this discussion because after Frank spoke
> in Chicago there were a number of scoping questions that remained unclear
> with possible misunderstandings and assumptions.
>
>
>
> It should be clear from the email exchanges on this list so far that the
> type of solution we develop will be considerably dependent on the scope of
> the problem.
>
>
>
> I don't believe we (well, not me anyway) are calling for a detailed
> requirements document. Not even a problem statement document. But we do
> need a clear scoping statement.
>
>
>
> Probably the best way to ensure Greg's concerns are answered is for Greg
> to ask specific targeted questions (such as, but not specifically, "Do you
> intend this technology to be used in kitchen appliances?"). Then Frank can
> suggest answers, others can chime in for consensus, and the result can be
> briefly captured in the document.
>
>
>
> Ciao,
>
> Adrian
>
>
>
> *From:* Carlos Pignataro (cpignata) [mailto:[email-masked]]
> *Sent:* 26 April 2017 15:21
> *To:* Greg Mirsky
> *Cc:* Adrian Farrel; IPPM Chairs; [email-masked]
>
> *Subject:* Re: [ippm] Vote at IPPM session
>
>
>
> Greg,
>
>
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Thu, 27 Apr 2017 17:20:20 +0000

Greg,

You seem to be presenting a position akin “half-glass is completely empty and has absolutely nothing in it”, or “it is empty, that glass, on one haf” – which is more polarized than “the glass is half empty”.

For #2, please read the threads. And the new text discussed on this list, which is forthcoming (so allow for more *concrete* comments on the text itself).
In this, Adrian has a great suggestion: pointed, targeted, narrow-scoped and explicit questions are more useful.

For #3, if your concern that amounts to technical unsoundness is the 4 instances of “out-of-band”, seems like an easy thing to resolve.

Thanks!

-- Carlos.

From: Greg Mirsky <[email-masked]>
Date: Thursday, April 27, 2017 at 12:59 PM
To: Adrian Farrel <[email-masked]>
Cc: Carlos Pignataro <[email-masked]>, IPPM Chairs <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [ippm] Vote at IPPM session

Hi Adrian,

  *   is the document reasonably readable;
  *   does it solve outstanding operational problem, i.e. yet unaddressed or operationally burdensome scenario;
  *   is the document technically sound.
I certainly say 'yes' to #1. I cannot say 'yes' neither on # 2, nor on #3:

  *   On #2. It appears to me that we're discussing what iOAM does and what it should not do even though the stated scope of the document is the data types, not the mechanism of the protocol itself. Does that indicate that the scope is not yet clearly defined? And if the mechanism of in-situ OAM is in scope of the document, then I cannot find clear statement of the problem(s) it solves or was intended to solve. If anyone can help me with the answer, would be much obliged.
  *   On #3. References to non-existent "out-of-band OAM" don't make the document technically sound, rather contrary to that. As the allusion that active OAM is inherently out-of-band and thus active OAM methods are not suitable to address needs of network operators.

Regards,
Greg

On Thu, Apr 27, 2017 at 4:05 AM, Adrian Farrel <[email-masked]<mailto:[email-masked]>> wrote:
Carlos is right, but we got to this discussion because after Frank spoke in Chicago there were a number of scoping questions that remained unclear with possible misunderstandings and assumptions.

Ciao,
Adrian

From: Carlos Pignataro (cpignata) [mailto:[email-masked]<mailto:[email-masked]>]
Sent: 26 April 2017 15:21
To: Greg Mirsky
Cc: Adrian Farrel; IPPM Chairs; [email-masked]<mailto:[email-masked]>

Greg,

— Carlos.

Dear Carlos,

Regards,
Greg

Thanks,

-- Carlos.

Hi Adrian, et. al,

Regards,
Greg

Cheers,
Adrian

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Fri, 28 Apr 2017 22:21:05 +0000

Thanks, Rüdiger — I like it too, and since this is targeting a wide range of developments (i.e., “Designers of carrier protocols for IOAM” includes disparate encaps), we might want to allow for some room and not be to prescriptive since there’s different capabilities available.

That said, as we are talking about edge-to-edge path constructs, there ought to think beyond “config knob” and also include operator collaboration in the solution.

I fully agree with you on roles.

Thanks,

Carlos.

> On Apr 27, 2017, at 11:43 AM, Adrian Farrel <[email-masked]> wrote:
> 
> That's nice Rüdiger,
>