Subject: Re: [spring] TSV-ART review of draft-ietf-spring-segment-routing-msdc-08

From: Gaurav Dawra <[email-masked]>
Date: Wed, 16 May 2018 19:43:51 -0700

Hi Martin,

Apologies from my end we had multiple discussions amongst authors.

For below comment - I will remove the citation of the really open paper. Will update in next rev:

Section will read like this now:

A flowlet is defined as a burst of packets from the same flow
followed by an idle interval.

With some ability to choose paths on the host, one may go from per-
flow load-sharing in the network to per-packet or per-flowlet.  The
host may select different segment routing instructions either per
packet, or per flowlet, and route them over different paths.  This
allows for solving the "elephant flow" problem in the data-center and
avoiding link imbalances.
 
Cheers,

Gaurav

> On Apr 9, 2018, at 8:02 AM, Martin Stiemerling <[email-masked]> wrote:
> 
> Hi Gaurav,
> 
> Am 01.03.18 um 16:01 schrieb Gaurav Dawra:
>> Hey Martin,
>> Seeking your feedback here. Regarding Section 7.1, The intent of the document is to describe one suggestive way and do not attempt to standardize any suggested mechanisms based considering the Informational nature. Authors do not attempt to fully solve – but to indicate how SR helps towards this.The details of how the hosts figure out what paths to take through the network such that the TCP and application are not affected is outside the scope of this document.
>> Would you prefer to add perhaps - “details outside the scope” or “this is one of the ways” or something else?
> 
> 
> Since the cited paper is really old and there is no up-to-date proof of any working implementation with real Internet traffic: Please remove this part.
> 
> This will require a lot of theoretical work, discussions the transport community, discussions with operating system vendors and testing in the wild open Internet.
> 
> Kind regards,
> 
>  Martin

---

From: Gaurav Dawra <[email-masked]>
Date: Wed, 16 May 2018 19:44:36 -0700

Hi Martin,

Apologies from my end we had few internal authors conversations on the points you have raised.

Please find below my response. I will be happy to discuss further, if needed.

<Gaurav> inline...

> On Apr 9, 2018, at 7:58 AM, Martin Stiemerling <[email-masked]> wrote:
> 
> Hi Gaurav,
> 
> This got lost on my end, sorry for this. The filter just moved these messages out of my sight... :-/
> 
> Am 15.02.18 um 05:47 schrieb Gaurav Dawra:
>> Hey Martin,
>> Sorry for late reply. Please see some comments inline[Gaurav]
>>> On Jan 9, 2018, at 2:25 PM, Martin Stiemerling <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>> Hi all,
>>> 
>>> I've reviewed this document as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors for their information and to allow them to address any issues raised. When done at the time of IETF Last Call, the authors should consider this review together with any other last-call comments they receive. Please always CC tsv-art@… if you reply to or forward this review.
>>> 
>>> Summary:
>>> This draft has serious issues in Section 7.1, 7.2 and in one part of Section3, described in the review, and needs to be rethought. The other sections are good AFAIK.
>>> 
>>> 
>>> Technicals:
>>> The overall draft looks ok, but the three points below look strange and need a fix before publication IMHO:
>>> 
>>> Both Sections, 7.1. and 7.2., are describing ideas, but not well proven funcationality and not even safe to use functionality. Both are some sort discussing that different paths in the network could be used by the end host traffic. This sounds pretty much like the Path Aware Networking Proposed Research Group (https://irtf.org/panrg) and hints to the fact that there is no commonly understand and accepted engineering solution in this space.
>>> 
>>> Section 7.1:
>>> [KANDULA04] is a really old reference that hasn't been followed up in recent times and even worse there is no evidence that this is going to work good enough or stable enough under real Internet traffic. Additionally, it is more than unclear how any modern TCP implementation will react to this
>> [Gaurav] Will get back on this.
> 
> I will reply to the other email dicussing this.
<Gaurav> I have replied to other thread.
> 
>>> 
>>> Section 7.2:
>>> This section describes an idea without detailing too much about any further aspects. Further it changes the commonly accepted notion of what an end host can do with the network. At best this would require a good definition of what an end host in your setting is, e.g., a highly modified piece of (at least) software that usually not found in OS availble on the market (yet?)
>>> Further communicating instantaneous path characteristics to a central point is potentially a bad idea, as the data is already outdated when reported by any node.
>> [Gaurav] I believe Authors are trying to highlight that Host which is defined in (https://tools.ietf.org/html/draft-ietf-spring-segment-routing-15) can influence the traffic based on the Calculations locally or jointly with the controller. Implementations can decide how much Centralized -vs- localized control is allowed at Host based on performance data collection.
> 
> Performance data collection (monitoring?) isn't crucial when it comes to timely (actually real-time) reaction. However, this section isn't just about performance data collection as it is about "Performance-aware routing" this seems to try to interact in real-time with the network behavior of TCP. This isn't even close to acceptable.
> 
> I would be ok to say that it is useful to collect performance data for offline analysis and improvement of the data network. However, this is at completely different magnitues of time scales.
> 
> I would recommend to remove the TCP part from this section entirely.
<Gaurav>Ack, will update in next rev:

Section will read like this:

;
 
   Knowing the path associated with flows/packets, the end host may
   deduce certain characteristics of the path on its own, and
   additionally use the information supplied with path information
   pushed from the controller or received via pull request.  The host
   may further share its path observations with the centralized agent,
   so that the latter may keep up-to-date network health map to assist
   other hosts with this information.
 
   For example, an application A.1 at HostA may pin a flow destined
   to HostZ via Spine node Node5 using label stack {16005, 16011}. The
   application A.1 may collect information on packet loss, deduced from
   Other offline mechanisms.  [There are some pingMesh mechanisms which 
   Can be used here]
 
    Through these mechanisms information to a centralized agent, e.g.
   after a flow completes, or periodically for longer lived flows.
   Next, using both local and/or global performance data, application
   A.1 as well as other applications sharing the same resources in the
   DC fabric may pick up the best path for the new flow, or update an
   existing path (e.g.: when informed of congestion on an existing
   path).
 
;

> 
> 
>>> 
>>> Section 3, 3rd bullet point:
>>> It is the foundation of TCP that the network is regarded as a black box and that you infer from the transmission of packets what the current state of the network path is. Inferring network path metrics (you mention SRTT, MSS, CWND ) is a bad idea, as this would required that all paths exhibit this and if not what is going to happen?
>>> It could be an interesting research field to change many points in TCP's behavior, but this once again points to the fact that this not the IETF works but IRTF or elsewhere.
>> [Gaurav] Martin, Authors are trying to suggest that TCP is rightly treating Network as Black Box. Authors are implying per path performance metrics as not cached. Is there some change in text you are suggesting?
> 
> 
> I would recommend to remove the 3rd bullet point completey. TCP isn't the place to remember "good" or "bad" paths. This is something the network could decide, e.g., rerouting TCP flows within the network or changing the forwarding path in the network for particular flows (if it is not routed).

<Gaurav> Ack, after discussion, we will remove the Section 3 - 3rd bullet point. Will update in next rev - coming shortly.

---

From: Gaurav Dawra <[email-masked]>
Date: Wed, 16 May 2018 19:47:04 -0700

Resending again with slight edits…

Hi Martin,

Please find below my response. I will be happy to discuss further, if required.

I will try to publish the next rev at the earliest.

<Gaurav> inline...

> On Apr 9, 2018, at 7:58 AM, Martin Stiemerling <[email-masked] <mailto:[email-masked]>> wrote:
> 
> Hi Gaurav,
> 
>>> On Jan 9, 2018, at 2:25 PM, Martin Stiemerling <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>>> 
>>> Hi all,
>>> 
>>> Both Sections, 7.1. and 7.2., are describing ideas, but not well proven funcationality and not even safe to use functionality. Both are some sort discussing that different paths in the network could be used by the end host traffic. This sounds pretty much like the Path Aware Networking Proposed Research Group (https://irtf.org/panrg <https://irtf.org/panrg>) and hints to the fact that there is no commonly understand and accepted engineering solution in this space.
>>> 
>>> Section 7.1:
>> [Gaurav] I believe Authors are trying to highlight that Host which is defined in (https://tools.ietf.org/html/draft-ietf-spring-segment-routing-15 <https://tools.ietf.org/html/draft-ietf-spring-segment-routing-15>) can influence the traffic based on the Calculations locally or jointly with the controller. Implementations can decide how much Centralized -vs- localized control is allowed at Host based on performance data collection.
> 

;

   For example, an application A.1 at HostA may pin a flow destined
   to HostZ via Spine node Node5 using label stack {16005, 16011}. The
   application A.1 may collect information on packet loss, deduced from
   other offline mechanisms.

;

---

From: Gaurav Dawra <[email-masked]>
Date: Wed, 23 May 2018 04:17:16 -0700

Hi Martin,

Thanks for the review. Any further comments here ? I will post the new version soon.

Cheers,

Gaurav

Sent from my iPhone

> On May 16, 2018, at 7:44 PM, Gaurav Dawra <[email-masked]> wrote:
> 
> Hi Martin,
>

---

From: Gaurav Dawra <[email-masked]>
Date: Fri, 25 May 2018 09:08:43 -0700

Hi Martin,

Thanks for review. I will post the new version. Hopefully, it will address all your comments and we can close this review.

Any updates on below response?

Cheers,

Gaurav

Sent from my iPhone

> On May 23, 2018, at 4:17 AM, Gaurav Dawra <[email-masked]> wrote:
> 
> Hi Martin,
>

---

From: Alvaro Retana <[email-masked]>
Date: Wed, 30 May 2018 15:55:36 -0400

Martin:

Hi!  How are you?

Gaurav just posted a revision.  Please take a look and let us know if the
changes address your concerns or not.

https://www.ietf.org/rfcdiff?url2=draft-ietf-spring-segment-routing-msdc-09

Thanks!!

Alvaro.

On May 25, 2018 at 12:08:46 PM, Gaurav Dawra ([email-masked]) wrote:

Hi Martin,

Thanks for review. I will post the new version. Hopefully, it will address
all your comments and we can close this review.

Cheers,

Gaurav

Sent from my iPhone

Hi Martin,

Thanks for the review. Any further comments here ? I will post the new
version soon.

Cheers,

Gaurav

Sent from my iPhone

Hi Martin,

Apologies from my end we had few internal authors conversations on the
points you have raised.

Please find below my response. I will be happy to discuss further, if
needed.

<Gaurav> inline...

Hi Gaurav,

This got lost on my end, sorry for this. The filter just moved these
messages out of my sight... :-/

Hey Martin,

On Jan 9, 2018, at 2:25 PM, Martin Stiemerling <[email-masked] <
mailto:[email-masked] <[email-masked]>>> wrote:

Hi all,

I've reviewed this document as part of the transport area review team's
ongoing effort to review key IETF documents. These comments were written
primarily for the transport area directors, but are copied to the
document's authors for their information and to allow them to address any
issues raised. When done at the time of IETF Last Call, the authors should
consider this review together with any other last-call comments they
receive. Please always CC tsv-art@… if you reply to or forward this review.

Summary:
This draft has serious issues in Section 7.1, 7.2 and in one part of
Section3, described in the review, and needs to be rethought. The other
sections are good AFAIK.

Technicals:
The overall draft looks ok, but the three points below look strange and
need a fix before publication IMHO:

Both Sections, 7.1. and 7.2., are describing ideas, but not well proven
funcationality and not even safe to use functionality. Both are some sort
discussing that different paths in the network could be used by the end
host traffic. This sounds pretty much like the Path Aware Networking
Proposed Research Group (https://irtf.org/panrg) and hints to the fact that
there is no commonly understand and accepted engineering solution in this
space.

Section 7.1:
[KANDULA04] is a really old reference that hasn't been followed up in
recent times and even worse there is no evidence that this is going to work
good enough or stable enough under real Internet traffic. Additionally, it
is more than unclear how any modern TCP implementation will react to this

Section 7.2:
This section describes an idea without detailing too much about any further
aspects. Further it changes the commonly accepted notion of what an end
host can do with the network. At best this would require a good definition
of what an end host in your setting is, e.g., a highly modified piece of
(at least) software that usually not found in OS availble on the market
(yet?)
Further communicating instantaneous path characteristics to a central point
is potentially a bad idea, as the data is already outdated when reported by
any node.

[Gaurav] I believe Authors are trying to highlight that Host which is
defined in (https://tools.ietf.org/html/draft-ietf-spring-segment-routing-15)
can influence the traffic based on the Calculations locally or jointly with
the controller. Implementations can decide how much Centralized -vs-
localized control is allowed at Host based on performance data collection.

Performance data collection (monitoring?) isn't crucial when it comes to
timely (actually real-time) reaction. However, this section isn't just
about performance data collection as it is about "Performance-aware
routing" this seems to try to interact in real-time with the network
behavior of TCP. This isn't even close to acceptable.

I would be ok to say that it is useful to collect performance data for
offline analysis and improvement of the data network. However, this is at
completely different magnitues of time scales.

;

*   Knowing the path associated with flows/packets, the end host may*

*   deduce certain characteristics of the path on its own, and*

*   additionally use the information supplied with path information*

*   pushed from the controller or received via pull request.  The host*

*   may further share its path observations with the centralized agent,*

*   so that the latter may keep up-to-date network health map to assist*

*   other hosts with this information.*

 *   For example, an application A.1 at HostA may pin a flow destined*

*   to HostZ via Spine node Node5 using label stack {16005, 16011}. The*

*   application A.1 may collect information on packet loss, deduced from*

*   Other offline mechanisms.  [There are some pingMesh mechanisms which *

*   Can be used here]*

 *    Through these mechanisms information to a centralized agent, e.g.*

*   after a flow completes, or periodically for longer lived flows.*

*   Next, using both local and/or global performance data, application*

*   A.1 as well as other applications sharing the same resources in the*

*   DC fabric may pick up the best path for the new flow, or update an*

*   existing path (e.g.: when informed of congestion on an existing*

*   path).*

;

Section 3, 3rd bullet point:
It is the foundation of TCP that the network is regarded as a black box and
that you infer from the transmission of packets what the current state of
the network path is. Inferring network path metrics (you mention SRTT, MSS,
CWND ) is a bad idea, as this would required that all paths exhibit this
and if not what is going to happen?
It could be an interesting research field to change many points in TCP's
behavior, but this once again points to the fact that this not the IETF
works but IRTF or elsewhere.

[Gaurav] Martin, Authors are trying to suggest that TCP is rightly treating
Network as Black Box. Authors are implying per path performance metrics as
not cached. Is there some change in text you are suggesting?

I would recommend to remove the 3rd bullet point completey. TCP isn't the
place to remember "good" or "bad" paths. This is something the network
could decide, e.g., rerouting TCP flows within the network or changing the
forwarding path in the network for particular flows (if it is not routed).

<Gaurav> Ack, after discussion, we will remove the Section 3 - 3rd bullet
point. Will update in next rev - coming shortly.

Kind regards,

 Martin