Subject: Re: [tcpm] WGLC for draft-ietf-tcpm-dctcp-04

From: "De Schepper, Koen (Nokia - BE)" <[email-masked]>
Date: Wed, 1 Mar 2017 10:18:12 +0000

I have a comment on the following text:

   It would be useful to implement DCTCP as additional actions on top of
   an existing congestion control algorithm like NewReno.  The DCTCP
   implementation MAY also allow configuration of resetting the value of
   DCTCP.Alpha as part of processing any loss episodes.

Setting alpha to its max value (btw resetting might imply setting to zero) has many 
complications and in the current Linux implementation turned out to not function.
The Linux patch I provided additionally halves the window (once per RTT) when a 
drop is encountered, independent from alpha. This is also, as far as I understood, 
how the Windows version works. Both implementations are working now in
drop-based networks next to Classic TCP flows.

As far as I know there is no working implementation that is setting alpha to max so 
I propose to describe the working implementation instead of a theoretical one.

For the rest I support publication.

Koen.

> -----Original Message-----
> From: tcpm [mailto:[email-masked]] On Behalf Of Scharf, Michael
> (Nokia - DE)
> Sent: dinsdag 14 februari 2017 17:32
> To: [email-masked]
> Cc: [email-masked]
> Subject: [tcpm] WGLC for draft-ietf-tcpm-dctcp-04
> 
> Hi,
> 
> TCPM chairs are not aware of outstanding issues in draft-ietf-tcpm-dctcp-04
> "Datacenter TCP (DCTCP): TCP Congestion Control for Datacenters".
> 
> Therefore, this e-mail starts a working group last call for draft-ietf-tcpm-
> dctcp-04.
> 
> The WGLC runs until Friday March 3. Please send any comments to the TCPM
> mailing list by then. Statements supporting the publication of the document
> are also helpful.
> 
> We assume that TCPM working group is aware of the IPR disclosure related
> to draft-bensley-tcpm-dctcp-00 (https://datatracker.ietf.org/ipr/2319/). A
> separate note will be sent out to confirm that all IPR has already been
> disclosed, in conformance with BCPs 78 and 79.
> 
> The draft is available at https://tools.ietf.org/html/draft-ietf-tcpm-dctcp-04
> 
> Thanks
> 
> Michael
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Praveen Balasubramanian <[email-masked]>
Date: Sat, 4 Mar 2017 00:41:02 +0000

Yes DCTCP adjusts cwnd at most once every round trip.

See step 4:

If the acknowledgment number is less than or equal to
       DCTCP.WindowEnd, stop processing.  Otherwise, the end of the
       observation window has been reached, so proceed to update the
       congestion estimate as follows:

So steps 5 through 8 are conditional and only executed once per round. What is a bit confusing is that the cwnd calculation is not indicated as step 9. I will fix this in an update to the draft.

Other than this we also have the following clarifying sentence:
Just as specified in [RFC3168<https://tools.ietf.org/html/rfc3168>], DCTCP does not react to congestion
   indications more than once for every window of data.

From: tcpm [mailto:[email-masked]] On Behalf Of Yuchung Cheng
Sent: Tuesday, February 28, 2017 3:03 PM
To: [email-masked]
Cc: Abdul Kabbani <[email-masked]>
Subject: Re: [tcpm] WGLC for draft-ietf-tcpm-dctcp-04

The wordings on cwnd adjustment in section 3.3 can be more clear:

"""
Rather than always halving the congestion window as described in
   [RFC3168], when the sender receives an indication of congestion
   (ECE), the sender MUST update cwnd as follows:

      cwnd = cwnd * (1 - DCTCP.Alpha / 2)

   Thus, when no sent byte experienced congestion, DCTCP.Alpha equals
   zero, and cwnd is left unchanged.  When all sent bytes experienced
   congestion, DCTCP.Alpha equals one, and cwnd is reduced by half.
   Lower levels of congestion will result in correspondingly smaller
   reductions to cwnd.
"""

AFAICT DCTCP paper only adjusts cwnd at most every round trip (which is also how Linux implements it). But the text implies the cwnd reduction can happen for every ECE mark?

On Tue, Feb 14, 2017 at 8:31 AM, Scharf, Michael (Nokia - DE) <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

TCPM chairs are not aware of outstanding issues in draft-ietf-tcpm-dctcp-04 "Datacenter TCP (DCTCP): TCP Congestion Control for Datacenters".

Therefore, this e-mail starts a working group last call for draft-ietf-tcpm-dctcp-04.

The WGLC runs until Friday March 3. Please send any comments to the TCPM mailing list by then. Statements supporting the publication of the document are also helpful.

We assume that TCPM working group is aware of the IPR disclosure related to draft-bensley-tcpm-dctcp-00 (https://datatracker.ietf.org/ipr/2319/). A separate note will be sent out to confirm that all IPR has already been disclosed, in conformance with BCPs 78 and 79.

Thanks

Michael
_______________________________________________
tcpm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tcpm

---

From: Praveen Balasubramanian <[email-masked]>
Date: Sat, 4 Mar 2017 00:52:13 +0000

The text around allowing a reset of Alpha was added on suggestions from the Linux implementation experience. The text already suggests that this is optional. The Windows implementation does not reset the value of Alpha or the other tracking variables. I think that we don't need to remove this text. If you still strongly object and I don't hear any other opinions, I will remove it in the next revision. 

>> The Linux patch I provided additionally halves the window (once per RTT) when a drop is encountered, independent from alpha
The handling of a timeout and dup acks should ideally be identical to NewReno. The only change in response is in presence of ECN marks.

Thanks

-----Original Message-----
From: tcpm [mailto:[email-masked]] On Behalf Of De Schepper, Koen (Nokia - BE)
Sent: Wednesday, March 1, 2017 2:18 AM
To: Scharf, Michael (Nokia - DE) <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: [tcpm] WGLC for draft-ietf-tcpm-dctcp-04

Setting alpha to its max value (btw resetting might imply setting to zero) has many complications and in the current Linux implementation turned out to not function.
The Linux patch I provided additionally halves the window (once per RTT) when a drop is encountered, independent from alpha. This is also, as far as I understood, how the Windows version works. Both implementations are working now in drop-based networks next to Classic TCP flows.

As far as I know there is no working implementation that is setting alpha to max so I propose to describe the working implementation instead of a theoretical one.

Koen.

> TCPM chairs are not aware of outstanding issues in 
> draft-ietf-tcpm-dctcp-04 "Datacenter TCP (DCTCP): TCP Congestion Control for Datacenters".
> 
> Therefore, this e-mail starts a working group last call for 
> draft-ietf-tcpm- dctcp-04.
> 
> The WGLC runs until Friday March 3. Please send any comments to the 
> TCPM mailing list by then. Statements supporting the publication of 
> the document are also helpful.
> 
> We assume that TCPM working group is aware of the IPR disclosure 
> related to draft-bensley-tcpm-dctcp-00 
> (https://datatracker.ietf.org/ipr/2319/). A separate note will be sent 
> out to confirm that all IPR has already been disclosed, in conformance with BCPs 78 and 79.
> 
> The draft is available at 
> https://tools.ietf.org/html/draft-ietf-tcpm-dctcp-04
> 
> Thanks
> 
> Michael

---

From: Yuchung Cheng <[email-masked]>
Date: Sun, 5 Mar 2017 08:34:10 -0800

On Fri, Mar 3, 2017 at 4:41 PM, Praveen Balasubramanian
<[email-masked]> wrote:
> Just as specified in [RFC3168], DCTCP does not react to congestion
>
Thanks for clarification. I notice the draft is also less clear about
the window reduction on packet losses. For example, if sender
experiences ECE and then losses in the same window of data. Does DCTCP
reduces the window twice (once on ECN, then again on Loss)?

The draft implies cwnd would be reduced twice:

A DCTCP sender MUST deal with loss episodes in the same way as
   conventional TCP.  In case of a timeout or fast retransmit or any
   change in delay (for delay based congestion control), the cwnd and
   other state variables like ssthresh must be changed in the same way
   that a conventional TCP would have changed them.

AFAICT Linux's implementation for both DCTCP and RFC3168 would only
reduce cwnd once on ECN + loss. Since this is a common scenario, it'd
be nice to be explicit in the DCTCP draft.

> On Tue, Feb 14, 2017 at 8:31 AM, Scharf, Michael (Nokia - DE) <[email-masked]> wrote:
>
> Hi,
>

---

From: marcelo bagnulo braun <[email-masked]>
Date: Sun, 5 Mar 2017 20:42:39 +0100

I support this document moving forward.

---

From: "De Schepper, Koen (Nokia - BE)" <[email-masked]>
Date: Wed, 8 Mar 2017 16:23:36 +0000

> From: Praveen Balasubramanian [mailto:[email-masked]]
> Sent: zaterdag 4 maart 2017 1:52
> To: De Schepper, Koen (Nokia - BE) <koen.de_schepper@nokia-bell-
> labs.com>; Scharf, Michael (Nokia - DE) <[email-masked]>;
> [email-masked]
> Cc: [email-masked]
> Subject: RE: WGLC for draft-ietf-tcpm-dctcp-04
> 
> The text around allowing a reset of Alpha was added on suggestions from the
> Linux implementation experience.

[KDS] I see it was added from the Linux implementation, but clearly not based on
the experience ;-). This implementation is not working.
I don't know if the FreeBSD implementation has drop compatibility, and how it works.
If that has a working implementation by setting alpha to max, we can keep the text.

> The text already suggests that this is
> optional.

[KDS] That is OK, as in the Linux patch it is also optional (default on). 
The new parameter is called dctcp_drop_compatible as it does not have anything
to do with alpha clamping.

Koen.

> The Windows implementation does not reset the value of Alpha or
> the other tracking variables. I think that we don't need to remove this text. If
> you still strongly object and I don't hear any other opinions, I will remove it in
> the next revision.
> 
> >> The Linux patch I provided additionally halves the window (once per RTT)
> when a drop is encountered, independent from alpha
> The handling of a timeout and dup acks should ideally be identical to
> NewReno. The only change in response is in presence of ECN marks.
> 
> Thanks
> 
> Setting alpha to its max value (btw resetting might imply setting to zero) has
> many complications and in the current Linux implementation turned out to
> not function.
> The Linux patch I provided additionally halves the window (once per RTT)
> when a drop is encountered, independent from alpha. This is also, as far as I
> understood, how the Windows version works. Both implementations are
> working now in drop-based networks next to Classic TCP flows.
> 
> As far as I know there is no working implementation that is setting alpha to
> max so I propose to describe the working implementation instead of a
> theoretical one.
>

---

From: Bob Briscoe <[email-masked]>
Date: Sun, 12 Mar 2017 01:45:00 +0000

I can confirm that all my earlier concerns have been addressed.
So I support this draft going forward, assuming the proposed 
improvements around loss handling currently being discussed will be sorted.

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Praveen Balasubramanian <[email-masked]>
Date: Mon, 13 Mar 2017 16:38:22 +0000

> Thanks for clarification. I notice the draft is also less clear about the window reduction on packet losses. For example, if sender experiences ECE and then losses in the same window of > data. Does DCTCP reduces the window twice (once on ECN, then again on Loss)?

DCTCP will reduce the window only once per window of data for the loss signals of ECN and duplicate ACKs. On a timeout cwnd will be unconditionally reduced to 1 MSS even if there was a preceding cwnd reduction due to a loss signal in a given window of data. 

>    that a conventional TCP would have changed them.

This wording comes from the fact that in Windows at least DCTCP is implemented as an extension to NewReno, in the sense that only the ECN handler behaves differently. I will change this wording to explicitly mention the above behavior of handling other loss events. 

> AFAICT Linux's implementation for both DCTCP and RFC3168 would only reduce cwnd once on ECN + loss. Since this is a common scenario, it'd be nice to be explicit in the DCTCP draft.

Yes agreed. We will publish an update soon. 

Thanks

-----Original Message-----
From: Yuchung Cheng [mailto:[email-masked]] 
Sent: Sunday, March 5, 2017 8:34 AM
To: Praveen Balasubramanian <[email-masked]>
Cc: [email-masked]; Abdul Kabbani <[email-masked]>
Subject: Re: [tcpm] WGLC for draft-ietf-tcpm-dctcp-04

>    indications more than once for every window of data.
Thanks for clarification. I notice the draft is also less clear about the window reduction on packet losses. For example, if sender experiences ECE and then losses in the same window of data. Does DCTCP reduces the window twice (once on ECN, then again on Loss)?

AFAICT Linux's implementation for both DCTCP and RFC3168 would only reduce cwnd once on ECN + loss. Since this is a common scenario, it'd be nice to be explicit in the DCTCP draft.

---

From: Praveen Balasubramanian <[email-masked]>
Date: Mon, 13 Mar 2017 16:41:08 +0000

> The DCTCP

Changing this to:

If experimental results show a benefit, a DCTCP implementation MAY reset the value of DCTCP.Alpha to 1 value as part of processing any loss episodes. 

Thanks

-----Original Message-----
From: De Schepper, Koen (Nokia - BE) [mailto:[email-masked]] 
Sent: Wednesday, March 8, 2017 8:24 AM
To: Praveen Balasubramanian <[email-masked]>; Scharf, Michael (Nokia - DE) <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: RE: WGLC for draft-ietf-tcpm-dctcp-04

> The text around allowing a reset of Alpha was added on suggestions 
> from the Linux implementation experience.

[KDS] I see it was added from the Linux implementation, but clearly not based on the experience ;-). This implementation is not working.
I don't know if the FreeBSD implementation has drop compatibility, and how it works.
If that has a working implementation by setting alpha to max, we can keep the text.

> The text already suggests that this is optional.

[KDS] That is OK, as in the Linux patch it is also optional (default on). 
The new parameter is called dctcp_drop_compatible as it does not have anything to do with alpha clamping.

Koen.

> The Windows implementation does not reset the value of Alpha or the 
> other tracking variables. I think that we don't need to remove this 
> text. If you still strongly object and I don't hear any other 
> opinions, I will remove it in the next revision.
> 
> when a drop is encountered, independent from alpha The handling of a 
> timeout and dup acks should ideally be identical to NewReno. The only 
> change in response is in presence of ECN marks.
> 
> Thanks
> 
> Setting alpha to its max value (btw resetting might imply setting to 
> zero) has many complications and in the current Linux implementation 
> turned out to not function.
> The Linux patch I provided additionally halves the window (once per 
> RTT) when a drop is encountered, independent from alpha. This is also, 
> as far as I understood, how the Windows version works. Both 
> implementations are working now in drop-based networks next to Classic TCP flows.
> 
> As far as I know there is no working implementation that is setting 
> alpha to max so I propose to describe the working implementation 
> instead of a theoretical one.
> 
> > From: tcpm [mailto:[email-masked]] On Behalf Of Scharf, 
> > Michael (Nokia - DE)
> > draft-ietf-tcpm-dctcp-04 "Datacenter TCP (DCTCP): TCP Congestion 
> > Control
> for Datacenters".
> >
> > (https://datatracker.ietf.org/ipr/2319/). A separate note will be 
> > sent out to confirm that all IPR has already been disclosed, in 
> > conformance with
> BCPs 78 and 79.
> >

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 13 Mar 2017 15:11:52 -0700

On Mon, Mar 13, 2017 at 9:38 AM, Praveen Balasubramanian <
[email-masked]> wrote:

> the window reduction on packet losses. For example, if sender experiences
> ECE and then losses in the same window of > data. Does DCTCP reduces the
> window twice (once on ECN, then again on Loss)?
>
> DCTCP will reduce the window only once per window of data for the loss
> signals of ECN and duplicate ACKs. On a timeout cwnd will be
> unconditionally reduced to 1 MSS even if there was a preceding cwnd
> reduction due to a loss signal in a given window of data.
>
> This wording comes from the fact that in Windows at least DCTCP is
> implemented as an extension to NewReno, in the sense that only the ECN
> handler behaves differently. I will change this wording to explicitly
> mention the above behavior of handling other loss events.
>
Thanks for the clarification. I assume you mean Reno not NewReno. NewReno
is not a congestion control... okay I really need to start a thread to
correct this.

> reduce cwnd once on ECN + loss. Since this is a common scenario, it'd be
> nice to be explicit in the DCTCP draft.
>
> > So steps 5 through 8 are conditional and only executed once per round.
> What is a bit confusing is that the cwnd calculation is not indicated as
> step 9. I will fix this in an update to the draft.
> >
> >
> >
> Thanks for clarification. I notice the draft is also less clear about the
> window reduction on packet losses. For example, if sender experiences ECE
> and then losses in the same window of data. Does DCTCP reduces the window
> twice (once on ECN, then again on Loss)?
>
> AFAICT Linux's implementation for both DCTCP and RFC3168 would only reduce
> cwnd once on ECN + loss. Since this is a common scenario, it'd be nice to
> be explicit in the DCTCP draft.
>
> >
> >
> >
> >
> >
> > AFAICT DCTCP paper only adjusts cwnd at most every round trip (which is
> also how Linux implements it). But the text implies the cwnd reduction can
> happen for every ECE mark?
> >
> >
> >
> >
> >
> draft-ietf-tcpm-dctcp-04.
> >
> to draft-bensley-tcpm-dctcp-00 (https://datatracker.ietf.org/ipr/2319/).
> A separate note will be sent out to confirm that all IPR has already been