Subject: Re: [tcpm] Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

From: Bob Briscoe <[email-masked]>
Date: Mon, 3 Jul 2017 01:42:49 +0100

Padma,

I think we have converged on most, if not all, issues - see comments 
inline below. I don't think we will have time to make further updates to 
the draft before the deadline in about 24hrs (cos I have other drafts 
too). But we will certainly do these ASAP, esp. if the draft is adopted.

[trimmed points on which we agree]

more...

On 28/06/17 15:53, Padma Bhooma wrote:
> Hi Bob
>
> Thanks for your responses. Please see inline.
>
>> On Jun 19, 2017, at 11:38 AM, Bob Briscoe <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> On 17/06/17 17:27, Padma Bhooma wrote:
>>> Hi Marcelo and Bob,
>>>
>>> I have reviewed the document draft-bagnulo-tcpm-generalized-ecn-04 
>>> and I am sending my comments below. Overall, I think there is value 
>>> in working on these additions or enhancements to RFC 3168 in tcpm 
>>> working group. I am willing to review and provide feedback as needed.
>>>
>>>   * *Section 3.2.1.3*- If the SYN-ACK does not support AccECN, TCP
>>>     must conservatively reduce its IW and SHOULD reduce it to 1
>>>     SMSS. This clause is very conservative because AccECN support
>>>     can be deployed on servers slowly. Even if both end hosts
>>>     support AccECN, there are networks with performance proxies that
>>>     will not allow the feedback to propagate correctly. On those
>>>     networks, TCP connections with ECT on SYN will always start with
>>>     1MSS of IW. I think this is too conservative and might add
>>>     multiple RTT to grow the window. I agree with the value in
>>>     adding ECT marking to SYN and SYN-ACK in terms of the latency
>>>     benefits that might bring to an end user.
>>>
>>>   * I think “pessimistic ECT and cache successes” strategy described
>>>     in 4.2.1 sounds like a reasonable one.
>>>
>> [BB] What's your reasoning for preferring this one? We recommended 
>> "optimistic ECT and cache failures”.
>
> [PB] I think, 1MSS may not be enough for clients that are not web 
> based. This assumption may not apply to connections that multiplex 
> data on a single connection. Starting at 1MSS will make the client 
> need more RTTs to reach the congestion window that is needed if the 
> client wants to send more than an HTTP get. From the client side, the 
> number of servers they connect to can be large (one-to-many) and using 
> 1MSS on every one of them can effect performance. When there are 
> performance proxies or other kind of middle boxes in the network that 
> will not make AccECN work end-to-end, trying optimistic ECT can 
> potentially have a performance hit.
[BB] If the client can somehow detect that a proxy is the cause of no 
ECN support while attached to a certain access network {Note 1}, it 
should be able to disable ECT until it attaches to another network. Then 
it will be able to use a full IW on all connections to every remote server.

{Note 1}: e.g. using a test access to a remote server known to support ECN.

We can suggest a strategy like this in the informative section, but I 
think this is not for standardization - implementers ought to have the 
freedom to innovate here.

>
>
>>
>>>
>>>   * *Section 3.2.3 ECT marking on Pure Acks*- It felt like the
>>>     document suggested further discussion on setting ECT on pure
>>>     Acks, please correct me if I am wrong here. Setting ECT on pure
>>>     Acks can increase the load on an AQM that might already be under
>>>     load. This is because the number of Acks can be large and the
>>>     information conveyed to the peer is redundant because asks are
>>>     cumulative. TCP acknowledgements also do not solicit a response
>>>     from the peer most of the times. So the chances of not echoing
>>>     CE on an Ack is high.
>>>
>> [BB] Let's take these points one at a time:
>> PB: Setting ECT on pure ACKs could increase the load on an AQM
>> BB: An AQM processes packets of all sizes anyway. If pure ACKs are 
>> not ECT, it still has to decide whether to drop each pure ACK. 
>> Marking instead of dropping doesn't make any more work for the AQM.
>
> [PB] Here by "load" I meant that the queue sizes can grow and increase 
> the probability of a drop for all traffic. I agree that if the queue 
> sizes are manageable it will be great to not drop Acks also.
[BB] I'm not so worried about this, cos:
* Unless loss/marking levels are ridiculously high, queue length will 
only increase by the marking probability (e.g. 1%)
* preserving a few ECT ACKs makes even less difference to a queue if it 
contains some data packets as well.
* once the control loop has had time to respond (1RTT) the queue gets 
regulated to the same length with or without ECN

All three factors together make this issue very small.

>
>>
>> PB: The information conveyed to the peer is redundant because ACKs 
>> are cumulative.
>> BB: Just because the ACKnumber info is superseded by the next ACK 
>> does not mean the ECN marking is superseded.
>
> [PB] Yes, agreed. That is also true for other information like SACK 
> that can be on an ACK.
>>
>> PB: TCP acknowledgements do not solicit a response from the peer most 
>> of the times. So the chances of not echoing CE on an Ack is high.
>> BB: Just because pure ACKs do not solicit a specific response (no ACK 
>> of an ACK) does not mean the data sender is not continuing to send 
>> data packets, on which ECE can be set (if RFC3168 feedback) or on 
>> which the CE counter can be increased (AccECN feedback).
>>
>> RFC3168 actually says this:
>>     (One simple possibility would be for the sender to reduce its
>>     congestion window when it receives a pure ACK packet with the CE
>>     codepoint set).
>> Consider this pure ACK example with data solely in the S->C 
>> direction, where S is the server.
>>   * Say 3 in 1000 pure ACKs in the C->S direction are CE-marked, 
>> meaning the C->S marking probability is 0.3%
>>   * This might be because 0.3% of data packets in other flows (e.g. a 
>> YouTube upload) are being CE-marked as well.
>>   * So S needs to tell C that it has seen 3 CE marked packets (which 
>> it will do on the data packets it is sending, whether with RFC3168 
>> feedback or AccECN feedback).
>>   * C is not (currently) sending any data to S, but it will reduce 
>> its cwnd for if it does start sending data.
>
> [PB] I think, it will be good to make AccECN feedback a strong 
> requirement for propagating CE marking on Acks also. Imagine a 
> bottleneck link that is congested due to a large flow and a few other 
> flows sending Acks alone. Then propagating CE marking on Acks can make 
> flows that will never send respond to congestion unnecessarily.
[BB] Surely it is OK to reduce cwnd on a flow that is not sending data, 
cos it will never use the reduced cwnd (so it will not "respond to 
congestion unnecessarily"). However, if it does start sending data, it 
will use the reduced cwnd (as we already discussed on a later point).

>>
>>
>>>
>>>   * *Section 3.2.4 Window Probe*- Adding ECT to window probes is
>>>     really good because this traffic is not too large on any
>>>     connection and it requires the peer to respond so that there is
>>>     a way to echo the CE marking. One modification that might be
>>>     worth discussing is to require congestion response to CE marking
>>>     on a window probe only when the receive window is opened. I will
>>>     try to explain why this is reasonable. If congestion at the
>>>     bottleneck link is transient, then lowering the congestion
>>>     window in response to CE on a window probe that does not open
>>>     the receive window might not be necessary because there won’t be
>>>     any data sent afterwards. This can actually happen multiple
>>>     times on a connection because a connection can go on for a long
>>>     time sending multiple window probes when the peer does not open
>>>     the window. So reducing congestion window only when the receive
>>>     window is opened as a response to CE seems to be accurate in
>>>     terms of timing and reaction to congestion notification.
>>>
>> [BB] On the other hand, if the window was constrained solely because 
>> the receive window was low and there had been a CE mark on a window 
>> probe many round trips ago, but not on more recent window probes, it 
>> would not be correct to reduce cwnd such a long time after the 
>> congestion had occurred.
>>
>> I think it would be better to reduce cwnd immediately there is a CE 
>> on a window probe, increase cwnd as normal for every non-CE-marked 
>> window probe (even if rwnd is still the limiting factor). But also 
>> rely on cwnd validation so that cwnd cannot grow excessively if it is 
>> not being used.
>
> [PB] I think, it will be good to have some clarifying text here. Both 
> the points that I made above (about Section 3.2.4 and Section 4.4.1) 
> are kind of related to connections that see CE marking on control 
> packets but do not have a lot of data to send or they are limited by 
> receive window. The mechanisms for maintaining congestion window 
> correctly when a CE marking is received in these scenarios is not well 
> documented in RFC 3168 or in this draft. I agree that there are 
> congestion window validation mechanisms (RFC 7661) that might apply 
> here. So it will be good to give some recommendation here.
[BB]: We'll write a summary of the discussion here. But bear in mind 
that this window probe case is already rather a corner-case. Whatever, I 
think this should only be advice, not normative requirements - I would 
hope implementers will experiment to find good strategies.

>
> [PB] Are there any changes needed on the AQM side in the network to 
> support CE marking on control packets? I do not think so but it will 
> be good to get a clarification.
[BB]: No. We could write into the draft that no network changes are 
needed. However, a TCP draft does not normally have to say that it 
requires no changes to the network.

>
>>
>>
>> More generally, I know some people are starting to get really 
>> confused over how to put together all the piecemeal modifications to 
>> ECN. I think that would require an informational "ECN roadmap" draft. 
>> No promises, but I guess I am someone who would be well-placed to 
>> know what to write in such a draft.
>>
>
> [PB] Yes, this is true, especially with the latest experimentation 
> around RFC 3168. It will be good to have an ECN roadmap.
>
>>
>>>
>>>  *
>>>     *Fallback mechanisms*- The document also discusses several
>>>     fallback mechanisms to handle incompatibilities in the network.
>>>     If we want implementations to consider these cases, it will be
>>>     useful to formalize these detection and fallback mechanisms. I
>>>     found that some of these mechanisms are being discussed in
>>>     section 4 which was tagged as informative.
>>>
>> [BB] I have just checked, and I cannot find any case where there is 
>> fall-back text in section 4 that ought to be normative. If you found 
>> any, pls say where. We tried to ensure that the recommended approach 
>> was always in section 3, even if section 4 discussed (then rejected) 
>> alternatives.
>
> [PB] Ok. Some measurements in the field will help to support the need 
> for these fallback mechanisms. I wonder if there is a way to make them 
> as "strong recommendations” so that the implementors do not think of 
> them as optional.
[BB]: If you are trying to make a "SHOULD" into a strong "SHOULD" that 
can be done by stating the only cases where the "SHOULD" does not apply 
(i.e. MUST ... unless ...). However, I don't know which text you are 
talking about. If you have an issue with particular sentences, pls point 
them out.

>
>>
>> In summary, as this conversation progresses, we might agree further 
>> changes. However, so far I have queued up the following list of 
>> changes to the text to address your points:
>> * Intro: Recommend complementary RFCs or drafts
>> * 3.2.1.3: Clarify that using IW=1SMSS only applies when using the 
>> strategies recommended in the previous two sections.
>> * When fall-back due to ECT on SYN (3.2.1.4) or SYN-ACK (3.2.2.3), 
>> consider disabling ECT on other control packets (esp. pure ACKs).
>> * Add detection of black-holing of pure ACKs as an open issue (hence 
>> why we have to take the approach in the previous bullet)
>> * Add extra section on general fall-back if no ECT packet has ever 
>> been ack'd
>> * And we will certainly acknowledge your contribution, and I hope you 
>> will continue to track changes to this draft carefully and let us 
>> know if you have implementation experience - thank you.
>
> Sounds good. Thank you!
[BB] Thank you.

Bob

>
> Padma
>
>>
>>
>> Bob
>>
>>>
>>> Thanks,
>>> Padma
>>>
>>>
>>>>
>>>>
>>>>   1. Working group acceptance call
>>>>      draft-bagnulo-tcpm-generalized-ecn-04
>>>>      (Scharf, Michael (Nokia - DE/Stuttgart))
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Fri, 16 Jun 2017 06:06:43 +0000
>>>> From: "Scharf, Michael (Nokia - DE/Stuttgart)"
>>>> <[email-masked] <mailto:[email-masked]>>
>>>> To: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
>>>> <mailto:[email-masked]>>
>>>> Cc: "[email-masked] <mailto:[email-masked]>" 
>>>> <[email-masked] <mailto:[email-masked]>>
>>>> Subject: [tcpm] Working group acceptance call
>>>> draft-bagnulo-tcpm-generalized-ecn-04
>>>> Message-ID:
>>>> <[email-masked] 
>>>> <mailto:[email-masked]>>
>>>>
>>>> Content-Type: text/plain; charset="us-ascii"
>>>>
>>>> Hi all,
>>>>
>>>> As requested by the authors, this e-mail starts a working group 
>>>> acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which 
>>>> will run on the list until June 30.
>>>>
>>>> The proposal is to add a new milestone to the  TCPM charter
>>>>
>>>>  Submit document on adding Explicit Congestion Notification (ECN) 
>>>> to TCP Control Packets to the IESG for publication as Experimental RFC
>>>>
>>>> and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.
>>>>
>>>> If you believe that TCPM should work on a document in this space, 
>>>> and in particular if you are willing to review it, please reply to 
>>>> this e-mails. Please also speak up if you have any concerns or 
>>>> objections.
>>>>
>>>> Thanks
>>>>
>>>> Michael
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL: 
>>>> <https://mailarchive.ietf.org/arch/browse/tcpm/attachments/20170616/151f93e9/attachment.html>
>>>>
>>>> ------------------------------
>>>>
>>>> Subject: Digest Footer
>>>>
>>>> _______________________________________________
>>>> tcpm mailing list
>>>> [email-masked] <mailto:[email-masked]>
>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> End of tcpm Digest, Vol 158, Issue 7
>>>> ************************************
>>>
>>>
>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>> -- 
>> ________________________________________________________________
>> Bob Briscoehttp://bobbriscoe.net/
>

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/

---

From: "Scharf, Michael (Nokia - DE/Stuttgart)" <[email-masked]>
Date: Mon, 10 Jul 2017 12:47:01 +0000

Hi all,

This e-mail confirms that it is working group consensus to adopt this document.

@authors: Please resubmit the next version as draft-ietf-tcpm-...

Thanks

Michael

From: Scharf, Michael (Nokia - DE/Stuttgart) [mailto:[email-masked]]
Sent: Friday, June 16, 2017 8:07 AM
To: [email-masked]
Cc: [email-masked]
Subject: Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

Hi all,

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

  Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

Thanks

Michael