Subject: [tcpm] Review of draft-ietf-tcpm-initcwnd-04

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 25 Jul 2012 18:11:40 +0200

Hi Jerry,

I'd say the draft now covers all the points discussed in tcpm and as such has 
all the content needed. As you actually only propose a small change to TCP, 
I'd prefer to have a quite short document describing mainly only the changes 
itself and move most of the reasoning to the appendix. Hence I'd like to 
propose quite a little restructuring before moving the document forward as it 
is hard to easily catch the main points at the moment. More concrete, I'd move  
at least section 4, 10 and 11, maybe also 5, 6 and 7 (or at least partly) to 
the appendix. The first two paragraph of section 7 might actually belong in 
the security consideration section. And section 15 (conclusion) is not needed 
at all. More detailed comments below.

Mirja

=============
Section 1 Introduction:
----------------------------
- s/roughly 15KB/maxium 14600B/

- 2. paragraph: Don't like this two sentences at all. I hope the reason to 
increase IW is not just the fact there the queues are large enough to store 10 
packets. You basically saying "Let's send more packets at once to blow the 
queues!". I'd say the reason is that flow sizes have increased and therefore 
IW10 would improve latency a lot. Only because queues are likely to be large 
than 10 packets today, it is possible at all to send more than 10 packet at 
once. Btw. even if the queue would be small, pacing could help to still allow 
an IW of 10. Moreover, the fact that there might be several concurrent flows, 
does not mean that the buffer has to be larger (only if those flow start at 
the same time with IW10). And even if there is a large buffer, if there is a 
long-lived TCP connection, this flow will fill up the queue and there might 
be by chance still no space in the queue for 10 additional packets.

- 3. paragraph: Don't agree. You might not know that there is a low speed link 
somewhere on the path. There are cases where the endpoint can protect its path 
by setting the receiver window but that's not the general case.

- 5. paragraph: "We recommend that all TCP implementations have a settable TCP 
IW parameter..."
  -> I'd like to see this not only in the introduction but also in the main 
section (2.) that is describing the changes and maybe even as a SHOULD

Section 2 TCP Modifications
-------------
- Maybe have a more specific title like "Setting the IW" and some subsections

- equation 1: say somewhere that numbers are in KByte

- 4. paragraph ("Note that...") does not belong in this section

- maybe new subsection for paragraph 5 "Resetting the IW after SYN loss"; 
  what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a number

- last paragraph: maybe s/implementations SHOULD fall back/implementations 
MUST fall back/ ?

Section 3 Implementation Issues 
-------------
- Maybe move the 3 paragraph into section 2 such that the setting of the 
receive window is a MUST or SHOULD of the changes to TCP

- 4. paragraph: Please give some explanation (or remove)

Section 4 Background
------------
- Move most parts in the appendix

- Move 2. paragraph into Introduction

- Maybe improve structure with subsections e.g. paragraph 5 and 6 
as "Recommendation for HTTP Traffic" or something

- 7. paragraph: "pacing will not be necessary" 
  -> I can't remember having seen any results on this. Please explain further 
or remove!

Section 5 Advantages of a Larger Initial Window
-----------
- section 5.1 Reducing Latency
   You should also mention that for larger transmission with several 100s of 
RTT a save of 4 RTT does not really help. Thus transmissions which will 
finish within a small number of RTT will benefit the most.

- section 5.2 
  You make some quite general statements here and then u only refer to google 
data. Maybe you can find some other references as well.

Section 6
----------
- paragraph 3 and 4 belong in an evaluation section/appendix (maybe own 
subsection on "Influence of simultaneous opened connections")

Section 7
--------
- Move 1. paragraph to security consideration

- Move rest in appendix or even parts in  the introduction

Section 9
----------
- move to main section which  is introducing the TCP changes as own subsection 
as a MUST is in here

- can you explain better why this is need or what would happen otherwise?

Section 10
----------
- Move to appendix, have more subsections and better titles for the 
subsections e.g. "Improvements in Latency", "Impact on the Retransmission 
Rate" and "Multiple TCP Connections"

- The numbers for the latency improvement are only on (google) web search. Of 
course there are quite large improvements possible but the Internet contains 
also other traffic. Not sure if your number are really that significant...?

Section 11
---------
- move to appendix

- It's nice that you list all the studies. Could you please quickly summarize 
their results/findings, too? Because I guess that the intersting part to 
know.

Section 12
--------
- Its good to have this section but it's quite unspecific. E.g. you say "An 
increased initial window MUST NOT be turned on by default on systems without 
such monitoring capabilities." But what exactly should be monitored and how 
frequently and so on. Also "any significant deterioration" doesn't say 
anything to me. Is it possible to have a concrete approach/algorithm here 
what to monitor when and what to do in which cases? And than have this even 
as part of the main section with TCP changes as this section has also some 
SHOULD and MUST and might be overread otherwise.

Section 14
---------
- "highly unlikely to lead to a persistent state of network congestion" -> 
Even a "highly unlikely" is concerning me in this case! Also there is no 
reasoning for this. But I guess you can even say, that this change cannot 
cause persistent network congestion as congestion control is still used....

Section 15 Conclusion
--------
- Is not needed here at all, as no new information

Appendix
--------- 
- "One notable exception is YouTube because we don't think
     the large initial window will have much material impact, either
     positive or negative, on bulk data services."
  -> Why is this document than not recommending to NOT use IW10 for bulk data 
services?

- "[CW10] contains some result from a testbed study on how short flows
     with a larger initial window might affect the throughput
     performance of other co-existing, long lived, bulk data transfers."
   -> Please also describe what the results are!

- paragraph on "Why 10 segment?"
  -> This is something I would prefer to have in the body of the document not 
in the appendix

- "Some simulation studies [JNDK10, JNDK10-1] have been conducted to
     study the effect of a larger initial window on wireless links from
     2G to 4G networks (EGDE/HSPA/LTE). The overall result seems mixed
     in both raw performance and the fairness index."
  -> Should it be recommend to not use IW10 in wireless scenarios (if known)?

---

From: Jerry Chu <[email-masked]>
Date: Wed, 25 Jul 2012 10:58:48 -0700

On Wed, Jul 25, 2012 at 9:11 AM, Mirja Kühlewind
<[email-masked]> wrote:
> itself and move most of the reasoning to the appendix.

A document describing only the change can be done, or at least summed up in
one line. But this is not how it is or should be done. We are proposing a small
change to a critical TCP parameter because the impact can be profound. (That's
why we've been wrestling with it for more than 2 years now!) So the
focus is really
on the ramification and justification of the change rather than the
change itself
(which can be summed up in one line). For this reason we've been following the
structure of RFC3390.

Moreover, the TOC provides a very clear index of the contents so a
savvy reader can
quickly pick the sections of interesting. E.g., if you only want to
focus on the change
you'll know to read section 1 and 2 only. That's it.

>Hence I'd like to

Yes it's doable but I think the current flow is fine if you realize
the main purpose of
the document is to justify the change, not the change itself. Also TOC
will make it
much easier to navigate through.

> at all. More detailed comments below.

Ok.

> be by chance still no space in the queue for 10 additional packets.

Guess you read the paragraph differently from what the authors had in mind.
What we have in mind was  "Ten segments are likely to fit into queue space"
because access link drains much faster these days (due to b/w growth)...

Will try to make it more clear.

Will continue later,

Jerry

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 25 Jul 2012 20:12:51 +0200

Hi Jerry,

> you'll know to read section 1 and 2 only. That's it.
And that's not really true (anymore). There are several SHOULD and MUST in the 
other sections as well. Thus you have to read all the doc to correctly 
implement the changes. I know that the intent was to have everything in 
section 1 and 2 that is of interest for implementers but due to the many 
changes that have been done in previous versions, it's not the case anymore. 
That why I said, all the content is there, would be nice to 
clean-up/restructure a little and then it's done.

It's good to have a similar structure than RFC3390 (didn't realize that you 
tried to have exactly the same sections here) but if the content would fit 
better with a different structure, I'd prefer to have a good readable doc 
over having a doc that looks similar to RFC3390.

Mirja

>
> >Hence I'd like to
> > propose quite a little restructuring before moving the document forward
> > as it is hard to easily catch the main points at the moment. More
> > concrete, I'd move at least section 4, 10 and 11, maybe also 5, 6 and 7
> > (or at least partly) to the appendix. The first two paragraph of section
> > 7 might actually belong in
>
> > the security consideration section. And section 15 (conclusion) is not
> > needed at all. More detailed comments below.
> >
> > Mirja
> >
> > =============
> > Section 1 Introduction:
> > - 2. paragraph: Don't like this two sentences at all. I hope the reason
> > to increase IW is not just the fact there the queues are large enough to
> > store 10 packets. You basically saying "Let's send more packets at once
> > to blow the queues!". I'd say the reason is that flow sizes have
> > increased and therefore IW10 would improve latency a lot. Only because
> > queues are likely to be large than 10 packets today, it is possible at
> > all to send more than 10 packet at once. Btw. even if the queue would be
> > small, pacing could help to still allow an IW of 10. Moreover, the fact
> > that there might be several concurrent flows, does not mean that the
> > buffer has to be larger (only if those flow start at the same time with
> > IW10). And even if there is a large buffer, if there is a long-lived TCP
> > connection, this flow will fill up the queue and there might be by chance
> > still no space in the queue for 10 additional packets.
>
> > - 3. paragraph: Don't agree. You might not know that there is a low speed
> > link somewhere on the path. There are cases where the endpoint can
> > protect its path by setting the receiver window but that's not the
> > general case.
> >
> > - 5. paragraph: "We recommend that all TCP implementations have a
> > settable TCP IW parameter..."
> >   -> I'd like to see this not only in the introduction but also in the
> > main section (2.) that is describing the changes and maybe even as a
> > SHOULD
> >
> >
> > - last paragraph: maybe s/implementations SHOULD fall
> > back/implementations MUST fall back/ ?
> >
> >
> >   -> I can't remember having seen any results on this. Please explain
> > further or remove!
> >
> >    You should also mention that for larger transmission with several 100s
> > of RTT a save of 4 RTT does not really help. Thus transmissions which
> > will finish within a small number of RTT will benefit the most.
> >
> > - section 5.2
> >   You make some quite general statements here and then u only refer to
> > google data. Maybe you can find some other references as well.
> >
> > Section 6
> > ----------
> > - move to main section which  is introducing the TCP changes as own
> > subsection as a MUST is in here
> >
> > - The numbers for the latency improvement are only on (google) web
> > search. Of course there are quite large improvements possible but the
> > Internet contains also other traffic. Not sure if your number are really
> > that significant...?
> >
> > Section 11
> > ---------
> > - move to appendix
> >
> > - It's nice that you list all the studies. Could you please quickly
> > summarize their results/findings, too? Because I guess that the
> > intersting part to know.
> >
> > Section 12
> > --------
> > - Its good to have this section but it's quite unspecific. E.g. you say
> > "An increased initial window MUST NOT be turned on by default on systems
> > without such monitoring capabilities." But what exactly should be
> > monitored and how frequently and so on. Also "any significant
> > deterioration" doesn't say anything to me. Is it possible to have a
> > concrete approach/algorithm here what to monitor when and what to do in
> > which cases? And than have this even as part of the main section with TCP
> > changes as this section has also some SHOULD and MUST and might be
> > overread otherwise.
> >
> > Section 14
> > ---------
> > - "highly unlikely to lead to a persistent state of network congestion"
> > -> Even a "highly unlikely" is concerning me in this case! Also there is
> > no reasoning for this. But I guess you can even say, that this change
> > cannot cause persistent network congestion as congestion control is still
> > used....
> >
> > Section 15 Conclusion
> > --------
> >   -> Why is this document than not recommending to NOT use IW10 for bulk
> > data services?
> >
> >   -> This is something I would prefer to have in the body of the document
> > not in the appendix
> >

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Jerry Chu <[email-masked]>
Date: Wed, 25 Jul 2012 14:58:48 -0700

> changes that have been done in previous versions, it's not the case anymore.

Actually the only other specification related languages is section 9,
and it's not
essential as it simply reiterates what is described in rfc6298.

Yes other SHOULD and MUST can be found in the newly added section 12
"Usage and Deployment Recommendations", but that section is mainly about
what one must do in any future large scale experiment, closely monitoring a
number of key metrics... and not really about protocol specification.

So to summarize, this draft is not just about protocol specification,
it also addresses
many other questions like why, and how to go about doing more
experiments safely.

> tried to have exactly the same sections here) but if the content would fit

It was spelled out up front in 1.  Introduction

   "This document proposes to raise the upper bound on TCP's initial
   window (IW) to 10 segments or roughly 15KB. It is patterned after and
   borrows heavily from RFC 3390 [RFC3390] and earlier work in this
   area."

> over having a doc that looks similar to RFC3390.

Let's stick to the current format but I will respond to your one other
criticism on
section 12 first. The section is quite specific about the metrics to monitor: "A
key metric to monitor is the rate of packet losses, ECN marking, or
segment retransmissions during the initial burst." But as others have
pointed out it can be
quite difficult to try to get more specific than that, i.e., how much
performance
deteriorates before one must back off. As such all we can do for now is to have
enough warning signs for now but leave some details for future investigation.

Jerry

---

From: Jerry Chu <[email-masked]>
Date: Tue, 31 Jul 2012 22:32:31 -0700

>>> > protect its path by setting the receiver window but that's not the

This is exactly the point - it's not the general case but may be the
vast majority.
I.e., 95% of severely b/w limited hosts may be just 1 hop away or even directly
attached to a slow link (e.g., dailup modem) hence can be mitigated much easily.
No one suggested a solution here to solve the general problem of
detecting bottleneck
b/w. That's the job TCP's CC algorithm.

>>> > Section 2 TCP Modifications

Plagiarized from RFC3390 :)

>>> > - equation 1: say somewhere that numbers are in KByte

Not really.

>>> > - 4. paragraph ("Note that...") does not belong in this section

Why not? (It's more or less a disclaim because no one has done any study
on non-Ethernet MTU.

>>> >   what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a

Ok. Changed to "more than one".

>>> > Section 3 Implementation Issues

Non-essential. Remember even IW10 is just an upperbound.

>>> > - 4. paragraph: Please give some explanation (or remove)

This was brought up and discussed on the list long ago.

>>> > Section 4 Background

See my other response. I prefer not to change the structure at this time.

>>> >   -> I can't remember having seen any results on this. Please explain

The statement was really "We suspect" pacing will not be necessary because
our test results do not show any severe congestion from IW10...

>>> > Section 5 Advantages of a Larger Initial Window

Isn't it obvious already?

>>> > google data. Maybe you can find some other references as well.

It's not just google data (see all the refs.)

>>> > Section 6

Non-essential because it simply restates what is said in rfc6298. It was added
to address some concern (or confusion) that was brought up in the past that
the initial burst of 10 pkts won't bid well for RTO hence causing
spurious rexmits.

>>> > - can you explain better why this is need or what would happen otherwise?

We just stated what we have.

>>> > summarize their results/findings, too? Because I guess that the

If you are interested please go read them yourself.

>>> > cannot cause persistent network congestion as congestion control is still

See 1st paragraph of section 7.

>>> > - Is not needed here at all, as no new information

Most rfcs have a conclusion section for those who have time only for
the 1st and last sections.

>>> >   -> Why is this document than not recommending to NOT use IW10 for bulk

Why should it recommend NOT? How would the TCP stack know apriori bulk
or not? So it will require input from the apps... Is it really worth
the complexity?

>>> >    -> Please also describe what the results are!

Read the slides please.

>>> >   -> Should it be recommend to not use IW10 in wireless scenarios (if

Studies are still on-going. I don't remember there has been a conclusive result.

Jerry