Subject: [tcpm] TCP and applications [was Re: Aid passive estimate of tcp performance]

From: Yuchung Cheng <[email-masked]>
Date: Wed, 8 Feb 2012 14:59:27 -0800

On Wed, Feb 8, 2012 at 11:20 AM, Jim Gettys <[email-masked]> wrote:
> Over the years, I have noted a distinct lack of communications between
> those who understand how the bits flow, and how they are used....  For
> example, the transient bufferbloat phenomena is something that is pretty
> obvious to those who understand TCP, and completely unobvious to most
> who do web browsers...
>
> I believe both groups need to talk to each other, strangely.  Sometimes
> you may find that the guys who do the applications actually have needs
> not being met by standards, that are well worth understanding; and
> conversely, the guys who do the apps may find that life is better for
> both them and others if they understand the standards.
+1

Can't agree more. We need a venue for two-way communications.

app dev: TCP sucks because blah blah blah
TCP dudes: app sucks because blah blah blah
[I am in a company where I have to hear both  :(  ]

Nothing gets done until both parties try to help each other by
listening to each other.

>
> Just go in with an open mind and say: "We observe the browsers/servers
> doing the following thing that we believe is much less than optimal, why
> are you doing this?" rather than "You are disobeying the rules, fix it",
> and I think you'll get a warm welcome.
>                - Jim
>
>
>>
>> Joe
>>
>> On 2/8/2012 11:03 AM, Jim Gettys wrote:
>>> On 02/08/2012 01:56 PM, Joe Touch wrote:
>>>>
>>>>
>>>> On 2/8/2012 10:05 AM, Smith, Donald wrote:
>>>>> Just agreeing with Joe, Many applications uses resets instead of a
>>>>> FIN handshake to terminate connections to speed things up especially
>>>>> browsers.
>>>>>
>>>>>
>>>>> http://pages.cpsc.ucalgary.ca/~carey/papers/2005/TCP-Resets.pdf
>>>>
>>>> It doesn't just "speed" things; it's reckless and dangerous, esp.
>>>> since the parties tossing around these RSTs typically don't maintain
>>>> TIME-WAIT before reusing those connection IDs.
>>>
>>> Joe, browsers have done this forever.
>>>
>>> If you don't like it, then it's (long past time) to make the case that
>>> they should change their behaviour.
>>>                          - Jim
>>>
>>>>
>>>> Joe
>>>>
>>>>>> -----Original Message-----
>>>>>> From: [email-masked] [mailto:[email-masked]] On
>>>>>> Behalf Of
>>>>>> Joe Touch
>>>>>> Sent: Wednesday, February 08, 2012 10:46 AM
>>>>>> To: Ángel Bartomeu Bonillo
>>>>>> Cc: [email-masked]
>>>>>> Subject: Re: [tcpm] Aid passive estimate of tcp performance
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2/8/2012 4:15 AM, Ángel Bartomeu Bonillo wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I recently faced the problem of passively estimating the packet loss
>>>>>>> of TCP connection. After a bit of investigating the literature I
>>>>>> found
>>>>>>> that there are tens of papers addressing this issue but I couldn't
>>>>>>> find any perfect solution. Since this is an important issue for
>>>>>>> network operators I have been thinking that there would be a very
>>>>>> easy
>>>>>>> solution by just making the two TCP connection end points to account
>>>>>>> for its sent segments and report it to the other end point in the
>>>>>>> FIN
>>>>>>> segment. This approach could be used as well for other measurable
>>>>>>> parameters along the TCP session without the need for any other
>>>>>>> offline method and It would be very helpful for detecting failures
>>>>>>> along networks.
>>>>>>>
>>>>>>> Any comments?
>>>>>>
>>>>>> Regardless of when you do this - either at the end of a
>>>>>> connection, or
>>>>>> during a connection - the ONLY information you can provide is
>>>>>> "here is
>>>>>> a list of segments I have not yet received".
>>>>>>
>>>>>> However, that list is irrelevant. Those segments might be in the
>>>>>> network and may still arrive - after you report, after the connection
>>>>>> is over, etc.
>>>>>>
>>>>>> There is no way to KNOW that a segment is lost vs. late.
>>>>>>
>>>>>> Even when you retransmit a segment (e.g., because it is basically
>>>>>> overdue, either because your side's timer expired or the other side
>>>>>> sent either a SACK or ACK indicating a gap), you don't know that the
>>>>>> first segment was lost - it could arrive after the timer expires.
>>>>>>
>>>>>> Even though segments have TTLs, and TTLs are *supposed* to be
>>>>>> decremented once every second or once every hop (whichever is
>>>>>> greater),
>>>>>> there are devices that queue packets for long periods (e.g., when a
>>>>>> link is interrupted) and send them on later (even though they should
>>>>>> have been dropped).
>>>>>>
>>>>>> So there is no such information as "lost segments"; the best you know
>>>>>> is retransmissions.
>>>>>>
>>>>>> However, keep in mind that there are lots of TCP connections that are
>>>>>> terminated without a FIN (e.g., using a RST - I don't think it's
>>>>>> appropriate in many cases, but the method is in widespread use).
>>>>>>
>>>>>> Joe
>>>>>> _______________________________________________
>>>>>> tcpm mailing list
>>>>>> [email-masked]
>>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>>
>>>>> This communication is the property of CenturyLink and may contain
>>>>> confidential or privileged information. Unauthorized use of this
>>>>> communication is strictly
>>>>> prohibited and may be unlawful.  If you have received this
>>>>> communication
>>>>> in error, please immediately notify the sender by reply e-mail and
>>>>> destroy
>>>>> all copies of the communication and any attachments.
>>>> _______________________________________________
>>>> tcpm mailing list
>>>> [email-masked]
>>>> https://www.ietf.org/mailman/listinfo/tcpm
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Mark Allman <[email-masked]>
Date: Wed, 08 Feb 2012 22:31:56 -0500

> TCP dudes: app sucks because blah blah blah

Wellllll..... that is one reason everyone rearranges all the furniture
in a big damn hotel three times a year.  But, alas, you can lead a horse
to water ...

allman

---

From: Ángel Bartomeu Bonillo <[email-masked]>
Date: Thu, 9 Feb 2012 09:15:45 +0100

Well, the case that I had in mind was calculating the packet loss passively from the end user. The estimation for packet loss would be the number of retransmissions minus the number of spurious retransmission. In the Uplink traffic you know the real number of retransmission and you can estimate the spurious ones with the info in the DSAck. However in the downlink it is impossible since you know the spurious but not the real number of packets sent out by the other peer any approach would fail in cases like having the remote peer in slow start and being all the packets dropped. You never know if you do not get packets because either there are no packets to send or because they were lost.

There is obviously no point in changing the TCP just to aid this case but I thought that maybe some of you would find a more generic usage for this idea that I considered quite simple and I thought I could be useful. I see now because of your comments that I was wrong since I didn't see all this consideration you just showed. 

Thanks anyways for your interesting comments.

---

From: Wesley Eddy <[email-masked]>
Date: Thu, 09 Feb 2012 09:44:17 -0500

> Well, the case that I had in mind was calculating the packet loss passively from the end user. The estimation for packet loss would be the number of retransmissions minus the number of spurious ret…

Even from the sender-side where you know about all of the
transmissions, you can still only compute an estimate due
to sources of error like lost ACKs carrying DSACK information
(as one easy example).

There is (in my opinion) a good description of some algorithms
for different types of connections (DSACK-supported, SACK but
no DSACK, and no SACK) and the associated sources of error in:

http://www.icir.org/mallman/papers/least-per-dec2003.ps
Mark Allman, Wesley Eddy, Shawn Ostermann.
Estimating Loss Rates With TCP.
ACM Performance Evaluation Review, 31(3), December 2003.

-- 
Wes Eddy
MTI Systems

---

From: Ángel Bartomeu Bonillo <[email-masked]>
Date: Thu, 9 Feb 2012 15:59:32 +0100

> ACM Performance Evaluation Review, 31(3), December 2003.

Yeah Wesley I have already read that and a few others. That's why I am interested in the downlink. AFAIK the downlink is not covered in any other paper.
Thank you.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 29 Feb 2012 18:01:51 -0500

> in a big damn hotel three times a year.  But, alas, you can lead a horse

[Apologies for appending to an old thread. I read tcpm in batches every
few weeks.]

Hi all.

I develop network gunk for firefox - if you think there is something we
could be doing to make the web more capable, more open, and more awesome
please reach out to me.

A few of you have worked on some topic with me in the past, and I like
to think we're both better for it. I'm a reasonable guy who loves to
talk about these interactions and what is realistic to do about them. My
counterparts at other browsers, at least the ones I know, are good
people too - all trying to make a better Internet while balancing many
needs and constraints.

So that's something concrete for you - a person to engage. I'll be in
Paris too.

BTW - I'm not trying to just throw this over the wall and say "call me
if you need me", I think tcpm is part of my community and try and keep
up and contribute. My community is just a big world (from HTML5 to
ethernet), so stuff slips through and the needs may not necessarily
align.

Congenially,
Patrick
aka [email-masked]