From nobody Thu May 1 01:12:28 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A342E1A88DD for ; Thu, 1 May 2014 01:12:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.887 X-Spam-Level: X-Spam-Status: No, score=-6.887 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4y1M4bA_Yx28 for ; Thu, 1 May 2014 01:12:17 -0700 (PDT) Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) by ietfa.amsl.com (Postfix) with ESMTP id 3BCC51A6EFE for ; Thu, 1 May 2014 01:12:17 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.97,963,1389772800"; d="asc'?scan'208";a="161483056" Received: from vmwexceht05-prd.hq.netapp.com ([10.106.77.35]) by mx12-out.netapp.com with ESMTP; 01 May 2014 01:12:02 -0700 Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.246]) by vmwexceht05-prd.hq.netapp.com ([10.106.77.35]) with mapi id 14.03.0123.003; Thu, 1 May 2014 01:12:02 -0700 From: "Eggert, Lars" To: "Fred Baker (fred)" Thread-Topic: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 RMCAT Flow competing with a long TCP Flow Thread-Index: Ac9jf090kpDicEcGShCy4a8kghdQ6gAK9R2AAAia8FAAJv0GcAAVl0KAACP0TQA= Date: Thu, 1 May 2014 08:12:00 +0000 Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5BD0F@ESESSMB205.ericsson.se> <81564C0D7D4D2A4B9A86C8C7404A13DA31F5BDA6@ESESSMB205.ericsson.se> <7BFDDE81-4472-4F13-94D6-B62B6B802C33@cisco.com> In-Reply-To: <7BFDDE81-4472-4F13-94D6-B62B6B802C33@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.29] Content-Type: multipart/signed; boundary="Apple-Mail=_C97B5BC3-8745-4446-9D9B-4536941FA51C"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/wD-hsQUYPHo_id5GoyqijTWSkEg Cc: Zaheduzzaman Sarker , Ingemar Johansson S , "rmcat WG \(rmcat@ietf.org\)" , "varun@comnet.tkk.fi" , "Michael Ramalho \(mramalho\)" , Erlendur Karlsson , "Xiaoqing Zhu \(xiaoqzhu\)" Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 RMCAT Flow competing with a long TCP Flow X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 08:12:22 -0000 --Apple-Mail=_C97B5BC3-8745-4446-9D9B-4536941FA51C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, one additional source of reordering we've seen is caused by the receiver = stack itself. At least Linux currently seems to introduce it when the = system gets fully loaded (e.g., with 40G NICs) and the scheduler moves = queue processing between cores. (This is anecdotal at the moment, we're = trying to quantify/qualify it better.) I'm sure this is a transient problem as far as the kernel source is = concerned, but boxes based on the current kernels may live in = deployments for quite some time before being updated. Lars On 2014-4-30, at 19:02, Fred Baker (fred) wrote: >=20 > On Apr 30, 2014, at 4:50 AM, Zaheduzzaman Sarker = wrote: >=20 >> Are you saying re-ordering is not a phenomena that we can observe in = the current Internet? And we should not have test cases to handle = re-ordering of packets in RMCAT? >=20 > Packet reordering, loss, and duplication happen. Where and to what = degree is a question of the structure of networks that packets pass = through and what happens in those networks. The last time I know of = someone doing an actual measurement and reporting the results was rather = a while ago, but it said that at that time about 5% of arriving traffic = was reordered. For systemic reasons, I would imagine this is reduced, = and very dependent on what networks and types of networks are crossed. >=20 > Going out on a limb a bit (this is what I suspect, not what I know), I = would imagine that today there are three major sources of traffic = reordering. One is loss; in radio networks (which include 3g/etc but = also include 802.11, 802.15, and 802.16), loss and retransmission are = pretty common. It would be interesting to have someone write a paper = measuring and analyzing the resultant =93best effort=94 behaviors in = those networks. Another is load balancing in the network; while it is = common for ECMP routing to send a subset of flows one way and another = subset on another, some networks schedule packets on the most available = bandwidth or go round robin. That is a recipe for reordering. The other = has to do with adaptive bitrate video and services that deliver it by = literally duplicating data. In these services, there is a path from the = CDN to the consumer that consists of two disjoint paths. The TCP = originator is configured with a link layer multicast address for its = first hop router, and there are in fact two such routers, one on each of = the parallel paths. The packets don=92t arrive at the endpoint, but they = do arrive at a point just before it, and the first of each pair of = packets to arrive is forwarded to the endpoint. You can imagine that the = collection point sees frequent duplication and some degree of = reordering. >=20 > The single biggest source of reordering, I will postulate, is none of = those; it is the result of loss due to TCP=92s AIMD behavior. When a = packet is dropped in flight due to queue depth or delay, the rest of the = packets in that RTT arrive before it, and then its retransmission = arrives. Voila! Packet reordering. >=20 > Should TCP handle packet reordering? Yes, of course. Making packet = arrival sequence completely random will certainly test that behavior; we = could discuss the beneficial effects of running TCP over CSNET in the = 1980=92s and the resulting hardening of TCP=92s algorithms. That doesn=92t= make that level of reordering a valid simulation of the Internet. A = more valid simulation will have one or both of two characteristics - = that a packet is dropped, the remaining packets in that RTT pass it, and = then it arrives, or that packets frequently arrive disordered but to a = degree that doesn=92t trigger TCP=92s Fast retransmit algorithm. --Apple-Mail=_C97B5BC3-8745-4446-9D9B-4536941FA51C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBU2IBztZcnpRveo1xAQJAAQP/bnr+l2gEJzXbEqUHHUCY39PcMz9GOZbj QOA0Dl2iLfF53DTz5tu3kdOYJWLW7r9hPrL4EBebQzX3s+IWx7588Lnj6Qj/AVzv yulzubRdYemurDMG/o5EBYlV2g4F3S3XAiw40Ve2LCkHCHph5skbBSa4YiD/h7+s u/ahl8E6hvw= =z13w -----END PGP SIGNATURE----- --Apple-Mail=_C97B5BC3-8745-4446-9D9B-4536941FA51C-- From nobody Fri May 2 02:50:28 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 076B11A09D4 for ; Fri, 2 May 2014 02:50:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WdKxyuN2sxCP for ; Fri, 2 May 2014 02:50:23 -0700 (PDT) Received: from r2-chicago.webserversystems.com (r2-chicago.webserversystems.com [173.236.101.58]) by ietfa.amsl.com (Postfix) with ESMTP id 339881A09A6 for ; Fri, 2 May 2014 02:50:23 -0700 (PDT) Received: from pool-71-175-4-197.phlapa.fios.verizon.net ([71.175.4.197]:3985 helo=[192.168.1.12]) by r2-chicago.webserversystems.com with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WgA6m-000ByR-CQ for rmcat@ietf.org; Fri, 02 May 2014 04:50:20 -0500 Message-ID: <53636A1C.4090501@jesup.org> Date: Fri, 02 May 2014 05:49:16 -0400 From: Randell Jesup User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: rmcat@ietf.org References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5BD0F@ESESSMB205.ericsson.se> <81564C0D7D4D2A4B9A86C8C7404A13DA31F5BDA6@ESESSMB205.ericsson.se> <7AC92FFB-76BE-494E-86E4-368BB0B6223C@csperkins.org> In-Reply-To: <7AC92FFB-76BE-494E-86E4-368BB0B6223C@csperkins.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - r2-chicago.webserversystems.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - jesup.org X-Get-Message-Sender-Via: r2-chicago.webserversystems.com: authenticated_id: randell@jesup.org Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/tc9cpWoWVMtx7m99Sa_hVjsGxTw Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 RMCAT Flow competing with a long TCP Flow X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 09:50:27 -0000 On 4/30/2014 1:51 PM, Colin Perkins wrote: > On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker wrote: >> Are you saying re-ordering is not a phenomena that we can observe in the current Internet? And we should not have test cases to handle re-ordering of packets in RMCAT? > One of my students collected some UDP streaming measurements from our university to hosts on residential networks, covering all the main UK ISPs, a couple of years ago. With everything using wired connections and UDP we saw a couple of hundred packets delivered out of order, from ~250 million packets sent. I can dig into the details of the re-ordering patterns if there’s interest. We didn’t analyse them too closely at the time because packet loss and delay spikes were much more common. Having it not fall over on (some) reordering: good. My experience has been only specific unusual network/hardware situations cause more than 0.0x% reordering of packet delivery (at the IP level, ignoring things like TCP retransmit, which is not an IP-level reordering in the way I'm discussing). I've seen significant reordering only on a network (an old office network of mine!) where we had multiple bonded T1's and the packets went across one of them randomly (I presume). I think I got to low-single-digit percent or more likely 0.n% (like 0.1-0.9%) reordered. There may also be some wireless networks where this is more likely than on normal wired; I haven't noticed it on WiFi though. It's easy to accidentally set up DummyNet/etc to cause massive packet reordering, however. I do not think that should be a primary or even secondary testcase, and (to be honest) I don't think we should care much about degenerate reordering cases. I'm not even sure we should bother spending much energy in our evaluation tests making it happen at all (which isn't to say that the algorithm authors shouldn't consider it and if they feel it would affect them, test it or discuss it - plus they will see a small amount of it on real networks). -- Randell Jesup -- rjesup a t mozilla d o t com From nobody Fri May 2 04:11:51 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EA6D71A6F07 for ; Fri, 2 May 2014 04:11:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.55 X-Spam-Level: X-Spam-Status: No, score=-2.55 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5H2kdBjxawMH for ; Fri, 2 May 2014 04:11:47 -0700 (PDT) Received: from mail-out1.uio.no (mail-out1.uio.no [IPv6:2001:700:100:10::57]) by ietfa.amsl.com (Postfix) with ESMTP id 4AD0B1A084E for ; Fri, 2 May 2014 04:11:46 -0700 (PDT) Received: from mail-mx6.uio.no ([129.240.10.40]) by mail-out1.uio.no with esmtp (Exim 4.75) (envelope-from ) id 1WgBNX-00069F-5C for rmcat@ietf.org; Fri, 02 May 2014 13:11:43 +0200 Received: from boomerang.ifi.uio.no ([129.240.68.135]) by mail-mx6.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80.1) (envelope-from ) id 1WgBNW-0005KM-ML for rmcat@ietf.org; Fri, 02 May 2014 13:11:43 +0200 From: Michael Welzl Content-Type: multipart/alternative; boundary="Apple-Mail=_27B81DC7-8649-4798-8B3B-BD29B69873F2" Date: Fri, 2 May 2014 13:11:42 +0200 Message-Id: <98B4752B-9609-4372-8145-5025992345DA@ifi.uio.no> To: rmcat WG Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 1 msgs/h 1 sum rcpts/h 9 sum msgs/h 5 total rcpts 15904 max rcpts/h 44 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-6.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.051, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: D54D82A1A0816FC26776F5327DCE17C5B6F1CC7A X-UiO-SPAM-Test: remote_host: 129.240.68.135 spam_score: -59 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 5129 max/h 16 blacklist 0 greylist 0 ratelimit 0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/ApTpeeqYmaxVwANEcKo_t7hz1Ko Subject: [rmcat] Update on coupled congestion control X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 11:11:50 -0000 --Apple-Mail=_27B81DC7-8649-4798-8B3B-BD29B69873F2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Dear all, We'd like to share an update about our work on coupled congestion = control: we have developed and evaluated an algorithm that we think = works pretty well, and documented our results in this technical report: http://safiquli.at.ifi.uio.no/paper/fse-tech-report.pdf A shorter version of this document was accepted for publication in the = ACM SIGCOMM'14 Capacity Sharing Workshop. The results were done with simulations, using TFRC (as a representative = for a multimedia-oriented congestion control scheme) and RAP (as a = rate-based AIMD scheme); we're currently working with LEDBAT (as a = representative for a delay-based scheme), and once we've covered that, = we'll approach real-life testing! The updated algorithm (algorithm 2 in the document, on page 7) isn't = incorporated in draft-welzl-rmcat-coupled-cc yet, we're working on an = update of this draft and plan to submit it soon, stay tuned! Any comments are very welcome! Cheers, Michael --Apple-Mail=_27B81DC7-8649-4798-8B3B-BD29B69873F2 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Dear = all,

We'd like to share an update about our work on = coupled congestion control: we have developed and evaluated an algorithm = that we think works pretty well, and documented our results in this = technical report:

A shorter version of this document was accepted for publication in the = ACM SIGCOMM'14 Capacity Sharing Workshop.

The = results were done with simulations, using TFRC (as a representative for = a multimedia-oriented congestion control scheme) and RAP (as a = rate-based AIMD scheme); we're currently working with LEDBAT (as a = representative for a delay-based scheme), and once we've covered that, = we'll approach real-life testing!

The updated = algorithm (algorithm 2 in the document, on page 7) isn't incorporated in = draft-welzl-rmcat-coupled-cc yet, we're working on an update of this = draft and plan to submit it soon, stay = tuned!

Any comments are very = welcome!

Cheers,
Michael

=
= --Apple-Mail=_27B81DC7-8649-4798-8B3B-BD29B69873F2-- From nobody Fri May 2 07:08:33 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ACECA1A6FE2 for ; Fri, 2 May 2014 07:08:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.929 X-Spam-Level: X-Spam-Status: No, score=-1.929 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wv8jEVw4iroc for ; Fri, 2 May 2014 07:08:28 -0700 (PDT) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 0363E1A6FDA for ; Fri, 2 May 2014 07:08:27 -0700 (PDT) Received: by mail-qg0-f47.google.com with SMTP id e89so4667619qgf.6 for ; Fri, 02 May 2014 07:08:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/6ZQa76tWjIdDjdER8am/ALrH9qZCjjWnq51oNe7lR0=; b=dx3gFcmejlR+FGL3yYk+GS8BItt3HPO9MTE4Ir2j4A5ysph6O/L46RxliARhuZVK+n VTWqgBT1OeL5c4uuL5qPxKamidlct57LGk4iqvgSZeEMNdkhTJ/s2uFelkPbI5Toyw97 0inDxCd+sBrDn3eZk4rGDaAjg/MnSqlX0OOBmKl72jRULeK3Vn/XciBem5R4M28s9SKS vBSZZrC9+cPA790mvu3ASYoNXDRMZoG5VdWUKoj/j2K6b/VPNpHghlnXjRTfh7z9Uomc 4F8bzhR7rg1SUDNbOEtfw0DNZ3bwlxjizHxoMBsIh6gWd7rSWjTM9yS/sRjVvvDEnvOK q1Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=/6ZQa76tWjIdDjdER8am/ALrH9qZCjjWnq51oNe7lR0=; b=VgNyNiXQCaQ9LgxFVsszAeT9TtFhhGMDCYakQiRAuV0qUqxG2fVgAgCDoP5O2AhE0q RRmDJnoKdjizj8su+Tgfx3m7NGjUfGw0o8WvvO4Zu3RQr+Dbp4cv1KJ1L6t+h18+4v8e RMzV39lIlFBdgiVavTzS2dyQu9zMwVo7+vu3vKI60nZ4pRrW5YK19yCCIgGway832lk+ 7PkB1WJIiJ2676K5NrXbIvFi3zcUx6YRZrpWD7ThMdygk9Q7PSrlgV7hRUkYmycNI8uf QlbIBZnIiqwJUGCdBqmjzVOEb3yd1i2J3lW67cgMMt/lYTfzMg3yt1OMkjNAiQ71/2s3 JeeQ== X-Gm-Message-State: ALoCoQneDRC+8fham3yCq8gFBN0obQ1iKOQTJIzZbIJ7TEz176s7yHXbV3/RqbP1grMaYpMvQy+1 MIME-Version: 1.0 X-Received: by 10.229.198.2 with SMTP id em2mr20347727qcb.21.1399039705319; Fri, 02 May 2014 07:08:25 -0700 (PDT) Sender: holmer@google.com Received: by 10.229.138.69 with HTTP; Fri, 2 May 2014 07:08:25 -0700 (PDT) In-Reply-To: <53636A1C.4090501@jesup.org> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5BD0F@ESESSMB205.ericsson.se> <81564C0D7D4D2A4B9A86C8C7404A13DA31F5BDA6@ESESSMB205.ericsson.se> <7AC92FFB-76BE-494E-86E4-368BB0B6223C@csperkins.org> <53636A1C.4090501@jesup.org> Date: Fri, 2 May 2014 16:08:25 +0200 X-Google-Sender-Auth: TP8xWA1LJsR9pIxckhiIVUTvUpk Message-ID: From: Stefan Holmer To: Randell Jesup Content-Type: multipart/alternative; boundary=001a11c2ee64a9a6d404f86b5047 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/9B9FOpkZXjuH8QMWh73MiZzbowg Cc: rmcat WG Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 RMCAT Flow competing with a long TCP Flow X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 14:08:29 -0000 --001a11c2ee64a9a6d404f86b5047 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, May 2, 2014 at 11:49 AM, Randell Jesup wrot= e: > On 4/30/2014 1:51 PM, Colin Perkins wrote: > >> On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker < >> zaheduzzaman.sarker@ericsson.com> wrote: >> >>> Are you saying re-ordering is not a phenomena that we can observe in th= e >>> current Internet? And we should not have test cases to handle re-orderi= ng >>> of packets in RMCAT? >>> >> One of my students collected some UDP streaming measurements from our >> university to hosts on residential networks, covering all the main UK IS= Ps, >> a couple of years ago. With everything using wired connections and UDP w= e >> saw a couple of hundred packets delivered out of order, from ~250 millio= n >> packets sent. I can dig into the details of the re-ordering patterns if >> there=E2=80=99s interest. We didn=E2=80=99t analyse them too closely at = the time because >> packet loss and delay spikes were much more common. >> > > Having it not fall over on (some) reordering: good. My experience has > been only specific unusual network/hardware situations cause more than > 0.0x% reordering of packet delivery (at the IP level, ignoring things lik= e > TCP retransmit, which is not an IP-level reordering in the way I'm > discussing). > > I've seen significant reordering only on a network (an old office network > of mine!) where we had multiple bonded T1's and the packets went across o= ne > of them randomly (I presume). I think I got to low-single-digit percent = or > more likely 0.n% (like 0.1-0.9%) reordered. There may also be some > wireless networks where this is more likely than on normal wired; I haven= 't > noticed it on WiFi though. > It's easy to accidentally set up DummyNet/etc to cause massive packet > reordering, however. I do not think that should be a primary or even > secondary testcase, and (to be honest) I don't think we should care much > about degenerate reordering cases. I'm not even sure we should bother > spending much energy in our evaluation tests making it happen at all (whi= ch > isn't to say that the algorithm authors shouldn't consider it and if they > feel it would affect them, test it or discuss it - plus they will see a > small amount of it on real networks). +1 > > > > -- > Randell Jesup -- rjesup a t mozilla d o t com > > --001a11c2ee64a9a6d404f86b5047 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Fri, May 2, 2014 at 11:49 AM, Randell Jesup <= randell-ietf@jesup.org> wrote:
On 4/30/2014 1:51 PM, Co= lin Perkins wrote:
On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker <zaheduzzaman.= sarker@ericsson.com> wrote:
Are you saying re-ordering is not a phenomena that we can observe in the cu= rrent Internet? And we should not have test cases to handle re-ordering of = packets in RMCAT?
One of my students collected some UDP streaming measurements from our unive= rsity to hosts on residential networks, covering all the main UK ISPs, a co= uple of years ago. With everything using wired connections and UDP we saw a= couple of hundred packets delivered out of order, from ~250 million packet= s sent. I can dig into the details of the re-ordering patterns if there=E2= =80=99s interest. We didn=E2=80=99t analyse them too closely at the time be= cause packet loss and delay spikes were much more common.

Having it not fall over on (some) reordering: good. =C2=A0My experience has= been only specific unusual network/hardware situations cause more than 0.0= x% reordering of packet delivery (at the IP level, ignoring things like TCP= retransmit, which is not an IP-level reordering in the way I'm discuss= ing).

I've seen significant reordering only on a network (an old office netwo= rk of mine!) where we had multiple bonded T1's and the packets went acr= oss one of them randomly (I presume). =C2=A0I think I got to low-single-dig= it percent or more likely 0.n% (like 0.1-0.9%) reordered. =C2=A0There may a= lso be some wireless networks where this is more likely than on normal wire= d; I haven't noticed it on WiFi though.

It's easy to accidentally set up DummyNet/etc to cause massive packet r= eordering, however. =C2=A0I do not think that should be a primary or even s= econdary testcase, and (to be honest) I don't think we should care much= about degenerate reordering cases. =C2=A0I'm not even sure we should b= other spending much energy in our evaluation tests making it happen at all = (which isn't to say that the algorithm authors shouldn't consider i= t and if they feel it would affect them, test it or discuss it - plus they = will see a small amount of it on real networks).

+1
=C2=A0



--
Randell Jesup -- rjesup a t mozilla d o t com


--001a11c2ee64a9a6d404f86b5047-- From nobody Fri May 2 13:10:13 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E702D1A6FBE for ; Fri, 2 May 2014 13:10:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.278 X-Spam-Level: X-Spam-Status: No, score=-1.278 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xgJTFt3NWa8N for ; Fri, 2 May 2014 13:10:10 -0700 (PDT) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) by ietfa.amsl.com (Postfix) with ESMTP id A71D31A6FF1 for ; Fri, 2 May 2014 13:10:10 -0700 (PDT) Received: by mail-qc0-f171.google.com with SMTP id c9so5283376qcz.2 for ; Fri, 02 May 2014 13:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:content-type:content-transfer-encoding; bh=1G4DVgug5gtDqP15iOhLjzkwrY0pRKB8Z44FGfBg6Bw=; b=N/xIJwfGMNpVGyVcQBmTHVjAloS+RxWEjaOTnwRTTXWqSPICqDMsoRVeLaRmQ87HYO /WfS868W+c2ipK3h/UM3rTWlnIT4beOAsPsjEwhCRQY13Okw+wFmjXivxVcMvXv1UOnf hLw+5AB0ifsLHFX+XGb514IRWKxA5vLYtOBgjazYiwB45y0TTMdR/7XqaNTRQTtCWfXy AgC+zKkfFPBv/SJubV5iwZScJong3/oTHMGVNvErbVVm0OB+4krl4QYZJ9+/6wcw6MoO 4iKKT7Ue+LV7L7xfDmfdOVe/krAcKURoZhUrBOZtldb+pCc+NAdZ7Rsdo0DkSYlWcHnm 3z4Q== X-Received: by 10.224.138.3 with SMTP id y3mr25394915qat.78.1399061408087; Fri, 02 May 2014 13:10:08 -0700 (PDT) MIME-Version: 1.0 Sender: winstein@gmail.com Received: by 10.229.28.69 with HTTP; Fri, 2 May 2014 13:09:28 -0700 (PDT) In-Reply-To: References: From: Keith Winstein Date: Fri, 2 May 2014 16:09:28 -0400 X-Google-Sender-Auth: jToPU6bHdOh2nCmkG4HDxtFse90 Message-ID: To: "rmcat@ietf.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/4Sdxc82sCBNB6o2OpxAzsK4IYsw Subject: Re: [rmcat] rmcat Digest, Vol 21, Issue 2 X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 20:10:12 -0000 Our experience with packet reordering has actually been quite different. We (Mosh) have found that packet reordering can be extremely common on 802.11n networks when the "block acks" feature is used and a link is saturated, at least with some receiving 802.11n chipsets, including the ones in MacBooks. In Mosh version 1.2.2, we turned on an advisory warning when packets were reordered in the network, because we thought that in practice reordering was rare enough to be notable and might signify an attempted attack. After the release, we got a lot of complaints that the warning was showing up frequently for users on enterprise 802.11n networks using the MacBook and a few other devices. On a full-throttle TCP download to a MacBook Pro, we found that something like 30-50% of all TCP segments were being delivered out of order! And Mosh's UDP datagrams were seeing similar treatment when there was a back-to-back flight of them. We ended up disabling the warning in v1.2.3. As I understand, the block acks feature means that a sender may transmit a whole chunk of frames (some of which will be received without error and some won't), and then the receiving station can tell the sender which frames were successfully received and which need to be retransmitted. In the meantime, the receiver can deliver the successfully-received datagrams to IP or wait. RFC 3366 3.1 essentially advises link designers that they should use a receive-side reordering buffer to try to put datagrams back in order before delivery to IP, but our testing suggests that common 802.11n receive chipsets don't do this in practice. So our experience has been that in the contemporary Internet (at least on some 802.11n networks), reordering can be much more common than we had initially believed. On Fri, May 2, 2014 at 3:00 PM, wrote: > ---------- Forwarded message ---------- > From: Randell Jesup > To: rmcat@ietf.org > Cc: > Date: Fri, 02 May 2014 05:49:16 -0400 > Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 RMCAT= Flow competing with a long TCP Flow > On 4/30/2014 1:51 PM, Colin Perkins wrote: >> >> On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker wrote: >>> >>> Are you saying re-ordering is not a phenomena that we can observe in th= e current Internet? And we should not have test cases to handle re-ordering= of packets in RMCAT? >> >> One of my students collected some UDP streaming measurements from our un= iversity to hosts on residential networks, covering all the main UK ISPs, a= couple of years ago. With everything using wired connections and UDP we sa= w a couple of hundred packets delivered out of order, from ~250 million pac= kets sent. I can dig into the details of the re-ordering patterns if there= =E2=80=99s interest. We didn=E2=80=99t analyse them too closely at the time= because packet loss and delay spikes were much more common. > > > Having it not fall over on (some) reordering: good. My experience has be= en only specific unusual network/hardware situations cause more than 0.0x% = reordering of packet delivery (at the IP level, ignoring things like TCP re= transmit, which is not an IP-level reordering in the way I'm discussing). > > I've seen significant reordering only on a network (an old office network= of mine!) where we had multiple bonded T1's and the packets went across on= e of them randomly (I presume). I think I got to low-single-digit percent = or more likely 0.n% (like 0.1-0.9%) reordered. There may also be some wire= less networks where this is more likely than on normal wired; I haven't not= iced it on WiFi though. > > It's easy to accidentally set up DummyNet/etc to cause massive packet reo= rdering, however. I do not think that should be a primary or even secondar= y testcase, and (to be honest) I don't think we should care much about dege= nerate reordering cases. I'm not even sure we should bother spending much = energy in our evaluation tests making it happen at all (which isn't to say = that the algorithm authors shouldn't consider it and if they feel it would = affect them, test it or discuss it - plus they will see a small amount of i= t on real networks). > From nobody Sun May 4 12:47:57 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4DD221A0125 for ; Sun, 4 May 2014 12:47:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.002 X-Spam-Level: X-Spam-Status: No, score=-0.002 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Junvr-5P_jyo for ; Sun, 4 May 2014 12:47:54 -0700 (PDT) Received: from sessmg22.ericsson.net (sessmg22.ericsson.net [193.180.251.58]) by ietfa.amsl.com (Postfix) with ESMTP id 65D381A0096 for ; Sun, 4 May 2014 12:47:53 -0700 (PDT) X-AuditID: c1b4fb3a-f79106d0000013ca-77-53669964bebd Received: from ESESSHC013.ericsson.se (Unknown_Domain [153.88.253.124]) by sessmg22.ericsson.net (Symantec Mail Security) with SMTP id 95.69.05066.46996635; Sun, 4 May 2014 21:47:48 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.151]) by ESESSHC013.ericsson.se ([153.88.183.57]) with mapi id 14.03.0174.001; Sun, 4 May 2014 21:47:48 +0200 From: Ingemar Johansson S To: Keith Winstein , "rmcat@ietf.org" Thread-Topic: Packet reordering Thread-Index: Ac9nz4UZwrkvruVzR5WRR6/WPOSJEg== Date: Sun, 4 May 2014 19:47:46 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.146] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrKLMWRmVeSWpSXmKPExsUyM+JvjW7KzLRgg5WTTSxOdtxislh98wOb A5PHkiU/mTyazhxlDmCK4rJJSc3JLEst0rdL4Mpoeq9acMOxYuf83ywNjA0OXYwcHBICJhJn Zpd0MXICmWISF+6tZ+ti5OIQEjjKKPHlxVYWCGcxo8TR1W/YQKrYBGwkVh76zghiiwi4SXz9 9BzMFhaQk/i5bz5UXF6i//0WdpAFIgJ6Egu3ZoOEWQRUJLrmnWcGsXkFfCVeT5nCBGIzCshK 3P9+jwXEZhYQl7j1ZD4TxEECEkv2QNRLCIhKvHz8jxXCVpL4seESC8h4ZgFNifW79CFaFSWm dD9khxgvKHFy5hOWCYzCs5BMnYXQMQtJxywkHQsYWVYxihanFhfnphsZ6aUWZSYXF+fn6eWl lmxiBIb7wS2/rXYwHnzueIhRgINRiYe3+EtksBBrYllxZe4hRmkOFiVx3kmL3IOFBNITS1Kz U1MLUovii0pzUosPMTJxcEo1MIZ1Snxd9trN//SURuk/Tw5eebW7yj/6f3LjnM12681eaN/Z EWnWtXLbmzU7WKw4hA4c3zyLd/eFpm9fTvYnPfJXiWFKevTruLWukun8Y1mB/87PvLai6lPh /KALSZp1HE/Fd34NvtRTtuHvo0M7DMod3kp194rvM2w4mnpVtmTyrcQH76LPZ5crsRRnJBpq MRcVJwIAoArIulgCAAA= Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/o_qlVJrlJ8vTONop_TtpjsnuBkk Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 19:47:56 -0000 SGkNCg0KQ2hhbmdlIHN1YmplY3QgbGluZSBhIGJpdC4NCg0KVGhhbmtzIGZvciBpbnRlcnN0aW5n IGlucHV0IEtlaXRoLiBEbyB5b3UgaGF2ZSBhbnkgZGF0YSBvbiBob3cgZGVlcCB0aGUgcmVwb3Jk ZXJpbmcgaXM/LiBJcyBpdCBkZWVwIGVub3VnaCB0byBjYXVzZSAzLWR1cGxpY2F0ZSBBQ0tzIGlu IFRDUCA/LCBJIHdvdWxkIGJlbGl2ZSB0aGF0IHBlb3BsZSB3b3VsZCBzdGFydCB0byBjb21wbGFp biBhYm91dCBwb29yIFRDUCBwZXJmb3JtYW5jZSA/LiBBZG1pdCB0aGF0IEkgb2NjYXNpb25hbGx5 IGV4cGVyaW5jZSBwb29yIFRDUCBwZXJmb3JtYW5jZSBpbiBteSBob21lIG5ldHdvcmsgYnV0IEkg aGF2ZSBzb2ZhciBibGFtZWQgaXQgbm90IG5vcm1hbCBwYWNrZXQgbG9zc2VzIGJ1dCBwZXJoYXBz IHRoZXJlIGNvdWxkIGJlIG90aGVyIGNhdXNlcy4NCg0KVGhlIHByZWNlbnNlIG9mIHJlb3JkZXJp bmcgaW4gM0dQUCBuZXR3b3JrcyBkZXBlbmQuIElmIHdlIGNvbnNpZGVyIGp1c3QgZGVmYXVsdCAo YmVzdCBlZmZvcnQpIGJlYXJlcnMgYW5kIGxvb2sgYXQgdGhlIHJhZGlvIGludGVyZmFjZSBvbmx5 IHRoZW4gdGhlIGFuc3dlciBpcyB0aGF0IHJlb3JkZXJpbmcgd2lsbCBub3QgaGFwcGVuLCB0aGUg cmVhc29uIGlzIHRoYXQgdGhlIGRlZmF1bHQgYmVhcmVycyB1c2UgQWNrbm93bGVkZ2VkIG1vZGUg dHJhbnNtaXNzaW9uIHdpdGggZ3VhcmFudGVlZCBpbiBvcmRlciBkZWxpdmVyeSB0byBoaWdoZXIg bGF5ZXJzLiBUaGVyZSBhcmUgb3RoZXIgZS5nIFFvUyBiZWFyZXJzIHRoYXQgdXNlIFVuYWNrbm93 bGVkZ2VkIG1vZGUgdHJhbnNtaXNzaW9uIGluIHdoaWNoIGNhc2UgcmVvcmRlcmluZyBtYXkgb2Nj dXIuDQpHb2luZyB0byB0aGUgYmFja2hhdWwuIEFzIEkgbWVudGlvbmVkIGluIGFuIGVhcmxpZXIg ZW1haWwsIHJlb3JkZXJpbmcgbWF5IG9jY3VyIGhlcmUgZHVlIHRvIGxvYWQgYmFsYW5jaW5nIGFu ZCBJIGhhdmUgc2VlbiBleGFtcGxlcyBvZiB0aGF0LCBob3dldmVyIG9ubHkgZW5vdWdoIHRvIHRy aWdnZXIgb25lIGR1cEFDSy4gSW4gdGVvcnkgR1RQLVUgc2VxdWVuY2UgbnVtYmVycyBjYW4gYmUg ZW5hYmxlZCBhbmQgdGhlIHBhY2tldCBvcmRlcmluZyBjYW4gYmUgcmVzdG9yZWQgYXQgdGhlIHR1 bm5lbCBlZ3Jlc3MsIGRvdWJ0IGhvd2V2ZXIgdGhhdCB0aGlzIGlzIHVzZWQuDQoNCkkgZG9uJ3Qg YXJndWUgYWdhaW5zdCB0ZXN0aW5nIHJvYnVzdG5lc3MgYWdhaW5zdCByZW9yZGVyaW5nICh3aXRo aW4gcmVhc29uKSBidXQgcGVyaGFwcyBpdCBzaG91bGQgdGhlbiBiZSBhIHNlcGFyYXRlIHRlc3Qg Y2FzZSBpZiBuZWNlc3NhcnkgPyBTb21laG93IEkgYmVsaWV2ZSBob3dldmVyIHRoYXQgcmVvcmRl cmluZyBpcyBhIGJpZ2dlciBjaGFsbGVuZ2UgZm9yIGF1ZGlvIGFuZCB2aWRlbyByZW5kZXJpbmcg KG5lY2Vzc2lhdGVzIGEgcmVjZWl2ZXIgYnVmZmVyKSB0aGFuIGZvciB0aGUgYWN0dWFsIHJhdGUg YWRhcHRhdGlvbiBhbGdvcml0aG1zLg0KDQovSW5nZW1hcg0KDQo+IC0tLS0tT3JpZ2luYWwgTWVz c2FnZS0tLS0tDQo+IEZyb206IEtlaXRoIFdpbnN0ZWluIFttYWlsdG86a2VpdGh3QG1pdC5lZHVd DQo+IFNlbnQ6IGRlbiAyIG1haiAyMDE0IDIyOjA5DQo+IFRvOiBybWNhdEBpZXRmLm9yZw0KPiBT dWJqZWN0OiBSZTogW3JtY2F0XSBybWNhdCBEaWdlc3QsIFZvbCAyMSwgSXNzdWUgMg0KPiANCj4g T3VyIGV4cGVyaWVuY2Ugd2l0aCBwYWNrZXQgcmVvcmRlcmluZyBoYXMgYWN0dWFsbHkgYmVlbiBx dWl0ZSBkaWZmZXJlbnQuIFdlDQo+IChNb3NoKSBoYXZlIGZvdW5kIHRoYXQgcGFja2V0IHJlb3Jk ZXJpbmcgY2FuIGJlIGV4dHJlbWVseSBjb21tb24gb24NCj4gODAyLjExbiBuZXR3b3JrcyB3aGVu IHRoZSAiYmxvY2sgYWNrcyIgZmVhdHVyZSBpcyB1c2VkIGFuZCBhIGxpbmsgaXMNCj4gc2F0dXJh dGVkLCBhdCBsZWFzdCB3aXRoIHNvbWUgcmVjZWl2aW5nIDgwMi4xMW4gY2hpcHNldHMsIGluY2x1 ZGluZyB0aGUgb25lcyBpbg0KPiBNYWNCb29rcy4NCj4gDQo+IEluIE1vc2ggdmVyc2lvbiAxLjIu Miwgd2UgdHVybmVkIG9uIGFuIGFkdmlzb3J5IHdhcm5pbmcgd2hlbiBwYWNrZXRzIHdlcmUNCj4g cmVvcmRlcmVkIGluIHRoZSBuZXR3b3JrLCBiZWNhdXNlIHdlIHRob3VnaHQgdGhhdCBpbiBwcmFj dGljZSByZW9yZGVyaW5nDQo+IHdhcyByYXJlIGVub3VnaCB0byBiZSBub3RhYmxlIGFuZCBtaWdo dCBzaWduaWZ5IGFuIGF0dGVtcHRlZCBhdHRhY2suDQo+IA0KPiBBZnRlciB0aGUgcmVsZWFzZSwg d2UgZ290IGEgbG90IG9mIGNvbXBsYWludHMgdGhhdCB0aGUgd2FybmluZyB3YXMgc2hvd2luZw0K PiB1cCBmcmVxdWVudGx5IGZvciB1c2VycyBvbiBlbnRlcnByaXNlIDgwMi4xMW4gbmV0d29ya3Mg dXNpbmcgdGhlIE1hY0Jvb2sNCj4gYW5kIGEgZmV3IG90aGVyIGRldmljZXMuIE9uIGEgZnVsbC10 aHJvdHRsZSBUQ1AgZG93bmxvYWQgdG8gYSBNYWNCb29rIFBybywNCj4gd2UgZm91bmQgdGhhdCBz b21ldGhpbmcgbGlrZSAzMC01MCUgb2YgYWxsIFRDUCBzZWdtZW50cyB3ZXJlIGJlaW5nDQo+IGRl bGl2ZXJlZCBvdXQgb2Ygb3JkZXIhIEFuZCBNb3NoJ3MgVURQIGRhdGFncmFtcyB3ZXJlIHNlZWlu ZyBzaW1pbGFyDQo+IHRyZWF0bWVudCB3aGVuIHRoZXJlIHdhcyBhIGJhY2stdG8tYmFjayBmbGln aHQgb2YgdGhlbS4gV2UgZW5kZWQgdXANCj4gZGlzYWJsaW5nIHRoZSB3YXJuaW5nIGluIHYxLjIu My4NCj4gDQo+IEFzIEkgdW5kZXJzdGFuZCwgdGhlIGJsb2NrIGFja3MgZmVhdHVyZSBtZWFucyB0 aGF0IGEgc2VuZGVyIG1heSB0cmFuc21pdCBhDQo+IHdob2xlIGNodW5rIG9mIGZyYW1lcyAoc29t ZSBvZiB3aGljaCB3aWxsIGJlIHJlY2VpdmVkIHdpdGhvdXQgZXJyb3IgYW5kDQo+IHNvbWUgd29u J3QpLCBhbmQgdGhlbiB0aGUgcmVjZWl2aW5nIHN0YXRpb24gY2FuIHRlbGwgdGhlIHNlbmRlciB3 aGljaCBmcmFtZXMNCj4gd2VyZSBzdWNjZXNzZnVsbHkgcmVjZWl2ZWQgYW5kIHdoaWNoIG5lZWQg dG8gYmUgcmV0cmFuc21pdHRlZC4gSW4gdGhlDQo+IG1lYW50aW1lLCB0aGUgcmVjZWl2ZXIgY2Fu IGRlbGl2ZXIgdGhlIHN1Y2Nlc3NmdWxseS1yZWNlaXZlZCBkYXRhZ3JhbXMgdG8gSVANCj4gb3Ig d2FpdC4NCj4gDQo+IFJGQyAzMzY2IDMuMSBlc3NlbnRpYWxseSBhZHZpc2VzIGxpbmsgZGVzaWdu ZXJzIHRoYXQgdGhleSBzaG91bGQgdXNlIGEgcmVjZWl2ZS0NCj4gc2lkZSByZW9yZGVyaW5nIGJ1 ZmZlciB0byB0cnkgdG8gcHV0IGRhdGFncmFtcyBiYWNrIGluIG9yZGVyIGJlZm9yZSBkZWxpdmVy eQ0KPiB0byBJUCwgYnV0IG91ciB0ZXN0aW5nIHN1Z2dlc3RzIHRoYXQgY29tbW9uIDgwMi4xMW4g cmVjZWl2ZSBjaGlwc2V0cyBkb24ndA0KPiBkbyB0aGlzIGluIHByYWN0aWNlLg0KPiANCj4gU28g b3VyIGV4cGVyaWVuY2UgaGFzIGJlZW4gdGhhdCBpbiB0aGUgY29udGVtcG9yYXJ5IEludGVybmV0 IChhdCBsZWFzdCBvbg0KPiBzb21lIDgwMi4xMW4gbmV0d29ya3MpLCByZW9yZGVyaW5nIGNhbiBi ZSBtdWNoIG1vcmUgY29tbW9uIHRoYW4gd2UNCj4gaGFkIGluaXRpYWxseSBiZWxpZXZlZC4NCj4g DQo+IE9uIEZyaSwgTWF5IDIsIDIwMTQgYXQgMzowMCBQTSwgIDxybWNhdC1yZXF1ZXN0QGlldGYu b3JnPiB3cm90ZToNCj4gPiAtLS0tLS0tLS0tIEZvcndhcmRlZCBtZXNzYWdlIC0tLS0tLS0tLS0N Cj4gPiBGcm9tOiBSYW5kZWxsIEplc3VwIDxyYW5kZWxsLWlldGZAamVzdXAub3JnPg0KPiA+IFRv OiBybWNhdEBpZXRmLm9yZw0KPiA+IENjOg0KPiA+IERhdGU6IEZyaSwgMDIgTWF5IDIwMTQgMDU6 NDk6MTYgLTA0MDANCj4gPiBTdWJqZWN0OiBSZTogW3JtY2F0XSBkcmFmdC1zYXJrZXItcm1jYXQt ZXZhbC10ZXN0LTAwLCB0ZXN0IGNhc2UgNC40DQo+ID4gUk1DQVQgRmxvdyBjb21wZXRpbmcgd2l0 aCBhIGxvbmcgVENQIEZsb3cgT24gNC8zMC8yMDE0IDE6NTEgUE0sIENvbGluDQo+IFBlcmtpbnMg d3JvdGU6DQo+ID4+DQo+ID4+IE9uIDMwIEFwciAyMDE0LCBhdCAwNzo1MCwgWmFoZWR1enphbWFu IFNhcmtlcg0KPiA8emFoZWR1enphbWFuLnNhcmtlckBlcmljc3Nvbi5jb20+IHdyb3RlOg0KPiA+ Pj4NCj4gPj4+IEFyZSB5b3Ugc2F5aW5nIHJlLW9yZGVyaW5nIGlzIG5vdCBhIHBoZW5vbWVuYSB0 aGF0IHdlIGNhbiBvYnNlcnZlIGluDQo+IHRoZSBjdXJyZW50IEludGVybmV0PyBBbmQgd2Ugc2hv dWxkIG5vdCBoYXZlIHRlc3QgY2FzZXMgdG8gaGFuZGxlIHJlLQ0KPiBvcmRlcmluZyBvZiBwYWNr ZXRzIGluIFJNQ0FUPw0KPiA+Pg0KPiA+PiBPbmUgb2YgbXkgc3R1ZGVudHMgY29sbGVjdGVkIHNv bWUgVURQIHN0cmVhbWluZyBtZWFzdXJlbWVudHMgZnJvbQ0KPiBvdXIgdW5pdmVyc2l0eSB0byBo b3N0cyBvbiByZXNpZGVudGlhbCBuZXR3b3JrcywgY292ZXJpbmcgYWxsIHRoZSBtYWluIFVLIElT UHMsDQo+IGEgY291cGxlIG9mIHllYXJzIGFnby4gV2l0aCBldmVyeXRoaW5nIHVzaW5nIHdpcmVk IGNvbm5lY3Rpb25zIGFuZCBVRFAgd2UNCj4gc2F3IGEgY291cGxlIG9mIGh1bmRyZWQgcGFja2V0 cyBkZWxpdmVyZWQgb3V0IG9mIG9yZGVyLCBmcm9tIH4yNTAgbWlsbGlvbg0KPiBwYWNrZXRzIHNl bnQuIEkgY2FuIGRpZyBpbnRvIHRoZSBkZXRhaWxzIG9mIHRoZSByZS1vcmRlcmluZyBwYXR0ZXJu cyBpZiB0aGVyZeKAmXMNCj4gaW50ZXJlc3QuIFdlIGRpZG7igJl0IGFuYWx5c2UgdGhlbSB0b28g Y2xvc2VseSBhdCB0aGUgdGltZSBiZWNhdXNlIHBhY2tldCBsb3NzDQo+IGFuZCBkZWxheSBzcGlr ZXMgd2VyZSBtdWNoIG1vcmUgY29tbW9uLg0KPiA+DQo+ID4NCj4gPiBIYXZpbmcgaXQgbm90IGZh bGwgb3ZlciBvbiAoc29tZSkgcmVvcmRlcmluZzogZ29vZC4gIE15IGV4cGVyaWVuY2UgaGFzDQo+ IGJlZW4gb25seSBzcGVjaWZpYyB1bnVzdWFsIG5ldHdvcmsvaGFyZHdhcmUgc2l0dWF0aW9ucyBj YXVzZSBtb3JlIHRoYW4NCj4gMC4weCUgcmVvcmRlcmluZyBvZiBwYWNrZXQgZGVsaXZlcnkgKGF0 IHRoZSBJUCBsZXZlbCwgaWdub3JpbmcgdGhpbmdzIGxpa2UgVENQDQo+IHJldHJhbnNtaXQsIHdo aWNoIGlzIG5vdCBhbiBJUC1sZXZlbCByZW9yZGVyaW5nIGluIHRoZSB3YXkgSSdtIGRpc2N1c3Np bmcpLg0KPiA+DQo+ID4gSSd2ZSBzZWVuIHNpZ25pZmljYW50IHJlb3JkZXJpbmcgb25seSBvbiBh IG5ldHdvcmsgKGFuIG9sZCBvZmZpY2UgbmV0d29yayBvZg0KPiBtaW5lISkgd2hlcmUgd2UgaGFk IG11bHRpcGxlIGJvbmRlZCBUMSdzIGFuZCB0aGUgcGFja2V0cyB3ZW50IGFjcm9zcyBvbmUNCj4g b2YgdGhlbSByYW5kb21seSAoSSBwcmVzdW1lKS4gIEkgdGhpbmsgSSBnb3QgdG8gbG93LXNpbmds ZS1kaWdpdCBwZXJjZW50IG9yDQo+IG1vcmUgbGlrZWx5IDAubiUgKGxpa2UgMC4xLTAuOSUpIHJl b3JkZXJlZC4gIFRoZXJlIG1heSBhbHNvIGJlIHNvbWUgd2lyZWxlc3MNCj4gbmV0d29ya3Mgd2hl cmUgdGhpcyBpcyBtb3JlIGxpa2VseSB0aGFuIG9uIG5vcm1hbCB3aXJlZDsgSSBoYXZlbid0IG5v dGljZWQgaXQNCj4gb24gV2lGaSB0aG91Z2guDQo+ID4NCj4gPiBJdCdzIGVhc3kgdG8gYWNjaWRl bnRhbGx5IHNldCB1cCBEdW1teU5ldC9ldGMgdG8gY2F1c2UgbWFzc2l2ZSBwYWNrZXQNCj4gcmVv cmRlcmluZywgaG93ZXZlci4gIEkgZG8gbm90IHRoaW5rIHRoYXQgc2hvdWxkIGJlIGEgcHJpbWFy eSBvciBldmVuDQo+IHNlY29uZGFyeSB0ZXN0Y2FzZSwgYW5kICh0byBiZSBob25lc3QpIEkgZG9u J3QgdGhpbmsgd2Ugc2hvdWxkIGNhcmUgbXVjaA0KPiBhYm91dCBkZWdlbmVyYXRlIHJlb3JkZXJp bmcgY2FzZXMuICBJJ20gbm90IGV2ZW4gc3VyZSB3ZSBzaG91bGQgYm90aGVyDQo+IHNwZW5kaW5n IG11Y2ggZW5lcmd5IGluIG91ciBldmFsdWF0aW9uIHRlc3RzIG1ha2luZyBpdCBoYXBwZW4gYXQg YWxsICh3aGljaA0KPiBpc24ndCB0byBzYXkgdGhhdCB0aGUgYWxnb3JpdGhtIGF1dGhvcnMgc2hv dWxkbid0IGNvbnNpZGVyIGl0IGFuZCBpZiB0aGV5IGZlZWwgaXQNCj4gd291bGQgYWZmZWN0IHRo ZW0sIHRlc3QgaXQgb3IgZGlzY3VzcyBpdCAtIHBsdXMgdGhleSB3aWxsIHNlZSBhIHNtYWxsIGFt b3VudCBvZiBpdA0KPiBvbiByZWFsIG5ldHdvcmtzKS4NCj4gPg0KPiANCg0K From nobody Sun May 4 16:17:48 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 21E251A0139 for ; Sun, 4 May 2014 16:17:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.278 X-Spam-Level: X-Spam-Status: No, score=-1.278 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nAJlYtlJrnba for ; Sun, 4 May 2014 16:17:45 -0700 (PDT) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) by ietfa.amsl.com (Postfix) with ESMTP id BB56B1A0136 for ; Sun, 4 May 2014 16:17:45 -0700 (PDT) Received: by mail-qa0-f45.google.com with SMTP id hw13so6181889qab.18 for ; Sun, 04 May 2014 16:17:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=HOl159SMd/W9p/P2LkWoh2r/7Wu6rgDwISKwBENKckA=; b=rrk9NVkWkYZCZcyNy/NA4JQjs42G3Wy9tL5XLpqElelr9RcSOPBmWG9ydL19bTB5w8 ePLzafr2tJw4RhUnNBmMReFj9cCCcz02JGDISTmDpk3UcxkbXmBhCPx7Mlqe5+JO0MRm 5CEJmF8Cko46ux3mRsRnW1bVZUY8f4aw4MiCPkz5nHeXEhafJly58WGl6HBNcc3KCUf1 YcQnMGg/IfNHr78xVfZQwzyq9UsnVo4JMgkijIPP6W3T7ddFWDDIYIIteXv7iTkndZQp Qz6yfxBpX3i9zubbZ6NtLYm3vF00WWz5bUmspf+SBwS7N2qLtXzU6O6tmHWw7A/wgRuB Iicw== X-Received: by 10.224.138.3 with SMTP id y3mr40078618qat.78.1399245462407; Sun, 04 May 2014 16:17:42 -0700 (PDT) MIME-Version: 1.0 Sender: winstein@gmail.com Received: by 10.229.28.69 with HTTP; Sun, 4 May 2014 16:17:02 -0700 (PDT) In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> From: Keith Winstein Date: Sun, 4 May 2014 19:17:02 -0400 X-Google-Sender-Auth: TphuL7D6gNMTD4kzzLK0f-KRrPk Message-ID: To: Ingemar Johansson S Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/2cVbTEAp09MEFdHS36dmC_5xF5E Cc: "rmcat@ietf.org" Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 23:17:47 -0000 Hello Ingemar, I'd have to do a more careful (and recent!) measurement to answer you clearly, but my recollection is that it was generally not deep enough to cause a triple dupack except perhaps very rarely. Generally speaking, as I recall the typical behavior was to see groups of 3-4 MSS-sized TCP segments randomly swizzled. My experience matches yours re: commercial service on 1xEV-DO, UMTS and LTE networks. Best regards, Keith On Sun, May 4, 2014 at 3:47 PM, Ingemar Johansson S wrote: > Hi > > Change subject line a bit. > > Thanks for intersting input Keith. Do you have any data on how deep the r= epordering is?. Is it deep enough to cause 3-duplicate ACKs in TCP ?, I wou= ld belive that people would start to complain about poor TCP performance ?.= Admit that I occasionally experince poor TCP performance in my home networ= k but I have sofar blamed it not normal packet losses but perhaps there cou= ld be other causes. > > The precense of reordering in 3GPP networks depend. If we consider just d= efault (best effort) bearers and look at the radio interface only then the = answer is that reordering will not happen, the reason is that the default b= earers use Acknowledged mode transmission with guaranteed in order delivery= to higher layers. There are other e.g QoS bearers that use Unacknowledged = mode transmission in which case reordering may occur. > Going to the backhaul. As I mentioned in an earlier email, reordering may= occur here due to load balancing and I have seen examples of that, however= only enough to trigger one dupACK. In teory GTP-U sequence numbers can be = enabled and the packet ordering can be restored at the tunnel egress, doubt= however that this is used. > > I don't argue against testing robustness against reordering (within reaso= n) but perhaps it should then be a separate test case if necessary ? Someho= w I believe however that reordering is a bigger challenge for audio and vid= eo rendering (necessiates a receiver buffer) than for the actual rate adapt= ation algorithms. > > /Ingemar > >> -----Original Message----- >> From: Keith Winstein [mailto:keithw@mit.edu] >> Sent: den 2 maj 2014 22:09 >> To: rmcat@ietf.org >> Subject: Re: [rmcat] rmcat Digest, Vol 21, Issue 2 >> >> Our experience with packet reordering has actually been quite different.= We >> (Mosh) have found that packet reordering can be extremely common on >> 802.11n networks when the "block acks" feature is used and a link is >> saturated, at least with some receiving 802.11n chipsets, including the = ones in >> MacBooks. >> >> In Mosh version 1.2.2, we turned on an advisory warning when packets wer= e >> reordered in the network, because we thought that in practice reordering >> was rare enough to be notable and might signify an attempted attack. >> >> After the release, we got a lot of complaints that the warning was showi= ng >> up frequently for users on enterprise 802.11n networks using the MacBook >> and a few other devices. On a full-throttle TCP download to a MacBook Pr= o, >> we found that something like 30-50% of all TCP segments were being >> delivered out of order! And Mosh's UDP datagrams were seeing similar >> treatment when there was a back-to-back flight of them. We ended up >> disabling the warning in v1.2.3. >> >> As I understand, the block acks feature means that a sender may transmit= a >> whole chunk of frames (some of which will be received without error and >> some won't), and then the receiving station can tell the sender which fr= ames >> were successfully received and which need to be retransmitted. In the >> meantime, the receiver can deliver the successfully-received datagrams t= o IP >> or wait. >> >> RFC 3366 3.1 essentially advises link designers that they should use a r= eceive- >> side reordering buffer to try to put datagrams back in order before deli= very >> to IP, but our testing suggests that common 802.11n receive chipsets don= 't >> do this in practice. >> >> So our experience has been that in the contemporary Internet (at least o= n >> some 802.11n networks), reordering can be much more common than we >> had initially believed. >> >> On Fri, May 2, 2014 at 3:00 PM, wrote: >> > ---------- Forwarded message ---------- >> > From: Randell Jesup >> > To: rmcat@ietf.org >> > Cc: >> > Date: Fri, 02 May 2014 05:49:16 -0400 >> > Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 >> > RMCAT Flow competing with a long TCP Flow On 4/30/2014 1:51 PM, Colin >> Perkins wrote: >> >> >> >> On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker >> wrote: >> >>> >> >>> Are you saying re-ordering is not a phenomena that we can observe in >> the current Internet? And we should not have test cases to handle re- >> ordering of packets in RMCAT? >> >> >> >> One of my students collected some UDP streaming measurements from >> our university to hosts on residential networks, covering all the main U= K ISPs, >> a couple of years ago. With everything using wired connections and UDP w= e >> saw a couple of hundred packets delivered out of order, from ~250 millio= n >> packets sent. I can dig into the details of the re-ordering patterns if = there=E2=80=99s >> interest. We didn=E2=80=99t analyse them too closely at the time because= packet loss >> and delay spikes were much more common. >> > >> > >> > Having it not fall over on (some) reordering: good. My experience has >> been only specific unusual network/hardware situations cause more than >> 0.0x% reordering of packet delivery (at the IP level, ignoring things li= ke TCP >> retransmit, which is not an IP-level reordering in the way I'm discussin= g). >> > >> > I've seen significant reordering only on a network (an old office netw= ork of >> mine!) where we had multiple bonded T1's and the packets went across one >> of them randomly (I presume). I think I got to low-single-digit percent= or >> more likely 0.n% (like 0.1-0.9%) reordered. There may also be some wire= less >> networks where this is more likely than on normal wired; I haven't notic= ed it >> on WiFi though. >> > >> > It's easy to accidentally set up DummyNet/etc to cause massive packet >> reordering, however. I do not think that should be a primary or even >> secondary testcase, and (to be honest) I don't think we should care much >> about degenerate reordering cases. I'm not even sure we should bother >> spending much energy in our evaluation tests making it happen at all (wh= ich >> isn't to say that the algorithm authors shouldn't consider it and if the= y feel it >> would affect them, test it or discuss it - plus they will see a small am= ount of it >> on real networks). >> > >> > From nobody Wed May 7 04:11:55 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D51EF1A06DF for ; Wed, 7 May 2014 04:11:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.55 X-Spam-Level: X-Spam-Status: No, score=-2.55 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HfmcoIZXpFn4 for ; Wed, 7 May 2014 04:11:51 -0700 (PDT) Received: from mail-out5.uio.no (mail-out5.uio.no [IPv6:2001:700:100:10::17]) by ietfa.amsl.com (Postfix) with ESMTP id 56AD11A06D0 for ; Wed, 7 May 2014 04:11:50 -0700 (PDT) Received: from mail-mx3.uio.no ([129.240.10.44]) by mail-out5.uio.no with esmtp (Exim 4.80.1) (envelope-from ) id 1WhzlJ-00026s-75 for rmcat@ietf.org; Wed, 07 May 2014 13:11:45 +0200 Received: from boomerang.ifi.uio.no ([129.240.68.135]) by mail-mx3.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from ) id 1WhzlI-0007cJ-BP for rmcat@ietf.org; Wed, 07 May 2014 13:11:45 +0200 From: Michael Welzl Content-Type: multipart/alternative; boundary="Apple-Mail=_6F004533-2FDA-453F-BF5C-BCAB29773C7D" Date: Wed, 7 May 2014 13:11:43 +0200 References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> To: rmcat WG Message-Id: Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 1 msgs/h 1 sum rcpts/h 1 sum msgs/h 1 total rcpts 16140 max rcpts/h 44 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-6.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.051, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 73C58ADF65C24A5C14CFC53D1DFB85318E9F73F4 X-UiO-SPAM-Test: remote_host: 129.240.68.135 spam_score: -59 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 5214 max/h 16 blacklist 0 greylist 0 ratelimit 0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/Txabq6CrAi5SR-sMQ9FW7AeKwbA Subject: [rmcat] Fwd: New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 11:11:54 -0000 --Apple-Mail=_6F004533-2FDA-453F-BF5C-BCAB29773C7D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Dear all, We have submitted an update of our coupled congestion control draft. = This now includes the new algorithm which should reduce queuing delay = and packet loss, as documented in = http://safiquli.at.ifi.uio.no/paper/fse-tech-report.pdf At this point, we'd really appreciate it if folks would take a look and = comment :-) Cheers, Michael Begin forwarded message: > From: internet-drafts@ietf.org > Subject: New Version Notification for = draft-welzl-rmcat-coupled-cc-03.txt > Date: 7. mai 2014 13:08:05 GMT+02:00 > To: Michael Welzl , "Safiqul Islam" = , "Stein Gjessing" , "Michael = Welzl" , Safiqul Islam , Stein = Gjessing >=20 >=20 > A new version of I-D, draft-welzl-rmcat-coupled-cc-03.txt > has been successfully submitted by Michael Welzl and posted to the > IETF repository. >=20 > Name: draft-welzl-rmcat-coupled-cc > Revision: 03 > Title: Coupled congestion control for RTP media > Document date: 2014-05-07 > Group: Individual Submission > Pages: 19 > URL: = http://www.ietf.org/internet-drafts/draft-welzl-rmcat-coupled-cc-03.txt > Status: = https://datatracker.ietf.org/doc/draft-welzl-rmcat-coupled-cc/ > Htmlized: = http://tools.ietf.org/html/draft-welzl-rmcat-coupled-cc-03 > Diff: = http://www.ietf.org/rfcdiff?url2=3Ddraft-welzl-rmcat-coupled-cc-03 >=20 > Abstract: > When multiple congestion controlled RTP sessions traverse the same > network bottleneck, it can be beneficial to combine their controls > such that the total on-the-wire behavior is improved. This document > describes such a method for flows that have the same sender, in a = way > that is as flexible and simple as possible while minimizing the > amount of changes needed to existing RTP applications. >=20 >=20 >=20 >=20 > Please note that it may take a couple of minutes from the time of = submission > until the htmlized version and diff are available at tools.ietf.org. >=20 > The IETF Secretariat >=20 --Apple-Mail=_6F004533-2FDA-453F-BF5C-BCAB29773C7D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Dear = all,

We have submitted an update of our coupled = congestion control draft. This now includes the new algorithm which = should reduce queuing delay and packet loss, as documented in http://sa= fiquli.at.ifi.uio.no/paper/fse-tech-report.pdf

At this point, we'd really appreciate it if folks would take a look = and comment   = :-)

Cheers,
Michael

=

Begin forwarded message:

Subject: New Version Notification for = draft-welzl-rmcat-coupled-cc-03.txt
Date: 7. mai 2014 = 13:08:05 GMT+02:00
To: Michael Welzl <michawe@ifi.uio.no>, "Safiqul = Islam" <safiquli@ifi.uio.no>, "Stein = Gjessing" <steing@ifi.uio.no>, "Michael = Welzl" <michawe@ifi.uio.no>, Safiqul = Islam <safiquli@ifi.uio.no>, Stein = Gjessing <steing@ifi.uio.no>


A new version of I-D, = draft-welzl-rmcat-coupled-cc-03.txt
has been successfully submitted = by Michael Welzl and posted to the
IETF repository.

Name: = draft-welzl-rmcat-coupled-cc
Revision: = 03
Title: Coupled congestion control for = RTP media
Document date: 2014-05-07
Group: = Individual Submission
Pages: 19
URL: =            http://www.ietf.org/internet-drafts/draft-welzl-rmcat-coupled-cc-03= .txt
Status:         ht= tps://datatracker.ietf.org/doc/draft-welzl-rmcat-coupled-cc/
Htmliz= ed:       http:/= /tools.ietf.org/html/draft-welzl-rmcat-coupled-cc-03
Diff: =           http://www.ietf.org/rfcdiff?url2=3Ddraft-welzl-rmcat-coupled-cc-03
Abstract:
  When multiple congestion controlled RTP = sessions traverse the same
  network bottleneck, it can be = beneficial to combine their controls
  such that the total = on-the-wire behavior is improved.  This document
=   describes such a method for flows that have the same sender, = in a way
  that is as flexible and simple as possible = while minimizing the
  amount of changes needed to = existing RTP applications.




Please note that it may = take a couple of minutes from the time of submission
until the = htmlized version and diff are available at tools.ietf.org.

The IETF = Secretariat


= --Apple-Mail=_6F004533-2FDA-453F-BF5C-BCAB29773C7D-- From nobody Thu May 8 00:12:21 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 554E41A010A for ; Thu, 8 May 2014 00:12:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.652 X-Spam-Level: X-Spam-Status: No, score=-0.652 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f8FVj7hIKzLW for ; Thu, 8 May 2014 00:12:14 -0700 (PDT) Received: from smtp-out-02.aalto.fi (smtp-out-02.aalto.fi [130.233.228.121]) by ietfa.amsl.com (Postfix) with ESMTP id 22DA71A0032 for ; Thu, 8 May 2014 00:12:13 -0700 (PDT) Received: from smtp-out-02.aalto.fi (localhost.localdomain [127.0.0.1]) by localhost (Email Security Appliance) with SMTP id 82B4A2711E1_36B2E48B for ; Thu, 8 May 2014 07:12:08 +0000 (GMT) Received: from EXHUB03.org.aalto.fi (exhub03.org.aalto.fi [130.233.222.116]) by smtp-out-02.aalto.fi (Sophos Email Appliance) with ESMTP id 0D7E8271088_36B2E48F for ; Thu, 8 May 2014 07:12:08 +0000 (GMT) Received: from [130.233.154.122] (130.233.154.122) by mail.aalto.fi (130.233.222.116) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 8 May 2014 10:12:07 +0300 Message-ID: <536B2E47.7090005@aalto.fi> Date: Thu, 8 May 2014 10:12:07 +0300 From: Lennart Schulte User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> In-Reply-To: <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> X-Enigmail-Version: 1.5.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SLjSbUr8BMvRiuKeWbKd5X98MxaU57f59" X-Originating-IP: [130.233.154.122] Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/qH5oGfgFutU3Me6SQj_pBH_VoAc Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 07:12:19 -0000 --SLjSbUr8BMvRiuKeWbKd5X98MxaU57f59 Content-Type: multipart/mixed; boundary="------------070404010704060306010903" This is a multi-part message in MIME format. --------------070404010704060306010903 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, while measuring 3G and 4G networks I also saw reordering: 35% of all connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of these reordering events the reordering extent was at least 3 packets large (which caused TCP to spuriously enter fast recovery). We are currently working on 2 drafts about TCP robustness to reordering, maybe they are of interest: - Detection and quantification http://tools.ietf.org/html/draft-zimmermann-tcpm-reordering-detection - Reaction and prevention http://tools.ietf.org/html/draft-zimmermann-tcpm-reordering-reaction Regards, Lennart On 05/05/2014 02:17 AM, Keith Winstein wrote: > Hello Ingemar, >=20 > I'd have to do a more careful (and recent!) measurement to answer you > clearly, but my recollection is that it was generally not deep enough > to cause a triple dupack except perhaps very rarely. Generally > speaking, as I recall the typical behavior was to see groups of 3-4 > MSS-sized TCP segments randomly swizzled. >=20 > My experience matches yours re: commercial service on 1xEV-DO, UMTS > and LTE networks. >=20 > Best regards, > Keith >=20 > On Sun, May 4, 2014 at 3:47 PM, Ingemar Johansson S > wrote: >> Hi >> >> Change subject line a bit. >> >> Thanks for intersting input Keith. Do you have any data on how deep th= e repordering is?. Is it deep enough to cause 3-duplicate ACKs in TCP ?, = I would belive that people would start to complain about poor TCP perform= ance ?. Admit that I occasionally experince poor TCP performance in my ho= me network but I have sofar blamed it not normal packet losses but perhap= s there could be other causes. >> >> The precense of reordering in 3GPP networks depend. If we consider jus= t default (best effort) bearers and look at the radio interface only then= the answer is that reordering will not happen, the reason is that the de= fault bearers use Acknowledged mode transmission with guaranteed in order= delivery to higher layers. There are other e.g QoS bearers that use Unac= knowledged mode transmission in which case reordering may occur. >> Going to the backhaul. As I mentioned in an earlier email, reordering = may occur here due to load balancing and I have seen examples of that, ho= wever only enough to trigger one dupACK. In teory GTP-U sequence numbers = can be enabled and the packet ordering can be restored at the tunnel egre= ss, doubt however that this is used. >> >> I don't argue against testing robustness against reordering (within re= ason) but perhaps it should then be a separate test case if necessary ? S= omehow I believe however that reordering is a bigger challenge for audio = and video rendering (necessiates a receiver buffer) than for the actual r= ate adaptation algorithms. >> >> /Ingemar >> >>> -----Original Message----- >>> From: Keith Winstein [mailto:keithw@mit.edu] >>> Sent: den 2 maj 2014 22:09 >>> To: rmcat@ietf.org >>> Subject: Re: [rmcat] rmcat Digest, Vol 21, Issue 2 >>> >>> Our experience with packet reordering has actually been quite differe= nt. We >>> (Mosh) have found that packet reordering can be extremely common on >>> 802.11n networks when the "block acks" feature is used and a link is >>> saturated, at least with some receiving 802.11n chipsets, including t= he ones in >>> MacBooks. >>> >>> In Mosh version 1.2.2, we turned on an advisory warning when packets = were >>> reordered in the network, because we thought that in practice reorder= ing >>> was rare enough to be notable and might signify an attempted attack. >>> >>> After the release, we got a lot of complaints that the warning was sh= owing >>> up frequently for users on enterprise 802.11n networks using the MacB= ook >>> and a few other devices. On a full-throttle TCP download to a MacBook= Pro, >>> we found that something like 30-50% of all TCP segments were being >>> delivered out of order! And Mosh's UDP datagrams were seeing similar >>> treatment when there was a back-to-back flight of them. We ended up >>> disabling the warning in v1.2.3. >>> >>> As I understand, the block acks feature means that a sender may trans= mit a >>> whole chunk of frames (some of which will be received without error a= nd >>> some won't), and then the receiving station can tell the sender which= frames >>> were successfully received and which need to be retransmitted. In the= >>> meantime, the receiver can deliver the successfully-received datagram= s to IP >>> or wait. >>> >>> RFC 3366 3.1 essentially advises link designers that they should use = a receive- >>> side reordering buffer to try to put datagrams back in order before d= elivery >>> to IP, but our testing suggests that common 802.11n receive chipsets = don't >>> do this in practice. >>> >>> So our experience has been that in the contemporary Internet (at leas= t on >>> some 802.11n networks), reordering can be much more common than we >>> had initially believed. >>> >>> On Fri, May 2, 2014 at 3:00 PM, wrote: >>>> ---------- Forwarded message ---------- >>>> From: Randell Jesup >>>> To: rmcat@ietf.org >>>> Cc: >>>> Date: Fri, 02 May 2014 05:49:16 -0400 >>>> Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 >>>> RMCAT Flow competing with a long TCP Flow On 4/30/2014 1:51 PM, Coli= n >>> Perkins wrote: >>>>> >>>>> On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker >>> wrote: >>>>>> >>>>>> Are you saying re-ordering is not a phenomena that we can observe = in >>> the current Internet? And we should not have test cases to handle re-= >>> ordering of packets in RMCAT? >>>>> >>>>> One of my students collected some UDP streaming measurements from >>> our university to hosts on residential networks, covering all the mai= n UK ISPs, >>> a couple of years ago. With everything using wired connections and UD= P we >>> saw a couple of hundred packets delivered out of order, from ~250 mil= lion >>> packets sent. I can dig into the details of the re-ordering patterns = if there=E2=80=99s >>> interest. We didn=E2=80=99t analyse them too closely at the time beca= use packet loss >>> and delay spikes were much more common. >>>> >>>> >>>> Having it not fall over on (some) reordering: good. My experience h= as >>> been only specific unusual network/hardware situations cause more tha= n >>> 0.0x% reordering of packet delivery (at the IP level, ignoring things= like TCP >>> retransmit, which is not an IP-level reordering in the way I'm discus= sing). >>>> >>>> I've seen significant reordering only on a network (an old office ne= twork of >>> mine!) where we had multiple bonded T1's and the packets went across = one >>> of them randomly (I presume). I think I got to low-single-digit perc= ent or >>> more likely 0.n% (like 0.1-0.9%) reordered. There may also be some w= ireless >>> networks where this is more likely than on normal wired; I haven't no= ticed it >>> on WiFi though. >>>> >>>> It's easy to accidentally set up DummyNet/etc to cause massive packe= t >>> reordering, however. I do not think that should be a primary or even= >>> secondary testcase, and (to be honest) I don't think we should care m= uch >>> about degenerate reordering cases. I'm not even sure we should bothe= r >>> spending much energy in our evaluation tests making it happen at all = (which >>> isn't to say that the algorithm authors shouldn't consider it and if = they feel it >>> would affect them, test it or discuss it - plus they will see a small= amount of it >>> on real networks). >>>> >>> >> >=20 --------------070404010704060306010903 Content-Type: application/pgp-keys; name="0x4534937E.asc" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0x4534937E.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQENBE17ak4BCADBT/sEL/sIqSKvq2i8kG6xnsiyThJhaE7NX9R7qlA6mL/AnuS5 t/XNDAHcMlgjUxwOhX6x1smTkeAftp4S2Gy8vLgu7BQPNUu/NijomVSGh/ncgtci Q3CMy2QHr+dWgYcKGZRz2r59IlHmx3EynCPwgT5u5OAQmWF17aOsHgHB5QHYV2g6 hBZ80lJKqrMRy0TGyYPue2Dv3jZx886J35IwzAJz8i3/OOBaqG4QY09CTY9lnVGL vl7VFil/h+XxZsHJG5JigznoVE48bKXe8ESyMnem2bo+HuSAg38jh8/jbE/+5Yb0 Rcuyt018t9FOpjH88g0ooBTo12cXnWrAnaJ3ABEBAAG0MExlbm5hcnQgU2NodWx0 ZSA8bGVubmFydC5zY2h1bHRlQHJ3dGgtYWFjaGVuLmRlPokBQQQTAQIAKwIbIwUJ CWYBgAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AFAk17besCGQEACgkQ8OibdUU0 k35w2wf+MibHd9ijw0WN5nJTJjJ9+vgEkGxCkFEKuYLR4a3g2ZiNoKa4w/qx5VP8 qycAinp8DeUnayaWKQI6sMHeLCqb5P0sn3nqmngxWRKDzT/bOdGUeN4fDte/IXhx M6drz5B1Ld6K62iNO5+E2wcvVlLHIbShQD4S35/vBBhWC021aVdm/3uIAp4myT/J KoC+6HZJcTugjXIIvk4TYLUmsFUODw0N5y8w0hBqFjncRw9dMX+e1eNN6oFbn/jH e1PEl03fJL7hZcpXd1GTV2Bk3Gw22ifPkC0n2HRe0xBsAsd3zR4rRWUdC3WXLcHS STGNZVGWn/DHzAPXCYIxuwTsN2G9ZokBPgQTAQIAKAUCTXtqTgIbIwUJCWYBgAYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ8OibdUU0k37Fzwf8DJvqOaehI7yX do/CFkSNsQ6opvmKQLslEnXja2QdPm3LgwgVLaS9RkV1e76AhKWxtjR8QJgaafVn k7cYA7rgI2OHv7MwbFcpX01VsWrCAmgBFtSN6E2aDi2Ys/Ch56BPghfEg+08EdRU bb7wGSBTwZ1DsdtGgmMsxNB9oGgwFD2qBsUnZs1LUdlvCGuXbRzrQDpt4lA/zTtZ rvXC2mXjkCTlDqDWtRDxRE6+LvuUeD2h5t69zbHluwqHWUIhH4lzsKmamY2QzboC uVuSuLs22p3743dDx5RtLJM20SNx70DHz+eMCU92WQ95/v2DXZUcd+Y8jx02sAii JEF1q8FCZbQ1TGVubmFydCBTY2h1bHRlIDxsZW5uYXJ0LnNjaHVsdGVAbmV0cy5y d3RoLWFhY2hlbi5kZT6JAT4EEwECACgFAk17bc4CGyMFCQlmAYAGCwkIBwMCBhUI AgkKCwQWAgMBAh4BAheAAAoJEPDom3VFNJN+L1kIAKdQU/pTsmitJTi65sUu80mi lJXIQruksZCA5cYPNqzB7uUD1hzBosoqilSji/HWols107aBakrafssuvKGLnK3h AVt7RSZQ0K/KmUlCay9UimLN8dsHLyeDNxdG2yil9c5/eI5EgAt3Lk+KcBcVoSQ/ eqIdeCMfvVSZStC/QpGNykY2g+MLATskol2w5IBCb1gIq8oKPzpuogv9tOh3qSaz p8pb99M8mmdAaoJFJBptLBdsYdD1g3cOkreJrXBxmFrcYfgIheiSJBvMg0Vn5krv 2OC4wMs2IQMgN4GiqD9mFZScJO6az79xkhhx6eTxr7qe8J84RCGPhdUeX//fv4a0 Kkxlbm5hcnQgU2NodWx0ZSA8bGhzY2h1bHRlQGdvb2dsZW1haWwuY29tPokBPgQT AQIAKAUCTXttvQIbIwUJCWYBgAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ 8OibdUU0k37Q+Qf+MKh1jAdMo7ppelCbOWGYIWYwrYUGMmeOpLVCbTRbW/CibwsM 5o0xZOX0XcRO5jLL1HdN49tN8jklfTEOX1waTh12I9R67iVT0W2kJgctdL5TRAP5 pXMbMkYJzQucczFSJZ5367MqAqL7c4170IhOxK/Tjf6MiCPGbBycPEMzZrau0nv/ iGojBTDl6aS3LHR95pIXl+1Crti9pjmDo87B1W1Jf9dsE6JRXT9ptm5S6uXG/0Yx AuZupDodj4bVc/5vkSC+lJrYrTPIzKJ70i8OTI+cn4+n7/BIV8AXnm0bL110ZhMB GR9stfguZjhIPTKEb7w6OQKGw18XjcRxyikpyrkBDQRNe2pOAQgAtzMfVkjxIKLd A8zzle5hWaUv5Fy2GGStEy0YojE7s5JpypEx8A5rvfHkv9+dtJmWPKusCzkl06nO v8ldgILlGETZpIV7zVrWvzVJwLK4tjOxHwQL+xTJSilqRxBMVvSk6HTbrKi7+cPq J3yds4645+Y4muoeDwtKH7XY2OJ0vMqD5co9ndyvP/IRNS8HtT/ZGViCG4k8BJ5w skHyVJaijfoCnu+NMxeD6Di471mErbgd5F/srFTdzZkwXxQUZHTFL6EbaVXJGSda SepLdjrham7+kSTRPTYukQHkPJI5+gKEVtyN2n6CoDE42n8TePVbSHcUp9NRPmly Yr8N8xB0AwARAQABiQElBBgBAgAPBQJNe2pOAhsMBQkJZgGAAAoJEPDom3VFNJN+ TPQH/i1/fmAyDVUMhS1Ouo9/nyGQZgiDyNbeyx5P0ETASskCvl52dyMlzZq+bChA e+qDBPt9SaqU4ycgkg6d0AJ0XBqRARhEiKJjDwVsEWrF0F3s9+SQbpMSsNr7NsLK c1jiwCpwuXKLTiRapdZsDK33seMFQ85o/RVLTahXhNsca7meD32IfUZ1bQSuT2aC WiIXaCqW4hxXrcD8y+FBfmT96V4+ItDcDJK50GninaQ7kU1PQxilPIV9/THVZA5z lFRkOdWWJN8kjdH7tZQuuyosZmcUJY4bkqL4BMJpIGEkn8NctzvsEPwjfplzBfRl 3peGDWBU+N7gIF0kDm/+WG/2HDE=3D =3DGv01 -----END PGP PUBLIC KEY BLOCK----- --------------070404010704060306010903-- --SLjSbUr8BMvRiuKeWbKd5X98MxaU57f59 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTay5HAAoJEPDom3VFNJN+JKgH/1FGKEtZiJ5kNYKxL+VuSMwb uZOrJ1jLIJfs3VPeCywN/JKk+Ucpkrf2tGPfzlFkfylAvY7pwB5avEN+eqF/UMzz /cVg+locbg7XQJtXL4bmpHfwsWus7HFhwB/ta2uNwJjCwVOcGHd6vz617MuR0rnT 2SQjOzrtCUfefS0W77tRgBjNPPOuE1mjmxgazCqEtpONrFXkkt2v2v3UtbZeaEJa 2ZYbkkJoUEfKPJYZHVRRa/0PQgNFRNxJOfjU+ONuPmzsG0DhiDm4XdApbk8Avuiw jkgVJS+W/k2PInXHMLWHuAahWo5QvNZVoLGF0CNAWE2+l1SuxmE78OOXQqGuBYQ= =5+eh -----END PGP SIGNATURE----- --SLjSbUr8BMvRiuKeWbKd5X98MxaU57f59-- From nobody Thu May 8 02:15:37 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB88A1A026A for ; Thu, 8 May 2014 02:15:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2EsrUs_z3CVx for ; Thu, 8 May 2014 02:15:32 -0700 (PDT) Received: from r2-chicago.webserversystems.com (r2-chicago.webserversystems.com [173.236.101.58]) by ietfa.amsl.com (Postfix) with ESMTP id C75D51A023B for ; Thu, 8 May 2014 02:15:32 -0700 (PDT) Received: from pool-71-175-4-197.phlapa.fios.verizon.net ([71.175.4.197]:3996 helo=[192.168.1.12]) by r2-chicago.webserversystems.com with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WiKQJ-0000hY-Gt for rmcat@ietf.org; Thu, 08 May 2014 04:15:27 -0500 Message-ID: <536B4AE5.1020000@jesup.org> Date: Thu, 08 May 2014 05:14:13 -0400 From: Randell Jesup User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: rmcat@ietf.org References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> In-Reply-To: <536B2E47.7090005@aalto.fi> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - r2-chicago.webserversystems.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - jesup.org X-Get-Message-Sender-Via: r2-chicago.webserversystems.com: authenticated_id: randell@jesup.org Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/5JLGwDLUNcb7hoCe2n-_PsoJR-A Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 09:15:35 -0000 On 5/8/2014 3:12 AM, Lennart Schulte wrote: > Hi, > while measuring 3G and 4G networks I also saw reordering: 35% of all > connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of > these reordering events the reordering extent was at least 3 packets > large (which caused TCP to spuriously enter fast recovery). 35% doesn't seem surprising.... I presume reordering can/will happen; my interest would be amount of reordering, and depth/distribution of reordering. (More than 30-50ms of reordering effectively == loss from an RTP standpoint, generally.) And unless it affects the congestion algorithm (which is of concern here), less than jitter depth reordering is pretty much "who cares" (maybe not quite, but close). Also, since the distribution is very uneven, are there networks (3G, 4G/LTE, WiFi (of various flavors) where this is dramatically more common? Thanks! -- Randell Jesup -- rjesup a t mozilla d o t com From nobody Thu May 8 05:10:34 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CF0E1A04F8 for ; Thu, 8 May 2014 05:10:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.901 X-Spam-Level: X-Spam-Status: No, score=-2.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nI3oc4tOohfH for ; Thu, 8 May 2014 05:10:29 -0700 (PDT) Received: from nasse.dc.kau.se (smtp.kau.se [193.10.220.39]) by ietfa.amsl.com (Postfix) with ESMTP id 9FB8F1A045F for ; Thu, 8 May 2014 05:10:29 -0700 (PDT) Date: Thu, 8 May 2014 14:10:19 +0200 From: Stefan Alfredsson To: rmcat@ietf.org Message-ID: <20140508121019.GB32203@sa-pc.cs.kau.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <536B2E47.7090005@aalto.fi> User-Agent: Mutt/1.5.21 (2010-09-15) Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/JDPHDF7VGBZRNKOuuampWQy_5uQ Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 12:10:33 -0000 Hi, * Quoting Lennart Schulte [08 May-14 10:12]: > while measuring 3G and 4G networks I also saw reordering: 35% of all > connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of > these reordering events the reordering extent was at least 3 packets > large (which caused TCP to spuriously enter fast recovery). 35% sounds very high. Have you done detailed studies to see if there is some common property of flows with reordering? We've done a lot of 3G/4G live measurements, and in one case we saw a high degree of reordering and TCP resets of many connections, especially for shorter flows (less than IW10). After investigation, the cause was found to be prioritization of small packets by the operator. It only affected one of four operators in the measurement, all else equal. One possible explanation can be that the operator wanted to prioritize e.g. DNS requests or VoIP packets over bulk data in times of queueing, but not excluding FIN packets lead to a very inefficient scenario: Imagine a short flow, fitting in the initial window: 1. TCP connection is setup 2. All data is sent in one window (e.g. 5 kb or such) 3. Sender closes socket, sends FIN. 4. Receiver should acknowledge data, and fully close. But since small packets were prioritized over the full sized data segments and all packets were sent back-to-back, the receiver *gets the FIN before the data segments*. Oops. The receiver thus SACK's the "missing" (=reordered) data. Reordered packets then arrive directly afterwards, gets acknowledged, final FIN is sent, and the connection is fully closed at the receiver. Meanwhile, the sender gets the SACK, retransmits the reordered segments, then gets the final FIN (presumably no uplink reordering?; but not investigated yet). Receiver now gets the retransmitted segments which arrive at the now closed socket, and RST packets go back. This reordering was also seen at the tail of longer flows, although to a much lesser extent. I can send packet captures if anyone wants a closer look. Regards, Stefan -- Stefan Alfredsson, PhD Tel: +46 (0) 54 700 1668 Datavetenskap Kontor: 21F-425 (Hus Vanern) Karlstads universitet PGP 0xB19B4B16 SE-651 88 Karlstad http://www.cs.kau.se/stefalfr/ From nobody Thu May 8 05:49:21 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A429F1A04B9 for ; Thu, 8 May 2014 05:49:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wugFLII6L6lw for ; Thu, 8 May 2014 05:49:17 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id 53CE21A063A for ; Thu, 8 May 2014 05:49:15 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 605EBA2; Thu, 8 May 2014 14:49:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1399553349; bh=MMRPAPP7qLsbu5Vh/23Iwhz1sDoK1c+xGniBAukuvfQ=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=XYXj8c4bwBQmIjHA3NdL0fePcgyM4kuBFWpa0UDfYC9YE1Z2gAU4BTiWoRZZZ+ZuE Iq+ngxE9a1QhdcnWh817rsEuD+phSisdTKEgQaRSBISfVBTspibBGUhYrF4TfvPyuj 0BXTNxWfCVyUKWGxYq1JEoraiEW4oktJ0/eDFGHQ= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 5669CA1; Thu, 8 May 2014 14:49:09 +0200 (CEST) Date: Thu, 8 May 2014 14:49:09 +0200 (CEST) From: Mikael Abrahamsson To: Randell Jesup In-Reply-To: <536B4AE5.1020000@jesup.org> Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/INwahvcVY8qgkbpPa0wGOXqpYKo Cc: rmcat@ietf.org Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 12:49:19 -0000 On Thu, 8 May 2014, Randell Jesup wrote: > On 5/8/2014 3:12 AM, Lennart Schulte wrote: >> Hi, >> while measuring 3G and 4G networks I also saw reordering: 35% of all >> connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of >> these reordering events the reordering extent was at least 3 packets >> large (which caused TCP to spuriously enter fast recovery). > > 35% doesn't seem surprising.... I presume reordering can/will happen; my > interest would be amount of reordering, and depth/distribution of reordering. > (More than 30-50ms of reordering effectively == loss from an RTP standpoint, > generally.) And unless it affects the congestion algorithm (which is of > concern here), less than jitter depth reordering is pretty much "who cares" > (maybe not quite, but close). > > Also, since the distribution is very uneven, are there networks (3G, 4G/LTE, > WiFi (of various flavors) where this is dramatically more common? I have historically proposed that applications should log these parameters as they happen, and provide this information in aggregate or per-session basis to present to the user. This is applicable to RTP and it's applicable to TCP (and more). As soon as you have sequence numbers you can detect this and you can count it. So while it's not core to this discussion, where is the best place to gain traction for suggestion to count and aggregate data on: Number of packets lost Number of packets arrived too late (PDV buffer miss) Number of packets that arrived out of order Number of packets that arrived out of order and caused PDV buffer miss If applications actually logged this to operators and users, they might be able to understand the problems and have a better understanding of the customers quality of experience, and also the user would have better data to do a fault report to the operator with. Basically, packet re-ordering shouldn't happen. Networks that cause packets to be re-ordered should know that this causes bad customer experience so they can stop. A lot of the time this is done out of ignorance. -- Mikael Abrahamsson email: swmike@swm.pp.se From nobody Thu May 8 08:01:31 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EF2681A0052 for ; Thu, 8 May 2014 08:01:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.277 X-Spam-Level: X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3AV8QHV9boFB for ; Thu, 8 May 2014 08:01:21 -0700 (PDT) Received: from mail-ob0-x230.google.com (mail-ob0-x230.google.com [IPv6:2607:f8b0:4003:c01::230]) by ietfa.amsl.com (Postfix) with ESMTP id 401CA1A004E for ; Thu, 8 May 2014 08:01:21 -0700 (PDT) Received: by mail-ob0-f176.google.com with SMTP id wo20so3132769obc.7 for ; Thu, 08 May 2014 08:01:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/k9Ki2B7wFQWqvUyCQnVtzJC2hi3YMnAxp5Zw7kkKrQ=; b=teTc2sZ2wF3eGFdf0fiVygqzG5q/6f6qwZpp/YysVSm7CZMdy8WgPtv1vypDpoV011 kOgLvptdPaibAoF0UOPfmJhoLhB+xldB7/IYu3tC5JB2dw61DY80sh88DyT4+c2eFE8Z usgEBI/bHDmdZ+/+BrZsZuwJ+GjQkwT8xu3wtVaHpodozr5fB9N2BlhPLX1sZ26Fer2c pUPBYUhZ6cNLFuRtW3u1mTyvMltucM725RRJY/5QXvbKvk4HGvalMuY+h8YbDg8LF79U v1osbMLoVj2R7H2eJCCg+XYIw0ELKLuDgeNC5QikcPMYR1VK1bims9jZFd7HmHQ8JwRo s4xg== MIME-Version: 1.0 X-Received: by 10.60.83.161 with SMTP id r1mr5433337oey.46.1399561276692; Thu, 08 May 2014 08:01:16 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.76.73.100 with HTTP; Thu, 8 May 2014 08:01:16 -0700 (PDT) In-Reply-To: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> Date: Thu, 8 May 2014 11:01:16 -0400 X-Google-Sender-Auth: AjbsbHZMgQarjeHZ8v-OFVkhlxM Message-ID: From: Jim Gettys To: Mikael Abrahamsson Content-Type: multipart/alternative; boundary=089e0112c460bd224d04f8e4c05a Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/LnyVYIWZS7ZbFAShsHRXQ-lBwOE Cc: Randell Jesup , "rmcat@ietf.org" Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 15:01:27 -0000 --089e0112c460bd224d04f8e4c05a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu, May 8, 2014 at 8:49 AM, Mikael Abrahamsson wrote= : > On Thu, 8 May 2014, Randell Jesup wrote: > > On 5/8/2014 3:12 AM, Lennart Schulte wrote: >> >>> Hi, >>> while measuring 3G and 4G networks I also saw reordering: 35% of all >>> connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of >>> these reordering events the reordering extent was at least 3 packets >>> large (which caused TCP to spuriously enter fast recovery). >>> >> >> 35% doesn't seem surprising.... I presume reordering can/will happen; my >> interest would be amount of reordering, and depth/distribution of >> reordering. (More than 30-50ms of reordering effectively =3D=3D loss fro= m an >> RTP standpoint, generally.) And unless it affects the congestion algori= thm >> (which is of concern here), less than jitter depth reordering is pretty >> much "who cares" (maybe not quite, but close). >> >> Also, since the distribution is very uneven, are there networks (3G, >> 4G/LTE, WiFi (of various flavors) where this is dramatically more common= ? >> > > I have historically proposed that applications should log these parameter= s > as they happen, and provide this information in aggregate or per-session > basis to present to the user. This is applicable to RTP and it's applicab= le > to TCP (and more). As soon as you have sequence numbers you can detect th= is > and you can count it. > > So while it's not core to this discussion, where is the best place to gai= n > traction for suggestion to count and aggregate data on: > > Number of packets lost > Number of packets arrived too late (PDV buffer miss) > Number of packets that arrived out of order > Number of packets that arrived out of order and caused PDV buffer miss > > If applications actually logged this to operators and users, they might b= e > able to understand the problems and have a better understanding of the > customers quality of experience, and also the user would have better data > to do a fault report to the operator with. > =E2=80=8BIt isn't just the cellular operators. It's often going to be your= home network.=E2=80=8B > > Basically, packet re-ordering shouldn't happen. Networks that cause > packets to be re-ordered should know that this causes bad customer > experience so they can stop. A lot of the time this is done out of > ignorance. =E2=80=8BThis is a naive point of view. First, current TCP implementations= are overly sensitive to reordering (though I believe there has been recent work in Linux to help this a lot, but it will take a long time before it is deployed widely enough to help). Also, remember today at least one hop of your path is likely to be over a wireless link (lossy, error prone), whether 802.11 or cellular. This has had a serious unintended follow-on effect.=E2=80=8B Since the consequences to TCP performance have been so bad, this has encouraged driver implementers on wireless (of all sorts) to put buffering in the receive path (needed anyway for aggregation on 802.11n, for example). In these situations, the receiver may "freeze" forwarding already received packets hoping that damaged packets get repaired quickly to avoid reordering. But this introduces delays before the packets get released. One of the "interesting" bufferbloat discoveries in the Linux WiFi stack, for example, was discovering that some drivers would try to repair the damaged packets effectively infinitely, causing spikes up to 1 second. And, btw, this kind of problem is rampant in the current generation of home routers. And what do you do if you can't repair the packet quickly (e.g. inside of a single transmit-op)? The receiver may decide to then forward the packets in the aggregate you have, but later (on a subsequent transmit opportunity) the missing packet(s) may be transmitted by the host, thereby causing out of order delivery. But I agree completely that if no information is provided to users the chance of getting anything fixed anywhere is zero. - Jim > > -- > Mikael Abrahamsson email: swmike@swm.pp.se > > --089e0112c460bd224d04f8e4c05a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Thu= , May 8, 2014 at 8:49 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On T= hu, 8 May 2014, Randell Jesup wrote:

On 5/8/2014 3:12 AM, Lennart Schulte wrote:
Hi,
while measuring 3G and 4G networks I also saw reordering: 35% of all
connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of
these reordering events the reordering extent was at least 3 packets
large (which caused TCP to spuriously enter fast recovery).

35% doesn't seem surprising.... I presume reordering can/will happen; m= y interest would be amount of reordering, and depth/distribution of reorder= ing. (More than 30-50ms of reordering effectively =3D=3D loss from an RTP s= tandpoint, generally.) =C2=A0And unless it affects the congestion algorithm= (which is of concern here), less than jitter depth reordering is pretty mu= ch "who cares" (maybe not quite, but close).

Also, since the distribution is very uneven, are there networks (3G, 4G/LTE= , WiFi (of various flavors) where this is dramatically more common?

I have historically proposed that applications should log these parameters = as they happen, and provide this information in aggregate or per-session ba= sis to present to the user. This is applicable to RTP and it's applicab= le to TCP (and more). As soon as you have sequence numbers you can detect t= his and you can count it.

So while it's not core to this discussion, where is the best place to g= ain traction for suggestion to count and aggregate data on:

Number of packets lost
Number of packets arrived too late (PDV buffer miss)
Number of packets that arrived out of order
Number of packets that arrived out of order and caused PDV buffer miss

If applications actually logged this to operators and users, they might be = able to understand the problems and have a better understanding of the cust= omers quality of experience, and also the user would have better data to do= a fault report to the operator with.

=E2=80=8BIt isn't just the cellular operators. =C2=A0It's o= ften going to be your home network.=E2=80=8B

Basically, packet re-ordering shouldn't happen. Networks that cause pac= kets to be re-ordered should know that this causes bad customer experience = so they can stop. A lot of the time this is done out of ignorance.

=E2= =80=8BThis is a naive point of view. =C2=A0First, current TCP implementatio= ns are overly sensitive to reordering (though I believe there has been rece= nt work in Linux to help this a lot, but it will take a long time before it= is deployed widely enough to help). =C2=A0

Also, remember today at least = one hop of your path is likely to be over a wireless link (lossy, error pro= ne), whether 802.11 or cellular.

This has had a serious uninten= ded follow-on effect.=E2=80=8B =C2=A0Since the consequences to TCP performa= nce have been so bad, this has encouraged driver implementers on wireless (= of all sorts) to put buffering in the receive path (needed anyway for aggre= gation on 802.11n, for example). =C2=A0In these situations, the receiver ma= y "freeze" forwarding already received packets hoping that damage= d packets get repaired quickly to avoid reordering. =C2=A0But this introduc= es delays before the packets get released. =C2=A0One of the "interesti= ng" bufferbloat discoveries in the Linux WiFi stack, for example, was = discovering that some drivers would try to repair the damaged packets effec= tively infinitely, causing spikes up to 1 second. =C2=A0And, btw, this kind= of problem is rampant in the current generation of home routers.

And what do you do if you can&= #39;t repair the packet quickly (e.g. inside of a single transmit-op)? =C2= =A0The receiver may decide to then forward the packets in the aggregate you= have, but later (on a subsequent transmit opportunity) the missing packet(= s) may be transmitted by the host, thereby causing out of order delivery.

But I agree completely that if= no information is provided to users the chance of getting anything fixed a= nywhere is zero.
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Jim



--
Mikael Abrahamsson =C2=A0 =C2=A0email: swmike@swm.pp.se


--089e0112c460bd224d04f8e4c05a-- From nobody Thu May 8 08:33:52 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 496711A0053 for ; Thu, 8 May 2014 08:33:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AUFyZ8eU7NBt for ; Thu, 8 May 2014 08:33:41 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id 62D911A004E for ; Thu, 8 May 2014 08:33:41 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 4C0DCA2; Thu, 8 May 2014 17:33:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1399563216; bh=QZu7T0/oEjromQQ6jhm8ndCiauPlfBhOKGCL1i25p0o=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=l0Qgr9tLBXY5mFZ7+0/yORzY5wtORAMj31Jsa5LLrMGM4MXVlHicOY0mEuqHn3fuU Kj0MuSVRpayPG3F0kXnfWh2N9AR6vd+rqexZFhHmrlRKm4qSNrsqa9aLEV8ceygWZm v3zfmoCdRcNxfsZSaZzkAaa3wi+TJaFI0p7vbtr4= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 486A5A1; Thu, 8 May 2014 17:33:36 +0200 (CEST) Date: Thu, 8 May 2014 17:33:36 +0200 (CEST) From: Mikael Abrahamsson To: Jim Gettys In-Reply-To: Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-137064504-1818948919-1399563216=:29282" Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/NVYt5q6QEs07Twg6kqcpmkxiutI Cc: Randell Jesup , "rmcat@ietf.org" Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 15:33:48 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---137064504-1818948919-1399563216=:29282 Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT On Thu, 8 May 2014, Jim Gettys wrote: > Also, remember today at least one hop of your path is likely to be over a > wireless link (lossy, error prone), whether 802.11 or cellular. 802.11 and cellular both have their own L2 re-transmit mechanisms, so I wouldn't call them lossy. > This has had a serious unintended follow-on effect.​ Since the > consequences to TCP performance have been so bad, this has encouraged > driver implementers on wireless (of all sorts) to put buffering in the > receive path (needed anyway for aggregation on 802.11n, for example). In > these situations, the receiver may "freeze" forwarding already received > packets hoping that damaged packets get repaired quickly to avoid > reordering. But this introduces delays before the packets get released. > One of the "interesting" bufferbloat discoveries in the Linux WiFi stack, > for example, was discovering that some drivers would try to repair the > damaged packets effectively infinitely, causing spikes up to 1 second. > And, btw, this kind of problem is rampant in the current generation of > home routers. Absolutely, these link layers often try too hard to avoid loss, which causes the problems you're describing. > But I agree completely that if no information is provided to users the > chance of getting anything fixed anywhere is zero. I pitched this to the TCP group 5 years back or so, but unfortunately there were no takers. People seem to want to do test traffic instead of measuring performance for real traffic. -- Mikael Abrahamsson email: swmike@swm.pp.se ---137064504-1818948919-1399563216=:29282-- From nobody Thu May 8 09:10:00 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EE6DD1A005E for ; Thu, 8 May 2014 09:09:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.886 X-Spam-Level: X-Spam-Status: No, score=-1.886 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gyVQVEWELEc5 for ; Thu, 8 May 2014 09:09:56 -0700 (PDT) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 725621A005A for ; Thu, 8 May 2014 09:09:56 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.97,1010,1389772800"; d="asc'?scan'208,217";a="121959390" Received: from vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) by mx11-out.netapp.com with ESMTP; 08 May 2014 09:09:51 -0700 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.187]) by vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) with mapi id 14.03.0123.003; Thu, 8 May 2014 09:09:51 -0700 From: "Zimmermann, Alexander" To: Jim Gettys Thread-Topic: [rmcat] Packet reordering Thread-Index: AQHPaozbGfHPc5r1Uk2yneWZzvq9QZs23ACAgAA8DYCAACTqAIAAEtuA Date: Thu, 8 May 2014 16:09:30 +0000 Message-ID: <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.30] Content-Type: multipart/signed; boundary="Apple-Mail=_229A8E8C-9A8D-4434-90A3-B625912648D3"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/OCXoD-vNSJmcItNyZU1onUaKJF8 Cc: Randell Jesup , "rmcat@ietf.org" , Mikael Abrahamsson Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 16:09:59 -0000 --Apple-Mail=_229A8E8C-9A8D-4434-90A3-B625912648D3 Content-Type: multipart/alternative; boundary="Apple-Mail=_5643961E-EA6A-44A7-9FF9-BBEDFDAE3676" --Apple-Mail=_5643961E-EA6A-44A7-9FF9-BBEDFDAE3676 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Jim, Am 08.05.2014 um 17:01 schrieb Jim Gettys : >=20 >=20 >=20 > On Thu, May 8, 2014 at 8:49 AM, Mikael Abrahamsson = wrote: > On Thu, 8 May 2014, Randell Jesup wrote: >=20 > On 5/8/2014 3:12 AM, Lennart Schulte wrote: > Hi, > while measuring 3G and 4G networks I also saw reordering: 35% of all > connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of > these reordering events the reordering extent was at least 3 packets > large (which caused TCP to spuriously enter fast recovery). >=20 > 35% doesn't seem surprising.... I presume reordering can/will happen; = my interest would be amount of reordering, and depth/distribution of = reordering. (More than 30-50ms of reordering effectively =3D=3D loss = from an RTP standpoint, generally.) And unless it affects the = congestion algorithm (which is of concern here), less than jitter depth = reordering is pretty much "who cares" (maybe not quite, but close). >=20 > Also, since the distribution is very uneven, are there networks (3G, = 4G/LTE, WiFi (of various flavors) where this is dramatically more = common? >=20 > I have historically proposed that applications should log these = parameters as they happen, and provide this information in aggregate or = per-session basis to present to the user. This is applicable to RTP and = it's applicable to TCP (and more). As soon as you have sequence numbers = you can detect this and you can count it. >=20 > So while it's not core to this discussion, where is the best place to = gain traction for suggestion to count and aggregate data on: >=20 > Number of packets lost > Number of packets arrived too late (PDV buffer miss) > Number of packets that arrived out of order > Number of packets that arrived out of order and caused PDV buffer miss >=20 > If applications actually logged this to operators and users, they = might be able to understand the problems and have a better understanding = of the customers quality of experience, and also the user would have = better data to do a fault report to the operator with. >=20 > =E2=80=8BIt isn't just the cellular operators. It's often going to be = your home network.=E2=80=8B >=20 > Basically, packet re-ordering shouldn't happen. Networks that cause = packets to be re-ordered should know that this causes bad customer = experience so they can stop. A lot of the time this is done out of = ignorance. >=20 > =E2=80=8BThis is a naive point of view. First, current TCP = implementations are overly sensitive to reordering (though I believe = there has been recent work in Linux to help this a lot, but it will take = a long time before it is deployed widely enough to help). =20 Only as a side note=E2=80=A6 Actually, there is a TCP reordering robustness in Linux for years. = However, it has weaknesses and performance not very well. See our TCPM presentation: = http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf=20 Alex >=20 > Also, remember today at least one hop of your path is likely to be = over a wireless link (lossy, error prone), whether 802.11 or cellular. >=20 > This has had a serious unintended follow-on effect.=E2=80=8B Since = the consequences to TCP performance have been so bad, this has = encouraged driver implementers on wireless (of all sorts) to put = buffering in the receive path (needed anyway for aggregation on 802.11n, = for example). In these situations, the receiver may "freeze" forwarding = already received packets hoping that damaged packets get repaired = quickly to avoid reordering. But this introduces delays before the = packets get released. One of the "interesting" bufferbloat discoveries = in the Linux WiFi stack, for example, was discovering that some drivers = would try to repair the damaged packets effectively infinitely, causing = spikes up to 1 second. And, btw, this kind of problem is rampant in the = current generation of home routers. >=20 > And what do you do if you can't repair the packet quickly (e.g. inside = of a single transmit-op)? The receiver may decide to then forward the = packets in the aggregate you have, but later (on a subsequent transmit = opportunity) the missing packet(s) may be transmitted by the host, = thereby causing out of order delivery. >=20 > But I agree completely that if no information is provided to users the = chance of getting anything fixed anywhere is zero. > - Jim >=20 >=20 >=20 > --=20 > Mikael Abrahamsson email: swmike@swm.pp.se >=20 >=20 --Apple-Mail=_5643961E-EA6A-44A7-9FF9-BBEDFDAE3676 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi = Jim,

Am 08.05.2014 um 17:01 schrieb Jim Gettys <jg@freedesktop.org>:




On Thu, May 8, = 2014 at 8:49 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On Thu, 8 May 2014, Randell Jesup = wrote:

On 5/8/2014 3:12 AM, Lennart Schulte wrote:
Hi,
while measuring 3G and 4G networks I also saw reordering: 35% of all
connections (12392 in 3G and 676 in 4G) had reordering. In about 2% = of
these reordering events the reordering extent was at least 3 packets
large (which caused TCP to spuriously enter fast recovery).

35% doesn't seem surprising.... I presume reordering can/will happen; my = interest would be amount of reordering, and depth/distribution of = reordering. (More than 30-50ms of reordering effectively =3D=3D loss = from an RTP standpoint, generally.)  And unless it affects the = congestion algorithm (which is of concern here), less than jitter depth = reordering is pretty much "who cares" (maybe not quite, but close).

Also, since the distribution is very uneven, are there networks (3G, = 4G/LTE, WiFi (of various flavors) where this is dramatically more = common?

I have historically proposed that applications should log these = parameters as they happen, and provide this information in aggregate or = per-session basis to present to the user. This is applicable to RTP and = it's applicable to TCP (and more). As soon as you have sequence numbers = you can detect this and you can count it.

So while it's not core to this discussion, where is the best place to = gain traction for suggestion to count and aggregate data on:

Number of packets lost
Number of packets arrived too late (PDV buffer miss)
Number of packets that arrived out of order
Number of packets that arrived out of order and caused PDV buffer = miss

If applications actually logged this to operators and users, they might = be able to understand the problems and have a better understanding of = the customers quality of experience, and also the user would have better = data to do a fault report to the operator with.

=E2=80=8BIt isn't just the cellular operators. =  It's often going to be your home network.=E2=80=8B

Basically, packet re-ordering shouldn't happen. Networks that cause = packets to be re-ordered should know that this causes bad customer = experience so they can stop. A lot of the time this is done out of = ignorance.

=E2=80=8BThis is a naive point of view. =  First, current TCP implementations are overly sensitive to = reordering (though I believe there has been recent work in Linux to help = this a lot, but it will take a long time before it is deployed widely = enough to help). =  

Only as a = side note=E2=80=A6
Actually, there is a TCP reordering = robustness in Linux for years. However, it has weaknesses and = performance not very well.

Alex


Also, remember today = at least one hop of your path is likely to be over a wireless link = (lossy, error prone), whether 802.11 or cellular.

This has had a serious = unintended follow-on effect.=E2=80=8B  Since the consequences to = TCP performance have been so bad, this has encouraged driver = implementers on wireless (of all sorts) to put buffering in the receive = path (needed anyway for aggregation on 802.11n, for example).  In = these situations, the receiver may "freeze" forwarding already received = packets hoping that damaged packets get repaired quickly to avoid = reordering.  But this introduces delays before the packets get = released.  One of the "interesting" bufferbloat discoveries in the = Linux WiFi stack, for example, was discovering that some drivers would = try to repair the damaged packets effectively infinitely, causing spikes = up to 1 second.  And, btw, this kind of problem is rampant in the = current generation of home routers.

And what do you do if = you can't repair the packet quickly (e.g. inside of a single = transmit-op)?  The receiver may decide to then forward the packets = in the aggregate you have, but later (on a subsequent transmit = opportunity) the missing packet(s) may be transmitted by the host, = thereby causing out of order delivery.

But I agree completely = that if no information is provided to users the chance of getting = anything fixed anywhere is zero.
    =                     =                     - = Jim



--
Mikael Abrahamsson    email: swmike@swm.pp.se



= --Apple-Mail=_5643961E-EA6A-44A7-9FF9-BBEDFDAE3676-- --Apple-Mail=_229A8E8C-9A8D-4434-90A3-B625912648D3 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlNrrA0ACgkQdyiq39b9uS5CxACaA9IDAJcU7Tg4ny3QPJG9J7B1 BIUAmgOO+d8uI3NLe6Q5cSle1Eupqsry =J5T/ -----END PGP SIGNATURE----- --Apple-Mail=_229A8E8C-9A8D-4434-90A3-B625912648D3-- From nobody Thu May 8 09:52:16 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4B7481A0066 for ; Thu, 8 May 2014 09:52:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cOiVl3cP-1nJ for ; Thu, 8 May 2014 09:52:10 -0700 (PDT) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) by ietfa.amsl.com (Postfix) with ESMTP id 403C11A0052 for ; Thu, 8 May 2014 09:52:10 -0700 (PDT) Received: by mail-wi0-f172.google.com with SMTP id hi2so18696wib.11 for ; Thu, 08 May 2014 09:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=nr5cvRGnvDyLxmkTbJ8SOhriuR9CowYCFsMg4ZT3FIo=; b=gJDfaKBb5BADe7bztSbUXgJR3RXSKA6Fcp6uebUUS5K7i1DmGJ1lIbl3vP6KPF8SXO 5swVpB3vu7Sa/4hffBw2S62S5UzcQ0FGAKmlPTG7D4Tf52l14ndciLnX3xEufPsVlvX7 Mc9kV48jwj1uMDiNFCDZJrfuBIggRQYjUvh5zGI736LFRbT/4v9hIzPLhvn5MZ42qJnI AJLA+gj1kxhwjzSBQdoFUtFOKxQr1LTOCDkaDwIDnc0QQxR1/c4aeUhEpdyDbsax+LPB GdIzR0Y2rEMNOrOxXfUF73cofSsNhKuHCUduRVa0C/UWHL1iv8HOr8NbfzQx4mkfFsMU rTqQ== MIME-Version: 1.0 X-Received: by 10.180.76.166 with SMTP id l6mr4413875wiw.17.1399567925290; Thu, 08 May 2014 09:52:05 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Thu, 8 May 2014 09:52:05 -0700 (PDT) In-Reply-To: <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> Date: Thu, 8 May 2014 09:52:05 -0700 Message-ID: From: Dave Taht To: "Zimmermann, Alexander" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/8_vcEmJ-tzeu0cMzcKXd-pokmF0 Cc: Randell Jesup , "rmcat@ietf.org" , Jim Gettys , Mikael Abrahamsson Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 16:52:13 -0000 On Thu, May 8, 2014 at 9:09 AM, Zimmermann, Alexander wrote: > Hi Jim, > > Am 08.05.2014 um 17:01 schrieb Jim Gettys : > > > > > On Thu, May 8, 2014 at 8:49 AM, Mikael Abrahamsson wro= te: >> >> On Thu, 8 May 2014, Randell Jesup wrote: >> >>> On 5/8/2014 3:12 AM, Lennart Schulte wrote: >>>> >>>> Hi, >>>> while measuring 3G and 4G networks I also saw reordering: 35% of all >>>> connections (12392 in 3G and 676 in 4G) had reordering. In about 2% of >>>> these reordering events the reordering extent was at least 3 packets >>>> large (which caused TCP to spuriously enter fast recovery). >>> >>> >>> 35% doesn't seem surprising.... I presume reordering can/will happen; m= y >>> interest would be amount of reordering, and depth/distribution of >>> reordering. (More than 30-50ms of reordering effectively =3D=3D loss fr= om an RTP >>> standpoint, generally.) And unless it affects the congestion algorithm >>> (which is of concern here), less than jitter depth reordering is pretty= much >>> "who cares" (maybe not quite, but close). >>> >>> Also, since the distribution is very uneven, are there networks (3G, >>> 4G/LTE, WiFi (of various flavors) where this is dramatically more commo= n? >> >> >> I have historically proposed that applications should log these paramete= rs >> as they happen, and provide this information in aggregate or per-session >> basis to present to the user. This is applicable to RTP and it's applica= ble >> to TCP (and more). As soon as you have sequence numbers you can detect t= his >> and you can count it. >> >> So while it's not core to this discussion, where is the best place to ga= in >> traction for suggestion to count and aggregate data on: >> >> Number of packets lost >> Number of packets arrived too late (PDV buffer miss) >> Number of packets that arrived out of order >> Number of packets that arrived out of order and caused PDV buffer miss >> >> If applications actually logged this to operators and users, they might = be >> able to understand the problems and have a better understanding of the >> customers quality of experience, and also the user would have better dat= a to >> do a fault report to the operator with. > > > It isn't just the cellular operators. It's often going to be your home > network. >> >> >> Basically, packet re-ordering shouldn't happen. Networks that cause >> packets to be re-ordered should know that this causes bad customer >> experience so they can stop. A lot of the time this is done out of >> ignorance. > > > This is a naive point of view. First, current TCP implementations are > overly sensitive to reordering (though I believe there has been recent wo= rk > in Linux to help this a lot, but it will take a long time before it is > deployed widely enough to help). And things like mosh & rtp mostly care about the most recent packets. If packets arrive out of order, the older packets are dropped anyway. Better that they not be delivered... and ledbat/quic don't mind packets arriving out of order. > > Only as a side note=E2=80=A6 > Actually, there is a TCP reordering robustness in Linux for years. Howeve= r, > it has weaknesses and performance not very well. > See our TCPM presentation: > http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf What version of linux tcp was this taken against? The entire stack has evolved considerably between linux 3.3 and linux 3.14. I am aware of a bunch of reordering fixes that went in fairly recently, and there's some interesting side effects from the new sch_fq + pacing stuff that I think make re-ordering events less likely. Also I note there was a fairly major bug in rx steering for that would result in ooo packets, induced and fixed back in the 3.8-3.11(?) timeframe, which might be relevant to an earlier comment on this thread... > > Alex > > > Also, remember today at least one hop of your path is likely to be over a > wireless link (lossy, error prone), whether 802.11 or cellular. > > This has had a serious unintended follow-on effect. Since the consequenc= es > to TCP performance have been so bad, this has encouraged driver implement= ers > on wireless (of all sorts) to put buffering in the receive path (needed > anyway for aggregation on 802.11n, for example). In these situations, th= e > receiver may "freeze" forwarding already received packets hoping that > damaged packets get repaired quickly to avoid reordering. But this > introduces delays before the packets get released. You have several separate, interrelated problems here. Some drivers will only release the entire aggregate of packets, in order, and do a lot of retries until they get them all, come hell or high water. Others will deliver a later aggregate if it arrives unscathed while retrying an earlier aggregate. Few will deliver "good" packets from an aggregate while retrying, and none I know of do anything other than give up after a long time out, rather than doing something saner like looking at flows and trying to ensure at least one packet is delivered from a flow, and dropping the rest when there are major problems. > One of the "interesting" > bufferbloat discoveries in the Linux WiFi stack, for example, was > discovering that some drivers would try to repair the damaged packets > effectively infinitely, causing spikes up to 1 second. And, btw, this ki= nd > of problem is rampant in the current generation of home routers. seen 36+ seconds. http://www.bufferbloat.net/issues/216 a year after we fixed this bug, broadcom found and fixed the same thing in one of their chipsets. Millions of devices in the field flooding the airwaves with pointless retries, with missing firmware updates. > And what do you do if you can't repair the packet quickly (e.g. inside of= a > single transmit-op)? The receiver may decide to then forward the packets= in > the aggregate you have, but later (on a subsequent transmit opportunity) = the > missing packet(s) may be transmitted by the host, thereby causing out of > order delivery. > > But I agree completely that if no information is provided to users the > chance of getting anything fixed anywhere is zero. > - Jim > >> >> >> -- >> Mikael Abrahamsson email: swmike@swm.pp.se >> > > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu May 8 12:33:14 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A3B81A010B for ; Thu, 8 May 2014 12:33:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.553 X-Spam-Level: X-Spam-Status: No, score=-2.553 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7zPoKc-dZ0pz for ; Thu, 8 May 2014 12:33:10 -0700 (PDT) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 811FD1A0100 for ; Thu, 8 May 2014 12:33:10 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.97,1012,1389772800"; d="asc'?scan'208";a="121991417" Received: from vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) by mx11-out.netapp.com with ESMTP; 08 May 2014 12:33:06 -0700 Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.246]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.03.0123.003; Thu, 8 May 2014 12:33:05 -0700 From: "Eggert, Lars" To: Dave Taht Thread-Topic: [rmcat] Packet reordering Thread-Index: Ac9nz4UZwrkvruVzR5WRR6/WPOSJEgCvVWUjABLsm4AAB4GogAAEnTcAAAJiDgAAAXy5gAAFnwEA Date: Thu, 8 May 2014 19:33:04 +0000 Message-ID: <0E97BDB9-6E3C-4461-AA3A-AA7DD1FBD472@netapp.com> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.26] Content-Type: multipart/signed; boundary="Apple-Mail=_813042DE-3E18-4D94-865A-85CAB47B07CF"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/TAadV2aprwpjIr4mw-8THmKyDzk Cc: Randell Jesup , "rmcat@ietf.org" , "Zimmermann, Alexander" , Jim Gettys , Mikael Abrahamsson Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 19:33:12 -0000 --Apple-Mail=_813042DE-3E18-4D94-865A-85CAB47B07CF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 2014-5-8, at 11:52, Dave Taht wrote: > Also I note there was a fairly major bug in rx steering for that would > result in ooo packets, induced and fixed back in the 3.8-3.11(?) > timeframe, which might be relevant to an earlier comment on this > thread... I think you refer to the comment I made. The anecdotal experience I = quoted was with 3.14. Lars --Apple-Mail=_813042DE-3E18-4D94-865A-85CAB47B07CF Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBU2vb7tZcnpRveo1xAQLkmwP+PGjXfpJ2aalQaPQ/2sY56AX9tcYWbDku UjGaDhOcsjHGO3jpBEjr7xUjS7rzugeEKz+gKtXwzhB4gbOKXQooUtCVYRLJXDB6 C9EFmRgKJZd2jpRFBCMj9eSIkJcg/l6YNOSrqwkZjNhDRnppUx9BwQ8FFmtregVr V2RwjeULtL8= =ij/9 -----END PGP SIGNATURE----- --Apple-Mail=_813042DE-3E18-4D94-865A-85CAB47B07CF-- From nobody Fri May 9 03:42:46 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE3DD1A0277 for ; Fri, 9 May 2014 03:42:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.553 X-Spam-Level: X-Spam-Status: No, score=-2.553 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6SlOEwzv_A5t for ; Fri, 9 May 2014 03:42:42 -0700 (PDT) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 8D05F1A024F for ; Fri, 9 May 2014 03:42:05 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.97,1017,1389772800"; d="asc'?scan'208";a="122098880" Received: from vmwexceht05-prd.hq.netapp.com ([10.106.77.35]) by mx11-out.netapp.com with ESMTP; 09 May 2014 03:42:00 -0700 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.187]) by vmwexceht05-prd.hq.netapp.com ([10.106.77.35]) with mapi id 14.03.0123.003; Fri, 9 May 2014 03:42:00 -0700 From: "Zimmermann, Alexander" To: Dave Taht Thread-Topic: [rmcat] Packet reordering Thread-Index: AQHPaozbGfHPc5r1Uk2yneWZzvq9QZs23ACAgAA8DYCAACTqAIAAEtuAgAAMG4CAASrsAA== Date: Fri, 9 May 2014 10:41:51 +0000 Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.20] Content-Type: multipart/signed; boundary="Apple-Mail=_626ECC9B-FC63-4A0B-A461-919834253C88"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/8scMWRotfA5QnycFo5T19OvAEd8 Cc: Randell Jesup , "rmcat@ietf.org" , Jim Gettys , Mikael Abrahamsson Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 10:42:45 -0000 --Apple-Mail=_626ECC9B-FC63-4A0B-A461-919834253C88 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Am 08.05.2014 um 18:52 schrieb Dave Taht : > On Thu, May 8, 2014 at 9:09 AM, Zimmermann, Alexander > wrote: >> Hi Jim, >>=20 >> Am 08.05.2014 um 17:01 schrieb Jim Gettys : >>=20 >>=20 >>=20 >>=20 >> On Thu, May 8, 2014 at 8:49 AM, Mikael Abrahamsson = wrote: >>>=20 >>> On Thu, 8 May 2014, Randell Jesup wrote: >>>=20 >>>> On 5/8/2014 3:12 AM, Lennart Schulte wrote: >>>>>=20 >>>>> Hi, >>>>> while measuring 3G and 4G networks I also saw reordering: 35% of = all >>>>> connections (12392 in 3G and 676 in 4G) had reordering. In about = 2% of >>>>> these reordering events the reordering extent was at least 3 = packets >>>>> large (which caused TCP to spuriously enter fast recovery). >>>>=20 >>>>=20 >>>> 35% doesn't seem surprising.... I presume reordering can/will = happen; my >>>> interest would be amount of reordering, and depth/distribution of >>>> reordering. (More than 30-50ms of reordering effectively =3D=3D = loss from an RTP >>>> standpoint, generally.) And unless it affects the congestion = algorithm >>>> (which is of concern here), less than jitter depth reordering is = pretty much >>>> "who cares" (maybe not quite, but close). >>>>=20 >>>> Also, since the distribution is very uneven, are there networks = (3G, >>>> 4G/LTE, WiFi (of various flavors) where this is dramatically more = common? >>>=20 >>>=20 >>> I have historically proposed that applications should log these = parameters >>> as they happen, and provide this information in aggregate or = per-session >>> basis to present to the user. This is applicable to RTP and it's = applicable >>> to TCP (and more). As soon as you have sequence numbers you can = detect this >>> and you can count it. >>>=20 >>> So while it's not core to this discussion, where is the best place = to gain >>> traction for suggestion to count and aggregate data on: >>>=20 >>> Number of packets lost >>> Number of packets arrived too late (PDV buffer miss) >>> Number of packets that arrived out of order >>> Number of packets that arrived out of order and caused PDV buffer = miss >>>=20 >>> If applications actually logged this to operators and users, they = might be >>> able to understand the problems and have a better understanding of = the >>> customers quality of experience, and also the user would have better = data to >>> do a fault report to the operator with. >>=20 >>=20 >> It isn't just the cellular operators. It's often going to be your = home >> network. >>>=20 >>>=20 >>> Basically, packet re-ordering shouldn't happen. Networks that cause >>> packets to be re-ordered should know that this causes bad customer >>> experience so they can stop. A lot of the time this is done out of >>> ignorance. >>=20 >>=20 >> This is a naive point of view. First, current TCP implementations = are >> overly sensitive to reordering (though I believe there has been = recent work >> in Linux to help this a lot, but it will take a long time before it = is >> deployed widely enough to help). >=20 > And things like mosh & rtp mostly care about the most recent packets. > If packets arrive out of order, the older packets are dropped anyway. > Better that they not be delivered... >=20 > and ledbat/quic don't mind packets arriving out of order. >=20 >=20 >=20 >>=20 >> Only as a side note=85 >> Actually, there is a TCP reordering robustness in Linux for years. = However, >> it has weaknesses and performance not very well. >> See our TCPM presentation: >> http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf >=20 > What version of linux tcp was this taken against? The entire stack has > evolved considerably between linux 3.3 and linux 3.14. I read netdev. I hope I know what=92s going on ;-) >=20 > I am aware of a bunch of reordering fixes that went in fairly > recently, and there's some interesting side effects from the new > sch_fq + pacing stuff that I think make re-ordering events less > likely. Understood. But I speak here about the reordering detection, = quantification and reaction mechanism in Linux. I=92m only aware of one to two recent = patches which fall in this category. This was patch from Yuchung which tries to still open the cwnd in case of packet reordering. >=20 > Also I note there was a fairly major bug in rx steering for that would > result in ooo packets, induced and fixed back in the 3.8-3.11(?) > timeframe, which might be relevant to an earlier comment on this > thread=85 >=20 That=92s interesting. I have to take a look into. Alex=20 >=20 >>=20 >> Alex >>=20 >>=20 >> Also, remember today at least one hop of your path is likely to be = over a >> wireless link (lossy, error prone), whether 802.11 or cellular. >>=20 >> This has had a serious unintended follow-on effect. Since the = consequences >> to TCP performance have been so bad, this has encouraged driver = implementers >> on wireless (of all sorts) to put buffering in the receive path = (needed >> anyway for aggregation on 802.11n, for example). In these = situations, the >> receiver may "freeze" forwarding already received packets hoping that >> damaged packets get repaired quickly to avoid reordering. But this >> introduces delays before the packets get released. >=20 > You have several separate, interrelated problems here. >=20 > Some drivers will only release the entire aggregate of packets, in > order, and do a lot of retries until they get them all, come hell or > high water. >=20 > Others will deliver a later aggregate if it arrives unscathed while > retrying an earlier aggregate. >=20 > Few will deliver "good" packets from an aggregate while retrying, and > none I know of do anything other than give up after a long time out, > rather than doing something saner like looking at flows and trying to > ensure at least one packet is delivered from a flow, and dropping the > rest > when there are major problems. >=20 >> One of the "interesting" >> bufferbloat discoveries in the Linux WiFi stack, for example, was >> discovering that some drivers would try to repair the damaged packets >> effectively infinitely, causing spikes up to 1 second. And, btw, = this kind >> of problem is rampant in the current generation of home routers. >=20 > seen 36+ seconds. http://www.bufferbloat.net/issues/216 >=20 > a year after we fixed this bug, broadcom found and fixed the same > thing in one of their chipsets. >=20 > Millions of devices in the field flooding the airwaves with pointless > retries, with missing firmware updates. >=20 >> And what do you do if you can't repair the packet quickly (e.g. = inside of a >> single transmit-op)? The receiver may decide to then forward the = packets in >> the aggregate you have, but later (on a subsequent transmit = opportunity) the >> missing packet(s) may be transmitted by the host, thereby causing out = of >> order delivery. >>=20 >> But I agree completely that if no information is provided to users = the >> chance of getting anything fixed anywhere is zero. >> - Jim >>=20 >>>=20 >>>=20 >>> -- >>> Mikael Abrahamsson email: swmike@swm.pp.se >>>=20 >>=20 >>=20 >=20 >=20 >=20 > --=20 > Dave T=E4ht >=20 > NSFW: = https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indec= ent.article --Apple-Mail=_626ECC9B-FC63-4A0B-A461-919834253C88 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlNssPYACgkQdyiq39b9uS7VrgCgsMlQG8l11ir/gTnImyg9Fh0b fagAoLSRJk2w0XSDz6QWTbxC3e7tYu7X =mEQN -----END PGP SIGNATURE----- --Apple-Mail=_626ECC9B-FC63-4A0B-A461-919834253C88-- From nobody Mon May 12 09:36:07 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EA7E11A0766 for ; Mon, 12 May 2014 09:36:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.03 X-Spam-Level: X-Spam-Status: No, score=-2.03 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ba7dlTtCxgYc for ; Mon, 12 May 2014 09:36:01 -0700 (PDT) Received: from mail-ig0-x22f.google.com (mail-ig0-x22f.google.com [IPv6:2607:f8b0:4001:c05::22f]) by ietfa.amsl.com (Postfix) with ESMTP id B71301A0759 for ; Mon, 12 May 2014 09:36:01 -0700 (PDT) Received: by mail-ig0-f175.google.com with SMTP id uq10so4042651igb.8 for ; Mon, 12 May 2014 09:35:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=t3oTYuKdkGNnClFJVGJbcA6PisiSMX2cLNOXWHjQD00=; b=Cvhg6T2dZl1Y1uZDIiS7UlJ7L9/lSE5tn4GEye3wuC3mlRK3XK7csTsIlV1xLoUxGt fAKr/+xd5IwOIBKcgbQOfVid6AH8pTH5/hBD22KqFsN6KNQYKUFX9aBa58uYC1/eTiDr /C8ip4cUmkPUni9BD7b99JYSXxRY6KATmfPsYbPifnjq4nSXbFUZx7Z0T5TqqQ4uwS84 hFUjdfJjgMgm5gFxSTtQpkzWtaOqs9CKSH5R9kdU+R4mkuHc8Fr5poT9tJKbzr1vElYK qwBbl+SQZ21W4vhK8qdlLCeL/xIwlV6ttVVt40A+0K3NCyHq+iIJDWbJXq73+cTGDe90 PWsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=t3oTYuKdkGNnClFJVGJbcA6PisiSMX2cLNOXWHjQD00=; b=fctobN1PlprtWJV+gvGGp0SzeKwgZHwJzgzuB0s6W6iPFACToxt6IsLvcr2PErQLMq LPBUzFrFZ92kFZYkF2fJTIcF17xqFWJFAZb5LkLJTlXyF0BxUYC3y0P0gr2vafZSheiw 5a2T/zUmrA1cNjvD4IcyMt7s3GGwcXqbU8MmT+q2xfNMoZIFsG27kZkla2OODTNbrBDk hfmGXSM93LsurJNNpzbesVbBc50a7XfkJtdalr0+Zc/R88VfRDxeBcSa/ptYNj5aocev WC1ZTCereXGaVCqITeqVC4nAwk1o/smn4v+2DyOGtz3fgZUPYeHqlYMKVP5VIjUR9Rgq fnJA== X-Gm-Message-State: ALoCoQknJb5p2Q2DeqUfkROQV9Y9+ImcsWtHmlqx4VU/hKqZ3K05mbCDhO0IqWmoPXZSWANswyll X-Received: by 10.50.143.34 with SMTP id sb2mr46043705igb.48.1399912555470; Mon, 12 May 2014 09:35:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.223.163 with HTTP; Mon, 12 May 2014 09:35:15 -0700 (PDT) In-Reply-To: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> From: Yuchung Cheng Date: Mon, 12 May 2014 09:35:15 -0700 Message-ID: To: Dave Taht Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/t8A-w1nfLtpxkwOnn1OSUz8Tbok Cc: "rmcat@ietf.org" Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 16:36:05 -0000 On Thu, May 8, 2014 at 9:52 AM, Dave Taht wrote: > On Thu, May 8, 2014 at 9:09 AM, Zimmermann, Alexander > wrote: >> Hi Jim, >> >> Am 08.05.2014 um 17:01 schrieb Jim Gettys : >> >> >> >> >> On Thu, May 8, 2014 at 8:49 AM, Mikael Abrahamsson wr= ote: >>> >>> On Thu, 8 May 2014, Randell Jesup wrote: >>> >>>> On 5/8/2014 3:12 AM, Lennart Schulte wrote: >>>>> >>>>> Hi, >>>>> while measuring 3G and 4G networks I also saw reordering: 35% of all >>>>> connections (12392 in 3G and 676 in 4G) had reordering. In about 2% o= f >>>>> these reordering events the reordering extent was at least 3 packets >>>>> large (which caused TCP to spuriously enter fast recovery). >>>> >>>> >>>> 35% doesn't seem surprising.... I presume reordering can/will happen; = my >>>> interest would be amount of reordering, and depth/distribution of >>>> reordering. (More than 30-50ms of reordering effectively =3D=3D loss f= rom an RTP >>>> standpoint, generally.) And unless it affects the congestion algorith= m >>>> (which is of concern here), less than jitter depth reordering is prett= y much >>>> "who cares" (maybe not quite, but close). >>>> >>>> Also, since the distribution is very uneven, are there networks (3G, >>>> 4G/LTE, WiFi (of various flavors) where this is dramatically more comm= on? >>> >>> >>> I have historically proposed that applications should log these paramet= ers >>> as they happen, and provide this information in aggregate or per-sessio= n >>> basis to present to the user. This is applicable to RTP and it's applic= able >>> to TCP (and more). As soon as you have sequence numbers you can detect = this >>> and you can count it. >>> >>> So while it's not core to this discussion, where is the best place to g= ain >>> traction for suggestion to count and aggregate data on: >>> >>> Number of packets lost >>> Number of packets arrived too late (PDV buffer miss) >>> Number of packets that arrived out of order >>> Number of packets that arrived out of order and caused PDV buffer miss >>> >>> If applications actually logged this to operators and users, they might= be >>> able to understand the problems and have a better understanding of the >>> customers quality of experience, and also the user would have better da= ta to >>> do a fault report to the operator with. >> >> >> It isn't just the cellular operators. It's often going to be your home >> network. >>> >>> >>> Basically, packet re-ordering shouldn't happen. Networks that cause >>> packets to be re-ordered should know that this causes bad customer >>> experience so they can stop. A lot of the time this is done out of >>> ignorance. >> >> >> This is a naive point of view. First, current TCP implementations are >> overly sensitive to reordering (though I believe there has been recent w= ork >> in Linux to help this a lot, but it will take a long time before it is >> deployed widely enough to help). > > And things like mosh & rtp mostly care about the most recent packets. > If packets arrive out of order, the older packets are dropped anyway. > Better that they not be delivered... > > and ledbat/quic don't mind packets arriving out of order. > > > >> >> Only as a side note=E2=80=A6 >> Actually, there is a TCP reordering robustness in Linux for years. Howev= er, >> it has weaknesses and performance not very well. >> See our TCPM presentation: >> http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf > > What version of linux tcp was this taken against? The entire stack has > evolved considerably between linux 3.3 and linux 3.14. It's in 3.12 http://patchwork.ozlabs.org/patch/268918/ http://patchwork.ozlabs.org/patch/272993/ (bug fix of the previous patch) > > I am aware of a bunch of reordering fixes that went in fairly > recently, and there's some interesting side effects from the new > sch_fq + pacing stuff that I think make re-ordering events less > likely. > > Also I note there was a fairly major bug in rx steering for that would > result in ooo packets, induced and fixed back in the 3.8-3.11(?) > timeframe, which might be relevant to an earlier comment on this > thread... > > >> >> Alex >> >> >> Also, remember today at least one hop of your path is likely to be over = a >> wireless link (lossy, error prone), whether 802.11 or cellular. >> >> This has had a serious unintended follow-on effect. Since the consequen= ces >> to TCP performance have been so bad, this has encouraged driver implemen= ters >> on wireless (of all sorts) to put buffering in the receive path (needed >> anyway for aggregation on 802.11n, for example). In these situations, t= he >> receiver may "freeze" forwarding already received packets hoping that >> damaged packets get repaired quickly to avoid reordering. But this >> introduces delays before the packets get released. > > You have several separate, interrelated problems here. > > Some drivers will only release the entire aggregate of packets, in > order, and do a lot of retries until they get them all, come hell or > high water. > > Others will deliver a later aggregate if it arrives unscathed while > retrying an earlier aggregate. > > Few will deliver "good" packets from an aggregate while retrying, and > none I know of do anything other than give up after a long time out, > rather than doing something saner like looking at flows and trying to > ensure at least one packet is delivered from a flow, and dropping the > rest > when there are major problems. > >> One of the "interesting" >> bufferbloat discoveries in the Linux WiFi stack, for example, was >> discovering that some drivers would try to repair the damaged packets >> effectively infinitely, causing spikes up to 1 second. And, btw, this k= ind >> of problem is rampant in the current generation of home routers. > > seen 36+ seconds. http://www.bufferbloat.net/issues/216 > > a year after we fixed this bug, broadcom found and fixed the same > thing in one of their chipsets. > > Millions of devices in the field flooding the airwaves with pointless > retries, with missing firmware updates. > >> And what do you do if you can't repair the packet quickly (e.g. inside o= f a >> single transmit-op)? The receiver may decide to then forward the packet= s in >> the aggregate you have, but later (on a subsequent transmit opportunity)= the >> missing packet(s) may be transmitted by the host, thereby causing out of >> order delivery. >> >> But I agree completely that if no information is provided to users the >> chance of getting anything fixed anywhere is zero. >> - Jim >> >>> >>> >>> -- >>> Mikael Abrahamsson email: swmike@swm.pp.se >>> >> >> > > > > -- > Dave T=C3=A4ht > > NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_029= 6_indecent.article > From nobody Mon May 12 09:40:46 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6D5A91A072A for ; Mon, 12 May 2014 09:40:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VcKITRf9Mdr7 for ; Mon, 12 May 2014 09:40:41 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id 33A7C1A0729 for ; Mon, 12 May 2014 09:40:41 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 08E379C; Mon, 12 May 2014 18:40:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1399912834; bh=HWpcLwMIbR+1kqiAbQFFAvL3sVoCXqJAx8rqYwGxC2U=; h=Date:From:To:Subject:In-Reply-To:References:From; b=IK4qHO+rO2KA5pjofnvFf+1BQlhlxl+1TrxQbOzLC2WKZCbYhr38kWxFAnlSPao0O wparz3R5nvhcsk/9PsG5ng/7t4e41y8S0dluY3q3CP7/EsINvbajFBQ/k7tT3qeMjm 9oUEhif+4QnNuL7MYJix2NfkRChab6qXZX0+JNHE= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 046E49A for ; Mon, 12 May 2014 18:40:34 +0200 (CEST) Date: Mon, 12 May 2014 18:40:33 +0200 (CEST) From: Mikael Abrahamsson To: "rmcat@ietf.org" In-Reply-To: Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-137064504-1921853490-1399912834=:29282" Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/asrfWqIVHGaDI2lCd0NXwnu7UNA Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 16:40:43 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---137064504-1921853490-1399912834=:29282 Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT On Thu, 8 May 2014, Dave Taht wrote: >> Only as a side note… >> Actually, there is a TCP reordering robustness in Linux for years. However, >> it has weaknesses and performance not very well. >> See our TCPM presentation: >> http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf > > What version of linux tcp was this taken against? The entire stack has > evolved considerably between linux 3.3 and linux 3.14. ... still no interest in my proposal to keep track of when these events are happening? I only saw discussion on how to improve handling of when it happens, not to actually count and present these events? -- Mikael Abrahamsson email: swmike@swm.pp.se ---137064504-1921853490-1399912834=:29282-- From nobody Mon May 12 09:45:55 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F2081A070E for ; Mon, 12 May 2014 09:45:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.553 X-Spam-Level: X-Spam-Status: No, score=-2.553 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 93ix54Vg9a0c for ; Mon, 12 May 2014 09:45:52 -0700 (PDT) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 71BA91A034E for ; Mon, 12 May 2014 09:45:52 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.97,1036,1389772800"; d="asc'?scan'208";a="122612962" Received: from vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) by mx11-out.netapp.com with ESMTP; 12 May 2014 09:45:46 -0700 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.187]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.03.0123.003; Mon, 12 May 2014 09:45:46 -0700 From: "Zimmermann, Alexander" To: Mikael Abrahamsson Thread-Topic: [rmcat] Packet reordering Thread-Index: AQHPaozbGfHPc5r1Uk2yneWZzvq9QZs23ACAgAA8DYCAACTqAIAAEtuAgAAMG4CABkYbgIAAAXIA Date: Mon, 12 May 2014 16:45:36 +0000 Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.20] Content-Type: multipart/signed; boundary="Apple-Mail=_087FAA85-73BE-416B-9554-2058ED9DD1B0"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/yaWQR5v5kFcfzw5eAZbK0FJzPHA Cc: "rmcat@ietf.org" Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 16:45:54 -0000 --Apple-Mail=_087FAA85-73BE-416B-9554-2058ED9DD1B0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Am 12.05.2014 um 18:40 schrieb Mikael Abrahamsson : > On Thu, 8 May 2014, Dave Taht wrote: >=20 >>> Only as a side note=85 >>> Actually, there is a TCP reordering robustness in Linux for years. = However, >>> it has weaknesses and performance not very well. >>> See our TCPM presentation: >>> http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf >>=20 >> What version of linux tcp was this taken against? The entire stack = has >> evolved considerably between linux 3.3 and linux 3.14. >=20 > ... still no interest in my proposal to keep track of when these = events are happening? I only saw discussion on how to improve handling = of when it happens, not to actually count and present these events? netstat -s? UdpLite: TcpExt: 54 TCP sockets finished time wait in fast timer 447 delayed acks sent Quick ack mode was activated 2 times 3651238 packets directly queued to recvmsg prequeue. 3385392 bytes directly in process context from backlog 3735300416 bytes directly received in process context from prequeue 5935 packet headers predicted 6097108 packets header predicted and directly queued to user 20251843 acknowledgments not containing data payload received 11398833 predicted acknowledgments 3562 times recovered from packet loss by selective acknowledgements Detected reordering 6 times using FACK Detected reordering 5 times using SACK Detected reordering 705 times using time stamp 2159 congestion windows fully recovered without slow start 208 congestion windows partially recovered using Hoe heuristic 1167 congestion windows recovered without slow start by DSACK 1 congestion windows recovered without slow start after partial ack 7469 fast retransmits 3 forward retransmits 2 other TCP timeouts TCPLossProbes: 1 TCPLossProbeRecovery: 1 2 DSACKs sent for old packets 3927 DSACKs received TCPDSACKIgnoredNoUndo: 183 TCPSackShifted: 24503 TCPSackMerged: 14139 TCPSackShiftFallback: 32545 TCPRcvCoalesce: 200 TCPSpuriousRtxHostQueues: 2 TCPAutoCorking: 2949971 TCPSynRetrans: 1 TCPOrigDataSent: 1031400383 >=20 > --=20 > Mikael Abrahamsson email: swmike@swm.pp.se --Apple-Mail=_087FAA85-73BE-416B-9554-2058ED9DD1B0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlNw+rgACgkQdyiq39b9uS4AXACeOjl8HQOqtL43be8czgJ5I7IT t7gAnRhA2c+AOBlv6RJVbWPDXfezc9a6 =tNZo -----END PGP SIGNATURE----- --Apple-Mail=_087FAA85-73BE-416B-9554-2058ED9DD1B0-- From nobody Mon May 12 10:33:31 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 70ACD1A0756 for ; Mon, 12 May 2014 10:33:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HQbNRkOPM6vH for ; Mon, 12 May 2014 10:33:27 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id 981E91A0757 for ; Mon, 12 May 2014 10:32:34 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 27DEB9C; Mon, 12 May 2014 19:32:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1399915948; bh=MtXFSQiJv8my0zgjI1np4lzZe2qTXZk7pJ9WBLHf6uA=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=3N7HA02vaEqRuonvYiYn0E34bPa5N/T+9YufYtuURIX/Z7vX0hfmGoAVwbVViyKfH 0KIvSrqoiTz23Zm4tQLS1IRgWbbNeImIDGKS8OGqb+ALMRUh++rgW84O+zXMsCOr8v kQ3q4eT6In75rFlCfUAkZz/hBB7aVKpapxSOb9AQ= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 2030C9A; Mon, 12 May 2014 19:32:28 +0200 (CEST) Date: Mon, 12 May 2014 19:32:28 +0200 (CEST) From: Mikael Abrahamsson To: "Zimmermann, Alexander" In-Reply-To: Message-ID: References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-137064504-1036844042-1399915948=:29282" Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/RwaLK4hULtNhgBEyF6crjm6F1S0 Cc: "rmcat@ietf.org" Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 17:33:29 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---137064504-1036844042-1399915948=:29282 Content-Type: TEXT/PLAIN; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8BIT On Mon, 12 May 2014, Zimmermann, Alexander wrote: > > Am 12.05.2014 um 18:40 schrieb Mikael Abrahamsson : > >> On Thu, 8 May 2014, Dave Taht wrote: >> >>>> Only as a side note… >>>> Actually, there is a TCP reordering robustness in Linux for years. However, >>>> it has weaknesses and performance not very well. >>>> See our TCPM presentation: >>>> http://tools.ietf.org/agenda/89/slides/slides-89-tcpm-7.pdf >>> >>> What version of linux tcp was this taken against? The entire stack has >>> evolved considerably between linux 3.3 and linux 3.14. >> >> ... still no interest in my proposal to keep track of when these events are happening? I only saw discussion on how to improve handling of when it happens, not to actually count and present these events? > > netstat -s? Sure. How do we present this to the user in a way that they can find usable, or helpdesks can find usable? I would like this information per destination, per session or some other metric so it can say "oh, it seems your connection primarily behaves a certain way, you're seeing a lot of *foo*". -- Mikael Abrahamsson email: swmike@swm.pp.se ---137064504-1036844042-1399915948=:29282-- From nobody Mon May 12 10:48:35 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7C8EB1A0772 for ; Mon, 12 May 2014 10:48:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZPb3kp7waNgS for ; Mon, 12 May 2014 10:48:24 -0700 (PDT) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) by ietfa.amsl.com (Postfix) with ESMTP id 584801A076A for ; Mon, 12 May 2014 10:48:23 -0700 (PDT) Received: by mail-we0-f181.google.com with SMTP id w61so7305215wes.12 for ; Mon, 12 May 2014 10:48:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=+Es9jLA/IwjxHsnExdDCFrUdt7DLP4Y6D9CvN5JSOc4=; b=tBQY+7WaTVUq2551JR0FAzHYeCQhAKy4H6TPZUvJORJV5XwK/touTTM7TkPewdRZkV 67Ort3ASgR7GehRrjnxdJRQqsZG/OXAuAu2kLqwVxoFDYTIcvv+XpQ7O8fznO1tWzgcS AYPBmXEFhfma/YJDLOtTIKDDXkrugZQWZ33pbgAHsU/qm6pV1jeJveHiRn+EARpoaRpM dHkOvnjnAMDwwLf7TCpHlsZWNjsoTBMlfgp466gwTRTs9rUFN5cfdyABREsX7P6P8lwI jhj8M+PHSThn0CgFviv0r4Yonfej2o+A7IqAYQ4pp6a2Hadi64g0a5NnBXWMq7siPgDg K5oQ== MIME-Version: 1.0 X-Received: by 10.180.14.233 with SMTP id s9mr16952364wic.53.1399916896850; Mon, 12 May 2014 10:48:16 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Mon, 12 May 2014 10:48:16 -0700 (PDT) In-Reply-To: <0E97BDB9-6E3C-4461-AA3A-AA7DD1FBD472@netapp.com> References: <81564C0D7D4D2A4B9A86C8C7404A13DA31F5D7C5@ESESSMB205.ericsson.se> <21014_1399245468_5366CA9C_21014_40_1_CAMzhQmPq5w2kVwEBRZKnJkbRyVPUkGp3d2TPy53bwwP-jzhBCw@mail.gmail.com> <536B2E47.7090005@aalto.fi> <536B4AE5.1020000@jesup.org> <484F1193-7B4D-426D-B725-B643DFF7B0CE@netapp.com> <0E97BDB9-6E3C-4461-AA3A-AA7DD1FBD472@netapp.com> Date: Mon, 12 May 2014 10:48:16 -0700 Message-ID: From: Dave Taht To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/8uEGfLdIijvK-Vk3jDMJZ-X8c3Q Cc: Randell Jesup , "rmcat@ietf.org" , "Zimmermann, Alexander" , Jim Gettys , Mikael Abrahamsson Subject: Re: [rmcat] Packet reordering X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 17:48:28 -0000 On Thu, May 8, 2014 at 12:33 PM, Eggert, Lars wrote: > On 2014-5-8, at 11:52, Dave Taht wrote: >> Also I note there was a fairly major bug in rx steering for that would >> result in ooo packets, induced and fixed back in the 3.8-3.11(?) >> timeframe, which might be relevant to an earlier comment on this >> thread... > > I think you refer to the comment I made. The anecdotal experience I quote= d was with 3.14. > > Lars This was the commit I was referring to, which was introduced and fixed somewhere between 3.10 and 3.12. So perhaps you are dealing with a newer issue. commit 50d1784ee4683f073c0362ee360bfae7a3333d6c Author: Eric Dumazet Date: Sat Sep 7 12:02:57 2013 -0700 net: fix multiqueue selection commit 416186fbf8c5b4e4465 ("net: Split core bits of netdev_pick_tx into __netdev_pick_tx") added a bug that disables caching of queue index in the socket. This is the source of packet reorders for TCP flows, and again this is happening more often when using FQ pacing. Old code was doing if (queue_index !=3D old_index) sk_tx_queue_set(sk, queue_index); Alexander renamed the variables but forgot to change sk_tx_queue_set() 2nd parameter. if (queue_index !=3D new_index) sk_tx_queue_set(sk, queue_index); This means we store -1 over and over in sk->sk_tx_queue_mapping --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu May 22 06:08:12 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 641E61A0091 for ; Thu, 22 May 2014 06:08:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uPdQhc_WkAMT for ; Thu, 22 May 2014 06:08:07 -0700 (PDT) Received: from mail-out2.uio.no (mail-out2.uio.no [IPv6:2001:700:100:10::58]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6680A1A001A for ; Thu, 22 May 2014 06:08:07 -0700 (PDT) Received: from mail-mx4.uio.no ([129.240.10.45]) by mail-out2.uio.no with esmtp (Exim 4.75) (envelope-from ) id 1WnSj6-0006RE-Mc for rmcat@ietf.org; Thu, 22 May 2014 15:08:04 +0200 Received: from 1x-193-157-254-236.uio.no ([193.157.254.236]) by mail-mx4.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from ) id 1WnSj5-0002IU-U6 for rmcat@ietf.org; Thu, 22 May 2014 15:08:04 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) From: Michael Welzl In-Reply-To: Date: Thu, 22 May 2014 15:08:03 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <38DDD51E-8858-47DA-98EE-D005BB044DC9@ifi.uio.no> References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> To: rmcat WG X-Mailer: Apple Mail (2.1510) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 1 msgs/h 1 sum rcpts/h 6 sum msgs/h 3 total rcpts 16638 max rcpts/h 44 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-6.1, required=5.0, autolearn=disabled, RP_MATCHES_RCVD=-1.051, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 5E5195E03AD67CF3535B25CC21B0198F69F933CF X-UiO-SPAM-Test: remote_host: 193.157.254.236 spam_score: -60 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 14 max/h 3 blacklist 0 greylist 0 ratelimit 0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/1Qe6-e8jSeSqPtBSUN3Ozw-bv3c Subject: Re: [rmcat] New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 13:08:10 -0000 ping =85 On May 7, 2014, at 1:11 PM, Michael Welzl wrote: > Dear all, >=20 > We have submitted an update of our coupled congestion control draft. = This now includes the new algorithm which should reduce queuing delay = and packet loss, as documented in = http://safiquli.at.ifi.uio.no/paper/fse-tech-report.pdf >=20 > At this point, we'd really appreciate it if folks would take a look = and comment :-) >=20 > Cheers, > Michael >=20 >=20 >=20 > Begin forwarded message: >=20 >> From: internet-drafts@ietf.org >> Subject: New Version Notification for = draft-welzl-rmcat-coupled-cc-03.txt >> Date: 7. mai 2014 13:08:05 GMT+02:00 >> To: Michael Welzl , "Safiqul Islam" = , "Stein Gjessing" , "Michael = Welzl" , Safiqul Islam , Stein = Gjessing >>=20 >>=20 >> A new version of I-D, draft-welzl-rmcat-coupled-cc-03.txt >> has been successfully submitted by Michael Welzl and posted to the >> IETF repository. >>=20 >> Name: draft-welzl-rmcat-coupled-cc >> Revision: 03 >> Title: Coupled congestion control for RTP media >> Document date: 2014-05-07 >> Group: Individual Submission >> Pages: 19 >> URL: = http://www.ietf.org/internet-drafts/draft-welzl-rmcat-coupled-cc-03.txt >> Status: = https://datatracker.ietf.org/doc/draft-welzl-rmcat-coupled-cc/ >> Htmlized: = http://tools.ietf.org/html/draft-welzl-rmcat-coupled-cc-03 >> Diff: = http://www.ietf.org/rfcdiff?url2=3Ddraft-welzl-rmcat-coupled-cc-03 >>=20 >> Abstract: >> When multiple congestion controlled RTP sessions traverse the same >> network bottleneck, it can be beneficial to combine their controls >> such that the total on-the-wire behavior is improved. This = document >> describes such a method for flows that have the same sender, in a = way >> that is as flexible and simple as possible while minimizing the >> amount of changes needed to existing RTP applications. >>=20 >>=20 >>=20 >>=20 >> Please note that it may take a couple of minutes from the time of = submission >> until the htmlized version and diff are available at tools.ietf.org. >>=20 >> The IETF Secretariat >>=20 >=20 From nobody Thu May 22 07:12:00 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9198A1A01B9 for ; Thu, 22 May 2014 07:11:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aT7bLvPB5Qe8 for ; Thu, 22 May 2014 07:11:49 -0700 (PDT) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 286B01A0176 for ; Thu, 22 May 2014 07:11:43 -0700 (PDT) Received: by mail-wi0-f175.google.com with SMTP id f8so9328179wiw.8 for ; Thu, 22 May 2014 07:11:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=7ADsHYnMsdE/nkmsgr8wMB1fxFvkOWholNnI/3WTPlI=; b=t63jWQb5g9WqaYJKeFaxiDQt24XAr51+UVG+TXuctCPKn3U/K+KQrFAWlYI+XDAzYI ZGCSBpICHQf5olJNOlvKUpIsZR784iC5VB3B/BaWz64FLv8zVtP/m0zu/zeXSZcD8DHo iuBkOOHeApqnwsKXZQlcqX1GjUkxKVGW+yUIs3PYQPU/JSjjmnrbipNvvh376ei0sT5s UuYkseyzRj5YinX8nPCDHDgYlAzwuaPmtMGY+Op8gpn3D7QmFAo/FkZ/oNc3VPPkAI3p Nehk80CmJpLM3VYRP5tdAinPdRM4joPm2zQdX00CkYFa8CmKjeJq96lONOHJuc0DNRiR L5sw== MIME-Version: 1.0 X-Received: by 10.180.14.233 with SMTP id s9mr16582210wic.53.1400767900894; Thu, 22 May 2014 07:11:40 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Thu, 22 May 2014 07:11:40 -0700 (PDT) In-Reply-To: References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> Date: Thu, 22 May 2014 07:11:40 -0700 Message-ID: From: Dave Taht To: Michael Welzl Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/S2HlpKtVG3D1LIzc18M9BecdSrE Cc: rmcat WG Subject: Re: [rmcat] Fwd: New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 14:11:56 -0000 Is there some reason why you do not use "weighted fair queuing" as analogous to "controllable fairness with prioritization" or cite any of that liturature? While being able to adjust the fairness dynamically is a good thing, the root technique is wfq, in my reading. re: 1 This (13 packet queue length) is based on the bandwidth delay product (BDP). We repeated our tests with dif- ferent queue lengths and found no significant differences. What different queue lengths? Against what competing traffic? On Wed, May 7, 2014 at 4:11 AM, Michael Welzl wrote: > Dear all, > > We have submitted an update of our coupled congestion control draft. This > now includes the new algorithm which should reduce queuing delay and pack= et > loss, as documented in > http://safiquli.at.ifi.uio.no/paper/fse-tech-report.pdf > > At this point, we'd really appreciate it if folks would take a look and > comment :-) > > Cheers, > Michael > > > > Begin forwarded message: > > From: internet-drafts@ietf.org > Subject: New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt > Date: 7. mai 2014 13:08:05 GMT+02:00 > To: Michael Welzl , "Safiqul Islam" > , "Stein Gjessing" , "Michael Wel= zl" > , Safiqul Islam , Stein Gjessing > > > > A new version of I-D, draft-welzl-rmcat-coupled-cc-03.txt > has been successfully submitted by Michael Welzl and posted to the > IETF repository. > > Name: draft-welzl-rmcat-coupled-cc > Revision: 03 > Title: Coupled congestion control for RTP media > Document date: 2014-05-07 > Group: Individual Submission > Pages: 19 > URL: > http://www.ietf.org/internet-drafts/draft-welzl-rmcat-coupled-cc-03.txt > Status: > https://datatracker.ietf.org/doc/draft-welzl-rmcat-coupled-cc/ > Htmlized: http://tools.ietf.org/html/draft-welzl-rmcat-coupled-cc-0= 3 > Diff: > http://www.ietf.org/rfcdiff?url2=3Ddraft-welzl-rmcat-coupled-cc-03 > > Abstract: > When multiple congestion controlled RTP sessions traverse the same > network bottleneck, it can be beneficial to combine their controls > such that the total on-the-wire behavior is improved. This document > describes such a method for flows that have the same sender, in a way > that is as flexible and simple as possible while minimizing the > amount of changes needed to existing RTP applications. > > > > > Please note that it may take a couple of minutes from the time of submiss= ion > until the htmlized version and diff are available at tools.ietf.org. > > The IETF Secretariat > > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu May 22 09:50:45 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 96AE11A0236 for ; Thu, 22 May 2014 09:50:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mqmFEjRNnCfy for ; Thu, 22 May 2014 09:50:42 -0700 (PDT) Received: from mail-out1.uio.no (mail-out1.uio.no [IPv6:2001:700:100:10::57]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF6331A023B for ; Thu, 22 May 2014 09:50:41 -0700 (PDT) Received: from mail-mx4.uio.no ([129.240.10.45]) by mail-out1.uio.no with esmtp (Exim 4.75) (envelope-from ) id 1WnWCV-0002hH-D4; Thu, 22 May 2014 18:50:39 +0200 Received: from 59.115.34.95.customer.cdi.no ([95.34.115.59] helo=[192.168.0.114]) by mail-mx4.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from ) id 1WnWCU-0000FM-Vt; Thu, 22 May 2014 18:50:39 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) From: Michael Welzl In-Reply-To: Date: Thu, 22 May 2014 18:50:38 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <39CD560F-5515-4265-A8C2-B1A7D141B158@ifi.uio.no> References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> To: Dave Taht X-Mailer: Apple Mail (2.1878.2) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 6 msgs/h 4 sum rcpts/h 11 sum msgs/h 6 total rcpts 16659 max rcpts/h 44 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: D4AB576E4726FC0A605D1000FE2F35C0C33A8BED X-UiO-SPAM-Test: remote_host: 95.34.115.59 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 4 total 1503 max/h 14 blacklist 0 greylist 0 ratelimit 0 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/GsfcGtHvGJ1HtiDcB17TGqwL6HI Cc: rmcat WG Subject: Re: [rmcat] New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 16:50:43 -0000 Hey, Thanks for that - glad see some interest at last! In line: On 22. mai 2014, at 16:11, Dave Taht wrote: > Is there some reason why you do not use "weighted fair queuing" as > analogous to "controllable fairness with prioritization" or cite any > of that liturature? >=20 > While being able to adjust the fairness dynamically is a good thing, > the root technique is wfq, in my reading. Well, wfq is about assigning a flow its weighted portion of the total = weight; this is what step (c) in our two active example algorithms does, = and it's also what wfq does. We didn't cite wfq because we don't think = of this as a scheduler, but since the active algorithm assigns rate = updates to all flows whenever any flow updates its rate, it's really = pretty much the same as a scheduler in fact. That's all - no major other reasons. We can put a wfq citation in there = if that makes you happy :) > re: >=20 > 1 This (13 packet queue length) is based on the bandwidth > delay product (BDP). We repeated our tests with dif- > ferent queue lengths and found no significant differences. >=20 > What different queue lengths? Against what competing traffic? I'll leave that for Safiqul to answer (he's got all the details). Cheers, Michael From nobody Thu May 22 11:34:27 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B77261A01E5 for ; Thu, 22 May 2014 11:34:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wfqPMCXUTDlF for ; Thu, 22 May 2014 11:34:23 -0700 (PDT) Received: from mail-out2.uio.no (mail-out2.uio.no [IPv6:2001:700:100:10::58]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 579531A0290 for ; Thu, 22 May 2014 11:34:23 -0700 (PDT) Received: from mail-mx2.uio.no ([129.240.10.30]) by mail-out2.uio.no with esmtp (Exim 4.75) (envelope-from ) id 1WnXoq-0007IQ-KI; Thu, 22 May 2014 20:34:20 +0200 Received: from 158.6.202.84.customer.cdi.no ([84.202.6.158] helo=[192.168.0.103]) by mail-mx2.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user safiquli (Exim 4.80) (envelope-from ) id 1WnXol-0006Cj-82; Thu, 22 May 2014 20:34:20 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Safiqul Islam In-Reply-To: <39CD560F-5515-4265-A8C2-B1A7D141B158@ifi.uio.no> Date: Thu, 22 May 2014 20:34:14 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> <39CD560F-5515-4265-A8C2-B1A7D141B158@ifi.uio.no> To: Dave Taht X-Mailer: Apple Mail (2.1874) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 6 msgs/h 4 sum rcpts/h 6 sum msgs/h 4 total rcpts 1541 max rcpts/h 24 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 4AF3BF863CB01677C9124D4AE6FF18B068B55ACD X-UiO-SPAM-Test: UIO-GREYLIST remote_host: 84.202.6.158 spam_score: -49 maxlevel 80 minaction 1 bait 0 mail/h: 4 total 199 max/h 10 blacklist 0 greylist 1 ratelimit 0 X-UiOonly: 2D45325B48FE85BE7FDC727B551B99645AE4B437 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/BS-tdyjon3957jg2G51EORVc2ns Cc: rmcat WG , Michael Welzl Subject: Re: [rmcat] New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 18:34:25 -0000 Hello, Thanks for your questions.=20 Inline:=20 On 22. mai 2014, at 18:50, Michael Welzl wrote: > Hey, >=20 > Thanks for that - glad see some interest at last! >=20 > In line: >=20 > On 22. mai 2014, at 16:11, Dave Taht wrote: >=20 >> Is there some reason why you do not use "weighted fair queuing" as >> analogous to "controllable fairness with prioritization" or cite any >> of that liturature? >>=20 >> While being able to adjust the fairness dynamically is a good thing, >> the root technique is wfq, in my reading. >=20 > Well, wfq is about assigning a flow its weighted portion of the total = weight; this is what step (c) in our two active example algorithms does, = and it's also what wfq does. We didn't cite wfq because we don't think = of this as a scheduler, but since the active algorithm assigns rate = updates to all flows whenever any flow updates its rate, it's really = pretty much the same as a scheduler in fact. >=20 > That's all - no major other reasons. We can put a wfq citation in = there if that makes you happy :) >=20 >=20 >> re: >>=20 >> 1 This (13 packet queue length) is based on the bandwidth >> delay product (BDP). We repeated our tests with dif- >> ferent queue lengths and found no significant differences. >>=20 >> What different queue lengths? Against what competing traffic? >=20 > I'll leave that for Safiqul to answer (he's got all the details). We ran our experiments while varying the queue lengths from half a BDP = to one and a half a BDP in order to see if the effects were similar or = not. Flows were competing with each other, and there was no = cross-traffic. Since we saw no change, we didn't stretch to smaller or = larger values because they didn't seem to have an effect on our = algorithm (there will probably be lower and upper bounds where the = congestion controllers don't properly work anymore, but that's then a = specialty of the congestion controller, not our coupling algorithm). Cheers, Safiqul From nobody Fri May 23 01:28:37 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 582791A03BD for ; Fri, 23 May 2014 01:28:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n_jkk1ejMQUc for ; Fri, 23 May 2014 01:28:32 -0700 (PDT) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8E0781A0108 for ; Fri, 23 May 2014 01:28:32 -0700 (PDT) Received: by mail-we0-f182.google.com with SMTP id t60so4586915wes.27 for ; Fri, 23 May 2014 01:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=j6Sylojzko4tTl/AB3pT30WXEICQk5QFnVcUBoXUyqs=; b=j71cPTakY2v2Hd2J87st5RU5LfCcrxgglHb5Krs/ChqjpMWXuT2QilpWyESmmU4gpk eIvvzlbL36Zj+M/vij61QcdGXLCEkdBiWCofNIHA820/MHiA/XfqtkGQt+Mg7wME+jML gYLmGPD114MpSuU2Rm3t1INS6ka4jpSq2Q22gy0JzvBetve/mWglIv6t7xFl+9875H6E nAaa82sg7GnKRQxfuzb6JgknX17I2sNJ1kgVtXif18xTpD+2qNP0UYWAMK1RvUMP6T0c YkdhsVtj926yN4+GfCknbAdS4NnoWipZlZoAly6UQ7982XAQAmgj3Jc1MCsDt2T3Kp5B VQrA== X-Received: by 10.180.86.74 with SMTP id n10mr1732865wiz.55.1400833709631; Fri, 23 May 2014 01:28:29 -0700 (PDT) Received: from [192.168.1.33] (61.Red-88-9-161.dynamicIP.rima-tde.net. [88.9.161.61]) by mx.google.com with ESMTPSA id vc2sm2983516wjc.2.2014.05.23.01.28.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 23 May 2014 01:28:28 -0700 (PDT) Message-ID: <537F06AF.6010105@gmail.com> Date: Fri, 23 May 2014 10:28:31 +0200 From: Sergio Garcia Murillo User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: rmcat@ietf.org References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> <39CD560F-5515-4265-A8C2-B1A7D141B158@ifi.uio.no> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/T9NqMKP8zhmy-ZZ5cid1KUPM838 Subject: Re: [rmcat] New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 08:28:35 -0000 El 22/05/2014 20:34, Safiqul Islam escribió: > Hello, > > Thanks for your questions. > > Inline: > > > On 22. mai 2014, at 18:50, Michael Welzl wrote: > >> Hey, >> >> Thanks for that - glad see some interest at last! >> >> In line: >> >> On 22. mai 2014, at 16:11, Dave Taht wrote: >> >>> Is there some reason why you do not use "weighted fair queuing" as >>> analogous to "controllable fairness with prioritization" or cite any >>> of that liturature? >>> >>> While being able to adjust the fairness dynamically is a good thing, >>> the root technique is wfq, in my reading. >> Well, wfq is about assigning a flow its weighted portion of the total weight; this is what step (c) in our two active example algorithms does, and it's also what wfq does. We didn't cite wfq because we don't think of this as a scheduler, but since the active algorithm assigns rate updates to all flows whenever any flow updates its rate, it's really pretty much the same as a scheduler in fact. >> >> That's all - no major other reasons. We can put a wfq citation in there if that makes you happy :) >> >> >>> re: >>> >>> 1 This (13 packet queue length) is based on the bandwidth >>> delay product (BDP). We repeated our tests with dif- >>> ferent queue lengths and found no significant differences. >>> >>> What different queue lengths? Against what competing traffic? >> I'll leave that for Safiqul to answer (he's got all the details). > > We ran our experiments while varying the queue lengths from half a BDP to one and a half a BDP in order to see if the effects were similar or not. Flows were competing with each other, and there was no cross-traffic. Since we saw no change, we didn't stretch to smaller or larger values because they didn't seem to have an effect on our algorithm (there will probably be lower and upper bounds where the congestion controllers don't properly work anymore, but that's then a specialty of the congestion controller, not our coupling algorithm). > > Cheers, > Safiqul > > Hi Safiqul, May I ask you which tools are you using to simulate changing conditions? I would like to run also some experiments on the new congestion algorithms. Best regards Sergio From nobody Fri May 23 04:46:45 2014 Return-Path: X-Original-To: rmcat@ietfa.amsl.com Delivered-To: rmcat@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 41C501A015E for ; Fri, 23 May 2014 04:46:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M6VEVYqocwOv for ; Fri, 23 May 2014 04:46:42 -0700 (PDT) Received: from mail-out1.uio.no (mail-out1.uio.no [IPv6:2001:700:100:10::57]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 09D261A0177 for ; Fri, 23 May 2014 04:46:41 -0700 (PDT) Received: from mail-mx3.uio.no ([129.240.10.44]) by mail-out1.uio.no with esmtp (Exim 4.75) (envelope-from ) id 1Wnnvq-0003vb-II; Fri, 23 May 2014 13:46:38 +0200 Received: from 1x-193-157-199-214.uio.no ([193.157.199.214]) by mail-mx3.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user safiquli (Exim 4.80) (envelope-from ) id 1Wnnvq-0004QE-1s; Fri, 23 May 2014 13:46:38 +0200 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Safiqul Islam In-Reply-To: <537F06AF.6010105@gmail.com> Date: Fri, 23 May 2014 13:46:37 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140507110805.18496.70131.idtracker@ietfa.amsl.com> <39CD560F-5515-4265-A8C2-B1A7D141B158@ifi.uio.no> <537F06AF.6010105@gmail.com> To: Sergio Garcia Murillo X-Mailer: Apple Mail (2.1874) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 2 msgs/h 1 sum rcpts/h 5 sum msgs/h 3 total rcpts 1543 max rcpts/h 24 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-6.1, required=5.0, autolearn=disabled, RP_MATCHES_RCVD=-1.051, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 463E5037F6478197C8B45C6BAE9C895D9503A965 X-UiO-SPAM-Test: remote_host: 193.157.199.214 spam_score: -60 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 36 max/h 6 blacklist 0 greylist 0 ratelimit 0 X-UiOonly: 9428C1580002BDE9B65E99E5B9119B246E4E6CC5 Archived-At: http://mailarchive.ietf.org/arch/msg/rmcat/vZSemedLdGgMpIv51xrk_iuL8cQ Cc: rmcat@ietf.org Subject: Re: [rmcat] New Version Notification for draft-welzl-rmcat-coupled-cc-03.txt X-BeenThere: rmcat@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 11:46:44 -0000 Hello Sergio, We used ns-2 for simulations, and changed the parameters according to = our needs. Regards, Safiqul On 23. mai 2014, at 10:28, Sergio Garcia Murillo = wrote: > El 22/05/2014 20:34, Safiqul Islam escribi=F3: >> Hello, >>=20 >> Thanks for your questions. >>=20 >> Inline: >>=20 >>=20 >> On 22. mai 2014, at 18:50, Michael Welzl wrote: >>=20 >>> Hey, >>>=20 >>> Thanks for that - glad see some interest at last! >>>=20 >>> In line: >>>=20 >>> On 22. mai 2014, at 16:11, Dave Taht wrote: >>>=20 >>>> Is there some reason why you do not use "weighted fair queuing" as >>>> analogous to "controllable fairness with prioritization" or cite = any >>>> of that liturature? >>>>=20 >>>> While being able to adjust the fairness dynamically is a good = thing, >>>> the root technique is wfq, in my reading. >>> Well, wfq is about assigning a flow its weighted portion of the = total weight; this is what step (c) in our two active example algorithms = does, and it's also what wfq does. We didn't cite wfq because we don't = think of this as a scheduler, but since the active algorithm assigns = rate updates to all flows whenever any flow updates its rate, it's = really pretty much the same as a scheduler in fact. >>>=20 >>> That's all - no major other reasons. We can put a wfq citation in = there if that makes you happy :) >>>=20 >>>=20 >>>> re: >>>>=20 >>>> 1 This (13 packet queue length) is based on the bandwidth >>>> delay product (BDP). We repeated our tests with dif- >>>> ferent queue lengths and found no significant differences. >>>>=20 >>>> What different queue lengths? Against what competing traffic? >>> I'll leave that for Safiqul to answer (he's got all the details). >>=20 >> We ran our experiments while varying the queue lengths from half a = BDP to one and a half a BDP in order to see if the effects were similar = or not. Flows were competing with each other, and there was no = cross-traffic. Since we saw no change, we didn't stretch to smaller or = larger values because they didn't seem to have an effect on our = algorithm (there will probably be lower and upper bounds where the = congestion controllers don't properly work anymore, but that's then a = specialty of the congestion controller, not our coupling algorithm). >>=20 >> Cheers, >> Safiqul >>=20 >>=20 >=20 > Hi Safiqul, >=20 > May I ask you which tools are you using to simulate changing = conditions? I would like to run also some experiments on the new = congestion algorithms. >=20 > Best regards > Sergio >=20