From nobody Tue Apr 1 05:12:02 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93DF31A0687 for ; Tue, 1 Apr 2014 05:12:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AnxkBCR14YQ6 for ; Tue, 1 Apr 2014 05:11:58 -0700 (PDT) Received: from atl4mhob12.myregisteredsite.com (atl4mhob12.myregisteredsite.com [209.17.115.50]) by ietfa.amsl.com (Postfix) with ESMTP id 8DF101A0681 for ; Tue, 1 Apr 2014 05:11:58 -0700 (PDT) Received: from mailpod.hostingplatform.com ([10.30.71.203]) by atl4mhob12.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id s31CBsuY023915 for ; Tue, 1 Apr 2014 08:11:54 -0400 Received: (qmail 15271 invoked by uid 0); 1 Apr 2014 12:11:54 -0000 X-TCPREMOTEIP: 69.81.143.143 X-Authenticated-UID: wes@mti-systems.com Received: from unknown (HELO ?192.168.1.108?) (wes@mti-systems.com@69.81.143.143) by 0 with ESMTPA; 1 Apr 2014 12:11:53 -0000 Message-ID: <533AAD07.103@mti-systems.com> Date: Tue, 01 Apr 2014 08:11:51 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "aqm@ietf.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/x4Q9g0ddTm-qtNRpa4sHbEN3NUY Subject: [aqm] publishing algorithms X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 12:12:00 -0000 Hello AQMers. As chairs, Richard and I had been planning to let the evaluation guidelines converge and then use those to guide adoption of algorithm documents. However, we now think there may be value in not waiting so long, and getting some algorithm documents moving along more quickly. We hope you can provide some feedback on the plan below: 1) Starting soon, we may look to adopt a small number of algorithm drafts for Experimental, with the goal that by doing so, it will increase the number of eyeballs and independent reviews of them, and enhance the quality, since people may be implementing to the drafts in order to test using the evaluation guidelines. Each algorithm *must* clearly identify which types of use cases / scenarios it is targeted for. 2) Adoption of an algorithm spec as a working group draft will require working group consensus that the algorithm looks attractive to experiment with for the stated scenarios, and multiple parties will plan to be looking at it, testing, analyzing, providing feedback, etc. 3) The evaluation guidelines / scenarios drafts being worked on separately will guide the later selection of one or more Experimental algorithms to become Proposed Standards with applicability statements for the scenarios they have been evaluated in. We're interested to know if the working group thinks this sounds like a good idea, bad idea, or any other thoughts. -- Wes Eddy MTI Systems From nobody Tue Apr 1 05:16:23 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 95C5E1A069F for ; Tue, 1 Apr 2014 05:16:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gvOYNk_x2m8j for ; Tue, 1 Apr 2014 05:16:12 -0700 (PDT) Received: from mail-out5.uio.no (mail-out5.uio.no [IPv6:2001:700:100:10::17]) by ietfa.amsl.com (Postfix) with ESMTP id 1D34D1A068C for ; Tue, 1 Apr 2014 05:16:12 -0700 (PDT) Received: from mail-mx3.uio.no ([129.240.10.44]) by mail-out5.uio.no with esmtp (Exim 4.80.1) (envelope-from ) id 1WUxbs-0003Xb-AN; Tue, 01 Apr 2014 14:16:08 +0200 Received: from [77.40.182.242] (helo=[10.50.5.82]) by mail-mx3.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from ) id 1WUxbr-00072I-Tg; Tue, 01 Apr 2014 14:16:08 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Michael Welzl In-Reply-To: <533AAD07.103@mti-systems.com> Date: Tue, 1 Apr 2014 14:16:06 +0200 Content-Transfer-Encoding: 7bit Message-Id: <7DA128B6-6274-4B6A-AD38-A82D02F7F2A0@ifi.uio.no> References: <533AAD07.103@mti-systems.com> To: Wesley Eddy X-Mailer: Apple Mail (2.1874) X-UiO-SPF-Received: X-UiO-Ratelimit-Test: rcpts/h 8 msgs/h 5 sum rcpts/h 12 sum msgs/h 6 total rcpts 14379 max rcpts/h 44 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 8CDD64C3DF8644A79D09DFC2435427E027CAC29F X-UiO-SPAM-Test: remote_host: 77.40.182.242 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 7 total 578 max/h 36 blacklist 0 greylist 0 ratelimit 0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/gm34YAWYEv60nqE31ktocxAtIe8 Cc: "aqm@ietf.org" Subject: Re: [aqm] publishing algorithms X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 12:16:17 -0000 On 1. apr. 2014, at 14:11, Wesley Eddy wrote: > Hello AQMers. As chairs, Richard and I had been planning to let > the evaluation guidelines converge and then use those to guide > adoption of algorithm documents. > > However, we now think there may be value in not waiting so long, > and getting some algorithm documents moving along more quickly. > > We hope you can provide some feedback on the plan below: > > 1) Starting soon, we may look to adopt a small number of algorithm > drafts for Experimental, with the goal that by doing so, it will > increase the number of eyeballs and independent reviews of them, and > enhance the quality, since people may be implementing to the drafts > in order to test using the evaluation guidelines. Each algorithm > *must* clearly identify which types of use cases / scenarios it is > targeted for. > > 2) Adoption of an algorithm spec as a working group draft will require > working group consensus that the algorithm looks attractive to > experiment with for the stated scenarios, and multiple parties will > plan to be looking at it, testing, analyzing, providing feedback, > etc. > > 3) The evaluation guidelines / scenarios drafts being worked on > separately will guide the later selection of one or more Experimental > algorithms to become Proposed Standards with applicability > statements for the scenarios they have been evaluated in. > > We're interested to know if the working group thinks this sounds like > a good idea, bad idea, or any other thoughts. I think it sounds like a good idea! Nice to get stuff documented and somewhat stabilized soon, IMHO. Cheers, Michael From nobody Tue Apr 1 09:34:20 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2BE8F1A0564 for ; Tue, 1 Apr 2014 09:34:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -114.511 X-Spam-Level: X-Spam-Status: No, score=-114.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DveOhegtSCcZ for ; Tue, 1 Apr 2014 09:34:17 -0700 (PDT) Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) by ietfa.amsl.com (Postfix) with ESMTP id 1CB6B1A0505 for ; Tue, 1 Apr 2014 09:34:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3802; q=dns/txt; s=iport; t=1396370054; x=1397579654; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=D0Dvby/C+sWNwxzYqVsdzQTbpp+wHMfazuVYp3qQLxk=; b=hQ8RKbL9W9VauqzeFKqTkQuGbMVAHnPwxOU3JVAOwi79NazP42sztm9H obXG40gsXE5Wn+mG6VIzx4rLznb/a/3amNoSyIj8k8eoFhUQCeYzmakfR Gl6rCGdi8GTuoJoGRxLYO0od3hjZEudIFMPVNnrDJP+1jfGJag6YYsTmR 0=; X-Files: signature.asc : 195 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgUFANXpOlOtJA2I/2dsb2JhbABPCoMGO1e8EYc1gRwWdIIlAQEBAwEBAQFrCwULAgEIDjgnCyUCBA4FDodjCA3RPhMEjhRcB4MkgRQEkFuBNYJag2ySOYMwgis X-IronPort-AV: E=Sophos;i="4.97,773,1389744000"; d="asc'?scan'208";a="314385830" Received: from alln-core-3.cisco.com ([173.36.13.136]) by rcdn-iport-3.cisco.com with ESMTP; 01 Apr 2014 16:34:12 +0000 Received: from xhc-aln-x01.cisco.com (xhc-aln-x01.cisco.com [173.36.12.75]) by alln-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id s31GYAwU003571 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 1 Apr 2014 16:34:11 GMT Received: from xmb-rcd-x09.cisco.com ([169.254.9.247]) by xhc-aln-x01.cisco.com ([173.36.12.75]) with mapi id 14.03.0123.003; Tue, 1 Apr 2014 11:34:09 -0500 From: "Fred Baker (fred)" To: Wesley Eddy Thread-Topic: [aqm] publishing algorithms Thread-Index: AQHPTcgzDWOtWFDxvEO+Ms+2IiSinw== Date: Tue, 1 Apr 2014 16:34:07 +0000 Message-ID: References: <533AAD07.103@mti-systems.com> In-Reply-To: <533AAD07.103@mti-systems.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.19.64.115] Content-Type: multipart/signed; boundary="Apple-Mail=_0A47871C-D6D2-4DD1-AA50-F330D0B2CD22"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/tnIPQ0ocLQYioY9PYpVzB6NPHWw Cc: "aqm@ietf.org" Subject: Re: [aqm] publishing algorithms X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 16:34:19 -0000 --Apple-Mail=_0A47871C-D6D2-4DD1-AA50-F330D0B2CD22 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Makes sense to me. I do have one question. Per charter, in December we = are supposed to "Submit first algorithm specification to IESG for = publication as Proposed Standard=94. Would this be a change of direction = for the charter? Note that I=92m not pushing a given algorithm, nor am I convinced that = there should be exactly one. In protocol design, we are worried about = interoperability, and everyone has to implement the protocol the same = way. In AQM, the different algorithms, and the ones we think of next, = have to produce a specific drop or mark rate under a specified = circumstance (which might be about queue depth, latency in queue, or = rate through a queue), and the end systems need to respond to that = predictably. The means by which the mark or drop rate is established is = semi-irrelevant if the rate itself is maintained. So I=92m not exactly = sure what the terms =93Experimental=94 or =93Proposed Standard=94 mean = in the context and using the definitions in RFC 2026. It would be nice = if we had a status that said =93recommended for consideration for = operational use=94, and we could put that status on several that meet = our requirements, whatever we decide those are. On Apr 1, 2014, at 5:11 AM, Wesley Eddy wrote: > Hello AQMers. As chairs, Richard and I had been planning to let > the evaluation guidelines converge and then use those to guide > adoption of algorithm documents. >=20 > However, we now think there may be value in not waiting so long, > and getting some algorithm documents moving along more quickly. >=20 > We hope you can provide some feedback on the plan below: >=20 > 1) Starting soon, we may look to adopt a small number of algorithm > drafts for Experimental, with the goal that by doing so, it will > increase the number of eyeballs and independent reviews of them, and > enhance the quality, since people may be implementing to the drafts > in order to test using the evaluation guidelines. Each algorithm > *must* clearly identify which types of use cases / scenarios it is > targeted for. >=20 > 2) Adoption of an algorithm spec as a working group draft will require > working group consensus that the algorithm looks attractive to > experiment with for the stated scenarios, and multiple parties will > plan to be looking at it, testing, analyzing, providing feedback, > etc. >=20 > 3) The evaluation guidelines / scenarios drafts being worked on > separately will guide the later selection of one or more = Experimental > algorithms to become Proposed Standards with applicability > statements for the scenarios they have been evaluated in. >=20 > We're interested to know if the working group thinks this sounds like > a good idea, bad idea, or any other thoughts. >=20 > --=20 > Wes Eddy > MTI Systems >=20 > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm If at first the idea is not absurd, then there is no hope for it. =20 Albert Einstein --Apple-Mail=_0A47871C-D6D2-4DD1-AA50-F330D0B2CD22 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iD8DBQFTOup6bjEdbHIsm0MRAr92AJ0SljhGdIqEab63GYN8onXUACH3UQCgr7td rYbgynIIGy+nxUdJDzhMUfU= =sjsJ -----END PGP SIGNATURE----- --Apple-Mail=_0A47871C-D6D2-4DD1-AA50-F330D0B2CD22-- From nobody Tue Apr 1 10:07:44 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 879CF1A0998 for ; Tue, 1 Apr 2014 10:07:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A6JK_j7iUdZl for ; Tue, 1 Apr 2014 10:07:40 -0700 (PDT) Received: from atl4mhob15.myregisteredsite.com (atl4mhob15.myregisteredsite.com [209.17.115.53]) by ietfa.amsl.com (Postfix) with ESMTP id 9340F1A0564 for ; Tue, 1 Apr 2014 10:07:40 -0700 (PDT) Received: from mailpod.hostingplatform.com ([10.30.71.204]) by atl4mhob15.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id s31H7YX1026127 for ; Tue, 1 Apr 2014 13:07:34 -0400 Received: (qmail 27750 invoked by uid 0); 1 Apr 2014 17:07:34 -0000 X-TCPREMOTEIP: 69.81.143.143 X-Authenticated-UID: wes@mti-systems.com Received: from unknown (HELO ?192.168.1.108?) (wes@mti-systems.com@69.81.143.143) by 0 with ESMTPA; 1 Apr 2014 17:07:34 -0000 Message-ID: <533AF252.5060608@mti-systems.com> Date: Tue, 01 Apr 2014 13:07:30 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Fred Baker (fred)" References: <533AAD07.103@mti-systems.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/JSqLFu7P1umbWVHRn_c1prT_4cs Cc: "aqm@ietf.org" Subject: Re: [aqm] publishing algorithms X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 17:07:42 -0000 n 4/1/2014 12:34 PM, Fred Baker (fred) wrote: > Makes sense to me. I do have one question. Per charter, in December > we are supposed to "Submit first algorithm specification to IESG for > publication as Proposed Standard”. Would this be a change of > direction for the charter? Yes, it would be a shift in plans, and we'd have to make up some new milestone targets. That's why we're looking for feedback before doing it, because it only makes sense to do if the people actually doing the work will go along with it :). > Note that I’m not pushing a given algorithm, nor am I convinced that > there should be exactly one. In protocol design, we are worried about > interoperability, and everyone has to implement the protocol the same > way. In AQM, the different algorithms, and the ones we think of next, > have to produce a specific drop or mark rate under a specified > circumstance (which might be about queue depth, latency in queue, or > rate through a queue), and the end systems need to respond to that > predictably. The means by which the mark or drop rate is established > is semi-irrelevant if the rate itself is maintained. So I’m not > exactly sure what the terms “Experimental” or “Proposed Standard” > mean in the context and using the definitions in RFC 2026. It would > be nice if we had a status that said “recommended for consideration > for operational use”, and we could put that status on several that > meet our requirements, whatever we decide those are. I think (well, I hope) that pretty much everyone agrees on this. My personal thought is that the Experimental ones may have some warts or "unknowns", and that should be okay with us, as long as they seem to be promising and there is wide interest in using them and finding out more about how they work or how they can be tweaked further, but a baseline is needed/desirable for multiple people to work from. The Standards Track algorithm(s) should have substantially less warts or unknowns about them, and people should be able to put them in their code and products with strong confidence that: 1) they're implementing from an unambiguous specification 2) it will perform with well-understood results For instance, a hypothetical Algorithm X may have been beat to death by one set of folks for some particular use case like a home gateway cable router. They speculate that it will do well for some other scenarios too, and there are other people interested in implementing and trying it out over a longer term, but nobody is fully sure that it's absolutely the best algorithm, and maybe there are some downsides like a bit of minor tuning or a hidden variable that has to be tweaked for other scenarios. That sounds like a good candidate for Experimental to me. Maybe people will go play with it, and either learn good things and fix it up for Standards Track, or learn bad things and drop it or make it Historic. -- Wes Eddy MTI Systems From nobody Wed Apr 9 08:35:22 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 05FE31A0391 for ; Wed, 9 Apr 2014 08:35:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lfg-9X57B8r4 for ; Wed, 9 Apr 2014 08:35:18 -0700 (PDT) Received: from atl4mhob13.myregisteredsite.com (atl4mhob13.myregisteredsite.com [209.17.115.51]) by ietfa.amsl.com (Postfix) with ESMTP id 74A671A0395 for ; Wed, 9 Apr 2014 08:35:18 -0700 (PDT) Received: from mailpod.hostingplatform.com ([10.30.71.210]) by atl4mhob13.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id s39FZC3B007431 for ; Wed, 9 Apr 2014 11:35:12 -0400 Received: (qmail 10266 invoked by uid 0); 9 Apr 2014 15:35:09 -0000 X-TCPREMOTEIP: 107.46.89.155 X-Authenticated-UID: wes@mti-systems.com Received: from unknown (HELO ?192.168.43.65?) (wes@mti-systems.com@107.46.89.155) by 0 with ESMTPA; 9 Apr 2014 15:35:09 -0000 Message-ID: <534568A8.7010909@mti-systems.com> Date: Wed, 09 Apr 2014 11:35:04 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "aqm@ietf.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/59jtNVpNfWCrIIOpr-Xzxf3gISM Subject: [aqm] working group LAST CALL on recommendations draft X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 15:35:20 -0000 We didn't receive any comments yet on the updated recommendations draft, which we were trying to have a working group last call on per Richard's email to the list on 3/5. Since we think people might not have noticed the last call, we're re-announcing it. In the next two weeks, please review this document: https://datatracker.ietf.org/doc/draft-ietf-aqm-recommenda tion/ and relay any comments, questions, corrections, words of support, etc. to this AQM mailing list. Thanks for your help in finishing this document! -- Wes Eddy MTI Systems From nobody Wed Apr 9 10:33:14 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 734121A02C2 for ; Wed, 9 Apr 2014 10:33:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 05AmZU01qZHQ for ; Wed, 9 Apr 2014 10:33:07 -0700 (PDT) Received: from mail-we0-x22e.google.com (mail-we0-x22e.google.com [IPv6:2a00:1450:400c:c03::22e]) by ietfa.amsl.com (Postfix) with ESMTP id 79C801A029B for ; Wed, 9 Apr 2014 10:33:07 -0700 (PDT) Received: by mail-we0-f174.google.com with SMTP id t60so2839893wes.33 for ; Wed, 09 Apr 2014 10:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=pD0VciIhhhkazJ/QRjvj1elKK8dAmHOudnsyn34oF2k=; b=pJPOo8sz5VfCXr1Lpdf0Zt4gixjTI/q1okPzLCrbHUA6hRtLvBe4wTqYasbhBCTL+u gZ98s3aaLZCpXWuhrE/hyX+WUMx/kNnC7YwqU44B2O2w5Baa+B+fS1nv0NgoesHhR9sp h9ZzxNcJyTWcvpNODeKWcYEulFe21SSOo+XeP3VjLb1YJoRnlSnuim5DiJpdYeokTWpo 26ARr9ZYtphxGaGVSmDTsQpqU1f0Qb8+tg8Tg0PP6ipLr8gpXCtYS51gMQvQXGBUhR5A R/6jdLiNC2Arqw4DJwV9NHDLOgAPaxS9HhLHAsxpeB7IUuNODwP3kqWjDWRFJF2sWs6i XXsg== MIME-Version: 1.0 X-Received: by 10.180.19.167 with SMTP id g7mr5391010wie.46.1397064786263; Wed, 09 Apr 2014 10:33:06 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Wed, 9 Apr 2014 10:33:06 -0700 (PDT) In-Reply-To: <534568A8.7010909@mti-systems.com> References: <534568A8.7010909@mti-systems.com> Date: Wed, 9 Apr 2014 10:33:06 -0700 Message-ID: From: Dave Taht To: Wesley Eddy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/eDqqhsttaHX4ggjnQ-rGj8e4yow Cc: "aqm@ietf.org" Subject: Re: [aqm] working group LAST CALL on recommendations draft X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 17:33:12 -0000 I still don't support wglc. a) Nit: "Network Working Group"? b) I have given up on using the term "AQM" to describe anything other than Active Queue "Length" Management algorithms. What I wrote about "SQM" is mostly outside the scope of the AQM guidelines document, but it's here: http://www.bufferbloat.net/projects/cerowrt/wiki/Smart_Queue_Management but I can live with the broad definition as used in this document. c) I also don't just mean "fair or flow queuing" when I say packet scheduling, we've identified an elephant in the room as is actually rate limiting (tbf, htb), or hybrid rate limiting/scheduling (hfsc), or powerboost style rate limiting/expansion.... Moving on: d) "The traditional technique for managing the queue length in a network device is to set a maximum length (in terms of packets)" - well, bytes is common on many devices like dlsams and cmts and modems... substitute: packets or bytes e) " 2. Provide a lower-delay interactive service" I tend to regard dns traffic also as rather significant and telnet is rather obsolete. ssh. f) "3. Non-TCP-friendly Transport Protocols" There's also the problem of DDOS attacks. g) "Another topic requiring consideration is the appropriate granularity of a "flow" when considering a queue management method. There are a few "natural" answers: 1) a transport (e.g. TCP or UDP) flow (source address/port, destination address/port, Differentiated Services Code Point - DSCP); 2) a source/destination host pair (IP addresses, DSCP); 3) a given source host or a given destination host. We suggest that the source/destination host pair gives the most appropriate granularity in many circumstances. " I don't suggest the last, as I have no data that backs it up. and my request to include a 5 tuple address/port destination address/port and protocol isn't in here, although the MF classifier is mentioned later on in section 4.4. Elsewhere (in webrtc) there is an assumption that the 5 tuple is addr/port daddr/dport protocol. I don't actually think doing a 5 tuple including the dscp rather than the protocol is a very good idea given the amount of misclassified flows I see transiting site boundries. I do think moving out dscp flows into their own queues and then 5 tupling with protocol is not a bad idea. As for the final recomendations: h) " 4. AQM algorithms SHOULD respond to measured congestion, not application profiles." I'm not sure if this precludes active classification and optimization measu= res? http://www.smallnetbuilder.com/lanwan/lanwan-features/32297-does-qualcomms-= streamboost-really-work " Not all applications transmit packets of the same size. Although applications may be characterized by particular profiles of packet size this should not be used as the basis for AQM (see next section)." >From a packet scheduling perspective I strongly support using some differentiation based on packet size at low bandwidths. 300 bytes works well. for AQM, don't care, just care about latency no matter if it comes from a pps problem or a packet size problem. I didn't mind pie's increasing probability of a drop based on packet size (Which so far as I know is still in cablemodem pie, and: it helps on competing voip traff= ic) i) 4.5 might want to also mention more modern protocols like uTP and QUIC " In 2013, an obvious example of further research is the need to consider the use of Map/Reduce applications in data centers; do we need to extend our taxonomy of TCP/SCTP sessions to include not only "mice" and "elephants", but "lemmings". "Lemmings" are flash crowds of "mice" that the network inadvertently try to signal to as if they were elephant flows, resulting in head of line blocking in data center applications." I like to talk about ANTS. Can suggest some language if you want. On Wed, Apr 9, 2014 at 8:35 AM, Wesley Eddy wrote: > We didn't receive any comments yet on the updated recommendations > draft, which we were trying to have a working group last call on > per Richard's email to the list on 3/5. > > Since we think people might not have noticed the last call, we're > re-announcing it. > > In the next two weeks, please review this document: > > https://datatracker.ietf.org/doc/draft-ietf-aqm-recommenda > tion/ > > and relay any comments, questions, corrections, words of support, > etc. to this AQM mailing list. > > Thanks for your help in finishing this document! > > -- > Wes Eddy > MTI Systems > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --=20 Dave T=E4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Wed Apr 9 17:33:57 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 437AC1A04A1 for ; Wed, 9 Apr 2014 17:33:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.772 X-Spam-Level: X-Spam-Status: No, score=-1.772 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id meo4cZaTiqCl for ; Wed, 9 Apr 2014 17:33:51 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by ietfa.amsl.com (Postfix) with ESMTP id 7E0761A03B3 for ; Wed, 9 Apr 2014 17:33:51 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id 430B6C94C9; Wed, 9 Apr 2014 20:33:47 -0400 (EDT) Date: Wed, 9 Apr 2014 20:33:47 -0400 From: John Leslie To: Wesley Eddy Message-ID: <20140410003347.GQ39240@verdi> References: <534568A8.7010909@mti-systems.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <534568A8.7010909@mti-systems.com> User-Agent: Mutt/1.4.1i Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/_9x48PowjH_RrjiX4D9O4DwNV84 Cc: "aqm@ietf.org" Subject: Re: [aqm] working group LAST CALL on recommendations draft X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 00:33:56 -0000 Wesley Eddy wrote: > > We didn't receive any comments yet on the updated recommendations > draft, which we were trying to have a working group last call on > per Richard's email to the list on 3/5. Not sure how I missed that; but alas I did. > Since we think people might not have noticed the last call, we're > re-announcing it. > > In the next two weeks, please review this document: > > https://datatracker.ietf.org/doc/draft-ietf-aqm-recommenda > tion/ > > and relay any comments, questions, corrections, words of support, > etc. to this AQM mailing list. First pass comments: This document is not ready for WGLC. A number of suggestions which seemed headed towards consensus have not been integrated. The questions put to the WG have been too limited, and also too general. We have mostly responded to them as motherhood and apple pie, without enough discussion of the actual wording -- least of all seeking consensus on the underlying principles we're trying to state. We have a number of areas outside of Section 4 which deserve our consideration. The whole "TCP-friendly" terminology is so 1998. ;^) Any meaning it may have had in 1998 has dissipated beyond recovery. We _really_ should come up with another way to say what we mean. I don't see discussion of multiple queues for different flows, which IMHO is necessary for effective Active Queue Management. We haven't discussed the misinterpretation of "congestion collapse". Congestion collapse came from re-transmitting dropped packets. A "non-responsive" flow _cannot_ lead to congestion collapse unless it increases its sending rate in response to congestion (packet drop, usually). It certainly _can_ lead to Denial of Service, but not congestion collapse. For that matter, we don't discuss denial-of-service, which is a very real issue, except for a brief mention in Security Considerations. I don't expect to solve that issue in this document, but I do believe we can say something about how Active Queue Managment might evolve to alleviate denial-of-service problems. (They're alarmingly common today!) Personally, I am not willing to let this document go without _some_ treatment of the issues raised by Bob Briscoe's proposal for "Immediate ECN". I don't mean that we must rubber-stamp Bob's text -- goodness knows I disagree with him about half the time -- but _sometime_ we have to escape the totally irrational belief that ECN can be "the same as" packet drop. (I would suggest placing this in the section 4.2.) In Section 4.2, I have a problem with the glib way we call "delaying packets in flight" a reasonable congestion signal. While I do not deny that some transports use delay as a signal, it's a dreadful signal and should be discouraged for AQMs going forward. (It will still be there as an incidental indicator for a long time; and I don't mean to forbid its use -- merely to encourage other mechanisms instead.) I'd like to see a presentation of how the services expected of the Internet have evolved: that we are now dominated by interactive uses, from web-browsing all the way to real-time conferencing. This will make it clear _why_ it's important to have a way to minimize delay; and will inform discussion of how to make the Internet more responsive to web-browsing while at the same time serving real-time speech. As I said, these are first-pass comments. I'll stop here. -- John Leslie From nobody Tue Apr 15 01:24:04 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 98C831A03F1 for ; Tue, 15 Apr 2014 01:24:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.15 X-Spam-Level: * X-Spam-Status: No, score=1.15 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_FR=0.35, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zvc2_hV8ysQF for ; Tue, 15 Apr 2014 01:23:57 -0700 (PDT) Received: from zproxy220.enst-bretagne.fr (zproxy220.enst-bretagne.fr [192.108.117.9]) by ietfa.amsl.com (Postfix) with ESMTP id 30E371A0395 for ; Tue, 15 Apr 2014 01:23:57 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTP id 9706EB010C; Tue, 15 Apr 2014 10:23:53 +0200 (CEST) X-Virus-Scanned: amavisd-new at zproxy220.enst-bretagne.fr Received: from zproxy220.enst-bretagne.fr ([127.0.0.1]) by localhost (zproxy220.enst-bretagne.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4VVyLvUXcjml; Tue, 15 Apr 2014 10:23:49 +0200 (CEST) Received: from [IPv6:2001:660:7301:3728:4d4:757:d938:f985] (passerellev6v4.enst-bretagne.fr [192.108.117.5]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTPSA id E2713B010E; Tue, 15 Apr 2014 10:23:49 +0200 (CEST) Content-Type: multipart/alternative; boundary="Apple-Mail=_AA9433B7-CC64-4854-BD29-7DF263707D54" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) From: Nicolas KUHN In-Reply-To: Date: Tue, 15 Apr 2014 10:23:49 +0200 Message-Id: <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> To: "Akhtar, Shahid (Shahid)" X-Mailer: Apple Mail (2.1503) Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/33aiciPzOZ_QS3Wy_3f3nKGco8E Cc: Richard Scheffenegger , "aqm@ietf.org" Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 08:24:01 -0000 --Apple-Mail=_AA9433B7-CC64-4854-BD29-7DF263707D54 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Shahid Akhtar, all, After the discussions in London IETF 89, there has been lots of = modifications to the draft organisation (as we already advertised on the = list).=20 The authors and I just discussed your points considering the size of the = files for the bulk transfer and the HTTP traffic.=20 In the current version of the draft, we consider the values mentioned in = the cablelabs work [0]. As a result, we consider bulk TCP transfers (repeating 5MB file = transmission) and realistic HTTP web traffic (repeated download of = 700kB). As a reminder, please find here the comments of Shahid Akhtar = regarding these values: > "Bulk TCP transfer: (continuous file transmission, or repeating 5MB = file transmission);" > The largest type of traffic on the Internet that uses bulk TCP - or = large files via TCP is progressive download. The average speed of a = Youtube video is over 3Mbps and it is on average 4.5 minutes long - so = the average file size is considerably larger than 5MB. We have used an = average of 38MB assuming 480p video and 4.5 minutes for some of our = experiments. The inter-arrival time of these files should have a = realistic random distribution as well as size of the files.=20 > "Realistic HTTP web traffic (repeated download of 700kB);" > According to statistics that Google collected from 2010 about web = traffic, the average GET file size for web traffic was only 7KB. It is = very important to have the size of these files correct since AQMs can = help transfers of small files when large file downloads are also = occurring. The effect of AQM on a 7KB file would be very different than = on a 700KB file. Web traffic also has a complex inter-arrival time = relationship between the GET requests as well as the pages that produce = these GET requests. However to make the testing simpler, a repeated = download of a 7KB file with a Pareto distributed size and Pareto = distributed inter-arrival time may be an OK compromise. We expect to stick with the cable labs values [0]. Please let us know if = you have any comments regarding this choice.=20 Kind regards,=20 Nicolas KUHN [0] = http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Managemen= t_Algorithms_DOCSIS_3_0.pdf On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid (Shahid)" = wrote: > Hi Nicholas and other authors, >=20 > Thank you for producing this document. I had some (long) comments = where I have also suggested some solutions/text, hopefully you find them = helpful. When quoting from the document, I have used quotes and I have = listed my comments as 1-4. >=20 >=20 > 1. The number of tests recommended per AQM scheme seems to be quite = large as well as many tests that may be best done together are done = independently. >=20 > For example, there are different tests for fairness between flows = (section 3.2.3), level of traffic (mild, medium, heavy) (section 3.2.4), = different types of bursty traffic (3.2.2) and different network = environments (section 3.3). In order to obtain a realistic estimate of = AQM comparison, many of these effects may need to be tested together. = For example to increase level of traffic (in section 3.2.4) only long = term TCP flows of 50s are proposed. It would be better to test the level = of congestion with realistic traffic (and with realistic proportions) as = proposed in section 3.2.4. Another example would be to test fairness = between flows of same type - e.g. long term TCP flows when multiple = types of TCP flows are present at a AQM controlled buffer or test = fairness between flows of different types - e.g. long term TCP flows and = web flows.=20 >=20 > I noticed that in section 3.3 different network environments are = mentioned. I suggest that tests with appropriate realistic traffic be = done on each of these enviroments with various levels of congestion. One = could measure fairness from these tests without the need for additional = tests. This may have the benefit of reducing overall number of tests as = well. >=20 >=20 > 2. There are four network enviroments mentioned in the document = (section 3.3): > Wifi - in home network. > Data-Center communications. > BB wireless/satellite. > Low and high buffers >=20 > This is a good set, however, recommed to add: >=20 > Access networks - and further split access networks into DSL/FTTH = networks and Cable access networks. Cable access networks may have much = larger number of flows than DSL/FTTH since they are shared by a large = number of customers >=20 > There is another type of network environment - narrowband wireless = (mobile) networks where the AQM could be placed on the eNodeB, but = perhaps this is a complicated scenario, and best left for later or in = another document. >=20 >=20 > 3. I noticed that the modeling of different types of realistic traffic = could be made more accurate (section 3.2.2) >=20 > "Bulk TCP transfer: (continuous file transmission, or repeating 5MB = file transmission);" > The largest type of traffic on the Internet that uses bulk TCP - or = large files via TCP is progressive download. The average speed of a = Youtube video is over 3Mbps and it is on average 4.5 minutes long - so = the average file size is considerably larger than 5MB. We have used an = average of 38MB assuming 480p video and 4.5 minutes for some of our = experiments. The inter-arrival time of these files should have a = realistic random distribution as well as size of the files.=20 >=20 > "Realistic HTTP web traffic (repeated download of 700kB);" > According to statistics that Google collected from 2010 about web = traffic, the average GET file size for web traffic was only 7KB. It is = very important to have the size of these files correct since AQMs can = help transfers of small files when large file downloads are also = occurring. The effect of AQM on a 7KB file would be very different than = on a 700KB file. Web traffic also has a complex inter-arrival time = relationship between the GET requests as well as the pages that produce = these GET requests. However to make the testing simpler, a repeated = download of a 7KB file with a Pareto distributed size and Pareto = distributed inter-arrival time may be an OK compromise. >=20 > I noticed that adaptive streaming is not modelled as a type of = realistic traffic. It is probably the most important and largest flow on = the Internet today - if we look at a recent Sandvine report of Internet = traffic, about 75% of Internet traffic is real-time entertainment of = which the largest component is adaptive streaming based video traffic. = In order to simulate adaptive streaming traffic, one would need a client = that changes video quality rates. To keep testing simpler, an option may = be to generate realistic chunk sized files at regular intervals. Average = netflix speeds are around 2Mbps. Typical chunk sizes range from 2s to = 10s, but 4s may be a good average. The chunk sizes should be varied with = some random distribution as is typically observed from real codecs.=20 >=20 >=20 > 4. In section 2.2.5, QoE metrics are mentioned. It may be valuable to = spell out some standard models or expressions to derive QoE from network = parameters so that these QoE can be compared between AQMs. In the = literature there is work that derives closed form expressions for QoE. = The following are examples and suggestions: >=20 > Video - the work by Johan de Vriendt et al , "Model for estimating QoE = of Video delivered using HTTP Adaptive Streaming", IFIP/IEEE Workshop on = QoE Centric Management, May 2013 may be useful here. The authors used = over 500 samples to derive their a closed form expressions. The authors = derive the MOS =3D alpha*(average bit rate) + beta*(standard deviation = of bit rates) + sigma*(no of bit rate switches) + gamma and then produce = good values for alpha, beta, sigma and gamma. >=20 > Voice - The work in "Voice quality prediction models and their = application in VoIP networks" by L. Sun et al has closed form = expressions for MOS scores for most commercial codes. They derive a long = closed form expression with parameters a-j for key codecs (and then = provide a-j for most commercial codecs) based on end-to-end and packet = loss. =20 >=20 > Web - The delay for one average web GET file could be used as the = representative for the page load time >=20 > Key metric for AQMs on access networks may be QoE since it provides = and end-user view. We have seen that often the absolute goodput does not = matter as much as the stability of the flow for Video (which tends to be = the dominant traffic). Larger goodput may provide higher VQ levels for = Video, but if higher volatility is also produced then that higher VQ = level may not produce a higher QoE. >=20 > On the other hand, best metrics inter data center traffic may be = goodput and delay since most of that traffic is likely to be inter = machine traffic. >=20 > Best regards, >=20 > Shahid Akhtar. >=20 > -----Original Message----- > From: aqm [mailto:aqm-bounces@ietf.org] On Behalf Of Nicolas KUHN > Sent: Friday, February 07, 2014 5:58 AM > To: Richard Scheffenegger > Cc: aqm@ietf.org > Subject: [aqm] [AQM Evaluation Guidelines] >=20 > Dear all,=20 >=20 > On the behalf of the contributors to the AQM Evaluation Guidelines, I = encourage active discussion on the draft that we have written.=20 > I just submitted the draft as an individual draft to the I-D = Submission Tool [0]. >=20 > Please let us know what you think of the current document.=20 >=20 > Kind regards,=20 >=20 > Nicolas KUHN >=20 > [0] http://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/ >=20 >=20 > On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" = wrote: >=20 >> Hi, >>=20 >> a new month, a new status report. >>=20 >> First of all, Wes and I as chairs would like to thank the editors who = have stepped forward to work on the AQM Evaluation Guideline draft. We = are really thankful for their burst of efforts in the last couple weeks! >>=20 >> We expect that that a document will be ready for submission into the = I-D repository well before cutoff, as an individual draft, so that the = WG can see what the state of thinking is currently. Also, once the = document is published on datatracker we'd like to encourage active = discussion on it. >>=20 >> If the submitted -00 draft is felt to have a fairly complete outline = of what the current thinking in the WG is, we may be able to ask the WG = for formal adoption during this IETF meeting, or shortly thereafter. >>=20 >>=20 >>=20 >>=20 >> - WG Milestones: >> - Submit AQM recommendations to IESG for publication, obsoleting RFC >> 2309 (Goal: January 2014) >> - draft-ietf-aqm-recommendation is accepted towards this milestone >> - http://datatracker.ietf.org/doc/draft-ietf-aqm-recommendation/ >> - the draft has been updated per comments received >> - if the authors are comfortable, a WGLC might be made on the next=20= >> revision >> - we would like to hear from other authors of RFC 2309 on this=20 >> document, if anyone has contacts to them. >>=20 >>=20 >> - Submit AQM algorithm evaluation guidelines to IESG for publication=20= >> as Informational (Goal: July 2014) >> - An editor team has come forth and is working on this >> - A draft should be available for discussion in the London IETF >> - We encourage discussion on the list and during the meeting, if = this=20 >> draft should be adopted by the working group. >>=20 >> - Submit first algorithm specification to IESG for publication as=20 >> Proposed Standard (Goal: December 2014) >> - Since any Proposed Standard algorithm should be in line with the=20= >> recommendations and be passable versus the evaluation guidelines, = this=20 >> milestone is dependend on the progress of the two work items above. >> - Currently the only algorithm spec with a complete and active=20 >> individual-submission draft is PIE >>=20 >> - Other items: >> - draft-pan-aqm-pie is under active work as a proposed algorithm: >> http://tools.ietf.org/id/draft-pan-aqm-pie-00.txt, however the = draft >> has expired and should be refreshed. >> - draft-nichols-tsvwg-codel is expired; Dave Taht or others may=20 >> revive it and/or describe pairing with FQ/SFQ algorithms: >> http://tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt >> - Other algorithm specifications are welcome! >> - Though, we are not planning on adopting algorithms until=20 >> recommendations and evaluation guidelines are mostly stable >>=20 >>=20 >> Richard Scheffenegger >>=20 >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm >=20 > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --Apple-Mail=_AA9433B7-CC64-4854-BD29-7DF263707D54 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi = Shahid Akhtar, all,

After the discussions in London = IETF 89, there has been lots of modifications to the draft organisation = (as we already advertised on the list). 
The authors and = I just discussed your points considering the size of the files for the = bulk transfer and the HTTP traffic. 
In the current = version of the draft, we consider the values mentioned in the = cablelabs work [0].

As a result, we consider = bulk TCP transfers (repeating 5MB file transmission) and realistic HTTP = web traffic (repeated download of 700kB). As a reminder, please find = here the comments of Shahid Akhtar regarding these = values:

"Bulk TCP = transfer: (continuous file transmission, or repeating 5MB file = transmission);"
The largest type of traffic on the Internet that uses = bulk TCP - or large files via TCP is progressive download. The average = speed of a Youtube video is over 3Mbps and it is on average 4.5 minutes = long - so the average file size is considerably larger than 5MB. We have = used an average of 38MB assuming 480p video and 4.5 minutes for some of = our experiments. The inter-arrival time of these files should have a = realistic random distribution as well as size of the = files. 

"Realistic HTTP web traffic (repeated download of = 700kB);"
According to statistics that Google collected from 2010 = about web traffic, the average GET file size for web traffic was only = 7KB. It is very important to have the size of these files correct since = AQMs can help transfers of small files when large file downloads are = also occurring. The effect of AQM on a 7KB file would be very different = than on a 700KB file. Web traffic also has a complex inter-arrival time = relationship between the GET requests as well as the pages that produce = these GET requests. However to make the testing simpler, a repeated = download of a 7KB file with a Pareto distributed size and Pareto = distributed inter-arrival time may be an OK = compromise.

We expect to stick = with the cable labs values [0]. Please let us know if you have any = comments regarding this choice. 

Kind = regards, 

Nicolas = KUHN

=

On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid = (Shahid)" <shahid.akhtar@alcatel-luc= ent.com> wrote:

Hi = Nicholas and other authors,

Thank you for producing this = document. I had some (long) comments where I have also suggested some = solutions/text, hopefully you find them helpful. When quoting from the = document, I have used quotes and I have listed my comments as = 1-4.


1. The number of tests recommended per AQM scheme seems = to be quite large as well as many tests that may be best done together = are done independently.

For example, there are different tests = for fairness between flows (section 3.2.3), level of traffic (mild, = medium, heavy) (section 3.2.4), different types of bursty traffic = (3.2.2) and different network environments (section 3.3). In order to = obtain a realistic estimate of AQM comparison, many of these effects may = need to be tested together. For example to increase level of traffic (in = section 3.2.4) only long term TCP flows of 50s are proposed. It would be = better to test the level of congestion with realistic traffic (and with = realistic proportions) as proposed in section 3.2.4. Another example = would be to test fairness between flows of same type - e.g. long term = TCP flows when multiple types of TCP flows are present at a AQM = controlled buffer or test fairness between flows of different types - = e.g. long term TCP flows and web flows.

I noticed that in = section 3.3 different network environments are mentioned. I suggest that = tests with appropriate realistic traffic be done on each of these = enviroments with various levels of congestion. One could measure = fairness from these tests without the need for additional tests. This = may have the benefit of reducing overall number of tests as = well.


2. There are four network enviroments mentioned in the = document (section 3.3):
Wifi - in home network.
Data-Center = communications.
BB wireless/satellite.
Low and high = buffers

This is a good set, however, recommed to = add:

Access networks - and further split access networks into = DSL/FTTH networks and Cable access networks. Cable access networks may = have much larger number of flows than DSL/FTTH since they are shared by = a large number of customers

There is another type of network = environment - narrowband wireless (mobile) networks where the AQM could = be placed on the eNodeB, but perhaps this is a complicated scenario, and = best left for later or in another document.


3. I noticed that = the modeling of different types of realistic traffic could be made more = accurate (section 3.2.2)

"Bulk TCP transfer: (continuous file = transmission, or repeating 5MB file transmission);"
The largest type = of traffic on the Internet that uses bulk TCP - or large files via TCP = is progressive download. The average speed of a Youtube video is over = 3Mbps and it is on average 4.5 minutes long - so the average file size = is considerably larger than 5MB. We have used an average of 38MB = assuming 480p video and 4.5 minutes for some of our experiments. The = inter-arrival time of these files should have a realistic random = distribution as well as size of the files.

"Realistic HTTP web = traffic (repeated download of 700kB);"
According to statistics that = Google collected from 2010 about web traffic, the average GET file size = for web traffic was only 7KB. It is very important to have the size of = these files correct since AQMs can help transfers of small files when = large file downloads are also occurring. The effect of AQM on a 7KB file = would be very different than on a 700KB file. Web traffic also has a = complex inter-arrival time relationship between the GET requests as well = as the pages that produce these GET requests. However to make the = testing simpler, a repeated download of a 7KB file with a Pareto = distributed size and Pareto distributed inter-arrival time may be an OK = compromise.

I noticed that adaptive streaming is not modelled as = a type of realistic traffic. It is probably the most important and = largest flow on the Internet today - if we look at a recent Sandvine = report of Internet traffic, about 75% of Internet traffic is real-time = entertainment of which the largest component is adaptive streaming based = video traffic. In order to simulate adaptive streaming traffic, one = would need a client that changes video quality rates. To keep testing = simpler, an option may be to generate realistic chunk sized files at = regular intervals. Average netflix speeds are around 2Mbps. Typical = chunk sizes range from 2s to 10s, but 4s may be a good average. The = chunk sizes should be varied with some random distribution as is = typically observed from real codecs.


4. In section 2.2.5, = QoE metrics are mentioned. It may be valuable to spell out some standard = models or expressions to derive QoE from network parameters so that = these QoE can be compared between AQMs. In the literature there is work = that derives closed form expressions for QoE. The following are examples = and suggestions:

Video - the work by Johan de Vriendt et al , = "Model for estimating QoE of Video delivered using HTTP Adaptive = Streaming", IFIP/IEEE Workshop on QoE Centric Management, May 2013 may = be useful here. The authors used over 500 samples to derive their a = closed form expressions. The authors derive the MOS =3D alpha*(average = bit rate) + beta*(standard deviation of bit rates) + sigma*(no of bit = rate switches) + gamma and then produce good values for alpha, beta, = sigma and gamma.

Voice - The work in "Voice quality prediction = models and their application in VoIP networks" by L. Sun et al has = closed form expressions for MOS scores for most commercial codes. They = derive a long closed form expression with parameters a-j for key codecs = (and then provide a-j for most commercial codecs) based on end-to-end = and packet loss.  

Web - The delay for one average web GET = file could be used as the representative for the page load = time

Key metric for AQMs on access networks may be QoE since it = provides and end-user view. We have seen that often the absolute goodput = does not matter as much as the stability of the flow for Video (which = tends to be the dominant traffic). Larger goodput may provide higher VQ = levels for Video, but if higher volatility is also produced then that = higher VQ level may not produce a higher QoE.

On the other hand, = best metrics inter data center traffic may be goodput and delay since = most of that traffic is likely to be inter machine traffic.

Best = regards,

Shahid Akhtar.

-----Original = Message-----
From: aqm [mailto:aqm-bounces@ietf.org] On Behalf Of = Nicolas KUHN
Sent: Friday, February 07, 2014 5:58 AM
To: Richard = Scheffenegger
Cc: aqm@ietf.org
Subject: [aqm] [AQM = Evaluation Guidelines]

Dear all,

On the behalf of the = contributors to the AQM Evaluation Guidelines, I encourage active = discussion on the draft that we have written.
I just submitted the = draft as an individual draft to the I-D Submission Tool = [0].

Please let us know what you think of the current document. =

Kind regards,

Nicolas KUHN

[0] h= ttp://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/

=
On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" <rs@netapp.com> = wrote:

Hi,

a new month, a new = status report.

First of all, Wes and I as chairs would like to = thank the editors who have stepped forward to work on the AQM Evaluation = Guideline draft. We are really thankful for their burst of efforts in = the last couple weeks!

We expect that that a document will be = ready for submission into the I-D repository well before cutoff, as an = individual draft, so that the WG can see what the state of thinking is = currently. Also, once the document is published on datatracker we'd like = to encourage active discussion on it.

If the submitted -00 draft = is felt to have a fairly complete outline of what the current thinking = in the WG is, we may be able to ask the WG for formal adoption during = this IETF meeting, or shortly thereafter.




- WG = Milestones:
 - Submit AQM recommendations to IESG for = publication, obsoleting RFC
2309 (Goal: January 2014)
=    - draft-ietf-aqm-recommendation is accepted towards = this milestone
   - ht= tp://datatracker.ietf.org/doc/draft-ietf-aqm-recommendation/
=    - the draft has been updated per comments received
=    - if the authors are comfortable, a WGLC might be made = on the next
revision
   - we would like to hear = from other authors of RFC 2309 on this
document, if anyone has = contacts to them.


 - Submit AQM algorithm evaluation = guidelines to IESG for publication
as Informational (Goal: July = 2014)
   - An editor team has come forth and is = working on this
   - A draft should be available for = discussion in the London IETF
   - We encourage = discussion on the list and during the meeting, if this
=       draft should be adopted by the = working group.

 - Submit first algorithm specification to = IESG for publication as
Proposed Standard (Goal: December 2014)
=    - Since any Proposed Standard algorithm should be in = line with the
recommendations and be passable versus the evaluation = guidelines, this
milestone is dependend on the progress of the two = work items above.
   - Currently the only algorithm = spec with a complete and active
individual-submission draft is = PIE

- Other items:
 - draft-pan-aqm-pie is under active = work as a proposed algorithm:
   http://tools.ie= tf.org/id/draft-pan-aqm-pie-00.txt, however the draft
=    has expired and should be refreshed.
 - = draft-nichols-tsvwg-codel is expired; Dave Taht or others may
revive = it and/or describe pairing with FQ/SFQ algorithms:
=    http://= tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt
 - Other = algorithm specifications are welcome!
   - Though, we = are not planning on adopting algorithms until
recommendations and = evaluation guidelines are mostly stable


Richard = Scheffenegger

_______________________________________________
aq= m mailing list
aqm@ietf.org
https://www.ietf.org/mail= man/listinfo/aqm

_____________________________________= __________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mail= man/listinfo/aqm

= --Apple-Mail=_AA9433B7-CC64-4854-BD29-7DF263707D54-- From nobody Tue Apr 15 03:28:44 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 25C8D1A03AE for ; Tue, 15 Apr 2014 03:28:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.726 X-Spam-Level: ** X-Spam-Status: No, score=2.726 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_DK=1.009, MIME_8BIT_HEADER=0.3, MIME_HEADER_CTYPE_ONLY=0.717] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0m_9s8fKAi20 for ; Tue, 15 Apr 2014 03:28:38 -0700 (PDT) Received: from mail2.tohojo.dk (mail2.tohojo.dk [IPv6:2a01:4f8:200:3141::101]) by ietfa.amsl.com (Postfix) with ESMTP id 48B971A02A7 for ; Tue, 15 Apr 2014 03:28:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at example.com Received: by alrua-kau.localdomain (Postfix, from userid 1000) id 07DE0BE19D; Tue, 15 Apr 2014 12:28:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toke.dk; s=201310; t=1397557706; bh=8c26BymewWAJyh2qu0yg5x4EAu1/nHvMl1pScElqsvI=; h=From:To:Cc:Subject:References:Date:In-Reply-To; b=rvl8VGpnbYgiWa/xXlIARHWXZCpShzKABJzJmSAqIMVjmVCcMybkFIudAKoe0Ddly mqAcdt1/QJEIcAHUo4KxmUs8WYNZ4tUyjud/ZyyyO6tFih0jwahaTQvpOmdww1+IAH +GKFuzmu9bMJ0TG3FwtiAdvfZJi8vMoZFEDjn0I8= From: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: Nicolas KUHN References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> Date: Tue, 15 Apr 2014 12:28:23 +0200 In-Reply-To: <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> (Nicolas KUHN's message of "Tue, 15 Apr 2014 10:23:49 +0200") Message-ID: <878ur6vqd4.fsf@toke.dk> Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/K6KC0qlcT2Tivqtxfg3X1JMx7Ms Cc: "Akhtar, Shahid \(Shahid\)" , "aqm@ietf.org" , Richard Scheffenegger Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 10:28:43 -0000 --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Nicolas KUHN writes: > and realistic HTTP web traffic (repeated download of 700kB). As a reminde= r, > please find here the comments of Shahid Akhtar regarding these values: The Cablelabs work doesn't specify web traffic as simply "repeated downloads of 700KB", though. Quoting from [0], the actual wording is: > "Webs" indicates the number of simultaneous web users (repeated > downloads of a 700 kB page as described in Appendix A of [White]), Where [White] refers to [1] which states (in the Appendix): > The file sizes are generated via a log-normal distribution, such that > the log10 of file size is drawn from a normal distribution with mean =3D > 3.34 and standard deviation =3D 0.84. The file sizes (yi) are calculated > from the resulting 100 draws (xi ) using the following formula, in > order to produce a set of 100 files whose total size =3D~ 600 kB (614400 > B): And in the main text it specifies (in section 3.2.3) the actual model for the web traffic used: > Model single user web page download as follows: > > - Web page modeled as single HTML page + 100 objects spread evenly > across 4 servers. Web object sizes are currently fixed at 25 kB each, > whereas the initial HTML page is 100 kB. Appendix A provides an > alternative page model that may be explored in future work. >=20 > - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms). >=20 > - Initial HTTP GET to retrieve a moderately sized object (100 kB HTML > page) from server 1. >=20 > - Once initial HTTP GET completes, initiate 24 simultaneous HTTP GETs > (via separate TCP connections), 6 connections each to 4 different > server nodes >=20 > - Once each individual HTTP GET completes, initiate a subsequent GET > to the same server, until 25 objects have been retrieved from each > server. Which is a pretty far cry from just saying "repeated downloads of 700 KB" and, while still somewhat bigger, matches the numbers from Google better in terms of distribution between page sizes and other objects. And, more importantly, it features the kind of parallelism and interactions that a real web browser does; which, as Shahid mentioned is (can be) quite important for the treatment it receives by an AQM. As such I would be strongly in favour of changing the draft to actually describe realistic web client behaviour, rather than just summarising it as "repeated downloads of 700KB". =2DToke [0] http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Manage= ment_Algorithms_DOCSIS_3_0.pdf [1] http://www.cablelabs.com/downloads/pubs/PreliminaryStudyOfCoDelAQM_DOCS= ISNetwork.pdf [2] https://developers.google.com/speed/articles/web-metrics --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBCAAGBQJTTQnHAAoJEENeEGz1+utPSkMH/0lGLjgjUbf4/BHF+6zwFH/i mlseWdn/b1NHb5GUozRwDFWolWZmvYhXxsDaFLSutvj2KSyC1EA9TZWUamrZia3X TvFVfLmWguPB3bTKqwxl0lcHmBplljm5ifnh6O45wivUKaID0omLmmmwmdyLz3vR K8aNeN3zeDQi2ZQ+6bpJ1QPbOc1wkwMXbpzwASVp1+KuCgqFVAevTnnYYQxTlHDc 9KDWL1AamtkBKAUpTf2dUh/rmPchI003Z3QPjsMff4UYRB30QubONTMPFn+VzGOo hj8FxdeD5Akn78/Uu9ED73FDVtiWXaTl/jYifRIt6fXGcjHgyHd/RicQ7uMWZkc= =IuSj -----END PGP SIGNATURE----- --=-=-=-- From nobody Tue Apr 15 06:57:38 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 281621A0312 for ; Tue, 15 Apr 2014 06:57:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.449 X-Spam-Level: * X-Spam-Status: No, score=1.449 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_FR=0.35, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q0yV8Tsiv6ye for ; Tue, 15 Apr 2014 06:57:28 -0700 (PDT) Received: from zproxy210.enst-bretagne.fr (zproxy210.enst-bretagne.fr [192.108.117.8]) by ietfa.amsl.com (Postfix) with ESMTP id A246C1A07B7 for ; Tue, 15 Apr 2014 06:57:26 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zproxy210.enst-bretagne.fr (Postfix) with ESMTP id 8BB506601A; Tue, 15 Apr 2014 15:57:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at zproxy210.enst-bretagne.fr Received: from zproxy210.enst-bretagne.fr ([127.0.0.1]) by localhost (zproxy210.enst-bretagne.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VykpMYQnKlt4; Tue, 15 Apr 2014 15:57:20 +0200 (CEST) Received: from [IPv6:2001:660:7301:3728:d9d2:9ee8:f972:d4c0] (passerellev6v4.enst-bretagne.fr [192.108.117.5]) by zproxy210.enst-bretagne.fr (Postfix) with ESMTPSA id EF18766019; Tue, 15 Apr 2014 15:57:19 +0200 (CEST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) From: Nicolas KUHN In-Reply-To: <878ur6vqd4.fsf@toke.dk> Date: Tue, 15 Apr 2014 15:57:19 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <2A8C5B3D-CD2C-4378-ABBC-AEE3678FD2F7@telecom-bretagne.eu> References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> <878ur6vqd4.fsf@toke.dk> To: =?iso-8859-1?Q?Toke_H=F8iland-J=F8rgensen?= X-Mailer: Apple Mail (2.1503) Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/yhUTUdkU6Bvvit44OT820BWOP2Q Cc: "Akhtar, Shahid \(Shahid\)" , "aqm@ietf.org" , Richard Scheffenegger Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 13:57:33 -0000 Thank you for detailing the content of the Cable Labs document and where = these 700kB come from.=20 Concerning your last point: > As such I would be strongly in favour of changing the draft to = actually > describe realistic web client behaviour, rather than just summarising = it > as "repeated downloads of 700KB". I understand that it may be a drastic simplification to just summarise = the web client behaviour as only repeated downloads of 700kB. However, = the draft may not detail realistic web client behaviour: I believe that = it may be out of topic and the draft cannot contain such level of = complexity for all the covered protocols/traffic.=20 I propose that the following changes:=20 Was:=20 - Realistic HTTP web traffic (repeated download of 700kB); Changed by: - Realistic HTTP web page downloads: the tester should at least = consider repeated downloads of 700kB - for a more accurate web traffic, = a single user web page download [White] may exploited; What do you think ? Regards,=20 Nicolas On Apr 15, 2014, at 12:28 PM, Toke H=F8iland-J=F8rgensen = wrote: > Nicolas KUHN writes: >=20 >> and realistic HTTP web traffic (repeated download of 700kB). As a = reminder, >> please find here the comments of Shahid Akhtar regarding these = values: >=20 > The Cablelabs work doesn't specify web traffic as simply "repeated > downloads of 700KB", though. Quoting from [0], the actual wording is: >=20 >> "Webs" indicates the number of simultaneous web users (repeated >> downloads of a 700 kB page as described in Appendix A of [White]), >=20 > Where [White] refers to [1] which states (in the Appendix): >=20 >> The file sizes are generated via a log-normal distribution, such that >> the log10 of file size is drawn from a normal distribution with mean = =3D >> 3.34 and standard deviation =3D 0.84. The file sizes (yi) are = calculated >> from the resulting 100 draws (xi ) using the following formula, in >> order to produce a set of 100 files whose total size =3D~ 600 kB = (614400 >> B): >=20 > And in the main text it specifies (in section 3.2.3) the actual model > for the web traffic used: >=20 >> Model single user web page download as follows: >>=20 >> - Web page modeled as single HTML page + 100 objects spread evenly >> across 4 servers. Web object sizes are currently fixed at 25 kB each, >> whereas the initial HTML page is 100 kB. Appendix A provides an >> alternative page model that may be explored in future work. >>=20 >> - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms). >>=20 >> - Initial HTTP GET to retrieve a moderately sized object (100 kB HTML >> page) from server 1. >>=20 >> - Once initial HTTP GET completes, initiate 24 simultaneous HTTP GETs >> (via separate TCP connections), 6 connections each to 4 different >> server nodes >>=20 >> - Once each individual HTTP GET completes, initiate a subsequent GET >> to the same server, until 25 objects have been retrieved from each >> server. >=20 >=20 > Which is a pretty far cry from just saying "repeated downloads of 700 > KB" and, while still somewhat bigger, matches the numbers from Google > better in terms of distribution between page sizes and other objects. > And, more importantly, it features the kind of parallelism and > interactions that a real web browser does; which, as Shahid mentioned = is > (can be) quite important for the treatment it receives by an AQM. >=20 > As such I would be strongly in favour of changing the draft to = actually > describe realistic web client behaviour, rather than just summarising = it > as "repeated downloads of 700KB". >=20 >=20 > -Toke >=20 >=20 > [0] = http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Managemen= t_Algorithms_DOCSIS_3_0.pdf >=20 > [1] = http://www.cablelabs.com/downloads/pubs/PreliminaryStudyOfCoDelAQM_DOCSISN= etwork.pdf >=20 > [2] https://developers.google.com/speed/articles/web-metrics > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm From nobody Tue Apr 15 09:31:04 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 141551A079F for ; Tue, 15 Apr 2014 09:31:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tMsBlTWolRK1 for ; Tue, 15 Apr 2014 09:30:58 -0700 (PDT) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) by ietfa.amsl.com (Postfix) with ESMTP id BDD6F1A0499 for ; Tue, 15 Apr 2014 09:30:57 -0700 (PDT) Received: by mail-wi0-f180.google.com with SMTP id q5so10815wiv.13 for ; Tue, 15 Apr 2014 09:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=YUp6gk/GLjsbziuR5oLln76hRoSzQMvxw5s9yfu0a8Y=; b=bqF/EOjxVFnJvR36x4fxv3ZplMgc+IkrnaSn2/ajw5+ohgonjCVZ0y8nWZfm9q2bdG aa0MdSbsTnvUzzO8B/azAK+640Zy32og4ZlgQzKKlCCeOG60MNx4zJ4G/DaaLi9DA+cz j3XM1s+1eOYGIOapkKHWkz9Jz+7HUURsmdfe1CHOAOIs7UrB4Js7o3tAVXffnLwuwaLd aPWRb405hrXqv48dYt/M0Mk/Le4Y6uUeWVGa7FFcATa12uuI/vl6tvp/KalX6i0sTg1h EZHQI5dBF5qah+TlUxnm2f4/UOhsGKikazZUA/tdbrfIDFPv0C6PX24+BBOiZqAo4deH LDgA== MIME-Version: 1.0 X-Received: by 10.180.19.167 with SMTP id g7mr3299140wie.46.1397579454402; Tue, 15 Apr 2014 09:30:54 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Tue, 15 Apr 2014 09:30:54 -0700 (PDT) In-Reply-To: <2A8C5B3D-CD2C-4378-ABBC-AEE3678FD2F7@telecom-bretagne.eu> References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> <878ur6vqd4.fsf@toke.dk> <2A8C5B3D-CD2C-4378-ABBC-AEE3678FD2F7@telecom-bretagne.eu> Date: Tue, 15 Apr 2014 09:30:54 -0700 Message-ID: From: Dave Taht To: Nicolas KUHN Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/vV-2YMcvw-vnRk3K9zJ-EEOteyc Cc: "Akhtar, Shahid \(Shahid\)" , "aqm@ietf.org" , Richard Scheffenegger Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 16:31:03 -0000 On Tue, Apr 15, 2014 at 6:57 AM, Nicolas KUHN wrote: > Thank you for detailing the content of the Cable Labs document and where = these 700kB come from. > Concerning your last point: > >> As such I would be strongly in favour of changing the draft to actually >> describe realistic web client behaviour, rather than just summarising it >> as "repeated downloads of 700KB". +100. > > > I understand that it may be a drastic simplification to just summarise th= e web client behaviour as only repeated downloads of 700kB. However, the dr= aft may not detail realistic web client behaviour: I believe that it may be= out of topic and the draft cannot contain such level of complexity for all= the covered protocols/traffic. > I propose that the following changes: > > Was: > - Realistic HTTP web traffic (repeated download of 700kB); > Changed by: > - Realistic HTTP web page downloads: the tester should at least c= onsider repeated downloads of 700kB - for a more accurate web traffic, a si= ngle user web page download [White] may exploited; > What do you think ? An AQM evaluation guide MUST include evaluations against real traffic patterns. Period. The white PLT model was decent; the repeated single flow 700k download proposal is nuts. (I can certainly see attempting to emulate DASH traffic, however) I have further pointed out some flaws in the white PLT model in previous emails - notably as to the effect of not emulating DNS traffic - and have been working towards acquiring a reasonable distribution of DNS hit, miss, and fill numbers to plug into it for some time. That has required work - work on finding a decent web benchmark - and work on acquiring statistics that make sense - and some of that work is beginning to bear fruit. Dnsmasq, for example has sprouted the ability to collect statistics, we are trying to get the chrome web page benchmarker working again, and so on. You ignore the overhead of DNS lookups to your peril. There are other overheads worth looking at, too... Similarly I regard testing a correct emulation of bittorrent's real-world behavior in an AQM'd environment as pretty critical. [white] was not even close in this respect. (but it was a good first try!) Overall I suggest that we also adopt the same tests that other WGs are proposing for their protocols. rmcat had a good starter set here: http://www.ietf.org/proceedings/89/slides/slides-89-rmcat-2.pdf > > Regards, > > Nicolas > > On Apr 15, 2014, at 12:28 PM, Toke H=C3=B8iland-J=C3=B8rgensen wrote: > >> Nicolas KUHN writes: >> >>> and realistic HTTP web traffic (repeated download of 700kB). As a remin= der, >>> please find here the comments of Shahid Akhtar regarding these values: >> >> The Cablelabs work doesn't specify web traffic as simply "repeated >> downloads of 700KB", though. Quoting from [0], the actual wording is: >> >>> "Webs" indicates the number of simultaneous web users (repeated >>> downloads of a 700 kB page as described in Appendix A of [White]), >> >> Where [White] refers to [1] which states (in the Appendix): >> >>> The file sizes are generated via a log-normal distribution, such that >>> the log10 of file size is drawn from a normal distribution with mean = =3D >>> 3.34 and standard deviation =3D 0.84. The file sizes (yi) are calculate= d >>> from the resulting 100 draws (xi ) using the following formula, in >>> order to produce a set of 100 files whose total size =3D~ 600 kB (61440= 0 >>> B): >> >> And in the main text it specifies (in section 3.2.3) the actual model >> for the web traffic used: >> >>> Model single user web page download as follows: >>> >>> - Web page modeled as single HTML page + 100 objects spread evenly >>> across 4 servers. Web object sizes are currently fixed at 25 kB each, >>> whereas the initial HTML page is 100 kB. Appendix A provides an >>> alternative page model that may be explored in future work. >>> >>> - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms). >>> >>> - Initial HTTP GET to retrieve a moderately sized object (100 kB HTML >>> page) from server 1. >>> >>> - Once initial HTTP GET completes, initiate 24 simultaneous HTTP GETs >>> (via separate TCP connections), 6 connections each to 4 different >>> server nodes >>> >>> - Once each individual HTTP GET completes, initiate a subsequent GET >>> to the same server, until 25 objects have been retrieved from each >>> server. >> >> >> Which is a pretty far cry from just saying "repeated downloads of 700 >> KB" and, while still somewhat bigger, matches the numbers from Google >> better in terms of distribution between page sizes and other objects. >> And, more importantly, it features the kind of parallelism and >> interactions that a real web browser does; which, as Shahid mentioned is >> (can be) quite important for the treatment it receives by an AQM. >> >> As such I would be strongly in favour of changing the draft to actually >> describe realistic web client behaviour, rather than just summarising it >> as "repeated downloads of 700KB". >> >> >> -Toke >> >> >> [0] http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Man= agement_Algorithms_DOCSIS_3_0.pdf >> >> [1] http://www.cablelabs.com/downloads/pubs/PreliminaryStudyOfCoDelAQM_D= OCSISNetwork.pdf >> >> [2] https://developers.google.com/speed/articles/web-metrics >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Tue Apr 15 10:19:52 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 770CB1A07EF for ; Tue, 15 Apr 2014 10:19:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A_WFJEWUsHYd for ; Tue, 15 Apr 2014 10:19:47 -0700 (PDT) Received: from nm23-vm8.access.bullet.mail.bf1.yahoo.com (nm23-vm8.access.bullet.mail.bf1.yahoo.com [216.109.115.167]) by ietfa.amsl.com (Postfix) with ESMTP id 048B21A07F7 for ; Tue, 15 Apr 2014 10:19:46 -0700 (PDT) Received: from [66.196.81.162] by nm23.access.bullet.mail.bf1.yahoo.com with NNFMP; 15 Apr 2014 17:19:44 -0000 Received: from [98.139.244.51] by tm8.access.bullet.mail.bf1.yahoo.com with NNFMP; 15 Apr 2014 17:19:44 -0000 Received: from [127.0.0.1] by smtp113.sbc.mail.bf1.yahoo.com with NNFMP; 15 Apr 2014 17:19:43 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1397582383; bh=5Sf6zxQeF2SJugXlir/RCsNDR4f+w0j76ojz7mOtZ6w=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:X-Enigmail-Version:Content-Type:Content-Transfer-Encoding; b=05KxfHER9dXs/ECWc1LNBfA0+dIFDdwK2SrLmgCjzFCsH3so13w3SvghS09RjrsM9Uc3hfDmbdsQ2BTc0AKKKucMl0v0mj3nIbZfhGiUvrXL/SLJQPlV8ccbUNCsx79eymOrQuqUDzCnF5jGrp2gXVCJHCHl+sriYjv/gvTYElw= X-Yahoo-Newman-Id: 989286.4752.bm@smtp113.sbc.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: JRhC32cVM1mfUPap5hpaTaz3iORUbT6PTWgMEydnwKZKkUC MrVoWigHL01A2ib.gYWWEi1QQBPCUdJqJmGU5V3uHpr5wz.goCiDP8VXyxhW bdYinBeoxgcxNem59o6wINznt6J._6mFGTm.8v2w4KI64k292tyvYnIAltzh a5KyP5v1vq65QSz9_SqbdqeZQ1M774IeYsETjQYgDLPzs9HyDQDaAFarzRA0 maw.rSIpzoDshMRUl6NpXh7ZVQ1fWws9zLK3dDDyDKGh27qd7V9ukAtSSw_z ELu6QQ4TUB5vIFRPfabt9dl7.ca97HIU3Ucgnxt12wYutDFnbNShBETIW.Nx aWcE4Q2wprNqAIAA1bJ1xdjme3FoAG6XmHuiyvP9aj8IZI_R4RDNWTUq5u80 d8N6uxRRqQo8MFEKUjkHus1oVvVVNbddqQ_bdab6pkmxHJ8Qs83GHG1i5o.a gWJJMWGSw3fOeJvJEVDLfE2Rt3XiRPBSaAZY3WufT8kA7V_xbRU0tN.1mlRN uUlKeewHFDpFYK9TVHceY2W.2jobZeZz4JpDGue8wW6cGNFvVWavLpqC61HD wJvvowP4qqkQalrDbezMVgmSKXTGARSLFqqAL_2aOwuoVApUCh9CG_gAi_X8 Q6Ff2IfF7PRtRMtJ4azsHIq.syw-- X-Yahoo-SMTP: sltvjZWswBCRD.ElTuB1l9j6s9wRYPpuyTNWOE5oEg-- X-Rocket-Received: from [10.111.100.146] (davec-b@74.213.188.46 with plain [98.139.221.125]) by smtp113.sbc.mail.bf1.yahoo.com with SMTP; 15 Apr 2014 10:19:43 -0700 PDT Message-ID: <534D6A2E.30409@rogers.com> Date: Tue, 15 Apr 2014 13:19:42 -0400 From: David Collier-Brown User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Dave Taht , Nicolas KUHN , "aqm@ietf.org" References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> <878ur6vqd4.fsf@toke.dk> <2A8C5B3D-CD2C-4378-ABBC-AEE3678FD2F7@telecom-bretagne.eu> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/9s9zzWYc3nYzQJ9O3nmM6WVvEyQ Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: davecb@spamcop.net List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 17:19:49 -0000 On 04/15/2014 12:30 PM, Dave Taht wrote: > Overall I suggest that we also adopt the same tests that other WGs are > proposing for their protocols. rmcat had a good starter set here: > http://www.ietf.org/proceedings/89/slides/slides-89-rmcat-2.pdf That might be a substantial time-saver in the end, although a time cost rigfht now... --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain From nobody Tue Apr 15 15:09:09 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B97031A045D for ; Tue, 15 Apr 2014 15:09:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.874 X-Spam-Level: X-Spam-Status: No, score=-1.874 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.272, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6JekAJUQJDm7 for ; Tue, 15 Apr 2014 15:09:03 -0700 (PDT) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 7ECE21A0239 for ; Tue, 15 Apr 2014 15:09:03 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.97,867,1389772800"; d="asc'?scan'208";a="117130455" Received: from vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) by mx11-out.netapp.com with ESMTP; 15 Apr 2014 15:08:39 -0700 Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.246]) by vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) with mapi id 14.03.0123.003; Tue, 15 Apr 2014 15:08:38 -0700 From: "Eggert, Lars" To: =?iso-8859-1?Q?Toke_H=F8iland-J=F8rgensen?= Thread-Topic: [aqm] [AQM Evaluation Guidelines] Thread-Index: AQHPI/v1WVwvqB4avE+7HEOeq/JuiJq+PNAAgFT4noD//62az4ABONkA Date: Tue, 15 Apr 2014 22:08:38 +0000 Message-ID: <10B093B6-3AF1-49A2-AC02-54E851A19222@netapp.com> References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> <878ur6vqd4.fsf@toke.dk> In-Reply-To: <878ur6vqd4.fsf@toke.dk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.30] Content-Type: multipart/signed; boundary="Apple-Mail=_8D71C576-C4B7-4890-AF9D-11F6598CBB79"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/mSodptxGwKSCN6HZk-zLkWd-mxA Cc: "Akhtar, Shahid \(Shahid\)" , "Scheffenegger, Richard" , Nicolas KUHN , "aqm@ietf.org" Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 22:09:08 -0000 --Apple-Mail=_8D71C576-C4B7-4890-AF9D-11F6598CBB79 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On 2014-4-15, at 3:28, Toke H=F8iland-J=F8rgensen wrote: > As such I would be strongly in favour of changing the draft to = actually > describe realistic web client behaviour, rather than just summarising = it > as "repeated downloads of 700KB". Fully agree. And if the 700KB download is left in, please call it something other = than "realistic HTTP web traffic", which it's certainly not. Maybe call = it "ringtone download" :-) Lars --Apple-Mail=_8D71C576-C4B7-4890-AF9D-11F6598CBB79 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBU02t5tZcnpRveo1xAQLmCwP/ZlQB14epYAXZE8MvgR8DiHL/+Um9s8vN 1DozQ49JmJn5R6gUgkpkA4adoLxiYPpPilwpgLr0xomC6ukU18gyRj3IK9ofyjGv ZEAWi95TC5K99Az2ON6DHS82qm7qJOB4Da5fooRdRzESpuAh/uA2F6Wp6G3MQ29w ujwENsHl0w4= =ezcG -----END PGP SIGNATURE----- --Apple-Mail=_8D71C576-C4B7-4890-AF9D-11F6598CBB79-- From nobody Tue Apr 15 15:47:32 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A15341A0004 for ; Tue, 15 Apr 2014 15:47:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.199 X-Spam-Level: X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wgiiW3Rdan50 for ; Tue, 15 Apr 2014 15:47:19 -0700 (PDT) Received: from ihemail1.lucent.com (ihemail1.lucent.com [135.245.0.33]) by ietfa.amsl.com (Postfix) with ESMTP id 6B1941A0002 for ; Tue, 15 Apr 2014 15:47:18 -0700 (PDT) Received: from us70tusmtp2.zam.alcatel-lucent.com (h135-5-2-64.lucent.com [135.5.2.64]) by ihemail1.lucent.com (8.13.8/IER-o) with ESMTP id s3FMlDGK010402 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 15 Apr 2014 17:47:14 -0500 (CDT) Received: from US70UWXCHHUB02.zam.alcatel-lucent.com (us70uwxchhub02.zam.alcatel-lucent.com [135.5.2.49]) by us70tusmtp2.zam.alcatel-lucent.com (GMO) with ESMTP id s3FMl0KR027553 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 15 Apr 2014 18:47:13 -0400 Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.57]) by US70UWXCHHUB02.zam.alcatel-lucent.com ([135.5.2.49]) with mapi id 14.02.0247.003; Tue, 15 Apr 2014 18:47:07 -0400 From: "Akhtar, Shahid (Shahid)" To: Nicolas KUHN Thread-Topic: [aqm] [AQM Evaluation Guidelines] Thread-Index: AQHPWIQGll9gehQHkUiB0C/rclcgBJsTMV2Q Date: Tue, 15 Apr 2014 22:47:06 +0000 Message-ID: References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> In-Reply-To: <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.5.27.16] Content-Type: multipart/alternative; boundary="_000_C0611F522A6FA9498A044C36073E49659D771B13US70UWXCHMBA01z_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/wUqH_c9Y_5ngx_wBmzdNZO16ruA Cc: Richard Scheffenegger , "aqm@ietf.org" Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 22:47:25 -0000 --_000_C0611F522A6FA9498A044C36073E49659D771B13US70UWXCHMBA01z_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Nicholas, Please see comments inline. Best regards, -Shahid. ________________________________ From: Nicolas KUHN [mailto:nicolas.kuhn@telecom-bretagne.eu] Sent: Tuesday, April 15, 2014 3:24 AM To: Akhtar, Shahid (Shahid) Cc: aqm@ietf.org; Richard Scheffenegger Subject: Re: [aqm] [AQM Evaluation Guidelines] Hi Shahid Akhtar, all, After the discussions in London IETF 89, there has been lots of modificatio= ns to the draft organisation (as we already advertised on the list). The authors and I just discussed your points considering the size of the fi= les for the bulk transfer and the HTTP traffic. In the current version of the draft, we consider the values mentioned in th= e cablelabs work [0]. As a result, we consider bulk TCP transfers (repeating 5MB file transmissio= n) and realistic HTTP web traffic (repeated download of 700kB). As a remind= er, please find here the comments of Shahid Akhtar regarding these values: "Bulk TCP transfer: (continuous file transmission, or repeating 5MB file tr= ansmission);" The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assumi= ng 480p video and 4.5 minutes for some of our experiments. The inter-arriva= l time of these files should have a realistic random distribution as well a= s size of the files. SA: The Cablelabs document as mentioned below has a set of tests for Bulk T= CP for uploading traffic, they use a single continuous TCP flow for certain= testcases and multiple TCP flows with 20MB or 250KB files repeatedly being= sent upstream for other testcases. Their application is upstream traffic w= hich behaves differently from downstream traffic. A Sandvine document on In= ternet traffic (https://www.sandvine.com/downloads/general/global-internet-= phenomena/2013/exposing-the-technical-and-commercial-factors-underlying-int= ernet-quality-of-experience.pdf) shows that real-time entertainment (Netfli= x, YouTube and others) constitute 68% of downstream traffic, (Figure 9) of = which YouTube (progessive download) is at 17% of total traffic. Using repea= ted transfers of TCP files as suggsted in the Cablelabs work would miss the= effect of the random on-off patterns that PD flows generate on the downstr= eam portion. So I would still recommend the use of PD (YouTube) statistics = such as above for modeling the Bulk TCP traffic downstream traffic. "Realistic HTTP web traffic (repeated download of 700kB);" According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM= on a 7KB file would be very different than on a 700KB file. Web traffic al= so has a complex inter-arrival time relationship between the GET requests a= s well as the pages that produce these GET requests. However to make the te= sting simpler, a repeated download of a 7KB file with a Pareto distributed = size and Pareto distributed inter-arrival time may be an OK compromise. SA: I believe Toke has already responded that the CableLabs work mentioned = below refers to another document, which uses 25KB for responses to web page= GETs. This is hugely better than using repeated 700KB downloads and I am O= K with using the model in the other document. I am still concerned that the= average size of the reponse maybe less than 25KB, The statistics collected= by Google (https://developers.google.com/speed/articles/web-metrics) show = an average size of 7KB, but they are 4 years old and I cannot find a more r= ecent number. We expect to stick with the cable labs values [0]. Please let us know if yo= u have any comments regarding this choice. The Cablelabs work does not mention any HTTP adaptive streaming (HAS) traff= ic models - would that be included in the guidelines? Kind regards, Nicolas KUHN [0] http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Manage= ment_Algorithms_DOCSIS_3_0.pdf On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid (Shahid)" > wrote: Hi Nicholas and other authors, Thank you for producing this document. I had some (long) comments where I h= ave also suggested some solutions/text, hopefully you find them helpful. Wh= en quoting from the document, I have used quotes and I have listed my comme= nts as 1-4. 1. The number of tests recommended per AQM scheme seems to be quite large a= s well as many tests that may be best done together are done independently. For example, there are different tests for fairness between flows (section = 3.2.3), level of traffic (mild, medium, heavy) (section 3.2.4), different t= ypes of bursty traffic (3.2.2) and different network environments (section = 3.3). In order to obtain a realistic estimate of AQM comparison, many of th= ese effects may need to be tested together. For example to increase level o= f traffic (in section 3.2.4) only long term TCP flows of 50s are proposed. = It would be better to test the level of congestion with realistic traffic (= and with realistic proportions) as proposed in section 3.2.4. Another examp= le would be to test fairness between flows of same type - e.g. long term TC= P flows when multiple types of TCP flows are present at a AQM controlled bu= ffer or test fairness between flows of different types - e.g. long term TCP= flows and web flows. I noticed that in section 3.3 different network environments are mentioned.= I suggest that tests with appropriate realistic traffic be done on each of= these enviroments with various levels of congestion. One could measure fai= rness from these tests without the need for additional tests. This may have= the benefit of reducing overall number of tests as well. 2. There are four network enviroments mentioned in the document (section 3.= 3): Wifi - in home network. Data-Center communications. BB wireless/satellite. Low and high buffers This is a good set, however, recommed to add: Access networks - and further split access networks into DSL/FTTH networks = and Cable access networks. Cable access networks may have much larger numbe= r of flows than DSL/FTTH since they are shared by a large number of custome= rs There is another type of network environment - narrowband wireless (mobile)= networks where the AQM could be placed on the eNodeB, but perhaps this is = a complicated scenario, and best left for later or in another document. 3. I noticed that the modeling of different types of realistic traffic coul= d be made more accurate (section 3.2.2) "Bulk TCP transfer: (continuous file transmission, or repeating 5MB file tr= ansmission);" The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assumi= ng 480p video and 4.5 minutes for some of our experiments. The inter-arriva= l time of these files should have a realistic random distribution as well a= s size of the files. "Realistic HTTP web traffic (repeated download of 700kB);" According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM= on a 7KB file would be very different than on a 700KB file. Web traffic al= so has a complex inter-arrival time relationship between the GET requests a= s well as the pages that produce these GET requests. However to make the te= sting simpler, a repeated download of a 7KB file with a Pareto distributed = size and Pareto distributed inter-arrival time may be an OK compromise. I noticed that adaptive streaming is not modelled as a type of realistic tr= affic. It is probably the most important and largest flow on the Internet t= oday - if we look at a recent Sandvine report of Internet traffic, about 75= % of Internet traffic is real-time entertainment of which the largest compo= nent is adaptive streaming based video traffic. In order to simulate adapti= ve streaming traffic, one would need a client that changes video quality ra= tes. To keep testing simpler, an option may be to generate realistic chunk = sized files at regular intervals. Average netflix speeds are around 2Mbps. = Typical chunk sizes range from 2s to 10s, but 4s may be a good average. The= chunk sizes should be varied with some random distribution as is typically= observed from real codecs. 4. In section 2.2.5, QoE metrics are mentioned. It may be valuable to spell= out some standard models or expressions to derive QoE from network paramet= ers so that these QoE can be compared between AQMs. In the literature there= is work that derives closed form expressions for QoE. The following are ex= amples and suggestions: Video - the work by Johan de Vriendt et al , "Model for estimating QoE of V= ideo delivered using HTTP Adaptive Streaming", IFIP/IEEE Workshop on QoE Ce= ntric Management, May 2013 may be useful here. The authors used over 500 sa= mples to derive their a closed form expressions. The authors derive the MOS= =3D alpha*(average bit rate) + beta*(standard deviation of bit rates) + si= gma*(no of bit rate switches) + gamma and then produce good values for alph= a, beta, sigma and gamma. Voice - The work in "Voice quality prediction models and their application = in VoIP networks" by L. Sun et al has closed form expressions for MOS score= s for most commercial codes. They derive a long closed form expression with= parameters a-j for key codecs (and then provide a-j for most commercial co= decs) based on end-to-end and packet loss. Web - The delay for one average web GET file could be used as the represent= ative for the page load time Key metric for AQMs on access networks may be QoE since it provides and end= -user view. We have seen that often the absolute goodput does not matter as= much as the stability of the flow for Video (which tends to be the dominan= t traffic). Larger goodput may provide higher VQ levels for Video, but if h= igher volatility is also produced then that higher VQ level may not produce= a higher QoE. On the other hand, best metrics inter data center traffic may be goodput an= d delay since most of that traffic is likely to be inter machine traffic. Best regards, Shahid Akhtar. -----Original Message----- From: aqm [mailto:aqm-bounces@ietf.org] On Behalf = Of Nicolas KUHN Sent: Friday, February 07, 2014 5:58 AM To: Richard Scheffenegger Cc: aqm@ietf.org Subject: [aqm] [AQM Evaluation Guidelines] Dear all, On the behalf of the contributors to the AQM Evaluation Guidelines, I encou= rage active discussion on the draft that we have written. I just submitted the draft as an individual draft to the I-D Submission Too= l [0]. Please let us know what you think of the current document. Kind regards, Nicolas KUHN [0] http://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/ On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" > wrote: Hi, a new month, a new status report. First of all, Wes and I as chairs would like to thank the editors who have = stepped forward to work on the AQM Evaluation Guideline draft. We are reall= y thankful for their burst of efforts in the last couple weeks! We expect that that a document will be ready for submission into the I-D re= pository well before cutoff, as an individual draft, so that the WG can see= what the state of thinking is currently. Also, once the document is publis= hed on datatracker we'd like to encourage active discussion on it. If the submitted -00 draft is felt to have a fairly complete outline of wha= t the current thinking in the WG is, we may be able to ask the WG for forma= l adoption during this IETF meeting, or shortly thereafter. - WG Milestones: - Submit AQM recommendations to IESG for publication, obsoleting RFC 2309 (Goal: January 2014) - draft-ietf-aqm-recommendation is accepted towards this milestone - http://datatracker.ietf.org/doc/draft-ietf-aqm-recommendation/ - the draft has been updated per comments received - if the authors are comfortable, a WGLC might be made on the next revision - we would like to hear from other authors of RFC 2309 on this document, if anyone has contacts to them. - Submit AQM algorithm evaluation guidelines to IESG for publication as Informational (Goal: July 2014) - An editor team has come forth and is working on this - A draft should be available for discussion in the London IETF - We encourage discussion on the list and during the meeting, if this draft should be adopted by the working group. - Submit first algorithm specification to IESG for publication as Proposed Standard (Goal: December 2014) - Since any Proposed Standard algorithm should be in line with the recommendations and be passable versus the evaluation guidelines, this milestone is dependend on the progress of the two work items above. - Currently the only algorithm spec with a complete and active individual-submission draft is PIE - Other items: - draft-pan-aqm-pie is under active work as a proposed algorithm: http://tools.ietf.org/id/draft-pan-aqm-pie-00.txt, however the draft has expired and should be refreshed. - draft-nichols-tsvwg-codel is expired; Dave Taht or others may revive it and/or describe pairing with FQ/SFQ algorithms: http://tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt - Other algorithm specifications are welcome! - Though, we are not planning on adopting algorithms until recommendations and evaluation guidelines are mostly stable Richard Scheffenegger _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm --_000_C0611F522A6FA9498A044C36073E49659D771B13US70UWXCHMBA01z_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Hi Nicholas,
 
Please see comments inline.
 
Best regards,
 
-Shahid.

From: Nicolas KUHN [mailto:nicolas.= kuhn@telecom-bretagne.eu]
Sent: Tuesday, April 15, 2014 3:24 AM
To: Akhtar, Shahid (Shahid)
Cc: aqm@ietf.org; Richard Scheffenegger
Subject: Re: [aqm] [AQM Evaluation Guidelines]

Hi Shahid Akhtar, all,

After the discussions in London IETF 89, there has been lots of modifi= cations to the draft organisation (as we already advertised on the list).&n= bsp;
The authors and I just discussed your points considering the size of t= he files for the bulk transfer and the HTTP traffic. 
In the current version of the draft, we consider the values mentioned = in the cablelabs work [0].

As a result, we consider bulk TCP transfers (repeating 5MB file transm= ission) and realistic HTTP web traffic (repeated download of 700kB). As a r= eminder, please find here the comments of Shahid Akhtar regarding these val= ues:

"Bulk TCP transfer: (continuous file transmission, or repeating 5= MB file transmission);"
The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assuming 480p video and 4.5 minutes f= or some of our experiments. The inter-arrival time of these files should ha= ve a realistic random distribution as well as size of the files.  
 
SA: The Cablelabs document as mentioned below has a se= t of tests for Bulk TCP for uploading traffic, they use a single continuous= TCP flow for certain testcases and multiple TCP flows with 20MB or 250KB files repeatedly being sent upstream for other te= stcases. Their application is upstream traffic which behaves differently fr= om downstream traffic. A Sandvine document on Internet traffic (https://www.sandvine.com/downloads/general/global-int= ernet-phenomena/2013/exposing-the-technical-and-commercial-factors-underlyi= ng-internet-quality-of-experience.pdf) shows that real-time entertainment (Netflix, YouTube and others) constitut= e 68% of downstream traffic, (Figure 9) of which YouTube (progess= ive download) is at 17% of total traffic. Using repeated transfers of TCP f= iles as suggsted in the Cablelabs work would miss the effect of the random on-off patterns that PD flows generate on th= e downstream portion. So I would still recommend the use of PD (YouTub= e) statistics such as above for modeling the Bulk TCP traffic downstream tr= affic.
"Realistic HTTP web traffic (repeated download of 700kB);" According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM on a 7KB file would be very differen= t than on a 700KB file. Web traffic also has a complex inter-arrival time r= elationship between the GET requests as well as the pages that produce thes= e GET requests. However to make the testing simpler, a repeated download of a 7KB file with a Pareto distr= ibuted size and Pareto distributed inter-arrival time may be an OK compromi= se. 
 
SA: I believe Toke has already responded that the= CableLabs work mentioned below refers to another document, which uses 25KB= for responses to web page GETs. This is hugely better than using repeated 700KB downloads and I am OK with using the mode= l in the other document. I am still concerned that the average size of the = reponse maybe less than 25KB, The statistics collected by Google (https://developers.goog= le.com/speed/articles/web-metrics) show an average size of 7KB, but they are 4 years old and I cannot find a = more recent number.
 
We expect to stick with the cable labs values [0]. Please let us know = if you have any comments regarding this choice.  
 
The Cablelabs work does not mention any HTTP adapt= ive streaming (HAS) traffic models - would that be included in the&nbs= p;guidelines? 
Kind regards, 

Nicolas KUHN


On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid (Shahid)" <<= a href=3D"mailto:shahid.akhtar@alcatel-lucent.com">shahid.akhtar@alcatel-lu= cent.com> wrote:

Hi Nicholas and other authors,

Thank you for producing this document. I had some (long) comments where I h= ave also suggested some solutions/text, hopefully you find them helpful. Wh= en quoting from the document, I have used quotes and I have listed my comme= nts as 1-4.


1. The number of tests recommended per AQM scheme seems to be quite large a= s well as many tests that may be best done together are done independently.=

For example, there are different tests for fairness between flows (section = 3.2.3), level of traffic (mild, medium, heavy) (section 3.2.4), different t= ypes of bursty traffic (3.2.2) and different network environments (section = 3.3). In order to obtain a realistic estimate of AQM comparison, many of these effects may need to be tested to= gether. For example to increase level of traffic (in section 3.2.4) only lo= ng term TCP flows of 50s are proposed. It would be better to test the level= of congestion with realistic traffic (and with realistic proportions) as proposed in section 3.2.4. Another exa= mple would be to test fairness between flows of same type - e.g. long term = TCP flows when multiple types of TCP flows are present at a AQM controlled = buffer or test fairness between flows of different types - e.g. long term TCP flows and web flows.

I noticed that in section 3.3 different network environments are mentioned.= I suggest that tests with appropriate realistic traffic be done on each of= these enviroments with various levels of congestion. One could measure fai= rness from these tests without the need for additional tests. This may have the benefit of reducing overall n= umber of tests as well.


2. There are four network enviroments mentioned in the document (section 3.= 3):
Wifi - in home network.
Data-Center communications.
BB wireless/satellite.
Low and high buffers

This is a good set, however, recommed to add:

Access networks - and further split access networks into DSL/FTTH networks = and Cable access networks. Cable access networks may have much larger numbe= r of flows than DSL/FTTH since they are shared by a large number of custome= rs

There is another type of network environment - narrowband wireless (mobile)= networks where the AQM could be placed on the eNodeB, but perhaps this is = a complicated scenario, and best left for later or in another document.


3. I noticed that the modeling of different types of realistic traffic coul= d be made more accurate (section 3.2.2)

"Bulk TCP transfer: (continuous file transmission, or repeating 5MB fi= le transmission);"
The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assuming 480p video and 4.5 minutes f= or some of our experiments. The inter-arrival time of these files should ha= ve a realistic random distribution as well as size of the files.

"Realistic HTTP web traffic (repeated download of 700kB);"
According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM on a 7KB file would be very differen= t than on a 700KB file. Web traffic also has a complex inter-arrival time r= elationship between the GET requests as well as the pages that produce thes= e GET requests. However to make the testing simpler, a repeated download of a 7KB file with a Pareto distr= ibuted size and Pareto distributed inter-arrival time may be an OK compromi= se.

I noticed that adaptive streaming is not modelled as a type of realistic tr= affic. It is probably the most important and largest flow on the Internet t= oday - if we look at a recent Sandvine report of Internet traffic, about 75= % of Internet traffic is real-time entertainment of which the largest component is adaptive streaming based v= ideo traffic. In order to simulate adaptive streaming traffic, one would ne= ed a client that changes video quality rates. To keep testing simpler, an o= ption may be to generate realistic chunk sized files at regular intervals. Average netflix speeds are around = 2Mbps. Typical chunk sizes range from 2s to 10s, but 4s may be a good avera= ge. The chunk sizes should be varied with some random distribution as is ty= pically observed from real codecs.


4. In section 2.2.5, QoE metrics are mentioned. It may be valuable to spell= out some standard models or expressions to derive QoE from network paramet= ers so that these QoE can be compared between AQMs. In the literature there= is work that derives closed form expressions for QoE. The following are examples and suggestions:

Video - the work by Johan de Vriendt et al , "Model for estimating QoE= of Video delivered using HTTP Adaptive Streaming", IFIP/IEEE Workshop= on QoE Centric Management, May 2013 may be useful here. The authors used o= ver 500 samples to derive their a closed form expressions. The authors derive the MOS =3D alpha*(average bit rate) += beta*(standard deviation of bit rates) + sigma*(no of bit rate switche= s) + gamma and then produce good values for alpha, beta, sigma and gamm= a.

Voice - The work in "Voice quality prediction models and their applica= tion in VoIP networks" by L. Sun et al has closed form expressions for= MOS scores for most commercial codes. They derive a long closed form expre= ssion with parameters a-j for key codecs (and then provide a-j for most commercial codecs) based on end-to-end and packe= t loss.  

Web - The delay for one average web GET file could be used as the represent= ative for the page load time

Key metric for AQMs on access networks may be QoE since it provides and end= -user view. We have seen that often the absolute goodput does not matter as= much as the stability of the flow for Video (which tends to be the dominan= t traffic). Larger goodput may provide higher VQ levels for Video, but if higher volatility is also produced then= that higher VQ level may not produce a higher QoE.

On the other hand, best metrics inter data center traffic may be goodput an= d delay since most of that traffic is likely to be inter machine traffic.
Best regards,

Shahid Akhtar.

-----Original Message-----
From: aqm [mailto:aqm-bounces@ietf.org<= /a>] On Behalf Of Nicolas KUHN
Sent: Friday, February 07, 2014 5:58 AM
To: Richard Scheffenegger
Cc: aqm@ietf.org
Subject: [aqm] [AQM Evaluation Guidelines]

Dear all,

On the behalf of the contributors to the AQM Evaluation Guidelines, I encou= rage active discussion on the draft that we have written.
I just submitted the draft as an individual draft to the I-D Submission Too= l [0].

Please let us know what you think of the current document.

Kind regards,

Nicolas KUHN

[0] http://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/

On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" <rs@netapp.com> wrote:

Hi,

a new month, a new status report.

First of all, Wes and I as chairs would like to thank the editors who have = stepped forward to work on the AQM Evaluation Guideline draft. We are reall= y thankful for their burst of efforts in the last couple weeks!

We expect that that a document will be ready for submission into the I-D re= pository well before cutoff, as an individual draft, so that the WG can see= what the state of thinking is currently. Also, once the document is publis= hed on datatracker we'd like to encourage active discussion on it.

If the submitted -00 draft is felt to have a fairly complete outline of wha= t the current thinking in the WG is, we may be able to ask the WG for forma= l adoption during this IETF meeting, or shortly thereafter.




- WG Milestones:
 - Submit AQM recommendations to IESG for publication, obsoleting RFC<= br> 2309 (Goal: January 2014)
   - draft-ietf-aqm-recommendation is accepted towards this = milestone
   - http://datatracker.ietf.org/doc/draft-ietf-aqm-recommen= dation/
   - the draft has been updated per comments received
   - if the authors are comfortable, a WGLC might be made on= the next
revision
   - we would like to hear from other authors of RFC 2309 on= this
document, if anyone has contacts to them.


 - Submit AQM algorithm evaluation guidelines to IESG for publication =
as Informational (Goal: July 2014)
   - An editor team has come forth and is working on this    - A draft should be available for discussion in the Londo= n IETF
   - We encourage discussion on the list and during the meet= ing, if this
      draft should be adopted by the working = group.

 - Submit first algorithm specification to IESG for publication as Proposed Standard (Goal: December 2014)
   - Since any Proposed Standard algorithm should be in line= with the
recommendations and be passable versus the evaluation guidelines, this
milestone is dependend on the progress of the two work items above.
   - Currently the only algorithm spec with a complete and a= ctive
individual-submission draft is PIE

- Other items:
 - draft-pan-aqm-pie is under active work as a proposed algorithm:
   http://tools.ietf.org/id/draft-pan-aqm-pie-00.txt, however the dra= ft
   has expired and should be refreshed.
 - draft-nichols-tsvwg-codel is expired; Dave Taht or others may
revive it and/or describe pairing with FQ/SFQ algorithms:
   http://tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt<= br>  - Other algorithm specifications are welcome!
   - Though, we are not planning on adopting algorithms unti= l
recommendations and evaluation guidelines are mostly stable


Richard Scheffenegger

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

--_000_C0611F522A6FA9498A044C36073E49659D771B13US70UWXCHMBA01z_-- From nobody Wed Apr 16 08:28:54 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DECD51A0188 for ; Wed, 16 Apr 2014 08:28:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.665 X-Spam-Level: ** X-Spam-Status: No, score=2.665 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.272] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id huRfNwLqRe7t for ; Wed, 16 Apr 2014 08:28:46 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id AF96B1A0165 for ; Wed, 16 Apr 2014 08:28:46 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id s3GFS7N8012112; Wed, 16 Apr 2014 09:28:07 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Wed, 16 Apr 2014 09:28:06 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0174.001; Wed, 16 Apr 2014 09:28:06 -0600 From: Greg White To: "Akhtar, Shahid (Shahid)" , Nicolas KUHN Thread-Topic: [aqm] [AQM Evaluation Guidelines] Thread-Index: AQHPI/vsfsEKmiUbyESwqMuBEFOkv5q+LAwAgFT4noCAAPEzAIAAsxYA Date: Wed, 16 Apr 2014 15:28:05 +0000 Message-ID: References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.9.131030 x-originating-ip: [10.5.0.27] Content-Type: multipart/alternative; boundary="_000_CF73EABC32A49gwhitecablelabscom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/2IJIYfcNCECSaao8Kwe1BOqSpgw Cc: Richard Scheffenegger , "aqm@ietf.org" Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 15:28:52 -0000 --_000_CF73EABC32A49gwhitecablelabscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable All, As has been mentioned, CableLabs work on AQM evaluations has been (so far) = entirely devoted to evaluating performance for upstream AQM algorithms that= would be implemented in a residential cable modem. While the traffic is,= of course, bi-directional, the focus was on identifying realistic test cas= es that would provide insight on the performance of the algorithms that we = were considering (in both an absolute and a relative sense). For evaluatin= g downstream AQM performance I would definitely add some additional traffic= types, and video traffic would be at the front of my mind there. In order= to do that justice, I think you'd want to have a HAS model that incorporat= es adaptive bitrate capability (but what algorithm?) and estimates QoE usin= g something like the de Vriendt model. For progressive download, it would = seem that the salient metrics would be initial buffering delay and then som= ething related to the number and duration of rebuffering events. I agree with others that reducing the web model to a single 700kB file down= load is unrealistic. Also, for recent work we've increased the object size= s to correspond to a projected 2018 page size (extrapolating from http://ht= tparchive.org/trends.php) of about 7MB. For our 2018 model of Internet traffic (again, upstream focused) where the = user's broadband speed is projected to be 1Gbps down and 200Mbps up, we're = using the following 9 traffic loads: N F1 F2 Fs W VG C T 1 0 0 - 1 1 0 0 2 1 1 inf 1 1 0 0 3 3 3 50MB 1 1 0 0 4 3 3 50MB 1 1 6 0 5 1 1 19MB* 1 1 0 0 6 3 3 50MB 4 4 0 0 7 3 3 50MB 4 4 6 0 8 5 5 2.5MB 4 4 6 0 9 3 3 50MB 4 1 0 100 where: N: traffic load index F1: number of simultaneous FTP uploads with 20ms RTT F2: number of simultaneous FTP uploads with 100ms RTT Fs: FTP filesize W: number of simultaneous web users VG: number of simultaneous VoIP/gaming sessions C: CBR data rate (Mbps) T: number of torrent (LEDBAT) connections *Filesize and repetition pattern chosen to exercise DOCSIS "powerboost" fea= ture Since evaluation comes before deployment (by a long lead time in some cases= ), and, once deployed, algorithms may have a long life, I would encourage t= he AQM Evaluation Guidelines draft to consider not just what traffic looks = like historically, but what we can reasonably predict for the future. -Greg From: , "Shahid (Shahid)" > Date: Tuesday, April 15, 2014 at 4:47 PM To: Nicolas KUHN > Cc: Richard Scheffenegger >, "aqm@ietf.= org" > Subject: Re: [aqm] [AQM Evaluation Guidelines] Hi Nicholas, Please see comments inline. Best regards, -Shahid. ________________________________ From: Nicolas KUHN [mailto:nicolas.kuhn@telecom-bretagne.eu] Sent: Tuesday, April 15, 2014 3:24 AM To: Akhtar, Shahid (Shahid) Cc: aqm@ietf.org; Richard Scheffenegger Subject: Re: [aqm] [AQM Evaluation Guidelines] Hi Shahid Akhtar, all, After the discussions in London IETF 89, there has been lots of modificatio= ns to the draft organisation (as we already advertised on the list). The authors and I just discussed your points considering the size of the fi= les for the bulk transfer and the HTTP traffic. In the current version of the draft, we consider the values mentioned in th= e cablelabs work [0]. As a result, we consider bulk TCP transfers (repeating 5MB file transmissio= n) and realistic HTTP web traffic (repeated download of 700kB). As a remind= er, please find here the comments of Shahid Akhtar regarding these values: "Bulk TCP transfer: (continuous file transmission, or repeating 5MB file tr= ansmission);" The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assumi= ng 480p video and 4.5 minutes for some of our experiments. The inter-arriva= l time of these files should have a realistic random distribution as well a= s size of the files. SA: The Cablelabs document as mentioned below has a set of tests for Bulk T= CP for uploading traffic, they use a single continuous TCP flow for certain= testcases and multiple TCP flows with 20MB or 250KB files repeatedly being= sent upstream for other testcases. Their application is upstream traffic w= hich behaves differently from downstream traffic. A Sandvine document on In= ternet traffic (https://www.sandvine.com/downloads/general/global-internet-= phenomena/2013/exposing-the-technical-and-commercial-factors-underlying-int= ernet-quality-of-experience.pdf) shows that real-time entertainment (Netfli= x, YouTube and others) constitute 68% of downstream traffic, (Figure 9) of = which YouTube (progessive download) is at 17% of total traffic. Using repea= ted transfers of TCP files as suggsted in the Cablelabs work would miss the= effect of the random on-off patterns that PD flows generate on the downstr= eam portion. So I would still recommend the use of PD (YouTube) statistics = such as above for modeling the Bulk TCP traffic downstream traffic. "Realistic HTTP web traffic (repeated download of 700kB);" According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM= on a 7KB file would be very different than on a 700KB file. Web traffic al= so has a complex inter-arrival time relationship between the GET requests a= s well as the pages that produce these GET requests. However to make the te= sting simpler, a repeated download of a 7KB file with a Pareto distributed = size and Pareto distributed inter-arrival time may be an OK compromise. SA: I believe Toke has already responded that the CableLabs work mentioned = below refers to another document, which uses 25KB for responses to web page= GETs. This is hugely better than using repeated 700KB downloads and I am O= K with using the model in the other document. I am still concerned that the= average size of the reponse maybe less than 25KB, The statistics collected= by Google (https://developers.google.com/speed/articles/web-metrics) show = an average size of 7KB, but they are 4 years old and I cannot find a more r= ecent number. We expect to stick with the cable labs values [0]. Please let us know if yo= u have any comments regarding this choice. The Cablelabs work does not mention any HTTP adaptive streaming (HAS) traff= ic models - would that be included in the guidelines? Kind regards, Nicolas KUHN [0] http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Manage= ment_Algorithms_DOCSIS_3_0.pdf On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid (Shahid)" > wrote: Hi Nicholas and other authors, Thank you for producing this document. I had some (long) comments where I h= ave also suggested some solutions/text, hopefully you find them helpful. Wh= en quoting from the document, I have used quotes and I have listed my comme= nts as 1-4. 1. The number of tests recommended per AQM scheme seems to be quite large a= s well as many tests that may be best done together are done independently. For example, there are different tests for fairness between flows (section = 3.2.3), level of traffic (mild, medium, heavy) (section 3.2.4), different t= ypes of bursty traffic (3.2.2) and different network environments (section = 3.3). In order to obtain a realistic estimate of AQM comparison, many of th= ese effects may need to be tested together. For example to increase level o= f traffic (in section 3.2.4) only long term TCP flows of 50s are proposed. = It would be better to test the level of congestion with realistic traffic (= and with realistic proportions) as proposed in section 3.2.4. Another examp= le would be to test fairness between flows of same type - e.g. long term TC= P flows when multiple types of TCP flows are present at a AQM controlled bu= ffer or test fairness between flows of different types - e.g. long term TCP= flows and web flows. I noticed that in section 3.3 different network environments are mentioned.= I suggest that tests with appropriate realistic traffic be done on each of= these enviroments with various levels of congestion. One could measure fai= rness from these tests without the need for additional tests. This may have= the benefit of reducing overall number of tests as well. 2. There are four network enviroments mentioned in the document (section 3.= 3): Wifi - in home network. Data-Center communications. BB wireless/satellite. Low and high buffers This is a good set, however, recommed to add: Access networks - and further split access networks into DSL/FTTH networks = and Cable access networks. Cable access networks may have much larger numbe= r of flows than DSL/FTTH since they are shared by a large number of custome= rs There is another type of network environment - narrowband wireless (mobile)= networks where the AQM could be placed on the eNodeB, but perhaps this is = a complicated scenario, and best left for later or in another document. 3. I noticed that the modeling of different types of realistic traffic coul= d be made more accurate (section 3.2.2) "Bulk TCP transfer: (continuous file transmission, or repeating 5MB file tr= ansmission);" The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assumi= ng 480p video and 4.5 minutes for some of our experiments. The inter-arriva= l time of these files should have a realistic random distribution as well a= s size of the files. "Realistic HTTP web traffic (repeated download of 700kB);" According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM= on a 7KB file would be very different than on a 700KB file. Web traffic al= so has a complex inter-arrival time relationship between the GET requests a= s well as the pages that produce these GET requests. However to make the te= sting simpler, a repeated download of a 7KB file with a Pareto distributed = size and Pareto distributed inter-arrival time may be an OK compromise. I noticed that adaptive streaming is not modelled as a type of realistic tr= affic. It is probably the most important and largest flow on the Internet t= oday - if we look at a recent Sandvine report of Internet traffic, about 75= % of Internet traffic is real-time entertainment of which the largest compo= nent is adaptive streaming based video traffic. In order to simulate adapti= ve streaming traffic, one would need a client that changes video quality ra= tes. To keep testing simpler, an option may be to generate realistic chunk = sized files at regular intervals. Average netflix speeds are around 2Mbps. = Typical chunk sizes range from 2s to 10s, but 4s may be a good average. The= chunk sizes should be varied with some random distribution as is typically= observed from real codecs. 4. In section 2.2.5, QoE metrics are mentioned. It may be valuable to spell= out some standard models or expressions to derive QoE from network paramet= ers so that these QoE can be compared between AQMs. In the literature there= is work that derives closed form expressions for QoE. The following are ex= amples and suggestions: Video - the work by Johan de Vriendt et al , "Model for estimating QoE of V= ideo delivered using HTTP Adaptive Streaming", IFIP/IEEE Workshop on QoE Ce= ntric Management, May 2013 may be useful here. The authors used over 500 sa= mples to derive their a closed form expressions. The authors derive the MOS= =3D alpha*(average bit rate) + beta*(standard deviation of bit rates) + si= gma*(no of bit rate switches) + gamma and then produce good values for alph= a, beta, sigma and gamma. Voice - The work in "Voice quality prediction models and their application = in VoIP networks" by L. Sun et al has closed form expressions for MOS score= s for most commercial codes. They derive a long closed form expression with= parameters a-j for key codecs (and then provide a-j for most commercial co= decs) based on end-to-end and packet loss. Web - The delay for one average web GET file could be used as the represent= ative for the page load time Key metric for AQMs on access networks may be QoE since it provides and end= -user view. We have seen that often the absolute goodput does not matter as= much as the stability of the flow for Video (which tends to be the dominan= t traffic). Larger goodput may provide higher VQ levels for Video, but if h= igher volatility is also produced then that higher VQ level may not produce= a higher QoE. On the other hand, best metrics inter data center traffic may be goodput an= d delay since most of that traffic is likely to be inter machine traffic. Best regards, Shahid Akhtar. -----Original Message----- From: aqm [mailto:aqm-bounces@ietf.org] On Behalf = Of Nicolas KUHN Sent: Friday, February 07, 2014 5:58 AM To: Richard Scheffenegger Cc: aqm@ietf.org Subject: [aqm] [AQM Evaluation Guidelines] Dear all, On the behalf of the contributors to the AQM Evaluation Guidelines, I encou= rage active discussion on the draft that we have written. I just submitted the draft as an individual draft to the I-D Submission Too= l [0]. Please let us know what you think of the current document. Kind regards, Nicolas KUHN [0] http://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/ On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" > wrote: Hi, a new month, a new status report. First of all, Wes and I as chairs would like to thank the editors who have = stepped forward to work on the AQM Evaluation Guideline draft. We are reall= y thankful for their burst of efforts in the last couple weeks! We expect that that a document will be ready for submission into the I-D re= pository well before cutoff, as an individual draft, so that the WG can see= what the state of thinking is currently. Also, once the document is publis= hed on datatracker we'd like to encourage active discussion on it. If the submitted -00 draft is felt to have a fairly complete outline of wha= t the current thinking in the WG is, we may be able to ask the WG for forma= l adoption during this IETF meeting, or shortly thereafter. - WG Milestones: - Submit AQM recommendations to IESG for publication, obsoleting RFC 2309 (Goal: January 2014) - draft-ietf-aqm-recommendation is accepted towards this milestone - http://datatracker.ietf.org/doc/draft-ietf-aqm-recommendation/ - the draft has been updated per comments received - if the authors are comfortable, a WGLC might be made on the next revision - we would like to hear from other authors of RFC 2309 on this document, if anyone has contacts to them. - Submit AQM algorithm evaluation guidelines to IESG for publication as Informational (Goal: July 2014) - An editor team has come forth and is working on this - A draft should be available for discussion in the London IETF - We encourage discussion on the list and during the meeting, if this draft should be adopted by the working group. - Submit first algorithm specification to IESG for publication as Proposed Standard (Goal: December 2014) - Since any Proposed Standard algorithm should be in line with the recommendations and be passable versus the evaluation guidelines, this milestone is dependend on the progress of the two work items above. - Currently the only algorithm spec with a complete and active individual-submission draft is PIE - Other items: - draft-pan-aqm-pie is under active work as a proposed algorithm: http://tools.ietf.org/id/draft-pan-aqm-pie-00.txt, however the draft has expired and should be refreshed. - draft-nichols-tsvwg-codel is expired; Dave Taht or others may revive it and/or describe pairing with FQ/SFQ algorithms: http://tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt - Other algorithm specifications are welcome! - Though, we are not planning on adopting algorithms until recommendations and evaluation guidelines are mostly stable Richard Scheffenegger _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm --_000_CF73EABC32A49gwhitecablelabscom_ Content-Type: text/html; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable
All,

As has been mentioned, CableLabs work on AQM evaluations has been (so far) = entirely devoted to evaluating performance for upstream AQM algorithms that= would be implemented in a residential cable modem.   While the traffi= c is, of course, bi-directional, the focus was on identifying realistic test cases that would provide insight o= n the performance of the algorithms that we were considering (in both an ab= solute and a relative sense).  For evaluating downstream AQM performan= ce I would definitely add some additional traffic types, and video traffic would be at the front of my mind there. &= nbsp;In order to do that justice, I think you'd want to have a HAS model th= at incorporates adaptive bitrate capability (but what algorithm?) and estim= ates QoE using something like the de Vriendt model.  For progressive download, it would seem that the sali= ent metrics would be initial buffering delay and then something related to = the number and duration of rebuffering events.

I agree with others that reducing the web model to a single 700kB file down= load is unrealistic.  Also, for recent work we've increased the object= sizes to correspond to a projected 2018 page size (extrapolating from = ;http://httparchive.org/trend= s.php) of about 7MB.

For our 2018 model of Internet traffic (again, upstream focused) where the = user's broadband speed is projected to be 1Gbps down and 200Mbps up, we're = using the following 9 traffic loads:

N F1 F2 Fs    W VG C T 
1 0  0  -     1 1  0 0 
2 1  1  inf   1 1  0 0
3 3  3  50MB  1 1  0 0
4 3  3  50MB  1 1  6 0
5 1  1  19MB* 1 1  0 0
6 3  3  50MB  4 4  0 0
7 3  3  50MB  4 4  6 0
8 5  5  2.5MB 4 4  6 0
9 3  3  50MB  4 1  0 100

where:
N: traffic load index
F1: number of simultaneous FTP uploads with 20ms RTT
F2: number of simultaneous FTP uploads wit= h 100ms RTT
Fs: FTP filesize
W: number of simultaneous web users
VG: number of simultane= ous VoIP/gaming sessions
C: CBR data rat= e (Mbps)
T: number of to= rrent (LEDBAT) connections 
*Filesize and r= epetition pattern chosen to exercise DOCSIS "powerboost" feature
   

Since evaluation comes before deployment (by a long lead time in some cases= ), and, once deployed, algorithms may have a long life, I would encourage t= he AQM Evaluation Guidelines draft to consider not just what traffic looks = like historically, but what we can reasonably predict for the future.

-Greg


From: <Akhtar>, "Shahid = (Shahid)" <shah= id.akhtar@alcatel-lucent.com>
Date: Tuesday, April 15, 2014 at 4:= 47 PM
To: Nicolas KUHN <nicolas.kuhn@telecom-bretagne.eu&= gt;
Cc: Richard Scheffenegger <rs@netapp.com>, "aqm@ietf.org" <aq= m@ietf.org>
Subject: Re: [aqm] [AQM Evaluation = Guidelines]

Hi Nicholas,
 
Please see comments inline.
 
Best regards,
 
-Shahid.

From: Nicolas KUHN [mailto:nicolas.kuhn@telecom-bretagne.e= u]
Sent: Tuesday, April 15, 2014 3:24 AM
To: Akhtar, Shahid (Shahid)
Cc: aqm@ietf.org; Richard Scheff= enegger
Subject: Re: [aqm] [AQM Evaluation Guidelines]

Hi Shahid Akhtar, all,

After the discussions in London IETF 89, there has been lots of modifi= cations to the draft organisation (as we already advertised on the list).&n= bsp;
The authors and I just discussed your points considering the size of t= he files for the bulk transfer and the HTTP traffic. 
In the current version of the draft, we consider the values mentioned = in the cablelabs work [0].

As a result, we consider bulk TCP transfers (repeating 5MB file transm= ission) and realistic HTTP web traffic (repeated download of 700kB). As a r= eminder, please find here the comments of Shahid Akhtar regarding these val= ues:

"Bulk TCP transfer: (continuous file transmission, or repeating 5= MB file transmission);"
The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assuming 480p video and 4.5 minutes f= or some of our experiments. The inter-arrival time of these files should ha= ve a realistic random distribution as well as size of the files.  
 
SA: The Cablelabs document as mentioned below has a se= t of tests for Bulk TCP for uploading traffic, they use a single continuous= TCP flow for certain testcases and multiple TCP flows with 20MB or 250KB files repeatedly being sent upstream for other te= stcases. Their application is upstream traffic which behaves differently fr= om downstream traffic. A Sandvine document on Internet traffic (https://www.sandvine.com/downloads/general/global-int= ernet-phenomena/2013/exposing-the-technical-and-commercial-factors-underlyi= ng-internet-quality-of-experience.pdf) shows that real-time entertainment (Netflix, YouTube and others) constitut= e 68% of downstream traffic, (Figure 9) of which YouTube (progess= ive download) is at 17% of total traffic. Using repeated transfers of TCP f= iles as suggsted in the Cablelabs work would miss the effect of the random on-off patterns that PD flows generate on th= e downstream portion. So I would still recommend the use of PD (YouTub= e) statistics such as above for modeling the Bulk TCP traffic downstream tr= affic.
"Realistic HTTP web traffic (repeated download of 700kB);" According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM on a 7KB file would be very differen= t than on a 700KB file. Web traffic also has a complex inter-arrival time r= elationship between the GET requests as well as the pages that produce thes= e GET requests. However to make the testing simpler, a repeated download of a 7KB file with a Pareto distr= ibuted size and Pareto distributed inter-arrival time may be an OK compromi= se. 
 
SA: I believe Toke has already responded that the= CableLabs work mentioned below refers to another document, which uses 25KB= for responses to web page GETs. This is hugely better than using repeated 700KB downloads and I am OK with using the mode= l in the other document. I am still concerned that the average size of the = reponse maybe less than 25KB, The statistics collected by Google (https://developers.goog= le.com/speed/articles/web-metrics) show an average size of 7KB, but they are 4 years old and I cannot find a = more recent number.
 
We expect to stick with the cable labs values [0]. Please let us know = if you have any comments regarding this choice.  
 
The Cablelabs work does not mention any HTTP adapt= ive streaming (HAS) traffic models - would that be included in the&nbs= p;guidelines? 
Kind regards, 

Nicolas KUHN


On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid (Shahid)" <<= a href=3D"mailto:shahid.akhtar@alcatel-lucent.com">shahid.akhtar@alcatel-lu= cent.com> wrote:

Hi Nicholas and other authors,

Thank you for producing this document. I had some (long) comments where I h= ave also suggested some solutions/text, hopefully you find them helpful. Wh= en quoting from the document, I have used quotes and I have listed my comme= nts as 1-4.


1. The number of tests recommended per AQM scheme seems to be quite large a= s well as many tests that may be best done together are done independently.=

For example, there are different tests for fairness between flows (section = 3.2.3), level of traffic (mild, medium, heavy) (section 3.2.4), different t= ypes of bursty traffic (3.2.2) and different network environments (section = 3.3). In order to obtain a realistic estimate of AQM comparison, many of these effects may need to be tested to= gether. For example to increase level of traffic (in section 3.2.4) only lo= ng term TCP flows of 50s are proposed. It would be better to test the level= of congestion with realistic traffic (and with realistic proportions) as proposed in section 3.2.4. Another exa= mple would be to test fairness between flows of same type - e.g. long term = TCP flows when multiple types of TCP flows are present at a AQM controlled = buffer or test fairness between flows of different types - e.g. long term TCP flows and web flows.

I noticed that in section 3.3 different network environments are mentioned.= I suggest that tests with appropriate realistic traffic be done on each of= these enviroments with various levels of congestion. One could measure fai= rness from these tests without the need for additional tests. This may have the benefit of reducing overall n= umber of tests as well.


2. There are four network enviroments mentioned in the document (section 3.= 3):
Wifi - in home network.
Data-Center communications.
BB wireless/satellite.
Low and high buffers

This is a good set, however, recommed to add:

Access networks - and further split access networks into DSL/FTTH networks = and Cable access networks. Cable access networks may have much larger numbe= r of flows than DSL/FTTH since they are shared by a large number of custome= rs

There is another type of network environment - narrowband wireless (mobile)= networks where the AQM could be placed on the eNodeB, but perhaps this is = a complicated scenario, and best left for later or in another document.


3. I noticed that the modeling of different types of realistic traffic coul= d be made more accurate (section 3.2.2)

"Bulk TCP transfer: (continuous file transmission, or repeating 5MB fi= le transmission);"
The largest type of traffic on the Internet that uses bulk TCP - or large f= iles via TCP is progressive download. The average speed of a Youtube video = is over 3Mbps and it is on average 4.5 minutes long - so the average file s= ize is considerably larger than 5MB. We have used an average of 38MB assuming 480p video and 4.5 minutes f= or some of our experiments. The inter-arrival time of these files should ha= ve a realistic random distribution as well as size of the files.

"Realistic HTTP web traffic (repeated download of 700kB);"
According to statistics that Google collected from 2010 about web traffic, = the average GET file size for web traffic was only 7KB. It is very importan= t to have the size of these files correct since AQMs can help transfers of = small files when large file downloads are also occurring. The effect of AQM on a 7KB file would be very differen= t than on a 700KB file. Web traffic also has a complex inter-arrival time r= elationship between the GET requests as well as the pages that produce thes= e GET requests. However to make the testing simpler, a repeated download of a 7KB file with a Pareto distr= ibuted size and Pareto distributed inter-arrival time may be an OK compromi= se.

I noticed that adaptive streaming is not modelled as a type of realistic tr= affic. It is probably the most important and largest flow on the Internet t= oday - if we look at a recent Sandvine report of Internet traffic, about 75= % of Internet traffic is real-time entertainment of which the largest component is adaptive streaming based v= ideo traffic. In order to simulate adaptive streaming traffic, one would ne= ed a client that changes video quality rates. To keep testing simpler, an o= ption may be to generate realistic chunk sized files at regular intervals. Average netflix speeds are around = 2Mbps. Typical chunk sizes range from 2s to 10s, but 4s may be a good avera= ge. The chunk sizes should be varied with some random distribution as is ty= pically observed from real codecs.


4. In section 2.2.5, QoE metrics are mentioned. It may be valuable to spell= out some standard models or expressions to derive QoE from network paramet= ers so that these QoE can be compared between AQMs. In the literature there= is work that derives closed form expressions for QoE. The following are examples and suggestions:

Video - the work by Johan de Vriendt et al , "Model for estimating QoE= of Video delivered using HTTP Adaptive Streaming", IFIP/IEEE Workshop= on QoE Centric Management, May 2013 may be useful here. The authors used o= ver 500 samples to derive their a closed form expressions. The authors derive the MOS =3D alpha*(average bit rate) += beta*(standard deviation of bit rates) + sigma*(no of bit rate switche= s) + gamma and then produce good values for alpha, beta, sigma and gamm= a.

Voice - The work in "Voice quality prediction models and their applica= tion in VoIP networks" by L. Sun et al has closed form expressions for= MOS scores for most commercial codes. They derive a long closed form expre= ssion with parameters a-j for key codecs (and then provide a-j for most commercial codecs) based on end-to-end and packe= t loss.  

Web - The delay for one average web GET file could be used as the represent= ative for the page load time

Key metric for AQMs on access networks may be QoE since it provides and end= -user view. We have seen that often the absolute goodput does not matter as= much as the stability of the flow for Video (which tends to be the dominan= t traffic). Larger goodput may provide higher VQ levels for Video, but if higher volatility is also produced then= that higher VQ level may not produce a higher QoE.

On the other hand, best metrics inter data center traffic may be goodput an= d delay since most of that traffic is likely to be inter machine traffic.
Best regards,

Shahid Akhtar.

-----Original Message-----
From: aqm [mailto:aqm-bounces@ietf.org] On Behalf Of Nicolas KUHN
Sent: Friday, February 07, 2014 5:58 AM
To: Richard Scheffenegger
Cc: aqm@ietf.org
Subject: [aqm] [AQM Evaluation Guidelines]

Dear all,

On the behalf of the contributors to the AQM Evaluation Guidelines, I encou= rage active discussion on the draft that we have written.
I just submitted the draft as an individual draft to the I-D Submission Too= l [0].

Please let us know what you think of the current document.

Kind regards,

Nicolas KUHN

[0] http://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/

On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" <rs@netapp.com> wrote:

Hi,

a new month, a new status report.

First of all, Wes and I as chairs would like to thank the editors who have = stepped forward to work on the AQM Evaluation Guideline draft. We are reall= y thankful for their burst of efforts in the last couple weeks!

We expect that that a document will be ready for submission into the I-D re= pository well before cutoff, as an individual draft, so that the WG can see= what the state of thinking is currently. Also, once the document is publis= hed on datatracker we'd like to encourage active discussion on it.

If the submitted -00 draft is felt to have a fairly complete outline of wha= t the current thinking in the WG is, we may be able to ask the WG for forma= l adoption during this IETF meeting, or shortly thereafter.




- WG Milestones:
 - Submit AQM recommendations to IESG for publication, obsoleting RFC<= br> 2309 (Goal: January 2014)
   - draft-ietf-aqm-recommendation is accepted towards this = milestone
   - http://datatracker.ietf.org/doc/draft-ietf-aqm-recommen= dation/
   - the draft has been updated per comments received
   - if the authors are comfortable, a WGLC might be made on= the next
revision
   - we would like to hear from other authors of RFC 2309 on= this
document, if anyone has contacts to them.


 - Submit AQM algorithm evaluation guidelines to IESG for publication =
as Informational (Goal: July 2014)
   - An editor team has come forth and is working on this    - A draft should be available for discussion in the Londo= n IETF
   - We encourage discussion on the list and during the meet= ing, if this
      draft should be adopted by the working = group.

 - Submit first algorithm specification to IESG for publication as Proposed Standard (Goal: December 2014)
   - Since any Proposed Standard algorithm should be in line= with the
recommendations and be passable versus the evaluation guidelines, this
milestone is dependend on the progress of the two work items above.
   - Currently the only algorithm spec with a complete and a= ctive
individual-submission draft is PIE

- Other items:
 - draft-pan-aqm-pie is under active work as a proposed algorithm:
   http://tools.ietf.org/id/draft-pan-aqm-pie-00.txt, however the dra= ft
   has expired and should be refreshed.
 - draft-nichols-tsvwg-codel is expired; Dave Taht or others may
revive it and/or describe pairing with FQ/SFQ algorithms:
   http://tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt<= br>  - Other algorithm specifications are welcome!
   - Though, we are not planning on adopting algorithms unti= l
recommendations and evaluation guidelines are mostly stable


Richard Scheffenegger

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/= mailman/listinfo/aqm

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/= mailman/listinfo/aqm

 --_000_CF73EABC32A49gwhitecablelabscom_-- From nobody Wed Apr 16 09:25:33 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B5BD1A025B for ; Wed, 16 Apr 2014 09:25:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TZdMQ0Xchs1H for ; Wed, 16 Apr 2014 09:25:22 -0700 (PDT) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) by ietfa.amsl.com (Postfix) with ESMTP id 065E81A0248 for ; Wed, 16 Apr 2014 09:25:21 -0700 (PDT) Received: by mail-wg0-f49.google.com with SMTP id a1so11395032wgh.32 for ; Wed, 16 Apr 2014 09:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=iP6Pg8vds4eOQKbtNi9rFhgO9Vc5fxqzgve54dC7daQ=; b=YBjxIwZUL4Z5VvC9VVgmxtEio+4B4FWUcdE+G753NbPM0uHWn1tL0DTXQ9Mda6YRS/ Eh/CsMlRqFHpCj2Ba4wQ02NUKEtPknUID0/+5a2+TDcZNyxgx0kUDKUbSxNXg+FQk7Tu 7KTeJSgtfyytZFPECYvI3Nuz2mSKDgiuUo6TspJzjHcORNWnD5ru7NvXcwt5ZXjVcEub IvTW3m6QivmVJxRmZA5eFeLJjbytco9F0Lc8Dk+tL+2jJiW3qFGyWNBUG1eemb7orRNm taijlk0NKsc/OfYbTDaV7VKDRMwpn5Hi52WdVImYPHsqokgLBEXP0JPhizaWx9Gdq/FV L23g== MIME-Version: 1.0 X-Received: by 10.194.185.148 with SMTP id fc20mr7817475wjc.27.1397665518273; Wed, 16 Apr 2014 09:25:18 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Wed, 16 Apr 2014 09:25:18 -0700 (PDT) In-Reply-To: References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> Date: Wed, 16 Apr 2014 09:25:18 -0700 Message-ID: From: Dave Taht To: Greg White Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/9E9c1dKc8sh30D6NzM3F5EcSTiA Cc: "Akhtar, Shahid \(Shahid\)" , "aqm@ietf.org" , Nicolas KUHN , Richard Scheffenegger Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 16:25:28 -0000 On Wed, Apr 16, 2014 at 8:28 AM, Greg White wrote: > All, > > As has been mentioned, CableLabs work on AQM evaluations has been (so far= ) > entirely devoted to evaluating performance for upstream AQM algorithms th= at > would be implemented in a residential cable modem. While the traffic is= , > of course, bi-directional, the focus was on identifying realistic test ca= ses > that would provide insight on the performance of the algorithms that we w= ere > considering (in both an absolute and a relative sense). For evaluating > downstream AQM performance I would definitely add some additional traffic > types, and video traffic would be at the front of my mind there. In orde= r > to do that justice, I think you'd want to have a HAS model that incorpora= tes > adaptive bitrate capability (but what algorithm?) and estimates QoE using > something like the de Vriendt model. +1 > For progressive download, it would > seem that the salient metrics would be initial buffering delay and then > something related to the number and duration of rebuffering events. > > I agree with others that reducing the web model to a single 700kB file > download is unrealistic. Also, for recent work we've increased the objec= t > sizes to correspond to a projected 2018 page size (extrapolating from > http://httparchive.org/trends.php) of about 7MB. While the trendline is impressively "up": http://httparchive.org/trends.php?s=3DAll&minlabel=3DNov+15+2010&maxlabel= =3DApr+1+2014 I'm reluctant to do a linear extrapolation of that trendline from 3 years of data. Perhaps a longer term store, like that of archive.org, could be explored. I'm not one to predict a "Peak Web", but the idea of a typical web page load cracking 7MB in 5 years disturbs me. > > For our 2018 model of Internet traffic (again, upstream focused) where th= e I do like, very much, trying to come up with projections for what traffic w= ill look like in 5 years. I'll ask around... > user's broadband speed is projected to be 1Gbps down and 200Mbps up, we'r= e > using the following 9 traffic loads: > > N F1 F2 Fs W VG C T > 1 0 0 - 1 1 0 0 > 2 1 1 inf 1 1 0 0 > 3 3 3 50MB 1 1 0 0 > 4 3 3 50MB 1 1 6 0 > 5 1 1 19MB* 1 1 0 0 > 6 3 3 50MB 4 4 0 0 > 7 3 3 50MB 4 4 6 0 > 8 5 5 2.5MB 4 4 6 0 > 9 3 3 50MB 4 1 0 100 > > where: > N: traffic load index > F1: number of simultaneous FTP uploads with 20ms RTT > F2: number of simultaneous FTP uploads with 100ms RTT I note that the average RTT in the US is apparently 38ms, and trending downward at about 1ms/year on wired connections. Key web sites in major cities (e.g. googles of the world) are below 5ms. I wish I knew netflix's typical RTT from ISP co-location facilities... high bandwidth needing providers will continue to also reduce their RTT's as much as they can. > Fs: FTP filesize > W: number of simultaneous web users I have two mental models for cable internet connectivity - one is in the home, the other is in small (and increasingly larger) business. Given the first concept, it is hard to extrapolate as far as your larger models, and given the second, I'd imagine web, voip, and videoconferencing traffic to be a much larger percentage of the overall model. > VG: number of simultaneous VoIP/gaming sessions > C: CBR data rate (Mbps) We really need a good model for videconferencing. I have no idea (except for live streaming delivery) what the CBR stream is intended to represent. Security cameras backhauled to Big Brother? Another item is the amount of "noise" traffic - dns, iot devices reporting tempurature, weather, etc, being transported. > T: number of torrent (LEDBAT) connections Torrent does not behave as per your model; I just had a long meeting with brahm over that. Presently per "new" torrent you typically see dozens = of sessions "connected", but only 6 or so sessions exchanging data in both directions, with a new connection explored and rotated to every 15 seconds or so. After a file is fully transferred, the same logic remains, but upload only. Anticipating that a given site would have 16 simultaneous torrents over a 100 active connections being redistributed seems overlarge, and I imagine that a typical number is bound much lower by the bittorrent software. > *Filesize and repetition pattern chosen to exercise DOCSIS "powerboost" > feature Yes, I'd like not only to have that but to actually have running code in linux that could emulate it... :) > > > Since evaluation comes before deployment (by a long lead time in some > cases), and, once deployed, algorithms may have a long life, I would > encourage the AQM Evaluation Guidelines draft to consider not just what > traffic looks like historically, but what we can reasonably predict for t= he > future. +1 Now that we are transporting hi resolution video, voice, web, and data in addition to the traditional email and chat, I have great difficulty assu= ming that bandwidth demands will continue to grow as they have, as the new stuff is mostly "fixed" in its bandwidth needs, rather than dynamic like tcp. ... Unless someone comes up with a way of encapsulating other senses or some application that truly demands high bandwidth that can work at multiple RTTs. There is a lot downward pressure to "do more with less", for example, as with things like webrtc that are bundled in the browser and don't require a lot of javascript support, and a lot of caching of core code that takes place, and a lot of work in reducing RTTs in new protocols like QUIC. (but I freely admit I could be wrong and would love to work on projections that make sense!) > -Greg > > > From: , "Shahid (Shahid)" > Date: Tuesday, April 15, 2014 at 4:47 PM > To: Nicolas KUHN > Cc: Richard Scheffenegger , "aqm@ietf.org" > > Subject: Re: [aqm] [AQM Evaluation Guidelines] > > Hi Nicholas, > > Please see comments inline. > > Best regards, > > -Shahid. > > ________________________________ > From: Nicolas KUHN [mailto:nicolas.kuhn@telecom-bretagne.eu] > Sent: Tuesday, April 15, 2014 3:24 AM > To: Akhtar, Shahid (Shahid) > Cc: aqm@ietf.org; Richard Scheffenegger > Subject: Re: [aqm] [AQM Evaluation Guidelines] > > Hi Shahid Akhtar, all, > > After the discussions in London IETF 89, there has been lots of > modifications to the draft organisation (as we already advertised on the > list). > The authors and I just discussed your points considering the size of the > files for the bulk transfer and the HTTP traffic. > In the current version of the draft, we consider the values mentioned in = the > cablelabs work [0]. > > As a result, we consider bulk TCP transfers (repeating 5MB file > transmission) and realistic HTTP web traffic (repeated download of 700kB)= . > As a reminder, please find here the comments of Shahid Akhtar regarding > these values: > > "Bulk TCP transfer: (continuous file transmission, or repeating 5MB file > transmission);" > The largest type of traffic on the Internet that uses bulk TCP - or large > files via TCP is progressive download. The average speed of a Youtube vid= eo > is over 3Mbps and it is on average 4.5 minutes long - so the average file > size is considerably larger than 5MB. We have used an average of 38MB > assuming 480p video and 4.5 minutes for some of our experiments. The > inter-arrival time of these files should have a realistic random > distribution as well as size of the files. > > SA: The Cablelabs document as mentioned below has a set of tests for Bulk > TCP for uploading traffic, they use a single continuous TCP flow for cert= ain > testcases and multiple TCP flows with 20MB or 250KB files repeatedly bein= g > sent upstream for other testcases. Their application is upstream traffic > which behaves differently from downstream traffic. A Sandvine document on > Internet traffic > (https://www.sandvine.com/downloads/general/global-internet-phenomena/201= 3/exposing-the-technical-and-commercial-factors-underlying-internet-quality= -of-experience.pdf) > shows that real-time entertainment (Netflix, YouTube and others) constitu= te > 68% of downstream traffic, (Figure 9) of which YouTube (progessive downlo= ad) > is at 17% of total traffic. Using repeated transfers of TCP files as > suggsted in the Cablelabs work would miss the effect of the random on-off > patterns that PD flows generate on the downstream portion. So I would sti= ll > recommend the use of PD (YouTube) statistics such as above for modeling t= he > Bulk TCP traffic downstream traffic. > > "Realistic HTTP web traffic (repeated download of 700kB);" > According to statistics that Google collected from 2010 about web traffic= , > the average GET file size for web traffic was only 7KB. It is very import= ant > to have the size of these files correct since AQMs can help transfers of > small files when large file downloads are also occurring. The effect of A= QM > on a 7KB file would be very different than on a 700KB file. Web traffic a= lso > has a complex inter-arrival time relationship between the GET requests as > well as the pages that produce these GET requests. However to make the > testing simpler, a repeated download of a 7KB file with a Pareto distribu= ted > size and Pareto distributed inter-arrival time may be an OK compromise. > > SA: I believe Toke has already responded that the CableLabs work mentione= d > below refers to another document, which uses 25KB for responses to web pa= ge > GETs. This is hugely better than using repeated 700KB downloads and I am = OK > with using the model in the other document. I am still concerned that the > average size of the reponse maybe less than 25KB, The statistics collecte= d > by Google (https://developers.google.com/speed/articles/web-metrics) show= an > average size of 7KB, but they are 4 years old and I cannot find a more > recent number. > > We expect to stick with the cable labs values [0]. Please let us know if = you > have any comments regarding this choice. > > The Cablelabs work does not mention any HTTP adaptive streaming (HAS) > traffic models - would that be included in the guidelines? > > Kind regards, > > Nicolas KUHN > > [0] > http://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Manageme= nt_Algorithms_DOCSIS_3_0.pdf > > On Feb 20, 2014, at 6:48 AM, "Akhtar, Shahid (Shahid)" > wrote: > > Hi Nicholas and other authors, > > Thank you for producing this document. I had some (long) comments where I > have also suggested some solutions/text, hopefully you find them helpful. > When quoting from the document, I have used quotes and I have listed my > comments as 1-4. > > > 1. The number of tests recommended per AQM scheme seems to be quite large= as > well as many tests that may be best done together are done independently. > > For example, there are different tests for fairness between flows (sectio= n > 3.2.3), level of traffic (mild, medium, heavy) (section 3.2.4), different > types of bursty traffic (3.2.2) and different network environments (secti= on > 3.3). In order to obtain a realistic estimate of AQM comparison, many of > these effects may need to be tested together. For example to increase lev= el > of traffic (in section 3.2.4) only long term TCP flows of 50s are propose= d. > It would be better to test the level of congestion with realistic traffic > (and with realistic proportions) as proposed in section 3.2.4. Another > example would be to test fairness between flows of same type - e.g. long > term TCP flows when multiple types of TCP flows are present at a AQM > controlled buffer or test fairness between flows of different types - e.g= . > long term TCP flows and web flows. > > I noticed that in section 3.3 different network environments are mentione= d. > I suggest that tests with appropriate realistic traffic be done on each o= f > these enviroments with various levels of congestion. One could measure > fairness from these tests without the need for additional tests. This may > have the benefit of reducing overall number of tests as well. > > > 2. There are four network enviroments mentioned in the document (section > 3.3): > Wifi - in home network. > Data-Center communications. > BB wireless/satellite. > Low and high buffers > > This is a good set, however, recommed to add: > > Access networks - and further split access networks into DSL/FTTH network= s > and Cable access networks. Cable access networks may have much larger num= ber > of flows than DSL/FTTH since they are shared by a large number of custome= rs > > There is another type of network environment - narrowband wireless (mobil= e) > networks where the AQM could be placed on the eNodeB, but perhaps this is= a > complicated scenario, and best left for later or in another document. > > > 3. I noticed that the modeling of different types of realistic traffic co= uld > be made more accurate (section 3.2.2) > > "Bulk TCP transfer: (continuous file transmission, or repeating 5MB file > transmission);" > The largest type of traffic on the Internet that uses bulk TCP - or large > files via TCP is progressive download. The average speed of a Youtube vid= eo > is over 3Mbps and it is on average 4.5 minutes long - so the average file > size is considerably larger than 5MB. We have used an average of 38MB > assuming 480p video and 4.5 minutes for some of our experiments. The > inter-arrival time of these files should have a realistic random > distribution as well as size of the files. > > "Realistic HTTP web traffic (repeated download of 700kB);" > According to statistics that Google collected from 2010 about web traffic= , > the average GET file size for web traffic was only 7KB. It is very import= ant > to have the size of these files correct since AQMs can help transfers of > small files when large file downloads are also occurring. The effect of A= QM > on a 7KB file would be very different than on a 700KB file. Web traffic a= lso > has a complex inter-arrival time relationship between the GET requests as > well as the pages that produce these GET requests. However to make the > testing simpler, a repeated download of a 7KB file with a Pareto distribu= ted > size and Pareto distributed inter-arrival time may be an OK compromise. > > I noticed that adaptive streaming is not modelled as a type of realistic > traffic. It is probably the most important and largest flow on the Intern= et > today - if we look at a recent Sandvine report of Internet traffic, about > 75% of Internet traffic is real-time entertainment of which the largest > component is adaptive streaming based video traffic. In order to simulate > adaptive streaming traffic, one would need a client that changes video > quality rates. To keep testing simpler, an option may be to generate > realistic chunk sized files at regular intervals. Average netflix speeds = are > around 2Mbps. Typical chunk sizes range from 2s to 10s, but 4s may be a g= ood > average. The chunk sizes should be varied with some random distribution a= s > is typically observed from real codecs. > > > 4. In section 2.2.5, QoE metrics are mentioned. It may be valuable to spe= ll > out some standard models or expressions to derive QoE from network > parameters so that these QoE can be compared between AQMs. In the literat= ure > there is work that derives closed form expressions for QoE. The following > are examples and suggestions: > > Video - the work by Johan de Vriendt et al , "Model for estimating QoE of > Video delivered using HTTP Adaptive Streaming", IFIP/IEEE Workshop on QoE > Centric Management, May 2013 may be useful here. The authors used over 50= 0 > samples to derive their a closed form expressions. The authors derive the > MOS =3D alpha*(average bit rate) + beta*(standard deviation of bit rates)= + > sigma*(no of bit rate switches) + gamma and then produce good values for > alpha, beta, sigma and gamma. > > Voice - The work in "Voice quality prediction models and their applicatio= n > in VoIP networks" by L. Sun et al has closed form expressions for MOS sco= res > for most commercial codes. They derive a long closed form expression with > parameters a-j for key codecs (and then provide a-j for most commercial > codecs) based on end-to-end and packet loss. > > Web - The delay for one average web GET file could be used as the > representative for the page load time > > Key metric for AQMs on access networks may be QoE since it provides and > end-user view. We have seen that often the absolute goodput does not matt= er > as much as the stability of the flow for Video (which tends to be the > dominant traffic). Larger goodput may provide higher VQ levels for Video, > but if higher volatility is also produced then that higher VQ level may n= ot > produce a higher QoE. > > On the other hand, best metrics inter data center traffic may be goodput = and > delay since most of that traffic is likely to be inter machine traffic. > > Best regards, > > Shahid Akhtar. > > -----Original Message----- > From: aqm [mailto:aqm-bounces@ietf.org] On Behalf Of Nicolas KUHN > Sent: Friday, February 07, 2014 5:58 AM > To: Richard Scheffenegger > Cc: aqm@ietf.org > Subject: [aqm] [AQM Evaluation Guidelines] > > Dear all, > > On the behalf of the contributors to the AQM Evaluation Guidelines, I > encourage active discussion on the draft that we have written. > I just submitted the draft as an individual draft to the I-D Submission T= ool > [0]. > > Please let us know what you think of the current document. > > Kind regards, > > Nicolas KUHN > > [0] http://datatracker.ietf.org/doc/draft-kuhn-aqm-eval-guidelines/ > > > On Feb 5, 2014, at 10:01 PM, "Scheffenegger, Richard" wro= te: > > Hi, > > a new month, a new status report. > > First of all, Wes and I as chairs would like to thank the editors who hav= e > stepped forward to work on the AQM Evaluation Guideline draft. We are rea= lly > thankful for their burst of efforts in the last couple weeks! > > We expect that that a document will be ready for submission into the I-D > repository well before cutoff, as an individual draft, so that the WG can > see what the state of thinking is currently. Also, once the document is > published on datatracker we'd like to encourage active discussion on it. > > If the submitted -00 draft is felt to have a fairly complete outline of w= hat > the current thinking in the WG is, we may be able to ask the WG for forma= l > adoption during this IETF meeting, or shortly thereafter. > > > > > - WG Milestones: > - Submit AQM recommendations to IESG for publication, obsoleting RFC > 2309 (Goal: January 2014) > - draft-ietf-aqm-recommendation is accepted towards this milestone > - http://datatracker.ietf.org/doc/draft-ietf-aqm-recommendation/ > - the draft has been updated per comments received > - if the authors are comfortable, a WGLC might be made on the next > revision > - we would like to hear from other authors of RFC 2309 on this > document, if anyone has contacts to them. > > > - Submit AQM algorithm evaluation guidelines to IESG for publication > as Informational (Goal: July 2014) > - An editor team has come forth and is working on this > - A draft should be available for discussion in the London IETF > - We encourage discussion on the list and during the meeting, if this > draft should be adopted by the working group. > > - Submit first algorithm specification to IESG for publication as > Proposed Standard (Goal: December 2014) > - Since any Proposed Standard algorithm should be in line with the > recommendations and be passable versus the evaluation guidelines, this > milestone is dependend on the progress of the two work items above. > - Currently the only algorithm spec with a complete and active > individual-submission draft is PIE > > - Other items: > - draft-pan-aqm-pie is under active work as a proposed algorithm: > http://tools.ietf.org/id/draft-pan-aqm-pie-00.txt, however the draft > has expired and should be refreshed. > - draft-nichols-tsvwg-codel is expired; Dave Taht or others may > revive it and/or describe pairing with FQ/SFQ algorithms: > http://tools.ietf.org/id/draft-nichols-tsvwg-codel-01.txt > - Other algorithm specifications are welcome! > - Though, we are not planning on adopting algorithms until > recommendations and evaluation guidelines are mostly stable > > > Richard Scheffenegger > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu Apr 17 00:14:20 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C2A9A1A0075 for ; Thu, 17 Apr 2014 00:14:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_HI=-5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2oP4KvTNg0yC for ; Thu, 17 Apr 2014 00:14:15 -0700 (PDT) Received: from hoemail2.alcatel.com (hoemail2.alcatel.com [192.160.6.149]) by ietfa.amsl.com (Postfix) with ESMTP id 46FAB1A00F2 for ; Thu, 17 Apr 2014 00:14:14 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (h135-239-2-42.lucent.com [135.239.2.42]) by hoemail2.alcatel.com (8.13.8/IER-o) with ESMTP id s3H7E7mq014013 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 17 Apr 2014 02:14:09 -0500 (CDT) Received: from FR711WXCHHUB02.zeu.alcatel-lucent.com (fr711wxchhub02.zeu.alcatel-lucent.com [135.239.2.112]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id s3H7E6pf025676 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 17 Apr 2014 09:14:07 +0200 Received: from FR712WXCHMBA13.zeu.alcatel-lucent.com ([169.254.5.3]) by FR711WXCHHUB02.zeu.alcatel-lucent.com ([135.239.2.112]) with mapi id 14.02.0247.003; Thu, 17 Apr 2014 09:14:06 +0200 From: "LAUTENSCHLAEGER, Wolfram (Wolfram)" To: "aqm@ietf.org" Thread-Topic: Re: [aqm][AQM Evaluation Guidelines] Thread-Index: Ac9aDAL+3BFLjfBnTCutyrji/UNqLw== Date: Thu, 17 Apr 2014 07:14:06 +0000 Message-ID: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/Dj9_ucfuLALGyymlkbMIlmZM1Zc Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 07:14:17 -0000 All, I'm not sure if we can reach good test coverage if we apply more and more d= etailed "realistic" test cases, particularly if we are looking into the fut= ure. The more detailed the test cases are, the more relevant others we leav= e out. Shouldn't we more concentrate on what we expect from a good AQM? The only thing we might expect from an AQM is to prevent greedy TCP sources= from drawing buffers permanently towards full state, instead of the much b= etter almost empty state, while maintaining full link utilization. In all o= ther situations, I'm convinced, AQM cannot improve a lot. But at least, in = these cases, it should not make things worse. Of course the greedy TCP case can be overlaid by others (unresponsive flows= , application paced flows, a renewal process of short lived flows, synchron= ized start of flows etc.etc.), and we should take this into account. But we= should not expect a lot if those "others" dominate the scenario. Example: An overlay of finite flows arriving as a renewal process with inte= r arrival time T (seconds) and flow size S (bits). And a link capacity of C= . In cases with C < T*S the link collapses with an infinite number of uncom= pleted flows, each of almost zero rate. In the opposite case, C > T*S the l= ink cannot be loaded permanently at 100% since the traffic offer is simply = not there. There is little to gain by AQM. It is only a thin range short be= fore the collapse. Wolfram From nobody Thu Apr 17 00:29:44 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9782D1A0048 for ; Thu, 17 Apr 2014 00:29:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.9 X-Spam-Level: X-Spam-Status: No, score=-6.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J-iU-ehJElJS for ; Thu, 17 Apr 2014 00:29:40 -0700 (PDT) Received: from hoemail1.alcatel.com (hoemail1.alcatel.com [192.160.6.148]) by ietfa.amsl.com (Postfix) with ESMTP id 761861A000E for ; Thu, 17 Apr 2014 00:29:39 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (h135-239-2-122.lucent.com [135.239.2.122]) by hoemail1.alcatel.com (8.13.8/IER-o) with ESMTP id s3H7TY7I008336 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 17 Apr 2014 02:29:36 -0500 (CDT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id s3H7TYXr007522 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 17 Apr 2014 09:29:34 +0200 Received: from FR712WXCHMBA13.zeu.alcatel-lucent.com ([169.254.5.3]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.02.0247.003; Thu, 17 Apr 2014 09:29:34 +0200 From: "LAUTENSCHLAEGER, Wolfram (Wolfram)" To: "aqm@ietf.org" Thread-Topic: [aqm] [AQM Evaluation Guidelines] Thread-Index: Ac9aDAL++yRrhaEE+02TdcZDll87BQAAZuxw Date: Thu, 17 Apr 2014 07:29:33 +0000 Message-ID: <0A452E1DADEF254C9A7AC1969B878128068975@FR712WXCHMBA13.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> In-Reply-To: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/YbBOOUuDLEj9Ozak7vEm_GTlxa8 Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 07:29:42 -0000 Sorry,=20 It should be C < S/T and C > S/T, respectively, see inline Wolfram > -----Urspr=FCngliche Nachricht----- > Von: aqm [mailto:aqm-bounces@ietf.org] Im Auftrag von > LAUTENSCHLAEGER, Wolfram (Wolfram) > Gesendet: Donnerstag, 17. April 2014 09:14 > An: aqm@ietf.org > Betreff: Re: [aqm] [AQM Evaluation Guidelines] >=20 > All, >=20 > I'm not sure if we can reach good test coverage if we apply more and more > detailed "realistic" test cases, particularly if we are looking into the = future. > The more detailed the test cases are, the more relevant others we leave o= ut. >=20 > Shouldn't we more concentrate on what we expect from a good AQM? >=20 > The only thing we might expect from an AQM is to prevent greedy TCP > sources from drawing buffers permanently towards full state, instead of t= he > much better almost empty state, while maintaining full link utilization. = In all > other situations, I'm convinced, AQM cannot improve a lot. But at least, = in > these cases, it should not make things worse. >=20 > Of course the greedy TCP case can be overlaid by others (unresponsive > flows, application paced flows, a renewal process of short lived flows, > synchronized start of flows etc.etc.), and we should take this into accou= nt. > But we should not expect a lot if those "others" dominate the scenario. >=20 > Example: An overlay of finite flows arriving as a renewal process with in= ter > arrival time T (seconds) and flow size S (bits). And a link capacity of C= . In cases > with C < T*S the link collapses with an infinite number of uncompleted fl= ows, Should be C < S/T > each of almost zero rate. In the opposite case, C > T*S the link cannot b= e Should be C > S/T > loaded permanently at 100% since the traffic offer is simply not there. T= here > is little to gain by AQM. It is only a thin range short before the collap= se. >=20 > Wolfram >=20 > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm From nobody Thu Apr 17 08:11:46 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55F761A0101 for ; Thu, 17 Apr 2014 08:11:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.573 X-Spam-Level: X-Spam-Status: No, score=-2.573 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gnNiYkiPxM7m for ; Thu, 17 Apr 2014 08:11:34 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by ietfa.amsl.com (Postfix) with ESMTP id 380D51A0145 for ; Thu, 17 Apr 2014 08:11:34 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id E78C5C94BD; Thu, 17 Apr 2014 11:11:28 -0400 (EDT) Date: Thu, 17 Apr 2014 11:11:28 -0400 From: John Leslie To: "LAUTENSCHLAEGER, Wolfram (Wolfram)" Message-ID: <20140417151128.GF66779@verdi> References: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> User-Agent: Mutt/1.4.1i Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/LBgTqJPOuGvKDldej2iwxDoyYqQ Cc: "aqm@ietf.org" Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 15:11:44 -0000 LAUTENSCHLAEGER, Wolfram (Wolfram) wrote: > > Shouldn't we more concentrate on what we expect from a good AQM? I'd love to... > The only thing we might expect from an AQM is to prevent greedy TCP > sources from drawing buffers permanently towards full state, I'm not the least bit sure that's possible. :^( However, it might be possible to separate the queue(s) of "greedy" TCP flows from other queues. There's a lot of room for discussion about how to do that... > instead of the much better almost empty state, Consider the class of real-time flows, which hardly ever want to stand in line for more than a millisecond or two. (They can generate redundancy to compensate for packet loss, or ideally have ECN-marked packets get through quickly.) > while maintaining full link utilization. That's a concern of _some_ traffic, but not real-time flows. > In all other situations, I'm convinced, AQM cannot improve a lot. "All other situations" covers a lot of ground... :^( And we may not all mean the same thing by "greedy TCP". Standard TCPs probe for additional capacity if they have anything to send. I consider that enough to call them "greedy"; but I'm sure others don't agree. > But at least, in these cases, it should not make things worse. "Make things worse" is perhaps insufficiently defined. Any delay at all "makes things worse" for real-time traffic. Failure to discover unused capacity "makes things worse" for TCP. > Of course the greedy TCP case can be overlaid by others (unresponsive > flows, application paced flows, a renewal process of short lived flows, > synchronized start of flows etc.etc.), These cover quite different situations... Unresponsive flows are a problem iff we buffer too much/many of them. Application-paced flows are a problem only if they're probing for increased capacity. Revewal flows are a problem if they guess badly at remaining capacity. > and we should take this into account. But we should not expect a lot > if those "others" dominate the scenario. I speculate that none of these would be a problem if we limited the extent which we buffer them. -- John Leslie From nobody Thu Apr 17 10:22:31 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EE95F1A014B for ; Thu, 17 Apr 2014 10:22:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 3 X-Spam-Level: *** X-Spam-Status: No, score=3 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MANGLED_EFFEX=2.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1cyfHyKhWD0W for ; Thu, 17 Apr 2014 10:22:24 -0700 (PDT) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 21AB91A0135 for ; Thu, 17 Apr 2014 10:22:23 -0700 (PDT) Received: by mail-wg0-f47.google.com with SMTP id x12so705011wgg.6 for ; Thu, 17 Apr 2014 10:22:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=TyLwUSvRk07LYyh6YqsUeY17lQWe0p4UaCc64DRgoJw=; b=BLcL/m/sy+Q1KyKcpzbjTMUNL5KIPhduThC5V/Sj23V6DBK9HAdm3f2TSbMi/1Ijtq ALlwSiFZgN9zdWvWcpWtt6M1u5tHWd57xAge1gkik5ojlT44rnJOrFZJYoYYKb66+Yzz hsfHaeQcu44BY79DOrbxwJfbC+TfpetkH3NIW+lX007uFDLJSNMP4KOr2fy3iQixi7wv HNkCd2OXIgrONg4gFrKhByuXRAnbrOVCXADAxiuohRLk7/7jGrQaJJ51a21fKGjW/3u8 xYurrGj7iBfc204Iif9CstTjBqTIfX4H73mlE2vzXnY2q2OervS0AtM/ZK/AGdav9lPE VRsA== MIME-Version: 1.0 X-Received: by 10.180.76.166 with SMTP id l6mr13249762wiw.17.1397755340037; Thu, 17 Apr 2014 10:22:20 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Thu, 17 Apr 2014 10:22:20 -0700 (PDT) Date: Thu, 17 Apr 2014 10:22:20 -0700 Message-ID: From: Dave Taht To: bloat , "aqm@ietf.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/qPSoYwo9u8Q8BDuabZL5fKJHHGE Subject: [aqm] The Effect of Network and Infrastructural Variables on SPDY's Performance X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 17:22:29 -0000 Last night's reading was quite good: http://arxiv.org/pdf/1401.6508.pdf "As RTT goes up, it becomes increasingly expensive for HTTPS to establish separate connections for each resource. Each HTTPS connection costs one round trip on TCP handshaking and a further two on negotiating SSL setup. SPDY does this only once (per server) and hence reduces such large waste by multiplexing streams over a single connection. " ... "the separation between RTT and bandwidth is not particularly distinct. This is because HTTPS tends to operate in a somewhat network-unfriendly manner, creating queueing delays where bandwidth is low. The bursty use of HTTPS' parallel connections cre- ates congestion at the gateway queues, causing upto 3% PLR and inflating RTT by upto 570%. In contrast, SPDY causes negligible packet loss at the gateway. The network friendly behaviour of SPDY is particularly interesting as Google has recently argued for the use of a larger IW for TCP [7]. The aim of this is to reduce round trips and speed up delivery | an idea which has been criticised for potentially causing congestion. One question here is whether or not this is a strategy that is speci cally designed to oper- ate in conjunction with SPDY. To explore this, we run further tests using and bandwidth xed at 1Mbps (all other parameters as above). For HTTPS, it appears that the critics are right: RTT and loss increase greatly with larger IWs. In contrast, SPDY achieves much higher gains when increasing the IW without these negative side e=0Bffects. " and then they inject packet loss: we inspect the impact of packet loss on SPDY's performance. We fix RTT at 150ms" Sigh, the rest of the paper is pretty good, but they should have looked at packet loss at 10-30ms at least. " and BW at 1Mbps, varying packet loss using the Linux kernel rewall with a stochastic proportional packet processing rule between 0 and 3%. . Figure 6 presents the results. Immediately, we see that SPDY is far more adversely a=0Bffected by packet loss than HTTPS is. This has been anticipated in other work [29] but never before tested. It is also contrary to what has been reported in the SPDY white paper [2], which states that SPDY is better able to deal with loss. The authors suggest because SPDY sends fewer packets, the negative e=0Bect of TCP backo=0B is mitigated. We nd that SPDY does, indeed, send fewer packets (up to 49% less due to TCP connection reuse). However, SPDY's multiplexed connections persist far longer compared to HTTPS. " --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu Apr 17 10:49:25 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0A151A0056 for ; Thu, 17 Apr 2014 10:49:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B9argkP2Ntxw for ; Thu, 17 Apr 2014 10:49:19 -0700 (PDT) Received: from mail-we0-x22a.google.com (mail-we0-x22a.google.com [IPv6:2a00:1450:400c:c03::22a]) by ietfa.amsl.com (Postfix) with ESMTP id 28D631A018D for ; Thu, 17 Apr 2014 10:49:17 -0700 (PDT) Received: by mail-we0-f170.google.com with SMTP id w61so749430wes.29 for ; Thu, 17 Apr 2014 10:49:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=NZ2cPcomXAuU7UhmBaMkN9zq/WBRGaGIaz2PBaISZYc=; b=mDTHmAeC5VlRrYQNqGxZyR4kwuK1K2PpdAF8OmoyXtwZK6ByuGz5BOPlxY0hV2BYue pC4VgFbXcHATrG/FUbwfeYWKsIdz4kLuAZIT4iATDqSLH7MRHvQE4G/mpNuzrSSex2ng VMFhirvrBpSNU/6yX/n7ltwMVWqOkGmStYuOAYuVrIBrmKetcPvepEwBbV0FVBLNbGvl RXHelT/FR9/GqG1sGKt1baS7SpmkvcjxvzNvGLYQomJalZqfTDW9VksyeTxpm3ETi+nA U7S/+UnXHVpykN1FUrmjMDlT4vOuRf0J5F0o5ddOEWKUoy4GfxZsbNKVyxEe6hRDv7Xd kOBQ== MIME-Version: 1.0 X-Received: by 10.194.9.8 with SMTP id v8mr3168291wja.53.1397756953071; Thu, 17 Apr 2014 10:49:13 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Thu, 17 Apr 2014 10:49:13 -0700 (PDT) Date: Thu, 17 Apr 2014 10:49:13 -0700 Message-ID: From: Dave Taht To: "aqm@ietf.org" , bloat , "cerowrt-devel@lists.bufferbloat.net" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/yJ83T2FWP82MTPsklfKRCQXrgXY Subject: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 17:49:24 -0000 Getting a grip on real web page load time behavior in an age of sharded websites, dozens of dns lookups, javascript, and fairly random behavior in ad service= s and cdns against how a modern browsers behaves is very, very hard. it turns out if you run google-chrome --enable-benchmarking --enable-net-benchmarking (Mac users have to embed these options in their startup script - see http://www.chromium.org/developers/how-tos/run-chromium-with-flags ) enable developer options and install and run the chrome web page benchmarke= r, ( https://chrome.google.com/webstore/detail/page-benchmarker/channimfdomahe= kjcahlbpccbgaopjll?hl=3Den ) that it works (at least for me, on a brief test of the latest chrome, on li= nux. Can someone try windows and mac?) You can then feed in a list of urls to test against, and post process the resulting .csv file to your hearts content. We used to use this benchmark a lot while trying to characterise typical web behaviors under aqm and packet scheduling systems under load. Running it simultaneously with a rrul test or one of the simpler tcp upload or down= load tests in the rrul suite was often quite interesting. It turned out the doc has been wrong a while as to the name of the second command lnie option. I was gearing up mentally for having to look at the source.... http://code.google.com/p/chromium/issues/detail?id=3D338705 /me happy --=20 Dave T=C3=A4ht Heartbleed POC on wifi campus networks with EAP auth: http://www.eduroam.edu.au/advisory.html From nobody Thu Apr 17 12:02:01 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8C211A0141 for ; Thu, 17 Apr 2014 12:01:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.35 X-Spam-Level: X-Spam-Status: No, score=-1.35 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I8Hr6Hr2uPQl for ; Thu, 17 Apr 2014 12:01:51 -0700 (PDT) Received: from mail-ve0-x229.google.com (mail-ve0-x229.google.com [IPv6:2607:f8b0:400c:c01::229]) by ietfa.amsl.com (Postfix) with ESMTP id F29DD1A0193 for ; Thu, 17 Apr 2014 12:01:50 -0700 (PDT) Received: by mail-ve0-f169.google.com with SMTP id pa12so1016030veb.0 for ; Thu, 17 Apr 2014 12:01:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hpOAtw6Lf/Xe/Nw61+NHIHmUNyZi4n0Xlh7MI76J/vU=; b=XN0ZzAsUWp2JBOuKBvwWDAdRRCkJ4j0OddABvibJapwB3QVXEw+LP0IJuFIgUbj3IZ 6vxOLH0xa5dyphEkBYMuUCw9GIHEx3KzzQKsOBW7JzwkAYyZwZixvTg60ixr99mTykLq Cw9Y9WdUbzuK5lV3JgxBFnLNVrRjqXd5nhBDweLaLooB+N3AysROs5XlcoCBmtDdJI7v z9f2VhvnXjmWg47uJEzE0xM1Ia4y2M6CpYuReEB3knTog60QlJJus0WK6Hicc1YxxvYj lP1IkuYuDlL1GqEDU99TxUzSnP5BvSIyfQoNfK+XIQAyTmb/cWBhOfih8arBFUkowvCX 20Gw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hpOAtw6Lf/Xe/Nw61+NHIHmUNyZi4n0Xlh7MI76J/vU=; b=M5E0Mdsc+rgmiO3+67gtJi0RTKJ5Lp+zt1F+VF5UQSN8FZRhvznRDKIFj5Jr9in33Z s2/f9mhNqH/aFyvKQE3nfivd8pXdqmE1kI4QlSfDwlkhLfEQCUOk2HW4wsngTgzX5god ZM+AsB30brxUfI29U9rJVJ7sV6v8Gl7A5jvG4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=hpOAtw6Lf/Xe/Nw61+NHIHmUNyZi4n0Xlh7MI76J/vU=; b=Vxj6qH3BaznEdHMvybxq37QSa+HXn8c/0mgEFZ3L1forWlJ4HewE9hk8FGms7pHUDa Z7Up98jDVcw7D9vBb6rIbV52tOhwcVBgx2gxBWv9D8ANgx0gSf2JmW8obV+iktASHByl tbYo6Ady3xNAQl1Vjy63KAhfVT+YrewUqO0soVlY9ncJBfyWEL+cCLhA9zAxkyacpNgr JZUGqNQBIIjwcuSnLC+LsWAfrB36DF8Z0wMVLzPEUTFcN70tJkQyJlfGFjUQUrbtRS5p KH/QFBcdf3sWpdyDYMQDLvZdryRL9kXWZwpVjFSThLDeYlOr5FT3xLfCN/msjI2807gq Ll0A== X-Gm-Message-State: ALoCoQnUWh3E2d1mS3oOAT8073g3/O4t2Wsb/1t239fgWDxdWv4uO9dqBVH/bMz6Bqo5CtgcEzLvX/jqNQnXlgI8AnFt/BBSmwcVTZNes7+OcYWB5yB5tEWbYqkmxA+WDzJYy4zTDF5GYJUsGZ3XmUblUM9XQ9uIhuqP4RBOGlbRoYCZnUr0Tf45WzbZqy34jZ7z1tR+1mEj MIME-Version: 1.0 X-Received: by 10.221.63.1 with SMTP id xc1mr1529112vcb.35.1397761307070; Thu, 17 Apr 2014 12:01:47 -0700 (PDT) Sender: willchan@google.com Received: by 10.52.38.101 with HTTP; Thu, 17 Apr 2014 12:01:46 -0700 (PDT) In-Reply-To: References: Date: Thu, 17 Apr 2014 12:01:46 -0700 X-Google-Sender-Auth: rXpkrkO22H4rGzSjw1F59Jpg2fQ Message-ID: From: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= To: Dave Taht Content-Type: multipart/alternative; boundary=001a1133499e3075d304f741aaec Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/uQ3jpyh-JC8bkaCHndLTI49vr5Q Cc: "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 19:01:57 -0000 --001a1133499e3075d304f741aaec Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Speaking as the primary Chromium developer in charge of this relevant code, I would like to caution putting too much trust in the numbers generated. Any statistical claims about the numbers are probably unreasonable to make. Reasons: * We don't actively maintain this code. It's behind the command line flags. They are broken. The fact that it still results in numbers on the benchmark extension is an example of where unmaintained code doesn't have the UI disabled, even though the internal workings of the code fail to guarantee correct operation. We haven't disabled it because, well, it's unmaintained. * We don't make sure to flush all the network state in between runs, so if you're using that option, don't trust it to work. * If you have an advanced Chromium setup, this definitely does not work. I advise using the benchmark extension only with a separate Chromium profile for testing purposes. Our flushing of sockets, caches, etc does not actually work correctly when you use the Chromium multiprofile feature and also fails to flush lots of our other network caches. * No one on Chromium really believes the time to paint numbers that we output :) It's complicated. Our graphics stack is complicated. The time from when Blink thinks it painted to when the GPU actually blits to the screen cannot currently be corroborated with any high degree of accuracy from within our code. * It has not been maintained since 2010. It is quite likely there are many other subtle inaccuracies here. In short, while you can expect it to give you a very high level understanding of performance issues, I advise against placing non-trivial confidence in the accuracy of the numbers generated by the benchmark extension. The fact that numbers are produced by the extension should not be treated as evidence that the extension actually functions correctly. Cheers. On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht wrote: > Getting a grip on real web page load time behavior in an age of > sharded websites, > dozens of dns lookups, javascript, and fairly random behavior in ad > services > and cdns against how a modern browsers behaves is very, very hard. > > it turns out if you run > > google-chrome --enable-benchmarking --enable-net-benchmarking > > (Mac users have to embed these options in their startup script - see > http://www.chromium.org/developers/how-tos/run-chromium-with-flags ) > > enable developer options and install and run the chrome web page > benchmarker, > ( > https://chrome.google.com/webstore/detail/page-benchmarker/channimfdomahe= kjcahlbpccbgaopjll?hl=3Den > ) > > that it works (at least for me, on a brief test of the latest chrome, on > linux. > Can someone try windows and mac?) > > You can then feed in a list of urls to test against, and post process > the resulting .csv file to your hearts content. We used to use this > benchmark a lot while trying to characterise typical web behaviors > under aqm and packet scheduling systems under load. Running > it simultaneously with a rrul test or one of the simpler tcp upload or > download > tests in the rrul suite was often quite interesting. > > It turned out the doc has been wrong a while as to the name of the second > command lnie option. I was gearing up mentally for having to look at > the source.... > > http://code.google.com/p/chromium/issues/detail?id=3D338705 > > /me happy > > -- > Dave T=E4ht > > Heartbleed POC on wifi campus networks with EAP auth: > http://www.eduroam.edu.au/advisory.html > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > --001a1133499e3075d304f741aaec Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Speaking as the primary Chromium developer in charge of th= is relevant code, I would like to caution putting too much trust in the num= bers generated. Any statistical claims about the numbers are probably unrea= sonable to make.

Reasons:
* We don't actively maintain this cod= e. It's behind the command line flags. They are broken. The fact that i= t still results in numbers on the benchmark extension is an example of wher= e unmaintained code doesn't have the UI disabled, even though the inter= nal workings of the code fail to guarantee correct operation. We haven'= t disabled it because, well, it's unmaintained.
* We don't make sure to flush all the network state in between run= s, so if you're using that option, don't trust it to work.
* If you have an advanced Chromium setup, this definitely does not work. = I advise using the benchmark extension only with a separate Chromium profil= e for testing purposes. Our flushing of sockets, caches, etc does not actua= lly work correctly when you use the Chromium multiprofile feature and also = fails to flush lots of our other network caches.
* No one on Chromium really believes the time to paint numbers that we= output :) It's complicated. Our graphics stack is complicated. The tim= e from when Blink thinks it painted to when the GPU actually blits to the s= creen cannot currently be corroborated with any high degree of accuracy fro= m within our code.
* It has not been maintained since 2010. It is quite likely there are = many other subtle inaccuracies here.

In short, whi= le you can expect it to give you a very high level understanding of perform= ance issues, I advise against placing non-trivial confidence in the accurac= y of the numbers generated by the benchmark extension. The fact that number= s are produced by the extension should not be treated as evidence that the = extension actually functions correctly.

Cheers.


<= div class=3D"gmail_quote">On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht <d= ave.taht@gmail.com> wrote:
Getting a grip on real web page load time be= havior in an age of
sharded websites,
dozens of dns lookups, javascript, and fairly random behavior in ad service= s
and cdns against how a modern browsers behaves is very, very hard.

it turns out if you run

google-chrome --enable-benchmarking --enable-net-benchmarking

(Mac users have to embed these options in their startup script - see
=A0http://www.chromium.org/developers/how-tos/run-chr= omium-with-flags )

enable developer options and install and run the chrome web page benchmarke= r,
( https://chrome.goo= gle.com/webstore/detail/page-benchmarker/channimfdomahekjcahlbpccbgaopjll?h= l=3Den
)

that it works (at least for me, on a brief test of the latest chrome, on li= nux.
Can someone try windows and mac?)

You can then feed in a list of urls to test against, and post process
the resulting .csv file to your hearts content. We used to use this
benchmark a lot while trying to characterise typical web behaviors
under aqm and packet scheduling systems under load. Running
it simultaneously with a rrul test or one of the simpler tcp upload or down= load
tests in the rrul suite was often quite interesting.

It turned out the doc has been wrong a while as to the name of the second command lnie option. I was gearing up mentally for having to look at
the source....

http://code.google.com/p/chromium/issues/detail?id=3D338705<= /a>

/me happy

--
Dave T=E4ht

Heartbleed POC on wifi campus networks with EAP auth:
http:= //www.eduroam.edu.au/advisory.html

_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm

--001a1133499e3075d304f741aaec-- From nobody Thu Apr 17 14:07:39 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B7CA51A0135 for ; Thu, 17 Apr 2014 14:07:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.199 X-Spam-Level: X-Spam-Status: No, score=0.199 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id djPHpGzdCRyD for ; Thu, 17 Apr 2014 14:07:32 -0700 (PDT) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) by ietfa.amsl.com (Postfix) with ESMTP id A69331A0054 for ; Thu, 17 Apr 2014 14:07:31 -0700 (PDT) Received: by mail-wi0-f176.google.com with SMTP id r20so34727wiv.15 for ; Thu, 17 Apr 2014 14:07:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Iv0CKcETV/OjBIZ/kywyTwlmpKsBK4KPihuTDsdi/4M=; b=ntfc3kPWGQHNviSOmPhIAiaMbWlI8T3TEn9nJUXd9D550lrbL3oNHnoqcafvXFKTrW PepaMSi+Ttbi2WpIANUhT7Xn7LRROvq4Qo4PH05tLqdFxsW+CDww64LkOWGZfBbwemlm G/spyV9b7XZ6HRr7E+4SGf3HBE6aUo+0K5gNkkdbjB683TOT7k2lsNa8aAVZGGH/31fc PtLcVVEaImiaMl9ZiieICYR4jMLRslt8hNdfG4pYB3CaLneIEc1iBCxvvuQ3bCLldDum TDWaFQn56vmTALvAj7lyHXOr9XcvbEHk5cUyYXNevtvx69Dz+mPBOHlwZKSfzEJHvxUu 2xEA== MIME-Version: 1.0 X-Received: by 10.180.78.225 with SMTP id e1mr25792619wix.17.1397768847422; Thu, 17 Apr 2014 14:07:27 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Thu, 17 Apr 2014 14:07:27 -0700 (PDT) In-Reply-To: References: Date: Thu, 17 Apr 2014 14:07:27 -0700 Message-ID: From: Dave Taht To: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/v7ZTZBzGZfjATEHlniNscK4QttE Cc: "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 21:07:36 -0000 On Thu, Apr 17, 2014 at 12:01 PM, William Chan (=E9=99=88=E6=99=BA=E6=98=8C= ) wrote: > Speaking as the primary Chromium developer in charge of this relevant cod= e, > I would like to caution putting too much trust in the numbers generated. = Any > statistical claims about the numbers are probably unreasonable to make. Sigh. Other benchmarks such as the apache ("ab") benchmark are primarily designed as stress testers for web servers, not as realistic traffic. Modern web traffic has such a high level of dynamicism in it, that static web page loads along any distribution, seem insufficient, passive analysis of aggregated traffic "feels" incorrect relative to the sorts of home and small business traffic I've seen, and so on. Famous papers, such as this one: http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-leland.pdf Seem possibly irrelevant to draw conclusions from given the kind of data they analysed and proceeding from an incorrect model or gut feel for how the network behaves today seems to be foolish. Even the most basic of tools, such as httping, had three basic bugs that I found in a few minutes of trying to come up with some basic behaviors yesterday: https://lists.bufferbloat.net/pipermail/bloat/2014-April/001890.html Those are going to be a lot easier to fix than diving into the chromium codebase! There are very few tools worth trusting, and I am always dubious of papers that publish results with unavailable tools and data. The only tools I have any faith in for network analysis are netperf, netperf-wrapper= , tcpdump and xplot.org, and to a large extent wireshark. Toke and I have been tearing apart d-itg and I hope to one day be able to add that to my trustable list... but better tools are needed! Tools that I don't have a lot of faith in include that, iperf, anything wri= tten in java or other high level languages, speedtest.net, and things like shaperprobe. Have very little faith in ns2, slightly more in ns3, and I've been meaning to look over the mininet and other simulators whenever I got some spare time; the mininet results stanford gets seem pretty reasonable and I adore their reproducing results effort. Haven't explored ndt, keep meaning to... > Reasons: > * We don't actively maintain this code. It's behind the command line flag= s. > They are broken. The fact that it still results in numbers on the benchma= rk > extension is an example of where unmaintained code doesn't have the UI > disabled, even though the internal workings of the code fail to guarantee > correct operation. We haven't disabled it because, well, it's unmaintaine= d. As I mentioned I was gearing up for a hacking run... The vast majority of results I look at are actually obtained via looking at packet captures. I mostly use benchmarks as abstractions to see if they make some sense relative to the captures and tend to compare different benchmarks against each other. I realize others don't go into that level of detail, so you have given fair warning! In our case we used the web page benchmarker as a means to try and rapidly acquire some typical distributions of get and tcp stream requests from things like the alexa top 1000, and as a way to A/B different aqm/packet scheduling setups. ... but the only easily publishable results were from the benchmark itself, and we (reluctantly) only published one graph from all the work that went into it 2+ years back and used it as a test driver for the famous ietf video, comparing two identical boxes running it at the same time under different network conditions: https://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos#IETF-demo-si= de-by-side-of-a-normal-cable-modem-vs-fq_codel from what I fiddled with today, it is at least still useful for that? moving on... The web model in the cablelabs work doesn't look much like my captures, in addition to not modeling dns at all, and using a smaller IW than google it looks like this: >> Model single user web page download as follows: >> - Web page modeled as single HTML page + 100 objects spread evenly >> across 4 servers. Web object sizes are currently fixed at 25 kB each, >> whereas the initial HTML page is 100 kB. Appendix A provides an >> alternative page model that may be explored in future work. Where what I see is a huge number of stuff that fits into a single iw10 slow start episode and some level of pipelining on larger stuff, so th= at a large number of object sizes of less than 7k with a lightly tailed distribu= tion outside of that makes more sense. (I'm not staring at appendix A right now, I'm under the impression it was better) I certainly would like more suggestions for models and types of web traffic, as well as simulation of https + pfs traffic, spdy, quic, etc.... >> - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms). Server RTTs from my own web history tend to be lower than 50ms. >> - Initial HTTP GET to retrieve a moderately sized object (100 kB HTML >> page) from server 1. An initial GET to google fits into iw10 - it's about 7k. >> - Once initial HTTP GET completes, initiate 24 simultaneous HTTP GETs >> (via separate TCP connections), 6 connections each to 4 different >> server nodes I usually don't see more than 15. and certainly not 25k sized objects. > - Once each individual HTTP GET completes, initiate a subsequent GET > to the same server, until 25 objects have been retrieved from each > server. > * We don't make sure to flush all the network state in between runs, so i= f > you're using that option, don't trust it to work. The typical scenario we used was a run against dozens or hundreds of urls, capturing traffic, while varying network conditions. Regarded the first run as the most interesting. Can exit the browser and restart after a run like that. At moment, merely plan to use the tool primarily to survey various web sites and load times while doing packet captures. Hope was to get valid data from the network portion of the load, tho... > * If you have an advanced Chromium setup, this definitely does not work. = I > advise using the benchmark extension only with a separate Chromium profil= e > for testing purposes. Our flushing of sockets, caches, etc does not actua= lly > work correctly when you use the Chromium multiprofile feature and also fa= ils > to flush lots of our other network caches. noted. > * No one on Chromium really believes the time to paint numbers that we > output :) It's complicated. Our graphics stack is complicated. The time f= rom I actually care only about time-to-full layout as that's a core network effect... > when Blink thinks it painted to when the GPU actually blits to the screen > cannot currently be corroborated with any high degree of accuracy from > within our code. > * It has not been maintained since 2010. It is quite likely there are man= y > other subtle inaccuracies here. Grok. > In short, while you can expect it to give you a very high level > understanding of performance issues, I advise against placing non-trivial > confidence in the accuracy of the numbers generated by the benchmark > extension. The fact that numbers are produced by the extension should not= be > treated as evidence that the extension actually functions correctly. OK, noted. Still delighted to be able to have a simple load generator that exercises the browsers and generates some results, however dubious. > > Cheers. > > > On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht wrote: >> >> Getting a grip on real web page load time behavior in an age of >> sharded websites, >> dozens of dns lookups, javascript, and fairly random behavior in ad >> services >> and cdns against how a modern browsers behaves is very, very hard. >> >> it turns out if you run >> >> google-chrome --enable-benchmarking --enable-net-benchmarking >> >> (Mac users have to embed these options in their startup script - see >> http://www.chromium.org/developers/how-tos/run-chromium-with-flags ) >> >> enable developer options and install and run the chrome web page >> benchmarker, >> ( >> https://chrome.google.com/webstore/detail/page-benchmarker/channimfdomah= ekjcahlbpccbgaopjll?hl=3Den >> ) >> >> that it works (at least for me, on a brief test of the latest chrome, on >> linux. >> Can someone try windows and mac?) >> >> You can then feed in a list of urls to test against, and post process >> the resulting .csv file to your hearts content. We used to use this >> benchmark a lot while trying to characterise typical web behaviors >> under aqm and packet scheduling systems under load. Running >> it simultaneously with a rrul test or one of the simpler tcp upload or >> download >> tests in the rrul suite was often quite interesting. >> >> It turned out the doc has been wrong a while as to the name of the secon= d >> command lnie option. I was gearing up mentally for having to look at >> the source.... >> >> http://code.google.com/p/chromium/issues/detail?id=3D338705 >> >> /me happy >> >> -- >> Dave T=C3=A4ht >> >> Heartbleed POC on wifi campus networks with EAP auth: >> http://www.eduroam.edu.au/advisory.html >> >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm > > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu Apr 17 14:58:35 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 12FA71A01F5 for ; Thu, 17 Apr 2014 14:58:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QKkcFq3TBDrz for ; Thu, 17 Apr 2014 14:58:28 -0700 (PDT) Received: from mail-we0-x22e.google.com (mail-we0-x22e.google.com [IPv6:2a00:1450:400c:c03::22e]) by ietfa.amsl.com (Postfix) with ESMTP id 5B14D1A01A5 for ; Thu, 17 Apr 2014 14:58:28 -0700 (PDT) Received: by mail-we0-f174.google.com with SMTP id t60so965974wes.5 for ; Thu, 17 Apr 2014 14:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=0GA9FbWCVb5l7cSP0hwcBTcDK1YkeN/1lzORQRUiXFE=; b=fbUpJqocB2n2miAESt7JbWwxLfVRayljAqZsQHgWOc4RjCgj2BCKeymkSBDBahJ0eC KDT7T+YfrQ/asKDo6k5Bd0IURA5XFseGxiwSFFRSXpaDoVWJsyZlhPSjeKxx2KI9s5m8 K0RuTAnSFIjKDVSTX60XNO/J12CHJiLPy6cwkL1w3Rmp11fdRlgTpgVTwxoLH/2PYVVu DnEkNZm2qTN3kjpSehRop8g3lsYoyORqSMIR7vxSdWQE8T+O+s/gCI3oXBLrn4hkohKM FEHzAbKnQ1SJ5aPYSa8El+4DD3+wgCCCzgY+uL8TllEEk9Lk7jMiMeR36VLF+T+HMBmn JtOw== MIME-Version: 1.0 X-Received: by 10.194.175.70 with SMTP id by6mr13657596wjc.3.1397771904185; Thu, 17 Apr 2014 14:58:24 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Thu, 17 Apr 2014 14:58:24 -0700 (PDT) In-Reply-To: <533AAD07.103@mti-systems.com> References: <533AAD07.103@mti-systems.com> Date: Thu, 17 Apr 2014 14:58:24 -0700 Message-ID: From: Dave Taht To: Wesley Eddy Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/K5alJfMbWjFh7Ez_UgCehldmUmQ Cc: "aqm@ietf.org" Subject: Re: [aqm] publishing algorithms X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 21:58:34 -0000 On Tue, Apr 1, 2014 at 5:11 AM, Wesley Eddy wrote: > Hello AQMers. As chairs, Richard and I had been planning to let > the evaluation guidelines converge and then use those to guide > adoption of algorithm documents. > > However, we now think there may be value in not waiting so long, > and getting some algorithm documents moving along more quickly. I saw this came out on april 1, and have a tendency to ignore stuff that comes out that day. :) > We hope you can provide some feedback on the plan below: > > 1) Starting soon, we may look to adopt a small number of algorithm > drafts for Experimental, with the goal that by doing so, it will > increase the number of eyeballs and independent reviews of them, and > enhance the quality, since people may be implementing to the drafts > in order to test using the evaluation guidelines. Each algorithm > *must* clearly identify which types of use cases / scenarios it is > targeted for. We submitted the codel and fq_codel drafts for consideration prior to ietf 89, and have got no feedback from anyone as yet. The first went into tsvwg https://datatracker.ietf.org/doc/draft-nichols-tsvwg-codel/?include_text=3D= 1 and the second is seemingly homeless as yet: https://tools.ietf.org/html/draft-hoeiland-joergensen-aqm-fq-codel-00 As they were being developed independently, they have a few nits in relating to each other that need to be resolved, presently. If this is a new requirement, and it seems like it is, then those will need to be revised more fully, although each has caveats and use cases described already late in the document. (sections 6 and 9 in the first, section 10 in the second) Are you proposing an explicit use cases and scenarios section? > 2) Adoption of an algorithm spec as a working group draft will require > working group consensus that the algorithm looks attractive to > experiment with for the stated scenarios, and multiple parties will > plan to be looking at it, testing, analyzing, providing feedback, > etc. +1 I note that I have another draft in progress that I hope to complete before ietf-90 describing multiple deployed aqm + packet scheduling + classificati= on systems in the hope to get the overall conversation on track. http://snapon.lab.bufferbloat.net/~d/draft-taht-home-gateway-best-practices= -00.html and I'd like to talk to that at ietf 90 if I'm not too broke to go. Otherwi= se hawaii. > > 3) The evaluation guidelines / scenarios drafts being worked on > separately will guide the later selection of one or more Experimental > algorithms to become Proposed Standards with applicability > statements for the scenarios they have been evaluated in. At this point I must confess to being deeply dissatisfied with the progress in the AQM guidelines document. I don't know what to do about it, I've tried to talk about the issues being totally missed in things like thi= s: http://www.bufferbloat.net/projects/cerowrt/wiki/Smart_Queue_Management and here on the list... and at the moment I'd prefer more folk have better models and understanding of actual traffic types on various media types before even trying to write anything down as to what valid tests are. I think I have finally come up for what I'd like this group to be doing, which is at least partially out of scope for the charter, but the acronym i= s nice: "CQM" - "Comprehensive Queue Management", and a goal is to try and design something that leverages the work other groups such as rmcat are doing. > We're interested to know if the working group thinks this sounds like > a good idea, bad idea, or any other thoughts. By all means, let's get folk fiddling with the working algorithms and code that exists today and is distributed now with every Linux box. Perhaps some more BSD support will show up soon, or hardware support in some other gear other than in what I describe above. I'm all in favor of new algorithms too, if anybody's got some... > > -- > Wes Eddy > MTI Systems > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Thu Apr 17 15:29:55 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1E7AB1A012D for ; Thu, 17 Apr 2014 15:29:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.35 X-Spam-Level: X-Spam-Status: No, score=-1.35 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5BQ9Fu6XesvZ for ; Thu, 17 Apr 2014 15:29:49 -0700 (PDT) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) by ietfa.amsl.com (Postfix) with ESMTP id 866AC1A0078 for ; Thu, 17 Apr 2014 15:29:49 -0700 (PDT) Received: by mail-ve0-f173.google.com with SMTP id oy12so1450017veb.18 for ; Thu, 17 Apr 2014 15:29:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5mMALEEhhv6zNdurybtJJZY0++zMSkmGBcOgFOyY0IM=; b=DW1JgcbzBWEK1Gsjfemckp7M7+IKJR1D7HFu/OIqqv2ww5jJIlu4fhpvTOh/u8Ogfc jGyaF+jk8pYhEQIFXzTR6HNkn6vQWOCq1HThCaq36kR2afsv6NriPKTzWAxVNWFYhSME o3iyOfFnZherO+m2OrGD7yUojgqgzS2lQfwAqlDYdFKMEklHFcIYjCDuXnQDeYlMWwv+ AN63+yPEdkkiHt8EMb1w3VackLZUIqxLFKqQdS3NLE8KIoaqFFAIygbzx0CLCQEFc4IX qrIp7Muegl0BnFj+A2719VEDwRG9GMNWcEkmYORY70VEjsL5Po3H+lR5n9PCFo4ue6O9 fxog== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5mMALEEhhv6zNdurybtJJZY0++zMSkmGBcOgFOyY0IM=; b=Pjsv/KGPxhh7zHKreoc0lTqQSoHLVvDXLIn/58F1O8bra+QLJ4XOryjD6jMKrl+ker JJHbR2EWG5t8+d9wofl3EC/20eVhk9qsgYYkS2FBr8/oaMEmthi3MDCpZNyIwepwOxPF 3xWQl4HO2VXZo995MwYwt7VquJTha2COnOPXM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=5mMALEEhhv6zNdurybtJJZY0++zMSkmGBcOgFOyY0IM=; b=QBvTOmZcXii5NtM4qhqr1JGBWchR/ufzyxhoW3gVF7k+chijXzt6KUEiQVe/k3xPlX bFnCzM4KLygXd3Q0Rko7jCVvK1pgurBrVu7K3vX3lGz/e9woECKdIwO6Qh5o47EGuL+B Mqhz6BcYbsitheTeiCdVtrNeccMI33rM4c93i/fL4pU8lpjqGPQLPbZc6Z5sSa53Z8zU pceZo61x88V2J/gvleMT5/lEl+1Ro9hllOp8yi7D84jJvARmRngP66Or71+mFCzB36Hr +3DfD3IKEulSDbV/EXL8AcYvLh4o2iSfz50bsQcUahSuLlZduv9+3SdnhD6GmJgEcICF 7GkA== X-Gm-Message-State: ALoCoQmIF3T2WIHHYThCJi+/Cleve7RFxWg5BfQWnPusuwJittJgZ7APy6tjrVVEX2uZGd4N/NRN9U30qe0X5xBNEmExKYLNUrraftjb4AujnFL6oAhhaQskkGfjl0ltRfWao5NIC0NpLDwWuMuG4GwwwWbjwOLpX3KhwHFdClJ7JlCn3n1TZPcviOWYiyaUzbnEVaslBj17 MIME-Version: 1.0 X-Received: by 10.58.38.166 with SMTP id h6mr9580723vek.22.1397773785447; Thu, 17 Apr 2014 15:29:45 -0700 (PDT) Sender: willchan@google.com Received: by 10.52.38.101 with HTTP; Thu, 17 Apr 2014 15:29:45 -0700 (PDT) In-Reply-To: References: Date: Thu, 17 Apr 2014 15:29:45 -0700 X-Google-Sender-Auth: -MZp4z2R7TkK-zxm5YVBb1Jf41g Message-ID: From: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= To: Dave Taht Content-Type: multipart/alternative; boundary=089e013a0d40f5709304f7449169 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/s0d3S38iNtLiarl7vGIhwi1su9E Cc: "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 22:29:54 -0000 --089e013a0d40f5709304f7449169 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I understand the desire for this code to work, and if I had more bandwidth, I would definitely fix it sooner. For now, all I can say is, be aware of the potential issues with the code and use at your own risk. Sorry that we don't have more bandwidth now to get this working sooner. On Thu, Apr 17, 2014 at 2:07 PM, Dave Taht wrote: > On Thu, Apr 17, 2014 at 12:01 PM, William Chan (=E9=99=88=E6=99=BA=E6=98= =8C) > wrote: > > Speaking as the primary Chromium developer in charge of this relevant > code, > > I would like to caution putting too much trust in the numbers generated= . > Any > > statistical claims about the numbers are probably unreasonable to make. > > Sigh. Other benchmarks such as the apache ("ab") benchmark > are primarily designed as stress testers for web servers, not as realisti= c > traffic. Modern web traffic has such a high level of dynamicism in it, > that static web page loads along any distribution, seem insufficient, > passive analysis of aggregated traffic "feels" incorrect relative to the > sorts of home and small business traffic I've seen, and so on. > > Famous papers, such as this one: > > http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-leland.pdf > > Seem possibly irrelevant to draw conclusions from given the kind > of data they analysed and proceeding from an incorrect model or > gut feel for how the network behaves today seems to be foolish. > > Even the most basic of tools, such as httping, had three basic bugs > that I found in a few minutes of trying to come up with some basic > behaviors yesterday: > > https://lists.bufferbloat.net/pipermail/bloat/2014-April/001890.html > > Those are going to be a lot easier to fix than diving into the chromium > codebase! > > There are very few tools worth trusting, and I am always dubious > of papers that publish results with unavailable tools and data. The only > tools I have any faith in for network analysis are netperf, > netperf-wrapper, > tcpdump and xplot.org, and to a large extent wireshark. Toke and I have > been tearing apart d-itg and I hope to one day be able to add that to > my trustable list... but better tools are needed! > > Tools that I don't have a lot of faith in include that, iperf, anything > written > in java or other high level languages, speedtest.net, and things like > shaperprobe. > > Have very little faith in ns2, slightly more in ns3, and I've been meanin= g > to look over the mininet and other simulators whenever I got some spare > time; the mininet results stanford gets seem pretty reasonable and I > adore their reproducing results effort. Haven't explored ndt, keep meanin= g > to... > > > Reasons: > > * We don't actively maintain this code. It's behind the command line > flags. > > They are broken. The fact that it still results in numbers on the > benchmark > > extension is an example of where unmaintained code doesn't have the UI > > disabled, even though the internal workings of the code fail to guarant= ee > > correct operation. We haven't disabled it because, well, it's > unmaintained. > > As I mentioned I was gearing up for a hacking run... > > The vast majority of results I look at are actually obtained via > looking at packet captures. I mostly use benchmarks as abstractions > to see if they make some sense relative to the captures and tend > to compare different benchmarks against each other. > > I realize others don't go into that level of detail, so you have given > fair warning! In our case we used the web page benchmarker as > a means to try and rapidly acquire some typical distributions of > get and tcp stream requests from things like the alexa top 1000, > and as a way to A/B different aqm/packet scheduling setups. > > ... but the only easily publishable results were from the benchmark itsel= f, > and we (reluctantly) only published one graph from all the work that > went into it 2+ years back and used it as a test driver for the famous > ietf video, comparing two identical boxes running it at the same time > under different network conditions: > > > https://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos#IETF-demo-= side-by-side-of-a-normal-cable-modem-vs-fq_codel > > from what I fiddled with today, it is at least still useful for that? > > moving on... > > The web model in the cablelabs work doesn't look much like my captures, > in addition to not modeling dns at all, and using a smaller IW than googl= e > it looks like this: > > >> Model single user web page download as follows: > > >> - Web page modeled as single HTML page + 100 objects spread evenly > >> across 4 servers. Web object sizes are currently fixed at 25 kB each, > >> whereas the initial HTML page is 100 kB. Appendix A provides an > >> alternative page model that may be explored in future work. > > Where what I see is a huge number of stuff that fits into a single > iw10 slow start episode and some level of pipelining on larger stuff, so > that a > large number of object sizes of less than 7k with a lightly tailed > distribution > outside of that makes more sense. > > (I'm not staring at appendix A right now, I'm under the impression > it was better) > > I certainly would like more suggestions for models and types > of web traffic, as well as simulation of https + pfs traffic, > spdy, quic, etc.... > > >> - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms). > > Server RTTs from my own web history tend to be lower than 50ms. > > >> - Initial HTTP GET to retrieve a moderately sized object (100 kB HTML > >> page) from server 1. > > An initial GET to google fits into iw10 - it's about 7k. > > >> - Once initial HTTP GET completes, initiate 24 simultaneous HTTP GETs > >> (via separate TCP connections), 6 connections each to 4 different > >> server nodes > > I usually don't see more than 15. and certainly not 25k sized objects. > > > - Once each individual HTTP GET completes, initiate a subsequent GET > > to the same server, until 25 objects have been retrieved from each > > server. > > > > * We don't make sure to flush all the network state in between runs, so > if > > you're using that option, don't trust it to work. > > The typical scenario we used was a run against dozens or hundreds of urls= , > capturing traffic, while varying network conditions. > > Regarded the first run as the most interesting. > > Can exit the browser and restart after a run like that. > > At moment, merely plan to use the tool primarily to survey various > web sites and load times while doing packet captures. Hope was > to get valid data from the network portion of the load, tho... > > > * If you have an advanced Chromium setup, this definitely does not work= . > I > > advise using the benchmark extension only with a separate Chromium > profile > > for testing purposes. Our flushing of sockets, caches, etc does not > actually > > work correctly when you use the Chromium multiprofile feature and also > fails > > to flush lots of our other network caches. > > noted. > > > > * No one on Chromium really believes the time to paint numbers that we > > output :) It's complicated. Our graphics stack is complicated. The time > from > > I actually care only about time-to-full layout as that's a core network > effect... > > > when Blink thinks it painted to when the GPU actually blits to the scre= en > > cannot currently be corroborated with any high degree of accuracy from > > within our code. > > > * It has not been maintained since 2010. It is quite likely there are > many > > other subtle inaccuracies here. > > Grok. > > > In short, while you can expect it to give you a very high level > > understanding of performance issues, I advise against placing non-trivi= al > > confidence in the accuracy of the numbers generated by the benchmark > > extension. The fact that numbers are produced by the extension should > not be > > treated as evidence that the extension actually functions correctly. > > OK, noted. Still delighted to be able to have a simple load generator > that exercises the browsers and generates some results, however > dubious. > > > > > Cheers. > > > > > > On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht wrote= : > >> > >> Getting a grip on real web page load time behavior in an age of > >> sharded websites, > >> dozens of dns lookups, javascript, and fairly random behavior in ad > >> services > >> and cdns against how a modern browsers behaves is very, very hard. > >> > >> it turns out if you run > >> > >> google-chrome --enable-benchmarking --enable-net-benchmarking > >> > >> (Mac users have to embed these options in their startup script - see > >> http://www.chromium.org/developers/how-tos/run-chromium-with-flags ) > >> > >> enable developer options and install and run the chrome web page > >> benchmarker, > >> ( > >> > https://chrome.google.com/webstore/detail/page-benchmarker/channimfdomahe= kjcahlbpccbgaopjll?hl=3Den > >> ) > >> > >> that it works (at least for me, on a brief test of the latest chrome, = on > >> linux. > >> Can someone try windows and mac?) > >> > >> You can then feed in a list of urls to test against, and post process > >> the resulting .csv file to your hearts content. We used to use this > >> benchmark a lot while trying to characterise typical web behaviors > >> under aqm and packet scheduling systems under load. Running > >> it simultaneously with a rrul test or one of the simpler tcp upload or > >> download > >> tests in the rrul suite was often quite interesting. > >> > >> It turned out the doc has been wrong a while as to the name of the > second > >> command lnie option. I was gearing up mentally for having to look at > >> the source.... > >> > >> http://code.google.com/p/chromium/issues/detail?id=3D338705 > >> > >> /me happy > >> > >> -- > >> Dave T=C3=A4ht > >> > >> Heartbleed POC on wifi campus networks with EAP auth: > >> http://www.eduroam.edu.au/advisory.html > >> > >> _______________________________________________ > >> aqm mailing list > >> aqm@ietf.org > >> https://www.ietf.org/mailman/listinfo/aqm > > > > > > > > -- > Dave T=C3=A4ht > > NSFW: > https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_inde= cent.article > --089e013a0d40f5709304f7449169 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I understand the desire for this code to work, and if I ha= d more bandwidth, I would definitely fix it sooner. For now, all I can say = is, be aware of the potential issues with the code and use at your own risk= . Sorry that we don't have more bandwidth now to get this working soone= r.


On Thu, Apr 1= 7, 2014 at 2:07 PM, Dave Taht <dave.taht@gmail.com> wrote:=
On Thu, Apr 17, 2014 at 12:01 PM, William Chan (=E9=99=88= =E6=99=BA=E6=98=8C)
<willchan@chromium.org> = wrote:
> Speaking as the primary Chromium developer in ch= arge of this relevant code,
> I would like to caution putting too much trust in the numbers generate= d. Any
> statistical claims about the numbers are probably unreasonable to make= .

Sigh. Other benchmarks such as the apache ("ab") benchmark<= br> are primarily designed as stress testers for web servers, not as realistic<= br> traffic. Modern web traffic has such a high level of dynamicism in it,
that static web page loads along any distribution, seem insufficient,
passive analysis of aggregated traffic "feels" incorrect relative= to the
sorts of home and small business traffic I've seen, and so on.

Famous papers, such as this one:

http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-leland.= pdf

Seem possibly irrelevant to draw conclusions from given the kind
of data they analysed and proceeding from an incorrect model or
gut feel for how the network behaves today seems to be foolish.

Even the most basic of tools, such as httping, had three basic bugs
that I found in a few minutes of trying to come up with some basic
behaviors yesterday:

https://lists.bufferbloat.net/pipermail/bloat/2014-= April/001890.html

Those are going to be a lot easier to fix than diving into the chromium
codebase!

There are very few tools worth trusting, and I am always dubious
of papers that publish results with unavailable tools and data. The only tools I have any faith in for network analysis are netperf, netperf-wrapper= ,
tcpdump and xplot.org, a= nd to a large extent wireshark. Toke and I have
been tearing apart d-itg and I hope to one day be able to add that to
my trustable list... but better tools are needed!

Tools that I don't have a lot of faith in include that, iperf, anything= written
in java or other high level languages, speedtest.net, and things like
shaperprobe.

Have very little faith in ns2, slightly more in ns3, and I've been mean= ing
to look over the mininet and other simulators whenever I got some spare
time; the mininet results stanford gets seem pretty reasonable and I
adore their reproducing results effort. Haven't explored ndt, keep mean= ing
to...

> Reasons:
> * We don't actively maintain this code. It's behind the comman= d line flags.
> They are broken. The fact that it still results in numbers on the benc= hmark
> extension is an example of where unmaintained code doesn't have th= e UI
> disabled, even though the internal workings of the code fail to guaran= tee
> correct operation. We haven't disabled it because, well, it's = unmaintained.

As I mentioned I was gearing up for a hacking run...

The vast majority of results I look at are actually obtained via
looking at packet captures. I mostly use benchmarks as abstractions
to see if they make some sense relative to the captures and tend
to compare different benchmarks against each other.

I realize others don't go into that level of detail, so you have given<= br> fair warning! In our case we used the web page benchmarker as
a means to try and rapidly acquire some typical distributions of
get and tcp stream requests from things like the alexa top 1000,
and as a way to A/B different aqm/packet scheduling setups.

... but the only easily publishable results were from the benchmark itself,=
and we (reluctantly) only published one graph from all the work that
went into it 2+ years back and used it as a test driver for the famous
ietf video, comparing two identical boxes running it at the same time
under different network conditions:

https://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos#IETF-demo-= side-by-side-of-a-normal-cable-modem-vs-fq_codel

from what I fiddled with today, it is at least still useful for that?

moving on...

The web model in the cablelabs work doesn't look much like my captures,=
in addition to not modeling dns at all, and using a smaller IW than google<= br> it looks like this:

>> Model single user web page download as follows:

>> - Web page modeled as single HTML page + 100 objects spread evenly=
>> across 4 servers. Web object sizes are currently fixed at 25 kB ea= ch,
>> whereas the initial HTML page is 100 kB. Appendix A provides an >> alternative page model that may be explored in future work.

Where what I see is a huge number of stuff that fits into a single
iw10 slow start episode and some level of pipelining on larger stuff, so th= at a
large number of object sizes of less than 7k with a lightly tailed distribu= tion
outside of that makes more sense.

(I'm not staring at appendix A right now, I'm under the impression<= br> =C2=A0it was better)

I certainly would like more suggestions for models and types
of web traffic, as well as simulation of https + pfs traffic,
spdy, quic, etc....

>> - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms).

Server RTTs from my own web history tend to be lower than 50ms.

>> - Initial HTTP GET to retrieve a moderately sized object (100 kB H= TML
>> page) from server 1.

An initial GET to google fits into iw10 - it's about 7k.

>> - Once initial HTTP GET completes, initiate 24 simultaneous HTTP G= ETs
>> (via separate TCP connections), 6 connections each to 4 different<= br> >> server nodes

I usually don't see more than 15. and certainly not 25k sized objects.<= br>
=C2=A0> - Once each individual HTTP GET completes, initiate a subsequent= GET
> to the same server, until 25 objects have been retrieved from each
> server.


> * We don't make sure to flush all the network state in between run= s, so if
> you're using that option, don't trust it to work.

The typical scenario we used was a run against dozens or hundreds of = urls,
capturing traffic, while varying network conditions.

Regarded the first run as the most interesting.

Can exit the browser and restart after a run like that.

At moment, merely plan to use the tool primarily to survey various
web sites and load times while doing packet captures. Hope was
to get valid data from the network portion of the load, tho...

> * If you have an advanced Chromium setup, this definitely does not wor= k. I
> advise using the benchmark extension only with a separate Chromium pro= file
> for testing purposes. Our flushing of sockets, caches, etc does not ac= tually
> work correctly when you use the Chromium multiprofile feature and also= fails
> to flush lots of our other network caches.

noted.


> * No one on Chromium really believes the time to paint numbers that we=
> output :) It's complicated. Our graphics stack is complicated. The= time from

I actually care only about time-to-full layout as that's a core n= etwork
effect...

> when Blink thinks it painted to when the GPU actually blits to the scr= een
> cannot currently be corroborated with any high degree of accuracy from=
> within our code.

> * It has not been maintained since 2010. It is quite likely there are = many
> other subtle inaccuracies here.

Grok.

> In short, while you can expect it to give you a very high level
> understanding of performance issues, I advise against placing non-triv= ial
> confidence in the accuracy of the numbers generated by the benchmark > extension. The fact that numbers are produced by the extension should = not be
> treated as evidence that the extension actually functions correctly.
OK, noted. Still delighted to be able to have a simple load generator=
that exercises the browsers and generates some results, however
dubious.

>
> Cheers.
>
>
> On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> Getting a grip on real web page load time behavior in an age of >> sharded websites,
>> dozens of dns lookups, javascript, and fairly random behavior in a= d
>> services
>> and cdns against how a modern browsers behaves is very, very hard.=
>>
>> it turns out if you run
>>
>> google-chrome --enable-benchmarking --enable-net-benchmarking
>>
>> (Mac users have to embed these options in their startup script - s= ee
>> =C2=A0http://www.chromium.org/developers/how= -tos/run-chromium-with-flags )
>>
>> enable developer options and install and run the chrome web page >> benchmarker,
>> (
>> https://chr= ome.google.com/webstore/detail/page-benchmarker/channimfdomahekjcahlbpccbga= opjll?hl=3Den
>> )
>>
>> that it works (at least for me, on a brief test of the latest chro= me, on
>> linux.
>> Can someone try windows and mac?)
>>
>> You can then feed in a list of urls to test against, and post proc= ess
>> the resulting .csv file to your hearts content. We used to use thi= s
>> benchmark a lot while trying to characterise typical web behaviors=
>> under aqm and packet scheduling systems under load. Running
>> it simultaneously with a rrul test or one of the simpler tcp uploa= d or
>> download
>> tests in the rrul suite was often quite interesting.
>>
>> It turned out the doc has been wrong a while as to the name of the= second
>> command lnie option. I was gearing up mentally for having to look = at
>> the source....
>>
>> http://code.google.com/p/chromium/issues/detail?id= =3D338705
>>
>> /me happy
>>
>> --
>> Dave T=C3=A4ht
>>
>> Heartbleed POC on wifi campus networks with EAP auth:
>> http://www.eduroam.edu.au/advisory.html
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>
>



--
Dave T=C3=A4ht

NSFW: https://w2.eff.org/Censorshi= p/Internet_censorship_bills/russell_0296_indecent.article

--089e013a0d40f5709304f7449169-- From nobody Fri Apr 18 11:15:47 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B09B91A02EB for ; Fri, 18 Apr 2014 11:15:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.964 X-Spam-Level: **** X-Spam-Status: No, score=4.964 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, MANGLED_SIDE=2.3, RP_MATCHES_RCVD=-0.272] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h39qbJV0xg1Z for ; Fri, 18 Apr 2014 11:15:39 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id E990F1A043F for ; Fri, 18 Apr 2014 11:15:38 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id s3IIFSWe016797; Fri, 18 Apr 2014 12:15:28 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 18 Apr 2014 12:15:28 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0174.001; Fri, 18 Apr 2014 12:15:28 -0600 From: Greg White To: Dave Taht Thread-Topic: [aqm] chrome web page benchmarker fixed Thread-Index: AQHPWmVePgFvJCotnEiEX+XB9sZQUJsWjr8AgAAjHoCAAP2yAA== Date: Fri, 18 Apr 2014 18:15:27 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.4.10.154] Content-Type: multipart/mixed; boundary="_002_CF76BDC532D89gwhitecablelabscom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/-Kb1bavcb9zhF0c_vgne2FaO9Og Cc: =?utf-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 18:15:43 -0000 --_002_CF76BDC532D89gwhitecablelabscom_ Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 RGF2ZSwgDQoNCldlIHVzZWQgdGhlIDI1ayBvYmplY3Qgc2l6ZSBmb3IgYSBzaG9ydCB0aW1lIGJh Y2sgaW4gMjAxMiB1bnRpbCB3ZSBoYWQNCnJlc291cmNlcyB0byBidWlsZCBhIG1vcmUgYWR2YW5j ZWQgbW9kZWwgKGFwcGVuZGl4IEEpLiAgSSBkaWQgYSBidW5jaCBvZg0KY2FwdHVyZXMgb2YgcmVh bCB3ZWIgcGFnZXMgYmFjayBpbiAyMDExIGFuZCBjb21wYXJlZCB0aGUgb2JqZWN0IHNpemUNCnN0 YXRpc3RpY3MgdG8gbW9kZWxzIHRoYXQgSSdkIHNlZW4gcHVibGlzaGVkLiAgTG9nbm9ybWFsIGRp ZG4ndCBzZWVtIHRvIGJlDQoqZXhhY3RseSogcmlnaHQsIGJ1dCBpdCB3YXNuJ3QgYSBiYWQgZml0 IHRvIHdoYXQgSSBzYXcuICBJJ3ZlIGF0dGFjaGVkIGENCkNERi4NCg0KVGhlIGNob2ljZSBvZiA0 IHNlcnZlcnMgd2FzIGJhc2VkIHNvbWV3aGF0IG9uIGxvZ2lzdGljcywgYW5kIGFsc28gb24gYQ0K ZmluZGluZyB0aGF0IGFjcm9zcyBvdXIgZGF0YSBzZXQsIHRoZSBhdmVyYWdlIHdlYiBwYWdlIHJl dHJpZXZlZCA4MSUgb2YNCml0cyByZXNvdXJjZXMgZnJvbSB0aGUgdG9wIDQgc2VydmVycy4gIElu Y3JlYXNpbmcgdG8gNSBzZXJ2ZXJzIG9ubHkNCmluY3JlYXNlZCB0aGF0IHBlcmNlbnRhZ2UgdG8g ODQlLg0KDQpUaGUgY2hvaWNlIG9mIFJUVHMgYWxzbyBjYW1lIGZyb20gdGhlIHdlYiB0cmFmZmlj IGNhcHR1cmVzLiBJIHNhdw0KUlRUbWluPTE2bXMsIFJUVG1lYW49NTMuOG1zLCBSVFRtYXg9MTM0 bXMuDQoNCk11Y2ggb2YgdGhpcyBjYW4gYmUgZm91bmQgaW4NCmh0dHBzOi8vdG9vbHMuaWV0Zi5v cmcvaHRtbC9kcmFmdC13aGl0ZS1odHRwYmlzLXNwZHktYW5hbHlzaXMtMDANCg0KSW4gbWFueSBv ZiB0aGUgY2FzZXMgdGhhdCB3ZSd2ZSBzaW11bGF0ZWQsIHRoZSBwYWNrZXQgZHJvcCBwcm9iYWJp bGl0eSBpcw0KbGVzcyB0aGFuIDElIGZvciBETlMgcGFja2V0cy4gIEluIG91ciB3ZWIgbW9kZWws IHRoZXJlIGFyZSBhIHRvdGFsIG9mIDQNCnNlcnZlcnMsIHNvIDQgRE5TIGxvb2t1cHMgYXNzdW1p bmcgbm9uZSBvZiB0aGUgYWRkcmVzc2VzIGFyZSBjYWNoZWQuIElmDQpQTFIgPSAxJSwgdGhlcmUg d291bGQgYmUgYSAzLjklIGNoYW5jZSBvZiBsb3Npbmcgb25lIG9yIG1vcmUgRE5TIHBhY2tldHMN Cih3aXRoIGEgcmVzdWx0aW5nIH41IHNlY29uZCBhZGRpdGlvbmFsIGRlbGF5IG9uIGxvYWQgdGlt ZSkuICBJJ3ZlIHByb2JhYmx5DQpvdmVyc2ltcGxpZmllZCB0aGlzLCBidXQgS2F0aGllIE4uIGFu ZCBJIG1hZGUgdGhlIGNhbGwgdGhhdCBpdCB3b3VsZCBiZQ0Kc2lnbmlmaWNhbnRseSBlYXNpZXIg dG8ganVzdCBkbyB0aGlzIG1hdGggdGhhbiB0byBidWlsZCBhIGRucw0KaW1wbGVtZW50YXRpb24g aW4gbnMyLiAgV2UndmUgb3BlbiBzb3VyY2VkIHRoZSB3ZWIgbW9kZWwgKGl0J3Mgb24gS2F0aGll J3MNCndlYiBwYWdlIGFuZCB3aWxsIGJlIHBhcnQgb2YgbnMyLjM2KSB3aXRoIGFuIGVuY291cmFn ZW1lbnQgdG8gdGhlDQpjb21tdW5pdHkgdG8gaW1wcm92ZSBvbiBpdC4gIElmIHlvdSdkIGxpa2Ug dG8gcG9ydCBpdCB0byBuczMgYW5kIGFkZCBhIGRucw0KbW9kZWwsIHRoYXQgd291bGQgYmUgZmFu dGFzdGljLg0KDQotR3JlZw0KDQoNCk9uIDQvMTcvMTQsIDM6MDcgUE0sICJEYXZlIFRhaHQiIDxk YXZlLnRhaHRAZ21haWwuY29tPiB3cm90ZToNCg0KPk9uIFRodSwgQXByIDE3LCAyMDE0IGF0IDEy OjAxIFBNLCBXaWxsaWFtIENoYW4gKOmZiOaZuuaYjCkNCj48d2lsbGNoYW5AY2hyb21pdW0ub3Jn PiB3cm90ZToNCj4+IFNwZWFraW5nIGFzIHRoZSBwcmltYXJ5IENocm9taXVtIGRldmVsb3BlciBp biBjaGFyZ2Ugb2YgdGhpcyByZWxldmFudA0KPj5jb2RlLA0KPj4gSSB3b3VsZCBsaWtlIHRvIGNh dXRpb24gcHV0dGluZyB0b28gbXVjaCB0cnVzdCBpbiB0aGUgbnVtYmVycw0KPj5nZW5lcmF0ZWQu IEFueQ0KPj4gc3RhdGlzdGljYWwgY2xhaW1zIGFib3V0IHRoZSBudW1iZXJzIGFyZSBwcm9iYWJs eSB1bnJlYXNvbmFibGUgdG8gbWFrZS4NCj4NCj5TaWdoLiBPdGhlciBiZW5jaG1hcmtzIHN1Y2gg YXMgdGhlIGFwYWNoZSAoImFiIikgYmVuY2htYXJrDQo+YXJlIHByaW1hcmlseSBkZXNpZ25lZCBh cyBzdHJlc3MgdGVzdGVycyBmb3Igd2ViIHNlcnZlcnMsIG5vdCBhcyByZWFsaXN0aWMNCj50cmFm ZmljLiBNb2Rlcm4gd2ViIHRyYWZmaWMgaGFzIHN1Y2ggYSBoaWdoIGxldmVsIG9mIGR5bmFtaWNp c20gaW4gaXQsDQo+dGhhdCBzdGF0aWMgd2ViIHBhZ2UgbG9hZHMgYWxvbmcgYW55IGRpc3RyaWJ1 dGlvbiwgc2VlbSBpbnN1ZmZpY2llbnQsDQo+cGFzc2l2ZSBhbmFseXNpcyBvZiBhZ2dyZWdhdGVk IHRyYWZmaWMgImZlZWxzIiBpbmNvcnJlY3QgcmVsYXRpdmUgdG8gdGhlDQo+c29ydHMgb2YgaG9t ZSBhbmQgc21hbGwgYnVzaW5lc3MgdHJhZmZpYyBJJ3ZlIHNlZW4sIGFuZCBzbyBvbi4NCj4NCj5G YW1vdXMgcGFwZXJzLCBzdWNoIGFzIHRoaXMgb25lOg0KPg0KPmh0dHA6Ly9jY3Iuc2lnY29tbS5v cmcvYXJjaGl2ZS8xOTk1L2phbjk1L2Njci05NTAxLWxlbGFuZC5wZGYNCj4NCj5TZWVtIHBvc3Np Ymx5IGlycmVsZXZhbnQgdG8gZHJhdyBjb25jbHVzaW9ucyBmcm9tIGdpdmVuIHRoZSBraW5kDQo+ b2YgZGF0YSB0aGV5IGFuYWx5c2VkIGFuZCBwcm9jZWVkaW5nIGZyb20gYW4gaW5jb3JyZWN0IG1v ZGVsIG9yDQo+Z3V0IGZlZWwgZm9yIGhvdyB0aGUgbmV0d29yayBiZWhhdmVzIHRvZGF5IHNlZW1z IHRvIGJlIGZvb2xpc2guDQo+DQo+RXZlbiB0aGUgbW9zdCBiYXNpYyBvZiB0b29scywgc3VjaCBh cyBodHRwaW5nLCBoYWQgdGhyZWUgYmFzaWMgYnVncw0KPnRoYXQgSSBmb3VuZCBpbiBhIGZldyBt aW51dGVzIG9mIHRyeWluZyB0byBjb21lIHVwIHdpdGggc29tZSBiYXNpYw0KPmJlaGF2aW9ycyB5 ZXN0ZXJkYXk6DQo+DQo+aHR0cHM6Ly9saXN0cy5idWZmZXJibG9hdC5uZXQvcGlwZXJtYWlsL2Js b2F0LzIwMTQtQXByaWwvMDAxODkwLmh0bWwNCj4NCj5UaG9zZSBhcmUgZ29pbmcgdG8gYmUgYSBs b3QgZWFzaWVyIHRvIGZpeCB0aGFuIGRpdmluZyBpbnRvIHRoZSBjaHJvbWl1bQ0KPmNvZGViYXNl IQ0KPg0KPlRoZXJlIGFyZSB2ZXJ5IGZldyB0b29scyB3b3J0aCB0cnVzdGluZywgYW5kIEkgYW0g YWx3YXlzIGR1YmlvdXMNCj5vZiBwYXBlcnMgdGhhdCBwdWJsaXNoIHJlc3VsdHMgd2l0aCB1bmF2 YWlsYWJsZSB0b29scyBhbmQgZGF0YS4gVGhlIG9ubHkNCj50b29scyBJIGhhdmUgYW55IGZhaXRo IGluIGZvciBuZXR3b3JrIGFuYWx5c2lzIGFyZSBuZXRwZXJmLA0KPm5ldHBlcmYtd3JhcHBlciwN Cj50Y3BkdW1wIGFuZCB4cGxvdC5vcmcsIGFuZCB0byBhIGxhcmdlIGV4dGVudCB3aXJlc2hhcmsu IFRva2UgYW5kIEkgaGF2ZQ0KPmJlZW4gdGVhcmluZyBhcGFydCBkLWl0ZyBhbmQgSSBob3BlIHRv IG9uZSBkYXkgYmUgYWJsZSB0byBhZGQgdGhhdCB0bw0KPm15IHRydXN0YWJsZSBsaXN0Li4uIGJ1 dCBiZXR0ZXIgdG9vbHMgYXJlIG5lZWRlZCENCj4NCj5Ub29scyB0aGF0IEkgZG9uJ3QgaGF2ZSBh IGxvdCBvZiBmYWl0aCBpbiBpbmNsdWRlIHRoYXQsIGlwZXJmLCBhbnl0aGluZw0KPndyaXR0ZW4N Cj5pbiBqYXZhIG9yIG90aGVyIGhpZ2ggbGV2ZWwgbGFuZ3VhZ2VzLCBzcGVlZHRlc3QubmV0LCBh bmQgdGhpbmdzIGxpa2UNCj5zaGFwZXJwcm9iZS4NCj4NCj5IYXZlIHZlcnkgbGl0dGxlIGZhaXRo IGluIG5zMiwgc2xpZ2h0bHkgbW9yZSBpbiBuczMsIGFuZCBJJ3ZlIGJlZW4gbWVhbmluZw0KPnRv IGxvb2sgb3ZlciB0aGUgbWluaW5ldCBhbmQgb3RoZXIgc2ltdWxhdG9ycyB3aGVuZXZlciBJIGdv dCBzb21lIHNwYXJlDQo+dGltZTsgdGhlIG1pbmluZXQgcmVzdWx0cyBzdGFuZm9yZCBnZXRzIHNl ZW0gcHJldHR5IHJlYXNvbmFibGUgYW5kIEkNCj5hZG9yZSB0aGVpciByZXByb2R1Y2luZyByZXN1 bHRzIGVmZm9ydC4gSGF2ZW4ndCBleHBsb3JlZCBuZHQsIGtlZXAgbWVhbmluZw0KPnRvLi4uDQo+ DQo+PiBSZWFzb25zOg0KPj4gKiBXZSBkb24ndCBhY3RpdmVseSBtYWludGFpbiB0aGlzIGNvZGUu IEl0J3MgYmVoaW5kIHRoZSBjb21tYW5kIGxpbmUNCj4+ZmxhZ3MuDQo+PiBUaGV5IGFyZSBicm9r ZW4uIFRoZSBmYWN0IHRoYXQgaXQgc3RpbGwgcmVzdWx0cyBpbiBudW1iZXJzIG9uIHRoZQ0KPj5i ZW5jaG1hcmsNCj4+IGV4dGVuc2lvbiBpcyBhbiBleGFtcGxlIG9mIHdoZXJlIHVubWFpbnRhaW5l ZCBjb2RlIGRvZXNuJ3QgaGF2ZSB0aGUgVUkNCj4+IGRpc2FibGVkLCBldmVuIHRob3VnaCB0aGUg aW50ZXJuYWwgd29ya2luZ3Mgb2YgdGhlIGNvZGUgZmFpbCB0bw0KPj5ndWFyYW50ZWUNCj4+IGNv cnJlY3Qgb3BlcmF0aW9uLiBXZSBoYXZlbid0IGRpc2FibGVkIGl0IGJlY2F1c2UsIHdlbGwsIGl0 J3MNCj4+dW5tYWludGFpbmVkLg0KPg0KPkFzIEkgbWVudGlvbmVkIEkgd2FzIGdlYXJpbmcgdXAg Zm9yIGEgaGFja2luZyBydW4uLi4NCj4NCj5UaGUgdmFzdCBtYWpvcml0eSBvZiByZXN1bHRzIEkg bG9vayBhdCBhcmUgYWN0dWFsbHkgb2J0YWluZWQgdmlhDQo+bG9va2luZyBhdCBwYWNrZXQgY2Fw dHVyZXMuIEkgbW9zdGx5IHVzZSBiZW5jaG1hcmtzIGFzIGFic3RyYWN0aW9ucw0KPnRvIHNlZSBp ZiB0aGV5IG1ha2Ugc29tZSBzZW5zZSByZWxhdGl2ZSB0byB0aGUgY2FwdHVyZXMgYW5kIHRlbmQN Cj50byBjb21wYXJlIGRpZmZlcmVudCBiZW5jaG1hcmtzIGFnYWluc3QgZWFjaCBvdGhlci4NCj4N Cj5JIHJlYWxpemUgb3RoZXJzIGRvbid0IGdvIGludG8gdGhhdCBsZXZlbCBvZiBkZXRhaWwsIHNv IHlvdSBoYXZlIGdpdmVuDQo+ZmFpciB3YXJuaW5nISBJbiBvdXIgY2FzZSB3ZSB1c2VkIHRoZSB3 ZWIgcGFnZSBiZW5jaG1hcmtlciBhcw0KPmEgbWVhbnMgdG8gdHJ5IGFuZCByYXBpZGx5IGFjcXVp cmUgc29tZSB0eXBpY2FsIGRpc3RyaWJ1dGlvbnMgb2YNCj5nZXQgYW5kIHRjcCBzdHJlYW0gcmVx dWVzdHMgZnJvbSB0aGluZ3MgbGlrZSB0aGUgYWxleGEgdG9wIDEwMDAsDQo+YW5kIGFzIGEgd2F5 IHRvIEEvQiBkaWZmZXJlbnQgYXFtL3BhY2tldCBzY2hlZHVsaW5nIHNldHVwcy4NCj4NCj4uLi4g YnV0IHRoZSBvbmx5IGVhc2lseSBwdWJsaXNoYWJsZSByZXN1bHRzIHdlcmUgZnJvbSB0aGUgYmVu Y2htYXJrDQo+aXRzZWxmLA0KPmFuZCB3ZSAocmVsdWN0YW50bHkpIG9ubHkgcHVibGlzaGVkIG9u ZSBncmFwaCBmcm9tIGFsbCB0aGUgd29yayB0aGF0DQo+d2VudCBpbnRvIGl0IDIrIHllYXJzIGJh Y2sgYW5kIHVzZWQgaXQgYXMgYSB0ZXN0IGRyaXZlciBmb3IgdGhlIGZhbW91cw0KPmlldGYgdmlk ZW8sIGNvbXBhcmluZyB0d28gaWRlbnRpY2FsIGJveGVzIHJ1bm5pbmcgaXQgYXQgdGhlIHNhbWUg dGltZQ0KPnVuZGVyIGRpZmZlcmVudCBuZXR3b3JrIGNvbmRpdGlvbnM6DQo+DQo+aHR0cHM6Ly93 d3cuYnVmZmVyYmxvYXQubmV0L3Byb2plY3RzL2Nlcm93cnQvd2lraS9CbG9hdC12aWRlb3MjSUVU Ri1kZW1vLXMNCj5pZGUtYnktc2lkZS1vZi1hLW5vcm1hbC1jYWJsZS1tb2RlbS12cy1mcV9jb2Rl bA0KPg0KPmZyb20gd2hhdCBJIGZpZGRsZWQgd2l0aCB0b2RheSwgaXQgaXMgYXQgbGVhc3Qgc3Rp bGwgdXNlZnVsIGZvciB0aGF0Pw0KPg0KPm1vdmluZyBvbi4uLg0KPg0KPlRoZSB3ZWIgbW9kZWwg aW4gdGhlIGNhYmxlbGFicyB3b3JrIGRvZXNuJ3QgbG9vayBtdWNoIGxpa2UgbXkgY2FwdHVyZXMs DQo+aW4gYWRkaXRpb24gdG8gbm90IG1vZGVsaW5nIGRucyBhdCBhbGwsIGFuZCB1c2luZyBhIHNt YWxsZXIgSVcgdGhhbiBnb29nbGUNCj5pdCBsb29rcyBsaWtlIHRoaXM6DQo+DQo+Pj4gTW9kZWwg c2luZ2xlIHVzZXIgd2ViIHBhZ2UgZG93bmxvYWQgYXMgZm9sbG93czoNCj4NCj4+PiAtIFdlYiBw YWdlIG1vZGVsZWQgYXMgc2luZ2xlIEhUTUwgcGFnZSArIDEwMCBvYmplY3RzIHNwcmVhZCBldmVu bHkNCj4+PiBhY3Jvc3MgNCBzZXJ2ZXJzLiBXZWIgb2JqZWN0IHNpemVzIGFyZSBjdXJyZW50bHkg Zml4ZWQgYXQgMjUga0IgZWFjaCwNCj4+PiB3aGVyZWFzIHRoZSBpbml0aWFsIEhUTUwgcGFnZSBp cyAxMDAga0IuIEFwcGVuZGl4IEEgcHJvdmlkZXMgYW4NCj4+PiBhbHRlcm5hdGl2ZSBwYWdlIG1v ZGVsIHRoYXQgbWF5IGJlIGV4cGxvcmVkIGluIGZ1dHVyZSB3b3JrLg0KPg0KPldoZXJlIHdoYXQg SSBzZWUgaXMgYSBodWdlIG51bWJlciBvZiBzdHVmZiB0aGF0IGZpdHMgaW50byBhIHNpbmdsZQ0K Pml3MTAgc2xvdyBzdGFydCBlcGlzb2RlIGFuZCBzb21lIGxldmVsIG9mIHBpcGVsaW5pbmcgb24g bGFyZ2VyIHN0dWZmLCBzbw0KPnRoYXQgYQ0KPmxhcmdlIG51bWJlciBvZiBvYmplY3Qgc2l6ZXMg b2YgbGVzcyB0aGFuIDdrIHdpdGggYSBsaWdodGx5IHRhaWxlZA0KPmRpc3RyaWJ1dGlvbg0KPm91 dHNpZGUgb2YgdGhhdCBtYWtlcyBtb3JlIHNlbnNlLg0KPg0KPihJJ20gbm90IHN0YXJpbmcgYXQg YXBwZW5kaXggQSByaWdodCBub3csIEknbSB1bmRlciB0aGUgaW1wcmVzc2lvbg0KPiBpdCB3YXMg YmV0dGVyKQ0KPg0KPkkgY2VydGFpbmx5IHdvdWxkIGxpa2UgbW9yZSBzdWdnZXN0aW9ucyBmb3Ig bW9kZWxzIGFuZCB0eXBlcw0KPm9mIHdlYiB0cmFmZmljLCBhcyB3ZWxsIGFzIHNpbXVsYXRpb24g b2YgaHR0cHMgKyBwZnMgdHJhZmZpYywNCj5zcGR5LCBxdWljLCBldGMuLi4uDQo+DQo+Pj4gLSBT ZXJ2ZXIgUlRUcyBzZXQgYXMgZm9sbG93cyAoMjAgbXMsIDMwIG1zLCA1MCBtcywgMTAwIG1zKS4N Cj4NCj5TZXJ2ZXIgUlRUcyBmcm9tIG15IG93biB3ZWIgaGlzdG9yeSB0ZW5kIHRvIGJlIGxvd2Vy IHRoYW4gNTBtcy4NCj4NCj4+PiAtIEluaXRpYWwgSFRUUCBHRVQgdG8gcmV0cmlldmUgYSBtb2Rl cmF0ZWx5IHNpemVkIG9iamVjdCAoMTAwIGtCIEhUTUwNCj4+PiBwYWdlKSBmcm9tIHNlcnZlciAx Lg0KPg0KPkFuIGluaXRpYWwgR0VUIHRvIGdvb2dsZSBmaXRzIGludG8gaXcxMCAtIGl0J3MgYWJv dXQgN2suDQo+DQo+Pj4gLSBPbmNlIGluaXRpYWwgSFRUUCBHRVQgY29tcGxldGVzLCBpbml0aWF0 ZSAyNCBzaW11bHRhbmVvdXMgSFRUUCBHRVRzDQo+Pj4gKHZpYSBzZXBhcmF0ZSBUQ1AgY29ubmVj dGlvbnMpLCA2IGNvbm5lY3Rpb25zIGVhY2ggdG8gNCBkaWZmZXJlbnQNCj4+PiBzZXJ2ZXIgbm9k ZXMNCj4NCj5JIHVzdWFsbHkgZG9uJ3Qgc2VlIG1vcmUgdGhhbiAxNS4gYW5kIGNlcnRhaW5seSBu b3QgMjVrIHNpemVkIG9iamVjdHMuDQo+DQo+ID4gLSBPbmNlIGVhY2ggaW5kaXZpZHVhbCBIVFRQ IEdFVCBjb21wbGV0ZXMsIGluaXRpYXRlIGEgc3Vic2VxdWVudCBHRVQNCj4+IHRvIHRoZSBzYW1l IHNlcnZlciwgdW50aWwgMjUgb2JqZWN0cyBoYXZlIGJlZW4gcmV0cmlldmVkIGZyb20gZWFjaA0K Pj4gc2VydmVyLg0KPg0KPg0KPj4gKiBXZSBkb24ndCBtYWtlIHN1cmUgdG8gZmx1c2ggYWxsIHRo ZSBuZXR3b3JrIHN0YXRlIGluIGJldHdlZW4gcnVucywgc28NCj4+aWYNCj4+IHlvdSdyZSB1c2lu ZyB0aGF0IG9wdGlvbiwgZG9uJ3QgdHJ1c3QgaXQgdG8gd29yay4NCj4NCj5UaGUgdHlwaWNhbCBz Y2VuYXJpbyB3ZSB1c2VkIHdhcyBhIHJ1biBhZ2FpbnN0IGRvemVucyBvciBodW5kcmVkcyBvZiB1 cmxzLA0KPmNhcHR1cmluZyB0cmFmZmljLCB3aGlsZSB2YXJ5aW5nIG5ldHdvcmsgY29uZGl0aW9u cy4NCj4NCj5SZWdhcmRlZCB0aGUgZmlyc3QgcnVuIGFzIHRoZSBtb3N0IGludGVyZXN0aW5nLg0K Pg0KPkNhbiBleGl0IHRoZSBicm93c2VyIGFuZCByZXN0YXJ0IGFmdGVyIGEgcnVuIGxpa2UgdGhh dC4NCj4NCj5BdCBtb21lbnQsIG1lcmVseSBwbGFuIHRvIHVzZSB0aGUgdG9vbCBwcmltYXJpbHkg dG8gc3VydmV5IHZhcmlvdXMNCj53ZWIgc2l0ZXMgYW5kIGxvYWQgdGltZXMgd2hpbGUgZG9pbmcg cGFja2V0IGNhcHR1cmVzLiBIb3BlIHdhcw0KPnRvIGdldCB2YWxpZCBkYXRhIGZyb20gdGhlIG5l dHdvcmsgcG9ydGlvbiBvZiB0aGUgbG9hZCwgdGhvLi4uDQo+DQo+PiAqIElmIHlvdSBoYXZlIGFu IGFkdmFuY2VkIENocm9taXVtIHNldHVwLCB0aGlzIGRlZmluaXRlbHkgZG9lcyBub3QNCj4+d29y ay4gSQ0KPj4gYWR2aXNlIHVzaW5nIHRoZSBiZW5jaG1hcmsgZXh0ZW5zaW9uIG9ubHkgd2l0aCBh IHNlcGFyYXRlIENocm9taXVtDQo+PnByb2ZpbGUNCj4+IGZvciB0ZXN0aW5nIHB1cnBvc2VzLiBP dXIgZmx1c2hpbmcgb2Ygc29ja2V0cywgY2FjaGVzLCBldGMgZG9lcyBub3QNCj4+YWN0dWFsbHkN Cj4+IHdvcmsgY29ycmVjdGx5IHdoZW4geW91IHVzZSB0aGUgQ2hyb21pdW0gbXVsdGlwcm9maWxl IGZlYXR1cmUgYW5kIGFsc28NCj4+ZmFpbHMNCj4+IHRvIGZsdXNoIGxvdHMgb2Ygb3VyIG90aGVy IG5ldHdvcmsgY2FjaGVzLg0KPg0KPm5vdGVkLg0KPg0KPg0KPj4gKiBObyBvbmUgb24gQ2hyb21p dW0gcmVhbGx5IGJlbGlldmVzIHRoZSB0aW1lIHRvIHBhaW50IG51bWJlcnMgdGhhdCB3ZQ0KPj4g b3V0cHV0IDopIEl0J3MgY29tcGxpY2F0ZWQuIE91ciBncmFwaGljcyBzdGFjayBpcyBjb21wbGlj YXRlZC4gVGhlIHRpbWUNCj4+ZnJvbQ0KPg0KPkkgYWN0dWFsbHkgY2FyZSBvbmx5IGFib3V0IHRp bWUtdG8tZnVsbCBsYXlvdXQgYXMgdGhhdCdzIGEgY29yZSBuZXR3b3JrDQo+ZWZmZWN0Li4uDQo+ DQo+PiB3aGVuIEJsaW5rIHRoaW5rcyBpdCBwYWludGVkIHRvIHdoZW4gdGhlIEdQVSBhY3R1YWxs eSBibGl0cyB0byB0aGUNCj4+c2NyZWVuDQo+PiBjYW5ub3QgY3VycmVudGx5IGJlIGNvcnJvYm9y YXRlZCB3aXRoIGFueSBoaWdoIGRlZ3JlZSBvZiBhY2N1cmFjeSBmcm9tDQo+PiB3aXRoaW4gb3Vy IGNvZGUuDQo+DQo+PiAqIEl0IGhhcyBub3QgYmVlbiBtYWludGFpbmVkIHNpbmNlIDIwMTAuIEl0 IGlzIHF1aXRlIGxpa2VseSB0aGVyZSBhcmUNCj4+bWFueQ0KPj4gb3RoZXIgc3VidGxlIGluYWNj dXJhY2llcyBoZXJlLg0KPg0KPkdyb2suDQo+DQo+PiBJbiBzaG9ydCwgd2hpbGUgeW91IGNhbiBl eHBlY3QgaXQgdG8gZ2l2ZSB5b3UgYSB2ZXJ5IGhpZ2ggbGV2ZWwNCj4+IHVuZGVyc3RhbmRpbmcg b2YgcGVyZm9ybWFuY2UgaXNzdWVzLCBJIGFkdmlzZSBhZ2FpbnN0IHBsYWNpbmcNCj4+bm9uLXRy aXZpYWwNCj4+IGNvbmZpZGVuY2UgaW4gdGhlIGFjY3VyYWN5IG9mIHRoZSBudW1iZXJzIGdlbmVy YXRlZCBieSB0aGUgYmVuY2htYXJrDQo+PiBleHRlbnNpb24uIFRoZSBmYWN0IHRoYXQgbnVtYmVy cyBhcmUgcHJvZHVjZWQgYnkgdGhlIGV4dGVuc2lvbiBzaG91bGQNCj4+bm90IGJlDQo+PiB0cmVh dGVkIGFzIGV2aWRlbmNlIHRoYXQgdGhlIGV4dGVuc2lvbiBhY3R1YWxseSBmdW5jdGlvbnMgY29y cmVjdGx5Lg0KPg0KPk9LLCBub3RlZC4gU3RpbGwgZGVsaWdodGVkIHRvIGJlIGFibGUgdG8gaGF2 ZSBhIHNpbXBsZSBsb2FkIGdlbmVyYXRvcg0KPnRoYXQgZXhlcmNpc2VzIHRoZSBicm93c2VycyBh bmQgZ2VuZXJhdGVzIHNvbWUgcmVzdWx0cywgaG93ZXZlcg0KPmR1YmlvdXMuDQo+DQo+Pg0KPj4g Q2hlZXJzLg0KPj4NCj4+DQo+PiBPbiBUaHUsIEFwciAxNywgMjAxNCBhdCAxMDo0OSBBTSwgRGF2 ZSBUYWh0IDxkYXZlLnRhaHRAZ21haWwuY29tPiB3cm90ZToNCj4+Pg0KPj4+IEdldHRpbmcgYSBn cmlwIG9uIHJlYWwgd2ViIHBhZ2UgbG9hZCB0aW1lIGJlaGF2aW9yIGluIGFuIGFnZSBvZg0KPj4+ IHNoYXJkZWQgd2Vic2l0ZXMsDQo+Pj4gZG96ZW5zIG9mIGRucyBsb29rdXBzLCBqYXZhc2NyaXB0 LCBhbmQgZmFpcmx5IHJhbmRvbSBiZWhhdmlvciBpbiBhZA0KPj4+IHNlcnZpY2VzDQo+Pj4gYW5k IGNkbnMgYWdhaW5zdCBob3cgYSBtb2Rlcm4gYnJvd3NlcnMgYmVoYXZlcyBpcyB2ZXJ5LCB2ZXJ5 IGhhcmQuDQo+Pj4NCj4+PiBpdCB0dXJucyBvdXQgaWYgeW91IHJ1bg0KPj4+DQo+Pj4gZ29vZ2xl LWNocm9tZSAtLWVuYWJsZS1iZW5jaG1hcmtpbmcgLS1lbmFibGUtbmV0LWJlbmNobWFya2luZw0K Pj4+DQo+Pj4gKE1hYyB1c2VycyBoYXZlIHRvIGVtYmVkIHRoZXNlIG9wdGlvbnMgaW4gdGhlaXIg c3RhcnR1cCBzY3JpcHQgLSBzZWUNCj4+PiAgaHR0cDovL3d3dy5jaHJvbWl1bS5vcmcvZGV2ZWxv cGVycy9ob3ctdG9zL3J1bi1jaHJvbWl1bS13aXRoLWZsYWdzICkNCj4+Pg0KPj4+IGVuYWJsZSBk ZXZlbG9wZXIgb3B0aW9ucyBhbmQgaW5zdGFsbCBhbmQgcnVuIHRoZSBjaHJvbWUgd2ViIHBhZ2UN Cj4+PiBiZW5jaG1hcmtlciwNCj4+PiAoDQo+Pj4gDQo+Pj5odHRwczovL2Nocm9tZS5nb29nbGUu Y29tL3dlYnN0b3JlL2RldGFpbC9wYWdlLWJlbmNobWFya2VyL2NoYW5uaW1mZG9tYWgNCj4+PmVr amNhaGxicGNjYmdhb3BqbGw/aGw9ZW4NCj4+PiApDQo+Pj4NCj4+PiB0aGF0IGl0IHdvcmtzIChh dCBsZWFzdCBmb3IgbWUsIG9uIGEgYnJpZWYgdGVzdCBvZiB0aGUgbGF0ZXN0IGNocm9tZSwNCj4+ Pm9uDQo+Pj4gbGludXguDQo+Pj4gQ2FuIHNvbWVvbmUgdHJ5IHdpbmRvd3MgYW5kIG1hYz8pDQo+ Pj4NCj4+PiBZb3UgY2FuIHRoZW4gZmVlZCBpbiBhIGxpc3Qgb2YgdXJscyB0byB0ZXN0IGFnYWlu c3QsIGFuZCBwb3N0IHByb2Nlc3MNCj4+PiB0aGUgcmVzdWx0aW5nIC5jc3YgZmlsZSB0byB5b3Vy IGhlYXJ0cyBjb250ZW50LiBXZSB1c2VkIHRvIHVzZSB0aGlzDQo+Pj4gYmVuY2htYXJrIGEgbG90 IHdoaWxlIHRyeWluZyB0byBjaGFyYWN0ZXJpc2UgdHlwaWNhbCB3ZWIgYmVoYXZpb3JzDQo+Pj4g dW5kZXIgYXFtIGFuZCBwYWNrZXQgc2NoZWR1bGluZyBzeXN0ZW1zIHVuZGVyIGxvYWQuIFJ1bm5p bmcNCj4+PiBpdCBzaW11bHRhbmVvdXNseSB3aXRoIGEgcnJ1bCB0ZXN0IG9yIG9uZSBvZiB0aGUg c2ltcGxlciB0Y3AgdXBsb2FkIG9yDQo+Pj4gZG93bmxvYWQNCj4+PiB0ZXN0cyBpbiB0aGUgcnJ1 bCBzdWl0ZSB3YXMgb2Z0ZW4gcXVpdGUgaW50ZXJlc3RpbmcuDQo+Pj4NCj4+PiBJdCB0dXJuZWQg b3V0IHRoZSBkb2MgaGFzIGJlZW4gd3JvbmcgYSB3aGlsZSBhcyB0byB0aGUgbmFtZSBvZiB0aGUN Cj4+PnNlY29uZA0KPj4+IGNvbW1hbmQgbG5pZSBvcHRpb24uIEkgd2FzIGdlYXJpbmcgdXAgbWVu dGFsbHkgZm9yIGhhdmluZyB0byBsb29rIGF0DQo+Pj4gdGhlIHNvdXJjZS4uLi4NCj4+Pg0KPj4+ IGh0dHA6Ly9jb2RlLmdvb2dsZS5jb20vcC9jaHJvbWl1bS9pc3N1ZXMvZGV0YWlsP2lkPTMzODcw NQ0KPj4+DQo+Pj4gL21lIGhhcHB5DQo+Pj4NCj4+PiAtLQ0KPj4+IERhdmUgVMOkaHQNCj4+Pg0K Pj4+IEhlYXJ0YmxlZWQgUE9DIG9uIHdpZmkgY2FtcHVzIG5ldHdvcmtzIHdpdGggRUFQIGF1dGg6 DQo+Pj4gaHR0cDovL3d3dy5lZHVyb2FtLmVkdS5hdS9hZHZpc29yeS5odG1sDQo+Pj4NCj4+PiBf X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPj4+IGFxbSBt YWlsaW5nIGxpc3QNCj4+PiBhcW1AaWV0Zi5vcmcNCj4+PiBodHRwczovL3d3dy5pZXRmLm9yZy9t YWlsbWFuL2xpc3RpbmZvL2FxbQ0KPj4NCj4+DQo+DQo+DQo+DQo+LS0gDQo+RGF2ZSBUw6RodA0K Pg0KPk5TRlc6IA0KPmh0dHBzOi8vdzIuZWZmLm9yZy9DZW5zb3JzaGlwL0ludGVybmV0X2NlbnNv cnNoaXBfYmlsbHMvcnVzc2VsbF8wMjk2X2luZGVjDQo+ZW50LmFydGljbGUNCj4NCj5fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPmFxbSBtYWlsaW5nIGxp c3QNCj5hcW1AaWV0Zi5vcmcNCj5odHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2FxbQ0KDQo= --_002_CF76BDC532D89gwhitecablelabscom_ Content-Type: application/octet-stream; name="content_length_cdf.eps" Content-Description: content_length_cdf.eps Content-Disposition: attachment; filename="content_length_cdf.eps"; size=17824; creation-date="Fri, 18 Apr 2014 18:15:27 GMT"; modification-date="Fri, 18 Apr 2014 18:15:27 GMT" Content-ID: <31F5D60375D5D04FB33E5F3E88F21B8E@cablelabs.com> Content-Transfer-Encoding: base64 JSFQUy1BZG9iZS0zLjAgRVBTRi0zLjAKJSVDcmVhdG9yOiBNQVRMQUIsIFRoZSBNYXRoV29ya3Ms IEluYy4gVmVyc2lvbiA3LjE0LjAuNzM5IChSMjAxMmEpLiBPcGVyYXRpbmcgU3lzdGVtOiBEYXJ3 aW4gMTIuMy4wIERhcndpbiBLZXJuZWwgVmVyc2lvbiAxMi4zLjA6IFN1biBKYW4gIDYgMjI6Mzc6 MTAgUFNUIDIwMTM7IHJvb3Q6eG51LTIwNTAuMjIuMTN+MS9SRUxFQVNFX1g4Nl82NCB4ODZfNjQu CiUlVGl0bGU6IC9Vc2Vycy9nd2hpdGUvRG9jdW1lbnRzL0Zhc3RlckludGVybmV0L1NQRFkvV2Vi c2l0ZSB0cmFjZXMvY29udGVudF9sZW5ndGhfY2RmLmVwcwolJUNyZWF0aW9uRGF0ZTogMDUvMDcv MjAxMyAgMTQ6MzU6MzUKJSVEb2N1bWVudE5lZWRlZEZvbnRzOiBIZWx2ZXRpY2EKJSVEb2N1bWVu dFByb2Nlc3NDb2xvcnM6IEN5YW4gTWFnZW50YSBZZWxsb3cgQmxhY2sKJSVMYW5ndWFnZUxldmVs OiAyCiUlUGFnZXM6IDEKJSVCb3VuZGluZ0JveDogIC0xMDQgICAxMjUgICA3MTcgICA2NjcKJSVF bmRDb21tZW50cwoKJSVCZWdpblByb2xvZwolIE1hdGhXb3JrcyBkaWN0aW9uYXJ5Ci9NYXRoV29y a3MgMTYwIGRpY3QgYmVnaW4KJSBkZWZpbml0aW9uIG9wZXJhdG9ycwovYmRlZiB7YmluZCBkZWZ9 IGJpbmQgZGVmCi9sZGVmIHtsb2FkIGRlZn0gYmluZCBkZWYKL3hkZWYge2V4Y2ggZGVmfSBiZGVm Ci94c3RvcmUge2V4Y2ggc3RvcmV9IGJkZWYKJSBvcGVyYXRvciBhYmJyZXZpYXRpb25zCi9jICAv Y2xpcCBsZGVmCi9jYyAvY29uY2F0IGxkZWYKL2NwIC9jbG9zZXBhdGggbGRlZgovZ3IgL2dyZXN0 b3JlIGxkZWYKL2dzIC9nc2F2ZSBsZGVmCi9tdCAvbW92ZXRvIGxkZWYKL25wIC9uZXdwYXRoIGxk ZWYKL2NtIC9jdXJyZW50bWF0cml4IGxkZWYKL3NtIC9zZXRtYXRyaXggbGRlZgovcm0gL3Jtb3Zl dG8gbGRlZgovcmwgL3JsaW5ldG8gbGRlZgovcyB7c2hvdyBuZXdwYXRofSBiZGVmCi9zYyB7c2V0 Y215a2NvbG9yfSBiZGVmCi9zciAvc2V0cmdiY29sb3IgbGRlZgovc2cgL3NldGdyYXkgbGRlZgov dyAvc2V0bGluZXdpZHRoIGxkZWYKL2ogL3NldGxpbmVqb2luIGxkZWYKL2NhcCAvc2V0bGluZWNh cCBsZGVmCi9yYyB7cmVjdGNsaXB9IGJkZWYKL3JmIHtyZWN0ZmlsbH0gYmRlZgolIHBhZ2Ugc3Rh dGUgY29udHJvbAovcGdzdiAoKSBkZWYKL2JwYWdlIHsvcGdzdiBzYXZlIGRlZn0gYmRlZgovZXBh Z2Uge3Bnc3YgcmVzdG9yZX0gYmRlZgovYnBsb3QgL2dzYXZlIGxkZWYKL2VwbG90IHtzdHJva2Ug Z3Jlc3RvcmV9IGJkZWYKJSBvcmllbnRhdGlvbiBzd2l0Y2gKL3BvcnRyYWl0TW9kZSAwIGRlZiAv bGFuZHNjYXBlTW9kZSAxIGRlZiAvcm90YXRlTW9kZSAyIGRlZgolIGNvb3JkaW5hdGUgc3lzdGVt IG1hcHBpbmdzCi9kcGkycG9pbnQgMCBkZWYKJSBmb250IGNvbnRyb2wKL0ZvbnRTaXplIDAgZGVm Ci9GTVMgey9Gb250U2l6ZSB4c3RvcmUgZmluZGZvbnQgW0ZvbnRTaXplIDAgMCBGb250U2l6ZSBu ZWcgMCAwXQogIG1ha2Vmb250IHNldGZvbnR9IGJkZWYKL3JlZW5jb2RlIHtleGNoIGR1cCB3aGVy ZSB7cG9wIGxvYWR9IHtwb3AgU3RhbmRhcmRFbmNvZGluZ30gaWZlbHNlCiAgZXhjaCBkdXAgMyAx IHJvbGwgZmluZGZvbnQgZHVwIGxlbmd0aCBkaWN0IGJlZ2luCiAgeyAxIGluZGV4IC9GSUQgbmUg e2RlZn17cG9wIHBvcH0gaWZlbHNlIH0gZm9yYWxsCiAgL0VuY29kaW5nIGV4Y2ggZGVmIGN1cnJl bnRkaWN0IGVuZCBkZWZpbmVmb250IHBvcH0gYmRlZgovaXNyb21hbiB7ZmluZGZvbnQgL0NoYXJT dHJpbmdzIGdldCAvQWdyYXZlIGtub3dufSBiZGVmCi9GTVNSIHszIDEgcm9sbCAxIGluZGV4IGR1 cCBpc3JvbWFuIHtyZWVuY29kZX0ge3BvcCBwb3B9IGlmZWxzZQogIGV4Y2ggRk1TfSBiZGVmCi9j c20gezEgZHBpMnBvaW50IGRpdiAtMSBkcGkycG9pbnQgZGl2IHNjYWxlIG5lZyB0cmFuc2xhdGUK IGR1cCBsYW5kc2NhcGVNb2RlIGVxIHtwb3AgLTkwIHJvdGF0ZX0KICB7cm90YXRlTW9kZSBlcSB7 OTAgcm90YXRlfSBpZn0gaWZlbHNlfSBiZGVmCiUgbGluZSB0eXBlczogc29saWQsIGRvdHRlZCwg ZGFzaGVkLCBkb3RkYXNoCi9TTyB7IFtdIDAgc2V0ZGFzaCB9IGJkZWYKL0RPIHsgWy41IGRwaTJw b2ludCBtdWwgNCBkcGkycG9pbnQgbXVsXSAwIHNldGRhc2ggfSBiZGVmCi9EQSB7IFs2IGRwaTJw b2ludCBtdWxdIDAgc2V0ZGFzaCB9IGJkZWYKL0REIHsgWy41IGRwaTJwb2ludCBtdWwgNCBkcGky cG9pbnQgbXVsIDYgZHBpMnBvaW50IG11bCA0CiAgZHBpMnBvaW50IG11bF0gMCBzZXRkYXNoIH0g YmRlZgolIG1hY3JvcyBmb3IgbGluZXMgYW5kIG9iamVjdHMKL0wge2xpbmV0byBzdHJva2V9IGJk ZWYKL01QIHszIDEgcm9sbCBtb3ZldG8gMSBzdWIge3JsaW5ldG99IHJlcGVhdH0gYmRlZgovQVAg e3tybGluZXRvfSByZXBlYXR9IGJkZWYKL1BEbHcgLTEgZGVmCi9XIHsvUERsdyBjdXJyZW50bGlu ZXdpZHRoIGRlZiBzZXRsaW5ld2lkdGh9IGRlZgovUFAge2Nsb3NlcGF0aCBlb2ZpbGx9IGJkZWYK L0RQIHtjbG9zZXBhdGggc3Ryb2tlfSBiZGVmCi9NUiB7NCAtMiByb2xsIG1vdmV0byBkdXAgIDAg ZXhjaCBybGluZXRvIGV4Y2ggMCBybGluZXRvCiAgbmVnIDAgZXhjaCBybGluZXRvIGNsb3NlcGF0 aH0gYmRlZgovRlIge01SIHN0cm9rZX0gYmRlZgovUFIge01SIGZpbGx9IGJkZWYKL0wxaSB7e2N1 cnJlbnRmaWxlIHBpY3N0ciByZWFkaGV4c3RyaW5nIHBvcH0gaW1hZ2V9IGJkZWYKL3RNYXRyaXgg bWF0cml4IGRlZgovTWFrZU92YWwge25ld3BhdGggdE1hdHJpeCBjdXJyZW50bWF0cml4IHBvcCB0 cmFuc2xhdGUgc2NhbGUKMCAwIDEgMCAzNjAgYXJjIHRNYXRyaXggc2V0bWF0cml4fSBiZGVmCi9G TyB7TWFrZU92YWwgc3Ryb2tlfSBiZGVmCi9QTyB7TWFrZU92YWwgZmlsbH0gYmRlZgovUEQge2N1 cnJlbnRsaW5ld2lkdGggMiBkaXYgMCAzNjAgYXJjIGZpbGwKICAgUERsdyAtMSBlcSBub3Qge1BE bHcgdyAvUERsdyAtMSBkZWZ9IGlmfSBkZWYKL0ZBIHtuZXdwYXRoIHRNYXRyaXggY3VycmVudG1h dHJpeCBwb3AgdHJhbnNsYXRlIHNjYWxlCiAgMCAwIDEgNSAtMiByb2xsIGFyYyB0TWF0cml4IHNl dG1hdHJpeCBzdHJva2V9IGJkZWYKL1BBIHtuZXdwYXRoIHRNYXRyaXggY3VycmVudG1hdHJpeCBw b3AJdHJhbnNsYXRlIDAgMCBtb3ZldG8gc2NhbGUKICAwIDAgMSA1IC0yIHJvbGwgYXJjIGNsb3Nl cGF0aCB0TWF0cml4IHNldG1hdHJpeCBmaWxsfSBiZGVmCi9GQW4ge25ld3BhdGggdE1hdHJpeCBj dXJyZW50bWF0cml4IHBvcCB0cmFuc2xhdGUgc2NhbGUKICAwIDAgMSA1IC0yIHJvbGwgYXJjbiB0 TWF0cml4IHNldG1hdHJpeCBzdHJva2V9IGJkZWYKL1BBbiB7bmV3cGF0aCB0TWF0cml4IGN1cnJl bnRtYXRyaXggcG9wIHRyYW5zbGF0ZSAwIDAgbW92ZXRvIHNjYWxlCiAgMCAwIDEgNSAtMiByb2xs IGFyY24gY2xvc2VwYXRoIHRNYXRyaXggc2V0bWF0cml4IGZpbGx9IGJkZWYKL3ZyYWRpdXMgMCBk ZWYgL2hyYWRpdXMgMCBkZWYgL2xyeSAwIGRlZgovbHJ4IDAgZGVmIC91bHkgMCBkZWYgL3VseCAw IGRlZiAvcmFkIDAgZGVmCi9NUlIgey92cmFkaXVzIHhkZWYgL2hyYWRpdXMgeGRlZiAvbHJ5IHhk ZWYgL2xyeCB4ZGVmIC91bHkgeGRlZgogIC91bHggeGRlZiBuZXdwYXRoIHRNYXRyaXggY3VycmVu dG1hdHJpeCBwb3AgdWx4IGhyYWRpdXMgYWRkIHVseQogIHZyYWRpdXMgYWRkIHRyYW5zbGF0ZSBo cmFkaXVzIHZyYWRpdXMgc2NhbGUgMCAwIDEgMTgwIDI3MCBhcmMgCiAgdE1hdHJpeCBzZXRtYXRy aXggbHJ4IGhyYWRpdXMgc3ViIHVseSB2cmFkaXVzIGFkZCB0cmFuc2xhdGUKICBocmFkaXVzIHZy YWRpdXMgc2NhbGUgMCAwIDEgMjcwIDM2MCBhcmMgdE1hdHJpeCBzZXRtYXRyaXgKICBscnggaHJh ZGl1cyBzdWIgbHJ5IHZyYWRpdXMgc3ViIHRyYW5zbGF0ZSBocmFkaXVzIHZyYWRpdXMgc2NhbGUK ICAwIDAgMSAwIDkwIGFyYyB0TWF0cml4IHNldG1hdHJpeCB1bHggaHJhZGl1cyBhZGQgbHJ5IHZy YWRpdXMgc3ViCiAgdHJhbnNsYXRlIGhyYWRpdXMgdnJhZGl1cyBzY2FsZSAwIDAgMSA5MCAxODAg YXJjIHRNYXRyaXggc2V0bWF0cml4CiAgY2xvc2VwYXRofSBiZGVmCi9GUlIge01SUiBzdHJva2Ug fSBiZGVmCi9QUlIge01SUiBmaWxsIH0gYmRlZgovTWxyUlIgey9scnkgeGRlZiAvbHJ4IHhkZWYg L3VseSB4ZGVmIC91bHggeGRlZiAvcmFkIGxyeSB1bHkgc3ViIDIgZGl2IGRlZgogIG5ld3BhdGgg dE1hdHJpeCBjdXJyZW50bWF0cml4IHBvcCB1bHggcmFkIGFkZCB1bHkgcmFkIGFkZCB0cmFuc2xh dGUKICByYWQgcmFkIHNjYWxlIDAgMCAxIDkwIDI3MCBhcmMgdE1hdHJpeCBzZXRtYXRyaXggbHJ4 IHJhZCBzdWIgbHJ5IHJhZAogIHN1YiB0cmFuc2xhdGUgcmFkIHJhZCBzY2FsZSAwIDAgMSAyNzAg OTAgYXJjIHRNYXRyaXggc2V0bWF0cml4CiAgY2xvc2VwYXRofSBiZGVmCi9GbHJSUiB7TWxyUlIg c3Ryb2tlIH0gYmRlZgovUGxyUlIge01sclJSIGZpbGwgfSBiZGVmCi9NdGJSUiB7L2xyeSB4ZGVm IC9scnggeGRlZiAvdWx5IHhkZWYgL3VseCB4ZGVmIC9yYWQgbHJ4IHVseCBzdWIgMiBkaXYgZGVm CiAgbmV3cGF0aCB0TWF0cml4IGN1cnJlbnRtYXRyaXggcG9wIHVseCByYWQgYWRkIHVseSByYWQg YWRkIHRyYW5zbGF0ZQogIHJhZCByYWQgc2NhbGUgMCAwIDEgMTgwIDM2MCBhcmMgdE1hdHJpeCBz ZXRtYXRyaXggbHJ4IHJhZCBzdWIgbHJ5IHJhZAogIHN1YiB0cmFuc2xhdGUgcmFkIHJhZCBzY2Fs ZSAwIDAgMSAwIDE4MCBhcmMgdE1hdHJpeCBzZXRtYXRyaXgKICBjbG9zZXBhdGh9IGJkZWYKL0Z0 YlJSIHtNdGJSUiBzdHJva2UgfSBiZGVmCi9QdGJSUiB7TXRiUlIgZmlsbCB9IGJkZWYKL3N0cmkg NiBhcnJheSBkZWYgL2R0cmkgNiBhcnJheSBkZWYKL3NtYXQgNiBhcnJheSBkZWYgL2RtYXQgNiBh cnJheSBkZWYKL3RtYXQxIDYgYXJyYXkgZGVmIC90bWF0MiA2IGFycmF5IGRlZiAvZGlmIDMgYXJy YXkgZGVmCi9hc3ViIHsvaW5kMiBleGNoIGRlZiAvaW5kMSBleGNoIGRlZiBkdXAgZHVwCiAgaW5k MSBnZXQgZXhjaCBpbmQyIGdldCBzdWIgZXhjaCB9IGJkZWYKL3RyaV90b19tYXRyaXggewogIDIg MCBhc3ViIDMgMSBhc3ViIDQgMCBhc3ViIDUgMSBhc3ViCiAgZHVwIDAgZ2V0IGV4Y2ggMSBnZXQg NyAtMSByb2xsIGFzdG9yZSB9IGJkZWYKL2NvbXB1dGVfdHJhbnNmb3JtIHsKICBkbWF0IGR0cmkg dHJpX3RvX21hdHJpeCB0bWF0MSBpbnZlcnRtYXRyaXggCiAgc21hdCBzdHJpIHRyaV90b19tYXRy aXggdG1hdDIgY29uY2F0bWF0cml4IH0gYmRlZgovZHMge3N0cmkgYXN0b3JlIHBvcH0gYmRlZgov ZHQge2R0cmkgYXN0b3JlIHBvcH0gYmRlZgovZGIgezIgY29weSAvY29scyB4ZGVmIC9yb3dzIHhk ZWYgbXVsIGR1cCAzIG11bCBzdHJpbmcKICBjdXJyZW50ZmlsZSAKICAzIGluZGV4IDAgZXEgey9B U0NJSUhleERlY29kZSBmaWx0ZXJ9CiAgey9BU0NJSTg1RGVjb2RlIGZpbHRlciAzIGluZGV4IDIg ZXEgey9SdW5MZW5ndGhEZWNvZGUgZmlsdGVyfSBpZiB9CiAgaWZlbHNlIGV4Y2ggcmVhZHN0cmlu ZyBwb3AKICBkdXAgMCAzIGluZGV4IGdldGludGVydmFsIC9yYm1hcCB4ZGVmCiAgZHVwIDIgaW5k ZXggZHVwIGdldGludGVydmFsIC9nYm1hcCB4ZGVmCiAgMSBpbmRleCBkdXAgMiBtdWwgZXhjaCBn ZXRpbnRlcnZhbCAvYmJtYXAgeGRlZiBwb3AgcG9wfWJkZWYKL2l0IHtncyBucCBkdHJpIGFsb2Fk IHBvcCBtb3ZldG8gbGluZXRvIGxpbmV0byBjcCBjCiAgY29scyByb3dzIDggY29tcHV0ZV90cmFu c2Zvcm0gCiAgcmJtYXAgZ2JtYXAgYmJtYXAgdHJ1ZSAzIGNvbG9yaW1hZ2UgZ3J9YmRlZgovaWwg e25ld3BhdGggbW92ZXRvIGxpbmV0byBzdHJva2V9YmRlZgpjdXJyZW50ZGljdCBlbmQgZGVmCiUl RW5kUHJvbG9nCgolJUJlZ2luU2V0dXAKTWF0aFdvcmtzIGJlZ2luCgowIGNhcAoKZW5kCiUlRW5k U2V0dXAKCiUlUGFnZTogMSAxCiUlQmVnaW5QYWdlU2V0dXAKJSVQYWdlQm91bmRpbmdCb3g6ICAt MTA0ICAgMTI1ICAgNzE3ICAgNjY3Ck1hdGhXb3JrcyBiZWdpbgpicGFnZQolJUVuZFBhZ2VTZXR1 cAoKJSVCZWdpbk9iamVjdDogb2JqMQpicGxvdAoKL2RwaTJwb2ludCAxMiBkZWYKcG9ydHJhaXRN b2RlIC0xMjQ4IDgwMDQgY3NtCgogICAgMCAgICAgMCAgOTg1MiAgNjQ5NSByYwo4NSBkaWN0IGJl Z2luICVDb2xvcnRhYmxlIGRpY3Rpb25hcnkKL2MwIHsgMC4wMDAwMDAgMC4wMDAwMDAgMC4wMDAw MDAgc3J9IGJkZWYKL2MxIHsgMS4wMDAwMDAgMS4wMDAwMDAgMS4wMDAwMDAgc3J9IGJkZWYKL2My IHsgMC45MDAwMDAgMC4wMDAwMDAgMC4wMDAwMDAgc3J9IGJkZWYKL2MzIHsgMC4wMDAwMDAgMC44 MjAwMDAgMC4wMDAwMDAgc3J9IGJkZWYKL2M0IHsgMC4wMDAwMDAgMC4wMDAwMDAgMC44MDAwMDAg c3J9IGJkZWYKL2M1IHsgMC45MTAwMDAgMC44MjAwMDAgMC4zMjAwMDAgc3J9IGJkZWYKL2M2IHsg MS4wMDAwMDAgMC4yNjAwMDAgMC44MjAwMDAgc3J9IGJkZWYKL2M3IHsgMC4wMDAwMDAgMC44MjAw MDAgMC44MjAwMDAgc3J9IGJkZWYKYzAKMSBqCjEgc2cKICAgMCAgICAwIDk4NTMgNjQ5NiByZgo2 IHcKMCA1Mjk0IDc2MzUgMCAwIC01Mjk0IDEyODEgNTc4MSA0IE1QClBQCi03NjM1IDAgMCA1Mjk0 IDc2MzUgMCAwIC01Mjk0IDEyODEgNTc4MSA1IE1QIHN0cm9rZQo0IHcKRE8KMCBzZwoxMjgxIDU3 ODEgbXQgMTI4MSAgNDg3IEwKMTI4MSAgNDg3IG10IDEyODEgIDQ4NyBMCjI1NTMgNTc4MSBtdCAy NTUzICA0ODcgTAoyNTUzICA0ODcgbXQgMjU1MyAgNDg3IEwKMzgyNiA1NzgxIG10IDM4MjYgIDQ4 NyBMCjM4MjYgIDQ4NyBtdCAzODI2ICA0ODcgTAo1MDk4IDU3ODEgbXQgNTA5OCAgNDg3IEwKNTA5 OCAgNDg3IG10IDUwOTggIDQ4NyBMCjYzNzEgNTc4MSBtdCA2MzcxICA0ODcgTAo2MzcxICA0ODcg bXQgNjM3MSAgNDg3IEwKNzY0MyA1NzgxIG10IDc2NDMgIDQ4NyBMCjc2NDMgIDQ4NyBtdCA3NjQz ICA0ODcgTAo4OTE2IDU3ODEgbXQgODkxNiAgNDg3IEwKODkxNiAgNDg3IG10IDg5MTYgIDQ4NyBM CjEyODEgNTc4MSBtdCA4OTE2IDU3ODEgTAo4OTE2IDU3ODEgbXQgODkxNiA1NzgxIEwKMTI4MSA1 MjUxIG10IDg5MTYgNTI1MSBMCjg5MTYgNTI1MSBtdCA4OTE2IDUyNTEgTAoxMjgxIDQ3MjIgbXQg ODkxNiA0NzIyIEwKODkxNiA0NzIyIG10IDg5MTYgNDcyMiBMCjEyODEgNDE5MiBtdCA4OTE2IDQx OTIgTAo4OTE2IDQxOTIgbXQgODkxNiA0MTkyIEwKMTI4MSAzNjYzIG10IDg5MTYgMzY2MyBMCjg5 MTYgMzY2MyBtdCA4OTE2IDM2NjMgTAoxMjgxIDMxMzQgbXQgODkxNiAzMTM0IEwKODkxNiAzMTM0 IG10IDg5MTYgMzEzNCBMCjEyODEgMjYwNCBtdCA4OTE2IDI2MDQgTAo4OTE2IDI2MDQgbXQgODkx NiAyNjA0IEwKMTI4MSAyMDc1IG10IDg5MTYgMjA3NSBMCjg5MTYgMjA3NSBtdCA4OTE2IDIwNzUg TAoxMjgxIDE1NDUgbXQgODkxNiAxNTQ1IEwKODkxNiAxNTQ1IG10IDg5MTYgMTU0NSBMCjEyODEg MTAxNiBtdCA4OTE2IDEwMTYgTAo4OTE2IDEwMTYgbXQgODkxNiAxMDE2IEwKMTI4MSAgNDg3IG10 IDg5MTYgIDQ4NyBMCjg5MTYgIDQ4NyBtdCA4OTE2ICA0ODcgTApTTwo2IHcKMTI4MSA1NzgxIG10 IDg5MTYgNTc4MSBMCjEyODEgIDQ4NyBtdCA4OTE2ICA0ODcgTAoxMjgxIDU3ODEgbXQgMTI4MSAg NDg3IEwKODkxNiA1NzgxIG10IDg5MTYgIDQ4NyBMCjEyODEgNTc4MSBtdCA4OTE2IDU3ODEgTAox MjgxIDU3ODEgbXQgMTI4MSAgNDg3IEwKMTI4MSA1NzgxIG10IDEyODEgNTc0MiBMCjEyODEgIDQ4 NyBtdCAxMjgxICA1MjUgTApETwoxMjgxIDU3ODEgbXQgMTI4MSAgNDg3IEwKMTI4MSAgNDg3IG10 IDEyODEgIDQ4NyBMClNPCjEyODEgNTc4MSBtdCAxMjgxIDU3MDQgTAoxMjgxICA0ODcgbXQgMTI4 MSAgNTYzIEwKJSVJbmNsdWRlUmVzb3VyY2U6IGZvbnQgSGVsdmV0aWNhCi9IZWx2ZXRpY2EgL0lT T0xhdGluMUVuY29kaW5nIDEyMCBGTVNSCgoxMTkzIDU5NjMgbXQgCigxMCkgcwolJUluY2x1ZGVS ZXNvdXJjZTogZm9udCBIZWx2ZXRpY2EKL0hlbHZldGljYSAvSVNPTGF0aW4xRW5jb2RpbmcgODAg Rk1TUgoKMTMyNiA1ODg5IG10IAooMCkgcwoxNjY0IDU3ODEgbXQgMTY2NCA1NzQyIEwKMTY2NCAg NDg3IG10IDE2NjQgIDUyNSBMCkRPCjE2NjQgNTc4MSBtdCAxNjY0ICA0ODcgTAoxNjY0ICA0ODcg bXQgMTY2NCAgNDg3IEwKU08KMTg4OCA1NzgxIG10IDE4ODggNTc0MiBMCjE4ODggIDQ4NyBtdCAx ODg4ICA1MjUgTApETwoxODg4IDU3ODEgbXQgMTg4OCAgNDg3IEwKMTg4OCAgNDg3IG10IDE4ODgg IDQ4NyBMClNPCjIwNDcgNTc4MSBtdCAyMDQ3IDU3NDIgTAoyMDQ3ICA0ODcgbXQgMjA0NyAgNTI1 IEwKRE8KMjA0NyA1NzgxIG10IDIwNDcgIDQ4NyBMCjIwNDcgIDQ4NyBtdCAyMDQ3ICA0ODcgTApT TwoyMTcwIDU3ODEgbXQgMjE3MCA1NzQyIEwKMjE3MCAgNDg3IG10IDIxNzAgIDUyNSBMCkRPCjIx NzAgNTc4MSBtdCAyMTcwICA0ODcgTAoyMTcwICA0ODcgbXQgMjE3MCAgNDg3IEwKU08KMjI3MSA1 NzgxIG10IDIyNzEgNTc0MiBMCjIyNzEgIDQ4NyBtdCAyMjcxICA1MjUgTApETwoyMjcxIDU3ODEg bXQgMjI3MSAgNDg3IEwKMjI3MSAgNDg3IG10IDIyNzEgIDQ4NyBMClNPCjIzNTYgNTc4MSBtdCAy MzU2IDU3NDIgTAoyMzU2ICA0ODcgbXQgMjM1NiAgNTI1IEwKRE8KMjM1NiA1NzgxIG10IDIzNTYg IDQ4NyBMCjIzNTYgIDQ4NyBtdCAyMzU2ICA0ODcgTApTTwoyNDMwIDU3ODEgbXQgMjQzMCA1NzQy IEwKMjQzMCAgNDg3IG10IDI0MzAgIDUyNSBMCkRPCjI0MzAgNTc4MSBtdCAyNDMwICA0ODcgTAoy NDMwICA0ODcgbXQgMjQzMCAgNDg3IEwKU08KMjQ5NSA1NzgxIG10IDI0OTUgNTc0MiBMCjI0OTUg IDQ4NyBtdCAyNDk1ICA1MjUgTApETwoyNDk1IDU3ODEgbXQgMjQ5NSAgNDg3IEwKMjQ5NSAgNDg3 IG10IDI0OTUgIDQ4NyBMClNPCjI1NTMgNTc4MSBtdCAyNTUzIDU3NDIgTAoyNTUzICA0ODcgbXQg MjU1MyAgNTI1IEwKRE8KMjU1MyA1NzgxIG10IDI1NTMgIDQ4NyBMCjI1NTMgIDQ4NyBtdCAyNTUz ICA0ODcgTApTTwoyNTUzIDU3ODEgbXQgMjU1MyA1NzA0IEwKMjU1MyAgNDg3IG10IDI1NTMgIDU2 MyBMCiUlSW5jbHVkZVJlc291cmNlOiBmb250IEhlbHZldGljYQovSGVsdmV0aWNhIC9JU09MYXRp bjFFbmNvZGluZyAxMjAgRk1TUgoKMjQ2NSA1OTYzIG10IAooMTApIHMKJSVJbmNsdWRlUmVzb3Vy Y2U6IGZvbnQgSGVsdmV0aWNhCi9IZWx2ZXRpY2EgL0lTT0xhdGluMUVuY29kaW5nIDgwIEZNU1IK CjI1OTggNTg4OSBtdCAKKDEpIHMKMjkzNiA1NzgxIG10IDI5MzYgNTc0MiBMCjI5MzYgIDQ4NyBt dCAyOTM2ICA1MjUgTApETwoyOTM2IDU3ODEgbXQgMjkzNiAgNDg3IEwKMjkzNiAgNDg3IG10IDI5 MzYgIDQ4NyBMClNPCjMxNjAgNTc4MSBtdCAzMTYwIDU3NDIgTAozMTYwICA0ODcgbXQgMzE2MCAg NTI1IEwKRE8KMzE2MCA1NzgxIG10IDMxNjAgIDQ4NyBMCjMxNjAgIDQ4NyBtdCAzMTYwICA0ODcg TApTTwozMzE5IDU3ODEgbXQgMzMxOSA1NzQyIEwKMzMxOSAgNDg3IG10IDMzMTkgIDUyNSBMCkRP CjMzMTkgNTc4MSBtdCAzMzE5ICA0ODcgTAozMzE5ICA0ODcgbXQgMzMxOSAgNDg3IEwKU08KMzQ0 MiA1NzgxIG10IDM0NDIgNTc0MiBMCjM0NDIgIDQ4NyBtdCAzNDQyICA1MjUgTApETwozNDQyIDU3 ODEgbXQgMzQ0MiAgNDg3IEwKMzQ0MiAgNDg3IG10IDM0NDIgIDQ4NyBMClNPCjM1NDMgNTc4MSBt dCAzNTQzIDU3NDIgTAozNTQzICA0ODcgbXQgMzU0MyAgNTI1IEwKRE8KMzU0MyA1NzgxIG10IDM1 NDMgIDQ4NyBMCjM1NDMgIDQ4NyBtdCAzNTQzICA0ODcgTApTTwozNjI4IDU3ODEgbXQgMzYyOCA1 NzQyIEwKMzYyOCAgNDg3IG10IDM2MjggIDUyNSBMCkRPCjM2MjggNTc4MSBtdCAzNjI4ICA0ODcg TAozNjI4ICA0ODcgbXQgMzYyOCAgNDg3IEwKU08KMzcwMiA1NzgxIG10IDM3MDIgNTc0MiBMCjM3 MDIgIDQ4NyBtdCAzNzAyICA1MjUgTApETwozNzAyIDU3ODEgbXQgMzcwMiAgNDg3IEwKMzcwMiAg NDg3IG10IDM3MDIgIDQ4NyBMClNPCjM3NjcgNTc4MSBtdCAzNzY3IDU3NDIgTAozNzY3ICA0ODcg bXQgMzc2NyAgNTI1IEwKRE8KMzc2NyA1NzgxIG10IDM3NjcgIDQ4NyBMCjM3NjcgIDQ4NyBtdCAz NzY3ICA0ODcgTApTTwozODI2IDU3ODEgbXQgMzgyNiA1NzQyIEwKMzgyNiAgNDg3IG10IDM4MjYg IDUyNSBMCkRPCjM4MjYgNTc4MSBtdCAzODI2ICA0ODcgTAozODI2ICA0ODcgbXQgMzgyNiAgNDg3 IEwKU08KMzgyNiA1NzgxIG10IDM4MjYgNTcwNCBMCjM4MjYgIDQ4NyBtdCAzODI2ICA1NjMgTAol JUluY2x1ZGVSZXNvdXJjZTogZm9udCBIZWx2ZXRpY2EKL0hlbHZldGljYSAvSVNPTGF0aW4xRW5j b2RpbmcgMTIwIEZNU1IKCjM3MzggNTk2MyBtdCAKKDEwKSBzCiUlSW5jbHVkZVJlc291cmNlOiBm b250IEhlbHZldGljYQovSGVsdmV0aWNhIC9JU09MYXRpbjFFbmNvZGluZyA4MCBGTVNSCgozODcx IDU4ODkgbXQgCigyKSBzCjQyMDkgNTc4MSBtdCA0MjA5IDU3NDIgTAo0MjA5ICA0ODcgbXQgNDIw OSAgNTI1IEwKRE8KNDIwOSA1NzgxIG10IDQyMDkgIDQ4NyBMCjQyMDkgIDQ4NyBtdCA0MjA5ICA0 ODcgTApTTwo0NDMzIDU3ODEgbXQgNDQzMyA1NzQyIEwKNDQzMyAgNDg3IG10IDQ0MzMgIDUyNSBM CkRPCjQ0MzMgNTc4MSBtdCA0NDMzICA0ODcgTAo0NDMzICA0ODcgbXQgNDQzMyAgNDg3IEwKU08K NDU5MiA1NzgxIG10IDQ1OTIgNTc0MiBMCjQ1OTIgIDQ4NyBtdCA0NTkyICA1MjUgTApETwo0NTky IDU3ODEgbXQgNDU5MiAgNDg3IEwKNDU5MiAgNDg3IG10IDQ1OTIgIDQ4NyBMClNPCjQ3MTUgNTc4 MSBtdCA0NzE1IDU3NDIgTAo0NzE1ICA0ODcgbXQgNDcxNSAgNTI1IEwKRE8KNDcxNSA1NzgxIG10 IDQ3MTUgIDQ4NyBMCjQ3MTUgIDQ4NyBtdCA0NzE1ICA0ODcgTApTTwo0ODE2IDU3ODEgbXQgNDgx NiA1NzQyIEwKNDgxNiAgNDg3IG10IDQ4MTYgIDUyNSBMCkRPCjQ4MTYgNTc4MSBtdCA0ODE2ICA0 ODcgTAo0ODE2ICA0ODcgbXQgNDgxNiAgNDg3IEwKU08KNDkwMSA1NzgxIG10IDQ5MDEgNTc0MiBM CjQ5MDEgIDQ4NyBtdCA0OTAxICA1MjUgTApETwo0OTAxIDU3ODEgbXQgNDkwMSAgNDg3IEwKNDkw MSAgNDg3IG10IDQ5MDEgIDQ4NyBMClNPCjQ5NzUgNTc4MSBtdCA0OTc1IDU3NDIgTAo0OTc1ICA0 ODcgbXQgNDk3NSAgNTI1IEwKRE8KNDk3NSA1NzgxIG10IDQ5NzUgIDQ4NyBMCjQ5NzUgIDQ4NyBt dCA0OTc1ICA0ODcgTApTTwo1MDQwIDU3ODEgbXQgNTA0MCA1NzQyIEwKNTA0MCAgNDg3IG10IDUw NDAgIDUyNSBMCkRPCjUwNDAgNTc4MSBtdCA1MDQwICA0ODcgTAo1MDQwICA0ODcgbXQgNTA0MCAg NDg3IEwKU08KNTA5OCA1NzgxIG10IDUwOTggNTc0MiBMCjUwOTggIDQ4NyBtdCA1MDk4ICA1MjUg TApETwo1MDk4IDU3ODEgbXQgNTA5OCAgNDg3IEwKNTA5OCAgNDg3IG10IDUwOTggIDQ4NyBMClNP CjUwOTggNTc4MSBtdCA1MDk4IDU3MDQgTAo1MDk4ICA0ODcgbXQgNTA5OCAgNTYzIEwKJSVJbmNs dWRlUmVzb3VyY2U6IGZvbnQgSGVsdmV0aWNhCi9IZWx2ZXRpY2EgL0lTT0xhdGluMUVuY29kaW5n IDEyMCBGTVNSCgo1MDEwIDU5NjMgbXQgCigxMCkgcwolJUluY2x1ZGVSZXNvdXJjZTogZm9udCBI ZWx2ZXRpY2EKL0hlbHZldGljYSAvSVNPTGF0aW4xRW5jb2RpbmcgODAgRk1TUgoKNTE0MyA1ODg5 IG10IAooMykgcwo1NDgxIDU3ODEgbXQgNTQ4MSA1NzQyIEwKNTQ4MSAgNDg3IG10IDU0ODEgIDUy NSBMCkRPCjU0ODEgNTc4MSBtdCA1NDgxICA0ODcgTAo1NDgxICA0ODcgbXQgNTQ4MSAgNDg3IEwK U08KNTcwNSA1NzgxIG10IDU3MDUgNTc0MiBMCjU3MDUgIDQ4NyBtdCA1NzA1ICA1MjUgTApETwo1 NzA1IDU3ODEgbXQgNTcwNSAgNDg3IEwKNTcwNSAgNDg3IG10IDU3MDUgIDQ4NyBMClNPCjU4NjQg NTc4MSBtdCA1ODY0IDU3NDIgTAo1ODY0ICA0ODcgbXQgNTg2NCAgNTI1IEwKRE8KNTg2NCA1Nzgx IG10IDU4NjQgIDQ4NyBMCjU4NjQgIDQ4NyBtdCA1ODY0ICA0ODcgTApTTwo1OTg3IDU3ODEgbXQg NTk4NyA1NzQyIEwKNTk4NyAgNDg3IG10IDU5ODcgIDUyNSBMCkRPCjU5ODcgNTc4MSBtdCA1OTg3 ICA0ODcgTAo1OTg3ICA0ODcgbXQgNTk4NyAgNDg3IEwKU08KNjA4OCA1NzgxIG10IDYwODggNTc0 MiBMCjYwODggIDQ4NyBtdCA2MDg4ICA1MjUgTApETwo2MDg4IDU3ODEgbXQgNjA4OCAgNDg3IEwK NjA4OCAgNDg3IG10IDYwODggIDQ4NyBMClNPCjYxNzMgNTc4MSBtdCA2MTczIDU3NDIgTAo2MTcz ICA0ODcgbXQgNjE3MyAgNTI1IEwKRE8KNjE3MyA1NzgxIG10IDYxNzMgIDQ4NyBMCjYxNzMgIDQ4 NyBtdCA2MTczICA0ODcgTApTTwo2MjQ3IDU3ODEgbXQgNjI0NyA1NzQyIEwKNjI0NyAgNDg3IG10 IDYyNDcgIDUyNSBMCkRPCjYyNDcgNTc4MSBtdCA2MjQ3ICA0ODcgTAo2MjQ3ICA0ODcgbXQgNjI0 NyAgNDg3IEwKU08KNjMxMiA1NzgxIG10IDYzMTIgNTc0MiBMCjYzMTIgIDQ4NyBtdCA2MzEyICA1 MjUgTApETwo2MzEyIDU3ODEgbXQgNjMxMiAgNDg3IEwKNjMxMiAgNDg3IG10IDYzMTIgIDQ4NyBM ClNPCjYzNzEgNTc4MSBtdCA2MzcxIDU3NDIgTAo2MzcxICA0ODcgbXQgNjM3MSAgNTI1IEwKRE8K NjM3MSA1NzgxIG10IDYzNzEgIDQ4NyBMCjYzNzEgIDQ4NyBtdCA2MzcxICA0ODcgTApTTwo2Mzcx IDU3ODEgbXQgNjM3MSA1NzA0IEwKNjM3MSAgNDg3IG10IDYzNzEgIDU2MyBMCiUlSW5jbHVkZVJl c291cmNlOiBmb250IEhlbHZldGljYQovSGVsdmV0aWNhIC9JU09MYXRpbjFFbmNvZGluZyAxMjAg Rk1TUgoKNjI4MyA1OTYzIG10IAooMTApIHMKJSVJbmNsdWRlUmVzb3VyY2U6IGZvbnQgSGVsdmV0 aWNhCi9IZWx2ZXRpY2EgL0lTT0xhdGluMUVuY29kaW5nIDgwIEZNU1IKCjY0MTYgNTg4OSBtdCAK KDQpIHMKNjc1NCA1NzgxIG10IDY3NTQgNTc0MiBMCjY3NTQgIDQ4NyBtdCA2NzU0ICA1MjUgTApE Two2NzU0IDU3ODEgbXQgNjc1NCAgNDg3IEwKNjc1NCAgNDg3IG10IDY3NTQgIDQ4NyBMClNPCjY5 NzggNTc4MSBtdCA2OTc4IDU3NDIgTAo2OTc4ICA0ODcgbXQgNjk3OCAgNTI1IEwKRE8KNjk3OCA1 NzgxIG10IDY5NzggIDQ4NyBMCjY5NzggIDQ4NyBtdCA2OTc4ICA0ODcgTApTTwo3MTM3IDU3ODEg bXQgNzEzNyA1NzQyIEwKNzEzNyAgNDg3IG10IDcxMzcgIDUyNSBMCkRPCjcxMzcgNTc4MSBtdCA3 MTM3ICA0ODcgTAo3MTM3ICA0ODcgbXQgNzEzNyAgNDg3IEwKU08KNzI2MCA1NzgxIG10IDcyNjAg NTc0MiBMCjcyNjAgIDQ4NyBtdCA3MjYwICA1MjUgTApETwo3MjYwIDU3ODEgbXQgNzI2MCAgNDg3 IEwKNzI2MCAgNDg3IG10IDcyNjAgIDQ4NyBMClNPCjczNjEgNTc4MSBtdCA3MzYxIDU3NDIgTAo3 MzYxICA0ODcgbXQgNzM2MSAgNTI1IEwKRE8KNzM2MSA1NzgxIG10IDczNjEgIDQ4NyBMCjczNjEg IDQ4NyBtdCA3MzYxICA0ODcgTApTTwo3NDQ2IDU3ODEgbXQgNzQ0NiA1NzQyIEwKNzQ0NiAgNDg3 IG10IDc0NDYgIDUyNSBMCkRPCjc0NDYgNTc4MSBtdCA3NDQ2ICA0ODcgTAo3NDQ2ICA0ODcgbXQg NzQ0NiAgNDg3IEwKU08KNzUyMCA1NzgxIG10IDc1MjAgNTc0MiBMCjc1MjAgIDQ4NyBtdCA3NTIw ICA1MjUgTApETwo3NTIwIDU3ODEgbXQgNzUyMCAgNDg3IEwKNzUyMCAgNDg3IG10IDc1MjAgIDQ4 NyBMClNPCjc1ODUgNTc4MSBtdCA3NTg1IDU3NDIgTAo3NTg1ICA0ODcgbXQgNzU4NSAgNTI1IEwK RE8KNzU4NSA1NzgxIG10IDc1ODUgIDQ4NyBMCjc1ODUgIDQ4NyBtdCA3NTg1ICA0ODcgTApTTwo3 NjQzIDU3ODEgbXQgNzY0MyA1NzQyIEwKNzY0MyAgNDg3IG10IDc2NDMgIDUyNSBMCkRPCjc2NDMg NTc4MSBtdCA3NjQzICA0ODcgTAo3NjQzICA0ODcgbXQgNzY0MyAgNDg3IEwKU08KNzY0MyA1Nzgx IG10IDc2NDMgNTcwNCBMCjc2NDMgIDQ4NyBtdCA3NjQzICA1NjMgTAolJUluY2x1ZGVSZXNvdXJj ZTogZm9udCBIZWx2ZXRpY2EKL0hlbHZldGljYSAvSVNPTGF0aW4xRW5jb2RpbmcgMTIwIEZNU1IK Cjc1NTUgNTk2MyBtdCAKKDEwKSBzCiUlSW5jbHVkZVJlc291cmNlOiBmb250IEhlbHZldGljYQov SGVsdmV0aWNhIC9JU09MYXRpbjFFbmNvZGluZyA4MCBGTVNSCgo3Njg4IDU4ODkgbXQgCig1KSBz CjgwMjYgNTc4MSBtdCA4MDI2IDU3NDIgTAo4MDI2ICA0ODcgbXQgODAyNiAgNTI1IEwKRE8KODAy NiA1NzgxIG10IDgwMjYgIDQ4NyBMCjgwMjYgIDQ4NyBtdCA4MDI2ICA0ODcgTApTTwo4MjUwIDU3 ODEgbXQgODI1MCA1NzQyIEwKODI1MCAgNDg3IG10IDgyNTAgIDUyNSBMCkRPCjgyNTAgNTc4MSBt dCA4MjUwICA0ODcgTAo4MjUwICA0ODcgbXQgODI1MCAgNDg3IEwKU08KODQwOSA1NzgxIG10IDg0 MDkgNTc0MiBMCjg0MDkgIDQ4NyBtdCA4NDA5ICA1MjUgTApETwo4NDA5IDU3ODEgbXQgODQwOSAg NDg3IEwKODQwOSAgNDg3IG10IDg0MDkgIDQ4NyBMClNPCjg1MzIgNTc4MSBtdCA4NTMyIDU3NDIg TAo4NTMyICA0ODcgbXQgODUzMiAgNTI1IEwKRE8KODUzMiA1NzgxIG10IDg1MzIgIDQ4NyBMCjg1 MzIgIDQ4NyBtdCA4NTMyICA0ODcgTApTTwo4NjMzIDU3ODEgbXQgODYzMyA1NzQyIEwKODYzMyAg NDg3IG10IDg2MzMgIDUyNSBMCkRPCjg2MzMgNTc4MSBtdCA4NjMzICA0ODcgTAo4NjMzICA0ODcg bXQgODYzMyAgNDg3IEwKU08KODcxOCA1NzgxIG10IDg3MTggNTc0MiBMCjg3MTggIDQ4NyBtdCA4 NzE4ICA1MjUgTApETwo4NzE4IDU3ODEgbXQgODcxOCAgNDg3IEwKODcxOCAgNDg3IG10IDg3MTgg IDQ4NyBMClNPCjg3OTIgNTc4MSBtdCA4NzkyIDU3NDIgTAo4NzkyICA0ODcgbXQgODc5MiAgNTI1 IEwKRE8KODc5MiA1NzgxIG10IDg3OTIgIDQ4NyBMCjg3OTIgIDQ4NyBtdCA4NzkyICA0ODcgTApT Two4ODU3IDU3ODEgbXQgODg1NyA1NzQyIEwKODg1NyAgNDg3IG10IDg4NTcgIDUyNSBMCkRPCjg4 NTcgNTc4MSBtdCA4ODU3ICA0ODcgTAo4ODU3ICA0ODcgbXQgODg1NyAgNDg3IEwKU08KODkxNiA1 NzgxIG10IDg5MTYgNTc0MiBMCjg5MTYgIDQ4NyBtdCA4OTE2ICA1MjUgTApETwo4OTE2IDU3ODEg bXQgODkxNiAgNDg3IEwKODkxNiAgNDg3IG10IDg5MTYgIDQ4NyBMClNPCjg5MTYgNTc4MSBtdCA4 OTE2IDU3MDQgTAo4OTE2ICA0ODcgbXQgODkxNiAgNTYzIEwKJSVJbmNsdWRlUmVzb3VyY2U6IGZv bnQgSGVsdmV0aWNhCi9IZWx2ZXRpY2EgL0lTT0xhdGluMUVuY29kaW5nIDEyMCBGTVNSCgo4ODI4 IDU5NjMgbXQgCigxMCkgcwolJUluY2x1ZGVSZXNvdXJjZTogZm9udCBIZWx2ZXRpY2EKL0hlbHZl dGljYSAvSVNPTGF0aW4xRW5jb2RpbmcgODAgRk1TUgoKODk2MSA1ODg5IG10IAooNikgcwoxMjgx IDU3ODEgbXQgMTM1NyA1NzgxIEwKODkxNiA1NzgxIG10IDg4MzkgNTc4MSBMCiUlSW5jbHVkZVJl c291cmNlOiBmb250IEhlbHZldGljYQovSGVsdmV0aWNhIC9JU09MYXRpbjFFbmNvZGluZyAxMjAg Rk1TUgoKMTE4MCA1ODI1IG10IAooMCkgcwoxMjgxIDUyNTEgbXQgMTM1NyA1MjUxIEwKODkxNiA1 MjUxIG10IDg4MzkgNTI1MSBMCjEwODAgNTI5NSBtdCAKKDAuMSkgcwoxMjgxIDQ3MjIgbXQgMTM1 NyA0NzIyIEwKODkxNiA0NzIyIG10IDg4MzkgNDcyMiBMCjEwODAgNDc2NiBtdCAKKDAuMikgcwox MjgxIDQxOTIgbXQgMTM1NyA0MTkyIEwKODkxNiA0MTkyIG10IDg4MzkgNDE5MiBMCjEwODAgNDIz NiBtdCAKKDAuMykgcwoxMjgxIDM2NjMgbXQgMTM1NyAzNjYzIEwKODkxNiAzNjYzIG10IDg4Mzkg MzY2MyBMCjEwODAgMzcwNyBtdCAKKDAuNCkgcwoxMjgxIDMxMzQgbXQgMTM1NyAzMTM0IEwKODkx NiAzMTM0IG10IDg4MzkgMzEzNCBMCjEwODAgMzE3OCBtdCAKKDAuNSkgcwoxMjgxIDI2MDQgbXQg MTM1NyAyNjA0IEwKODkxNiAyNjA0IG10IDg4MzkgMjYwNCBMCjEwODAgMjY0OCBtdCAKKDAuNikg cwoxMjgxIDIwNzUgbXQgMTM1NyAyMDc1IEwKODkxNiAyMDc1IG10IDg4MzkgMjA3NSBMCjEwODAg MjExOSBtdCAKKDAuNykgcwoxMjgxIDE1NDUgbXQgMTM1NyAxNTQ1IEwKODkxNiAxNTQ1IG10IDg4 MzkgMTU0NSBMCjEwODAgMTU4OSBtdCAKKDAuOCkgcwoxMjgxIDEwMTYgbXQgMTM1NyAxMDE2IEwK ODkxNiAxMDE2IG10IDg4MzkgMTAxNiBMCjEwODAgMTA2MCBtdCAKKDAuOSkgcwoxMjgxICA0ODcg bXQgMTM1NyAgNDg3IEwKODkxNiAgNDg3IG10IDg4MzkgIDQ4NyBMCjExODAgIDUzMSBtdCAKKDEp IHMKMTI4MSA1NzgxIG10IDg5MTYgNTc4MSBMCjEyODEgIDQ4NyBtdCA4OTE2ICA0ODcgTAoxMjgx IDU3ODEgbXQgMTI4MSAgNDg3IEwKODkxNiA1NzgxIG10IDg5MTYgIDQ4NyBMCmdzIDEyODEgNDg3 IDc2MzYgNTI5NSByYwovYzggeyAwLjAwMDAwMCAwLjAwMDAwMCAxLjAwMDAwMCBzcn0gYmRlZgpj OAozNSAtNyAzNCAwIDM1IC0zIDM1IC03IDM1IC03IDM1IC00IDM1IDAgMzUgMCAKMzUgLTMgMzUg LTQgMzQgLTQgMzUgMCAzNSAwIDM1IC03IDM1IC0zIDM1IC03IAozNSAtNCAzNSAtNyAzNCAtNyAz NSAtNyAzNSAtNyAzNSAtMTUgMzUgMCAzNSAtMTAgCjM1IC03IDM1IC03IDM1IDAgMzQgLTE4IDM1 IC0xNCAzNSAtMzkgMzUgLTE1IDM1IC00MiAKMzUgLTQzIDM1IC0yNSAzNSAtMzkgMzQgLTUwIDM1 IC0yOCAzNSAtMzYgMzUgLTM5IDM1IC0zNSAKMzUgLTM5IDM1IC0zNiAzNSAtODIgMzQgLTQ5IDM1 IC0zNiAzNSAtMzIgMzUgLTUwIDM1IC00OSAKMzUgLTI1IDM1IC0yOSAzNSAtMjQgMzUgLTIyIDM0 IC02MCAzNSAtNDMgMzUgLTI1IDM1IC0zMiAKMzUgLTE3IDM1IC0yOSAzNSAtMzUgMzUgLTU3IDM0 IC00NiAzNSAtNDMgMzUgLTUzIDM1IC0yNSAKMzUgLTY4IDM1IC04MSAzNSAtOTYgMzUgLTcxIDM1 IC04OSAzNCAtODYgMzUgLTQ2IDM1IC05MiAKMzUgLTEwMCAzNSAtNzggMzUgLTcxIDM1IC02MCAz NSAtODkgMzQgLTgyIDM1IC03MSAzNSAtNTAgCjM1IC03NCAzNSAtNjEgMzUgLTc0IDM1IC01MyAz NSAtODIgMzQgLTYxIDM1IC02NyAzNSAtNzEgCjM1IC0zNiAzNSAtNTMgMzUgLTU3IDM1IC0zOSAz NSAtMzUgMzUgLTUwIDM0IC0zNiAzNSAtMzkgCjM1IC0yNSAzNSAtNjQgMzUgLTYwIDM1IC01NyAz NSAtMzIgMzUgLTI1IDM0IC0zNSAzNSAtNTAgCjM1IC0zMiAzNSAtNTMgMzUgLTQ2IDM1IC0xNSAz NSAtNTcgMzUgLTE3IDM1IC0zMiAzNCAtMjUgCjM1IC0zNiAzNSAtNDkgMzUgLTMyIDM1IC00NyAz NSAtMjggMzUgLTI4IDM1IC0yNSAzNCAtNDYgCjM1IC0xNSAzNSAtMTQgMzUgLTE4IDM1IC03IDM1 IC0yMSAzNSAtMzYgMzUgLTI4IDM1IC0yNSAKMzQgLTI4IDM1IC03IDM1IC0xOCAzNSAtMTEgMzUg LTIxIDM1IC0zMiAzNSAtNCAzNSAtNyAKMzQgLTcgMzUgLTQgMzUgLTcxIDM1IC03MSAzNSAtMjI3 IDM1IC00IDM1IDAgMzUgLTIxIAozNCAwIDM1IDAgMzUgLTMgMzUgMCAzNSAwIDM1IDAgMzUgMCAz NSAwIAozNSAtMTggMzQgMCAzNSAwIDM1IC03IDM1IDAgMzUgMCAzNSAwIDM1IC00IAozNSAtMyAz NCAwIDM1IC0xMSAzNSAwIDM1IDAgMzUgMCAzNSAwIDM1IDAgCjM1IDAgMzUgMCAzNCAwIDM1IDAg MzUgMCAzNSAwIDM1IDAgMzUgMCAKMzUgMCAzNSAwIDM0IDAgMzUgMCAzNSAwIDM1IDAgMzUgMCAz NSAwIAozNSAwIDM1IDAgMzQgMCAzNSAwIDM1IDAgMzUgLTExIDM1IDAgMzUgMCAKMzUgMCAzNSAw IDM1IDAgMzQgMCAzNSAwIDM1IDAgMzUgMCAxMjk4IDU3NjMgMjAwIE1QIHN0cm9rZQpncgoKYzgK MCBzZwozNDQ3ICAzOTIgbXQgCihDREYgb2Ygb2JqZWN0IHNpemUgZm9yIDE1IHBvcHVsYXIgd2Vi c2l0ZXMgXCgxNDkwIHRvdGFsIG9iamVcCmN0c1wpKSBzCiA5OTggMzcyOSBtdCAgLTkwIHJvdGF0 ZQooQ3VtdWxhdGl2ZSBEaXN0aWJ1dGlvbikgcwo5MCByb3RhdGUKNDU5MiA2MTA2IG10IAooT2Jq ZWN0IFNpemUgXChieXRlc1wpKSBzCgplbmQgJSVDb2xvciBEaWN0CgplcGxvdAolJUVuZE9iamVj dAoKZXBhZ2UKZW5kCgpzaG93cGFnZQoKJSVUcmFpbGVyCiUlRU9GCg== --_002_CF76BDC532D89gwhitecablelabscom_-- From nobody Fri Apr 18 11:48:19 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1148C1A0445 for ; Fri, 18 Apr 2014 11:48:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 3.1 X-Spam-Level: *** X-Spam-Status: No, score=3.1 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, MANGLED_SIDE=2.3, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lb7V-68FzFsg for ; Fri, 18 Apr 2014 11:48:13 -0700 (PDT) Received: from smtp78.iad3a.emailsrvr.com (smtp78.iad3a.emailsrvr.com [173.203.187.78]) by ietfa.amsl.com (Postfix) with ESMTP id 34CD31A014B for ; Fri, 18 Apr 2014 11:48:13 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp26.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id E5FBE3380B5; Fri, 18 Apr 2014 14:48:08 -0400 (EDT) X-Virus-Scanned: OK Received: from app44.wa-webapps.iad3a (relay.iad3a.rsapps.net [172.27.255.110]) by smtp26.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id C1ADF338097; Fri, 18 Apr 2014 14:48:08 -0400 (EDT) Received: from reed.com (localhost.localdomain [127.0.0.1]) by app44.wa-webapps.iad3a (Postfix) with ESMTP id AF19618007A; Fri, 18 Apr 2014 14:48:08 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Fri, 18 Apr 2014 14:48:08 -0400 (EDT) Date: Fri, 18 Apr 2014 14:48:08 -0400 (EDT) From: dpreed@reed.com To: "Greg White" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20140418144808000000_69530" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: Message-ID: <1397846888.714812176@apps.rackspace.com> X-Mailer: webmail7.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/d6FUqJH7qq1suP-EYbZ1TteN4fs Cc: bloat , "aqm@ietf.org" , Dave Taht , "cerowrt-devel@lists.bufferbloat.net" , =?utf-8?Q?William_Chan_=28=E9=99=88=E6=99=BA=E6=98=8C=29?= Subject: Re: [aqm] [Cerowrt-devel] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 18:48:18 -0000 ------=_20140418144808000000_69530 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AWhy is the DNS PLR so high? 1% is pretty depressing.=0A =0AAlso, it see= ms odd to eliminate 19% of the content retrieval because the tail is fat an= d long rather than short. Wouldn't it be better to have 1000 servers?=0A = =0A =0A=0A=0AOn Friday, April 18, 2014 2:15pm, "Greg White" said:=0A=0A=0A=0A> Dave,=0A> =0A> We used the 25k object size for = a short time back in 2012 until we had=0A> resources to build a more advanc= ed model (appendix A). I did a bunch of=0A> captures of real web pages bac= k in 2011 and compared the object size=0A> statistics to models that I'd se= en published. Lognormal didn't seem to be=0A> *exactly* right, but it wasn= 't a bad fit to what I saw. I've attached a=0A> CDF.=0A> =0A> The choice o= f 4 servers was based somewhat on logistics, and also on a=0A> finding that= across our data set, the average web page retrieved 81% of=0A> its resourc= es from the top 4 servers. Increasing to 5 servers only=0A> increased that= percentage to 84%.=0A> =0A> The choice of RTTs also came from the web traf= fic captures. I saw=0A> RTTmin=3D16ms, RTTmean=3D53.8ms, RTTmax=3D134ms.=0A= > =0A> Much of this can be found in=0A> https://tools.ietf.org/html/draft-w= hite-httpbis-spdy-analysis-00=0A> =0A> In many of the cases that we've simu= lated, the packet drop probability is=0A> less than 1% for DNS packets. In= our web model, there are a total of 4=0A> servers, so 4 DNS lookups assumi= ng none of the addresses are cached. If=0A> PLR =3D 1%, there would be a 3.= 9% chance of losing one or more DNS packets=0A> (with a resulting ~5 second= additional delay on load time). I've probably=0A> oversimplified this, bu= t Kathie N. and I made the call that it would be=0A> significantly easier t= o just do this math than to build a dns=0A> implementation in ns2. We've o= pen sourced the web model (it's on Kathie's=0A> web page and will be part o= f ns2.36) with an encouragement to the=0A> community to improve on it. If = you'd like to port it to ns3 and add a dns=0A> model, that would be fantast= ic.=0A> =0A> -Greg=0A> =0A> =0A> On 4/17/14, 3:07 PM, "Dave Taht" wrote:=0A> =0A> >On Thu, Apr 17, 2014 at 12:01 PM, William Cha= n (=E9=99=88=E6=99=BA=E6=98=8C)=0A> > wrote:=0A> >> = Speaking as the primary Chromium developer in charge of this relevant=0A> >= >code,=0A> >> I would like to caution putting too much trust in the numbers= =0A> >>generated. Any=0A> >> statistical claims about the numbers are proba= bly unreasonable to make.=0A> >=0A> >Sigh. Other benchmarks such as the apa= che ("ab") benchmark=0A> >are primarily designed as stress testers for web = servers, not as realistic=0A> >traffic. Modern web traffic has such a high = level of dynamicism in it,=0A> >that static web page loads along any distri= bution, seem insufficient,=0A> >passive analysis of aggregated traffic "fee= ls" incorrect relative to the=0A> >sorts of home and small business traffic= I've seen, and so on.=0A> >=0A> >Famous papers, such as this one:=0A> >=0A= > >http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-leland.pdf=0A> >=0A> = >Seem possibly irrelevant to draw conclusions from given the kind=0A> >of d= ata they analysed and proceeding from an incorrect model or=0A> >gut feel f= or how the network behaves today seems to be foolish.=0A> >=0A> >Even the m= ost basic of tools, such as httping, had three basic bugs=0A> >that I found= in a few minutes of trying to come up with some basic=0A> >behaviors yeste= rday:=0A> >=0A> >https://lists.bufferbloat.net/pipermail/bloat/2014-April/0= 01890.html=0A> >=0A> >Those are going to be a lot easier to fix than diving= into the chromium=0A> >codebase!=0A> >=0A> >There are very few tools worth= trusting, and I am always dubious=0A> >of papers that publish results with= unavailable tools and data. The only=0A> >tools I have any faith in for ne= twork analysis are netperf,=0A> >netperf-wrapper,=0A> >tcpdump and xplot.or= g, and to a large extent wireshark. Toke and I have=0A> >been tearing apart= d-itg and I hope to one day be able to add that to=0A> >my trustable list.= .. but better tools are needed!=0A> >=0A> >Tools that I don't have a lot of= faith in include that, iperf, anything=0A> >written=0A> >in java or other = high level languages, speedtest.net, and things like=0A> >shaperprobe.=0A> = >=0A> >Have very little faith in ns2, slightly more in ns3, and I've been m= eaning=0A> >to look over the mininet and other simulators whenever I got so= me spare=0A> >time; the mininet results stanford gets seem pretty reasonabl= e and I=0A> >adore their reproducing results effort. Haven't explored ndt, = keep meaning=0A> >to...=0A> >=0A> >> Reasons:=0A> >> * We don't actively ma= intain this code. It's behind the command line=0A> >>flags.=0A> >> They are= broken. The fact that it still results in numbers on the=0A> >>benchmark= =0A> >> extension is an example of where unmaintained code doesn't have the= UI=0A> >> disabled, even though the internal workings of the code fail to= =0A> >>guarantee=0A> >> correct operation. We haven't disabled it because, = well, it's=0A> >>unmaintained.=0A> >=0A> >As I mentioned I was gearing up f= or a hacking run...=0A> >=0A> >The vast majority of results I look at are a= ctually obtained via=0A> >looking at packet captures. I mostly use benchmar= ks as abstractions=0A> >to see if they make some sense relative to the capt= ures and tend=0A> >to compare different benchmarks against each other.=0A> = >=0A> >I realize others don't go into that level of detail, so you have giv= en=0A> >fair warning! In our case we used the web page benchmarker as=0A> >= a means to try and rapidly acquire some typical distributions of=0A> >get a= nd tcp stream requests from things like the alexa top 1000,=0A> >and as a w= ay to A/B different aqm/packet scheduling setups.=0A> >=0A> >... but the on= ly easily publishable results were from the benchmark=0A> >itself,=0A> >and= we (reluctantly) only published one graph from all the work that=0A> >went= into it 2+ years back and used it as a test driver for the famous=0A> >iet= f video, comparing two identical boxes running it at the same time=0A> >und= er different network conditions:=0A> >=0A> >https://www.bufferbloat.net/pro= jects/cerowrt/wiki/Bloat-videos#IETF-demo-s=0A> >ide-by-side-of-a-normal-ca= ble-modem-vs-fq_codel=0A> >=0A> >from what I fiddled with today, it is at l= east still useful for that?=0A> >=0A> >moving on...=0A> >=0A> >The web mode= l in the cablelabs work doesn't look much like my captures,=0A> >in additio= n to not modeling dns at all, and using a smaller IW than google=0A> >it lo= oks like this:=0A> >=0A> >>> Model single user web page download as follows= :=0A> >=0A> >>> - Web page modeled as single HTML page + 100 objects spread= evenly=0A> >>> across 4 servers. Web object sizes are currently fixed at 2= 5 kB=0A> each,=0A> >>> whereas the initial HTML page is 100 kB. Appendix A = provides an=0A> >>> alternative page model that may be explored in future w= ork.=0A> >=0A> >Where what I see is a huge number of stuff that fits into a= single=0A> >iw10 slow start episode and some level of pipelining on larger= stuff, so=0A> >that a=0A> >large number of object sizes of less than 7k wi= th a lightly tailed=0A> >distribution=0A> >outside of that makes more sense= .=0A> >=0A> >(I'm not staring at appendix A right now, I'm under the impres= sion=0A> > it was better)=0A> >=0A> >I certainly would like more suggestion= s for models and types=0A> >of web traffic, as well as simulation of https = + pfs traffic,=0A> >spdy, quic, etc....=0A> >=0A> >>> - Server RTTs set as = follows (20 ms, 30 ms, 50 ms, 100 ms).=0A> >=0A> >Server RTTs from my own w= eb history tend to be lower than 50ms.=0A> >=0A> >>> - Initial HTTP GET to = retrieve a moderately sized object (100 kB=0A> HTML=0A> >>> page) from serv= er 1.=0A> >=0A> >An initial GET to google fits into iw10 - it's about 7k.= =0A> >=0A> >>> - Once initial HTTP GET completes, initiate 24 simultaneous = HTTP=0A> GETs=0A> >>> (via separate TCP connections), 6 connections each to= 4 different=0A> >>> server nodes=0A> >=0A> >I usually don't see more than = 15. and certainly not 25k sized objects.=0A> >=0A> > > - Once each individu= al HTTP GET completes, initiate a subsequent GET=0A> >> to the same server,= until 25 objects have been retrieved from each=0A> >> server.=0A> >=0A> >= =0A> >> * We don't make sure to flush all the network state in between runs= , so=0A> >>if=0A> >> you're using that option, don't trust it to work.=0A> = >=0A> >The typical scenario we used was a run against dozens or hundreds of= urls,=0A> >capturing traffic, while varying network conditions.=0A> >=0A> = >Regarded the first run as the most interesting.=0A> >=0A> >Can exit the br= owser and restart after a run like that.=0A> >=0A> >At moment, merely plan = to use the tool primarily to survey various=0A> >web sites and load times w= hile doing packet captures. Hope was=0A> >to get valid data from the networ= k portion of the load, tho...=0A> >=0A> >> * If you have an advanced Chromi= um setup, this definitely does not=0A> >>work. I=0A> >> advise using the be= nchmark extension only with a separate Chromium=0A> >>profile=0A> >> for te= sting purposes. Our flushing of sockets, caches, etc does not=0A> >>actuall= y=0A> >> work correctly when you use the Chromium multiprofile feature and = also=0A> >>fails=0A> >> to flush lots of our other network caches.=0A> >=0A= > >noted.=0A> >=0A> >=0A> >> * No one on Chromium really believes the time = to paint numbers that we=0A> >> output :) It's complicated. Our graphics st= ack is complicated. The time=0A> >>from=0A> >=0A> >I actually care only abo= ut time-to-full layout as that's a core network=0A> >effect...=0A> >=0A> >>= when Blink thinks it painted to when the GPU actually blits to the=0A> >>s= creen=0A> >> cannot currently be corroborated with any high degree of accur= acy from=0A> >> within our code.=0A> >=0A> >> * It has not been maintained = since 2010. It is quite likely there are=0A> >>many=0A> >> other subtle ina= ccuracies here.=0A> >=0A> >Grok.=0A> >=0A> >> In short, while you can expec= t it to give you a very high level=0A> >> understanding of performance issu= es, I advise against placing=0A> >>non-trivial=0A> >> confidence in the acc= uracy of the numbers generated by the benchmark=0A> >> extension. The fact = that numbers are produced by the extension should=0A> >>not be=0A> >> treat= ed as evidence that the extension actually functions correctly.=0A> >=0A> >= OK, noted. Still delighted to be able to have a simple load generator=0A> >= that exercises the browsers and generates some results, however=0A> >dubiou= s.=0A> >=0A> >>=0A> >> Cheers.=0A> >>=0A> >>=0A> >> On Thu, Apr 17, 2014 at= 10:49 AM, Dave Taht =0A> wrote:=0A> >>>=0A> >>> Getti= ng a grip on real web page load time behavior in an age of=0A> >>> sharded = websites,=0A> >>> dozens of dns lookups, javascript, and fairly random beha= vior in ad=0A> >>> services=0A> >>> and cdns against how a modern browsers = behaves is very, very hard.=0A> >>>=0A> >>> it turns out if you run=0A> >>>= =0A> >>> google-chrome --enable-benchmarking --enable-net-benchmarking=0A> = >>>=0A> >>> (Mac users have to embed these options in their startup script = - see=0A> >>> http://www.chromium.org/developers/how-tos/run-chromium-with= -flags=0A> )=0A> >>>=0A> >>> enable developer options and install and run t= he chrome web page=0A> >>> benchmarker,=0A> >>> (=0A> >>>=0A> >>>https://ch= rome.google.com/webstore/detail/page-benchmarker/channimfdomah=0A> >>>ekjca= hlbpccbgaopjll?hl=3Den=0A> >>> )=0A> >>>=0A> >>> that it works (at least fo= r me, on a brief test of the latest=0A> chrome,=0A> >>>on=0A> >>> linux.=0A= > >>> Can someone try windows and mac?)=0A> >>>=0A> >>> You can then feed i= n a list of urls to test against, and post=0A> process=0A> >>> the resultin= g .csv file to your hearts content. We used to use this=0A> >>> benchmark a= lot while trying to characterise typical web behaviors=0A> >>> under aqm a= nd packet scheduling systems under load. Running=0A> >>> it simultaneously = with a rrul test or one of the simpler tcp upload=0A> or=0A> >>> download= =0A> >>> tests in the rrul suite was often quite interesting.=0A> >>>=0A> >= >> It turned out the doc has been wrong a while as to the name of the=0A> >= >>second=0A> >>> command lnie option. I was gearing up mentally for having = to look at=0A> >>> the source....=0A> >>>=0A> >>> http://code.google.com/p/= chromium/issues/detail?id=3D338705=0A> >>>=0A> >>> /me happy=0A> >>>=0A> >>= > --=0A> >>> Dave T=C3=A4ht=0A> >>>=0A> >>> Heartbleed POC on wifi campus n= etworks with EAP auth:=0A> >>> http://www.eduroam.edu.au/advisory.html=0A> = >>>=0A> >>> _______________________________________________=0A> >>> aqm mai= ling list=0A> >>> aqm@ietf.org=0A> >>> https://www.ietf.org/mailman/listinf= o/aqm=0A> >>=0A> >>=0A> >=0A> >=0A> >=0A> >--=0A> >Dave T=C3=A4ht=0A> >=0A>= >NSFW:=0A> >https://w2.eff.org/Censorship/Internet_censorship_bills/russel= l_0296_indec=0A> >ent.article=0A> >=0A> >__________________________________= _____________=0A> >aqm mailing list=0A> >aqm@ietf.org=0A> >https://www.ietf= .org/mailman/listinfo/aqm=0A> =0A> ________________________________________= _______=0A> Cerowrt-devel mailing list=0A> Cerowrt-devel@lists.bufferbloat.= net=0A> https://lists.bufferbloat.net/listinfo/cerowrt-devel=0A> ------=_20140418144808000000_69530 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Why is the= DNS PLR so high?  1% is pretty depressing.

=0A

 

=0A

Also, it seems o= dd to eliminate 19% of the content retrieval because the tail is fat and lo= ng rather than short.  Wouldn't it be better to have 1000 servers?

= =0A

 

=0A

 

=0A=0A



On Friday, April 1= 8, 2014 2:15pm, "Greg White" <g.white@CableLabs.com> said:

=0A
=0A

> Dave,
>
> We used the 25k object size for a short t= ime back in 2012 until we had
> resources to build a more advanced = model (appendix A). I did a bunch of
> captures of real web pages = back in 2011 and compared the object size
> statistics to models th= at I'd seen published. Lognormal didn't seem to be
> *exactly* rig= ht, but it wasn't a bad fit to what I saw. I've attached a
> CDF.<= br />>
> The choice of 4 servers was based somewhat on logistic= s, and also on a
> finding that across our data set, the average we= b page retrieved 81% of
> its resources from the top 4 servers. In= creasing to 5 servers only
> increased that percentage to 84%.
>
> The choice of RTTs also came from the web traffic captures= . I saw
> RTTmin=3D16ms, RTTmean=3D53.8ms, RTTmax=3D134ms.
>= ;
> Much of this can be found in
> https://tools.ietf.org/= html/draft-white-httpbis-spdy-analysis-00
>
> In many of t= he cases that we've simulated, the packet drop probability is
> les= s than 1% for DNS packets. In our web model, there are a total of 4
&= gt; servers, so 4 DNS lookups assuming none of the addresses are cached. If=
> PLR =3D 1%, there would be a 3.9% chance of losing one or more D= NS packets
> (with a resulting ~5 second additional delay on load t= ime). I've probably
> oversimplified this, but Kathie N. and I mad= e the call that it would be
> significantly easier to just do this = math than to build a dns
> implementation in ns2. We've open sourc= ed the web model (it's on Kathie's
> web page and will be part of n= s2.36) with an encouragement to the
> community to improve on it. = If you'd like to port it to ns3 and add a dns
> model, that would b= e fantastic.
>
> -Greg
>
>
> On = 4/17/14, 3:07 PM, "Dave Taht" <dave.taht@gmail.com> wrote:
> =
> >On Thu, Apr 17, 2014 at 12:01 PM, William Chan (=E9=99=88=E6= =99=BA=E6=98=8C)
> ><willchan@chromium.org> wrote:
&g= t; >> Speaking as the primary Chromium developer in charge of this re= levant
> >>code,
> >> I would like to caution p= utting too much trust in the numbers
> >>generated. Any
= > >> statistical claims about the numbers are probably unreasonabl= e to make.
> >
> >Sigh. Other benchmarks such as the = apache ("ab") benchmark
> >are primarily designed as stress test= ers for web servers, not as realistic
> >traffic. Modern web tra= ffic has such a high level of dynamicism in it,
> >that static w= eb page loads along any distribution, seem insufficient,
> >pass= ive analysis of aggregated traffic "feels" incorrect relative to the
&= gt; >sorts of home and small business traffic I've seen, and so on.
> >
> >Famous papers, such as this one:
> >> >http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-leland.pdf> >
> >Seem possibly irrelevant to draw conclusions fr= om given the kind
> >of data they analysed and proceeding from a= n incorrect model or
> >gut feel for how the network behaves tod= ay seems to be foolish.
> >
> >Even the most basic of= tools, such as httping, had three basic bugs
> >that I found in= a few minutes of trying to come up with some basic
> >behaviors= yesterday:
> >
> >https://lists.bufferbloat.net/pipe= rmail/bloat/2014-April/001890.html
> >
> >Those are g= oing to be a lot easier to fix than diving into the chromium
> >= codebase!
> >
> >There are very few tools worth trust= ing, and I am always dubious
> >of papers that publish results w= ith unavailable tools and data. The only
> >tools I have any fai= th in for network analysis are netperf,
> >netperf-wrapper,
> >tcpdump and xplot.org, and to a large extent wireshark. Toke and = I have
> >been tearing apart d-itg and I hope to one day be able= to add that to
> >my trustable list... but better tools are nee= ded!
> >
> >Tools that I don't have a lot of faith in= include that, iperf, anything
> >written
> >in java = or other high level languages, speedtest.net, and things like
> >= ;shaperprobe.
> >
> >Have very little faith in ns2, s= lightly more in ns3, and I've been meaning
> >to look over the m= ininet and other simulators whenever I got some spare
> >time; t= he mininet results stanford gets seem pretty reasonable and I
> >= ;adore their reproducing results effort. Haven't explored ndt, keep meaning=
> >to...
> >
> >> Reasons:
> &= gt;> * We don't actively maintain this code. It's behind the command lin= e
> >>flags.
> >> They are broken. The fact tha= t it still results in numbers on the
> >>benchmark
> = >> extension is an example of where unmaintained code doesn't have th= e UI
> >> disabled, even though the internal workings of the = code fail to
> >>guarantee
> >> correct operati= on. We haven't disabled it because, well, it's
> >>unmaintain= ed.
> >
> >As I mentioned I was gearing up for a hack= ing run...
> >
> >The vast majority of results I look= at are actually obtained via
> >looking at packet captures. I m= ostly use benchmarks as abstractions
> >to see if they make some= sense relative to the captures and tend
> >to compare different= benchmarks against each other.
> >
> >I realize othe= rs don't go into that level of detail, so you have given
> >fair= warning! In our case we used the web page benchmarker as
> >a m= eans to try and rapidly acquire some typical distributions of
> >= ;get and tcp stream requests from things like the alexa top 1000,
>= >and as a way to A/B different aqm/packet scheduling setups.
> = >
> >... but the only easily publishable results were from th= e benchmark
> >itself,
> >and we (reluctantly) only p= ublished one graph from all the work that
> >went into it 2+ yea= rs back and used it as a test driver for the famous
> >ietf vide= o, comparing two identical boxes running it at the same time
> >= under different network conditions:
> >
> >https://ww= w.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos#IETF-demo-s
> = >ide-by-side-of-a-normal-cable-modem-vs-fq_codel
> >
>= ; >from what I fiddled with today, it is at least still useful for that?=
> >
> >moving on...
> >
> >The= web model in the cablelabs work doesn't look much like my captures,
&= gt; >in addition to not modeling dns at all, and using a smaller IW than= google
> >it looks like this:
> >
> >>= > Model single user web page download as follows:
> >
&g= t; >>> - Web page modeled as single HTML page + 100 objects spread= evenly
> >>> across 4 servers. Web object sizes are curre= ntly fixed at 25 kB
> each,
> >>> whereas the init= ial HTML page is 100 kB. Appendix A provides an
> >>> alte= rnative page model that may be explored in future work.
> >
> >Where what I see is a huge number of stuff that fits into a singl= e
> >iw10 slow start episode and some level of pipelining on lar= ger stuff, so
> >that a
> >large number of object siz= es of less than 7k with a lightly tailed
> >distribution
&g= t; >outside of that makes more sense.
> >
> >(I'm = not staring at appendix A right now, I'm under the impression
> >= ; it was better)
> >
> >I certainly would like more s= uggestions for models and types
> >of web traffic, as well as si= mulation of https + pfs traffic,
> >spdy, quic, etc....
>= ; >
> >>> - Server RTTs set as follows (20 ms, 30 ms, 5= 0 ms, 100 ms).
> >
> >Server RTTs from my own web his= tory tend to be lower than 50ms.
> >
> >>> - In= itial HTTP GET to retrieve a moderately sized object (100 kB
> HTML=
> >>> page) from server 1.
> >
> >A= n initial GET to google fits into iw10 - it's about 7k.
> >
> >>> - Once initial HTTP GET completes, initiate 24 simultane= ous HTTP
> GETs
> >>> (via separate TCP connection= s), 6 connections each to 4 different
> >>> server nodes> >
> >I usually don't see more than 15. and certainly= not 25k sized objects.
> >
> > > - Once each indi= vidual HTTP GET completes, initiate a subsequent GET
> >> to = the same server, until 25 objects have been retrieved from each
> &= gt;> server.
> >
> >
> >> * We don't= make sure to flush all the network state in between runs, so
> >= ;>if
> >> you're using that option, don't trust it to work= .
> >
> >The typical scenario we used was a run again= st dozens or hundreds of urls,
> >capturing traffic, while varyi= ng network conditions.
> >
> >Regarded the first run = as the most interesting.
> >
> >Can exit the browser = and restart after a run like that.
> >
> >At moment, = merely plan to use the tool primarily to survey various
> >web s= ites and load times while doing packet captures. Hope was
> >to = get valid data from the network portion of the load, tho...
> ><= br />> >> * If you have an advanced Chromium setup, this definitel= y does not
> >>work. I
> >> advise using the be= nchmark extension only with a separate Chromium
> >>profile> >> for testing purposes. Our flushing of sockets, caches, et= c does not
> >>actually
> >> work correctly whe= n you use the Chromium multiprofile feature and also
> >>fail= s
> >> to flush lots of our other network caches.
> &= gt;
> >noted.
> >
> >
> >> *= No one on Chromium really believes the time to paint numbers that we
= > >> output :) It's complicated. Our graphics stack is complicated= . The time
> >>from
> >
> >I actually c= are only about time-to-full layout as that's a core network
> >e= ffect...
> >
> >> when Blink thinks it painted to = when the GPU actually blits to the
> >>screen
> >&= gt; cannot currently be corroborated with any high degree of accuracy from<= br />> >> within our code.
> >
> >> * It = has not been maintained since 2010. It is quite likely there are
> = >>many
> >> other subtle inaccuracies here.
> &= gt;
> >Grok.
> >
> >> In short, while y= ou can expect it to give you a very high level
> >> understan= ding of performance issues, I advise against placing
> >>non-= trivial
> >> confidence in the accuracy of the numbers genera= ted by the benchmark
> >> extension. The fact that numbers ar= e produced by the extension should
> >>not be
> >&= gt; treated as evidence that the extension actually functions correctly.> >
> >OK, noted. Still delighted to be able to have a = simple load generator
> >that exercises the browsers and generat= es some results, however
> >dubious.
> >
> &g= t;>
> >> Cheers.
> >>
> >>
> >> On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht <dave.taht@g= mail.com>
> wrote:
> >>>
> >>>= Getting a grip on real web page load time behavior in an age of
> = >>> sharded websites,
> >>> dozens of dns lookups= , javascript, and fairly random behavior in ad
> >>> servi= ces
> >>> and cdns against how a modern browsers behaves i= s very, very hard.
> >>>
> >>> it turns o= ut if you run
> >>>
> >>> google-chrome -= -enable-benchmarking --enable-net-benchmarking
> >>>
= > >>> (Mac users have to embed these options in their startup s= cript - see
> >>> http://www.chromium.org/developers/how-= tos/run-chromium-with-flags
> )
> >>>
> &g= t;>> enable developer options and install and run the chrome web page=
> >>> benchmarker,
> >>> (
> >= ;>>
> >>>https://chrome.google.com/webstore/detail/p= age-benchmarker/channimfdomah
> >>>ekjcahlbpccbgaopjll?hl= =3Den
> >>> )
> >>>
> >>>= ; that it works (at least for me, on a brief test of the latest
> c= hrome,
> >>>on
> >>> linux.
> >= ;>> Can someone try windows and mac?)
> >>>
>= ; >>> You can then feed in a list of urls to test against, and pos= t
> process
> >>> the resulting .csv file to your = hearts content. We used to use this
> >>> benchmark a lot = while trying to characterise typical web behaviors
> >>> u= nder aqm and packet scheduling systems under load. Running
> >&g= t;> it simultaneously with a rrul test or one of the simpler tcp upload<= br />> or
> >>> download
> >>> tests i= n the rrul suite was often quite interesting.
> >>>
&= gt; >>> It turned out the doc has been wrong a while as to the nam= e of the
> >>>second
> >>> command lnie o= ption. I was gearing up mentally for having to look at
> >>&g= t; the source....
> >>>
> >>> http://code= .google.com/p/chromium/issues/detail?id=3D338705
> >>>
> >>> /me happy
> >>>
> >>>= --
> >>> Dave T=C3=A4ht
> >>>
> = >>> Heartbleed POC on wifi campus networks with EAP auth:
>= ; >>> http://www.eduroam.edu.au/advisory.html
> >>&g= t;
> >>> _______________________________________________> >>> aqm mailing list
> >>> aqm@ietf.org<= br />> >>> https://www.ietf.org/mailman/listinfo/aqm
> = >>
> >>
> >
> >
> >> >--
> >Dave T=C3=A4ht
> >
> >NS= FW:
> >https://w2.eff.org/Censorship/Internet_censorship_bills/r= ussell_0296_indec
> >ent.article
> >
> >__= _____________________________________________
> >aqm mailing lis= t
> >aqm@ietf.org
> >https://www.ietf.org/mailman/lis= tinfo/aqm
>
> ____________________________________________= ___
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.buff= erbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel<= br />>

=0A
 ------=_20140418144808000000_69530-- From nobody Fri Apr 18 12:05:26 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DA9031A043F for ; Fri, 18 Apr 2014 12:05:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.5 X-Spam-Level: ** X-Spam-Status: No, score=2.5 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, J_CHICKENPOX_44=0.6, J_CHICKENPOX_64=0.6, J_CHICKENPOX_74=0.6, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r0MzPhQ0dY-D for ; Fri, 18 Apr 2014 12:05:20 -0700 (PDT) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) by ietfa.amsl.com (Postfix) with ESMTP id B28171A0471 for ; Fri, 18 Apr 2014 12:05:19 -0700 (PDT) Received: by mail-wi0-f174.google.com with SMTP id d1so1013063wiv.7 for ; Fri, 18 Apr 2014 12:05:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=TsM/UE6svtZ+HxVxDpOsOG6MzeitFZAeHWaE++U67gY=; b=C25A1jV0dueoBE0bEpMehyo5XLwI14WbosOIuLxsccT09dznDPU/CVFFC3aJDD8NAB hZxoCRtdMgT708YQnBHF9BgHjcBpG2Vf6UHpXOCo97fNDR5TeQtsOvug+ZlRqihDQAg/ NTZG9uslcbYGqhJG3vFlavrzkjWcm14AilQExwHNfGZHbWz6stHZS4rcBKIWdx8pgiB/ 8kthxUxbJ9ZuGmq0cD+eptRa32ADwOTrGn8+GA61EJMvgQB/EM/1dHyH1wd0sd6/o4UO MFjAnCDBYKCCTa1pqgJ+Yn8WCZOhKcFo81zMjLafIqJ/xNgbg7wCwdz/UNH2HR1TmPsT UKHw== MIME-Version: 1.0 X-Received: by 10.180.81.138 with SMTP id a10mr3446993wiy.53.1397847915068; Fri, 18 Apr 2014 12:05:15 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Fri, 18 Apr 2014 12:05:15 -0700 (PDT) In-Reply-To: References: Date: Fri, 18 Apr 2014 12:05:15 -0700 Message-ID: From: Dave Taht To: Greg White Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/_H4nJnJSaCj_Mk-om7zwT9POz9A Cc: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 19:05:25 -0000 On Fri, Apr 18, 2014 at 11:15 AM, Greg White wrote: > Dave, > > We used the 25k object size for a short time back in 2012 until we had > resources to build a more advanced model (appendix A). I did a bunch of > captures of real web pages back in 2011 and compared the object size > statistics to models that I'd seen published. Lognormal didn't seem to b= e > *exactly* right, but it wasn't a bad fit to what I saw. I've attached a > CDF. That does seem a bit large on the initial 20%. Hmm. There is a second kind of major case, where you are moving around on the same web property, and hopefully many core portions of the web page(s) such as the css and javascript, basic logos and other images, are cached. Caching is handled two ways, one is to explicitly mark the data as cacheable for a certain period, the other is an if-modified-since request, which costs RTTs for setup and the query. I am under the impression that we generally see a lot more of the latter than the former these days. > The choice of 4 servers was based somewhat on logistics, and also on a > finding that across our data set, the average web page retrieved 81% of > its resources from the top 4 servers. Increasing to 5 servers only > increased that percentage to 84%. > > The choice of RTTs also came from the web traffic captures. I saw > RTTmin=3D16ms, RTTmean=3D53.8ms, RTTmax=3D134ms. Get a median? My own stats are probably quite skewed lower from being in california, and doing some tests from places like isc.org in redwood city, which is insanely well co-located. > Much of this can be found in > https://tools.ietf.org/html/draft-white-httpbis-spdy-analysis-00 Thx! > In many of the cases that we've simulated, the packet drop probability is > less than 1% for DNS packets. In our web model, there are a total of 4 I think we have the ability to get a better number for dns loss now. > servers, so 4 DNS lookups assuming none of the addresses are cached. If > PLR =3D 1%, there would be a 3.9% chance of losing one or more DNS packet= s > (with a resulting ~5 second additional delay on load time). I've probabl= y > oversimplified this, but Kathie N. and I made the call that it would be > significantly easier to just do this math than to build a dns > implementation in ns2. The specific thing I've been concerned about was not the probability of a dns loss, although as you note the consequences are huge - but the frequency and cost of a cache miss and the resulting fill. This is a very simple namebench test against the alexa top 1000: http://snapon.lab.bufferbloat.net/~d/namebench/namebench_2014-03-20_1255.ht= ml This is a more comprehensive one taken against my own recent web history fi= le. http://snapon.lab.bufferbloat.net/~d/namebench/namebench_2014-03-24_1541.ht= ml Both of these were taken against the default SQM system in cerowrt against a cable modem, so you can pretty safely assume the ~20ms (middle) knee in the curve is basically based on physical RTT to the nearest upstream DNS server. And it's a benchmark so I don't generally believe in the relative hit ratios vs a vs "normal traffic", but do think the baseline RTT, and the knees in the curves in the cost of a miss and file are relevant. (it's also not clear to me if all cable modems run a local dns server) Recently simon kelly added support for gathering hit and miss statistics to dnsmasq 2.69. They can be obtained via a simple dns lookup as answers to queries of class CHAOS and type TXT in domain bind. The domain names are cachesize.bind, insertions.bind, evictions.bind, misses.bind, hits.bind, auth.bind and servers.bind. An example command to query this, using the dig utility would be dig +short chaos txt cachesize.bind It would be very interesting to see the differences between dnsmasq without DNSSEC, with DNSSEC and with DNSSEC and --dnssec-check-unsigned (checking for proof of non-existence) - we've been a bit concerned about the overheads of the last in particular. Getting more elaborate stats (hit, miss, and fill costs) is under discussio= n. > We've open sourced the web model (it's on Kathie's > web page and will be part of ns2.36) with an encouragement to the > community to improve on it. If you'd like to port it to ns3 and add a dn= s > model, that would be fantastic. As part of the google summer of code I am signed up to mentor a student with tom for the *codel related bits in ns3, and certainly plan to get fingers dirty in the cablelabs drop, and there was a very encouraging patch set distributed around for tcp-cubic with hystart support recently as well as a halfway decent 802.11 mac emulation. As usual, I have no funding, personally, to tackle the job, but I'll do what I can anyway. It would be wonderful to finally have all the ns2 and ns3 code mainlined for more people to use it. --=20 Dave T=C3=A4ht From nobody Fri Apr 18 12:13:51 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EA46A1A04A1 for ; Fri, 18 Apr 2014 12:13:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.664 X-Spam-Level: ** X-Spam-Status: No, score=2.664 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, RP_MATCHES_RCVD=-0.272] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g3pT528rKtIp for ; Fri, 18 Apr 2014 12:13:45 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id A65E91A0471 for ; Fri, 18 Apr 2014 12:13:45 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id s3IJDdT6022407; Fri, 18 Apr 2014 13:13:39 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 18 Apr 2014 13:13:39 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0174.001; Fri, 18 Apr 2014 13:13:38 -0600 From: Greg White To: Dave Taht Thread-Topic: [aqm] [AQM Evaluation Guidelines] Thread-Index: AQHPI/vsfsEKmiUbyESwqMuBEFOkv5q+LAwAgFT4noCAAPEzAIAAsxYAgAB0kwCAAu8dgA== Date: Fri, 18 Apr 2014 19:13:37 +0000 Message-ID: References: <52CDAAF7.2060307@mti-systems.com> <012C3117EDDB3C4781FD802A8C27DD4F2602EB77@SACEXCMBX02-PRD.hq.netapp.com> <0D6E78FE-D859-4C44-8611-7966FA1D3859@telecom-bretagne.eu> <1840EEB0-7079-4C03-ACE1-EA828225973F@telecom-bretagne.eu> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.4.10.154] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/rC8gWFtX2Z-zMB1RKHLeq9rfeJg Cc: "Akhtar, Shahid \(Shahid\)" , Richard Scheffenegger , "aqm@ietf.org" , Nicolas KUHN Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 19:13:50 -0000 On 4/16/14, 10:25 AM, "Dave Taht" wrote: >On Wed, Apr 16, 2014 at 8:28 AM, Greg White wrote: >> >> I agree with others that reducing the web model to a single 700kB file >> download is unrealistic. Also, for recent work we've increased the >>object >> sizes to correspond to a projected 2018 page size (extrapolating from >> http://httparchive.org/trends.php) of about 7MB. > >While the trendline is impressively "up": > >http://httparchive.org/trends.php?s=3DAll&minlabel=3DNov+15+2010&maxlabel= =3DApr+ >1+2014 > >I'm reluctant to do a linear extrapolation of that trendline from 3 >years of data. Perhaps >a longer term store, like that of archive.org, could be explored. I'm >not one to predict >a "Peak Web", but the idea of a typical web page load cracking 7MB in >5 years disturbs >me. > >> >> For our 2018 model of Internet traffic (again, upstream focused) where >>the > >I do like, very much, trying to come up with projections for what traffic >will >look like in 5 years. I'll ask around... Yes, it would be interesting to get some other views on this. >> T: number of torrent (LEDBAT) connections > >Torrent does not behave as per your model; I just had a long meeting >with brahm over that. Presently per "new" torrent you typically see >dozens of >sessions "connected", but only 6 or so sessions exchanging data in >both directions, >with a new connection explored and rotated to every 15 seconds or so. > >After a file is fully transferred, the same logic remains, but upload >only. >Anticipating that a given site would have 16 simultaneous torrents over >a 100 active connections being redistributed seems overlarge, and I >imagine >that a typical number is bound much lower by the bittorrent software. 6 sessions * 16 torrents seems to end up pretty close to 100 total sessions. Am I missing something? I neglected to mention that the torrent traffic is limited to 50% of the user's upstream data rate, based on evidence (like you suggest) that torrent clients by default limit their traffic, and it is apparently well known that running it unlimited is not recommended (probably due partly to bufferbloat, but once established, the mindset might still remain even after bufferbloat is eliminated). >> *Filesize and repetition pattern chosen to exercise DOCSIS "powerboost" >> feature > >Yes, I'd like not only to have that but to actually have running code in >linux that could emulate it... :) The code we've released for the ns2 model is available if you want to use that as a start. Or, I can walk you through the math if you'd rather. >> Since evaluation comes before deployment (by a long lead time in some >> cases), and, once deployed, algorithms may have a long life, I would >> encourage the AQM Evaluation Guidelines draft to consider not just what >> traffic looks like historically, but what we can reasonably predict for >>the >> future. > >+1 > >Now that we are transporting hi resolution video, voice, web, and data >in addition to the traditional email and chat, I have great difficulty >assuming >that bandwidth demands will continue to grow as they have, as the new >stuff >is mostly "fixed" in its bandwidth needs, rather than dynamic like tcp. > >... Unless someone comes up with a way of encapsulating other senses or >some application that truly demands high bandwidth that can work at >multiple RTTs. Yes, I hear you and don't disagree. On the other hand it is hard to argue strongly against 30+ years of rock-solid consistent 50% annual increase... Plus, I'm probably not the only one who can remember at one point thinking "I'll never fill a 20 MB hard drive, why, that's 20 *million* bytes!" >There is a lot downward pressure to "do more with less", for example, >as with things like webrtc that are bundled in the browser and don't >require a lot of javascript support, and a lot of caching of core code >that takes place, and a lot of work in reducing RTTs in new protocols >like QUIC. > >(but I freely admit I could be wrong and would love to work on projections > that make sense!) From nobody Fri Apr 18 12:28:04 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C78571A0495 for ; Fri, 18 Apr 2014 12:28:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.965 X-Spam-Level: **** X-Spam-Status: No, score=4.965 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, MANGLED_SIDE=2.3, RP_MATCHES_RCVD=-0.272] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vHb6-2RxBI-a for ; Fri, 18 Apr 2014 12:27:56 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 9A6171A0182 for ; Fri, 18 Apr 2014 12:27:56 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id s3IJRkrt023743; Fri, 18 Apr 2014 13:27:46 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 18 Apr 2014 13:27:45 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0174.001; Fri, 18 Apr 2014 13:27:45 -0600 From: Greg White To: "dpreed@reed.com" Thread-Topic: [Cerowrt-devel] [aqm] chrome web page benchmarker fixed Thread-Index: AQHPWmVePgFvJCotnEiEX+XB9sZQUJsWjr8AgAAjHoCAAP2yAIAAbbYA//+mfQA= Date: Fri, 18 Apr 2014 19:27:44 +0000 Message-ID: References: <1397846888.714812176@apps.rackspace.com> In-Reply-To: <1397846888.714812176@apps.rackspace.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.4.10.154] Content-Type: multipart/alternative; boundary="_000_CF76D02032E2Dgwhitecablelabscom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/SvKdgCCJNIDvTpRoJfOBeDGLlfw Cc: bloat , "aqm@ietf.org" , Dave Taht , "cerowrt-devel@lists.bufferbloat.net" , =?utf-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= Subject: Re: [aqm] [Cerowrt-devel] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 19:28:03 -0000 --_000_CF76D02032E2Dgwhitecablelabscom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SW4gbW9zdCBjYXNlcyBpdCB3YXMgKmZhciogbGVzcyB0aGFuIDElLiAgSW4gb3VyIERPQ1NJUyAz LjEgc2NlbmFyaW9zIGl0IGlzIG1vcmUgbGlrZSAxZS01IHRvIDFlLTQgKDFlLTMgd29yc3QgY2Fz ZSBmb3IgdGhlIGJ1ZmZlcmJsb2F0ZWQgZHJvcCB0YWlsIGJhc2VsaW5lKSB3aGljaCBtZWFucyB0 aGF0IGltcGxlbWVudGluZyBkbnMgaW4gdGhlIG1vZGVsIHdvdWxkIGhhdmUgYWZmZWN0ZWQgb24g dGhlIG9yZGVyIG9mIDEgb3V0IG9mIGV2ZXJ5IDI1MDAgcGFnZSBsb2Fkcy4NCg0KQXMgd2l0aCBh bnkgbW9kZWwgb2YgcmVhbGl0eSwgdGhlcmUgaXMgYSB0cmFkZW9mZiBiZXR3ZWVuIGNvbXBsZXhp dHkgYW5kIGZpZGVsaXR5LCByaWdodD8gV2UgY2hvc2UgNCBzZXJ2ZXJzIGFzIGJlaW5nIGEgcmVh c29uYWJsZSBhbmQgcHJhY3RpY2FsIG51bWJlci4gIE1hbnkgb2YgdGhlIG1vc3QgcG9wdWxhciBz aXRlcyBhY3R1YWxseSBkb24ndCBkaXN0cmlidXRlIGNvbnRlbnQgdGhhdCB3aWRlbHkgKGUuZy4g R29vZ2xlIG9ubHkgdXNlZCAzIHNlcnZlcnMgaW4gb3VyIG1lYXN1cmVtZW50cyksIGFuZCB0aGUg d29yc3QgY2FzZSB3YXMgMzIgc2VydmVycy4gICBJbiByZWFsaXR5IGV2ZXJ5IHdlYiBwYWdlIGlz IGRpZmZlcmVudCwgYW5kIHlvdSBjb3VsZCBjb21lIHVwIHdpdGggYW4gYXJiaXRyYXJpbHkgY29t cGxleCBtb2RlbCBhbmQgaXQgc3RpbGwgd291bGRuJ3QgbWF0Y2ggc29tZSB2ZXJzaW9uIG9mIHJl YWxpdHkuICBJbiBteSB2aWV3LCB3ZSB3YW50IHRvIGhhdmUgc29tZXRoaW5nIHRoYXQgaXMgcHJh Y3RpY2FsIHRvIGltcGxlbWVudCBhbmQgY2xvc2UgZW5vdWdoIHRvIHJlYWxpdHkgdG8gYmUgdXNl ZnVsLiAgSW4gdGhlIEFRTS1ldmFsdWF0aW9uIGRyYWZ0IGN1cnJlbnRseSwgdGhlIG1vZGVsIGlz IGEgc2luZ2xlIDcwMGtCIGZpbGUgc2VydmVkIGZyb20gYSBzaW5nbGUgc2VydmVyLg0KDQpHcmVn DQoNCkZyb206ICJkcHJlZWRAcmVlZC5jb208bWFpbHRvOmRwcmVlZEByZWVkLmNvbT4iIDxkcHJl ZWRAcmVlZC5jb208bWFpbHRvOmRwcmVlZEByZWVkLmNvbT4+DQpEYXRlOiBGcmlkYXksIEFwcmls IDE4LCAyMDE0IGF0IDEyOjQ4IFBNDQpUbzogR3JlZyBXaGl0ZSA8Zy53aGl0ZUBjYWJsZWxhYnMu Y29tPG1haWx0bzpnLndoaXRlQGNhYmxlbGFicy5jb20+Pg0KQ2M6IERhdmUgVGFodCA8ZGF2ZS50 YWh0QGdtYWlsLmNvbTxtYWlsdG86ZGF2ZS50YWh0QGdtYWlsLmNvbT4+LCAiV2lsbGlhbSBDaGFu ICjpmYjmmbrmmIwpIiA8d2lsbGNoYW5AY2hyb21pdW0ub3JnPG1haWx0bzp3aWxsY2hhbkBjaHJv bWl1bS5vcmc+PiwgImFxbUBpZXRmLm9yZzxtYWlsdG86YXFtQGlldGYub3JnPiIgPGFxbUBpZXRm Lm9yZzxtYWlsdG86YXFtQGlldGYub3JnPj4sICJjZXJvd3J0LWRldmVsQGxpc3RzLmJ1ZmZlcmJs b2F0Lm5ldDxtYWlsdG86Y2Vyb3dydC1kZXZlbEBsaXN0cy5idWZmZXJibG9hdC5uZXQ+IiA8Y2Vy b3dydC1kZXZlbEBsaXN0cy5idWZmZXJibG9hdC5uZXQ8bWFpbHRvOmNlcm93cnQtZGV2ZWxAbGlz dHMuYnVmZmVyYmxvYXQubmV0Pj4sIGJsb2F0IDxibG9hdEBsaXN0cy5idWZmZXJibG9hdC5uZXQ8 bWFpbHRvOmJsb2F0QGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldD4+DQpTdWJqZWN0OiBSZTogW0Nlcm93 cnQtZGV2ZWxdIFthcW1dIGNocm9tZSB3ZWIgcGFnZSBiZW5jaG1hcmtlciBmaXhlZA0KDQoNCldo eSBpcyB0aGUgRE5TIFBMUiBzbyBoaWdoPyAgMSUgaXMgcHJldHR5IGRlcHJlc3NpbmcuDQoNCg0K DQpBbHNvLCBpdCBzZWVtcyBvZGQgdG8gZWxpbWluYXRlIDE5JSBvZiB0aGUgY29udGVudCByZXRy aWV2YWwgYmVjYXVzZSB0aGUgdGFpbCBpcyBmYXQgYW5kIGxvbmcgcmF0aGVyIHRoYW4gc2hvcnQu ICBXb3VsZG4ndCBpdCBiZSBiZXR0ZXIgdG8gaGF2ZSAxMDAwIHNlcnZlcnM/DQoNCg0KDQoNCg0K DQpPbiBGcmlkYXksIEFwcmlsIDE4LCAyMDE0IDI6MTVwbSwgIkdyZWcgV2hpdGUiIDxnLndoaXRl QENhYmxlTGFicy5jb208bWFpbHRvOmcud2hpdGVAQ2FibGVMYWJzLmNvbT4+IHNhaWQ6DQoNCg0K PiBEYXZlLA0KPg0KPiBXZSB1c2VkIHRoZSAyNWsgb2JqZWN0IHNpemUgZm9yIGEgc2hvcnQgdGlt ZSBiYWNrIGluIDIwMTIgdW50aWwgd2UgaGFkDQo+IHJlc291cmNlcyB0byBidWlsZCBhIG1vcmUg YWR2YW5jZWQgbW9kZWwgKGFwcGVuZGl4IEEpLiBJIGRpZCBhIGJ1bmNoIG9mDQo+IGNhcHR1cmVz IG9mIHJlYWwgd2ViIHBhZ2VzIGJhY2sgaW4gMjAxMSBhbmQgY29tcGFyZWQgdGhlIG9iamVjdCBz aXplDQo+IHN0YXRpc3RpY3MgdG8gbW9kZWxzIHRoYXQgSSdkIHNlZW4gcHVibGlzaGVkLiBMb2du b3JtYWwgZGlkbid0IHNlZW0gdG8gYmUNCj4gKmV4YWN0bHkqIHJpZ2h0LCBidXQgaXQgd2Fzbid0 IGEgYmFkIGZpdCB0byB3aGF0IEkgc2F3LiBJJ3ZlIGF0dGFjaGVkIGENCj4gQ0RGLg0KPg0KPiBU aGUgY2hvaWNlIG9mIDQgc2VydmVycyB3YXMgYmFzZWQgc29tZXdoYXQgb24gbG9naXN0aWNzLCBh bmQgYWxzbyBvbiBhDQo+IGZpbmRpbmcgdGhhdCBhY3Jvc3Mgb3VyIGRhdGEgc2V0LCB0aGUgYXZl cmFnZSB3ZWIgcGFnZSByZXRyaWV2ZWQgODElIG9mDQo+IGl0cyByZXNvdXJjZXMgZnJvbSB0aGUg dG9wIDQgc2VydmVycy4gSW5jcmVhc2luZyB0byA1IHNlcnZlcnMgb25seQ0KPiBpbmNyZWFzZWQg dGhhdCBwZXJjZW50YWdlIHRvIDg0JS4NCj4NCj4gVGhlIGNob2ljZSBvZiBSVFRzIGFsc28gY2Ft ZSBmcm9tIHRoZSB3ZWIgdHJhZmZpYyBjYXB0dXJlcy4gSSBzYXcNCj4gUlRUbWluPTE2bXMsIFJU VG1lYW49NTMuOG1zLCBSVFRtYXg9MTM0bXMuDQo+DQo+IE11Y2ggb2YgdGhpcyBjYW4gYmUgZm91 bmQgaW4NCj4gaHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LXdoaXRlLWh0dHBiaXMt c3BkeS1hbmFseXNpcy0wMA0KPg0KPiBJbiBtYW55IG9mIHRoZSBjYXNlcyB0aGF0IHdlJ3ZlIHNp bXVsYXRlZCwgdGhlIHBhY2tldCBkcm9wIHByb2JhYmlsaXR5IGlzDQo+IGxlc3MgdGhhbiAxJSBm b3IgRE5TIHBhY2tldHMuIEluIG91ciB3ZWIgbW9kZWwsIHRoZXJlIGFyZSBhIHRvdGFsIG9mIDQN Cj4gc2VydmVycywgc28gNCBETlMgbG9va3VwcyBhc3N1bWluZyBub25lIG9mIHRoZSBhZGRyZXNz ZXMgYXJlIGNhY2hlZC4gSWYNCj4gUExSID0gMSUsIHRoZXJlIHdvdWxkIGJlIGEgMy45JSBjaGFu Y2Ugb2YgbG9zaW5nIG9uZSBvciBtb3JlIEROUyBwYWNrZXRzDQo+ICh3aXRoIGEgcmVzdWx0aW5n IH41IHNlY29uZCBhZGRpdGlvbmFsIGRlbGF5IG9uIGxvYWQgdGltZSkuIEkndmUgcHJvYmFibHkN Cj4gb3ZlcnNpbXBsaWZpZWQgdGhpcywgYnV0IEthdGhpZSBOLiBhbmQgSSBtYWRlIHRoZSBjYWxs IHRoYXQgaXQgd291bGQgYmUNCj4gc2lnbmlmaWNhbnRseSBlYXNpZXIgdG8ganVzdCBkbyB0aGlz IG1hdGggdGhhbiB0byBidWlsZCBhIGRucw0KPiBpbXBsZW1lbnRhdGlvbiBpbiBuczIuIFdlJ3Zl IG9wZW4gc291cmNlZCB0aGUgd2ViIG1vZGVsIChpdCdzIG9uIEthdGhpZSdzDQo+IHdlYiBwYWdl IGFuZCB3aWxsIGJlIHBhcnQgb2YgbnMyLjM2KSB3aXRoIGFuIGVuY291cmFnZW1lbnQgdG8gdGhl DQo+IGNvbW11bml0eSB0byBpbXByb3ZlIG9uIGl0LiBJZiB5b3UnZCBsaWtlIHRvIHBvcnQgaXQg dG8gbnMzIGFuZCBhZGQgYSBkbnMNCj4gbW9kZWwsIHRoYXQgd291bGQgYmUgZmFudGFzdGljLg0K Pg0KPiAtR3JlZw0KPg0KPg0KPiBPbiA0LzE3LzE0LCAzOjA3IFBNLCAiRGF2ZSBUYWh0IiA8ZGF2 ZS50YWh0QGdtYWlsLmNvbTxtYWlsdG86ZGF2ZS50YWh0QGdtYWlsLmNvbT4+IHdyb3RlOg0KPg0K PiA+T24gVGh1LCBBcHIgMTcsIDIwMTQgYXQgMTI6MDEgUE0sIFdpbGxpYW0gQ2hhbiAo6ZmI5pm6 5piMKQ0KPiA+PHdpbGxjaGFuQGNocm9taXVtLm9yZzxtYWlsdG86d2lsbGNoYW5AY2hyb21pdW0u b3JnPj4gd3JvdGU6DQo+ID4+IFNwZWFraW5nIGFzIHRoZSBwcmltYXJ5IENocm9taXVtIGRldmVs b3BlciBpbiBjaGFyZ2Ugb2YgdGhpcyByZWxldmFudA0KPiA+PmNvZGUsDQo+ID4+IEkgd291bGQg bGlrZSB0byBjYXV0aW9uIHB1dHRpbmcgdG9vIG11Y2ggdHJ1c3QgaW4gdGhlIG51bWJlcnMNCj4g Pj5nZW5lcmF0ZWQuIEFueQ0KPiA+PiBzdGF0aXN0aWNhbCBjbGFpbXMgYWJvdXQgdGhlIG51bWJl cnMgYXJlIHByb2JhYmx5IHVucmVhc29uYWJsZSB0byBtYWtlLg0KPiA+DQo+ID5TaWdoLiBPdGhl ciBiZW5jaG1hcmtzIHN1Y2ggYXMgdGhlIGFwYWNoZSAoImFiIikgYmVuY2htYXJrDQo+ID5hcmUg cHJpbWFyaWx5IGRlc2lnbmVkIGFzIHN0cmVzcyB0ZXN0ZXJzIGZvciB3ZWIgc2VydmVycywgbm90 IGFzIHJlYWxpc3RpYw0KPiA+dHJhZmZpYy4gTW9kZXJuIHdlYiB0cmFmZmljIGhhcyBzdWNoIGEg aGlnaCBsZXZlbCBvZiBkeW5hbWljaXNtIGluIGl0LA0KPiA+dGhhdCBzdGF0aWMgd2ViIHBhZ2Ug bG9hZHMgYWxvbmcgYW55IGRpc3RyaWJ1dGlvbiwgc2VlbSBpbnN1ZmZpY2llbnQsDQo+ID5wYXNz aXZlIGFuYWx5c2lzIG9mIGFnZ3JlZ2F0ZWQgdHJhZmZpYyAiZmVlbHMiIGluY29ycmVjdCByZWxh dGl2ZSB0byB0aGUNCj4gPnNvcnRzIG9mIGhvbWUgYW5kIHNtYWxsIGJ1c2luZXNzIHRyYWZmaWMg SSd2ZSBzZWVuLCBhbmQgc28gb24uDQo+ID4NCj4gPkZhbW91cyBwYXBlcnMsIHN1Y2ggYXMgdGhp cyBvbmU6DQo+ID4NCj4gPmh0dHA6Ly9jY3Iuc2lnY29tbS5vcmcvYXJjaGl2ZS8xOTk1L2phbjk1 L2Njci05NTAxLWxlbGFuZC5wZGYNCj4gPg0KPiA+U2VlbSBwb3NzaWJseSBpcnJlbGV2YW50IHRv IGRyYXcgY29uY2x1c2lvbnMgZnJvbSBnaXZlbiB0aGUga2luZA0KPiA+b2YgZGF0YSB0aGV5IGFu YWx5c2VkIGFuZCBwcm9jZWVkaW5nIGZyb20gYW4gaW5jb3JyZWN0IG1vZGVsIG9yDQo+ID5ndXQg ZmVlbCBmb3IgaG93IHRoZSBuZXR3b3JrIGJlaGF2ZXMgdG9kYXkgc2VlbXMgdG8gYmUgZm9vbGlz aC4NCj4gPg0KPiA+RXZlbiB0aGUgbW9zdCBiYXNpYyBvZiB0b29scywgc3VjaCBhcyBodHRwaW5n LCBoYWQgdGhyZWUgYmFzaWMgYnVncw0KPiA+dGhhdCBJIGZvdW5kIGluIGEgZmV3IG1pbnV0ZXMg b2YgdHJ5aW5nIHRvIGNvbWUgdXAgd2l0aCBzb21lIGJhc2ljDQo+ID5iZWhhdmlvcnMgeWVzdGVy ZGF5Og0KPiA+DQo+ID5odHRwczovL2xpc3RzLmJ1ZmZlcmJsb2F0Lm5ldC9waXBlcm1haWwvYmxv YXQvMjAxNC1BcHJpbC8wMDE4OTAuaHRtbA0KPiA+DQo+ID5UaG9zZSBhcmUgZ29pbmcgdG8gYmUg YSBsb3QgZWFzaWVyIHRvIGZpeCB0aGFuIGRpdmluZyBpbnRvIHRoZSBjaHJvbWl1bQ0KPiA+Y29k ZWJhc2UhDQo+ID4NCj4gPlRoZXJlIGFyZSB2ZXJ5IGZldyB0b29scyB3b3J0aCB0cnVzdGluZywg YW5kIEkgYW0gYWx3YXlzIGR1YmlvdXMNCj4gPm9mIHBhcGVycyB0aGF0IHB1Ymxpc2ggcmVzdWx0 cyB3aXRoIHVuYXZhaWxhYmxlIHRvb2xzIGFuZCBkYXRhLiBUaGUgb25seQ0KPiA+dG9vbHMgSSBo YXZlIGFueSBmYWl0aCBpbiBmb3IgbmV0d29yayBhbmFseXNpcyBhcmUgbmV0cGVyZiwNCj4gPm5l dHBlcmYtd3JhcHBlciwNCj4gPnRjcGR1bXAgYW5kIHhwbG90Lm9yZywgYW5kIHRvIGEgbGFyZ2Ug ZXh0ZW50IHdpcmVzaGFyay4gVG9rZSBhbmQgSSBoYXZlDQo+ID5iZWVuIHRlYXJpbmcgYXBhcnQg ZC1pdGcgYW5kIEkgaG9wZSB0byBvbmUgZGF5IGJlIGFibGUgdG8gYWRkIHRoYXQgdG8NCj4gPm15 IHRydXN0YWJsZSBsaXN0Li4uIGJ1dCBiZXR0ZXIgdG9vbHMgYXJlIG5lZWRlZCENCj4gPg0KPiA+ VG9vbHMgdGhhdCBJIGRvbid0IGhhdmUgYSBsb3Qgb2YgZmFpdGggaW4gaW5jbHVkZSB0aGF0LCBp cGVyZiwgYW55dGhpbmcNCj4gPndyaXR0ZW4NCj4gPmluIGphdmEgb3Igb3RoZXIgaGlnaCBsZXZl bCBsYW5ndWFnZXMsIHNwZWVkdGVzdC5uZXQsIGFuZCB0aGluZ3MgbGlrZQ0KPiA+c2hhcGVycHJv YmUuDQo+ID4NCj4gPkhhdmUgdmVyeSBsaXR0bGUgZmFpdGggaW4gbnMyLCBzbGlnaHRseSBtb3Jl IGluIG5zMywgYW5kIEkndmUgYmVlbiBtZWFuaW5nDQo+ID50byBsb29rIG92ZXIgdGhlIG1pbmlu ZXQgYW5kIG90aGVyIHNpbXVsYXRvcnMgd2hlbmV2ZXIgSSBnb3Qgc29tZSBzcGFyZQ0KPiA+dGlt ZTsgdGhlIG1pbmluZXQgcmVzdWx0cyBzdGFuZm9yZCBnZXRzIHNlZW0gcHJldHR5IHJlYXNvbmFi bGUgYW5kIEkNCj4gPmFkb3JlIHRoZWlyIHJlcHJvZHVjaW5nIHJlc3VsdHMgZWZmb3J0LiBIYXZl bid0IGV4cGxvcmVkIG5kdCwga2VlcCBtZWFuaW5nDQo+ID50by4uLg0KPiA+DQo+ID4+IFJlYXNv bnM6DQo+ID4+ICogV2UgZG9uJ3QgYWN0aXZlbHkgbWFpbnRhaW4gdGhpcyBjb2RlLiBJdCdzIGJl aGluZCB0aGUgY29tbWFuZCBsaW5lDQo+ID4+ZmxhZ3MuDQo+ID4+IFRoZXkgYXJlIGJyb2tlbi4g VGhlIGZhY3QgdGhhdCBpdCBzdGlsbCByZXN1bHRzIGluIG51bWJlcnMgb24gdGhlDQo+ID4+YmVu Y2htYXJrDQo+ID4+IGV4dGVuc2lvbiBpcyBhbiBleGFtcGxlIG9mIHdoZXJlIHVubWFpbnRhaW5l ZCBjb2RlIGRvZXNuJ3QgaGF2ZSB0aGUgVUkNCj4gPj4gZGlzYWJsZWQsIGV2ZW4gdGhvdWdoIHRo ZSBpbnRlcm5hbCB3b3JraW5ncyBvZiB0aGUgY29kZSBmYWlsIHRvDQo+ID4+Z3VhcmFudGVlDQo+ ID4+IGNvcnJlY3Qgb3BlcmF0aW9uLiBXZSBoYXZlbid0IGRpc2FibGVkIGl0IGJlY2F1c2UsIHdl bGwsIGl0J3MNCj4gPj51bm1haW50YWluZWQuDQo+ID4NCj4gPkFzIEkgbWVudGlvbmVkIEkgd2Fz IGdlYXJpbmcgdXAgZm9yIGEgaGFja2luZyBydW4uLi4NCj4gPg0KPiA+VGhlIHZhc3QgbWFqb3Jp dHkgb2YgcmVzdWx0cyBJIGxvb2sgYXQgYXJlIGFjdHVhbGx5IG9idGFpbmVkIHZpYQ0KPiA+bG9v a2luZyBhdCBwYWNrZXQgY2FwdHVyZXMuIEkgbW9zdGx5IHVzZSBiZW5jaG1hcmtzIGFzIGFic3Ry YWN0aW9ucw0KPiA+dG8gc2VlIGlmIHRoZXkgbWFrZSBzb21lIHNlbnNlIHJlbGF0aXZlIHRvIHRo ZSBjYXB0dXJlcyBhbmQgdGVuZA0KPiA+dG8gY29tcGFyZSBkaWZmZXJlbnQgYmVuY2htYXJrcyBh Z2FpbnN0IGVhY2ggb3RoZXIuDQo+ID4NCj4gPkkgcmVhbGl6ZSBvdGhlcnMgZG9uJ3QgZ28gaW50 byB0aGF0IGxldmVsIG9mIGRldGFpbCwgc28geW91IGhhdmUgZ2l2ZW4NCj4gPmZhaXIgd2Fybmlu ZyEgSW4gb3VyIGNhc2Ugd2UgdXNlZCB0aGUgd2ViIHBhZ2UgYmVuY2htYXJrZXIgYXMNCj4gPmEg bWVhbnMgdG8gdHJ5IGFuZCByYXBpZGx5IGFjcXVpcmUgc29tZSB0eXBpY2FsIGRpc3RyaWJ1dGlv bnMgb2YNCj4gPmdldCBhbmQgdGNwIHN0cmVhbSByZXF1ZXN0cyBmcm9tIHRoaW5ncyBsaWtlIHRo ZSBhbGV4YSB0b3AgMTAwMCwNCj4gPmFuZCBhcyBhIHdheSB0byBBL0IgZGlmZmVyZW50IGFxbS9w YWNrZXQgc2NoZWR1bGluZyBzZXR1cHMuDQo+ID4NCj4gPi4uLiBidXQgdGhlIG9ubHkgZWFzaWx5 IHB1Ymxpc2hhYmxlIHJlc3VsdHMgd2VyZSBmcm9tIHRoZSBiZW5jaG1hcmsNCj4gPml0c2VsZiwN Cj4gPmFuZCB3ZSAocmVsdWN0YW50bHkpIG9ubHkgcHVibGlzaGVkIG9uZSBncmFwaCBmcm9tIGFs bCB0aGUgd29yayB0aGF0DQo+ID53ZW50IGludG8gaXQgMisgeWVhcnMgYmFjayBhbmQgdXNlZCBp dCBhcyBhIHRlc3QgZHJpdmVyIGZvciB0aGUgZmFtb3VzDQo+ID5pZXRmIHZpZGVvLCBjb21wYXJp bmcgdHdvIGlkZW50aWNhbCBib3hlcyBydW5uaW5nIGl0IGF0IHRoZSBzYW1lIHRpbWUNCj4gPnVu ZGVyIGRpZmZlcmVudCBuZXR3b3JrIGNvbmRpdGlvbnM6DQo+ID4NCj4gPmh0dHBzOi8vd3d3LmJ1 ZmZlcmJsb2F0Lm5ldC9wcm9qZWN0cy9jZXJvd3J0L3dpa2kvQmxvYXQtdmlkZW9zI0lFVEYtZGVt by1zDQo+ID5pZGUtYnktc2lkZS1vZi1hLW5vcm1hbC1jYWJsZS1tb2RlbS12cy1mcV9jb2RlbA0K PiA+DQo+ID5mcm9tIHdoYXQgSSBmaWRkbGVkIHdpdGggdG9kYXksIGl0IGlzIGF0IGxlYXN0IHN0 aWxsIHVzZWZ1bCBmb3IgdGhhdD8NCj4gPg0KPiA+bW92aW5nIG9uLi4uDQo+ID4NCj4gPlRoZSB3 ZWIgbW9kZWwgaW4gdGhlIGNhYmxlbGFicyB3b3JrIGRvZXNuJ3QgbG9vayBtdWNoIGxpa2UgbXkg Y2FwdHVyZXMsDQo+ID5pbiBhZGRpdGlvbiB0byBub3QgbW9kZWxpbmcgZG5zIGF0IGFsbCwgYW5k IHVzaW5nIGEgc21hbGxlciBJVyB0aGFuIGdvb2dsZQ0KPiA+aXQgbG9va3MgbGlrZSB0aGlzOg0K PiA+DQo+ID4+PiBNb2RlbCBzaW5nbGUgdXNlciB3ZWIgcGFnZSBkb3dubG9hZCBhcyBmb2xsb3dz Og0KPiA+DQo+ID4+PiAtIFdlYiBwYWdlIG1vZGVsZWQgYXMgc2luZ2xlIEhUTUwgcGFnZSArIDEw MCBvYmplY3RzIHNwcmVhZCBldmVubHkNCj4gPj4+IGFjcm9zcyA0IHNlcnZlcnMuIFdlYiBvYmpl Y3Qgc2l6ZXMgYXJlIGN1cnJlbnRseSBmaXhlZCBhdCAyNSBrQg0KPiBlYWNoLA0KPiA+Pj4gd2hl cmVhcyB0aGUgaW5pdGlhbCBIVE1MIHBhZ2UgaXMgMTAwIGtCLiBBcHBlbmRpeCBBIHByb3ZpZGVz IGFuDQo+ID4+PiBhbHRlcm5hdGl2ZSBwYWdlIG1vZGVsIHRoYXQgbWF5IGJlIGV4cGxvcmVkIGlu IGZ1dHVyZSB3b3JrLg0KPiA+DQo+ID5XaGVyZSB3aGF0IEkgc2VlIGlzIGEgaHVnZSBudW1iZXIg b2Ygc3R1ZmYgdGhhdCBmaXRzIGludG8gYSBzaW5nbGUNCj4gPml3MTAgc2xvdyBzdGFydCBlcGlz b2RlIGFuZCBzb21lIGxldmVsIG9mIHBpcGVsaW5pbmcgb24gbGFyZ2VyIHN0dWZmLCBzbw0KPiA+ dGhhdCBhDQo+ID5sYXJnZSBudW1iZXIgb2Ygb2JqZWN0IHNpemVzIG9mIGxlc3MgdGhhbiA3ayB3 aXRoIGEgbGlnaHRseSB0YWlsZWQNCj4gPmRpc3RyaWJ1dGlvbg0KPiA+b3V0c2lkZSBvZiB0aGF0 IG1ha2VzIG1vcmUgc2Vuc2UuDQo+ID4NCj4gPihJJ20gbm90IHN0YXJpbmcgYXQgYXBwZW5kaXgg QSByaWdodCBub3csIEknbSB1bmRlciB0aGUgaW1wcmVzc2lvbg0KPiA+IGl0IHdhcyBiZXR0ZXIp DQo+ID4NCj4gPkkgY2VydGFpbmx5IHdvdWxkIGxpa2UgbW9yZSBzdWdnZXN0aW9ucyBmb3IgbW9k ZWxzIGFuZCB0eXBlcw0KPiA+b2Ygd2ViIHRyYWZmaWMsIGFzIHdlbGwgYXMgc2ltdWxhdGlvbiBv ZiBodHRwcyArIHBmcyB0cmFmZmljLA0KPiA+c3BkeSwgcXVpYywgZXRjLi4uLg0KPiA+DQo+ID4+ PiAtIFNlcnZlciBSVFRzIHNldCBhcyBmb2xsb3dzICgyMCBtcywgMzAgbXMsIDUwIG1zLCAxMDAg bXMpLg0KPiA+DQo+ID5TZXJ2ZXIgUlRUcyBmcm9tIG15IG93biB3ZWIgaGlzdG9yeSB0ZW5kIHRv IGJlIGxvd2VyIHRoYW4gNTBtcy4NCj4gPg0KPiA+Pj4gLSBJbml0aWFsIEhUVFAgR0VUIHRvIHJl dHJpZXZlIGEgbW9kZXJhdGVseSBzaXplZCBvYmplY3QgKDEwMCBrQg0KPiBIVE1MDQo+ID4+PiBw YWdlKSBmcm9tIHNlcnZlciAxLg0KPiA+DQo+ID5BbiBpbml0aWFsIEdFVCB0byBnb29nbGUgZml0 cyBpbnRvIGl3MTAgLSBpdCdzIGFib3V0IDdrLg0KPiA+DQo+ID4+PiAtIE9uY2UgaW5pdGlhbCBI VFRQIEdFVCBjb21wbGV0ZXMsIGluaXRpYXRlIDI0IHNpbXVsdGFuZW91cyBIVFRQDQo+IEdFVHMN Cj4gPj4+ICh2aWEgc2VwYXJhdGUgVENQIGNvbm5lY3Rpb25zKSwgNiBjb25uZWN0aW9ucyBlYWNo IHRvIDQgZGlmZmVyZW50DQo+ID4+PiBzZXJ2ZXIgbm9kZXMNCj4gPg0KPiA+SSB1c3VhbGx5IGRv bid0IHNlZSBtb3JlIHRoYW4gMTUuIGFuZCBjZXJ0YWlubHkgbm90IDI1ayBzaXplZCBvYmplY3Rz Lg0KPiA+DQo+ID4gPiAtIE9uY2UgZWFjaCBpbmRpdmlkdWFsIEhUVFAgR0VUIGNvbXBsZXRlcywg aW5pdGlhdGUgYSBzdWJzZXF1ZW50IEdFVA0KPiA+PiB0byB0aGUgc2FtZSBzZXJ2ZXIsIHVudGls IDI1IG9iamVjdHMgaGF2ZSBiZWVuIHJldHJpZXZlZCBmcm9tIGVhY2gNCj4gPj4gc2VydmVyLg0K PiA+DQo+ID4NCj4gPj4gKiBXZSBkb24ndCBtYWtlIHN1cmUgdG8gZmx1c2ggYWxsIHRoZSBuZXR3 b3JrIHN0YXRlIGluIGJldHdlZW4gcnVucywgc28NCj4gPj5pZg0KPiA+PiB5b3UncmUgdXNpbmcg dGhhdCBvcHRpb24sIGRvbid0IHRydXN0IGl0IHRvIHdvcmsuDQo+ID4NCj4gPlRoZSB0eXBpY2Fs IHNjZW5hcmlvIHdlIHVzZWQgd2FzIGEgcnVuIGFnYWluc3QgZG96ZW5zIG9yIGh1bmRyZWRzIG9m IHVybHMsDQo+ID5jYXB0dXJpbmcgdHJhZmZpYywgd2hpbGUgdmFyeWluZyBuZXR3b3JrIGNvbmRp dGlvbnMuDQo+ID4NCj4gPlJlZ2FyZGVkIHRoZSBmaXJzdCBydW4gYXMgdGhlIG1vc3QgaW50ZXJl c3RpbmcuDQo+ID4NCj4gPkNhbiBleGl0IHRoZSBicm93c2VyIGFuZCByZXN0YXJ0IGFmdGVyIGEg cnVuIGxpa2UgdGhhdC4NCj4gPg0KPiA+QXQgbW9tZW50LCBtZXJlbHkgcGxhbiB0byB1c2UgdGhl IHRvb2wgcHJpbWFyaWx5IHRvIHN1cnZleSB2YXJpb3VzDQo+ID53ZWIgc2l0ZXMgYW5kIGxvYWQg dGltZXMgd2hpbGUgZG9pbmcgcGFja2V0IGNhcHR1cmVzLiBIb3BlIHdhcw0KPiA+dG8gZ2V0IHZh bGlkIGRhdGEgZnJvbSB0aGUgbmV0d29yayBwb3J0aW9uIG9mIHRoZSBsb2FkLCB0aG8uLi4NCj4g Pg0KPiA+PiAqIElmIHlvdSBoYXZlIGFuIGFkdmFuY2VkIENocm9taXVtIHNldHVwLCB0aGlzIGRl ZmluaXRlbHkgZG9lcyBub3QNCj4gPj53b3JrLiBJDQo+ID4+IGFkdmlzZSB1c2luZyB0aGUgYmVu Y2htYXJrIGV4dGVuc2lvbiBvbmx5IHdpdGggYSBzZXBhcmF0ZSBDaHJvbWl1bQ0KPiA+PnByb2Zp bGUNCj4gPj4gZm9yIHRlc3RpbmcgcHVycG9zZXMuIE91ciBmbHVzaGluZyBvZiBzb2NrZXRzLCBj YWNoZXMsIGV0YyBkb2VzIG5vdA0KPiA+PmFjdHVhbGx5DQo+ID4+IHdvcmsgY29ycmVjdGx5IHdo ZW4geW91IHVzZSB0aGUgQ2hyb21pdW0gbXVsdGlwcm9maWxlIGZlYXR1cmUgYW5kIGFsc28NCj4g Pj5mYWlscw0KPiA+PiB0byBmbHVzaCBsb3RzIG9mIG91ciBvdGhlciBuZXR3b3JrIGNhY2hlcy4N Cj4gPg0KPiA+bm90ZWQuDQo+ID4NCj4gPg0KPiA+PiAqIE5vIG9uZSBvbiBDaHJvbWl1bSByZWFs bHkgYmVsaWV2ZXMgdGhlIHRpbWUgdG8gcGFpbnQgbnVtYmVycyB0aGF0IHdlDQo+ID4+IG91dHB1 dCA6KSBJdCdzIGNvbXBsaWNhdGVkLiBPdXIgZ3JhcGhpY3Mgc3RhY2sgaXMgY29tcGxpY2F0ZWQu IFRoZSB0aW1lDQo+ID4+ZnJvbQ0KPiA+DQo+ID5JIGFjdHVhbGx5IGNhcmUgb25seSBhYm91dCB0 aW1lLXRvLWZ1bGwgbGF5b3V0IGFzIHRoYXQncyBhIGNvcmUgbmV0d29yaw0KPiA+ZWZmZWN0Li4u DQo+ID4NCj4gPj4gd2hlbiBCbGluayB0aGlua3MgaXQgcGFpbnRlZCB0byB3aGVuIHRoZSBHUFUg YWN0dWFsbHkgYmxpdHMgdG8gdGhlDQo+ID4+c2NyZWVuDQo+ID4+IGNhbm5vdCBjdXJyZW50bHkg YmUgY29ycm9ib3JhdGVkIHdpdGggYW55IGhpZ2ggZGVncmVlIG9mIGFjY3VyYWN5IGZyb20NCj4g Pj4gd2l0aGluIG91ciBjb2RlLg0KPiA+DQo+ID4+ICogSXQgaGFzIG5vdCBiZWVuIG1haW50YWlu ZWQgc2luY2UgMjAxMC4gSXQgaXMgcXVpdGUgbGlrZWx5IHRoZXJlIGFyZQ0KPiA+Pm1hbnkNCj4g Pj4gb3RoZXIgc3VidGxlIGluYWNjdXJhY2llcyBoZXJlLg0KPiA+DQo+ID5Hcm9rLg0KPiA+DQo+ ID4+IEluIHNob3J0LCB3aGlsZSB5b3UgY2FuIGV4cGVjdCBpdCB0byBnaXZlIHlvdSBhIHZlcnkg aGlnaCBsZXZlbA0KPiA+PiB1bmRlcnN0YW5kaW5nIG9mIHBlcmZvcm1hbmNlIGlzc3VlcywgSSBh ZHZpc2UgYWdhaW5zdCBwbGFjaW5nDQo+ID4+bm9uLXRyaXZpYWwNCj4gPj4gY29uZmlkZW5jZSBp biB0aGUgYWNjdXJhY3kgb2YgdGhlIG51bWJlcnMgZ2VuZXJhdGVkIGJ5IHRoZSBiZW5jaG1hcmsN Cj4gPj4gZXh0ZW5zaW9uLiBUaGUgZmFjdCB0aGF0IG51bWJlcnMgYXJlIHByb2R1Y2VkIGJ5IHRo ZSBleHRlbnNpb24gc2hvdWxkDQo+ID4+bm90IGJlDQo+ID4+IHRyZWF0ZWQgYXMgZXZpZGVuY2Ug dGhhdCB0aGUgZXh0ZW5zaW9uIGFjdHVhbGx5IGZ1bmN0aW9ucyBjb3JyZWN0bHkuDQo+ID4NCj4g Pk9LLCBub3RlZC4gU3RpbGwgZGVsaWdodGVkIHRvIGJlIGFibGUgdG8gaGF2ZSBhIHNpbXBsZSBs b2FkIGdlbmVyYXRvcg0KPiA+dGhhdCBleGVyY2lzZXMgdGhlIGJyb3dzZXJzIGFuZCBnZW5lcmF0 ZXMgc29tZSByZXN1bHRzLCBob3dldmVyDQo+ID5kdWJpb3VzLg0KPiA+DQo+ID4+DQo+ID4+IENo ZWVycy4NCj4gPj4NCj4gPj4NCj4gPj4gT24gVGh1LCBBcHIgMTcsIDIwMTQgYXQgMTA6NDkgQU0s IERhdmUgVGFodCA8ZGF2ZS50YWh0QGdtYWlsLmNvbTxtYWlsdG86ZGF2ZS50YWh0QGdtYWlsLmNv bT4+DQo+IHdyb3RlOg0KPiA+Pj4NCj4gPj4+IEdldHRpbmcgYSBncmlwIG9uIHJlYWwgd2ViIHBh Z2UgbG9hZCB0aW1lIGJlaGF2aW9yIGluIGFuIGFnZSBvZg0KPiA+Pj4gc2hhcmRlZCB3ZWJzaXRl cywNCj4gPj4+IGRvemVucyBvZiBkbnMgbG9va3VwcywgamF2YXNjcmlwdCwgYW5kIGZhaXJseSBy YW5kb20gYmVoYXZpb3IgaW4gYWQNCj4gPj4+IHNlcnZpY2VzDQo+ID4+PiBhbmQgY2RucyBhZ2Fp bnN0IGhvdyBhIG1vZGVybiBicm93c2VycyBiZWhhdmVzIGlzIHZlcnksIHZlcnkgaGFyZC4NCj4g Pj4+DQo+ID4+PiBpdCB0dXJucyBvdXQgaWYgeW91IHJ1bg0KPiA+Pj4NCj4gPj4+IGdvb2dsZS1j aHJvbWUgLS1lbmFibGUtYmVuY2htYXJraW5nIC0tZW5hYmxlLW5ldC1iZW5jaG1hcmtpbmcNCj4g Pj4+DQo+ID4+PiAoTWFjIHVzZXJzIGhhdmUgdG8gZW1iZWQgdGhlc2Ugb3B0aW9ucyBpbiB0aGVp ciBzdGFydHVwIHNjcmlwdCAtIHNlZQ0KPiA+Pj4gaHR0cDovL3d3dy5jaHJvbWl1bS5vcmcvZGV2 ZWxvcGVycy9ob3ctdG9zL3J1bi1jaHJvbWl1bS13aXRoLWZsYWdzDQo+ICkNCj4gPj4+DQo+ID4+ PiBlbmFibGUgZGV2ZWxvcGVyIG9wdGlvbnMgYW5kIGluc3RhbGwgYW5kIHJ1biB0aGUgY2hyb21l IHdlYiBwYWdlDQo+ID4+PiBiZW5jaG1hcmtlciwNCj4gPj4+ICgNCj4gPj4+DQo+ID4+Pmh0dHBz Oi8vY2hyb21lLmdvb2dsZS5jb20vd2Vic3RvcmUvZGV0YWlsL3BhZ2UtYmVuY2htYXJrZXIvY2hh bm5pbWZkb21haA0KPiA+Pj5la2pjYWhsYnBjY2JnYW9wamxsP2hsPWVuDQo+ID4+PiApDQo+ID4+ Pg0KPiA+Pj4gdGhhdCBpdCB3b3JrcyAoYXQgbGVhc3QgZm9yIG1lLCBvbiBhIGJyaWVmIHRlc3Qg b2YgdGhlIGxhdGVzdA0KPiBjaHJvbWUsDQo+ID4+Pm9uDQo+ID4+PiBsaW51eC4NCj4gPj4+IENh biBzb21lb25lIHRyeSB3aW5kb3dzIGFuZCBtYWM/KQ0KPiA+Pj4NCj4gPj4+IFlvdSBjYW4gdGhl biBmZWVkIGluIGEgbGlzdCBvZiB1cmxzIHRvIHRlc3QgYWdhaW5zdCwgYW5kIHBvc3QNCj4gcHJv Y2Vzcw0KPiA+Pj4gdGhlIHJlc3VsdGluZyAuY3N2IGZpbGUgdG8geW91ciBoZWFydHMgY29udGVu dC4gV2UgdXNlZCB0byB1c2UgdGhpcw0KPiA+Pj4gYmVuY2htYXJrIGEgbG90IHdoaWxlIHRyeWlu ZyB0byBjaGFyYWN0ZXJpc2UgdHlwaWNhbCB3ZWIgYmVoYXZpb3JzDQo+ID4+PiB1bmRlciBhcW0g YW5kIHBhY2tldCBzY2hlZHVsaW5nIHN5c3RlbXMgdW5kZXIgbG9hZC4gUnVubmluZw0KPiA+Pj4g aXQgc2ltdWx0YW5lb3VzbHkgd2l0aCBhIHJydWwgdGVzdCBvciBvbmUgb2YgdGhlIHNpbXBsZXIg dGNwIHVwbG9hZA0KPiBvcg0KPiA+Pj4gZG93bmxvYWQNCj4gPj4+IHRlc3RzIGluIHRoZSBycnVs IHN1aXRlIHdhcyBvZnRlbiBxdWl0ZSBpbnRlcmVzdGluZy4NCj4gPj4+DQo+ID4+PiBJdCB0dXJu ZWQgb3V0IHRoZSBkb2MgaGFzIGJlZW4gd3JvbmcgYSB3aGlsZSBhcyB0byB0aGUgbmFtZSBvZiB0 aGUNCj4gPj4+c2Vjb25kDQo+ID4+PiBjb21tYW5kIGxuaWUgb3B0aW9uLiBJIHdhcyBnZWFyaW5n IHVwIG1lbnRhbGx5IGZvciBoYXZpbmcgdG8gbG9vayBhdA0KPiA+Pj4gdGhlIHNvdXJjZS4uLi4N Cj4gPj4+DQo+ID4+PiBodHRwOi8vY29kZS5nb29nbGUuY29tL3AvY2hyb21pdW0vaXNzdWVzL2Rl dGFpbD9pZD0zMzg3MDUNCj4gPj4+DQo+ID4+PiAvbWUgaGFwcHkNCj4gPj4+DQo+ID4+PiAtLQ0K PiA+Pj4gRGF2ZSBUw6RodA0KPiA+Pj4NCj4gPj4+IEhlYXJ0YmxlZWQgUE9DIG9uIHdpZmkgY2Ft cHVzIG5ldHdvcmtzIHdpdGggRUFQIGF1dGg6DQo+ID4+PiBodHRwOi8vd3d3LmVkdXJvYW0uZWR1 LmF1L2Fkdmlzb3J5Lmh0bWwNCj4gPj4+DQo+ID4+PiBfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXw0KPiA+Pj4gYXFtIG1haWxpbmcgbGlzdA0KPiA+Pj4gYXFt QGlldGYub3JnPG1haWx0bzphcW1AaWV0Zi5vcmc+DQo+ID4+PiBodHRwczovL3d3dy5pZXRmLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL2FxbQ0KPiA+Pg0KPiA+Pg0KPiA+DQo+ID4NCj4gPg0KPiA+LS0N Cj4gPkRhdmUgVMOkaHQNCj4gPg0KPiA+TlNGVzoNCj4gPmh0dHBzOi8vdzIuZWZmLm9yZy9DZW5z b3JzaGlwL0ludGVybmV0X2NlbnNvcnNoaXBfYmlsbHMvcnVzc2VsbF8wMjk2X2luZGVjDQo+ID5l bnQuYXJ0aWNsZQ0KPiA+DQo+ID5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXw0KPiA+YXFtIG1haWxpbmcgbGlzdA0KPiA+YXFtQGlldGYub3JnPG1haWx0bzph cW1AaWV0Zi5vcmc+DQo+ID5odHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Fx bQ0KPg0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0K PiBDZXJvd3J0LWRldmVsIG1haWxpbmcgbGlzdA0KPiBDZXJvd3J0LWRldmVsQGxpc3RzLmJ1ZmZl cmJsb2F0Lm5ldDxtYWlsdG86Q2Vyb3dydC1kZXZlbEBsaXN0cy5idWZmZXJibG9hdC5uZXQ+DQo+ IGh0dHBzOi8vbGlzdHMuYnVmZmVyYmxvYXQubmV0L2xpc3RpbmZvL2Nlcm93cnQtZGV2ZWwNCj4N Cg== --_000_CF76D02032E2Dgwhitecablelabscom_ Content-Type: text/html; charset="utf-8" Content-ID: <113DC101E6318542B0604BD07A51100F@cablelabs.com> Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgLXdlYmtpdC1saW5lLWJy ZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAx NHB4OyBmb250LWZhbWlseTogQ2FsaWJyaSwgc2Fucy1zZXJpZjsiPg0KPGRpdj5JbiBtb3N0IGNh c2VzIGl0IHdhcyAqZmFyKiBsZXNzIHRoYW4gMSUuICZuYnNwO0luIG91ciBET0NTSVMgMy4xIHNj ZW5hcmlvcyBpdCBpcyBtb3JlIGxpa2UgMWUtNSB0byAxZS00ICgxZS0zIHdvcnN0IGNhc2UgZm9y IHRoZSBidWZmZXJibG9hdGVkIGRyb3AgdGFpbCBiYXNlbGluZSkgd2hpY2ggbWVhbnMgdGhhdCBp bXBsZW1lbnRpbmcgZG5zIGluIHRoZSBtb2RlbCB3b3VsZCBoYXZlIGFmZmVjdGVkIG9uIHRoZSBv cmRlciBvZiAxIG91dCBvZiBldmVyeQ0KIDI1MDAgcGFnZSBsb2Fkcy48L2Rpdj4NCjxkaXY+PGJy Pg0KPC9kaXY+DQo8ZGl2PkFzIHdpdGggYW55IG1vZGVsIG9mIHJlYWxpdHksIHRoZXJlIGlzIGEg dHJhZGVvZmYgYmV0d2VlbiBjb21wbGV4aXR5IGFuZCBmaWRlbGl0eSwgcmlnaHQ/IFdlIGNob3Nl IDQgc2VydmVycyBhcyBiZWluZyBhIHJlYXNvbmFibGUgYW5kIHByYWN0aWNhbCBudW1iZXIuICZu YnNwO01hbnkgb2YgdGhlIG1vc3QgcG9wdWxhciBzaXRlcyBhY3R1YWxseSBkb24ndCBkaXN0cmli dXRlIGNvbnRlbnQgdGhhdCB3aWRlbHkgKGUuZy4gR29vZ2xlIG9ubHkgdXNlZA0KIDMgc2VydmVy cyBpbiBvdXIgbWVhc3VyZW1lbnRzKSwgYW5kIHRoZSB3b3JzdCBjYXNlIHdhcyAzMiBzZXJ2ZXJz LiAmbmJzcDsgSW4gcmVhbGl0eSBldmVyeSB3ZWIgcGFnZSBpcyBkaWZmZXJlbnQsIGFuZCB5b3Ug Y291bGQgY29tZSB1cCB3aXRoIGFuIGFyYml0cmFyaWx5IGNvbXBsZXggbW9kZWwgYW5kIGl0IHN0 aWxsIHdvdWxkbid0IG1hdGNoIHNvbWUgdmVyc2lvbiBvZiByZWFsaXR5LiAmbmJzcDtJbiBteSB2 aWV3LCB3ZSB3YW50IHRvIGhhdmUgc29tZXRoaW5nDQogdGhhdCBpcyBwcmFjdGljYWwgdG8gaW1w bGVtZW50IGFuZCBjbG9zZSBlbm91Z2ggdG8gcmVhbGl0eSB0byBiZSB1c2VmdWwuICZuYnNwO0lu IHRoZSBBUU0tZXZhbHVhdGlvbiBkcmFmdCBjdXJyZW50bHksIHRoZSBtb2RlbCBpcyBhIHNpbmds ZSA3MDBrQiBmaWxlIHNlcnZlZCBmcm9tIGEgc2luZ2xlIHNlcnZlci48L2Rpdj4NCjxkaXY+PGJy Pg0KPC9kaXY+DQo8ZGl2PkdyZWc8L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8c3BhbiBpZD0i T0xLX1NSQ19CT0RZX1NFQ1RJT04iPg0KPGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaTsg Zm9udC1zaXplOjExcHQ7IHRleHQtYWxpZ246bGVmdDsgY29sb3I6YmxhY2s7IEJPUkRFUi1CT1RU T006IG1lZGl1bSBub25lOyBCT1JERVItTEVGVDogbWVkaXVtIG5vbmU7IFBBRERJTkctQk9UVE9N OiAwaW47IFBBRERJTkctTEVGVDogMGluOyBQQURESU5HLVJJR0hUOiAwaW47IEJPUkRFUi1UT1A6 ICNiNWM0ZGYgMXB0IHNvbGlkOyBCT1JERVItUklHSFQ6IG1lZGl1bSBub25lOyBQQURESU5HLVRP UDogM3B0Ij4NCjxzcGFuIHN0eWxlPSJmb250LXdlaWdodDpib2xkIj5Gcm9tOiA8L3NwYW4+JnF1 b3Q7PGEgaHJlZj0ibWFpbHRvOmRwcmVlZEByZWVkLmNvbSI+ZHByZWVkQHJlZWQuY29tPC9hPiZx dW90OyAmbHQ7PGEgaHJlZj0ibWFpbHRvOmRwcmVlZEByZWVkLmNvbSI+ZHByZWVkQHJlZWQuY29t PC9hPiZndDs8YnI+DQo8c3BhbiBzdHlsZT0iZm9udC13ZWlnaHQ6Ym9sZCI+RGF0ZTogPC9zcGFu PkZyaWRheSwgQXByaWwgMTgsIDIwMTQgYXQgMTI6NDggUE08YnI+DQo8c3BhbiBzdHlsZT0iZm9u dC13ZWlnaHQ6Ym9sZCI+VG86IDwvc3Bhbj5HcmVnIFdoaXRlICZsdDs8YSBocmVmPSJtYWlsdG86 Zy53aGl0ZUBjYWJsZWxhYnMuY29tIj5nLndoaXRlQGNhYmxlbGFicy5jb208L2E+Jmd0Ozxicj4N CjxzcGFuIHN0eWxlPSJmb250LXdlaWdodDpib2xkIj5DYzogPC9zcGFuPkRhdmUgVGFodCAmbHQ7 PGEgaHJlZj0ibWFpbHRvOmRhdmUudGFodEBnbWFpbC5jb20iPmRhdmUudGFodEBnbWFpbC5jb208 L2E+Jmd0OywgJnF1b3Q7V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwpJnF1b3Q7ICZsdDs8YSBocmVm PSJtYWlsdG86d2lsbGNoYW5AY2hyb21pdW0ub3JnIj53aWxsY2hhbkBjaHJvbWl1bS5vcmc8L2E+ Jmd0OywgJnF1b3Q7PGEgaHJlZj0ibWFpbHRvOmFxbUBpZXRmLm9yZyI+YXFtQGlldGYub3JnPC9h PiZxdW90OyAmbHQ7PGEgaHJlZj0ibWFpbHRvOmFxbUBpZXRmLm9yZyI+YXFtQGlldGYub3JnPC9h PiZndDssDQogJnF1b3Q7PGEgaHJlZj0ibWFpbHRvOmNlcm93cnQtZGV2ZWxAbGlzdHMuYnVmZmVy YmxvYXQubmV0Ij5jZXJvd3J0LWRldmVsQGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldDwvYT4mcXVvdDsg Jmx0OzxhIGhyZWY9Im1haWx0bzpjZXJvd3J0LWRldmVsQGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldCI+ Y2Vyb3dydC1kZXZlbEBsaXN0cy5idWZmZXJibG9hdC5uZXQ8L2E+Jmd0OywgYmxvYXQgJmx0Ozxh IGhyZWY9Im1haWx0bzpibG9hdEBsaXN0cy5idWZmZXJibG9hdC5uZXQiPmJsb2F0QGxpc3RzLmJ1 ZmZlcmJsb2F0Lm5ldDwvYT4mZ3Q7PGJyPg0KPHNwYW4gc3R5bGU9ImZvbnQtd2VpZ2h0OmJvbGQi PlN1YmplY3Q6IDwvc3Bhbj5SZTogW0Nlcm93cnQtZGV2ZWxdIFthcW1dIGNocm9tZSB3ZWIgcGFn ZSBiZW5jaG1hcmtlciBmaXhlZDxicj4NCjwvZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxkaXY+ DQo8ZGl2Pjxmb250IGZhY2U9ImFyaWFsIiBzaXplPSIyIj4NCjxwIHN0eWxlPSJtYXJnaW46MDtw YWRkaW5nOjA7Ij5XaHkgaXMgdGhlIEROUyBQTFIgc28gaGlnaD8gJm5ic3A7MSUgaXMgcHJldHR5 IGRlcHJlc3NpbmcuPC9wPg0KPHAgc3R5bGU9Im1hcmdpbjowO3BhZGRpbmc6MDsiPiZuYnNwOzwv cD4NCjxwIHN0eWxlPSJtYXJnaW46MDtwYWRkaW5nOjA7Ij5BbHNvLCBpdCBzZWVtcyBvZGQgdG8g ZWxpbWluYXRlIDE5JSBvZiB0aGUgY29udGVudCByZXRyaWV2YWwgYmVjYXVzZSB0aGUgdGFpbCBp cyBmYXQgYW5kIGxvbmcgcmF0aGVyIHRoYW4gc2hvcnQuICZuYnNwO1dvdWxkbid0IGl0IGJlIGJl dHRlciB0byBoYXZlIDEwMDAgc2VydmVycz88L3A+DQo8cCBzdHlsZT0ibWFyZ2luOjA7cGFkZGlu ZzowOyI+Jm5ic3A7PC9wPg0KPHAgc3R5bGU9Im1hcmdpbjowO3BhZGRpbmc6MDsiPiZuYnNwOzwv cD4NCjwhLS1XTV9DT01QT1NFX1NJR05BVFVSRV9TVEFSVC0tPjwhLS1XTV9DT01QT1NFX1NJR05B VFVSRV9FTkQtLT4NCjxwIHN0eWxlPSJtYXJnaW46MDtwYWRkaW5nOjA7Ij48YnI+DQo8YnI+DQpP biBGcmlkYXksIEFwcmlsIDE4LCAyMDE0IDI6MTVwbSwgJnF1b3Q7R3JlZyBXaGl0ZSZxdW90OyAm bHQ7PGEgaHJlZj0ibWFpbHRvOmcud2hpdGVAQ2FibGVMYWJzLmNvbSI+Zy53aGl0ZUBDYWJsZUxh YnMuY29tPC9hPiZndDsgc2FpZDo8YnI+DQo8YnI+DQo8L3A+DQo8ZGl2IGlkPSJTYWZlU3R5bGVz MTM5Nzg0Njc1NCI+DQo8cCBzdHlsZT0ibWFyZ2luOjA7cGFkZGluZzowOyI+Jmd0OyBEYXZlLDxi cj4NCiZndDsgPGJyPg0KJmd0OyBXZSB1c2VkIHRoZSAyNWsgb2JqZWN0IHNpemUgZm9yIGEgc2hv cnQgdGltZSBiYWNrIGluIDIwMTIgdW50aWwgd2UgaGFkPGJyPg0KJmd0OyByZXNvdXJjZXMgdG8g YnVpbGQgYSBtb3JlIGFkdmFuY2VkIG1vZGVsIChhcHBlbmRpeCBBKS4gSSBkaWQgYSBidW5jaCBv Zjxicj4NCiZndDsgY2FwdHVyZXMgb2YgcmVhbCB3ZWIgcGFnZXMgYmFjayBpbiAyMDExIGFuZCBj b21wYXJlZCB0aGUgb2JqZWN0IHNpemU8YnI+DQomZ3Q7IHN0YXRpc3RpY3MgdG8gbW9kZWxzIHRo YXQgSSdkIHNlZW4gcHVibGlzaGVkLiBMb2dub3JtYWwgZGlkbid0IHNlZW0gdG8gYmU8YnI+DQom Z3Q7ICpleGFjdGx5KiByaWdodCwgYnV0IGl0IHdhc24ndCBhIGJhZCBmaXQgdG8gd2hhdCBJIHNh dy4gSSd2ZSBhdHRhY2hlZCBhPGJyPg0KJmd0OyBDREYuPGJyPg0KJmd0OyA8YnI+DQomZ3Q7IFRo ZSBjaG9pY2Ugb2YgNCBzZXJ2ZXJzIHdhcyBiYXNlZCBzb21ld2hhdCBvbiBsb2dpc3RpY3MsIGFu ZCBhbHNvIG9uIGE8YnI+DQomZ3Q7IGZpbmRpbmcgdGhhdCBhY3Jvc3Mgb3VyIGRhdGEgc2V0LCB0 aGUgYXZlcmFnZSB3ZWIgcGFnZSByZXRyaWV2ZWQgODElIG9mPGJyPg0KJmd0OyBpdHMgcmVzb3Vy Y2VzIGZyb20gdGhlIHRvcCA0IHNlcnZlcnMuIEluY3JlYXNpbmcgdG8gNSBzZXJ2ZXJzIG9ubHk8 YnI+DQomZ3Q7IGluY3JlYXNlZCB0aGF0IHBlcmNlbnRhZ2UgdG8gODQlLjxicj4NCiZndDsgPGJy Pg0KJmd0OyBUaGUgY2hvaWNlIG9mIFJUVHMgYWxzbyBjYW1lIGZyb20gdGhlIHdlYiB0cmFmZmlj IGNhcHR1cmVzLiBJIHNhdzxicj4NCiZndDsgUlRUbWluPTE2bXMsIFJUVG1lYW49NTMuOG1zLCBS VFRtYXg9MTM0bXMuPGJyPg0KJmd0OyA8YnI+DQomZ3Q7IE11Y2ggb2YgdGhpcyBjYW4gYmUgZm91 bmQgaW48YnI+DQomZ3Q7IDxhIGhyZWY9Imh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFm dC13aGl0ZS1odHRwYmlzLXNwZHktYW5hbHlzaXMtMDAiPmh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcv aHRtbC9kcmFmdC13aGl0ZS1odHRwYmlzLXNwZHktYW5hbHlzaXMtMDA8L2E+PGJyPg0KJmd0OyA8 YnI+DQomZ3Q7IEluIG1hbnkgb2YgdGhlIGNhc2VzIHRoYXQgd2UndmUgc2ltdWxhdGVkLCB0aGUg cGFja2V0IGRyb3AgcHJvYmFiaWxpdHkgaXM8YnI+DQomZ3Q7IGxlc3MgdGhhbiAxJSBmb3IgRE5T IHBhY2tldHMuIEluIG91ciB3ZWIgbW9kZWwsIHRoZXJlIGFyZSBhIHRvdGFsIG9mIDQ8YnI+DQom Z3Q7IHNlcnZlcnMsIHNvIDQgRE5TIGxvb2t1cHMgYXNzdW1pbmcgbm9uZSBvZiB0aGUgYWRkcmVz c2VzIGFyZSBjYWNoZWQuIElmPGJyPg0KJmd0OyBQTFIgPSAxJSwgdGhlcmUgd291bGQgYmUgYSAz LjklIGNoYW5jZSBvZiBsb3Npbmcgb25lIG9yIG1vcmUgRE5TIHBhY2tldHM8YnI+DQomZ3Q7ICh3 aXRoIGEgcmVzdWx0aW5nIH41IHNlY29uZCBhZGRpdGlvbmFsIGRlbGF5IG9uIGxvYWQgdGltZSku IEkndmUgcHJvYmFibHk8YnI+DQomZ3Q7IG92ZXJzaW1wbGlmaWVkIHRoaXMsIGJ1dCBLYXRoaWUg Ti4gYW5kIEkgbWFkZSB0aGUgY2FsbCB0aGF0IGl0IHdvdWxkIGJlPGJyPg0KJmd0OyBzaWduaWZp Y2FudGx5IGVhc2llciB0byBqdXN0IGRvIHRoaXMgbWF0aCB0aGFuIHRvIGJ1aWxkIGEgZG5zPGJy Pg0KJmd0OyBpbXBsZW1lbnRhdGlvbiBpbiBuczIuIFdlJ3ZlIG9wZW4gc291cmNlZCB0aGUgd2Vi IG1vZGVsIChpdCdzIG9uIEthdGhpZSdzPGJyPg0KJmd0OyB3ZWIgcGFnZSBhbmQgd2lsbCBiZSBw YXJ0IG9mIG5zMi4zNikgd2l0aCBhbiBlbmNvdXJhZ2VtZW50IHRvIHRoZTxicj4NCiZndDsgY29t bXVuaXR5IHRvIGltcHJvdmUgb24gaXQuIElmIHlvdSdkIGxpa2UgdG8gcG9ydCBpdCB0byBuczMg YW5kIGFkZCBhIGRuczxicj4NCiZndDsgbW9kZWwsIHRoYXQgd291bGQgYmUgZmFudGFzdGljLjxi cj4NCiZndDsgPGJyPg0KJmd0OyAtR3JlZzxicj4NCiZndDsgPGJyPg0KJmd0OyA8YnI+DQomZ3Q7 IE9uIDQvMTcvMTQsIDM6MDcgUE0sICZxdW90O0RhdmUgVGFodCZxdW90OyAmbHQ7PGEgaHJlZj0i bWFpbHRvOmRhdmUudGFodEBnbWFpbC5jb20iPmRhdmUudGFodEBnbWFpbC5jb208L2E+Jmd0OyB3 cm90ZTo8YnI+DQomZ3Q7IDxicj4NCiZndDsgJmd0O09uIFRodSwgQXByIDE3LCAyMDE0IGF0IDEy OjAxIFBNLCBXaWxsaWFtIENoYW4gKOmZiOaZuuaYjCk8YnI+DQomZ3Q7ICZndDsmbHQ7PGEgaHJl Zj0ibWFpbHRvOndpbGxjaGFuQGNocm9taXVtLm9yZyI+d2lsbGNoYW5AY2hyb21pdW0ub3JnPC9h PiZndDsgd3JvdGU6PGJyPg0KJmd0OyAmZ3Q7Jmd0OyBTcGVha2luZyBhcyB0aGUgcHJpbWFyeSBD aHJvbWl1bSBkZXZlbG9wZXIgaW4gY2hhcmdlIG9mIHRoaXMgcmVsZXZhbnQ8YnI+DQomZ3Q7ICZn dDsmZ3Q7Y29kZSw8YnI+DQomZ3Q7ICZndDsmZ3Q7IEkgd291bGQgbGlrZSB0byBjYXV0aW9uIHB1 dHRpbmcgdG9vIG11Y2ggdHJ1c3QgaW4gdGhlIG51bWJlcnM8YnI+DQomZ3Q7ICZndDsmZ3Q7Z2Vu ZXJhdGVkLiBBbnk8YnI+DQomZ3Q7ICZndDsmZ3Q7IHN0YXRpc3RpY2FsIGNsYWltcyBhYm91dCB0 aGUgbnVtYmVycyBhcmUgcHJvYmFibHkgdW5yZWFzb25hYmxlIHRvIG1ha2UuPGJyPg0KJmd0OyAm Z3Q7PGJyPg0KJmd0OyAmZ3Q7U2lnaC4gT3RoZXIgYmVuY2htYXJrcyBzdWNoIGFzIHRoZSBhcGFj aGUgKCZxdW90O2FiJnF1b3Q7KSBiZW5jaG1hcms8YnI+DQomZ3Q7ICZndDthcmUgcHJpbWFyaWx5 IGRlc2lnbmVkIGFzIHN0cmVzcyB0ZXN0ZXJzIGZvciB3ZWIgc2VydmVycywgbm90IGFzIHJlYWxp c3RpYzxicj4NCiZndDsgJmd0O3RyYWZmaWMuIE1vZGVybiB3ZWIgdHJhZmZpYyBoYXMgc3VjaCBh IGhpZ2ggbGV2ZWwgb2YgZHluYW1pY2lzbSBpbiBpdCw8YnI+DQomZ3Q7ICZndDt0aGF0IHN0YXRp YyB3ZWIgcGFnZSBsb2FkcyBhbG9uZyBhbnkgZGlzdHJpYnV0aW9uLCBzZWVtIGluc3VmZmljaWVu dCw8YnI+DQomZ3Q7ICZndDtwYXNzaXZlIGFuYWx5c2lzIG9mIGFnZ3JlZ2F0ZWQgdHJhZmZpYyAm cXVvdDtmZWVscyZxdW90OyBpbmNvcnJlY3QgcmVsYXRpdmUgdG8gdGhlPGJyPg0KJmd0OyAmZ3Q7 c29ydHMgb2YgaG9tZSBhbmQgc21hbGwgYnVzaW5lc3MgdHJhZmZpYyBJJ3ZlIHNlZW4sIGFuZCBz byBvbi48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDtGYW1vdXMgcGFwZXJzLCBzdWNoIGFz IHRoaXMgb25lOjxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0OzxhIGhyZWY9Imh0dHA6Ly9j Y3Iuc2lnY29tbS5vcmcvYXJjaGl2ZS8xOTk1L2phbjk1L2Njci05NTAxLWxlbGFuZC5wZGYiPmh0 dHA6Ly9jY3Iuc2lnY29tbS5vcmcvYXJjaGl2ZS8xOTk1L2phbjk1L2Njci05NTAxLWxlbGFuZC5w ZGY8L2E+PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7U2VlbSBwb3NzaWJseSBpcnJlbGV2 YW50IHRvIGRyYXcgY29uY2x1c2lvbnMgZnJvbSBnaXZlbiB0aGUga2luZDxicj4NCiZndDsgJmd0 O29mIGRhdGEgdGhleSBhbmFseXNlZCBhbmQgcHJvY2VlZGluZyBmcm9tIGFuIGluY29ycmVjdCBt b2RlbCBvcjxicj4NCiZndDsgJmd0O2d1dCBmZWVsIGZvciBob3cgdGhlIG5ldHdvcmsgYmVoYXZl cyB0b2RheSBzZWVtcyB0byBiZSBmb29saXNoLjxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0 O0V2ZW4gdGhlIG1vc3QgYmFzaWMgb2YgdG9vbHMsIHN1Y2ggYXMgaHR0cGluZywgaGFkIHRocmVl IGJhc2ljIGJ1Z3M8YnI+DQomZ3Q7ICZndDt0aGF0IEkgZm91bmQgaW4gYSBmZXcgbWludXRlcyBv ZiB0cnlpbmcgdG8gY29tZSB1cCB3aXRoIHNvbWUgYmFzaWM8YnI+DQomZ3Q7ICZndDtiZWhhdmlv cnMgeWVzdGVyZGF5Ojxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0OzxhIGhyZWY9Imh0dHBz Oi8vbGlzdHMuYnVmZmVyYmxvYXQubmV0L3BpcGVybWFpbC9ibG9hdC8yMDE0LUFwcmlsLzAwMTg5 MC5odG1sIj5odHRwczovL2xpc3RzLmJ1ZmZlcmJsb2F0Lm5ldC9waXBlcm1haWwvYmxvYXQvMjAx NC1BcHJpbC8wMDE4OTAuaHRtbDwvYT48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDtUaG9z ZSBhcmUgZ29pbmcgdG8gYmUgYSBsb3QgZWFzaWVyIHRvIGZpeCB0aGFuIGRpdmluZyBpbnRvIHRo ZSBjaHJvbWl1bTxicj4NCiZndDsgJmd0O2NvZGViYXNlITxicj4NCiZndDsgJmd0Ozxicj4NCiZn dDsgJmd0O1RoZXJlIGFyZSB2ZXJ5IGZldyB0b29scyB3b3J0aCB0cnVzdGluZywgYW5kIEkgYW0g YWx3YXlzIGR1YmlvdXM8YnI+DQomZ3Q7ICZndDtvZiBwYXBlcnMgdGhhdCBwdWJsaXNoIHJlc3Vs dHMgd2l0aCB1bmF2YWlsYWJsZSB0b29scyBhbmQgZGF0YS4gVGhlIG9ubHk8YnI+DQomZ3Q7ICZn dDt0b29scyBJIGhhdmUgYW55IGZhaXRoIGluIGZvciBuZXR3b3JrIGFuYWx5c2lzIGFyZSBuZXRw ZXJmLDxicj4NCiZndDsgJmd0O25ldHBlcmYtd3JhcHBlciw8YnI+DQomZ3Q7ICZndDt0Y3BkdW1w IGFuZCB4cGxvdC5vcmcsIGFuZCB0byBhIGxhcmdlIGV4dGVudCB3aXJlc2hhcmsuIFRva2UgYW5k IEkgaGF2ZTxicj4NCiZndDsgJmd0O2JlZW4gdGVhcmluZyBhcGFydCBkLWl0ZyBhbmQgSSBob3Bl IHRvIG9uZSBkYXkgYmUgYWJsZSB0byBhZGQgdGhhdCB0bzxicj4NCiZndDsgJmd0O215IHRydXN0 YWJsZSBsaXN0Li4uIGJ1dCBiZXR0ZXIgdG9vbHMgYXJlIG5lZWRlZCE8YnI+DQomZ3Q7ICZndDs8 YnI+DQomZ3Q7ICZndDtUb29scyB0aGF0IEkgZG9uJ3QgaGF2ZSBhIGxvdCBvZiBmYWl0aCBpbiBp bmNsdWRlIHRoYXQsIGlwZXJmLCBhbnl0aGluZzxicj4NCiZndDsgJmd0O3dyaXR0ZW48YnI+DQom Z3Q7ICZndDtpbiBqYXZhIG9yIG90aGVyIGhpZ2ggbGV2ZWwgbGFuZ3VhZ2VzLCBzcGVlZHRlc3Qu bmV0LCBhbmQgdGhpbmdzIGxpa2U8YnI+DQomZ3Q7ICZndDtzaGFwZXJwcm9iZS48YnI+DQomZ3Q7 ICZndDs8YnI+DQomZ3Q7ICZndDtIYXZlIHZlcnkgbGl0dGxlIGZhaXRoIGluIG5zMiwgc2xpZ2h0 bHkgbW9yZSBpbiBuczMsIGFuZCBJJ3ZlIGJlZW4gbWVhbmluZzxicj4NCiZndDsgJmd0O3RvIGxv b2sgb3ZlciB0aGUgbWluaW5ldCBhbmQgb3RoZXIgc2ltdWxhdG9ycyB3aGVuZXZlciBJIGdvdCBz b21lIHNwYXJlPGJyPg0KJmd0OyAmZ3Q7dGltZTsgdGhlIG1pbmluZXQgcmVzdWx0cyBzdGFuZm9y ZCBnZXRzIHNlZW0gcHJldHR5IHJlYXNvbmFibGUgYW5kIEk8YnI+DQomZ3Q7ICZndDthZG9yZSB0 aGVpciByZXByb2R1Y2luZyByZXN1bHRzIGVmZm9ydC4gSGF2ZW4ndCBleHBsb3JlZCBuZHQsIGtl ZXAgbWVhbmluZzxicj4NCiZndDsgJmd0O3RvLi4uPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAm Z3Q7Jmd0OyBSZWFzb25zOjxicj4NCiZndDsgJmd0OyZndDsgKiBXZSBkb24ndCBhY3RpdmVseSBt YWludGFpbiB0aGlzIGNvZGUuIEl0J3MgYmVoaW5kIHRoZSBjb21tYW5kIGxpbmU8YnI+DQomZ3Q7 ICZndDsmZ3Q7ZmxhZ3MuPGJyPg0KJmd0OyAmZ3Q7Jmd0OyBUaGV5IGFyZSBicm9rZW4uIFRoZSBm YWN0IHRoYXQgaXQgc3RpbGwgcmVzdWx0cyBpbiBudW1iZXJzIG9uIHRoZTxicj4NCiZndDsgJmd0 OyZndDtiZW5jaG1hcms8YnI+DQomZ3Q7ICZndDsmZ3Q7IGV4dGVuc2lvbiBpcyBhbiBleGFtcGxl IG9mIHdoZXJlIHVubWFpbnRhaW5lZCBjb2RlIGRvZXNuJ3QgaGF2ZSB0aGUgVUk8YnI+DQomZ3Q7 ICZndDsmZ3Q7IGRpc2FibGVkLCBldmVuIHRob3VnaCB0aGUgaW50ZXJuYWwgd29ya2luZ3Mgb2Yg dGhlIGNvZGUgZmFpbCB0bzxicj4NCiZndDsgJmd0OyZndDtndWFyYW50ZWU8YnI+DQomZ3Q7ICZn dDsmZ3Q7IGNvcnJlY3Qgb3BlcmF0aW9uLiBXZSBoYXZlbid0IGRpc2FibGVkIGl0IGJlY2F1c2Us IHdlbGwsIGl0J3M8YnI+DQomZ3Q7ICZndDsmZ3Q7dW5tYWludGFpbmVkLjxicj4NCiZndDsgJmd0 Ozxicj4NCiZndDsgJmd0O0FzIEkgbWVudGlvbmVkIEkgd2FzIGdlYXJpbmcgdXAgZm9yIGEgaGFj a2luZyBydW4uLi48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDtUaGUgdmFzdCBtYWpvcml0 eSBvZiByZXN1bHRzIEkgbG9vayBhdCBhcmUgYWN0dWFsbHkgb2J0YWluZWQgdmlhPGJyPg0KJmd0 OyAmZ3Q7bG9va2luZyBhdCBwYWNrZXQgY2FwdHVyZXMuIEkgbW9zdGx5IHVzZSBiZW5jaG1hcmtz IGFzIGFic3RyYWN0aW9uczxicj4NCiZndDsgJmd0O3RvIHNlZSBpZiB0aGV5IG1ha2Ugc29tZSBz ZW5zZSByZWxhdGl2ZSB0byB0aGUgY2FwdHVyZXMgYW5kIHRlbmQ8YnI+DQomZ3Q7ICZndDt0byBj b21wYXJlIGRpZmZlcmVudCBiZW5jaG1hcmtzIGFnYWluc3QgZWFjaCBvdGhlci48YnI+DQomZ3Q7 ICZndDs8YnI+DQomZ3Q7ICZndDtJIHJlYWxpemUgb3RoZXJzIGRvbid0IGdvIGludG8gdGhhdCBs ZXZlbCBvZiBkZXRhaWwsIHNvIHlvdSBoYXZlIGdpdmVuPGJyPg0KJmd0OyAmZ3Q7ZmFpciB3YXJu aW5nISBJbiBvdXIgY2FzZSB3ZSB1c2VkIHRoZSB3ZWIgcGFnZSBiZW5jaG1hcmtlciBhczxicj4N CiZndDsgJmd0O2EgbWVhbnMgdG8gdHJ5IGFuZCByYXBpZGx5IGFjcXVpcmUgc29tZSB0eXBpY2Fs IGRpc3RyaWJ1dGlvbnMgb2Y8YnI+DQomZ3Q7ICZndDtnZXQgYW5kIHRjcCBzdHJlYW0gcmVxdWVz dHMgZnJvbSB0aGluZ3MgbGlrZSB0aGUgYWxleGEgdG9wIDEwMDAsPGJyPg0KJmd0OyAmZ3Q7YW5k IGFzIGEgd2F5IHRvIEEvQiBkaWZmZXJlbnQgYXFtL3BhY2tldCBzY2hlZHVsaW5nIHNldHVwcy48 YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsuLi4gYnV0IHRoZSBvbmx5IGVhc2lseSBwdWJs aXNoYWJsZSByZXN1bHRzIHdlcmUgZnJvbSB0aGUgYmVuY2htYXJrPGJyPg0KJmd0OyAmZ3Q7aXRz ZWxmLDxicj4NCiZndDsgJmd0O2FuZCB3ZSAocmVsdWN0YW50bHkpIG9ubHkgcHVibGlzaGVkIG9u ZSBncmFwaCBmcm9tIGFsbCB0aGUgd29yayB0aGF0PGJyPg0KJmd0OyAmZ3Q7d2VudCBpbnRvIGl0 IDImIzQzOyB5ZWFycyBiYWNrIGFuZCB1c2VkIGl0IGFzIGEgdGVzdCBkcml2ZXIgZm9yIHRoZSBm YW1vdXM8YnI+DQomZ3Q7ICZndDtpZXRmIHZpZGVvLCBjb21wYXJpbmcgdHdvIGlkZW50aWNhbCBi b3hlcyBydW5uaW5nIGl0IGF0IHRoZSBzYW1lIHRpbWU8YnI+DQomZ3Q7ICZndDt1bmRlciBkaWZm ZXJlbnQgbmV0d29yayBjb25kaXRpb25zOjxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0Ozxh IGhyZWY9Imh0dHBzOi8vd3d3LmJ1ZmZlcmJsb2F0Lm5ldC9wcm9qZWN0cy9jZXJvd3J0L3dpa2kv QmxvYXQtdmlkZW9zI0lFVEYtZGVtby1zIj5odHRwczovL3d3dy5idWZmZXJibG9hdC5uZXQvcHJv amVjdHMvY2Vyb3dydC93aWtpL0Jsb2F0LXZpZGVvcyNJRVRGLWRlbW8tczwvYT48YnI+DQomZ3Q7 ICZndDtpZGUtYnktc2lkZS1vZi1hLW5vcm1hbC1jYWJsZS1tb2RlbS12cy1mcV9jb2RlbDxicj4N CiZndDsgJmd0Ozxicj4NCiZndDsgJmd0O2Zyb20gd2hhdCBJIGZpZGRsZWQgd2l0aCB0b2RheSwg aXQgaXMgYXQgbGVhc3Qgc3RpbGwgdXNlZnVsIGZvciB0aGF0Pzxicj4NCiZndDsgJmd0Ozxicj4N CiZndDsgJmd0O21vdmluZyBvbi4uLjxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0O1RoZSB3 ZWIgbW9kZWwgaW4gdGhlIGNhYmxlbGFicyB3b3JrIGRvZXNuJ3QgbG9vayBtdWNoIGxpa2UgbXkg Y2FwdHVyZXMsPGJyPg0KJmd0OyAmZ3Q7aW4gYWRkaXRpb24gdG8gbm90IG1vZGVsaW5nIGRucyBh dCBhbGwsIGFuZCB1c2luZyBhIHNtYWxsZXIgSVcgdGhhbiBnb29nbGU8YnI+DQomZ3Q7ICZndDtp dCBsb29rcyBsaWtlIHRoaXM6PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsg TW9kZWwgc2luZ2xlIHVzZXIgd2ViIHBhZ2UgZG93bmxvYWQgYXMgZm9sbG93czo8YnI+DQomZ3Q7 ICZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyAtIFdlYiBwYWdlIG1vZGVsZWQgYXMgc2luZ2xl IEhUTUwgcGFnZSAmIzQzOyAxMDAgb2JqZWN0cyBzcHJlYWQgZXZlbmx5PGJyPg0KJmd0OyAmZ3Q7 Jmd0OyZndDsgYWNyb3NzIDQgc2VydmVycy4gV2ViIG9iamVjdCBzaXplcyBhcmUgY3VycmVudGx5 IGZpeGVkIGF0IDI1IGtCPGJyPg0KJmd0OyBlYWNoLDxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IHdo ZXJlYXMgdGhlIGluaXRpYWwgSFRNTCBwYWdlIGlzIDEwMCBrQi4gQXBwZW5kaXggQSBwcm92aWRl cyBhbjxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IGFsdGVybmF0aXZlIHBhZ2UgbW9kZWwgdGhhdCBt YXkgYmUgZXhwbG9yZWQgaW4gZnV0dXJlIHdvcmsuPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAm Z3Q7V2hlcmUgd2hhdCBJIHNlZSBpcyBhIGh1Z2UgbnVtYmVyIG9mIHN0dWZmIHRoYXQgZml0cyBp bnRvIGEgc2luZ2xlPGJyPg0KJmd0OyAmZ3Q7aXcxMCBzbG93IHN0YXJ0IGVwaXNvZGUgYW5kIHNv bWUgbGV2ZWwgb2YgcGlwZWxpbmluZyBvbiBsYXJnZXIgc3R1ZmYsIHNvPGJyPg0KJmd0OyAmZ3Q7 dGhhdCBhPGJyPg0KJmd0OyAmZ3Q7bGFyZ2UgbnVtYmVyIG9mIG9iamVjdCBzaXplcyBvZiBsZXNz IHRoYW4gN2sgd2l0aCBhIGxpZ2h0bHkgdGFpbGVkPGJyPg0KJmd0OyAmZ3Q7ZGlzdHJpYnV0aW9u PGJyPg0KJmd0OyAmZ3Q7b3V0c2lkZSBvZiB0aGF0IG1ha2VzIG1vcmUgc2Vuc2UuPGJyPg0KJmd0 OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7KEknbSBub3Qgc3RhcmluZyBhdCBhcHBlbmRpeCBBIHJpZ2h0 IG5vdywgSSdtIHVuZGVyIHRoZSBpbXByZXNzaW9uPGJyPg0KJmd0OyAmZ3Q7IGl0IHdhcyBiZXR0 ZXIpPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7SSBjZXJ0YWlubHkgd291bGQgbGlrZSBt b3JlIHN1Z2dlc3Rpb25zIGZvciBtb2RlbHMgYW5kIHR5cGVzPGJyPg0KJmd0OyAmZ3Q7b2Ygd2Vi IHRyYWZmaWMsIGFzIHdlbGwgYXMgc2ltdWxhdGlvbiBvZiBodHRwcyAmIzQzOyBwZnMgdHJhZmZp Yyw8YnI+DQomZ3Q7ICZndDtzcGR5LCBxdWljLCBldGMuLi4uPGJyPg0KJmd0OyAmZ3Q7PGJyPg0K Jmd0OyAmZ3Q7Jmd0OyZndDsgLSBTZXJ2ZXIgUlRUcyBzZXQgYXMgZm9sbG93cyAoMjAgbXMsIDMw IG1zLCA1MCBtcywgMTAwIG1zKS48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDtTZXJ2ZXIg UlRUcyBmcm9tIG15IG93biB3ZWIgaGlzdG9yeSB0ZW5kIHRvIGJlIGxvd2VyIHRoYW4gNTBtcy48 YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyAtIEluaXRpYWwgSFRUUCBHRVQg dG8gcmV0cmlldmUgYSBtb2RlcmF0ZWx5IHNpemVkIG9iamVjdCAoMTAwIGtCPGJyPg0KJmd0OyBI VE1MPGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgcGFnZSkgZnJvbSBzZXJ2ZXIgMS48YnI+DQomZ3Q7 ICZndDs8YnI+DQomZ3Q7ICZndDtBbiBpbml0aWFsIEdFVCB0byBnb29nbGUgZml0cyBpbnRvIGl3 MTAgLSBpdCdzIGFib3V0IDdrLjxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0OyZndDsmZ3Q7 IC0gT25jZSBpbml0aWFsIEhUVFAgR0VUIGNvbXBsZXRlcywgaW5pdGlhdGUgMjQgc2ltdWx0YW5l b3VzIEhUVFA8YnI+DQomZ3Q7IEdFVHM8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyAodmlhIHNlcGFy YXRlIFRDUCBjb25uZWN0aW9ucyksIDYgY29ubmVjdGlvbnMgZWFjaCB0byA0IGRpZmZlcmVudDxi cj4NCiZndDsgJmd0OyZndDsmZ3Q7IHNlcnZlciBub2Rlczxicj4NCiZndDsgJmd0Ozxicj4NCiZn dDsgJmd0O0kgdXN1YWxseSBkb24ndCBzZWUgbW9yZSB0aGFuIDE1LiBhbmQgY2VydGFpbmx5IG5v dCAyNWsgc2l6ZWQgb2JqZWN0cy48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsgJmd0OyAt IE9uY2UgZWFjaCBpbmRpdmlkdWFsIEhUVFAgR0VUIGNvbXBsZXRlcywgaW5pdGlhdGUgYSBzdWJz ZXF1ZW50IEdFVDxicj4NCiZndDsgJmd0OyZndDsgdG8gdGhlIHNhbWUgc2VydmVyLCB1bnRpbCAy NSBvYmplY3RzIGhhdmUgYmVlbiByZXRyaWV2ZWQgZnJvbSBlYWNoPGJyPg0KJmd0OyAmZ3Q7Jmd0 OyBzZXJ2ZXIuPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0 OyAqIFdlIGRvbid0IG1ha2Ugc3VyZSB0byBmbHVzaCBhbGwgdGhlIG5ldHdvcmsgc3RhdGUgaW4g YmV0d2VlbiBydW5zLCBzbzxicj4NCiZndDsgJmd0OyZndDtpZjxicj4NCiZndDsgJmd0OyZndDsg eW91J3JlIHVzaW5nIHRoYXQgb3B0aW9uLCBkb24ndCB0cnVzdCBpdCB0byB3b3JrLjxicj4NCiZn dDsgJmd0Ozxicj4NCiZndDsgJmd0O1RoZSB0eXBpY2FsIHNjZW5hcmlvIHdlIHVzZWQgd2FzIGEg cnVuIGFnYWluc3QgZG96ZW5zIG9yIGh1bmRyZWRzIG9mIHVybHMsPGJyPg0KJmd0OyAmZ3Q7Y2Fw dHVyaW5nIHRyYWZmaWMsIHdoaWxlIHZhcnlpbmcgbmV0d29yayBjb25kaXRpb25zLjxicj4NCiZn dDsgJmd0Ozxicj4NCiZndDsgJmd0O1JlZ2FyZGVkIHRoZSBmaXJzdCBydW4gYXMgdGhlIG1vc3Qg aW50ZXJlc3RpbmcuPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Q2FuIGV4aXQgdGhlIGJy b3dzZXIgYW5kIHJlc3RhcnQgYWZ0ZXIgYSBydW4gbGlrZSB0aGF0Ljxicj4NCiZndDsgJmd0Ozxi cj4NCiZndDsgJmd0O0F0IG1vbWVudCwgbWVyZWx5IHBsYW4gdG8gdXNlIHRoZSB0b29sIHByaW1h cmlseSB0byBzdXJ2ZXkgdmFyaW91czxicj4NCiZndDsgJmd0O3dlYiBzaXRlcyBhbmQgbG9hZCB0 aW1lcyB3aGlsZSBkb2luZyBwYWNrZXQgY2FwdHVyZXMuIEhvcGUgd2FzPGJyPg0KJmd0OyAmZ3Q7 dG8gZ2V0IHZhbGlkIGRhdGEgZnJvbSB0aGUgbmV0d29yayBwb3J0aW9uIG9mIHRoZSBsb2FkLCB0 aG8uLi48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7ICogSWYgeW91IGhhdmUgYW4g YWR2YW5jZWQgQ2hyb21pdW0gc2V0dXAsIHRoaXMgZGVmaW5pdGVseSBkb2VzIG5vdDxicj4NCiZn dDsgJmd0OyZndDt3b3JrLiBJPGJyPg0KJmd0OyAmZ3Q7Jmd0OyBhZHZpc2UgdXNpbmcgdGhlIGJl bmNobWFyayBleHRlbnNpb24gb25seSB3aXRoIGEgc2VwYXJhdGUgQ2hyb21pdW08YnI+DQomZ3Q7 ICZndDsmZ3Q7cHJvZmlsZTxicj4NCiZndDsgJmd0OyZndDsgZm9yIHRlc3RpbmcgcHVycG9zZXMu IE91ciBmbHVzaGluZyBvZiBzb2NrZXRzLCBjYWNoZXMsIGV0YyBkb2VzIG5vdDxicj4NCiZndDsg Jmd0OyZndDthY3R1YWxseTxicj4NCiZndDsgJmd0OyZndDsgd29yayBjb3JyZWN0bHkgd2hlbiB5 b3UgdXNlIHRoZSBDaHJvbWl1bSBtdWx0aXByb2ZpbGUgZmVhdHVyZSBhbmQgYWxzbzxicj4NCiZn dDsgJmd0OyZndDtmYWlsczxicj4NCiZndDsgJmd0OyZndDsgdG8gZmx1c2ggbG90cyBvZiBvdXIg b3RoZXIgbmV0d29yayBjYWNoZXMuPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7bm90ZWQu PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0OyAqIE5vIG9u ZSBvbiBDaHJvbWl1bSByZWFsbHkgYmVsaWV2ZXMgdGhlIHRpbWUgdG8gcGFpbnQgbnVtYmVycyB0 aGF0IHdlPGJyPg0KJmd0OyAmZ3Q7Jmd0OyBvdXRwdXQgOikgSXQncyBjb21wbGljYXRlZC4gT3Vy IGdyYXBoaWNzIHN0YWNrIGlzIGNvbXBsaWNhdGVkLiBUaGUgdGltZTxicj4NCiZndDsgJmd0OyZn dDtmcm9tPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7SSBhY3R1YWxseSBjYXJlIG9ubHkg YWJvdXQgdGltZS10by1mdWxsIGxheW91dCBhcyB0aGF0J3MgYSBjb3JlIG5ldHdvcms8YnI+DQom Z3Q7ICZndDtlZmZlY3QuLi48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7IHdoZW4g QmxpbmsgdGhpbmtzIGl0IHBhaW50ZWQgdG8gd2hlbiB0aGUgR1BVIGFjdHVhbGx5IGJsaXRzIHRv IHRoZTxicj4NCiZndDsgJmd0OyZndDtzY3JlZW48YnI+DQomZ3Q7ICZndDsmZ3Q7IGNhbm5vdCBj dXJyZW50bHkgYmUgY29ycm9ib3JhdGVkIHdpdGggYW55IGhpZ2ggZGVncmVlIG9mIGFjY3VyYWN5 IGZyb208YnI+DQomZ3Q7ICZndDsmZ3Q7IHdpdGhpbiBvdXIgY29kZS48YnI+DQomZ3Q7ICZndDs8 YnI+DQomZ3Q7ICZndDsmZ3Q7ICogSXQgaGFzIG5vdCBiZWVuIG1haW50YWluZWQgc2luY2UgMjAx MC4gSXQgaXMgcXVpdGUgbGlrZWx5IHRoZXJlIGFyZTxicj4NCiZndDsgJmd0OyZndDttYW55PGJy Pg0KJmd0OyAmZ3Q7Jmd0OyBvdGhlciBzdWJ0bGUgaW5hY2N1cmFjaWVzIGhlcmUuPGJyPg0KJmd0 OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7R3Jvay48YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsm Z3Q7IEluIHNob3J0LCB3aGlsZSB5b3UgY2FuIGV4cGVjdCBpdCB0byBnaXZlIHlvdSBhIHZlcnkg aGlnaCBsZXZlbDxicj4NCiZndDsgJmd0OyZndDsgdW5kZXJzdGFuZGluZyBvZiBwZXJmb3JtYW5j ZSBpc3N1ZXMsIEkgYWR2aXNlIGFnYWluc3QgcGxhY2luZzxicj4NCiZndDsgJmd0OyZndDtub24t dHJpdmlhbDxicj4NCiZndDsgJmd0OyZndDsgY29uZmlkZW5jZSBpbiB0aGUgYWNjdXJhY3kgb2Yg dGhlIG51bWJlcnMgZ2VuZXJhdGVkIGJ5IHRoZSBiZW5jaG1hcms8YnI+DQomZ3Q7ICZndDsmZ3Q7 IGV4dGVuc2lvbi4gVGhlIGZhY3QgdGhhdCBudW1iZXJzIGFyZSBwcm9kdWNlZCBieSB0aGUgZXh0 ZW5zaW9uIHNob3VsZDxicj4NCiZndDsgJmd0OyZndDtub3QgYmU8YnI+DQomZ3Q7ICZndDsmZ3Q7 IHRyZWF0ZWQgYXMgZXZpZGVuY2UgdGhhdCB0aGUgZXh0ZW5zaW9uIGFjdHVhbGx5IGZ1bmN0aW9u cyBjb3JyZWN0bHkuPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7T0ssIG5vdGVkLiBTdGls bCBkZWxpZ2h0ZWQgdG8gYmUgYWJsZSB0byBoYXZlIGEgc2ltcGxlIGxvYWQgZ2VuZXJhdG9yPGJy Pg0KJmd0OyAmZ3Q7dGhhdCBleGVyY2lzZXMgdGhlIGJyb3dzZXJzIGFuZCBnZW5lcmF0ZXMgc29t ZSByZXN1bHRzLCBob3dldmVyPGJyPg0KJmd0OyAmZ3Q7ZHViaW91cy48YnI+DQomZ3Q7ICZndDs8 YnI+DQomZ3Q7ICZndDsmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0OyBDaGVlcnMuPGJyPg0KJmd0OyAm Z3Q7Jmd0Ozxicj4NCiZndDsgJmd0OyZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7IE9uIFRodSwgQXBy IDE3LCAyMDE0IGF0IDEwOjQ5IEFNLCBEYXZlIFRhaHQgJmx0OzxhIGhyZWY9Im1haWx0bzpkYXZl LnRhaHRAZ21haWwuY29tIj5kYXZlLnRhaHRAZ21haWwuY29tPC9hPiZndDs8YnI+DQomZ3Q7IHdy b3RlOjxicj4NCiZndDsgJmd0OyZndDsmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgR2V0dGlu ZyBhIGdyaXAgb24gcmVhbCB3ZWIgcGFnZSBsb2FkIHRpbWUgYmVoYXZpb3IgaW4gYW4gYWdlIG9m PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgc2hhcmRlZCB3ZWJzaXRlcyw8YnI+DQomZ3Q7ICZndDsm Z3Q7Jmd0OyBkb3plbnMgb2YgZG5zIGxvb2t1cHMsIGphdmFzY3JpcHQsIGFuZCBmYWlybHkgcmFu ZG9tIGJlaGF2aW9yIGluIGFkPGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgc2VydmljZXM8YnI+DQom Z3Q7ICZndDsmZ3Q7Jmd0OyBhbmQgY2RucyBhZ2FpbnN0IGhvdyBhIG1vZGVybiBicm93c2VycyBi ZWhhdmVzIGlzIHZlcnksIHZlcnkgaGFyZC48YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0Ozxicj4NCiZn dDsgJmd0OyZndDsmZ3Q7IGl0IHR1cm5zIG91dCBpZiB5b3UgcnVuPGJyPg0KJmd0OyAmZ3Q7Jmd0 OyZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyBnb29nbGUtY2hyb21lIC0tZW5hYmxlLWJlbmNo bWFya2luZyAtLWVuYWJsZS1uZXQtYmVuY2htYXJraW5nPGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDs8 YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyAoTWFjIHVzZXJzIGhhdmUgdG8gZW1iZWQgdGhlc2Ugb3B0 aW9ucyBpbiB0aGVpciBzdGFydHVwIHNjcmlwdCAtIHNlZTxicj4NCiZndDsgJmd0OyZndDsmZ3Q7 IDxhIGhyZWY9Imh0dHA6Ly93d3cuY2hyb21pdW0ub3JnL2RldmVsb3BlcnMvaG93LXRvcy9ydW4t Y2hyb21pdW0td2l0aC1mbGFncyI+DQpodHRwOi8vd3d3LmNocm9taXVtLm9yZy9kZXZlbG9wZXJz L2hvdy10b3MvcnVuLWNocm9taXVtLXdpdGgtZmxhZ3M8L2E+PGJyPg0KJmd0OyApPGJyPg0KJmd0 OyAmZ3Q7Jmd0OyZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyBlbmFibGUgZGV2ZWxvcGVyIG9w dGlvbnMgYW5kIGluc3RhbGwgYW5kIHJ1biB0aGUgY2hyb21lIHdlYiBwYWdlPGJyPg0KJmd0OyAm Z3Q7Jmd0OyZndDsgYmVuY2htYXJrZXIsPGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgKDxicj4NCiZn dDsgJmd0OyZndDsmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDs8YSBocmVmPSJodHRwczovL2No cm9tZS5nb29nbGUuY29tL3dlYnN0b3JlL2RldGFpbC9wYWdlLWJlbmNobWFya2VyL2NoYW5uaW1m ZG9tYWgiPmh0dHBzOi8vY2hyb21lLmdvb2dsZS5jb20vd2Vic3RvcmUvZGV0YWlsL3BhZ2UtYmVu Y2htYXJrZXIvY2hhbm5pbWZkb21haDwvYT48YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0O2VramNhaGxi cGNjYmdhb3BqbGw/aGw9ZW48YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyApPGJyPg0KJmd0OyAmZ3Q7 Jmd0OyZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyB0aGF0IGl0IHdvcmtzIChhdCBsZWFzdCBm b3IgbWUsIG9uIGEgYnJpZWYgdGVzdCBvZiB0aGUgbGF0ZXN0PGJyPg0KJmd0OyBjaHJvbWUsPGJy Pg0KJmd0OyAmZ3Q7Jmd0OyZndDtvbjxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IGxpbnV4Ljxicj4N CiZndDsgJmd0OyZndDsmZ3Q7IENhbiBzb21lb25lIHRyeSB3aW5kb3dzIGFuZCBtYWM/KTxicj4N CiZndDsgJmd0OyZndDsmZ3Q7PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgWW91IGNhbiB0aGVuIGZl ZWQgaW4gYSBsaXN0IG9mIHVybHMgdG8gdGVzdCBhZ2FpbnN0LCBhbmQgcG9zdDxicj4NCiZndDsg cHJvY2Vzczxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IHRoZSByZXN1bHRpbmcgLmNzdiBmaWxlIHRv IHlvdXIgaGVhcnRzIGNvbnRlbnQuIFdlIHVzZWQgdG8gdXNlIHRoaXM8YnI+DQomZ3Q7ICZndDsm Z3Q7Jmd0OyBiZW5jaG1hcmsgYSBsb3Qgd2hpbGUgdHJ5aW5nIHRvIGNoYXJhY3RlcmlzZSB0eXBp Y2FsIHdlYiBiZWhhdmlvcnM8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyB1bmRlciBhcW0gYW5kIHBh Y2tldCBzY2hlZHVsaW5nIHN5c3RlbXMgdW5kZXIgbG9hZC4gUnVubmluZzxicj4NCiZndDsgJmd0 OyZndDsmZ3Q7IGl0IHNpbXVsdGFuZW91c2x5IHdpdGggYSBycnVsIHRlc3Qgb3Igb25lIG9mIHRo ZSBzaW1wbGVyIHRjcCB1cGxvYWQ8YnI+DQomZ3Q7IG9yPGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsg ZG93bmxvYWQ8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyB0ZXN0cyBpbiB0aGUgcnJ1bCBzdWl0ZSB3 YXMgb2Z0ZW4gcXVpdGUgaW50ZXJlc3RpbmcuPGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDs8YnI+DQom Z3Q7ICZndDsmZ3Q7Jmd0OyBJdCB0dXJuZWQgb3V0IHRoZSBkb2MgaGFzIGJlZW4gd3JvbmcgYSB3 aGlsZSBhcyB0byB0aGUgbmFtZSBvZiB0aGU8YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0O3NlY29uZDxi cj4NCiZndDsgJmd0OyZndDsmZ3Q7IGNvbW1hbmQgbG5pZSBvcHRpb24uIEkgd2FzIGdlYXJpbmcg dXAgbWVudGFsbHkgZm9yIGhhdmluZyB0byBsb29rIGF0PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsg dGhlIHNvdXJjZS4uLi48YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0Ozxicj4NCiZndDsgJmd0OyZndDsm Z3Q7IDxhIGhyZWY9Imh0dHA6Ly9jb2RlLmdvb2dsZS5jb20vcC9jaHJvbWl1bS9pc3N1ZXMvZGV0 YWlsP2lkPTMzODcwNSI+aHR0cDovL2NvZGUuZ29vZ2xlLmNvbS9wL2Nocm9taXVtL2lzc3Vlcy9k ZXRhaWw/aWQ9MzM4NzA1PC9hPjxicj4NCiZndDsgJmd0OyZndDsmZ3Q7PGJyPg0KJmd0OyAmZ3Q7 Jmd0OyZndDsgL21lIGhhcHB5PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDs8YnI+DQomZ3Q7ICZndDsm Z3Q7Jmd0OyAtLTxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IERhdmUgVMOkaHQ8YnI+DQomZ3Q7ICZn dDsmZ3Q7Jmd0Ozxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IEhlYXJ0YmxlZWQgUE9DIG9uIHdpZmkg Y2FtcHVzIG5ldHdvcmtzIHdpdGggRUFQIGF1dGg6PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgPGEg aHJlZj0iaHR0cDovL3d3dy5lZHVyb2FtLmVkdS5hdS9hZHZpc29yeS5odG1sIj5odHRwOi8vd3d3 LmVkdXJvYW0uZWR1LmF1L2Fkdmlzb3J5Lmh0bWw8L2E+PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDs8 YnI+DQomZ3Q7ICZndDsmZ3Q7Jmd0OyBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXzxicj4NCiZndDsgJmd0OyZndDsmZ3Q7IGFxbSBtYWlsaW5nIGxpc3Q8YnI+ DQomZ3Q7ICZndDsmZ3Q7Jmd0OyA8YSBocmVmPSJtYWlsdG86YXFtQGlldGYub3JnIj5hcW1AaWV0 Zi5vcmc8L2E+PGJyPg0KJmd0OyAmZ3Q7Jmd0OyZndDsgPGEgaHJlZj0iaHR0cHM6Ly93d3cuaWV0 Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9hcW0iPmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4v bGlzdGluZm8vYXFtPC9hPjxicj4NCiZndDsgJmd0OyZndDs8YnI+DQomZ3Q7ICZndDsmZ3Q7PGJy Pg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7 LS08YnI+DQomZ3Q7ICZndDtEYXZlIFTDpGh0PGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7 TlNGVzo8YnI+DQomZ3Q7ICZndDs8YSBocmVmPSJodHRwczovL3cyLmVmZi5vcmcvQ2Vuc29yc2hp cC9JbnRlcm5ldF9jZW5zb3JzaGlwX2JpbGxzL3J1c3NlbGxfMDI5Nl9pbmRlYyI+aHR0cHM6Ly93 Mi5lZmYub3JnL0NlbnNvcnNoaXAvSW50ZXJuZXRfY2Vuc29yc2hpcF9iaWxscy9ydXNzZWxsXzAy OTZfaW5kZWM8L2E+PGJyPg0KJmd0OyAmZ3Q7ZW50LmFydGljbGU8YnI+DQomZ3Q7ICZndDs8YnI+ DQomZ3Q7ICZndDtfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f Xzxicj4NCiZndDsgJmd0O2FxbSBtYWlsaW5nIGxpc3Q8YnI+DQomZ3Q7ICZndDs8YSBocmVmPSJt YWlsdG86YXFtQGlldGYub3JnIj5hcW1AaWV0Zi5vcmc8L2E+PGJyPg0KJmd0OyAmZ3Q7PGEgaHJl Zj0iaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9hcW0iPmh0dHBzOi8vd3d3 LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vYXFtPC9hPjxicj4NCiZndDsgPGJyPg0KJmd0OyBf X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzxicj4NCiZndDsg Q2Vyb3dydC1kZXZlbCBtYWlsaW5nIGxpc3Q8YnI+DQomZ3Q7IDxhIGhyZWY9Im1haWx0bzpDZXJv d3J0LWRldmVsQGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldCI+Q2Vyb3dydC1kZXZlbEBsaXN0cy5idWZm ZXJibG9hdC5uZXQ8L2E+PGJyPg0KJmd0OyA8YSBocmVmPSJodHRwczovL2xpc3RzLmJ1ZmZlcmJs b2F0Lm5ldC9saXN0aW5mby9jZXJvd3J0LWRldmVsIj5odHRwczovL2xpc3RzLmJ1ZmZlcmJsb2F0 Lm5ldC9saXN0aW5mby9jZXJvd3J0LWRldmVsPC9hPjxicj4NCiZndDs8L3A+DQo8L2Rpdj4NCjwv Zm9udD48L2Rpdj4NCjwvZGl2Pg0KPC9zcGFuPg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_CF76D02032E2Dgwhitecablelabscom_-- From nobody Fri Apr 18 12:41:05 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C6511A0212 for ; Fri, 18 Apr 2014 12:41:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wBykyBBcB0Ss for ; Fri, 18 Apr 2014 12:40:57 -0700 (PDT) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) by ietfa.amsl.com (Postfix) with ESMTP id 736521A01D9 for ; Fri, 18 Apr 2014 12:40:57 -0700 (PDT) Received: by mail-wi0-f178.google.com with SMTP id bs8so1031743wib.17 for ; Fri, 18 Apr 2014 12:40:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=huRzyduJCbGVE4GprxWO/SHWpDnlLoBcWpnJdcecSK4=; b=a++5dt9tZvjJoBn81IANKuIfc3hL198GkJY7c6/lYmc6f+sac2S8Sr6Xanp9WGxW9l GISk5mVsEqVorM60mzT6o/b+5aatXeZot4F9C50bKSH8NUy74nHR3DoDNJaqnCEfOJP7 raO0rU58dy7CnuXq1ZmLRmNtDhVUlEuLz2/cmslM9zS5jB26Sfnj1gG+P/a4SrUvgb+5 09GB9YJDGSppFU8Hg7obk2eQeHomyniPetJRNnmk1Tpo7Iz6P1dzmp5LE3LNkwBNRoKR CO9b5mKQFIj65xnR8Gamz1Cd2spT6+ompXvyKcTSCY0ZODbn82pstVBVRw9BI94ERhyH Qf6A== MIME-Version: 1.0 X-Received: by 10.180.76.166 with SMTP id l6mr3596182wiw.17.1397850052958; Fri, 18 Apr 2014 12:40:52 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Fri, 18 Apr 2014 12:40:52 -0700 (PDT) In-Reply-To: References: Date: Fri, 18 Apr 2014 12:40:52 -0700 Message-ID: From: Dave Taht To: Greg White Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/Pt4Sy-hEJfitQR1EGmyVtO_o4iU Cc: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 19:41:02 -0000 > The specific thing I've been concerned about was not the probability of > a dns loss, although as you note the consequences are huge - > but the frequency and cost of a cache miss and the resulting fill. > > This is a very simple namebench test against the alexa top 1000: > > http://snapon.lab.bufferbloat.net/~d/namebench/namebench_2014-03-20_1255.= html > > This is a more comprehensive one taken against my own recent web history = file. > > http://snapon.lab.bufferbloat.net/~d/namebench/namebench_2014-03-24_1541.= html > > Both of these were taken against the default SQM system in cerowrt > against a cable modem, so you can > pretty safely assume the ~20ms (middle) knee in the curve is basically > based on physical > RTT to the nearest upstream DNS server. > > And it's a benchmark so I don't generally believe in the relative hit > ratios vs a vs "normal traffic", but do think the baseline RTT, and > the knees in the curves in the cost of a miss and file are relevant. To be utterly clear here, the ~20ms knee in this data is a variable depende= nt on the RTT to the nearest upstream DNS server. Most - nearly all - ISPs - have a hefty dns server in their co-location facility, but the baseline physical RTT is dependent on the actual technology in use. The actual RTT of an query is dependent on the outstanding queue length in a single queue AQM, unless otherwise prioritized. If prioritized (cerowrt's 3 band system does this for queries coming from the router) I imagine the packet loss rate drops hugely, also. To give an extreme example of the DNS rtt problem, dns lookups over satellite links take 800+ms, and this is one reason why web proxy servers are so common in such environments as the whole query is shipped to a local-to-the-intern= et proxy server so as to avoid this rtt cost. This technique is of increasingl= y limited value in an age of e2e encryption. Also: recently we've seen increasing use of non-local or otherwise redirect= ed dns servers such as here http://www.cnet.com/news/google-confirms-turkey-is-blocking-its-dns-service= / It would be a good research project for someone to catagorize typical nearest-upstream DNS RTTs, the availability of local-to-site dns servers, hit/miss ratios in homes and small business, the cost of dnssec, etc. --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Fri Apr 18 13:41:22 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 431821A022D for ; Fri, 18 Apr 2014 13:41:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.036 X-Spam-Level: X-Spam-Status: No, score=-0.036 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, RP_MATCHES_RCVD=-0.272] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bKHURIrmgGIn for ; Fri, 18 Apr 2014 13:41:15 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id E4B791A0208 for ; Fri, 18 Apr 2014 13:41:14 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id s3IKf42Q031862; Fri, 18 Apr 2014 14:41:04 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 18 Apr 2014 14:41:04 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0174.001; Fri, 18 Apr 2014 14:41:04 -0600 From: Greg White To: Dave Taht Thread-Topic: [aqm] chrome web page benchmarker fixed Thread-Index: AQHPWmVePgFvJCotnEiEX+XB9sZQUJsWjr8AgAAjHoCAAP2yAIAAcn6A//+2MYA= Date: Fri, 18 Apr 2014 20:41:03 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.4.10.154] Content-Type: text/plain; charset="gb2312" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/UzPMKdjU-UCqH60KJPr3lLp--xo Cc: =?gb2312?B?V2lsbGlhbSBDaGFuICizwtbHsv0p?= , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 20:41:19 -0000 T24gNC8xOC8xNCwgMTowNSBQTSwgIkRhdmUgVGFodCIgPGRhdmUudGFodEBnbWFpbC5jb20+IHdy b3RlOg0KDQo+T24gRnJpLCBBcHIgMTgsIDIwMTQgYXQgMTE6MTUgQU0sIEdyZWcgV2hpdGUgPGcu d2hpdGVAY2FibGVsYWJzLmNvbT4NCj53cm90ZToNCj4+DQo+PiBUaGUgY2hvaWNlIG9mIFJUVHMg YWxzbyBjYW1lIGZyb20gdGhlIHdlYiB0cmFmZmljIGNhcHR1cmVzLiBJIHNhdw0KPj4gUlRUbWlu PTE2bXMsIFJUVG1lYW49NTMuOG1zLCBSVFRtYXg9MTM0bXMuDQo+DQo+R2V0IGEgbWVkaWFuPw0K DQpNZWRpYW4gdmFsdWUgd2FzIDYybXMuDQoNCj4NCj5NeSBvd24gc3RhdHMgYXJlIHByb2JhYmx5 IHF1aXRlIHNrZXdlZCBsb3dlciBmcm9tIGJlaW5nIGluIGNhbGlmb3JuaWEsDQo+YW5kIGRvaW5n IHNvbWUgdGVzdHMgZnJvbSBwbGFjZXMgbGlrZSBpc2Mub3JnIGluIHJlZHdvb2QgY2l0eSwgd2hp Y2gNCj5pcyBpbnNhbmVseSB3ZWxsDQo+Y28tbG9jYXRlZC4NCg0KTWluZSBhcmUgcHJvYmFibHkg c2tld2VkIHRvby4gSSB3YXMgdG9sZCB0aGF0IGdsb2JhbCBtZWRpYW4gKGF0IHRoZSB0aW1lIEkN CmNvbGxlY3RlZCB0aGlzIGRhdGEpIHdhcyBhcm91bmQgMTAwbXMuDQoNCg== From nobody Sat Apr 19 12:29:17 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49A771A00A5 for ; Sat, 19 Apr 2014 12:29:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CMnzU9G83zWl for ; Sat, 19 Apr 2014 12:29:10 -0700 (PDT) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) by ietfa.amsl.com (Postfix) with ESMTP id 779D41A00BD for ; Sat, 19 Apr 2014 12:29:10 -0700 (PDT) Received: by mail-wi0-f180.google.com with SMTP id q5so611628wiv.7 for ; Sat, 19 Apr 2014 12:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=8Ge6l1RQRnezeq3J7KCDNQvL3oYuiA6+P+uOBMEtPlc=; b=jJfcLUsX0xJxT9YOxUj3/QrpeuUOhJXzmgOFXBGfmFXRJtQHooGsxkBjgF3D5M7E4n 25HaDnuhG9fkeLbbA/XbAtQif1qbL9l0CGIMvN0lZlMsMfkdB2VkSKT+3z0oaBKTJygn u3CIMzQk8/sz7NAA5idB7ZQ9SaWa+S5j4H17f7VX/tDF15W6ZTYdYCjA8Da9HWHIPdDU qYNX8FOQjcaDYm9GwGjALpK8ImfCuP0kJpnO6HdPGkPVQE/A5ta5B9Mz4irnpgvwGwhq v193JEiTyhkE3ZIoHlrKIrihT/RWCVWt2SpENJiEgArrrGoCijBkBqLuGAjTxdX03IQ9 SyEw== MIME-Version: 1.0 X-Received: by 10.180.78.225 with SMTP id e1mr7602989wix.17.1397935745644; Sat, 19 Apr 2014 12:29:05 -0700 (PDT) Received: by 10.216.177.10 with HTTP; Sat, 19 Apr 2014 12:29:05 -0700 (PDT) In-Reply-To: References: Date: Sat, 19 Apr 2014 12:29:05 -0700 Message-ID: From: Dave Taht To: Greg White Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/awMk7bawSlIfofkAhPsMSTokrjo Cc: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 19:29:15 -0000 On Fri, Apr 18, 2014 at 1:41 PM, Greg White wrote: > On 4/18/14, 1:05 PM, "Dave Taht" wrote: > >>On Fri, Apr 18, 2014 at 11:15 AM, Greg White >>wrote: >>> >>> The choice of RTTs also came from the web traffic captures. I saw >>> RTTmin=3D16ms, RTTmean=3D53.8ms, RTTmax=3D134ms. >> >>Get a median? > > Median value was 62ms. > >> >>My own stats are probably quite skewed lower from being in california, >>and doing some tests from places like isc.org in redwood city, which >>is insanely well >>co-located. > > Mine are probably skewed too. I was told that global median (at the time = I > collected this data) was around 100ms. Well, the future is already here, just not evenly distributed. Nearly every sample I'd taken at the same time as from, almost entirely from major cities, came in at under 70ms median. It strikes me that a possibly useful metric would be object size vs RTT, over time. > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From y.elkhatib@lancaster.ac.uk Sun Apr 20 06:14:21 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1E19D1A01E6 for ; Sun, 20 Apr 2014 06:14:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.381 X-Spam-Level: ** X-Spam-Status: No, score=2.381 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_FONT_FACE_BAD=0.981, HTML_MESSAGE=0.001, J_CHICKENPOX_18=0.6, MANGLED_EFFEX=2.3, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q8rIelZGOSNk for ; Sun, 20 Apr 2014 06:14:15 -0700 (PDT) Received: from lipwig.lancs.ac.uk (lipwig.lancs.ac.uk [148.88.25.19]) by ietfa.amsl.com (Postfix) with ESMTP id 224901A01E5 for ; Sun, 20 Apr 2014 06:14:15 -0700 (PDT) Received: from ex-1-ht0.lancs.ac.uk ([10.42.18.57] helo=EX-1-HT0.lancs.local) by lipwig.lancs.ac.uk with esmtp (Exim 4.72) (envelope-from ) id 1WbrZR-0007ex-9m; Sun, 20 Apr 2014 14:14:09 +0100 Received: from EX-0-MB0.lancs.local ([fe80::7431:cb18:af99:98c8]) by EX-1-HT0.lancs.local ([fe80::d9e8:ad10:d075:a6b6%12]) with mapi id 14.03.0174.001; Sun, 20 Apr 2014 14:14:07 +0100 From: "El Khatib, Yehia (elkhatib)" To: "dave.taht@gmail.com" Thread-Topic: [aqm] The Effect of Network and Infrastructural Variables on SPDY's Performance Thread-Index: AQHPXJpoeMtKHLhZ7UyeZD0LOV2E1Q== Date: Sun, 20 Apr 2014 13:14:06 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.9.131030 x-originating-ip: [86.179.219.210] x-iss-local-domain: 1 Content-Type: multipart/alternative; boundary="_000_CF7986AC248Ayelkhatiblancasteracuk_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/USRkT8UC-ZUiTRUz8VEao54jOE0 Cc: "aqm@ietf.org" , "bloat@lists.bufferbloat.net" Subject: Re: [aqm] The Effect of Network and Infrastructural Variables on SPDY's Performance X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 13:15:35 -0000 --_000_CF7986AC248Ayelkhatiblancasteracuk_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi Dave, Thanks for your interest. The value of control variables were guided by Google=92s assertion that SPD= Y is targeted towards mobile environments. Hence, when we had to fix a para= meter, we picked a value that would be representative of many a mobile sett= ing as described in the literature. Cheers, Yehia -- Dr. Yehia Elkhatib School of Computing & Communications Lancaster University, LA1 4WA, UK y.elkhatib lancaster.ac.uk http://www.comp.lancs.ac.uk/~elkhatib From: Dave Taht > Subject: [aqm] The Effect of Network and Infrastructural Variables on SPDY'= s Performance Date: 17. april 2014 19:22:20 CEST To: bloat >= , "aqm@ietf.org" > Last night's reading was quite good: http://arxiv.org/pdf/1401.6508.pdf "As RTT goes up, it becomes increasingly expensive for HTTPS to establish separate connections for each resource. Each HTTPS connection costs one round trip on TCP handshaking and a further two on negotiating SSL setup. SPDY does this only once (per server) and hence reduces such large waste by multiplexing streams over a single connection. " ... "the separation between RTT and bandwidth is not particularly distinct. This is because HTTPS tends to operate in a somewhat network-unfriendly manner, creating queueing delays where bandwidth is low. The bursty use of HTTPS' parallel connections cre- ates congestion at the gateway queues, causing upto 3% PLR and inflating RTT by upto 570%. In contrast, SPDY causes negligible packet loss at the gateway. The network friendly behaviour of SPDY is particularly interesting as Google has recently argued for the use of a larger IW for TCP [7]. The aim of this is to reduce round trips and speed up delivery | an idea which has been criticised for potentially causing congestion. One question here is whether or not this is a strategy that is speci cally designed to oper- ate in conjunction with SPDY. To explore this, we run further tests using and bandwidth xed at 1Mbps (all other parameters as above). For HTTPS, it appears that the critics are right: RTT and loss increase greatly with larger IWs. In contrast, SPDY achieves much higher gains when increasing the IW without these negative side e ffects. " and then they inject packet loss: we inspect the impact of packet loss on SPDY's performance. We fix RTT at 150ms" Sigh, the rest of the paper is pretty good, but they should have looked at packet loss at 10-30ms at least. " and BW at 1Mbps, varying packet loss using the Linux kernel rewall with a stochastic proportional packet processing rule between 0 and 3%. . Figure 6 presents the results. Immediately, we see that SPDY is far more adversely a ffected by packet loss than HTTPS is. This has been anticipated in other work [29] but never before tested. It is also contrary to what has been reported in the SPDY white paper [2], which states that SPDY is better able to deal with loss. The authors suggest because SPDY sends fewer packets, the negative e ect of TCP backo is mitigated. We nd that SPDY does, indeed, send fewer packets (up to 49% less due to TCP connection reuse). However, SPDY's multiplexed connections persist far longer compared to HTTPS. " -- Dave T=E4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm --_000_CF7986AC248Ayelkhatiblancasteracuk_ Content-Type: text/html; charset="Windows-1252" Content-ID: <08348A20F156F9449AD91383FC3849C1@exchange.lancs.ac.uk> Content-Transfer-Encoding: quoted-printable

Hi Dave,

Thanks for you= r interest.

The value of contr= ol variables were guided by Google= =92s assertion that SPDY is ta= rgeted towards mobile environm= ents. Hence, when we had to fix a parameter, we picked a value that would be rep= resentative of many a mobile setting as described in the literature.=


Cheers,=

Yehia

-- 
Dr. Yehia Elkhatib
School of Computing & Communications
Lancaster University, LA1 4WA, UK
y.elkhatib <then add> lancaster.ac.uk
http://www.comp.lancs.ac.uk/~elkhatib

From: Dave Taht <dave.taht@gmail.com>
Subject: [aqm] The Effect of Net= work and Infrastructural Variables on SPDY's Performance
Date: 17. april 2014 19:22:20 CEST<= br>

Last night's reading was quite good:

http://arxiv.org/pdf/1401.65= 08.pdf


"As RTT goes up, it becomes increasingly
expensive for HTTPS to establish separate connections for each resource. Each HTTPS connection costs one round trip on TCP handshaking and
a further two on negotiating SSL setup. SPDY does this only once (per
server) and hence reduces such large waste by multiplexing streams over a single connection. "

...

"the separation between RTT
and bandwidth is not particularly distinct. This is because HTTPS tends to<= br> operate in a somewhat network-unfriendly manner, creating queueing delays where bandwidth is low. The bursty use of HTTPS' parallel connections cre-<= br> ates congestion at the gateway queues, causing upto 3% PLR and inflating RTT by upto 570%. In contrast, SPDY causes negligible packet loss at the gateway.

The network friendly behaviour of SPDY is particularly interesting as
Google has recently argued for the use of a larger IW for TCP [7]. The
aim of this is to reduce round trips and speed up delivery | an idea which<= br> has been criticised for potentially causing congestion. One question here is whether or not this is a strategy that is speci cally designed to oper-<= br> ate in conjunction with SPDY. To explore this, we run further tests using
and bandwidth xed at 1Mbps (all other parameters as above). For HTTPS,
it appears that the critics are right: RTT and loss increase greatly
with larger IWs. In contrast, SPDY achieves much higher gains when
increasing the IW without these negative side e ffects. "

and then they inject packet loss:

we inspect the impact of packet loss on SPDY's performance. We
fix RTT at 150ms"

Sigh, the rest of the paper is pretty good, but they should have looked at<= br> packet loss at 10-30ms at least.

" and BW at 1Mbps, varying packet loss using the Linux
kernel rewall with a stochastic proportional packet processing rule between=
0 and 3%.
. Figure 6 presents the results.
Immediately, we see that SPDY is far more adversely a ffected by packet
loss than HTTPS is. This has been anticipated in other work [29] but never<= br> before tested. It is also contrary to what has been reported in the SPDY white paper [2], which states that SPDY is better able to deal with loss. The authors suggest because SPDY sends fewer packets, the negative e ect of TCP backo is mitigated. We nd that SPDY does, indeed, send fewer
packets (up to 49% less due to TCP connection reuse). However, SPDY's
multiplexed connections persist far longer compared to HTTPS. "


-- 
Dave T=E4ht

NSFW: https://w2.eff.org/Censorship/Internet_ce= nsorship_bills/russell_0296_indecent.article

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/= mailman/listinfo/aqm

--_000_CF7986AC248Ayelkhatiblancasteracuk_-- From nobody Tue Apr 22 00:39:27 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 445A71A0116 for ; Tue, 22 Apr 2014 00:39:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.9 X-Spam-Level: X-Spam-Status: No, score=-6.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KVWQ3v4o5trY for ; Tue, 22 Apr 2014 00:39:20 -0700 (PDT) Received: from hoemail2.alcatel.com (hoemail2.alcatel.com [192.160.6.149]) by ietfa.amsl.com (Postfix) with ESMTP id B8B681A00D3 for ; Tue, 22 Apr 2014 00:39:20 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (h135-239-2-42.lucent.com [135.239.2.42]) by hoemail2.alcatel.com (8.13.8/IER-o) with ESMTP id s3M7dDQ3014327 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 22 Apr 2014 02:39:15 -0500 (CDT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id s3M7dCrY025548 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 22 Apr 2014 09:39:13 +0200 Received: from FR712WXCHMBA13.zeu.alcatel-lucent.com ([169.254.5.3]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.02.0247.003; Tue, 22 Apr 2014 09:39:12 +0200 From: "LAUTENSCHLAEGER, Wolfram (Wolfram)" To: "aqm@ietf.org" Thread-Topic: [aqm] [AQM Evaluation Guidelines] Thread-Index: AQHPWk9Tm5747IlyTkWHT6fvKQY6y5sdNrZg Date: Tue, 22 Apr 2014 07:39:08 +0000 Message-ID: <0A452E1DADEF254C9A7AC1969B878128068CD5@FR712WXCHMBA13.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> <20140417151128.GF66779@verdi> In-Reply-To: <20140417151128.GF66779@verdi> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/IohZFD2YCQ3hCEV2gtIqgMn4x-Q Cc: John Leslie Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 07:39:25 -0000 See comments in line > > Shouldn't we more concentrate on what we expect from a good AQM? >=20 > I'd love to... >=20 > > The only thing we might expect from an AQM is to prevent greedy TCP > > sources from drawing buffers permanently towards full state, >=20 > I'm not the least bit sure that's possible. :^( >=20 > However, it might be possible to separate the queue(s) of "greedy" > TCP flows from other queues. There's a lot of room for discussion about h= ow > to do that... You are talking about scheduling, not AQM. I agree with you that perfect scheduling everywhere could obsolete any AQM. But as long ... >=20 > > instead of the much better almost empty state, >=20 > Consider the class of real-time flows, which hardly ever want to stand= in line > for more than a millisecond or two. (They can generate redundancy to > compensate for packet loss, or ideally have ECN-marked packets get throug= h > quickly.) >=20 > > while maintaining full link utilization. >=20 > That's a concern of _some_ traffic, but not real-time flows. That's a concern for _all_ traffic if _some_ greedy traffic is involved,=20 no matter if real-time flows are there, too. >=20 > > In all other situations, I'm convinced, AQM cannot improve a lot. >=20 > "All other situations" covers a lot of ground... :^( >=20 > And we may not all mean the same thing by "greedy TCP". Standard TCPs > probe for additional capacity if they have anything to send. I consider t= hat > enough to call them "greedy"; but I'm sure others don't agree. I mean with "greedy TCP" a flow that is probing for additional capacity _at_this_ queue. Many TCP flows are bandwidth restricted elsewhere (up- or downstream), and as such not greedy to my understanding. >=20 > > But at least, in these cases, it should not make things worse. >=20 > "Make things worse" is perhaps insufficiently defined. >=20 > Any delay at all "makes things worse" for real-time traffic. >=20 > Failure to discover unused capacity "makes things worse" for TCP. That's what I meant. >=20 > > Of course the greedy TCP case can be overlaid by others (unresponsive > > flows, application paced flows, a renewal process of short lived > > flows, synchronized start of flows etc.etc.), >=20 > These cover quite different situations... >=20 > Unresponsive flows are a problem iff we buffer too much/many of them. Unresponsive flows are a problem if they absorb the AQM's=20 drop/mark signals, so that the responsive flows miss these and continue=20 to grow their sending rates. >=20 > Application-paced flows are a problem only if they're probing for incr= eased > capacity. >=20 > Revewal flows are a problem if they guess badly at remaining capacity. You missed my point: A renewal process of TCP flows is not a problem for AQ= M. But it is a problem for AQM _evaluation_. The 100% capacity utilization can= not be reached, no matter how good or bad the AQM is. The capacity utilization = is solely defined by the traffic definition and not by the AQM. Nevertheless w= e have such traffic in reality. If we want to evaluate this, we first should = be clear what we expect. >=20 > > and we should take this into account. But we should not expect a lot > > if those "others" dominate the scenario. >=20 > I speculate that none of these would be a problem if we limited the ex= tent > which we buffer them. >=20 > -- > John Leslie Wolfram From nobody Tue Apr 22 05:36:42 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9D5051A03EF for ; Tue, 22 Apr 2014 05:36:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id owtRGUBFXnnS for ; Tue, 22 Apr 2014 05:36:35 -0700 (PDT) Received: from nm12-vm2.access.bullet.mail.bf1.yahoo.com (nm12-vm2.access.bullet.mail.bf1.yahoo.com [216.109.114.241]) by ietfa.amsl.com (Postfix) with ESMTP id 1D29A1A03EC for ; Tue, 22 Apr 2014 05:36:35 -0700 (PDT) Received: from [66.196.81.166] by nm12.access.bullet.mail.bf1.yahoo.com with NNFMP; 22 Apr 2014 12:36:29 -0000 Received: from [98.139.221.156] by tm12.access.bullet.mail.bf1.yahoo.com with NNFMP; 22 Apr 2014 12:36:29 -0000 Received: from [127.0.0.1] by smtp116.sbc.mail.bf1.yahoo.com with NNFMP; 22 Apr 2014 12:36:29 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1398170189; bh=aBV1/gXkAS2XUYhoKiJ9U+m5ua9AhJ0uOm0ZC04Pkww=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:X-Enigmail-Version:Content-Type:Content-Transfer-Encoding; b=Wvw4OTlQO8m3dgx206Tq0hN7NvtDdk2nuWgnCFx4dIu70kor4ERusgujqBc1EnnxOq5B+2e+5E1KL+ovzrvcqq+8ta1asQ7OHMzGdWPJGuiA0gy89db8SYKzT8DMvZtr2isXHm/gBGpKegirAg73EQ29WpCj2WDzxcgmi07HBgE= X-Yahoo-Newman-Id: 563280.41041.bm@smtp116.sbc.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: Br9.6ywVM1lwi7zU1hIZuTf_yxP6gJAT7tbd3ByWvuj_Wew RTiQ7hI0EjrzQn0SsjDVHQMA13nZ2hmpQCquref9dkS8bo6KZI7qBpWbQqQH Inpb1yGRtEkObMHWKhj8L.a3BCp.9XugNpu3YV7nZ2RgqT41dJBAdHM2XG3q e_djZ7yxUlDDs1GrYp1gapfHEVtd9PCNzVh6cvJsyttsm.XcM1BWEX9XO4.k k.5.gMUzF5Zp3Cgfk_xec3hno3RumEZ88JR2aHR21aPqaaf7oWTuv.R4ZAs2 nxU9fgOVxXkXaSNNr7YtngUnqN8293P1VCQW8BnV_DW_QtnU84YcaH0_xoOv KyLEStXsyjqqm_ZCgSC5Oio45HfRrk0qFALlyrmDDTxu.DGSbE8b6qOjtIUt 5hlgwjbSfR6b3oK_asipy3qjXgArbDjum0TGm2X8jnkhgmJ4asECADw15n2C zg8pa_qrMUF0d_PrLB2i5aF5CZG94QUBsvfAh6KkVG6SwZbVsDe8A.RDWzx9 MzKfgpxbNQvNnq0xc4j9Gdh7VgXwaQXH1kg-- X-Yahoo-SMTP: sltvjZWswBCRD.ElTuB1l9j6s9wRYPpuyTNWOE5oEg-- X-Rocket-Received: from [10.111.100.146] (davec-b@74.213.188.46 with plain [98.139.221.125]) by smtp116.sbc.mail.bf1.yahoo.com with SMTP; 22 Apr 2014 12:36:29 +0000 UTC Message-ID: <5356624B.2000808@rogers.com> Date: Tue, 22 Apr 2014 08:36:27 -0400 From: David Collier-Brown User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: aqm@ietf.org References: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> <20140417151128.GF66779@verdi> <0A452E1DADEF254C9A7AC1969B878128068CD5@FR712WXCHMBA13.zeu.alcatel-lucent.com> In-Reply-To: <0A452E1DADEF254C9A7AC1969B878128068CD5@FR712WXCHMBA13.zeu.alcatel-lucent.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/gKXpnS_rXgSJ9wDziFN5zQMPaw4 Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: davecb@spamcop.net List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 12:36:40 -0000 Just a clarification: a greedy TCP will try to send more packets than will fit, and with a bad queuing algorithm will fill buffers, while with a good one will cause drops. The buffer-filling is a secondary effect, and can be avoided. The harm to other streams caused as a direct effect is the problem, and may indeed be hard to deal with, or distinguish from a flow enthusiastically probing to see if it can go faster through the bottleneck link. The fix for the latter is drops, of course. The fix for a TCP who ignores drops is ... maybe *lots* of drops? (:-)) --dave [I'm not being /totally/ serious, but improving the existing bottleneck signalling is worth discussing. Someone with knowledge of a bad TCP, please comnent] On 04/22/2014 03:39 AM, LAUTENSCHLAEGER, Wolfram (Wolfram) wrote: > See comments in line > >>> Shouldn't we more concentrate on what we expect from a good AQM? >> >> I'd love to... >> >>> The only thing we might expect from an AQM is to prevent greedy TCP >>> sources from drawing buffers permanently towards full state, >> >> I'm not the least bit sure that's possible. :^( >> >> However, it might be possible to separate the queue(s) of "greedy" >> TCP flows from other queues. There's a lot of room for discussion about how >> to do that... > > You are talking about scheduling, not AQM. I agree with you that > perfect scheduling everywhere could obsolete any AQM. But as long ... > >> >>> instead of the much better almost empty state, >> >> Consider the class of real-time flows, which hardly ever want to stand in line >> for more than a millisecond or two. (They can generate redundancy to >> compensate for packet loss, or ideally have ECN-marked packets get through >> quickly.) >> >>> while maintaining full link utilization. >> >> That's a concern of _some_ traffic, but not real-time flows. > > That's a concern for _all_ traffic if _some_ greedy traffic is involved, > no matter if real-time flows are there, too. > >> >>> In all other situations, I'm convinced, AQM cannot improve a lot. >> >> "All other situations" covers a lot of ground... :^( >> >> And we may not all mean the same thing by "greedy TCP". Standard TCPs >> probe for additional capacity if they have anything to send. I consider that >> enough to call them "greedy"; but I'm sure others don't agree. > > I mean with "greedy TCP" a flow that is probing for additional capacity > _at_this_ queue. Many TCP flows are bandwidth restricted elsewhere > (up- or downstream), and as such not greedy to my understanding. > >> >>> But at least, in these cases, it should not make things worse. >> >> "Make things worse" is perhaps insufficiently defined. >> >> Any delay at all "makes things worse" for real-time traffic. >> >> Failure to discover unused capacity "makes things worse" for TCP. > > That's what I meant. > >> >>> Of course the greedy TCP case can be overlaid by others (unresponsive >>> flows, application paced flows, a renewal process of short lived >>> flows, synchronized start of flows etc.etc.), >> >> These cover quite different situations... >> >> Unresponsive flows are a problem iff we buffer too much/many of them. > > Unresponsive flows are a problem if they absorb the AQM's > drop/mark signals, so that the responsive flows miss these and continue > to grow their sending rates. > >> >> Application-paced flows are a problem only if they're probing for increased >> capacity. >> >> Revewal flows are a problem if they guess badly at remaining capacity. > > You missed my point: A renewal process of TCP flows is not a problem for AQM. > But it is a problem for AQM _evaluation_. The 100% capacity utilization cannot > be reached, no matter how good or bad the AQM is. The capacity utilization is > solely defined by the traffic definition and not by the AQM. Nevertheless we > have such traffic in reality. If we want to evaluate this, we first should be > clear what we expect. > >> >>> and we should take this into account. But we should not expect a lot >>> if those "others" dominate the scenario. >> >> I speculate that none of these would be a problem if we limited the extent >> which we buffer them. >> >> -- >> John Leslie > > > Wolfram > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain From nobody Tue Apr 22 11:47:51 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 382F61A022B for ; Tue, 22 Apr 2014 11:47:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.472 X-Spam-Level: X-Spam-Status: No, score=-4.472 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 23KJ0cP4XhFH for ; Tue, 22 Apr 2014 11:47:44 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by ietfa.amsl.com (Postfix) with ESMTP id 858301A0226 for ; Tue, 22 Apr 2014 11:47:44 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id B474AC94BE; Tue, 22 Apr 2014 14:47:36 -0400 (EDT) Date: Tue, 22 Apr 2014 14:47:36 -0400 From: John Leslie To: davecb@spamcop.net Message-ID: <20140422184736.GE86778@verdi> References: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> <20140417151128.GF66779@verdi> <0A452E1DADEF254C9A7AC1969B878128068CD5@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5356624B.2000808@rogers.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5356624B.2000808@rogers.com> User-Agent: Mutt/1.4.1i Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/WvodqnXQdjsSzrAk63Mft0nZ6AU Cc: aqm@ietf.org Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 18:47:49 -0000 David Collier-Brown wrote: > > Just a clarification: a greedy TCP will try to send more packets than > will fit, Clarification is good! However, I'd like to work out a sufficiently precise definition of "greedy TCP" here. My personal preference would be to call a TCP "greedy" when it is probing for additional capacity, and only then. (I'm quite willing to work with a different definition; but I think we need _one_ definition when we talk of "greedy TCP". > and with a bad queuing algorithm will fill buffers, while with a > good one will cause drops. Please forgive me if you can: I like to think in terms of whether packets are arriving faster than they can be forwarded, or slower -- not in terms of what's in the buffer(s). > The buffer-filling is a secondary effect, and can be avoided. I'd say it SHOULD be avoided, except in limited cases. When TCP is _not_ probing for additional capacity, buffering helps TCP. When TCP _is_ probing for additional capacity, buffering increases the depth of the sawtooth. Buffering _always_ tends to increase delay for real-time flows. > The harm to other streams caused as a direct effect is the problem, > and may indeed be hard to deal with, Maintaining multiple buffers can help _a_lot_ here. > or distinguish from a flow enthusiastically probing to see if it can > go faster through the > bottleneck link. Yes, distinguishing the probing is hard. :^( I suggest we think in terms of assuming TCP is probing, absent any actual indicator that it isn't. > The fix for the latter is drops, of course. Dropping packets is a _poor_ fix -- though often it's the best fix available. :^( In an "ideal" world, we'd be able to identify the next packet of that flow to go out and ECN-mark it as Congestion Experienced. That's hard, of course, but it would be a better fix if everything worked that way. > The fix for a TCP who ignores drops is ... maybe *lots* of drops? (:-)) Excellent question! But first we should ask, can we tell that's happening? If we could, I'd suggest turning off all buffering for that flow. That would probably drop *lots* of packets when we turn it off. And it seems quite fair to deny the benefits of buffering to any TCP flow that doesn't cut back its sending rate. -- John Leslie From davecb.42@gmail.com Tue Apr 22 12:17:26 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F17DF1A003D for ; Tue, 22 Apr 2014 12:17:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.75 X-Spam-Level: X-Spam-Status: No, score=-1.75 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xi3j9_axZhel for ; Tue, 22 Apr 2014 12:17:21 -0700 (PDT) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) by ietfa.amsl.com (Postfix) with ESMTP id A894D1A0076 for ; Tue, 22 Apr 2014 12:17:21 -0700 (PDT) Received: by mail-ie0-f179.google.com with SMTP id lx4so5705938iec.38 for ; Tue, 22 Apr 2014 12:17:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=7ScfK+YYWRkgt9V+AJ7v6EBJK8eT94ok60T5uolJl9U=; b=biVtchX95yj5VKpoeG68AFg8EAm2tybcj5P+ymuKEpBGA7sx4X/NNYE7bj82qi0skN 6zt1RS64JJEIYHGk5gi/52K21tPB7QmUwQaWxvz3bSLryNeXv4k/VhqugItOrrNtNRgt ExD79B1f0n4LATRIE978GunwKeetd/Pd3h/uGa0mTeFD88UDqlv5x8s6xS43KIWnjazt lAmJ3kJaHpMsK9ndoOWzPSkjRzPLycvE6bvpHQg8TQm+GT6aZEFbj+bU3zQAljqJFvzL Yvy7shpKtym5zeOGtEj/M9PaPspkETGJEv1sYsCWsXFCXOtQAhNgZpkO4CY+i0jUYWvm yVaA== X-Received: by 10.42.92.82 with SMTP id s18mr10362716icm.64.1398194236173; Tue, 22 Apr 2014 12:17:16 -0700 (PDT) Received: from [10.111.100.146] ([74.213.188.46]) by mx.google.com with ESMTPSA id lp4sm30296374igb.12.2014.04.22.12.17.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 22 Apr 2014 12:17:14 -0700 (PDT) X-Google-Original-From: David Collier-Brown Message-ID: <5356C037.50803@spamcop.net> Date: Tue, 22 Apr 2014 15:17:11 -0400 From: David Collier-Brown User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Leslie References: <0A452E1DADEF254C9A7AC1969B878128068957@FR712WXCHMBA13.zeu.alcatel-lucent.com> <20140417151128.GF66779@verdi> <0A452E1DADEF254C9A7AC1969B878128068CD5@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5356624B.2000808@rogers.com> <20140422184736.GE86778@verdi> In-Reply-To: <20140422184736.GE86778@verdi> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/P3i8RI0es1kU96p135lQs2H-dLQ X-Mailman-Approved-At: Wed, 23 Apr 2014 04:59:18 -0700 Cc: aqm@ietf.org Subject: Re: [aqm] [AQM Evaluation Guidelines] X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 19:18:40 -0000 Strongly agree!!! I suspect it might be useful to think of the probing as a "greedy algorithm" that is stopped by drops, so that a nutso-evil scheme gets stopped by drops the same way. I just don't know much about what I've now defined as nutso-evil TCP (:-)) --dave [This is *not* suggested as a new term of art] On 04/22/2014 02:47 PM, John Leslie wrote: > David Collier-Brown wrote: >> Just a clarification: a greedy TCP will try to send more packets than >> will fit, > Clarification is good! > > However, I'd like to work out a sufficiently precise definition of > "greedy TCP" here. > > My personal preference would be to call a TCP "greedy" when it is > probing for additional capacity, and only then. > > (I'm quite willing to work with a different definition; but I think > we need _one_ definition when we talk of "greedy TCP". > >> and with a bad queuing algorithm will fill buffers, while with a >> good one will cause drops. > Please forgive me if you can: I like to think in terms of whether > packets are arriving faster than they can be forwarded, or slower -- > not in terms of what's in the buffer(s). > >> The buffer-filling is a secondary effect, and can be avoided. > I'd say it SHOULD be avoided, except in limited cases. > > When TCP is _not_ probing for additional capacity, buffering helps > TCP. > > When TCP _is_ probing for additional capacity, buffering increases > the depth of the sawtooth. > > Buffering _always_ tends to increase delay for real-time flows. > >> The harm to other streams caused as a direct effect is the problem, >> and may indeed be hard to deal with, > Maintaining multiple buffers can help _a_lot_ here. > >> or distinguish from a flow enthusiastically probing to see if it can >> go faster through the > bottleneck link. > Yes, distinguishing the probing is hard. :^( > > I suggest we think in terms of assuming TCP is probing, absent any > actual indicator that it isn't. > >> The fix for the latter is drops, of course. > Dropping packets is a _poor_ fix -- though often it's the best fix > available. :^( > > In an "ideal" world, we'd be able to identify the next packet of > that flow to go out and ECN-mark it as Congestion Experienced. That's > hard, of course, but it would be a better fix if everything worked > that way. > >> The fix for a TCP who ignores drops is ... maybe *lots* of drops? (:-)) > Excellent question! > > But first we should ask, can we tell that's happening? > > If we could, I'd suggest turning off all buffering for that flow. > That would probably drop *lots* of packets when we turn it off. And > it seems quite fair to deny the benefits of buffering to any TCP flow > that doesn't cut back its sending rate. > > -- > John Leslie > -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain From nobody Sun Apr 27 12:06:56 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5FB551A0693 for ; Sun, 27 Apr 2014 12:06:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lMYiStq5CKa3 for ; Sun, 27 Apr 2014 12:06:51 -0700 (PDT) Received: from nm13-vm9.access.bullet.mail.gq1.yahoo.com (nm13-vm9.access.bullet.mail.gq1.yahoo.com [216.39.63.251]) by ietfa.amsl.com (Postfix) with ESMTP id E0E341A0675 for ; Sun, 27 Apr 2014 12:06:50 -0700 (PDT) Received: from [216.39.60.166] by nm13.access.bullet.mail.gq1.yahoo.com with NNFMP; 27 Apr 2014 19:06:50 -0000 Received: from [67.195.22.118] by tm2.access.bullet.mail.gq1.yahoo.com with NNFMP; 27 Apr 2014 19:06:50 -0000 Received: from [127.0.0.1] by smtp113.sbc.mail.gq1.yahoo.com with NNFMP; 27 Apr 2014 19:06:50 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1398625610; bh=QktU5tDG2FBNyNvmRjc2l+CmzQgho4mp4kwWf53KN0s=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:X-Enigmail-Version:X-Forwarded-Message-Id:Content-Type:Content-Transfer-Encoding; b=tHP/q3TkGvCxfsUH62eQWCJtyMZNKe4wLKSMobaqon1CSa0bkjyywbYfURqX7D6UuoA2RfjDY7m5XPZEECHBLfkczVpOLs2Gnz6u5x49rOtn9dukPgEEjBqaf/rttXHBDEJ0BN4ewNG8C6L7tfdBtlM4ALAwcBCS77FYVuWMn6k= X-Yahoo-Newman-Id: 705177.71730.bm@smtp113.sbc.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: DOWxPtYVM1mDR2C6j.MrIoI.ylm5iGhF9zk9Iy7_QzsrLuR JjhWpjfcycW_C59nkEc.heIpeB7uVYxNNutT4Tso1ohUYDlaHlC6_lLM8Vxr kZD14PzSeaDycJAczEmJ4VYh80L0z7OqdS_lCJiMI08O2R419GgwuoWinHl0 vvHoJ8CXbDhJW3.L3Lif3KplnQUyVg.gjwTbaV4yi_yA10DiXaXW4tGZYXYL ombTFCjW.81pr9PZAOKy90GBkL_e9XFAdzBtgwK6iOm8UYhuIMJoTn4xt0UH Hyl_N8P0mzPXCbxbH_gkgM8irUfblo6y7MdYnjcbkBKgGVWFYjQaxVyWIAGf kY3sNAWmEu5rOvtSDQ7tPz2MjlsJ7Q3In.vtyCiomrdtK0TUGR4wnoKjKAFJ LB21wyUhTcwuT2q1lzoYuC2zi_99gRLDP3Pewg8ixzQAWbJaAguZ9WcmCnS0 8ud_XuUqULMENdHp4Uo567zuTTVftjjfxMq5LtFTSVQHosVAGKhnVvh20iwo 7JjI_n2jhFM7YC4RniatMf9Dq51F2yL6alQ0adbpszW6lUVgFTQeLU0shCtM B.iWRUSHPvtzQ2h1Ukvuv6QrVxbctxN7ZEPsQiQ-- X-Yahoo-SMTP: sltvjZWswBCRD.ElTuB1l9j6s9wRYPpuyTNWOE5oEg-- X-Rocket-Received: from [192.168.0.13] (davec-b@99.225.150.135 with plain [67.195.22.104]) by smtp113.sbc.mail.gq1.yahoo.com with SMTP; 27 Apr 2014 12:06:50 -0700 PDT Message-ID: <535D5548.1000208@rogers.com> Date: Sun, 27 Apr 2014 15:06:48 -0400 From: David Collier-Brown User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: "aqm@ietf.org" , bloat@lists.bufferbloat.net References: <535D4F22.6060602@cobb.uk.net> In-Reply-To: <535D4F22.6060602@cobb.uk.net> X-Enigmail-Version: 1.6 X-Forwarded-Message-Id: <535D4F22.6060602@cobb.uk.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/ejJTvepcBeJh3rHOsiyWJeBmD2E Subject: [aqm] Slightly off-topic: [Internet Policy] FCC's latest action - two questions X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: davecb@spamcop.net List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Apr 2014 19:06:53 -0000 While this discussion is about policy, the implementation is either - TCP flow/congestion control, or - stream throtteling >From dealing with providers of the latter, I'm not prepared to swear that trying to implement the latter won't blow up the former (;-)) The on-topic question might be, will attempts to do network capacity management necessarily interfere with congestion control and aqm. I can certainly imagine - a management tool attempting to artificially buffer and delay a stream, adding far more latency than it restricts bandwidth, or - a management tool forging aqm "slow down" signals, or - discarding packets to force retransmission and slowdowns. I wonder what's worst? Best? --dave -------- Original Message -------- Subject: Re: [Internet Policy] FCC's latest action - two questions Date: Sun, 27 Apr 2014 19:40:34 +0100 From: Graham Cobb To: internetpolicy@elists.isoc.org On 27/04/14 09:13, Patrik Fältström wrote: > I do not get it either, and would like to have a clarification (if > someone exists) from people that have followed this closer than I > have (I am trying to follow the European discussions, which fills > my $day). I do not know exactly what the FCC proposals mean, nor do I have any insight into what the US ISPs want to do. Full disclosure: my employer supplies equipment to do policy control (and many other things) to ISPs (including the US ISPs). However, I have no knowledge of their plans and, in any case, these comments are my own views, not those of my employer. However, I do know what some of the industry discussions (in many countries) are about in this area, and what SOME ISPs (and vendors) would *like* to do. > Given these rules, it is already clear an access provider have two > different ways of getting more money. A. negotiate other peering > and transit relationships or B. charge downstream more for the > Internet Access. It is rather more complicated than that. What many ISPs would like to do is to sell all the parties involved in communication many additional services (normally known as "value added services" -- VAS). For example, one service that someone like Netflix might be interested in buying from an ISP is "charge to bill" -- allowing users to charge the video rental to their phone bill. But there are many, many more services ISPs are hoping to sell to OTT (over-the-top) players: marketing/advertising, space on their app store, ordering/customer care, payments/charging, location, device information, QoS, CDN/proxy, and loads more. Many of those VAS have nothing to do with transmitting bits -- but the ISPs want to bundle up many services and offer them as a package to people like Netflix. In addition, ISPs want to offer additional value added services to their users. One of those is bandwidth (another is latency -- particularly aimed at gamers, another is packet loss rates). It is fairly easy to see that in mobile networks, ISPs can offer users a choice of maximum bandwidth (a trivial example is whether the user can use 4G or is limited to 3G even where 4G is available -- in practice there are a lot more options which allow many bandwidth choices to be offered in both 3G and 4G networks). In fixed line networks there is less option to offer different access network speeds (although it does exist) -- it is more normal to offer a choice of congestion options on the backhaul (your access line may do 16Mbps but if you are sharing a 50Mbps backhaul between 50 people, you are unlikely to get more than 1Mbps at busy times). The idea is that ISPs will offer different peak-time bandwidth, at different prices: $10/month for 1Mbps, $20/month for 20Mbps, $35/month for access to the maximum bandwidth available on your connection, say. Alongside those tiers, the ISP will offer a large number of options to upgrade (permanently or temporarily, or for some protocols but not others). So, if you subscribe to 1Mbps speed but need 10Mbps to watch a movie for the next 2 hours you could pay $2 to enable that (say). The next step is to allow the OTT partner (Netflix, say) to pay that add on fee for you. So Netflix can say "watch this video for $5 even if you only subscribe to the basic ISP service" -- and behind the scenes Netflix would pay part of their $5 to the ISP to temporarily configure your connection for a higher bandwidth for access to the Netflix CDN, for the next 2 hours. All this *could* be done without reducing the service level provided to the users who are not paying for the add-ons (by careful provisioning of access network and backhaul capacity). But will it? In my mind, the questions for NN are these two: 1) Will the add-ons and VAS packages be offered equally, on FRAND commercial terms, to all players, including the ISPs own services, and 2) Will the add-ons cause the basic service to become unusable or will a reasonable level of service be maintained even for users who are not paying additional charges (directly or indirectly). > Because of this, to be able to implement "fast lane" connections > must by definition be full. No, it doesn't. Modern policy control systems are very sophisticated. They can control the maximum bandwidth (and other traffic control parameters, including whether access is allowed at all) available by stream, identifying streams by a combination of source/destination IP address, protocol etc, even using DPI to allow identification of particular web sites or even particular pages within protocols such as HTTP. Rules can be created which use that information along with subscriber info and preferences, device type, previous history, quotas, limits, tier, add-ons, location, service being accessed, etc, etc. And third party web services (or apps on the device) can use APIs to request (and pay for) changes to those rules. The sorts of deals being talked about using this equipment would be "buy a new Samsung HD tablet and get 20 days free Netflix upgrade to HD from any location in central Mumbai". Or "buy a new pair of football boots from this store and get free access to all the Manchester United goals on every Saturday afternoon of the season". Or even "disable your access to porn videos for the upcoming religious period and have us send an SMS to your priest to confirm you have done it"! The ideas are limited only to the imaginations of the marketing depts. Some of these are probably good ideas. The question is how to allow them without also allowing undesirable effects. Graham _______________________________________________ To manage your ISOC subscriptions or unsubscribe, please log into the ISOC Member Portal: https://portal.isoc.org/ Then choose Interests & Subscriptions from the My Account menu. From nobody Mon Apr 28 18:24:33 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A60E11A8870 for ; Mon, 28 Apr 2014 18:24:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.6 X-Spam-Level: X-Spam-Status: No, score=-0.6 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yCJ6xQ1FVGeL for ; Mon, 28 Apr 2014 18:24:30 -0700 (PDT) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) by ietfa.amsl.com (Postfix) with ESMTP id A17491A8872 for ; Mon, 28 Apr 2014 18:24:21 -0700 (PDT) Received: by mail-wg0-f50.google.com with SMTP id k14so2733347wgh.9 for ; Mon, 28 Apr 2014 18:24:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=6oa4QZCEmiO/+fgBRy2BcDEUXf9YPrE8Wq5LfWPn2PE=; b=UTVopokYeFO7x4bXhmrOgqT8U2CSWxMkHgNfNAzHzt2pOWtyPkVyOFkfsN/82fQo9E bSe1sQ5ylaw7KZjHxDIZjJqQBAV99ymnObTtFj+1u+53QNf04SfhBcP8C1FsqGyDfXTv y2vgxJYm+RYgtAuF4Fg9a6rDqjn/e+YW+HNu05+dxpHxOcc5HcujaUB6RSe4JN15wyjR nGj0qeSMutmuO5UWRv9CrB/xXcALSoJRYDzTAQoNxQNIMJAUQTuZDmlDOWBJ/KICXs8a fzGMNqW0S4vscjAxMYokxBfxpaKbg9h5DayiEn2gY/O1CkN+wl95gBHrsIkdfThLrndD gfqQ== MIME-Version: 1.0 X-Received: by 10.180.198.46 with SMTP id iz14mr8766194wic.46.1398734660229; Mon, 28 Apr 2014 18:24:20 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Mon, 28 Apr 2014 18:24:20 -0700 (PDT) Date: Mon, 28 Apr 2014 18:24:20 -0700 Message-ID: From: Dave Taht To: bloat , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/TAUOgDBdtVbpp9e1X4quYU9g9PY Subject: [aqm] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 01:24:32 -0000 pretty wonderful experiment and video http://livingwithlag.com/ --=20 Dave T=C3=A4ht From nobody Tue Apr 29 00:08:57 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 738701A884F for ; Tue, 29 Apr 2014 00:08:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z69hpSjZuf8B for ; Tue, 29 Apr 2014 00:08:52 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id 20DF51A884E for ; Tue, 29 Apr 2014 00:08:51 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id DCD01A2; Tue, 29 Apr 2014 09:08:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1398755329; bh=lLRVbBUCU3viiH2ULott6d5LQEgmMaFgroFiVBjkd7M=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=sjy0p6zjhQE0hC+ckEBxhytJ8KzT4/jY60EeDiXM9JflF22qKwf1USyGIT9tYg01l fpjRWuTm/ni9dAkYkFb2iOWlKuFq+deozNbMLJZqYOEqpZfMnPQ4brLIYyJJnE2v9A tCA5IMxwyr92h2YN9vdHa6SyrvDNgwYA97FTwYgA= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id D4398A1; Tue, 29 Apr 2014 09:08:49 +0200 (CEST) Date: Tue, 29 Apr 2014 09:08:49 +0200 (CEST) From: Mikael Abrahamsson To: Dave Taht In-Reply-To: Message-ID: References: User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/07hvwjFomgSdzkGWeLAfl5A-OF0 Cc: "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 07:08:55 -0000 On Mon, 28 Apr 2014, Dave Taht wrote: > pretty wonderful experiment and video http://livingwithlag.com/ Just so that everybody realises that this is an advertisement. Also, what access method has 300 ms access latency, let alone 3 seconds? None that I know of, the meaningful comparison would be ADSL2+ at around 25ms and 3G at around 50-100ms. -- Mikael Abrahamsson email: swmike@swm.pp.se From nobody Tue Apr 29 00:21:37 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 761921A074E for ; Tue, 29 Apr 2014 00:21:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -115.152 X-Spam-Level: X-Spam-Status: No, score=-115.152 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RdqHw87T8kIh for ; Tue, 29 Apr 2014 00:21:34 -0700 (PDT) Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) by ietfa.amsl.com (Postfix) with ESMTP id DF12F1A0746 for ; Tue, 29 Apr 2014 00:21:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2272; q=dns/txt; s=iport; t=1398756093; x=1399965693; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=7SiAE4a4iBrxHhimWIUl7iyIMwtct23xHtU5GQDWK1E=; b=GDWDc6eyw0rW+/QTIWpU82Uf7q6tL3lqXCv8WkutexqgCszxatvQpJDe fqMyDKwdTLdnk78jLHTG0oRChz1vUZ2KTqoxJVMi/+3D+pEAbtUw2WNSX BbLyfKq24EDLYAenyCLSJwCCiRIw43zLOsD0qsOy2kMpeS/ukEFLKr86r 4=; X-Files: signature.asc : 195 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApgFAK9SX1OtJA2E/2dsb2JhbABZgwZPV709hzmBHBZ0giUBAQEDAQEBAWsLBQsCAQgOCi4nCyUCBA4FDogrCA3JFBeJL4Q+EQFQBwmDG4EVBIU7i0eBOIZTgTyRJYMxgXI5 X-IronPort-AV: E=Sophos;i="4.97,949,1389744000"; d="asc'?scan'208";a="320905872" Received: from alln-core-10.cisco.com ([173.36.13.132]) by rcdn-iport-1.cisco.com with ESMTP; 29 Apr 2014 07:21:32 +0000 Received: from xhc-aln-x02.cisco.com (xhc-aln-x02.cisco.com [173.36.12.76]) by alln-core-10.cisco.com (8.14.5/8.14.5) with ESMTP id s3T7LWJD010325 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 29 Apr 2014 07:21:32 GMT Received: from xmb-rcd-x09.cisco.com ([169.254.9.100]) by xhc-aln-x02.cisco.com ([173.36.12.76]) with mapi id 14.03.0123.003; Tue, 29 Apr 2014 02:21:32 -0500 From: "Fred Baker (fred)" To: Mikael Abrahamsson Thread-Topic: [aqm] the side effects of 330ms lag in the real world Thread-Index: AQHPY3ulkMTE4Rpp6E+HqfC8bSk3Tw== Date: Tue, 29 Apr 2014 07:21:31 +0000 Message-ID: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.21.85.48] Content-Type: multipart/signed; boundary="Apple-Mail=_E58775AF-F777-4FB6-9447-70E863C1440E"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/2j0ODA9blzGBuSmd4b9EOMLdycI Cc: bloat , Dave Taht , "cerowrt-devel@lists.bufferbloat.net" , "aqm@ietf.org" Subject: Re: [aqm] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 07:21:35 -0000 --Apple-Mail=_E58775AF-F777-4FB6-9447-70E863C1440E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 29, 2014, at 3:08 AM, Mikael Abrahamsson = wrote: > On Mon, 28 Apr 2014, Dave Taht wrote: >=20 >> pretty wonderful experiment and video http://livingwithlag.com/ >=20 > Just so that everybody realises that this is an advertisement. >=20 > Also, what access method has 300 ms access latency, let alone 3 = seconds? None that I know of, the meaningful comparison would be ADSL2+ = at around 25ms and 3G at around 50-100ms. Well, we could discuss international communications. I happen to be at = Infocom in Toronto, VPN=92d into Cisco San Jose, and did a ping to you: ping -c 10 swm.pp.se PING swm.pp.se (212.247.200.143): 56 data bytes ... --- swm.pp.se ping statistics --- 10 packets transmitted, 10 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 249.368/342.038/456.376/71.902 ms 3 seconds is unusually high, although naval satcom is frequently in the = 1.5-2.5 ms range. But I have a sample that I measured in a Dublin hotel = that observed standing latencies of around 280 ms to San Jose, frequent = latencies of seven seconds, and a peak of 9286 ms. Yes, the hotel DSL = was horrendously misconfigured. It makes a great graphic for = presentations. > --=20 > Mikael Abrahamsson email: swmike@swm.pp.se >=20 > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm ---------------------------------------------------- The ignorance of how to use new knowledge stockpiles exponentially.=20 - Marshall McLuhan --Apple-Mail=_E58775AF-F777-4FB6-9447-70E863C1440E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iD8DBQFTX1L7bjEdbHIsm0MRAjUZAJ4nbQAVUezS4Ja6bwczez+VvfNBlgCg4Bis xZwDo3YmgEt0PjUcC1cJcCs= =2UXv -----END PGP SIGNATURE----- --Apple-Mail=_E58775AF-F777-4FB6-9447-70E863C1440E-- From nobody Tue Apr 29 00:56:17 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DBF61A0746 for ; Tue, 29 Apr 2014 00:56:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fdKRZphD0c6j for ; Tue, 29 Apr 2014 00:56:15 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id 4EAB11A888F for ; Tue, 29 Apr 2014 00:56:12 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id E369AA5; Tue, 29 Apr 2014 09:56:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1398758170; bh=XZYM49qj3oRRho+RKjAUR5klzpmKCnGWUzsckVcMo5M=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=vEr6VUkasXIWkDU5ZelbLn3nEmYoGy8NEgwRUUTTJoS2UCDhTiSs1feORlwxXauLY cHrNWtUy4I6/gh6G1lwrvHoH0lM0QMDANKKZv+LTGCpflpczOcIiHKSLPpXiYm37IM zlpJeCdPrgRWxSNyYmZ02rAOLhT1vvIimC3oLvu4= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id DB60BA3; Tue, 29 Apr 2014 09:56:10 +0200 (CEST) Date: Tue, 29 Apr 2014 09:56:10 +0200 (CEST) From: Mikael Abrahamsson To: "Fred Baker (fred)" In-Reply-To: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> Message-ID: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-137064504-577767-1398758170=:29282" Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/UqBhSECJZLEPJADsQOUiacSZ360 Cc: bloat , Dave Taht , "cerowrt-devel@lists.bufferbloat.net" , "aqm@ietf.org" Subject: Re: [aqm] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 07:56:16 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---137064504-577767-1398758170=:29282 Content-Type: TEXT/PLAIN; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8BIT On Tue, 29 Apr 2014, Fred Baker (fred) wrote: > Well, we could discuss international communications. I happen to be at > Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you: Yes, but as soon as you hit the long distance network the latency is the same regardless of access method. So while I agree that understanding the effect of latency is important, it's no longer a meaningful way of selling fiber access. If your last-mile is fiber instead of ADSL2+ won't improve your long distance latency. -- Mikael Abrahamsson email: swmike@swm.pp.se ---137064504-577767-1398758170=:29282-- From nobody Tue Apr 29 03:09:19 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 236961A0735 for ; Mon, 28 Apr 2014 00:12:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.651 X-Spam-Level: X-Spam-Status: No, score=-1.651 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001, NML_ADSP_CUSTOM_MED=0.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tWecM7bHBA1F for ; Mon, 28 Apr 2014 00:12:49 -0700 (PDT) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by ietfa.amsl.com (Postfix) with ESMTP id E53B31A0447 for ; Mon, 28 Apr 2014 00:12:48 -0700 (PDT) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Wefk7-0007I1-IM for aqm@ietf.org; Mon, 28 Apr 2014 09:12:47 +0200 Received: from 66-87-113-231.pools.spcsdns.net ([66.87.113.231]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 28 Apr 2014 09:12:47 +0200 Received: from eternaleye by 66-87-113-231.pools.spcsdns.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 28 Apr 2014 09:12:47 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: aqm@ietf.org From: Alex Elsayed Date: Mon, 28 Apr 2014 00:12:35 -0700 Lines: 174 Message-ID: References: <535D4F22.6060602@cobb.uk.net> <535D5548.1000208@rogers.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8Bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 66-87-113-231.pools.spcsdns.net User-Agent: KNode/4.12 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/Mj_pd16B5a9bI-F0vbcxfCqTmWA X-Mailman-Approved-At: Tue, 29 Apr 2014 03:09:17 -0700 Cc: bloat@lists.bufferbloat.net Subject: Re: [aqm] Slightly off-topic: [Internet Policy] FCC's latest action - two questions X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 07:12:51 -0000 Well, I doubt any one option will be an absolute worst/best - simply because there are multiple metrics at play here, which likely conflict. - How much overhead is added (retransmission, extra computation)? - How easy/likely is the mechanism to be bypassed? - If it is bypassed, how much damage does that do elsewhere? - How likely is it to break other protocols? For instance, interposing with some sort of TCP-level proxy that does CC differently on each side (this is already done in some cases for e.g. high- latency links) adds little network overhead but potentially significant computational burden, is bypassed by using UDP (which may do significant damage if poorly implemented, potentially causing Yet Another Congestion Collapse), and breaks the end-to-end property of TCP flow control. Throttling by packet-drop adds retransmission overhead, but very little computational. It's also only possible to bypass if you manage to evade whatever the classification scheme is, and is unlikely to break any protocol since it's an expected failure mode of the internet. Throttling by ECN saves the retransmission overhead, but makes it easier to bypass (ignore the ECN bit), and might cause major long-term damage by providing an incentive to ignore ECN entirely. Again, however, it's unlikely to break other protocols _directly_. Overall, it starts looking like the solution is the same as the basic case of AQM, in the rough: 1.) If something is too fast, and ECN is supported, ECN 2.) If no ECN, drop 3.) If ECN but stream does not respond, drop (perhaps with penalty) You then do some sort of weighted FQ, and use some sort of ruleset to control the weights and streams of each queue. David Collier-Brown wrote: > While this discussion is about policy, the implementation is either > - TCP flow/congestion control, or > - stream throtteling > >>From dealing with providers of the latter, I'm not prepared to swear > that trying to implement the latter won't blow up the former (;-)) > > The on-topic question might be, will attempts to do network capacity > management necessarily interfere with congestion control and aqm. > I can certainly imagine > - a management tool attempting to artificially buffer and delay a > stream, adding far more latency than it restricts bandwidth, or > - a management tool forging aqm "slow down" signals, or > - discarding packets to force retransmission and slowdowns. > > I wonder what's worst? Best? > > --dave > > -------- Original Message -------- > Subject: Re: [Internet Policy] FCC's latest action - two questions > Date: Sun, 27 Apr 2014 19:40:34 +0100 > From: Graham Cobb > To: internetpolicy@elists.isoc.org > > On 27/04/14 09:13, Patrik Fältström wrote: >> I do not get it either, and would like to have a clarification (if >> someone exists) from people that have followed this closer than I >> have (I am trying to follow the European discussions, which fills >> my $day). > > I do not know exactly what the FCC proposals mean, nor do I have any > insight into what the US ISPs want to do. Full disclosure: my > employer supplies equipment to do policy control (and many other > things) to ISPs (including the US ISPs). However, I have no knowledge > of their plans and, in any case, these comments are my own views, not > those of my employer. > > However, I do know what some of the industry discussions (in many > countries) are about in this area, and what SOME ISPs (and vendors) > would *like* to do. > >> Given these rules, it is already clear an access provider have two >> different ways of getting more money. A. negotiate other peering >> and transit relationships or B. charge downstream more for the >> Internet Access. > > It is rather more complicated than that. What many ISPs would like to > do is to sell all the parties involved in communication many > additional services (normally known as "value added services" -- VAS). > For example, one service that someone like Netflix might be > interested in buying from an ISP is "charge to bill" -- allowing users > to charge the video rental to their phone bill. But there are many, > many more services ISPs are hoping to sell to OTT (over-the-top) > players: marketing/advertising, space on their app store, > ordering/customer care, payments/charging, location, device > information, QoS, CDN/proxy, and loads more. > > Many of those VAS have nothing to do with transmitting bits -- but the > ISPs want to bundle up many services and offer them as a package to > people like Netflix. > > In addition, ISPs want to offer additional value added services to > their users. One of those is bandwidth (another is latency -- > particularly aimed at gamers, another is packet loss rates). It is > fairly easy to see that in mobile networks, ISPs can offer users a > choice of maximum bandwidth (a trivial example is whether the user can > use 4G or is limited to 3G even where 4G is available -- in practice > there are a lot more options which allow many bandwidth choices to be > offered in both 3G and 4G networks). In fixed line networks there is > less option to offer different access network speeds (although it does > exist) -- it is more normal to offer a choice of congestion options on > the backhaul (your access line may do 16Mbps but if you are sharing a > 50Mbps backhaul between 50 people, you are unlikely to get more than > 1Mbps at busy times). > > The idea is that ISPs will offer different peak-time bandwidth, at > different prices: $10/month for 1Mbps, $20/month for 20Mbps, $35/month > for access to the maximum bandwidth available on your connection, say. > Alongside those tiers, the ISP will offer a large number of options > to upgrade (permanently or temporarily, or for some protocols but not > others). So, if you subscribe to 1Mbps speed but need 10Mbps to watch > a movie for the next 2 hours you could pay $2 to enable that (say). > > The next step is to allow the OTT partner (Netflix, say) to pay that > add on fee for you. So Netflix can say "watch this video for $5 even > if you only subscribe to the basic ISP service" -- and behind the > scenes Netflix would pay part of their $5 to the ISP to temporarily > configure your connection for a higher bandwidth for access to the > Netflix CDN, for the next 2 hours. > > All this *could* be done without reducing the service level provided > to the users who are not paying for the add-ons (by careful > provisioning of access network and backhaul capacity). But will it? In > my mind, the questions for NN are these two: > > 1) Will the add-ons and VAS packages be offered equally, on FRAND > commercial terms, to all players, including the ISPs own services, and > > 2) Will the add-ons cause the basic service to become unusable or will > a reasonable level of service be maintained even for users who are not > paying additional charges (directly or indirectly). > >> Because of this, to be able to implement "fast lane" connections >> must by definition be full. > > No, it doesn't. Modern policy control systems are very sophisticated. > They can control the maximum bandwidth (and other traffic control > parameters, including whether access is allowed at all) available by > stream, identifying streams by a combination of source/destination IP > address, protocol etc, even using DPI to allow identification of > particular web sites or even particular pages within protocols such as > HTTP. Rules can be created which use that information along with > subscriber info and preferences, device type, previous history, > quotas, limits, tier, add-ons, location, service being accessed, etc, > etc. And third party web services (or apps on the device) can use > APIs to request (and pay for) changes to those rules. > > The sorts of deals being talked about using this equipment would be > "buy a new Samsung HD tablet and get 20 days free Netflix upgrade to > HD from any location in central Mumbai". Or "buy a new pair of > football boots from this store and get free access to all the > Manchester United goals on every Saturday afternoon of the season". > Or even "disable your access to porn videos for the upcoming religious > period and have us send an SMS to your priest to confirm you have done > it"! The ideas are limited only to the imaginations of the marketing > depts. > > Some of these are probably good ideas. The question is how to allow > them without also allowing undesirable effects. > > Graham > _______________________________________________ > To manage your ISOC subscriptions or unsubscribe, > please log into the ISOC Member Portal: > https://portal.isoc.org/ > Then choose Interests & Subscriptions from the My Account menu. From mithrandi@mithrandi.net Tue Apr 29 00:43:42 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83C9E1A0453 for ; Tue, 29 Apr 2014 00:43:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.378 X-Spam-Level: X-Spam-Status: No, score=-1.378 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x-1ZQRwtRp43 for ; Tue, 29 Apr 2014 00:43:41 -0700 (PDT) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) by ietfa.amsl.com (Postfix) with ESMTP id 12E2E1A884E for ; Tue, 29 Apr 2014 00:43:40 -0700 (PDT) Received: by mail-la0-f44.google.com with SMTP id hr17so323777lab.17 for ; Tue, 29 Apr 2014 00:43:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mithrandi.net; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JA9SI1CS1k9hpYUMEG2Whz6yd8kRRDTP7eYFsY7oyTE=; b=cITQH3hPA4oELxvRqhFeMaDdEfnq2mFrshheUrbXhUhlYI08seY2TDmiQRpFr9exTr e8hCtfTxKT78TO2wVRT/su5OckPHDM1HBRSV+sWippWDWAfF+1asgS8Am5/eOmaAfmbL 3q5fpentuvEkg9/tQaB2bO1/0J8V7rYDxteCg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=JA9SI1CS1k9hpYUMEG2Whz6yd8kRRDTP7eYFsY7oyTE=; b=hd9srd/VwHYv8Pm86ngn1haHx26DAiOX28vUz31Ttxrw1vPJMM8lel3V/iN0UV0XL/ tDQED5Ue/cbSAFkI+/B68U8OjbBxxnGAuobN6Audyf56Dr20pwoHqTHveccp/GnxJ7XO 5+JaQ/YU6WcCRAOryQlWJoFbThkbDdV2hpb5+h7cF/U8A4yUZ4vdx0UsnfFZH0qrrFh1 LN5P7KTJcQuAdRhgPyi+Nswjz1D6yJM8Y9yMT89WszarRNelGOqjDXwpqtmkyzRIvVSq JBC1zGjwlzVe0Z39wpk6WN2zPPm64WpvmKpwF4qiefGI/5zhvbOFoWLXJiRhNkPIRrko ldLg== X-Gm-Message-State: ALoCoQle17XRvL7vdlc91cM++WmhDLoKqfjeuRGlBFwkSD+qCUj8jc+QrE4EFkXu0KjNygwHZJ0u MIME-Version: 1.0 X-Received: by 10.112.52.104 with SMTP id s8mr21366779lbo.7.1398757419130; Tue, 29 Apr 2014 00:43:39 -0700 (PDT) Received: by 10.114.157.10 with HTTP; Tue, 29 Apr 2014 00:43:39 -0700 (PDT) In-Reply-To: References: Date: Tue, 29 Apr 2014 09:43:39 +0200 Message-ID: From: Tristan Seligmann To: Mikael Abrahamsson Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/xGDuotvPppGylIM92vn2HC70MaE X-Mailman-Approved-At: Tue, 29 Apr 2014 03:10:41 -0700 Cc: bloat , Dave Taht , "cerowrt-devel@lists.bufferbloat.net" , "aqm@ietf.org" Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 07:45:15 -0000 On 29 April 2014 09:08, Mikael Abrahamsson wrote: > Also, what access method has 300 ms access latency, let alone 3 seconds? > None that I know of, the meaningful comparison would be ADSL2+ at around > 25ms and 3G at around 50-100ms. Latencies on my ADSL2+ connection in Johannesburg, South Africa to my ISP, a host in the UK, and a host in the USA: --- www.mweb.co.za ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9011ms rtt min/avg/max/mdev = 33.720/38.594/66.841/9.507 ms --- anor.aduial.net ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9010ms rtt min/avg/max/mdev = 197.966/201.981/211.580/4.619 ms --- speedtest.atlanta.linode.com ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9001ms rtt min/avg/max/mdev = 285.859/292.451/331.214/13.285 ms And of course, I have bufferbloat mitigation in play. -- mithrandi, i Ainil en-Balandor, a faer Ambar From nobody Tue Apr 29 03:11:15 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 084311A6FEE for ; Mon, 28 Apr 2014 14:00:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.379 X-Spam-Level: X-Spam-Status: No, score=-1.379 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_NUMERIC_HELO=1.164, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_FSL_HELO_BARE_IP_2=0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yZ-JUXE3guvs for ; Mon, 28 Apr 2014 14:00:05 -0700 (PDT) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by ietfa.amsl.com (Postfix) with ESMTP id 5148F1A6FBB for ; Mon, 28 Apr 2014 14:00:05 -0700 (PDT) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Weseh-0005u8-Qt for aqm@ietf.org; Mon, 28 Apr 2014 23:00:03 +0200 Received: from 32.97.110.57 ([32.97.110.57]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 28 Apr 2014 23:00:03 +0200 Received: from wmf by 32.97.110.57 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 28 Apr 2014 23:00:03 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: aqm@ietf.org From: Wes Felter Date: Mon, 28 Apr 2014 15:57:15 -0500 Lines: 7 Message-ID: References: <535D4F22.6060602@cobb.uk.net> <535D5548.1000208@rogers.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 32.97.110.57 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 In-Reply-To: <535D5548.1000208@rogers.com> Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/rVnDXdGG8mCHfSDxa6n2SlVX1x8 X-Mailman-Approved-At: Tue, 29 Apr 2014 03:11:10 -0700 Cc: bloat@lists.bufferbloat.net Subject: Re: [aqm] Slightly off-topic: [Internet Policy] FCC's latest action - two questions X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 21:00:07 -0000 I would expect ISPs to disable (or never enable) AQM on the "slow lane" to maximize the difference between the two services. After all, people won't pay for the fast lane if it's the same speed as the slow lane. -- Wes Felter From nobody Tue Apr 29 08:46:52 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F8E01A0776 for ; Tue, 29 Apr 2014 08:46:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HlOeTgXNjQ_D for ; Tue, 29 Apr 2014 08:46:49 -0700 (PDT) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) by ietfa.amsl.com (Postfix) with ESMTP id CA2971A08F0 for ; Tue, 29 Apr 2014 08:46:47 -0700 (PDT) Received: by mail-wi0-f177.google.com with SMTP id cc10so689023wib.16 for ; Tue, 29 Apr 2014 08:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=BONCxR/spyWM5/uTx0EhoJGPlOJRdmeTFON8M+mFjII=; b=TEpEIzbgwOAH6TH1AmKIES1tRPl/yahNhDSGzIQKyx9KBi7ZX7L6+PDamNfY0Y0BRU +eogcHv6DzADk5v77EalO50Rd3H/bRex0DqklS120PgBvra/KoB1z3CIRiW2ooxGADDf K8/9vtOe6bTPt2pjCPGRTB+YgVa0a31JcaEqNyI34uMqxY6XG0nYehoBX/+36SYCAv0q 0qJBq5v+bLCRgbk/M/mR5+lXU/BezUrs6MRiunSlyklurXvHVJDDNdoBGCT4VNVB91f2 n6joQcCJx0MUaeZrwttHx30zERfc/yLQtKGBozczX1PMPDgpKIDQU8StUTdF2xKKQMIC UqXg== MIME-Version: 1.0 X-Received: by 10.194.6.106 with SMTP id z10mr79312wjz.1.1398786406224; Tue, 29 Apr 2014 08:46:46 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Tue, 29 Apr 2014 08:46:46 -0700 (PDT) In-Reply-To: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> Date: Tue, 29 Apr 2014 08:46:46 -0700 Message-ID: From: Dave Taht To: Mikael Abrahamsson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/pJA7HWPg7xFwE9lxRNjfQ8__M3c Cc: bloat , "Fred Baker \(fred\)" , "cerowrt-devel@lists.bufferbloat.net" , "aqm@ietf.org" Subject: Re: [aqm] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 15:46:50 -0000 On Tue, Apr 29, 2014 at 12:56 AM, Mikael Abrahamsson wro= te: > On Tue, 29 Apr 2014, Fred Baker (fred) wrote: A couple points here. 1) The video went viral, and garnered over 600,000 new hits in the 12 hours since I posted it here. there is pent up demand for less latency. While the ad conflates bandwidth with latency, they could have published their RTTs on their local fiber network, which is probably a great deal less than dsl or cable. That counts for a lot when accessing local services. 2) There is a lot of things an ISP can do to improve apparent latency on the long haul A) co-locating with a major dns server like f-root to reduce dns latency B) co-locating with major services like google and netflix publishing ping times to google for example might be a good tactic. C) Better peering >> Well, we could discuss international communications. I happen to be at >> Infocom in Toronto, VPN=E2=80=99d into Cisco San Jose, and did a ping to= you: > > > Yes, but as soon as you hit the long distance network the latency is the > same regardless of access method. So while I agree that understanding the > effect of latency is important, it's no longer a meaningful way of sellin= g > fiber access. If your last-mile is fiber instead of ADSL2+ won't improve > your long distance latency. Well, it chops a great deal from the baseline physical latency, and most people tend to access resources closer to them than farther away. An american in paris might want to access the NYT, but Parisians La Monde. Similarly most major websites are replicated and use CDNs to distribute their data closer to the user. The physical RTT matters more and more in the last mile the more resources are co-located in the local data center= . --=20 Dave T=C3=A4ht From nobody Tue Apr 29 09:44:45 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C9F5B1A066A for ; Tue, 29 Apr 2014 09:44:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.277 X-Spam-Level: X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PociWQ4boOtF for ; Tue, 29 Apr 2014 09:44:40 -0700 (PDT) Received: from mail-oa0-x229.google.com (mail-oa0-x229.google.com [IPv6:2607:f8b0:4003:c02::229]) by ietfa.amsl.com (Postfix) with ESMTP id 14ED91A01DA for ; Tue, 29 Apr 2014 09:44:40 -0700 (PDT) Received: by mail-oa0-f41.google.com with SMTP id j17so542438oag.14 for ; Tue, 29 Apr 2014 09:44:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=1nEzBnZ/G3HYJg36TzSf0h8BzI7UpSHnXYg4zDQGJuk=; b=Rw1fdhIVCVQaG5S3z5A4dKAajzYwnNr4iU2jUHu2dLscOsMwk5AmknsZ1HvDCOIrxo St15Dm8/m6U4snbsNOgkRl2KLE1QCcVqtcaL4RbA7ycVtLrae6ZiIpE678yq/4ai3Vh/ mzrGfQPXRRqkC5pjhQDGRQgp+GJSs7z4tAW9z95khM5A5BQ2WZERmJwOQv5LSy9m8/x5 JMR7AmWQjRlNcunIRG/nKlXAm5H7FpJJ+01buh1s8KSyKNsR8DKixJ0+UnkfGfqueSL0 56ECBCPxTGZWCuaKzPimZkq4k6RsXDif+X+atyZ9Fo9THarL7AZrtRKEO7gNd7ln17bY Q4ww== MIME-Version: 1.0 X-Received: by 10.60.125.195 with SMTP id ms3mr421153oeb.3.1398789878674; Tue, 29 Apr 2014 09:44:38 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.76.73.100 with HTTP; Tue, 29 Apr 2014 09:44:38 -0700 (PDT) In-Reply-To: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> Date: Tue, 29 Apr 2014 12:44:38 -0400 X-Google-Sender-Auth: JDTl2IEp_p49TBfekRvNUJzevAQ Message-ID: From: Jim Gettys To: Mikael Abrahamsson Content-Type: multipart/alternative; boundary=047d7b33cf28d57c0404f83125b7 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/HxKbRbdu6V17UNxPt9AO7wC1NW4 Cc: "aqm@ietf.org" , "Fred Baker \(fred\)" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 16:44:43 -0000 --047d7b33cf28d57c0404f83125b7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson wrote= : > On Tue, 29 Apr 2014, Fred Baker (fred) wrote: > > Well, we could discuss international communications. I happen to be at >> Infocom in Toronto, VPN=E2=80=99d into Cisco San Jose, and did a ping to= you: >> > > Yes, but as soon as you hit the long distance network the latency is the > same regardless of access method. So while I agree that understanding the > effect of latency is important, it's no longer a meaningful way of sellin= g > fiber access. If your last-mile is fiber instead of ADSL2+ won't improve > your long distance latency. =E2=80=8BFIOS bufferbloat is a problem too. Measured bufferbloat, symmetric 25/25 service in New Jersey at my inlaw's house is 200ms (on the ethernet port of the Actiontec router provided by Verizon). So latency under load is the usual problem. Why would you think the GPON guys are any better in principle than cable or DSL? Cable and DSL may be somewhat worse, just because it is older and downward compatibility means that new modems on low bandwidth tiers are even more grossly over buffered. You can look at the netalyzr scatter plots in http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-th= row-stones-at-another/ Now, if someone gives me real fiber to the home, with a real switch fabric upstream, rather than gpon life might be somewhat better (if the switches aren't themselves overbuffered.... But so far, it isn't. - Jim - Jim - Jim =E2=80=8B > > > -- > Mikael Abrahamsson email: swmike@swm.pp.se > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > > --047d7b33cf28d57c0404f83125b7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Tue= , Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On Tue, 29 Apr 2014, Fred Baker (fred) wro= te:

Well, we could discuss international communications. I happen to be at Info= com in Toronto, VPN=E2=80=99d into Cisco San Jose, and did a ping to you:

Yes, but as soon as you hit the long distance network the latency is the sa= me regardless of access method. So while I agree that understanding the eff= ect of latency is important, it's no longer a meaningful way of selling= fiber access. If your last-mile is fiber instead of ADSL2+ won't impro= ve your long distance latency.

=E2= =80=8BFIOS bufferbloat is a problem too.

Measured bufferbloat, symmetric 25/25 service in New Jersey= at my inlaw's house is 200ms (on the ethernet port of the Actiontec ro= uter provided by Verizon). =C2=A0So latency under load is the usual problem= .

Why would you think the GPON g= uys are any better in principle than cable or DSL? =C2=A0Cable and DSL may = be somewhat worse, just because it is older and downward compatibility mean= s that new modems on low bandwidth tiers are even more grossly over buffere= d.
You can look at the = netalyzr scatter plots in=C2=A0http://ge= ttys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stone= s-at-another/

Now, if someone gives me real = fiber to the home, with a real switch fabric upstream, rather than gpon lif= e might be somewhat better (if the switches aren't themselves overbuffe= red.... =C2=A0But so far, it isn't.
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Jim

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Jim

=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0- Jim

=E2=80=8B<= /div>


--
Mikael Abrahamsson =C2=A0 =C2=A0email: swmike@swm.pp.se

_____________________= __________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
= https://lists.bufferbloat.net/listinfo/bloat


--047d7b33cf28d57c0404f83125b7-- From nobody Tue Apr 29 09:57:35 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 67A331A08F0 for ; Tue, 29 Apr 2014 09:57:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.277 X-Spam-Level: X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ecYHs_q_CCar for ; Tue, 29 Apr 2014 09:57:33 -0700 (PDT) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) by ietfa.amsl.com (Postfix) with ESMTP id D48021A07D9 for ; Tue, 29 Apr 2014 09:57:32 -0700 (PDT) Received: by mail-ob0-f170.google.com with SMTP id vb8so565241obc.15 for ; Tue, 29 Apr 2014 09:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=wdzfQaFg8FjLD7FJjqn0TmoQ/ScmW5GPi90O3qWN9G4=; b=VpP4D3kzIb36OHBdRAn64XS3z9AYzC3QU03CLW08kFRQrs1DLkiZUxXLvQMaAjvRnJ coqbtU9pzoK4BWFvO06ql6DvrUI2KrnZxbgmsa2+rfn8BRv7G9E8hKKhzlfL37FkjvE3 7wHovZpuBIQB/JhZpv9nCUVdLLAGYOZxhwPZfQcKJ1SNGfFHKYlHFPXRfVvVGHhdZn1p Jgz/LWp2CeDYl5coPHlI9wtNpeCahr998rPM/QSlYzXr05lMoNl7jTZQyGfeX670d1Hk IPTQXfkx2bitg+MciIs0BJe9JYP4cDFSF3qIKsgq88usjCs6+Dy9oFSUiYbuSztzmgEp iCuA== MIME-Version: 1.0 X-Received: by 10.182.27.133 with SMTP id t5mr2984095obg.65.1398790651583; Tue, 29 Apr 2014 09:57:31 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.76.73.100 with HTTP; Tue, 29 Apr 2014 09:57:31 -0700 (PDT) In-Reply-To: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> Date: Tue, 29 Apr 2014 12:57:31 -0400 X-Google-Sender-Auth: GeVG0icCwQpqzrRjNIaq90cCa1E Message-ID: From: Jim Gettys To: Mikael Abrahamsson Content-Type: multipart/alternative; boundary=001a113356c6e7e37404f8315305 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/lCk5TDem1_KzYVnVyhYkIYPtwwU Cc: "aqm@ietf.org" , "Fred Baker \(fred\)" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 16:57:34 -0000 --001a113356c6e7e37404f8315305 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Apr 29, 2014 at 12:44 PM, Jim Gettys wrote: > > > > On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson wro= te: > >> On Tue, 29 Apr 2014, Fred Baker (fred) wrote: >> >> Well, we could discuss international communications. I happen to be at >>> Infocom in Toronto, VPN=E2=80=99d into Cisco San Jose, and did a ping t= o you: >>> >> >> Yes, but as soon as you hit the long distance network the latency is the >> same regardless of access method. So while I agree that understanding th= e >> effect of latency is important, it's no longer a meaningful way of selli= ng >> fiber access. If your last-mile is fiber instead of ADSL2+ won't improve >> your long distance latency. > > > =E2=80=8BFIOS bufferbloat is a problem too. > > Measured bufferbloat, symmetric 25/25 service in New Jersey at my inlaw's > house is 200ms (on the ethernet port of the Actiontec router provided by > Verizon). So latency under load is the usual problem. > > Why would you think the GPON guys are any better in principle than cable > or DSL? Cable and DSL may be somewhat worse, just because it is older an= d > downward compatibility means that new modems on low bandwidth tiers are > even more grossly over buffered. > You can look at the netalyzr scatter plots in > http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-= throw-stones-at-another/ > > =E2=80=8BOh, I forgot to note: Netalyzr topped out at about 20M=E2=80=8Bbps= : most of why GPON looks good on those plots is just that the common lowest tier of service (at the time that data was taken) it can't easily detect the bloat. But if you perform other tests that don't have a bandwidth limit, the bloat is there. This reminds me I should ask Nick Weaver if he has more recent data... - Jim Now, if someone gives me real fiber to the home, with a real switch fabric > upstream, rather than gpon life might be somewhat better (if the switches > aren't themselves overbuffered.... But so far, it isn't. > - Jim > > > >> >> >> -- >> Mikael Abrahamsson email: swmike@swm.pp.se >> >> _______________________________________________ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat >> >> > --001a113356c6e7e37404f8315305 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Tue= , Apr 29, 2014 at 12:44 PM, Jim Gettys <jg@freedesktop.org>= wrote:



On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson <<= a href=3D"mailto:swmike@swm.pp.se" target=3D"_blank">swmike@swm.pp.se&g= t; wrote:
On Tue, 29 Apr 2014, Fred Baker (fred) wrote:

Well, we could discuss international communications. I happen to be at Info= com in Toronto, VPN=E2=80=99d into Cisco San Jose, and did a ping to you:

Yes, but as soon as you hit the long distance network the latency is the sa= me regardless of access method. So while I agree that understanding the eff= ect of latency is important, it's no longer a meaningful way of selling= fiber access. If your last-mile is fiber instead of ADSL2+ won't impro= ve your long distance latency.

=E2=80=8BFIOS buf= ferbloat is a problem too.

Measured bufferbloat, symmetric 25/25 service = in New Jersey at my inlaw's house is 200ms (on the ethernet port of the= Actiontec router provided by Verizon). =C2=A0So latency under load is the = usual problem.

Why= would you think the GPON guys are any better in principle than cable or DS= L? =C2=A0Cable and DSL may be somewhat worse, just because it is older and = downward compatibility means that new modems on low bandwidth tiers are eve= n more grossly over buffered.


=E2=80= =8BOh, I forgot to note: Netalyzr topped out at about 20M=E2=80=8Bbps: most= of why
GPON looks good on those plots is just that the common lowest tier of servi= ce (at the time that data was taken) it can't easily detect the bloat. = But if you perform other tests that don't have a bandwidth limit, the b= loat is there.

This reminds me I should ask N= ick Weaver if he has more recent data...
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0- Jim

Now, if someone gives me rea= l fiber to the home, with a real switch fabric upstream, rather than gpon l= ife might be somewhat better (if the switches aren't themselves overbuf= fered.... =C2=A0But so far, it isn't.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 - Jim

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0


--
Mikael Abrahamsson =C2=A0 =C2=A0email: swmike@swm.pp.se

_______________= ________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
= https://lists.bufferbloat.net/listinfo/bloat



--001a113356c6e7e37404f8315305-- From nobody Tue Apr 29 10:01:52 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1E54B1A093E for ; Tue, 29 Apr 2014 10:01:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.024 X-Spam-Level: X-Spam-Status: No, score=0.024 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_DK=1.009, MIME_8BIT_HEADER=0.3, MIME_HEADER_CTYPE_ONLY=0.717, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WMj_e6qZLN0m for ; Tue, 29 Apr 2014 10:01:47 -0700 (PDT) Received: from mail2.tohojo.dk (mail2.tohojo.dk [IPv6:2a01:4f8:200:3141::101]) by ietfa.amsl.com (Postfix) with ESMTP id C5D651A04B6 for ; Tue, 29 Apr 2014 10:01:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at mail2.tohojo.dk Sender: toke@toke.dk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toke.dk; s=201310; t=1398790898; bh=d9oHF3VHMI89FKco8/j/JiJrinvArL1VqcDgYN0Hmsk=; h=From:To:Cc:Subject:References:Date:In-Reply-To; b=NIO31E6IFBb2XSccIpIyK5gQZmUhImABN4IZ+94rP+qnUU/4hN6oDHNKHpuMGQAG3 mQGBITg/FBXL6bT4C5TKh3fUhb17wOGXoo15orsz+a0gKI9aKeGvcdNkkWCKqt0A6D lUWVz5Uo0yKcRVCZpxUWkClNI2TGoqQCvnVEYt5E= Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id A3B5120F9C; Tue, 29 Apr 2014 19:01:38 +0200 (CEST) From: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: Jim Gettys References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> Date: Tue, 29 Apr 2014 19:01:36 +0200 In-Reply-To: (Jim Gettys's message of "Tue, 29 Apr 2014 12:44:38 -0400") Message-ID: <87bnvkkr2n.fsf@toke.dk> Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/blQZRm5w66oiqJsEmw6_IQGSvMw Cc: bloat , "Fred Baker \(fred\)" , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , Mikael Abrahamsson Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 17:01:50 -0000 --=-=-= Content-Type: text/plain Jim Gettys writes: > Now, if someone gives me real fiber to the home, with a real switch fabric > upstream, rather than gpon life might be somewhat better (if the switches aren't > themselves overbuffered.... But so far, it isn't. As a data point for this, I have fibre to my apartment building and ethernet into the apartment. I get .5 ms to my upstream gateway and about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while running at 100 Mbps... http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png However, as that graph shows, it is quite possible to completely avoid bufferbloat by deploying the right shaping. And in that case fibre *does* have a significant latency advantage. The best latency I've seen to the upstream gateway on DSL has been ~12 ms. -Toke --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBCAAGBQJTX9rwAAoJEENeEGz1+utPcDkH/iozkvZUxjqcTn34AvL7l4Vi 2fpLUwIoVJkYmJ3z49nf+7ps/quPQDIkz3nHha7krsQYQT1To9WbXvszzKSgKnBl r4AaQCJWHCCiyNs4Ncbdm0wg5jlASLlBhmpaqM7e0Nb5vfW5SbQkg7hInxNKBCs6 H2peb2Tb8nQRdyY+q/FzL1aZYmdidbTbtfqBgrmO2KSE9+883upD3ouFzXmV7vRm 2QoDpl9R1ZksBAGfDkBaqOz83aSBAr4UFvV045wY16nvtP06QGjYVE+6R35Fl1lR 2uFjdZpyY3oH6LjgOTpZQH9toFrlFXGLvzGEdaI345VonOHPRtsLpWMgU3aLrBY= =w0Hb -----END PGP SIGNATURE----- --=-=-=-- From nobody Tue Apr 29 10:02:14 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E80041A0912 for ; Tue, 29 Apr 2014 10:02:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SroGzRYNdRxe for ; Tue, 29 Apr 2014 10:02:10 -0700 (PDT) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) by ietfa.amsl.com (Postfix) with ESMTP id F11531A0943 for ; Tue, 29 Apr 2014 10:02:09 -0700 (PDT) Received: by mail-wi0-f173.google.com with SMTP id z2so7780782wiv.6 for ; Tue, 29 Apr 2014 10:02:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=d0L6SFr7u+Agobug1t9QyILuZ4vOMA66GeDi/9P3t3Y=; b=KV/rjNklSTgxwYjx+me07dTTxzGyAmi6+FdnXmaJOl4QK289xx7J6lvmYPRjib8tG8 xCd0iYFtAc6jmF/tkyKu7hobU5U6lJf577YqH50lWnT9h0uik9tvKQuzuBWmb9a0/Qr9 GCNcs8OVe9uj2R3Is1oHzZSjrJJbiOMTPKf3mYzpOORx0qLu42b05gYSGNg6s4Ao59ph GYJm9I/uVni1cgnLRpH8xPibu1ixWS6c4wIDLZv0QBJz7z9ITGOYNsuvcz9MZuLRSkZN hI0n7VoqodptIiIS4JoCyd1qZHuJqxSQExue6kg/JuYAE4iKHW0buBogdMKb5oDFuUz1 tXUg== MIME-Version: 1.0 X-Received: by 10.194.63.196 with SMTP id i4mr543351wjs.50.1398790928290; Tue, 29 Apr 2014 10:02:08 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Tue, 29 Apr 2014 10:02:08 -0700 (PDT) In-Reply-To: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> Date: Tue, 29 Apr 2014 10:02:08 -0700 Message-ID: From: Dave Taht To: Jim Gettys Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/SmcCP7wpBBfqONO2ky1LEb8i1VY Cc: bloat , "Fred Baker \(fred\)" , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , Mikael Abrahamsson Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 17:02:13 -0000 On Tue, Apr 29, 2014 at 9:44 AM, Jim Gettys wrote: > > > > On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson > wrote: >> >> On Tue, 29 Apr 2014, Fred Baker (fred) wrote: >> >>> Well, we could discuss international communications. I happen to be at >>> Infocom in Toronto, VPN=E2=80=99d into Cisco San Jose, and did a ping t= o you: >> >> >> Yes, but as soon as you hit the long distance network the latency is the >> same regardless of access method. So while I agree that understanding th= e >> effect of latency is important, it's no longer a meaningful way of selli= ng >> fiber access. If your last-mile is fiber instead of ADSL2+ won't improve >> your long distance latency. > > > FIOS bufferbloat is a problem too. > > Measured bufferbloat, symmetric 25/25 service in New Jersey at my inlaw's > house is 200ms (on the ethernet port of the Actiontec router provided by > Verizon). So latency under load is the usual problem. ESR's link, before and after the cerowrt SQM treatment: https://www.bufferbloat.net/projects/codel/wiki/RRUL_Rogues_Gallery#Verizon= -FIOS-Testing-at-25Mbit-up-and-25Mbit-down > Why would you think the GPON guys are any better in principle than cable = or > DSL? Cable and DSL may be somewhat worse, just because it is older and > downward compatibility means that new modems on low bandwidth tiers are e= ven > more grossly over buffered. Well, buffering on the DSLAM or CMTS needs to be more actively managed. Fixed limits are much like conventional policing, always either too large or too small to handle sustained or bursty traffic respectively. I have been fiddling with Tim Shepard's "udpburst" tool as a quick means of measuring head end buffering, even with fq_codel present on the inbound. (It's not suitable for open internet use as yet, but code in progress can be had or enhanced at https://github.com/dtaht/isochronous ). I just added ecn and tos setting support to it. server: ./udpburst -S -E -D 32 # Server mode, enable ECN marking, set dscp to 0x20 (CS1) client: This is from a 22Mbit down CMTS d@nuc:~/git/isochronous$ ./udpburst -f 149.20.63.30 -E -C -d -n 400 -s 1400 1400 bytes -- received 382 of 400 -- 365 consecutive 0 ooo 0 dups 2 ect ........................................................................ ........................................................................ ........................................................................ ........................................................................ ........................................................................ .... .. . ... . ... . .. .. . . 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 or roughly 512k of buffering. A DSL link (6400 down) d@puck:~/git/isochronous$ ./udpburst -f snapon.lab.bufferbloat.net -n 100 -C -d -s 1000 1000 bytes -- received 71 of 100 -- 71 consecutive 0 ooo 0 dups 0 ect ...................................................................... 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 or roughly 64k worth of buffering. Interestingly the bandwidth disparity be= tween the server (gigE in isc.org's co-lo), is so great that fq_codel can't kick in before the 64k dslam buffer is overrun. > You can look at the netalyzr scatter plots in > http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-= throw-stones-at-another/ > > Now, if someone gives me real fiber to the home, with a real switch fabri= c > upstream, rather than gpon life might be somewhat better (if the switches > aren't themselves overbuffered.... But so far, it isn't. > - Jim > > - Jim > > - Jim > >> >> >> -- >> Mikael Abrahamsson email: swmike@swm.pp.se >> >> _______________________________________________ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat >> > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Tue Apr 29 10:07:32 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 112B01A0941 for ; Tue, 29 Apr 2014 10:07:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.977 X-Spam-Level: X-Spam-Status: No, score=-0.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KCFgkG6ZePeM for ; Tue, 29 Apr 2014 10:07:23 -0700 (PDT) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) by ietfa.amsl.com (Postfix) with ESMTP id CBC641A04B6 for ; Tue, 29 Apr 2014 10:07:22 -0700 (PDT) Received: by mail-oa0-f54.google.com with SMTP id i7so580988oag.27 for ; Tue, 29 Apr 2014 10:07:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=MJWG9YPI4KE0jWEiRFlvA8qEbbyNpxnFzm/JdfWmay4=; b=RXyTFZRCMuI7YE+2PHLgSJlXnzQQrUaHlynnZVj8YVrLRB7Ukv9LsmwWpOdpePnWDv WfV7ZPKEvpYi9dz2OcSdYIJikPqSsjkgqX3J729W52O1UR8PbZbz7/yy8nG3qHVeWny7 DccU/LlhPevRiIMZgQOnlG8rWMHppRb5ZNsLnB/0YnGCUbd35n9swibxOockpkgt78zr oMUt6kEUKJrwVbI8fLEtwi5OGYTT+JhEGnEbm+HpJ+aAcvH29nx9tA+AqwJ9oUAuuH99 Mj5azPAR3jn+ePj9/BoEUqKERztv8qONX2ONidV+bEJ/MTJnndSNykfuaZI0SqoglKs3 Hz3w== MIME-Version: 1.0 X-Received: by 10.182.38.199 with SMTP id i7mr3013343obk.68.1398791241584; Tue, 29 Apr 2014 10:07:21 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.76.73.100 with HTTP; Tue, 29 Apr 2014 10:07:21 -0700 (PDT) In-Reply-To: <87bnvkkr2n.fsf@toke.dk> References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> <87bnvkkr2n.fsf@toke.dk> Date: Tue, 29 Apr 2014 13:07:21 -0400 X-Google-Sender-Auth: LKaC8kagsl7Au9-8GCEVDakmWIY Message-ID: From: Jim Gettys To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Content-Type: multipart/alternative; boundary=001a11c32ddc11da6c04f83177b6 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/CaAxdSVDNvuN2rntAaej7Lpr5Mk Cc: bloat , "Fred Baker \(fred\)" , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , Mikael Abrahamsson Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 17:07:24 -0000 --001a11c32ddc11da6c04f83177b6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Apr 29, 2014 at 1:01 PM, Toke H=C3=B8iland-J=C3=B8rgensen wrote: > Jim Gettys writes: > > > Now, if someone gives me real fiber to the home, with a real switch > fabric > > upstream, rather than gpon life might be somewhat better (if the > switches aren't > > themselves overbuffered.... But so far, it isn't. > > As a data point for this, I have fibre to my apartment building and > ethernet into the apartment. I get .5 ms to my upstream gateway and > about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while > running at 100 Mbps... > > http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png > > However, as that graph shows, it is quite possible to completely avoid > bufferbloat by deploying the right shaping=E2=80=8B And in that case fibre > *does* have a significant latency advantage. The best latency I've seen > to the upstream gateway on DSL has been ~12 ms. > =E2=80=8BMedia access is a killer on Cable too, putting the latency floor a= t around 8ms on my Docsis 3.0 Comcast service, though you can sometimes get lucky and piggyback. to somewhat lower latency, IIRC conversations with Greg White about how cable works. - Jim > -Toke > --001a11c32ddc11da6c04f83177b6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Tue= , Apr 29, 2014 at 1:01 PM, Toke H=C3=B8iland-J=C3=B8rgensen <toke@toke.dk&g= t; wrote:
Jim Gettys <jg@freedesktop.org> writes:

> Now, if someone gives me real fiber to the home, with a real switch fa= bric
> upstream, rather than gpon life might be somewhat better (if the switc= hes aren't
> themselves overbuffered.... But so far, it isn't.

As a data point for this, I have fibre to my apartment building and ethernet into the apartment. I get .5 ms to my upstream gateway and
about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while
running at 100 Mbps...

http://files.toke.dk/bufferbloat/data/karlstad/cdf_com= parison.png

However, as that graph shows, it is quite possible to completely avoid
bufferbloat by deploying the right shaping=E2=80=8B
And in that case fibre
*does* have a significant latency advantage. The best latency I've seen=
to the upstream gateway on DSL has been ~12 ms.

=E2=80=8BMedia a= ccess is a killer on Cable too, putting the latency floor at around 8ms on = my Docsis 3.0 Comcast service, though you can sometimes get lucky and piggy= back. to somewhat lower latency, IIRC conversations with Greg White about h= ow cable works.
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0- Jim


-Toke

--001a11c32ddc11da6c04f83177b6-- From nobody Tue Apr 29 11:09:40 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1BFE41A093E for ; Tue, 29 Apr 2014 11:09:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.114 X-Spam-Level: X-Spam-Status: No, score=-0.114 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RP_MATCHES_RCVD=-0.651] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CXkbgvD-p45k for ; Tue, 29 Apr 2014 11:09:23 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 7CC6C1A04AF for ; Tue, 29 Apr 2014 11:09:23 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id s3TI9Ggp002772; Tue, 29 Apr 2014 12:09:17 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Tue, 29 Apr 2014 12:09:16 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0174.001; Tue, 29 Apr 2014 12:09:16 -0600 From: Greg White To: Jim Gettys , =?utf-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Thread-Topic: [aqm] [Bloat] the side effects of 330ms lag in the real world Thread-Index: AQHPY821FCT9UMBvhkKEiW4/EzD9ypso5KYA Date: Tue, 29 Apr 2014 18:09:15 +0000 Message-ID: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> <87bnvkkr2n.fsf@toke.dk> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.4.10.154] Content-Type: multipart/alternative; boundary="_000_CF85404933612gwhitecablelabscom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/JXKj4zDuTnq3Er4dWYkuZmj-SG4 Cc: Mikael Abrahamsson , "aqm@ietf.org" , "Fred Baker \(fred\)" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 18:09:27 -0000 --_000_CF85404933612gwhitecablelabscom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 RllJIHBpZ2d5YmFja2VkIHJlcXVlc3RzIGluIERPQ1NJUyBlbGltaW5hdGUgdGhlIHBvc3NpYmls aXR5IG9mIHJlcXVlc3QgY29sbGlzaW9ucyAoYW5kIHJlc3VsdGluZyBiYWNrb2ZmL3JldHJ5KS4g IEluIERPQ1NJUyAzLjAgdGhleSB3aWxsIHJlc3VsdCBpbiBsb3dlciBsYXRlbmN5IG9ubHkgYXMg YSByZXN1bHQgb2YgZWxpbWluYXRpbmcgdGhlc2UgZXZlbnRzLiAgSW4gYSBsaWdodGx5IGxvYWRl ZCBET0NTSVMgMy4wIG5ldHdvcmsgKGZldyBuZWlnaGJvcnMgY29udGVuZGluZyBmb3IgYmFuZHdp ZHRoKSB0aGUgY29sbGlzaW9uIHByb2JhYmlsaXR5IHdpbGwgYmUgbG93IGFueXdheSwgc28gaXQg d29uJ3QgbWFrZSBtdWNoIGRpZmZlcmVuY2UuICBJbiBhIGhpZ2hseSB1dGlsaXplZCBuZXR3b3Jr IChlc3BlY2lhbGx5IGluIHRoZSBjYXNlIHdoZXJlIGEgbG90IG9mIG5laWdoYm9ycyBhcmUgc2Vu ZGluZyBhdCByYXRlcyBiZWxvdyB0aGUgcGlnZ3liYWNraW5nIHRocmVzaG9sZCkgaXQgY2FuIG1h a2UgYSBiaWdnZXIgZGlmZmVyZW5jZS4NCg0KUGFja2V0IHJhdGVzIGFib3ZlIH4yMDBwcHMgd2ls bCBiZSBzdWZmaWNpZW50IHRvIGVuc3VyZSBwaWdneWJhY2tpbmcuDQoNCkkgdXN1YWxseSBjb25z aWRlciB+Nm1zIHRvIGJlIHRoZSBub21pbmFsIHVwc3RyZWFtIE1BQyBsYXRlbmN5LCBhbmQgfjAu N21zIHRvIGJlIHRoZSBlcXVpdmFsZW50IG9uIGRvd25zdHJlYW0sIG1ha2luZyB0aGUgUlRUIG9u IHRoZSBET0NTSVMgMy4wIGxpbmsgY2xvc2UgdG8gN21zIChkZXBlbmRpbmcgb24gY29uZmlndXJh dGlvbikuDQoNCi1HcmVnDQoNCkZyb206IEppbSBHZXR0eXMgPGpnQGZyZWVkZXNrdG9wLm9yZzxt YWlsdG86amdAZnJlZWRlc2t0b3Aub3JnPj4NCkRhdGU6IFR1ZXNkYXksIEFwcmlsIDI5LCAyMDE0 IGF0IDExOjA3IEFNDQpUbzogVG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2VuIDx0b2tlQHRva2UuZGs8 bWFpbHRvOnRva2VAdG9rZS5kaz4+DQpDYzogYmxvYXQgPGJsb2F0QGxpc3RzLmJ1ZmZlcmJsb2F0 Lm5ldDxtYWlsdG86YmxvYXRAbGlzdHMuYnVmZmVyYmxvYXQubmV0Pj4sICJGcmVkIEJha2VyIChm cmVkKSIgPGZyZWRAY2lzY28uY29tPG1haWx0bzpmcmVkQGNpc2NvLmNvbT4+LCAiYXFtQGlldGYu b3JnPG1haWx0bzphcW1AaWV0Zi5vcmc+IiA8YXFtQGlldGYub3JnPG1haWx0bzphcW1AaWV0Zi5v cmc+PiwgImNlcm93cnQtZGV2ZWxAbGlzdHMuYnVmZmVyYmxvYXQubmV0PG1haWx0bzpjZXJvd3J0 LWRldmVsQGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldD4iIDxjZXJvd3J0LWRldmVsQGxpc3RzLmJ1ZmZl cmJsb2F0Lm5ldDxtYWlsdG86Y2Vyb3dydC1kZXZlbEBsaXN0cy5idWZmZXJibG9hdC5uZXQ+Piwg TWlrYWVsIEFicmFoYW1zc29uIDxzd21pa2VAc3dtLnBwLnNlPG1haWx0bzpzd21pa2VAc3dtLnBw LnNlPj4NClN1YmplY3Q6IFJlOiBbYXFtXSBbQmxvYXRdIHRoZSBzaWRlIGVmZmVjdHMgb2YgMzMw bXMgbGFnIGluIHRoZSByZWFsIHdvcmxkDQoNCg0KDQoNCk9uIFR1ZSwgQXByIDI5LCAyMDE0IGF0 IDE6MDEgUE0sIFRva2UgSMO4aWxhbmQtSsO4cmdlbnNlbiA8dG9rZUB0b2tlLmRrPG1haWx0bzp0 b2tlQHRva2UuZGs+PiB3cm90ZToNCkppbSBHZXR0eXMgPGpnQGZyZWVkZXNrdG9wLm9yZzxtYWls dG86amdAZnJlZWRlc2t0b3Aub3JnPj4gd3JpdGVzOg0KDQo+IE5vdywgaWYgc29tZW9uZSBnaXZl cyBtZSByZWFsIGZpYmVyIHRvIHRoZSBob21lLCB3aXRoIGEgcmVhbCBzd2l0Y2ggZmFicmljDQo+ IHVwc3RyZWFtLCByYXRoZXIgdGhhbiBncG9uIGxpZmUgbWlnaHQgYmUgc29tZXdoYXQgYmV0dGVy IChpZiB0aGUgc3dpdGNoZXMgYXJlbid0DQo+IHRoZW1zZWx2ZXMgb3ZlcmJ1ZmZlcmVkLi4uLiBC dXQgc28gZmFyLCBpdCBpc24ndC4NCg0KQXMgYSBkYXRhIHBvaW50IGZvciB0aGlzLCBJIGhhdmUg ZmlicmUgdG8gbXkgYXBhcnRtZW50IGJ1aWxkaW5nIGFuZA0KZXRoZXJuZXQgaW50byB0aGUgYXBh cnRtZW50LiBJIGdldCAuNSBtcyB0byBteSB1cHN0cmVhbSBnYXRld2F5IGFuZA0KYWJvdXQgNiBt cyB0byBHb29nbGUuIFN0aWxsIG1lYXN1cmVkIHVwIHRvIH4yMCBtcyBvZiBidWZmZXJibG9hdCB3 aGlsZQ0KcnVubmluZyBhdCAxMDAgTWJwcy4uLg0KDQpodHRwOi8vZmlsZXMudG9rZS5kay9idWZm ZXJibG9hdC9kYXRhL2thcmxzdGFkL2NkZl9jb21wYXJpc29uLnBuZw0KDQpIb3dldmVyLCBhcyB0 aGF0IGdyYXBoIHNob3dzLCBpdCBpcyBxdWl0ZSBwb3NzaWJsZSB0byBjb21wbGV0ZWx5IGF2b2lk DQpidWZmZXJibG9hdCBieSBkZXBsb3lpbmcgdGhlIHJpZ2h0IHNoYXBpbmfigIsNCkFuZCBpbiB0 aGF0IGNhc2UgZmlicmUNCipkb2VzKiBoYXZlIGEgc2lnbmlmaWNhbnQgbGF0ZW5jeSBhZHZhbnRh Z2UuIFRoZSBiZXN0IGxhdGVuY3kgSSd2ZSBzZWVuDQp0byB0aGUgdXBzdHJlYW0gZ2F0ZXdheSBv biBEU0wgaGFzIGJlZW4gfjEyIG1zLg0KDQrigItNZWRpYSBhY2Nlc3MgaXMgYSBraWxsZXIgb24g Q2FibGUgdG9vLCBwdXR0aW5nIHRoZSBsYXRlbmN5IGZsb29yIGF0IGFyb3VuZCA4bXMgb24gbXkg RG9jc2lzIDMuMCBDb21jYXN0IHNlcnZpY2UsIHRob3VnaCB5b3UgY2FuIHNvbWV0aW1lcyBnZXQg bHVja3kgYW5kIHBpZ2d5YmFjay4gdG8gc29tZXdoYXQgbG93ZXIgbGF0ZW5jeSwgSUlSQyBjb252 ZXJzYXRpb25zIHdpdGggR3JlZyBXaGl0ZSBhYm91dCBob3cgY2FibGUgd29ya3MuDQogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtIEppbQ0KDQoNCi1Ub2tlDQoNCg== --_000_CF85404933612gwhitecablelabscom_ Content-Type: text/html; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgLXdlYmtpdC1saW5lLWJy ZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAx NHB4OyBmb250LWZhbWlseTogQ2FsaWJyaSwgc2Fucy1zZXJpZjsiPg0KPGRpdj5GWUkgcGlnZ3li YWNrZWQgcmVxdWVzdHMgaW4gRE9DU0lTIGVsaW1pbmF0ZSB0aGUgcG9zc2liaWxpdHkgb2YgcmVx dWVzdCBjb2xsaXNpb25zIChhbmQgcmVzdWx0aW5nIGJhY2tvZmYvcmV0cnkpLiAmbmJzcDtJbiBE T0NTSVMgMy4wIHRoZXkgd2lsbCByZXN1bHQgaW4gbG93ZXIgbGF0ZW5jeSBvbmx5IGFzIGEgcmVz dWx0IG9mIGVsaW1pbmF0aW5nIHRoZXNlIGV2ZW50cy4gJm5ic3A7SW4gYSBsaWdodGx5IGxvYWRl ZCBET0NTSVMgMy4wIG5ldHdvcmsgKGZldw0KIG5laWdoYm9ycyBjb250ZW5kaW5nIGZvciBiYW5k d2lkdGgpIHRoZSBjb2xsaXNpb24gcHJvYmFiaWxpdHkgd2lsbCBiZSBsb3cgYW55d2F5LCBzbyBp dCB3b24ndCBtYWtlIG11Y2ggZGlmZmVyZW5jZS4gJm5ic3A7SW4gYSBoaWdobHkgdXRpbGl6ZWQg bmV0d29yayAoZXNwZWNpYWxseSBpbiB0aGUgY2FzZSB3aGVyZSBhIGxvdCBvZiBuZWlnaGJvcnMg YXJlIHNlbmRpbmcgYXQgcmF0ZXMgYmVsb3cgdGhlIHBpZ2d5YmFja2luZyB0aHJlc2hvbGQpIGl0 IGNhbg0KIG1ha2UgYSBiaWdnZXIgZGlmZmVyZW5jZS48L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+ DQo8ZGl2PlBhY2tldCByYXRlcyBhYm92ZSB+MjAwcHBzIHdpbGwgYmUgc3VmZmljaWVudCB0byBl bnN1cmUgcGlnZ3liYWNraW5nLjwvZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxkaXY+SSB1c3Vh bGx5IGNvbnNpZGVyIH42bXMgdG8gYmUgdGhlIG5vbWluYWwgdXBzdHJlYW0gTUFDIGxhdGVuY3ks IGFuZCB+MC43bXMgdG8gYmUgdGhlIGVxdWl2YWxlbnQgb24gZG93bnN0cmVhbSwgbWFraW5nIHRo ZSBSVFQgb24gdGhlIERPQ1NJUyAzLjAgbGluayBjbG9zZSB0byA3bXMgKGRlcGVuZGluZyBvbiBj b25maWd1cmF0aW9uKS4mbmJzcDs8L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2Pi1HcmVn PC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPHNwYW4gaWQ9Ik9MS19TUkNfQk9EWV9TRUNUSU9O Ij4NCjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGlicmk7IGZvbnQtc2l6ZToxMXB0OyB0ZXh0 LWFsaWduOmxlZnQ7IGNvbG9yOmJsYWNrOyBCT1JERVItQk9UVE9NOiBtZWRpdW0gbm9uZTsgQk9S REVSLUxFRlQ6IG1lZGl1bSBub25lOyBQQURESU5HLUJPVFRPTTogMGluOyBQQURESU5HLUxFRlQ6 IDBpbjsgUEFERElORy1SSUdIVDogMGluOyBCT1JERVItVE9QOiAjYjVjNGRmIDFwdCBzb2xpZDsg Qk9SREVSLVJJR0hUOiBtZWRpdW0gbm9uZTsgUEFERElORy1UT1A6IDNwdCI+DQo8c3BhbiBzdHls ZT0iZm9udC13ZWlnaHQ6Ym9sZCI+RnJvbTogPC9zcGFuPkppbSBHZXR0eXMgJmx0OzxhIGhyZWY9 Im1haWx0bzpqZ0BmcmVlZGVza3RvcC5vcmciPmpnQGZyZWVkZXNrdG9wLm9yZzwvYT4mZ3Q7PGJy Pg0KPHNwYW4gc3R5bGU9ImZvbnQtd2VpZ2h0OmJvbGQiPkRhdGU6IDwvc3Bhbj5UdWVzZGF5LCBB cHJpbCAyOSwgMjAxNCBhdCAxMTowNyBBTTxicj4NCjxzcGFuIHN0eWxlPSJmb250LXdlaWdodDpi b2xkIj5UbzogPC9zcGFuPlRva2UgSMO4aWxhbmQtSsO4cmdlbnNlbiAmbHQ7PGEgaHJlZj0ibWFp bHRvOnRva2VAdG9rZS5kayI+dG9rZUB0b2tlLmRrPC9hPiZndDs8YnI+DQo8c3BhbiBzdHlsZT0i Zm9udC13ZWlnaHQ6Ym9sZCI+Q2M6IDwvc3Bhbj5ibG9hdCAmbHQ7PGEgaHJlZj0ibWFpbHRvOmJs b2F0QGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldCI+YmxvYXRAbGlzdHMuYnVmZmVyYmxvYXQubmV0PC9h PiZndDssICZxdW90O0ZyZWQgQmFrZXIgKGZyZWQpJnF1b3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86 ZnJlZEBjaXNjby5jb20iPmZyZWRAY2lzY28uY29tPC9hPiZndDssICZxdW90OzxhIGhyZWY9Im1h aWx0bzphcW1AaWV0Zi5vcmciPmFxbUBpZXRmLm9yZzwvYT4mcXVvdDsgJmx0OzxhIGhyZWY9Im1h aWx0bzphcW1AaWV0Zi5vcmciPmFxbUBpZXRmLm9yZzwvYT4mZ3Q7LA0KICZxdW90OzxhIGhyZWY9 Im1haWx0bzpjZXJvd3J0LWRldmVsQGxpc3RzLmJ1ZmZlcmJsb2F0Lm5ldCI+Y2Vyb3dydC1kZXZl bEBsaXN0cy5idWZmZXJibG9hdC5uZXQ8L2E+JnF1b3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86Y2Vy b3dydC1kZXZlbEBsaXN0cy5idWZmZXJibG9hdC5uZXQiPmNlcm93cnQtZGV2ZWxAbGlzdHMuYnVm ZmVyYmxvYXQubmV0PC9hPiZndDssIE1pa2FlbCBBYnJhaGFtc3NvbiAmbHQ7PGEgaHJlZj0ibWFp bHRvOnN3bWlrZUBzd20ucHAuc2UiPnN3bWlrZUBzd20ucHAuc2U8L2E+Jmd0Ozxicj4NCjxzcGFu IHN0eWxlPSJmb250LXdlaWdodDpib2xkIj5TdWJqZWN0OiA8L3NwYW4+UmU6IFthcW1dIFtCbG9h dF0gdGhlIHNpZGUgZWZmZWN0cyBvZiAzMzBtcyBsYWcgaW4gdGhlIHJlYWwgd29ybGQ8YnI+DQo8 L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXYgZGlyPSJsdHIiPg0K PGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImZvbnQtc2l6ZTpzbWFsbCI+PGJyPg0K PC9kaXY+DQo8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGJyPg0KPGJyPg0KPGRpdiBjbGFzcz0i Z21haWxfcXVvdGUiPk9uIFR1ZSwgQXByIDI5LCAyMDE0IGF0IDE6MDEgUE0sIFRva2UgSMO4aWxh bmQtSsO4cmdlbnNlbiA8c3BhbiBkaXI9Imx0ciI+DQombHQ7PGEgaHJlZj0ibWFpbHRvOnRva2VA dG9rZS5kayIgdGFyZ2V0PSJfYmxhbmsiPnRva2VAdG9rZS5kazwvYT4mZ3Q7PC9zcGFuPiB3cm90 ZTo8YnI+DQo8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAw IDAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxk aXYgY2xhc3M9IiI+SmltIEdldHR5cyAmbHQ7PGEgaHJlZj0ibWFpbHRvOmpnQGZyZWVkZXNrdG9w Lm9yZyI+amdAZnJlZWRlc2t0b3Aub3JnPC9hPiZndDsgd3JpdGVzOjxicj4NCjxicj4NCiZndDsg Tm93LCBpZiBzb21lb25lIGdpdmVzIG1lIHJlYWwgZmliZXIgdG8gdGhlIGhvbWUsIHdpdGggYSBy ZWFsIHN3aXRjaCBmYWJyaWM8YnI+DQomZ3Q7IHVwc3RyZWFtLCByYXRoZXIgdGhhbiBncG9uIGxp ZmUgbWlnaHQgYmUgc29tZXdoYXQgYmV0dGVyIChpZiB0aGUgc3dpdGNoZXMgYXJlbid0PGJyPg0K Jmd0OyB0aGVtc2VsdmVzIG92ZXJidWZmZXJlZC4uLi4gQnV0IHNvIGZhciwgaXQgaXNuJ3QuPGJy Pg0KPGJyPg0KPC9kaXY+DQpBcyBhIGRhdGEgcG9pbnQgZm9yIHRoaXMsIEkgaGF2ZSBmaWJyZSB0 byBteSBhcGFydG1lbnQgYnVpbGRpbmcgYW5kPGJyPg0KZXRoZXJuZXQgaW50byB0aGUgYXBhcnRt ZW50LiBJIGdldCAuNSBtcyB0byBteSB1cHN0cmVhbSBnYXRld2F5IGFuZDxicj4NCmFib3V0IDYg bXMgdG8gR29vZ2xlLiBTdGlsbCBtZWFzdXJlZCB1cCB0byB+MjAgbXMgb2YgYnVmZmVyYmxvYXQg d2hpbGU8YnI+DQpydW5uaW5nIGF0IDEwMCBNYnBzLi4uPGJyPg0KPGJyPg0KPGEgaHJlZj0iaHR0 cDovL2ZpbGVzLnRva2UuZGsvYnVmZmVyYmxvYXQvZGF0YS9rYXJsc3RhZC9jZGZfY29tcGFyaXNv bi5wbmciIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vZmlsZXMudG9rZS5kay9idWZmZXJibG9hdC9k YXRhL2thcmxzdGFkL2NkZl9jb21wYXJpc29uLnBuZzwvYT48YnI+DQo8YnI+DQpIb3dldmVyLCBh cyB0aGF0IGdyYXBoIHNob3dzLCBpdCBpcyBxdWl0ZSBwb3NzaWJsZSB0byBjb21wbGV0ZWx5IGF2 b2lkPGJyPg0KYnVmZmVyYmxvYXQgYnkgZGVwbG95aW5nIHRoZSByaWdodCBzaGFwaW5n4oCLPC9i bG9ja3F1b3RlPg0KPGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2lu OjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+ DQpBbmQgaW4gdGhhdCBjYXNlIGZpYnJlPGJyPg0KKmRvZXMqIGhhdmUgYSBzaWduaWZpY2FudCBs YXRlbmN5IGFkdmFudGFnZS4gVGhlIGJlc3QgbGF0ZW5jeSBJJ3ZlIHNlZW48YnI+DQp0byB0aGUg dXBzdHJlYW0gZ2F0ZXdheSBvbiBEU0wgaGFzIGJlZW4gfjEyIG1zLjxicj4NCjwvYmxvY2txdW90 ZT4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSJnbWFpbF9kZWZhdWx0IiBzdHlsZT0i Zm9udC1zaXplOnNtYWxsIj7igItNZWRpYSBhY2Nlc3MgaXMgYSBraWxsZXIgb24gQ2FibGUgdG9v LCBwdXR0aW5nIHRoZSBsYXRlbmN5IGZsb29yIGF0IGFyb3VuZCA4bXMgb24gbXkgRG9jc2lzIDMu MCBDb21jYXN0IHNlcnZpY2UsIHRob3VnaCB5b3UgY2FuIHNvbWV0aW1lcyBnZXQgbHVja3kgYW5k IHBpZ2d5YmFjay4gdG8gc29tZXdoYXQgbG93ZXIgbGF0ZW5jeSwgSUlSQyBjb252ZXJzYXRpb25z DQogd2l0aCBHcmVnIFdoaXRlIGFib3V0IGhvdyBjYWJsZSB3b3Jrcy48L2Rpdj4NCjxkaXYgY2xh c3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LXNpemU6c21hbGwiPiZuYnNwOyAmbmJzcDsg Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAm bmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZu YnNwOyAmbmJzcDstIEppbTwvZGl2Pg0KPGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9 ImZvbnQtc2l6ZTpzbWFsbCI+PGJyPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxf cXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xp ZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxzcGFuIGNsYXNzPSJIT0VuWmIiPjxmb250IGNvbG9yPSIj ODg4ODg4Ij48YnI+DQotVG9rZTxicj4NCjwvZm9udD48L3NwYW4+PC9ibG9ja3F1b3RlPg0KPC9k aXY+DQo8YnI+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvc3Bhbj4NCjwvYm9k eT4NCjwvaHRtbD4NCg== --_000_CF85404933612gwhitecablelabscom_-- From nobody Tue Apr 29 20:22:10 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0C351A09A2 for ; Tue, 29 Apr 2014 20:22:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.7 X-Spam-Level: X-Spam-Status: No, score=-1.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M9f0mfU48x37 for ; Tue, 29 Apr 2014 20:22:02 -0700 (PDT) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) by ietfa.amsl.com (Postfix) with ESMTP id D0B081A0985 for ; Tue, 29 Apr 2014 20:22:01 -0700 (PDT) Received: by mail-we0-f182.google.com with SMTP id u57so1087261wes.13 for ; Tue, 29 Apr 2014 20:22:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=c53+Hx+6AyIxGWdvz4lK/FywyXdwbb8aRKbdwBGb4Bk=; b=WZEa7OXxcDoLAlz2reCvnYEGq7Qwx5tKBeIM4OKOLBY/JM2qVrN6vCcS1SrrRLvOGH HFhlFzWszNwk5w5T5zqLPgNRw2bpvYJTXlRw8FreD+gsDVWMhJM70hKvgFgsj1Zxm3HF 8WiSMNYgw8/t+gQwrihjekRkBPKvfp3MvurxIb98bT3R20OMjxUUwtugFYr9j2vDlAfz /MskjTDAMG9ObnSa1XsLmo+CgeIJqQDgyJV8YGEH2aBaCu+OpqEXRkb6YFc0iSiKMpuu dr3WEKU+MZ/hpZZFUCH7hjVPI4zNDo8H4OS1lQAfcH5SFXXnnBYnRJK3hwoIKtsrD6Kh 1Vqw== MIME-Version: 1.0 X-Received: by 10.180.198.46 with SMTP id iz14mr1353563wic.46.1398828120022; Tue, 29 Apr 2014 20:22:00 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Tue, 29 Apr 2014 20:21:59 -0700 (PDT) In-Reply-To: <87bnvkkr2n.fsf@toke.dk> References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> <87bnvkkr2n.fsf@toke.dk> Date: Tue, 29 Apr 2014 20:21:59 -0700 Message-ID: From: Dave Taht To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/HyfBSEHO8woVSO54I7Srszkgujk Cc: Jim Gettys , "Fred Baker \(fred\)" , "cerowrt-devel@lists.bufferbloat.net" , bloat , "aqm@ietf.org" , Mikael Abrahamsson Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 03:22:06 -0000 On Tue, Apr 29, 2014 at 10:01 AM, Toke H=C3=B8iland-J=C3=B8rgensen wrote: > Jim Gettys writes: > >> Now, if someone gives me real fiber to the home, with a real switch fabr= ic >> upstream, rather than gpon life might be somewhat better (if the switche= s aren't >> themselves overbuffered.... But so far, it isn't. > > As a data point for this, I have fibre to my apartment building and > ethernet into the apartment. I get .5 ms to my upstream gateway and > about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while > running at 100 Mbps... > > http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png I need to note that what this wonderfully flat CDF for the measurement stream shows is that short flows under fq_codel leap to the head of the queue ever better as you get more and more bandwidth available. The background load flows not shown on this graph are experiencing 5-20ms worth of latency in each direction as per codel's algorithm. A better test (in progress) would measure typical voip behaviors.... > However, as that graph shows, it is quite possible to completely avoid > bufferbloat by deploying the right shaping. It does not "completely avoid bufferbloat", the fq_codel "fast queue" merely eliminates queuing delay for sparse flows, things like arp, syn, syn/ack, dns, ntp, etc, as well as the first packet of any flow that has not built up a queue yet. (which is, admittedly, quite a lot of bufferbloat reduction) The rest of the magic comes from codel. > And in that case fibre > *does* have a significant latency advantage. The best latency I've seen > to the upstream gateway on DSL has been ~12 ms. And reduced RTT =3D money. this piece states observed average RTTs at peak times were 17ms for fiber, 28ms for cable, and 44ms for DSL. http://www.igvita.com/2012/07/19/latency-the-new-web-performance-bottleneck= / I don't know if the underlying report measures baseline unloaded last mile = RTT. > > -Toke > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article From nobody Tue Apr 29 23:17:07 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BE00B1A09DF for ; Tue, 29 Apr 2014 23:16:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.602 X-Spam-Level: X-Spam-Status: No, score=-4.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gdZF0UYpbBk1 for ; Tue, 29 Apr 2014 23:16:51 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) by ietfa.amsl.com (Postfix) with ESMTP id D869B1A09C5 for ; Tue, 29 Apr 2014 23:16:50 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 4E3AEA6; Wed, 30 Apr 2014 08:16:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1398838608; bh=cw8BNmj0MzIs9ATMpyyuEDh3G0LnFGeTEsula+cgTc4=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=Q7ADiIWfYdp5Kh8VwfQ05M74OkCogFDfS97TBIxmOTWwecFmUBR4mIRkyDt0YmcPc 5WrLfr2YfNl4NqnBrXqDdLTUN6/t1F4nL07y26lK62ye43UVlB2UdWS743Wo1gcmKO /wdRghjJ/3UImbNlaHlWyesD3q/edOy0UCBHvOeE= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 47F3AA5; Wed, 30 Apr 2014 08:16:48 +0200 (CEST) Date: Wed, 30 Apr 2014 08:16:48 +0200 (CEST) From: Mikael Abrahamsson To: Jim Gettys In-Reply-To: Message-ID: References: <4130D000-FE28-4A5E-B824-3371C1602472@cisco.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/jQIAQQvkIkaQMGQGWYyiTgaTTg8 Cc: "aqm@ietf.org" , "Fred Baker \(fred\)" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 06:16:55 -0000 On Tue, 29 Apr 2014, Jim Gettys wrote: > Why would you think the GPON guys are any better in principle than cable or Fiber != GPON. Fiber in Sweden is 99% active ethernet. And the advertisement isn't about bufferbloat, I doubt they even know what that is... Upside is that the equipment used in Sweden is usually cheap ethernet switches which don't have buffering and might also use policing instead of shaping which makes bufferbloat problem basically go away, but on the other hand creates other problems. -- Mikael Abrahamsson email: swmike@swm.pp.se From nobody Wed Apr 30 07:09:37 2014 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C859B1A6FA9 for ; Wed, 30 Apr 2014 07:09:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qy7t1UgKquTK for ; Wed, 30 Apr 2014 07:09:35 -0700 (PDT) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) by ietfa.amsl.com (Postfix) with ESMTP id 3D61A1A08DF for ; Wed, 30 Apr 2014 07:09:35 -0700 (PDT) Received: by mail-wi0-f177.google.com with SMTP id cc10so2259678wib.4 for ; Wed, 30 Apr 2014 07:09:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uHVfseZh3SP9jAhs+6/I8w+nhyqGGTusgvr3j9iv2Q4=; b=cJxYPq9/c0lndrlu2cjpZnX8TBtcLr+VOUkU2M8R1bR4e5LKuBUEMZHWALtfb/bKfE wTiHJ3d5rwkQVq35TBo7Qw61RpyNGAKRDEFlkJKzfWQ2JfrtiYpYTX9jDxceP8UW1GeR iKGewGzSnAsLvbpEuEMr6c6pjaR1ms4OWQH7tVQEO1oJ9DFlP5E0Ks+X5zC+Mrrfgxde m4//EssXyi1nmyBR4vrWKocz1sZQoNhaH6tBDvzP4opz6F5uERVW9HluQEDQHLfzSrkW vf+C8XkIMgbTA6hiF8NfGPWDM7CYiwiSL+kkF//dZifiziasRKJsZKoaDxgJZ8zoAkm9 jocw== MIME-Version: 1.0 X-Received: by 10.180.76.166 with SMTP id l6mr3876539wiw.17.1398866973363; Wed, 30 Apr 2014 07:09:33 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Wed, 30 Apr 2014 07:09:33 -0700 (PDT) In-Reply-To: References: Date: Wed, 30 Apr 2014 07:09:33 -0700 Message-ID: From: Dave Taht To: Greg White Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/aqm/7FHJ_p9MBmy7s4kZuBmG9hDdsUM Cc: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= , "aqm@ietf.org" , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [aqm] chrome web page benchmarker fixed X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 14:09:36 -0000 Doug Orr recommended to us that we give http://www.chromium.org/developers/telemetry a shot in generating reproducible web traffic models.