From mirja.kuehlewind@ikr.uni-stuttgart.de Tue Jun 4 07:58:01 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40A0321F9EB1 for ; Tue, 4 Jun 2013 07:58:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.707 X-Spam-Level: X-Spam-Status: No, score=-0.707 tagged_above=-999 required=5 tests=[AWL=1.015, BAYES_00=-2.599, HELO_EQ_DE=0.35, MIME_8BIT_HEADER=0.3, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c2wfoahbw-ar for ; Tue, 4 Jun 2013 07:57:47 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by ietfa.amsl.com (Postfix) with ESMTP id B916221F99F3 for ; Tue, 4 Jun 2013 06:11:16 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id 27BBB601FA; Tue, 4 Jun 2013 15:11:15 +0200 (CEST) Received: from vpn-2-cl195 (vpn-2-cl195 [10.41.21.195]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with ESMTP id 220AE601F8; Tue, 4 Jun 2013 15:11:15 +0200 (CEST) From: Mirja =?iso-8859-1?q?K=FChlewind?= To: Wesley Eddy Date: Tue, 4 Jun 2013 15:11:14 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20101217.1207316) References: <5190FB21.5080100@mti-systems.com> <201305301420.51606.mirja.kuehlewind@ikr.uni-stuttgart.de> <51A77AB5.9040302@mti-systems.com> In-Reply-To: <51A77AB5.9040302@mti-systems.com> X-KMail-QuotePrefix: > MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201306041511.14779.mkuehle@ikr.uni-stuttgart.de> Cc: aqm@ietf.org Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jun 2013 14:58:01 -0000 Hi Wes, thanks, that's already very helpful. I was maybe even thinking about a milestone on something like 'requirements for initial deployment'... don't know if this makes sense though. Mirja On Thursday 30 May 2013 18:13:41 Wesley Eddy wrote: > On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: > > But my point is, we should not only to standardize more and more AQM > > mechanisms because that's a large research area which maybe even should > > be homed in the IRTF, but we need to find actually deployment strategies. > > Having a working group on AQM will already help to make people aware of > > the topic and maybe think about using AQM. But only standardizing more > > and more AQM queue, might end up in investing a lot of working that > > no-one will ever use. > > I suggest appending something like this to the charter: > > Many AQM algorithms have been proposed in academic literature, but > very few are widely implemented and deployed. A goal of the working > group is to produce recommendations that will actually be used, and > algorithms that will actually be implemented, deployed in equipment, > and enabled. Towards these ends, the group actively encourages > participation from operators and implementers, and will coordinate > with the IETF OPS area and other relevant parts of the IETF and > Internet community. Wider research and evaluation of AQM mechanisms > shall be coordinated with the IRTF/ICCRG, and significant > participation in this WG from the academic and research community is > highly desirable, when it is directly relevant to implementation and > deployment. > > > We will definitely need to get engagement from some of the operators > that participate in the IETF, and should solicit participation more > widely outside as well (e.g. advertise to NANOG list, bufferbloat > list, etc.) in order to attract operators and implementers that aren't > already aware of this activity. > > Would this help to address your concern? From gettysjim@gmail.com Tue Jun 4 10:27:24 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A7A4D21F9B9E for ; Tue, 4 Jun 2013 10:27:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.363 X-Spam-Level: X-Spam-Status: No, score=-2.363 tagged_above=-999 required=5 tests=[AWL=0.386, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IkADlCPWgddU for ; Tue, 4 Jun 2013 10:27:11 -0700 (PDT) Received: from mail-qe0-f45.google.com (mail-qe0-f45.google.com [209.85.128.45]) by ietfa.amsl.com (Postfix) with ESMTP id 02A2621F9702 for ; Tue, 4 Jun 2013 08:47:10 -0700 (PDT) Received: by mail-qe0-f45.google.com with SMTP id q19so286512qeb.32 for ; Tue, 04 Jun 2013 08:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=mcomXxOjW3FkgrDdz9rwmk+KA/rhzrlLG8aB9nUJWDk=; b=iiDO0/3NPgzOtQFr5ypHsk/8aLC9yg1H7+hYK9ZvUa8W7Zv3PC/2nJ/cS5y6jfYlzC Bz34rPUDSN5v58bzEXFp52GHUjFtSy87G4sWED6yabTnEZ0intac843IZTTXWBvw2FmI nGBDzdXWQqOwcOSq5AY+gh1gL+HeScDDCjg064v1CJbTK3eNCCL6M3NLTYXqu+YhNAV8 n0YLkE8hNolR9N4xkriJeWhbL4nxf316sjlXF0yn6qvHmmTDxhtX/mXcKXqT+5eQ91Ec S7lpLRPJUeDWn/2dQ0UVF11pT8OhVTkqEwS0QIDavFYz3ml8vx67ehIVgA8Lrt5G19Z1 R6Fw== MIME-Version: 1.0 X-Received: by 10.49.4.194 with SMTP id m2mr28036935qem.3.1370360830383; Tue, 04 Jun 2013 08:47:10 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.49.2.67 with HTTP; Tue, 4 Jun 2013 08:47:10 -0700 (PDT) In-Reply-To: <51A77AB5.9040302@mti-systems.com> References: <5190FB21.5080100@mti-systems.com> <201305301420.51606.mirja.kuehlewind@ikr.uni-stuttgart.de> <51A77AB5.9040302@mti-systems.com> Date: Tue, 4 Jun 2013 11:47:10 -0400 X-Google-Sender-Auth: _bhgjywWdtKxIIABkr002y5qSgA Message-ID: From: Jim Gettys To: Wesley Eddy Content-Type: multipart/alternative; boundary=047d7bb70dd68266c904de55feb7 Cc: Mirja Kuehlewind , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jun 2013 17:27:27 -0000 --047d7bb70dd68266c904de55feb7 Content-Type: text/plain; charset=ISO-8859-1 There are a number of things I want to see: 1) packet scheduling is as important as drop/mark algorithms: it is the two together that is such a win with fq_codel. So I, for one, would be *very* unhappy to see a charter that meant it was out of scope to discuss and document scheduling and flow queuing strategies. And packet scheduling can interact with drop mark algorithms. Optimal scheduling depends on where you are in the network (e.g. a host, versus home router, vs. ISP head end, etc.); documenting what makes the most sense where would likely help both implementors of software and hardware and operators of those networks. I've given some thought to the area, but until we have running code, it's hand waving, but probably won't be hand waving in a year or two. 2) many mark/drop algorithms can co-exist (and generating informational RFC's to help customers write RFP's is worth while): but so far, there is no universal "one size fits all" mark/drop algorithm that is optimally effective everywhere. Kathy and Van say a primary goal of CoDel was "first, do no harm", rather than "optimal under all circumstances". The goal was an algorithm that never hurt you and would always be safe to have enabled. CoDel's constants were chosen so that CoDel will work at the edge of the network with planetary length paths in the face of highly variable bandwidth. But CoDel inside a datacenter (unless you mess with its constants) is not effective in a timely fashion. People have feared tuneable algorithms such as (W)RED can hurt them, so what we have had for AQM is sometimes not present at all, or not enabled (even in nodes that it would help greatly). Note that one of Dave Taht's discoveries was that RED had been broken in the Linux source tree for 5 years and *no one had noticed*. Maybe someday we'll reach nirvana with a drop/mark algorithm that "just works" everywhere well all the time and can never hurt and always works perfectly, but that's a lofty goal left for research; in the meanwhile, helping people understand what to use where in the network and the tradeoffs is important. But if CoDel (as we believe) or some other algorithm really is "first, do no harm" without tuning, it has great value to be on by default just to prevent the disaster an operator not enabling any AQM at all; the world will never be worse than that, and usually be radically better. Anything that *requires* tuning of knobs to work is a not a candidate.... 3) Some algorithms may be hard to retrofit into existing platforms. For example, if you can't timestamp a packet, it's really hard to use CoDel, which is an algorithm easy to implement from scratch, but could be really hard retrofit in an existing hardware design. I don't want to discourage people doing things a lot better than nothing as a stop-gap, but people need to understand that it may be a second best solution. Again, advice as to what/where algorithms may be applicable will help designers and operators both. We face a 5-10 year transition here, even if we succeed this time. 4) There are "feagtures" or lack of features that today's hardware does that can/does make life for software very hard. Changes could make life much easier. For example, today's hardware makes it hard to do true head drop (which hardware assist could make trivial). Some have noticed that at low bandwidth we may adjust CoDel parameters to work around the lack of head drop and unaccounted for time in the driver's ring buffers (which CoDel needs), since Linux's qdisc's can't see the time in the rings. So there are a set of recommendations to hardware and OS designers that would help the state of the network: this simple head drop feature, packet pacing, making GSO/TSO less evil, and so on, war stories on all the places we've been working on debloating Linux to guide other OS developers, etc. This is really the other side of queue management: try to encourage traffic and systems that is less bursty and gentle to the network, so the AQM/scheduling algorithms don't have to work as hard. Such things should go someplace in the IETF, and if not here, where? Doing nothing anywhere is what I want to avoid. Best regards, Jim On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy wrote: > On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: > > > > But my point is, we should not only to standardize more and more AQM > > mechanisms because that's a large research area which maybe even should > be > > homed in the IRTF, but we need to find actually deployment strategies. > Having > > a working group on AQM will already help to make people aware of the > topic > > and maybe think about using AQM. But only standardizing more and more AQM > > queue, might end up in investing a lot of working that no-one will ever > use. > > > I suggest appending something like this to the charter: > > Many AQM algorithms have been proposed in academic literature, but > very few are widely implemented and deployed. A goal of the working > group is to produce recommendations that will actually be used, and > algorithms that will actually be implemented, deployed in equipment, > and enabled. Towards these ends, the group actively encourages > participation from operators and implementers, and will coordinate > with the IETF OPS area and other relevant parts of the IETF and > Internet community. Wider research and evaluation of AQM mechanisms > shall be coordinated with the IRTF/ICCRG, and significant > participation in this WG from the academic and research community is > highly desirable, when it is directly relevant to implementation and > deployment. > > > We will definitely need to get engagement from some of the operators > that participate in the IETF, and should solicit participation more > widely outside as well (e.g. advertise to NANOG list, bufferbloat > list, etc.) in order to attract operators and implementers that aren't > already aware of this activity. > > Would this help to address your concern? > > > -- > Wes Eddy > MTI Systems > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > --047d7bb70dd68266c904de55feb7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
The= re are a number of things I want to see:

1) packet scheduling is as important as drop/mark algorithms: it is the two= together that is such a win with fq_codel. =A0So I, for one, would be *ver= y* unhappy to see a charter that meant it was out of scope to discuss and d= ocument scheduling and flow queuing strategies. =A0And packet scheduling ca= n interact with drop mark algorithms.

Optimal scheduling depends on = where you are in the network (e.g. a host, versus home router, vs. ISP head= end, etc.); documenting what makes the most sense where would likely help = both implementors of software and hardware and operators of those networks.= =A0I've given some thought to the area, but until we have running code= , it's hand waving, but probably won't be hand waving in a year or = two.

2) many mark/drop algorithms c= an co-exist (and generating informational RFC's to help customers write= RFP's is worth while): but so far, there is no universal "one siz= e fits all" mark/drop algorithm that is optimally effective everywhere= . =A0Kathy and Van say a primary goal of CoDel was "first, do no harm&= quot;, rather than "optimal under all circumstances". =A0The goal= was an algorithm that never hurt you and would always be safe to have enab= led. CoDel's constants were chosen so that CoDel will work at the edge = of the network with planetary length paths in the face of highly variable b= andwidth.=A0

But CoDel inside a datacenter = (unless you mess with its constants) is not effective in a timely fashion.= =A0

People have feared tuneable al= gorithms such as (W)RED can hurt them, so what we have had for AQM is somet= imes not present at all, or not enabled (even in nodes that it would help g= reatly). =A0Note that one of Dave Taht's discoveries was that RED had b= een broken in the Linux source tree for 5 years and *no one had noticed*.

Maybe someday we'll reach = nirvana with a drop/mark algorithm that "just works" everywhere w= ell all the time and can never hurt and always works perfectly, but that= 9;s a lofty goal left for research; in the meanwhile, helping people unders= tand what to use where in the network and the tradeoffs is important.

But if CoDel (as we believe) o= r some other algorithm really is "first, do no harm" without tuni= ng, =A0it has great value to be on by default just to prevent the disaster = an operator not enabling any AQM at all; the world will never be worse than= that, and usually be radically better. Anything that *requires* tuning of = knobs to work is a not a candidate....

3) Some algorithms may be hard= to retrofit into existing platforms. =A0For example, if you can't time= stamp a packet, it's really hard to use CoDel, which is an algorithm ea= sy to implement from scratch, but could be really hard retrofit in an exist= ing hardware design. I don't want to discourage people doing things a l= ot better than nothing as a stop-gap, but people need to understand that it= may be a second best solution.

Again, advice as to what/where= algorithms may be applicable will help designers and operators both. =A0We= face a 5-10 year transition here, even if we succeed this time.

4) There are "feagtures&q= uot; or lack of features that today's hardware does that can/does make = life for software very hard. Changes could make life much easier.=A0

For example, today's hardw= are makes it hard to do true head drop (which hardware assist could make tr= ivial). =A0Some have noticed that at low bandwidth we may adjust CoDel para= meters to work around the lack of head drop and unaccounted for time in the= driver's ring buffers (which CoDel needs), since Linux's qdisc'= ;s can't see the time in the rings. So there are a set of recommendatio= ns to hardware and OS designers that would help the state of the network: t= his simple head drop feature, packet pacing, making GSO/TSO less evil, and = so on, war stories on all the places we've been working on debloating L= inux to guide other OS developers, etc.

This is really the other side = of queue management: try to encourage traffic and systems that is less burs= ty and gentle to the network, so the AQM/scheduling algorithms don't ha= ve to work as hard.

Such things should go someplac= e in the IETF, and if not here, where? =A0Doing nothing anywhere is what I = want to avoid.

Best regards,
=A0 =A0 Jim



On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy <wes@mti-systems.com> wrote:
On 5/30/2013 8:20 AM, Mirj= a Kuehlewind wrote:
>
> But my point is, we should not only to standardize more and more AQM > mechanisms because that's a large research area which maybe even s= hould be
> homed in the IRTF, but we need to find actually deployment strategies.= Having
> a working group on AQM will already help to make people aware of the t= opic
> and maybe think about using AQM. But only standardizing more and more = AQM
> queue, might end up in investing a lot of working that no-one will eve= r use.


I suggest appending something like this to the charter:

=A0 Many AQM algorithms have been proposed in academic literature, but
=A0 very few are widely implemented and deployed. =A0A goal of the working<= br> =A0 group is to produce recommendations that will actually be used, and
=A0 algorithms that will actually be implemented, deployed in equipment, =A0 and enabled. =A0Towards these ends, the group actively encourages
=A0 participation from operators and implementers, and will coordinate
=A0 with the IETF OPS area and other relevant parts of the IETF and
=A0 Internet community. =A0Wider research and evaluation of AQM mechanisms<= br> =A0 shall be coordinated with the IRTF/ICCRG, and significant
=A0 participation in this WG from the academic and research community is =A0 highly desirable, when it is directly relevant to implementation and =A0 deployment.


We will definitely need to get engagement from some of the operators
that participate in the IETF, and should solicit participation more
widely outside as well (e.g. advertise to NANOG list, bufferbloat
list, etc.) in order to attract operators and implementers that aren't<= br> already aware of this activity.

Would this help to address your concern?


--
Wes Eddy
MTI Systems
_____________________________= __________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm

--047d7bb70dd68266c904de55feb7-- From fred@cisco.com Tue Jun 4 15:17:24 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79F5021F9990 for ; Tue, 4 Jun 2013 15:17:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.486 X-Spam-Level: X-Spam-Status: No, score=-110.486 tagged_above=-999 required=5 tests=[AWL=-0.114, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SmvrHa4ejQBv for ; Tue, 4 Jun 2013 15:17:19 -0700 (PDT) Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76]) by ietfa.amsl.com (Postfix) with ESMTP id 92B4D21F99A4 for ; Tue, 4 Jun 2013 15:17:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3359; q=dns/txt; s=iport; t=1370384239; x=1371593839; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=QlYBYz08uRKELdxNDn9PYD9tAu9uC/3de2hRzTtY3zc=; b=Nr2qBevjsG2dQBRdhaFY+hOCpM04pYLyd/fSAkOhJrO/UvhMBGcVZE29 MYrm2u5cd+C6NBBXRdndPLscUbnPeuXYtArvN3g7DsVRxoa7F5EGWG6o9 qRKCK71tZu/oPYrF1um7injYgtBVRwkykp93AQx6vs7ypf/5rzOb/8rJq s=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ag4FANxmrlGtJV2c/2dsb2JhbABaDoJ7ML8pfRZ0giMBAQEDAQEBAWsLBQsCAQgYCiQnCyUCBA4FCId/Bgy9WwSNVxWBBQIxB4J6YQOof4JRPoFoPw X-IronPort-AV: E=Sophos;i="4.87,802,1363132800"; d="scan'208";a="218817249" Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-5.cisco.com with ESMTP; 04 Jun 2013 22:17:19 +0000 Received: from xhc-rcd-x15.cisco.com (xhc-rcd-x15.cisco.com [173.37.183.89]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id r54MHIDa022434 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 4 Jun 2013 22:17:18 GMT Received: from xmb-rcd-x09.cisco.com ([169.254.9.220]) by xhc-rcd-x15.cisco.com ([173.37.183.89]) with mapi id 14.02.0318.004; Tue, 4 Jun 2013 17:17:18 -0500 From: "Fred Baker (fred)" To: grenville armitage Thread-Topic: [aqm] Question re draft-baker-aqm-recommendations recomendation #4 Thread-Index: AQHOYXFGJTwo/KmyTU+rweDV3ZdVtg== Date: Tue, 4 Jun 2013 22:17:17 +0000 Message-ID: <8C48B86A895913448548E6D15DA7553B903FEB@xmb-rcd-x09.cisco.com> References: <8C48B86A895913448548E6D15DA7553B850ECE@xmb-rcd-x09.cisco.com> <41E8D91E-658B-4B44-92D2-5EB0329781A5@ifi.uio.no> <8C48B86A895913448548E6D15DA7553B8512B5@xmb-rcd-x09.cisco.com> <20130507133724.GY23227@verdi> <51916F7F.1020601@swin.edu.au> In-Reply-To: <51916F7F.1020601@swin.edu.au> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.79.96.32] Content-Type: text/plain; charset="Windows-1252" Content-ID: <88CBBD89671B9945A979668FC14AC5DF@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "" Subject: Re: [aqm] Question re draft-baker-aqm-recommendations recomendation #4 X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jun 2013 22:17:24 -0000 On May 14, 2013, at 6:55 AM, grenville armitage wro= te: >=20 >=20 > On 05/07/2013 23:37, John Leslie wrote: >> Fred Baker (fred) wrote: >>> On May 6, 2013, at 7:25 AM, Michael Welzl wrote: >>>=20 >>>> ...sounds as if it would be the most normal thing in the world for an >>>> AQM algorithm to make a decision based on the transport protocol, >>>> which I think it shouldn't... >>>=20 >>> I certainly agree that we should not be making transport-specific >>> modifications... >>> My point in this, if you can think of a better way to phrase it, is >>> that the AQM algorithm someone implements needs to demonstrably work >>> with the transports and applications it will be affecting. >>=20 >> Hearing no differing suggestions, those words seem appropriate. Thus >> Section 4.4 might become: >> " >> " 4.4. Active Queue Management algorithms deployed SHOULD be effective o= n >> " all common Internet traffic >> " >> " Active Queue Management algorithms typically are verified to work with >> " TCP [RFC0793] and a limited number of applications of it. This no >> " longer represents a sufficient selection of actual traffic. We have >> " significant use of UDP [RFC0768] in voice and video services, as well >> " as SCTP [RFC4960] and DCCP [RFC4340]. Hence, Active Queue Management >> " algorithms should demonstrably work with other transports as well as >> " TCP, and with a wide variety of applications. >=20 > I see a difference between the scope of Fred's "...demonstrably > work with the transports and applications it will be affecting" > and the scope of the section heading ("...be effective on all > common Internet traffic") or even "..as well as TCP, and with > a wide variety of applications." >=20 > I certainly agree that an AQM algorithm ought not necessarily > be transport-layer aware. But the wording above could lead to > a situation where acceptance of a future AQM scheme is derailed > because it doesn't "work" with some (yet to be defined) "wide > variety of applications". >=20 > I prefer the wiggle room allowed by "..the transports and > applications it will be affecting" rather than "...all common > Internet traffic". I'll accept the change of language. I need to post an updated version for t= he IETF meeting, and am working on that with Srinivasan Keshav. My biggest concern in the text is, as you say, it can't be limited to TCP. = It has to work with two RFCs at a minimum: https://tools.ietf.org/html/rfc3168 3168 The Addition of Explicit Congestion Notification (ECN) to IP. K. Ramakrishnan, S. Floyd, D. Black. September 2001. (Format: TXT=3D17096= 6 bytes) (Obsoletes RFC2481) (Updates RFC2003, RFC2474, RFC2401, RFC0793) (Updated by RFC4301, RFC6040) (Status: PROPOSED STANDARD) https://tools.ietf.org/html/rfc6679 6679 Explicit Congestion Notification (ECN) for RTP over UDP. M. Westerlund, I. Johansson, C. Perkins, P. O'Hanlon, K. Carlberg. August 2012. (Format: TXT=3D148560 bytes) (Status: PROPOSED STANDARD) > cheers, > gja >=20 > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm =95 Make things as simple as possible, but not simpler. Albert Einstein From ropan@cisco.com Wed Jun 5 00:09:29 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA8AE21F9A5A for ; Wed, 5 Jun 2013 00:09:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.371 X-Spam-Level: X-Spam-Status: No, score=-10.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N4M1j+h-5qWR for ; Wed, 5 Jun 2013 00:09:24 -0700 (PDT) Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) by ietfa.amsl.com (Postfix) with ESMTP id 9558121F9A4E for ; Wed, 5 Jun 2013 00:09:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=20337; q=dns/txt; s=iport; t=1370416164; x=1371625764; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=gbvhIYZ9tycwjUHoT/pi+sT69WaDT3FbsUK3mOG6/ug=; b=nCal8bpxcaVK+OSLuQhcKKnEGeJ5uNyxhAvVhoE0WraUYnRAF/wpc9rg lYiCxlot05QJTR759nU2C5e17tRCMODPizOrNdWo3qnDFusKcQF0nGDii 4A5QWpJT4905BeUZThymoLTC7KZam5O7bcWLul8822dTz61qD3mKkUhm8 s=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgQFAB/jrlGtJXHA/2dsb2JhbABagkVEML8qgQAWdIIjAQEBBAEBAWsLEgEIDgMDAQIBCh0uCxQJCAIEAQ0FCAwFAgOHbwy8ZASOcyANBAeCemEDk22VEoMPgic X-IronPort-AV: E=Sophos;i="4.87,804,1363132800"; d="scan'208,217";a="219012651" Received: from rcdn-core2-5.cisco.com ([173.37.113.192]) by rcdn-iport-4.cisco.com with ESMTP; 05 Jun 2013 07:09:23 +0000 Received: from xhc-aln-x09.cisco.com (xhc-aln-x09.cisco.com [173.36.12.83]) by rcdn-core2-5.cisco.com (8.14.5/8.14.5) with ESMTP id r5579N8M025341 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 5 Jun 2013 07:09:23 GMT Received: from xmb-aln-x15.cisco.com ([169.254.9.55]) by xhc-aln-x09.cisco.com ([173.36.12.83]) with mapi id 14.02.0318.004; Wed, 5 Jun 2013 02:09:23 -0500 From: "Rong Pan (ropan)" To: Jim Gettys , Wesley Eddy Thread-Topic: [aqm] floating a draft charter Thread-Index: AQHOT+e7sbTg87RfgUiCkEk4FI/R8JkeFEGAgABBDYCAB9RAAIAAjE4A Date: Wed, 5 Jun 2013 07:09:21 +0000 Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.2.1.120420 x-originating-ip: [10.21.119.205] Content-Type: multipart/alternative; boundary="_000_BF2CBE9A994B89438F7F3B9FD58C2B7F185CD31Cxmbalnx15ciscoc_" MIME-Version: 1.0 Cc: Mirja Kuehlewind , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2013 07:09:29 -0000 --_000_BF2CBE9A994B89438F7F3B9FD58C2B7F185CD31Cxmbalnx15ciscoc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I understand packet scheduling is important. However, most network devices = today treat these two issues separately: classed-based scheduling/queueing = and drop/mark algorithms. In class-based queueing, vendors differ in how ma= ny classes that they support: 2, 4 or 8. In drop/mark algorithm, vendors te= nd to implement both tail-drop and WRED. I think we can continue to treat these two issues separately as it has been= done in practice. We can prove the values of delay-based AQM and we can al= so prove the value of FQ. The goal of AQM is to give early congestion notif= ication before the buffer becomes full or the delay gets big. Scheduling al= gorithms determine bandwidth allocation and weighted fairness among differe= nt classes/flows. We don't have to mandate a particular scheduling algorith= m with a particular AQM scheme. Vendors shall be free to choose their best = performance-cost tradeoff point. I think it would be challenging to force v= endors to adopt a fixed set. Best Regards, Rong From: Jim Gettys > Date: Tuesday, June 4, 2013 8:47 AM To: Wesley Eddy > Cc: Mirja Kuehlewind >, "aqm@ietf.org" > Subject: Re: [aqm] floating a draft charter There are a number of things I want to see: 1) packet scheduling is as important as drop/mark algorithms: it is the two= together that is such a win with fq_codel. So I, for one, would be *very*= unhappy to see a charter that meant it was out of scope to discuss and doc= ument scheduling and flow queuing strategies. And packet scheduling can in= teract with drop mark algorithms. Optimal scheduling depends on where you are in the network (e.g. a host, ve= rsus home router, vs. ISP head end, etc.); documenting what makes the most = sense where would likely help both implementors of software and hardware an= d operators of those networks. I've given some thought to the area, but un= til we have running code, it's hand waving, but probably won't be hand wavi= ng in a year or two. 2) many mark/drop algorithms can co-exist (and generating informational RFC= 's to help customers write RFP's is worth while): but so far, there is no u= niversal "one size fits all" mark/drop algorithm that is optimally effectiv= e everywhere. Kathy and Van say a primary goal of CoDel was "first, do no = harm", rather than "optimal under all circumstances". The goal was an algo= rithm that never hurt you and would always be safe to have enabled. CoDel's= constants were chosen so that CoDel will work at the edge of the network w= ith planetary length paths in the face of highly variable bandwidth. But CoDel inside a datacenter (unless you mess with its constants) is not e= ffective in a timely fashion. People have feared tuneable algorithms such as (W)RED can hurt them, so wha= t we have had for AQM is sometimes not present at all, or not enabled (even= in nodes that it would help greatly). Note that one of Dave Taht's discov= eries was that RED had been broken in the Linux source tree for 5 years and= *no one had noticed*. Maybe someday we'll reach nirvana with a drop/mark algorithm that "just wor= ks" everywhere well all the time and can never hurt and always works perfec= tly, but that's a lofty goal left for research; in the meanwhile, helping p= eople understand what to use where in the network and the tradeoffs is impo= rtant. But if CoDel (as we believe) or some other algorithm really is "first, do n= o harm" without tuning, it has great value to be on by default just to pre= vent the disaster an operator not enabling any AQM at all; the world will n= ever be worse than that, and usually be radically better. Anything that *re= quires* tuning of knobs to work is a not a candidate.... 3) Some algorithms may be hard to retrofit into existing platforms. For ex= ample, if you can't timestamp a packet, it's really hard to use CoDel, whic= h is an algorithm easy to implement from scratch, but could be really hard = retrofit in an existing hardware design. I don't want to discourage people = doing things a lot better than nothing as a stop-gap, but people need to un= derstand that it may be a second best solution. Again, advice as to what/where algorithms may be applicable will help desig= ners and operators both. We face a 5-10 year transition here, even if we s= ucceed this time. 4) There are "feagtures" or lack of features that today's hardware does tha= t can/does make life for software very hard. Changes could make life much e= asier. For example, today's hardware makes it hard to do true head drop (which har= dware assist could make trivial). Some have noticed that at low bandwidth = we may adjust CoDel parameters to work around the lack of head drop and una= ccounted for time in the driver's ring buffers (which CoDel needs), since L= inux's qdisc's can't see the time in the rings. So there are a set of recom= mendations to hardware and OS designers that would help the state of the ne= twork: this simple head drop feature, packet pacing, making GSO/TSO less ev= il, and so on, war stories on all the places we've been working on debloati= ng Linux to guide other OS developers, etc. This is really the other side of queue management: try to encourage traffic= and systems that is less bursty and gentle to the network, so the AQM/sche= duling algorithms don't have to work as hard. Such things should go someplace in the IETF, and if not here, where? Doing= nothing anywhere is what I want to avoid. Best regards, Jim On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy > wrote: On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: > > But my point is, we should not only to standardize more and more AQM > mechanisms because that's a large research area which maybe even should b= e > homed in the IRTF, but we need to find actually deployment strategies. Ha= ving > a working group on AQM will already help to make people aware of the topi= c > and maybe think about using AQM. But only standardizing more and more AQM > queue, might end up in investing a lot of working that no-one will ever u= se. I suggest appending something like this to the charter: Many AQM algorithms have been proposed in academic literature, but very few are widely implemented and deployed. A goal of the working group is to produce recommendations that will actually be used, and algorithms that will actually be implemented, deployed in equipment, and enabled. Towards these ends, the group actively encourages participation from operators and implementers, and will coordinate with the IETF OPS area and other relevant parts of the IETF and Internet community. Wider research and evaluation of AQM mechanisms shall be coordinated with the IRTF/ICCRG, and significant participation in this WG from the academic and research community is highly desirable, when it is directly relevant to implementation and deployment. We will definitely need to get engagement from some of the operators that participate in the IETF, and should solicit participation more widely outside as well (e.g. advertise to NANOG list, bufferbloat list, etc.) in order to attract operators and implementers that aren't already aware of this activity. Would this help to address your concern? -- Wes Eddy MTI Systems _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm --_000_BF2CBE9A994B89438F7F3B9FD58C2B7F185CD31Cxmbalnx15ciscoc_ Content-Type: text/html; charset="us-ascii" Content-ID: <1EA423F69D922B4D925C15EC4C79D9C8@emea.cisco.com> Content-Transfer-Encoding: quoted-printable
I understand packet scheduling is important. However, most network dev= ices today treat these two issues separately: classed-based scheduling/queu= eing and drop/mark algorithms. In class-based queueing, vendors differ= in how many classes that they support: 2, 4 or 8. In drop/mark algorithm, vendors tend to implement both tail-dro= p and WRED. 

I think we can continue to treat these two issues separately as it has= been done in practice. We can prove the values of delay-based AQM and= we can also prove the value of FQ. The goal of AQM is to give early conges= tion notification before the buffer becomes full or the delay gets big. Scheduling algorithms determine bandwidth allo= cation and weighted fairness among different classes/flows. We don't have t= o mandate a particular scheduling algorithm with a particular AQM scheme. V= endors shall be free to choose their best performance-cost tradeoff point. I think it would be challenging to f= orce vendors to adopt a fixed set. 

Best Regards,

Rong 



From: Jim Gettys <jg@freedesktop.org>
Date: Tuesday, June 4, 2013 8:47 AM=
To: Wesley Eddy <wes@mti-systems.com>
Cc: Mirja Kuehlewind <mirja.kuehlewind@ikr.uni-stu= ttgart.de>, "aqm@ietf.org&q= uot; <aqm@ietf.org>
Subject: Re: [aqm] floating a draft= charter

There are a number o= f things I want to see:

1) packet scheduling= is as important as drop/mark algorithms: it is the two together that is su= ch a win with fq_codel.  So I, for one, would be *very* unhappy to see= a charter that meant it was out of scope to discuss and document scheduling and flow queuing strategies.  And = packet scheduling can interact with drop mark algorithms.

Optimal scheduling d= epends on where you are in the network (e.g. a host, versus home router, vs= . ISP head end, etc.); documenting what makes the most sense where would li= kely help both implementors of software and hardware and operators of those networks.  I've given some though= t to the area, but until we have running code, it's hand waving, but probab= ly won't be hand waving in a year or two.

2) many mark/drop al= gorithms can co-exist (and generating informational RFC's to help customers= write RFP's is worth while): but so far, there is no universal "one s= ize fits all" mark/drop algorithm that is optimally effective everywhere.  Kathy and Van say a primary goal of = CoDel was "first, do no harm", rather than "optimal under al= l circumstances".  The goal was an algorithm that never hurt you = and would always be safe to have enabled. CoDel's constants were chosen so that CoDel will work at the edge of the network with planetary l= ength paths in the face of highly variable bandwidth. 

But CoDel inside a d= atacenter (unless you mess with its constants) is not effective in a timely= fashion. 

People have feared t= uneable algorithms such as (W)RED can hurt them, so what we have had for AQ= M is sometimes not present at all, or not enabled (even in nodes that it wo= uld help greatly).  Note that one of Dave Taht's discoveries was that RED had been broken in the Linux source t= ree for 5 years and *no one had noticed*.

Maybe someday we'll = reach nirvana with a drop/mark algorithm that "just works" everyw= here well all the time and can never hurt and always works perfectly, but t= hat's a lofty goal left for research; in the meanwhile, helping people understand what to use where in the network and the tradeof= fs is important.

But if CoDel (as we = believe) or some other algorithm really is "first, do no harm" wi= thout tuning,  it has great value to be on by default just to prevent = the disaster an operator not enabling any AQM at all; the world will never be worse than that, and usually be radically better. = Anything that *requires* tuning of knobs to work is a not a candidate....

3) Some algorithms m= ay be hard to retrofit into existing platforms.  For example, if you c= an't timestamp a packet, it's really hard to use CoDel, which is an algorit= hm easy to implement from scratch, but could be really hard retrofit in an existing hardware design. I don't want to di= scourage people doing things a lot better than nothing as a stop-gap, but p= eople need to understand that it may be a second best solution.

Again, advice as to = what/where algorithms may be applicable will help designers and operators b= oth.  We face a 5-10 year transition here, even if we succeed this tim= e.

4) There are "f= eagtures" or lack of features that today's hardware does that can/does= make life for software very hard. Changes could make life much easier.&nbs= p;

For example, today's= hardware makes it hard to do true head drop (which hardware assist could m= ake trivial).  Some have noticed that at low bandwidth we may adjust C= oDel parameters to work around the lack of head drop and unaccounted for time in the driver's ring buffers (which CoD= el needs), since Linux's qdisc's can't see the time in the rings. So there = are a set of recommendations to hardware and OS designers that would help t= he state of the network: this simple head drop feature, packet pacing, making GSO/TSO less evil, and so on, war= stories on all the places we've been working on debloating Linux to guide = other OS developers, etc.

This is really the o= ther side of queue management: try to encourage traffic and systems that is= less bursty and gentle to the network, so the AQM/scheduling algorithms do= n't have to work as hard.

Such things should g= o someplace in the IETF, and if not here, where?  Doing nothing anywhe= re is what I want to avoid.

Best regards,
    Jim



On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy <wes@mti-system= s.com> wrote:
On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote:
>
> But my point is, we should not only to standardize more and more AQM > mechanisms because that's a large research area which maybe even shoul= d be
> homed in the IRTF, but we need to find actually deployment strategies.= Having
> a working group on AQM will already help to make people aware of the t= opic
> and maybe think about using AQM. But only standardizing more and more = AQM
> queue, might end up in investing a lot of working that no-one will eve= r use.


I suggest appending something like this to the charter:

  Many AQM algorithms have been proposed in academic literature, but   very few are widely implemented and deployed.  A goal of the wo= rking
  group is to produce recommendations that will actually be used, and<= br>   algorithms that will actually be implemented, deployed in equipment,=
  and enabled.  Towards these ends, the group actively encourages=
  participation from operators and implementers, and will coordinate   with the IETF OPS area and other relevant parts of the IETF and
  Internet community.  Wider research and evaluation of AQM mecha= nisms
  shall be coordinated with the IRTF/ICCRG, and significant
  participation in this WG from the academic and research community is=
  highly desirable, when it is directly relevant to implementation and=
  deployment.


We will definitely need to get engagement from some of the operators
that participate in the IETF, and should solicit participation more
widely outside as well (e.g. advertise to NANOG list, bufferbloat
list, etc.) in order to attract operators and implementers that aren't
already aware of this activity.

Would this help to address your concern?


--
Wes Eddy
MTI Systems
_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm

--_000_BF2CBE9A994B89438F7F3B9FD58C2B7F185CD31Cxmbalnx15ciscoc_-- From dave.taht@gmail.com Wed Jun 5 00:44:06 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F7E621F9A97 for ; Wed, 5 Jun 2013 00:44:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.372 X-Spam-Level: X-Spam-Status: No, score=-2.372 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1j2f8AENzhZJ for ; Wed, 5 Jun 2013 00:44:04 -0700 (PDT) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) by ietfa.amsl.com (Postfix) with ESMTP id 1DFC121F9A2A for ; Wed, 5 Jun 2013 00:44:01 -0700 (PDT) Received: by mail-ie0-f170.google.com with SMTP id e14so2864208iej.15 for ; Wed, 05 Jun 2013 00:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UdrNiB3Hur2zo2ITQYoL2/B4X9zbmjm4Pb5OuJWLfao=; b=t0plKSiFAJhG8uE0FiN9O010irZ5ijhbIiD5ymk2lzv+cDXDEupRM5wKjtwOr9AnbV oYbuE3i2NloZbaTzdiprprWZIxQBkdjNXvJWKKrY1G+scy7VEOYvHgqdOzKodLfxoA5j CCtvnhrBEvp2rcple+AgS2R8Tqgy8GdJtk0F7BnyMZBdAx5JYP87w90tv8j3m1pI7P+h zFewE5G4fyK2SVQ9mEEyFW0oRb5te2gqEcU8enC9FSL/D5VsZ6EvLrq/KgSF3rAwfjv6 LiVFyyGTVrUbK+E2BjRb1O65fw3TtVvEUaKqYVB1p9B9WcXdo+tnLoUIfoWzNycKJ4fc aMnw== MIME-Version: 1.0 X-Received: by 10.50.118.69 with SMTP id kk5mr2585501igb.36.1370418240675; Wed, 05 Jun 2013 00:44:00 -0700 (PDT) Received: by 10.64.35.44 with HTTP; Wed, 5 Jun 2013 00:44:00 -0700 (PDT) In-Reply-To: References: Date: Wed, 5 Jun 2013 00:44:00 -0700 Message-ID: From: Dave Taht To: "Rong Pan (ropan)" Content-Type: multipart/alternative; boundary=e89a8f6433826de4d504de635ca7 Cc: Wesley Eddy , Mirja Kuehlewind , Jim Gettys , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2013 07:44:06 -0000 --e89a8f6433826de4d504de635ca7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Wed, Jun 5, 2013 at 12:09 AM, Rong Pan (ropan) wrote: > I understand packet scheduling is important. However, most network > devices today treat these two issues separately: classed-based > scheduling/queueing and drop/mark algorithms. In class-based queueing, > vendors differ in how many classes that they support: 2, 4 or 8. In > drop/mark algorithm, vendors tend to implement both tail-drop and WRED. > I really have a great deal of trouble extrapolating to "vendors" here. I suppose you mean "big iron vendors", like Juniper, or Allied Telesis, which haven't weighed in on this list yet. In the CMTS market there are three players, and while I know what Arris can do, I would prefer an Arris representative to say. In terms of chipset makers ethernet support, most don't don't support anything in hardware, neither do the low end bridge chips, and thus they rely on software to do either AQM or FQ, in addition to doing useful things like firewalling or NAT. In servers, home routers, and handhelds, linux based devices are prevailant and can easily do just about anything. (I recently had a theorist badger me about doing a particle model for scheduling packets, because the GPU is effectively free on many common chipsets. I'm sooo not going there, but he is). Wifi chipsets generally have no strategy other than 802.11e for managing queues. Most of the France Telecom's dsl devices run shortest queue first, with longest queue drop. The list goes on and on... > I think we can continue to treat these two issues separately as it has > been done in practice. > Scheduling and drop algorithms have long been used in combination in practice. Examples include the venerable wondershaper, and openwrt's QoS system, which for most of the last 7 years or so was HFSC + SFQ + RED. Dozens of other examples exist, but not on the gear you are familiar with. > We can prove the values of delay-based AQM and we can also prove the valu= e > of FQ. > Sure. Thing is, we've pretty much proven that the two in combination are better than either, singly. Do you not agree with that assessment? > The goal of AQM is to give early congestion notification before the buffe= r > becomes full or the delay gets big. > If you want to define AQM as Active Queue LENGTH management, then go right ahead and try. Actually I was thinking hard about expanding the scope of this proposed working group to be "Smart Queue Management" - which neatly not only encompasses FQ and queue length management, but also extends out to nagging problems like sanely rate limiting ipv6 ICMP responses to someone probing someone's entire /64 address space, and dealing with other sorts of attack like that, and problems like incast. Similarly, the problem of dealing with diffserv marking remains, as to how (or if) it would correctly interact with a SQM system. Things like iptables don't seem to have a home within the ietf presently (?= ) Scheduling algorithms determine bandwidth allocation and weighted fairness > among different classes/flows. We don't have to mandate a particular > scheduling algorithm with a particular AQM scheme. > Concur with that very much. I continue to experiment with QFQ+, DRR, WFQ, SFQ, and other techniques. > Vendors shall be free to choose their best performance-cost tradeoff > point. I think it would be challenging to force vendors to adopt a fixed > set. > Concur. Let the best set of ideas win both here and in the marketplace! But an end goal should be some effective form of SQM *on by default*, on everything. > Best Regards, > > Rong > > > > From: Jim Gettys > Date: Tuesday, June 4, 2013 8:47 AM > To: Wesley Eddy > Cc: Mirja Kuehlewind , " > aqm@ietf.org" > > Subject: Re: [aqm] floating a draft charter > > There are a number of things I want to see: > > 1) packet scheduling is as important as drop/mark algorithms: it is the > two together that is such a win with fq_codel. So I, for one, would be > *very* unhappy to see a charter that meant it was out of scope to discuss > and document scheduling and flow queuing strategies. And packet scheduli= ng > can interact with drop mark algorithms. > > Optimal scheduling depends on where you are in the network (e.g. a host, > versus home router, vs. ISP head end, etc.); documenting what makes the > most sense where would likely help both implementors of software and > hardware and operators of those networks. I've given some thought to the > area, but until we have running code, it's hand waving, but probably won'= t > be hand waving in a year or two. > > 2) many mark/drop algorithms can co-exist (and generating informational > RFC's to help customers write RFP's is worth while): but so far, there is > no universal "one size fits all" mark/drop algorithm that is optimally > effective everywhere. Kathy and Van say a primary goal of CoDel was > "first, do no harm", rather than "optimal under all circumstances". The > goal was an algorithm that never hurt you and would always be safe to hav= e > enabled. CoDel's constants were chosen so that CoDel will work at the edg= e > of the network with planetary length paths in the face of highly variable > bandwidth. > > But CoDel inside a datacenter (unless you mess with its constants) is > not effective in a timely fashion. > > People have feared tuneable algorithms such as (W)RED can hurt them, so > what we have had for AQM is sometimes not present at all, or not enabled > (even in nodes that it would help greatly). Note that one of Dave Taht's > discoveries was that RED had been broken in the Linux source tree for 5 > years and *no one had noticed*. > > Maybe someday we'll reach nirvana with a drop/mark algorithm that "just > works" everywhere well all the time and can never hurt and always works > perfectly, but that's a lofty goal left for research; in the meanwhile, > helping people understand what to use where in the network and the > tradeoffs is important. > > But if CoDel (as we believe) or some other algorithm really is "first, > do no harm" without tuning, it has great value to be on by default just = to > prevent the disaster an operator not enabling any AQM at all; the world > will never be worse than that, and usually be radically better. Anything > that *requires* tuning of knobs to work is a not a candidate.... > > 3) Some algorithms may be hard to retrofit into existing platforms. For > example, if you can't timestamp a packet, it's really hard to use CoDel, > which is an algorithm easy to implement from scratch, but could be really > hard retrofit in an existing hardware design. I don't want to discourage > people doing things a lot better than nothing as a stop-gap, but people > need to understand that it may be a second best solution. > > Again, advice as to what/where algorithms may be applicable will help > designers and operators both. We face a 5-10 year transition here, even = if > we succeed this time. > > 4) There are "feagtures" or lack of features that today's hardware does > that can/does make life for software very hard. Changes could make life > much easier. > > For example, today's hardware makes it hard to do true head drop (which > hardware assist could make trivial). Some have noticed that at low > bandwidth we may adjust CoDel parameters to work around the lack of head > drop and unaccounted for time in the driver's ring buffers (which CoDel > needs), since Linux's qdisc's can't see the time in the rings. So there a= re > a set of recommendations to hardware and OS designers that would help the > state of the network: this simple head drop feature, packet pacing, makin= g > GSO/TSO less evil, and so on, war stories on all the places we've been > working on debloating Linux to guide other OS developers, etc. > > This is really the other side of queue management: try to encourage > traffic and systems that is less bursty and gentle to the network, so the > AQM/scheduling algorithms don't have to work as hard. > > Such things should go someplace in the IETF, and if not here, where? > Doing nothing anywhere is what I want to avoid. > > Best regards, > Jim > > > > On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy wrote= : > >> On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: >> > >> > But my point is, we should not only to standardize more and more AQM >> > mechanisms because that's a large research area which maybe even shoul= d >> be >> > homed in the IRTF, but we need to find actually deployment strategies. >> Having >> > a working group on AQM will already help to make people aware of the >> topic >> > and maybe think about using AQM. But only standardizing more and more >> AQM >> > queue, might end up in investing a lot of working that no-one will eve= r >> use. >> >> >> I suggest appending something like this to the charter: >> >> Many AQM algorithms have been proposed in academic literature, but >> very few are widely implemented and deployed. A goal of the working >> group is to produce recommendations that will actually be used, and >> algorithms that will actually be implemented, deployed in equipment, >> and enabled. Towards these ends, the group actively encourages >> participation from operators and implementers, and will coordinate >> with the IETF OPS area and other relevant parts of the IETF and >> Internet community. Wider research and evaluation of AQM mechanisms >> shall be coordinated with the IRTF/ICCRG, and significant >> participation in this WG from the academic and research community is >> highly desirable, when it is directly relevant to implementation and >> deployment. >> >> >> We will definitely need to get engagement from some of the operators >> that participate in the IETF, and should solicit participation more >> widely outside as well (e.g. advertise to NANOG list, bufferbloat >> list, etc.) in order to attract operators and implementers that aren't >> already aware of this activity. >> >> Would this help to address your concern? >> >> >> -- >> Wes Eddy >> MTI Systems >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm >> > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html --e89a8f6433826de4d504de635ca7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Wed, Jun 5, 2013 at 12:09 AM, Rong Pa= n (ropan) <ropan@cisco.com> wrote:
I understand packet scheduling is important. However, most network dev= ices today treat these two issues separately: classed-based scheduling/queu= eing and drop/mark algorithms.=A0In class-based queueing, vendors differ in= how many classes that they support: 2, 4 or 8. In drop/mark algorithm, vendors tend to implement both tail-dro= p and WRED.=A0

I really have a great deal = of trouble extrapolating to "vendors" here. I suppose you mean &q= uot;big iron vendors", like Juniper, or Allied Telesis, which haven= 9;t weighed in on this list yet. In the CMTS market there are three players= , and while I know what Arris can do, I would prefer an Arris representativ= e to say. In terms of chipset makers ethernet support, most don't don&#= 39;t support anything in hardware, neither do the low end bridge chips, and= thus they rely on software to do either AQM or FQ, in addition to doing us= eful things like firewalling or NAT. In servers, home routers, and handheld= s, linux based devices are prevailant and can easily do just about anything= . (I recently had a theorist badger me about doing a particle model for sch= eduling packets, because the GPU is effectively free on many common chipset= s. I'm sooo not going there, but he is). Wifi chipsets generally have n= o strategy other than 802.11e for managing queues. Most of the France Telec= om's dsl devices run shortest queue first, with longest queue drop. The= list goes on and on...



I think we can continue to treat these two issues separately as it has= been done in practice.
=A0
=A0= Scheduling and drop algorithms have long been used in combination in practi= ce. Examples include the venerable wondershaper, and openwrt's QoS syst= em, which for most of the last 7 years or so was HFSC + SFQ + RED. Dozens o= f other examples exist, but not on the gear you are familiar with.
=A0
We can prove the va= lues of delay-based AQM and we can also prove the value of FQ.

Sure. Thing is, we've pretty much proven th= at the two in combination are better than either, singly. Do you not agree = with that assessment?
=A0
The goal of AQM is to give early congestion notification before= the buffer becomes full or the delay gets big.

If you want = to define AQM as Active Queue LENGTH management, then go right ahead and tr= y. Actually I was thinking hard about expanding the scope of this proposed = working group to be "Smart Queue Management" - which neatly not o= nly encompasses FQ and queue length management, but also extends out to nag= ging problems like sanely rate limiting ipv6 ICMP responses to someone prob= ing someone's entire /64 address space, and dealing with other sorts of= attack like that, and problems like incast.

Similarly, the problem of dealing with diffserv marking remains, as to = how (or if) it would correctly interact with a SQM system.
=A0
Things= like iptables don't seem to have a home within the ietf presently (?)<= br>
Scheduling algorithms = determine bandwidth allocation and weighted fairness among different classe= s/flows. We don't have to mandate a particular scheduling algorithm wit= h a particular AQM scheme.

Concur with that very much. I continue to exper= iment with QFQ+, DRR, WFQ, SFQ, and other techniques.
=A0
Vendors shall be free to choose their best performance-cost tradeoff point. I think it would be challenging to f= orce vendors to adopt a fixed set.=A0

Conc= ur. Let the best set of ideas win both here and in the marketplace! But an = end goal should be some effective form of SQM *on by default*, on everythin= g.


Best Regards,

Rong=A0



From: Jim Gettys <jg@freedesktop.org>
Date: Tuesday, June 4, 2013 8:47 AM=
To: Wesley Eddy <wes@mti-systems.com>
Cc: Mirja Kuehlewind <mirja.kueh= lewind@ikr.uni-stuttgart.de>, "aqm@ietf.org" <aqm@ietf.org>

Subject: Re: [aqm] floating a draft= charter

There are a number o= f things I want to see:

1) packet scheduling= is as important as drop/mark algorithms: it is the two together that is su= ch a win with fq_codel. =A0So I, for one, would be *very* unhappy to see a = charter that meant it was out of scope to discuss and document scheduling and flow queuing strategies. =A0And pac= ket scheduling can interact with drop mark algorithms.

Optimal scheduling d= epends on where you are in the network (e.g. a host, versus home router, vs= . ISP head end, etc.); documenting what makes the most sense where would li= kely help both implementors of software and hardware and operators of those networks. =A0I've given some thoug= ht to the area, but until we have running code, it's hand waving, but p= robably won't be hand waving in a year or two.

2) many mark/drop al= gorithms can co-exist (and generating informational RFC's to help custo= mers write RFP's is worth while): but so far, there is no universal &qu= ot;one size fits all" mark/drop algorithm that is optimally effective everywhere. =A0Kathy and Van say a primary goal of CoD= el was "first, do no harm", rather than "optimal under all c= ircumstances". =A0The goal was an algorithm that never hurt you and wo= uld always be safe to have enabled. CoDel's constants were chosen so that CoDel will work at the edge of the network with planetary l= ength paths in the face of highly variable bandwidth.=A0

But CoDel inside a d= atacenter (unless you mess with its constants) is not effective in a timely= fashion.=A0

People have feared t= uneable algorithms such as (W)RED can hurt them, so what we have had for AQ= M is sometimes not present at all, or not enabled (even in nodes that it wo= uld help greatly). =A0Note that one of Dave Taht's discoveries was that RED had been broken in the Linux sour= ce tree for 5 years and *no one had noticed*.

Maybe someday we'= ;ll reach nirvana with a drop/mark algorithm that "just works" ev= erywhere well all the time and can never hurt and always works perfectly, b= ut that's a lofty goal left for research; in the meanwhile, helping people understand what to use where in the network and the tradeof= fs is important.

But if CoDel (as we = believe) or some other algorithm really is "first, do no harm" wi= thout tuning, =A0it has great value to be on by default just to prevent the= disaster an operator not enabling any AQM at all; the world will never be worse than that, and usually be radically better. = Anything that *requires* tuning of knobs to work is a not a candidate....

3) Some algorithms m= ay be hard to retrofit into existing platforms. =A0For example, if you can&= #39;t timestamp a packet, it's really hard to use CoDel, which is an al= gorithm easy to implement from scratch, but could be really hard retrofit in an existing hardware design. I don't want t= o discourage people doing things a lot better than nothing as a stop-gap, b= ut people need to understand that it may be a second best solution.

Again, advice as to = what/where algorithms may be applicable will help designers and operators b= oth. =A0We face a 5-10 year transition here, even if we succeed this time.<= /div>

4) There are "f= eagtures" or lack of features that today's hardware does that can/= does make life for software very hard. Changes could make life much easier.= =A0

For example, today&#= 39;s hardware makes it hard to do true head drop (which hardware assist cou= ld make trivial). =A0Some have noticed that at low bandwidth we may adjust = CoDel parameters to work around the lack of head drop and unaccounted for time in the driver's ring buffers (which= CoDel needs), since Linux's qdisc's can't see the time in the = rings. So there are a set of recommendations to hardware and OS designers t= hat would help the state of the network: this simple head drop feature, packet pacing, making GSO/TSO less evil, and so on, war= stories on all the places we've been working on debloating Linux to gu= ide other OS developers, etc.

This is really the o= ther side of queue management: try to encourage traffic and systems that is= less bursty and gentle to the network, so the AQM/scheduling algorithms do= n't have to work as hard.

Such things should g= o someplace in the IETF, and if not here, where? =A0Doing nothing anywhere = is what I want to avoid.

Best regards,
=A0 =A0 Jim



On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy <wes@mti-system= s.com> wrote:
On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote:
>
> But my point is, we should not only to standardize more and more AQM > mechanisms because that's a large research area which maybe even s= hould be
> homed in the IRTF, but we need to find actually deployment strategies.= Having
> a working group on AQM will already help to make people aware of the t= opic
> and maybe think about using AQM. But only standardizing more and more = AQM
> queue, might end up in investing a lot of working that no-one will eve= r use.


I suggest appending something like this to the charter:

=A0 Many AQM algorithms have been proposed in academic literature, but
=A0 very few are widely implemented and deployed. =A0A goal of the working<= br> =A0 group is to produce recommendations that will actually be used, and
=A0 algorithms that will actually be implemented, deployed in equipment, =A0 and enabled. =A0Towards these ends, the group actively encourages
=A0 participation from operators and implementers, and will coordinate
=A0 with the IETF OPS area and other relevant parts of the IETF and
=A0 Internet community. =A0Wider research and evaluation of AQM mechanisms<= br> =A0 shall be coordinated with the IRTF/ICCRG, and significant
=A0 participation in this WG from the academic and research community is =A0 highly desirable, when it is directly relevant to implementation and =A0 deployment.


We will definitely need to get engagement from some of the operators
that participate in the IETF, and should solicit participation more
widely outside as well (e.g. advertise to NANOG list, bufferbloat
list, etc.) in order to attract operators and implementers that aren't<= br> already aware of this activity.

Would this help to address your concern?


--
Wes Eddy
MTI Systems
_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm


_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm




--
Dave T=E4ht

= Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscrib= e.html=20 --e89a8f6433826de4d504de635ca7-- From andrewmcgr@google.com Wed Jun 5 01:28:15 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C6F621F99FF for ; Wed, 5 Jun 2013 01:28:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.863 X-Spam-Level: X-Spam-Status: No, score=-2.863 tagged_above=-999 required=5 tests=[AWL=-0.113, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UmLUiVvCyAcs for ; Wed, 5 Jun 2013 01:28:10 -0700 (PDT) Received: from mail-vc0-f173.google.com (mail-vc0-f173.google.com [209.85.220.173]) by ietfa.amsl.com (Postfix) with ESMTP id 6D60921F997B for ; Wed, 5 Jun 2013 01:28:07 -0700 (PDT) Received: by mail-vc0-f173.google.com with SMTP id ht11so904535vcb.4 for ; Wed, 05 Jun 2013 01:28:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lCHb5u01UqkmLPG5frDi8hdnrhoiF63K4vv/teG1nOU=; b=EuTVVusDrRMolk4Jw0MqyYYiEyQT9VrGhJ4Qrq/4OJpHho2up1wrWRI3iQ0JhyuT8I 1vMnWPblSPAdxB4Gj/i1BqwVzY/ebDOTnXfWCsYdDFWxvKnv4amFjLcoaWCaQSaUQGEQ BVtW8oUVA1UTMSlbkY8bC2HZKQDX8DZbUDkPqVRfNL77oDY9QdVMKbYxZJK/VsL3DAQU TsdR/8RegdsGWj6N61Z3HBEEhT6EaC22lJuaUzdtoOvVPgbo2BTWkxq0PSyiUsNDGuPJ 9VkSuxUbtJ5k3YyfE1eysO1dhYQEDbID/sHcb+oluHZne9/KgZa649BLLz87pmTLg1WS SU2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=lCHb5u01UqkmLPG5frDi8hdnrhoiF63K4vv/teG1nOU=; b=YTznYxjIQ+8xDyZwIrXMl7hBoZpU0SVY+E1bPZfQuH1u7YHOrh8OSgM/97aoyjjpfn 8nhX4D8Pkl6VBzReV7eA0Uc97Y3yWzE0kbvSkailroJBpr0Dhsc7dCBwxl2lr+jAAt1G Cj74x8MWPaRYu9zCE1SnsWN47qvJPRIDIQ9L3CEr5ylDMCRXYmoMiRDDHV3HccZKVmKF rjfUPkoMr3t9GJiOFPwpNeB56Q0FfAU6NAtKD3MI77hwtXj4ny6/FHiXKpgMiHUglKAl O/7gFmo7XyOlsTyGnImF0SBYclMm2Zvl+7PvxFbvq40hNiw1tht8eh+Txa+UHKNd+6R9 sjfA== MIME-Version: 1.0 X-Received: by 10.221.4.131 with SMTP id oc3mr19848713vcb.49.1370420884372; Wed, 05 Jun 2013 01:28:04 -0700 (PDT) Received: by 10.220.209.20 with HTTP; Wed, 5 Jun 2013 01:28:04 -0700 (PDT) In-Reply-To: References: Date: Wed, 5 Jun 2013 18:28:04 +1000 Message-ID: From: Andrew Mcgregor To: "Rong Pan (ropan)" Content-Type: multipart/alternative; boundary=089e01293fe201a50804de63fad6 X-Gm-Message-State: ALoCoQnVJ8q91+FR7IT+G+LTvg+RvxR6TRbNDBdmw30Q5pHu9HHeFIXePS+lSG9hY+x/YQ+tb+sp/2UZ08V+145lP2ae8sZie24BPMlzGxO0g8TXOjF1K8b2JjltJSeROoyVjMRlAB5U2CGAH79+UY4TDt0fm+EDx7sqbVhB7lCoFw6BFQSXFO0GaztQPEapuHC5uY86h9lO Cc: Wesley Eddy , Mirja Kuehlewind , Jim Gettys , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2013 08:28:15 -0000 --089e01293fe201a50804de63fad6 Content-Type: text/plain; charset=ISO-8859-1 On 5 June 2013 17:09, Rong Pan (ropan) wrote: > I understand packet scheduling is important. However, most network > devices today treat these two issues separately: classed-based > scheduling/queueing and drop/mark algorithms. In class-based queueing, > vendors differ in how many classes that they support: 2, 4 or 8. In > drop/mark algorithm, vendors tend to implement both tail-drop and WRED. > I'm aware of chips, not including software forwarding, with 1, 2, 4, 5, 6, 8, 9, 16, 64 and arbitrary numbers of classes limited by available memory, with strict- or soft-priority, DRR, WRR, and drop/mark algorithms including tail-drop, head-drop, WRED, priority- or class- weighted buffer pressure management with head or tail eviction, and that's without using rate ACL mechanisms. So even restricting the scope to ethernet switch chips there is far more variety than you're suggesting. Software can of course do nearly anything, and given that these days software forwarding is slightly cheaper than silicon forwarding even up to around 32x10G levels, it is entirely realistic to expect to run software forwarding in many parts of the network. > I think we can continue to treat these two issues separately as it has > been done in practice. > Any form of buffer pressure management that takes classes into account couples these, and two widely deployed switch silicon families that I know of have that. That amounts to weak AQM. > We can prove the values of delay-based AQM and we can also prove the value > of FQ. The goal of AQM is to give early congestion notification before the > buffer becomes full or the delay gets big. Scheduling algorithms determine > bandwidth allocation and weighted fairness among different classes/flows. > We don't have to mandate a particular scheduling algorithm with a > particular AQM scheme. Vendors shall be free to choose their best > performance-cost tradeoff point. I think it would be challenging to force > vendors to adopt a fixed set. > > Best Regards, > > Rong > > More than challenging, that's impossible. Not to mention completely undesirable, because we're well aware that anything we have right now is imperfect. Andrew > > > From: Jim Gettys > Date: Tuesday, June 4, 2013 8:47 AM > To: Wesley Eddy > Cc: Mirja Kuehlewind , " > aqm@ietf.org" > > Subject: Re: [aqm] floating a draft charter > > There are a number of things I want to see: > > 1) packet scheduling is as important as drop/mark algorithms: it is the > two together that is such a win with fq_codel. So I, for one, would be > *very* unhappy to see a charter that meant it was out of scope to discuss > and document scheduling and flow queuing strategies. And packet scheduling > can interact with drop mark algorithms. > > Optimal scheduling depends on where you are in the network (e.g. a host, > versus home router, vs. ISP head end, etc.); documenting what makes the > most sense where would likely help both implementors of software and > hardware and operators of those networks. I've given some thought to the > area, but until we have running code, it's hand waving, but probably won't > be hand waving in a year or two. > > 2) many mark/drop algorithms can co-exist (and generating informational > RFC's to help customers write RFP's is worth while): but so far, there is > no universal "one size fits all" mark/drop algorithm that is optimally > effective everywhere. Kathy and Van say a primary goal of CoDel was > "first, do no harm", rather than "optimal under all circumstances". The > goal was an algorithm that never hurt you and would always be safe to have > enabled. CoDel's constants were chosen so that CoDel will work at the edge > of the network with planetary length paths in the face of highly variable > bandwidth. > > But CoDel inside a datacenter (unless you mess with its constants) is > not effective in a timely fashion. > > People have feared tuneable algorithms such as (W)RED can hurt them, so > what we have had for AQM is sometimes not present at all, or not enabled > (even in nodes that it would help greatly). Note that one of Dave Taht's > discoveries was that RED had been broken in the Linux source tree for 5 > years and *no one had noticed*. > > Maybe someday we'll reach nirvana with a drop/mark algorithm that "just > works" everywhere well all the time and can never hurt and always works > perfectly, but that's a lofty goal left for research; in the meanwhile, > helping people understand what to use where in the network and the > tradeoffs is important. > > But if CoDel (as we believe) or some other algorithm really is "first, > do no harm" without tuning, it has great value to be on by default just to > prevent the disaster an operator not enabling any AQM at all; the world > will never be worse than that, and usually be radically better. Anything > that *requires* tuning of knobs to work is a not a candidate.... > > 3) Some algorithms may be hard to retrofit into existing platforms. For > example, if you can't timestamp a packet, it's really hard to use CoDel, > which is an algorithm easy to implement from scratch, but could be really > hard retrofit in an existing hardware design. I don't want to discourage > people doing things a lot better than nothing as a stop-gap, but people > need to understand that it may be a second best solution. > > Again, advice as to what/where algorithms may be applicable will help > designers and operators both. We face a 5-10 year transition here, even if > we succeed this time. > > 4) There are "feagtures" or lack of features that today's hardware does > that can/does make life for software very hard. Changes could make life > much easier. > > For example, today's hardware makes it hard to do true head drop (which > hardware assist could make trivial). Some have noticed that at low > bandwidth we may adjust CoDel parameters to work around the lack of head > drop and unaccounted for time in the driver's ring buffers (which CoDel > needs), since Linux's qdisc's can't see the time in the rings. So there are > a set of recommendations to hardware and OS designers that would help the > state of the network: this simple head drop feature, packet pacing, making > GSO/TSO less evil, and so on, war stories on all the places we've been > working on debloating Linux to guide other OS developers, etc. > > This is really the other side of queue management: try to encourage > traffic and systems that is less bursty and gentle to the network, so the > AQM/scheduling algorithms don't have to work as hard. > > Such things should go someplace in the IETF, and if not here, where? > Doing nothing anywhere is what I want to avoid. > > Best regards, > Jim > > > > On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy wrote: > >> On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: >> > >> > But my point is, we should not only to standardize more and more AQM >> > mechanisms because that's a large research area which maybe even should >> be >> > homed in the IRTF, but we need to find actually deployment strategies. >> Having >> > a working group on AQM will already help to make people aware of the >> topic >> > and maybe think about using AQM. But only standardizing more and more >> AQM >> > queue, might end up in investing a lot of working that no-one will ever >> use. >> >> >> I suggest appending something like this to the charter: >> >> Many AQM algorithms have been proposed in academic literature, but >> very few are widely implemented and deployed. A goal of the working >> group is to produce recommendations that will actually be used, and >> algorithms that will actually be implemented, deployed in equipment, >> and enabled. Towards these ends, the group actively encourages >> participation from operators and implementers, and will coordinate >> with the IETF OPS area and other relevant parts of the IETF and >> Internet community. Wider research and evaluation of AQM mechanisms >> shall be coordinated with the IRTF/ICCRG, and significant >> participation in this WG from the academic and research community is >> highly desirable, when it is directly relevant to implementation and >> deployment. >> >> >> We will definitely need to get engagement from some of the operators >> that participate in the IETF, and should solicit participation more >> widely outside as well (e.g. advertise to NANOG list, bufferbloat >> list, etc.) in order to attract operators and implementers that aren't >> already aware of this activity. >> >> Would this help to address your concern? >> >> >> -- >> Wes Eddy >> MTI Systems >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm >> > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > -- Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128 --089e01293fe201a50804de63fad6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable



On 5 June 2013 17:09, Rong Pan (ropan) <ropan@cisco.com> wrote:
I understand packet scheduling is important. However, most network dev= ices today treat these two issues separately: classed-based scheduling/queu= eing and drop/mark algorithms.=A0In class-based queueing, vendors differ in= how many classes that they support: 2, 4 or 8. In drop/mark algorithm, vendors tend to implement both tail-dro= p and WRED.=A0

I'm aw= are of chips, not including software forwarding, with 1, 2, 4, 5, 6, 8, 9, = 16, 64 and arbitrary numbers of classes limited by available memory, with s= trict- or soft-priority, DRR, WRR, and drop/mark algorithms including tail-= drop, head-drop, WRED, priority- or class- weighted buffer pressure managem= ent with head or tail eviction, and that's without using rate ACL mecha= nisms. =A0So even restricting the scope to ethernet switch chips there is f= ar more variety than you're suggesting.

Software can of course do nearly anything, = and given that these days software forwarding is slightly cheaper than sili= con forwarding even up to around 32x10G levels, it is entirely realistic to= expect to run software forwarding in many parts of the network.
=A0
I think we can continue to treat these two issues separately as it has= been done in practice.=A0

Any form of buffer pressure management that takes classes into account co= uples these, and two widely deployed switch silicon families that I know of= have that. =A0That amounts to weak AQM.
=A0
We can prove the v= alues of delay-based AQM and we can also prove the value of FQ. The goal of= AQM is to give early congestion notification before the buffer becomes full or the delay gets big. Scheduling algorithms determine bandwidth allo= cation and weighted fairness among different classes/flows. We don't ha= ve to mandate a particular scheduling algorithm with a particular AQM schem= e. Vendors shall be free to choose their best performance-cost tradeoff point. I think it would be challenging to f= orce vendors to adopt a fixed set.=A0

Best Regards,

Rong=A0


More than chall= enging, that's impossible. =A0Not to mention completely undesirable, be= cause we're well aware that anything we have right now is imperfect.

Andrew
=A0


From: Jim Gettys <jg@freedesktop.org>
Date: Tuesday, June 4, 2013 8:47 AM=
To: Wesley Eddy <wes@mti-systems.com>
Cc: Mirja Kuehlewind <mirja.kueh= lewind@ikr.uni-stuttgart.de>, "aqm@ietf.org" <aqm@ietf.org>

Subject: Re: [aqm] floating a draft= charter

There are a number of things I want to see:<= /div>

1) packet scheduling is as important as drop= /mark algorithms: it is the two together that is such a win with fq_codel. = =A0So I, for one, would be *very* unhappy to see a charter that meant it wa= s out of scope to discuss and document scheduling and flow queuing strategies. =A0And pac= ket scheduling can interact with drop mark algorithms.

Optimal scheduling depends on where you are = in the network (e.g. a host, versus home router, vs. ISP head end, etc.); d= ocumenting what makes the most sense where would likely help both implement= ors of software and hardware and operators of those networks. =A0I've given some thoug= ht to the area, but until we have running code, it's hand waving, but p= robably won't be hand waving in a year or two.

2) many mark/drop algorithms can co-exist (a= nd generating informational RFC's to help customers write RFP's is = worth while): but so far, there is no universal "one size fits all&quo= t; mark/drop algorithm that is optimally effective everywhere. =A0Kathy and Van say a primary goal of CoD= el was "first, do no harm", rather than "optimal under all c= ircumstances". =A0The goal was an algorithm that never hurt you and wo= uld always be safe to have enabled. CoDel's constants were chosen so that CoDel will work at the edge of the network with planetary l= ength paths in the face of highly variable bandwidth.=A0

But CoDel inside a datacenter (unless you me= ss with its constants) is not effective in a timely fashion.=A0

People have feared tuneable algorithms such = as (W)RED can hurt them, so what we have had for AQM is sometimes not prese= nt at all, or not enabled (even in nodes that it would help greatly). =A0No= te that one of Dave Taht's discoveries was that RED had been broken in the Linux sour= ce tree for 5 years and *no one had noticed*.

Maybe someday we'll reach nirvana with a= drop/mark algorithm that "just works" everywhere well all the ti= me and can never hurt and always works perfectly, but that's a lofty go= al left for research; in the meanwhile, helping people understand what to use where in the network and the tradeof= fs is important.

But if CoDel (as we believe) or some other a= lgorithm really is "first, do no harm" without tuning, =A0it has = great value to be on by default just to prevent the disaster an operator no= t enabling any AQM at all; the world will never be worse than that, and usually be radically better. = Anything that *requires* tuning of knobs to work is a not a candidate....

3) Some algorithms may be hard to retrofit i= nto existing platforms. =A0For example, if you can't timestamp a packet= , it's really hard to use CoDel, which is an algorithm easy to implemen= t from scratch, but could be really hard retrofit in an existing hardware design. I don't want t= o discourage people doing things a lot better than nothing as a stop-gap, b= ut people need to understand that it may be a second best solution.

Again, advice as to what/where algorithms ma= y be applicable will help designers and operators both. =A0We face a 5-10 y= ear transition here, even if we succeed this time.

4) There are "feagtures" or lack o= f features that today's hardware does that can/does make life for softw= are very hard. Changes could make life much easier.=A0

For example, today's hardware makes it h= ard to do true head drop (which hardware assist could make trivial). =A0Som= e have noticed that at low bandwidth we may adjust CoDel parameters to work= around the lack of head drop and unaccounted for time in the driver's ring buffers (which= CoDel needs), since Linux's qdisc's can't see the time in the = rings. So there are a set of recommendations to hardware and OS designers t= hat would help the state of the network: this simple head drop feature, packet pacing, making GSO/TSO less evil, and so on, war= stories on all the places we've been working on debloating Linux to gu= ide other OS developers, etc.

This is really the other side of queue manag= ement: try to encourage traffic and systems that is less bursty and gentle = to the network, so the AQM/scheduling algorithms don't have to work as = hard.

Such things should go someplace in the IETF,= and if not here, where? =A0Doing nothing anywhere is what I want to avoid.=

Best regards,
=A0 =A0 Jim



On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy <wes@mti-system= s.com> wrote:
On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote:
>
> But my point is, we should not only to standardize more and more AQM > mechanisms because that's a large research area which maybe even s= hould be
> homed in the IRTF, but we need to find actually deployment strategies.= Having
> a working group on AQM will already help to make people aware of the t= opic
> and maybe think about using AQM. But only standardizing more and more = AQM
> queue, might end up in investing a lot of working that no-one will eve= r use.


I suggest appending something like this to the charter:

=A0 Many AQM algorithms have been proposed in academic literature, but
=A0 very few are widely implemented and deployed. =A0A goal of the working<= br> =A0 group is to produce recommendations that will actually be used, and
=A0 algorithms that will actually be implemented, deployed in equipment, =A0 and enabled. =A0Towards these ends, the group actively encourages
=A0 participation from operators and implementers, and will coordinate
=A0 with the IETF OPS area and other relevant parts of the IETF and
=A0 Internet community. =A0Wider research and evaluation of AQM mechanisms<= br> =A0 shall be coordinated with the IRTF/ICCRG, and significant
=A0 participation in this WG from the academic and research community is =A0 highly desirable, when it is directly relevant to implementation and =A0 deployment.


We will definitely need to get engagement from some of the operators
that participate in the IETF, and should solicit participation more
widely outside as well (e.g. advertise to NANOG list, bufferbloat
list, etc.) in order to attract operators and implementers that aren't<= br> already aware of this activity.

Would this help to address your concern?


--
Wes Eddy
MTI Systems
_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm


_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm




--
Andrew McGregor=A0|<= /span>=A0SRE=A0|=A0andrewmcgr@google.com=A0|=A0+61 4 8143 7128
--089e01293fe201a50804de63fad6-- From luca.muscariello@orange.com Wed Jun 5 01:45:02 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79CCD21F9A52 for ; Wed, 5 Jun 2013 01:45:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.021 X-Spam-Level: X-Spam-Status: No, score=-6.021 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_FR=0.35, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SjnjLcEtE+uy for ; Wed, 5 Jun 2013 01:44:58 -0700 (PDT) Received: from p-mail1.rd.orange.com (p-mail1.rd.orange.com [195.101.245.15]) by ietfa.amsl.com (Postfix) with ESMTP id 887CD21F9A1B for ; Wed, 5 Jun 2013 01:44:57 -0700 (PDT) Received: from p-mail1.rd.orange.com (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 2937441021B; Wed, 5 Jun 2013 10:44:56 +0200 (CEST) Received: from ftrdsmtp1.rd.francetelecom.fr (unknown [10.192.128.46]) by p-mail1.rd.orange.com (Postfix) with ESMTP id A6E66410223; Wed, 5 Jun 2013 10:44:55 +0200 (CEST) Received: from ftrdmel10.rd.francetelecom.fr ([10.192.128.44]) by ftrdsmtp1.rd.francetelecom.fr with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jun 2013 10:44:55 +0200 Received: from [10.193.161.45] ([10.193.161.45]) by ftrdmel10.rd.francetelecom.fr with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jun 2013 10:44:54 +0200 Message-ID: <51AEFA86.1070802@orange.com> Date: Wed, 05 Jun 2013 10:44:54 +0200 From: Luca MUSCARIELLO User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130510 Thunderbird/17.0.6 MIME-Version: 1.0 To: Dave Taht References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------040906070605000801050000" X-OriginalArrivalTime: 05 Jun 2013 08:44:54.0379 (UTC) FILETIME=[F3501BB0:01CE61C8] Cc: Wesley Eddy , "Rong Pan \(ropan\)" , Mirja Kuehlewind , Jim Gettys , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2013 08:45:02 -0000 This is a multi-part message in MIME format. --------------040906070605000801050000 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit AFAIK, Chipcos for home gateways (Broadcom, Ikanos, Intel), network processors (Cavium), but also Cisco, Juniper and Alcatel-Lucent make a difference between class based queuing and dropping algorithms. The first is implemented everywhere in different ways and I do not think there would be relevant work to do there (except for HTB in Linux which is really bad in terms of performance, IMHO). Dropping algorithms is used frequently as a term to describe AQM, e.g. in CLI based configuration systems you may find (in commercial Linux based products) SFQ among the dropping algorithms. It is clearly an abuse of language but common practice. I unfortunately agree with Dave, and we discussed about that in Paris, that chipcos do not take care about flow based queue management (typically withing a given class). Only Linux does that, but many products make use of Linux only for the slow path while the fast path is managed in hardware. Deploying QoS (including de-bloating in QoS) based on Linux is soon broken by the trend to distribute packet processing in dedicated processing units and chipco ignore completely queue management there. Anytime I ask whether the chip implements deficit round round robin the answer is "of course!" and in the end it turns out to be *modified* deficit round robin, i.e. class based queuing with up to say 64 queues. Which is not what I am looking for. One solution would be to move everything to software, but the cost of the chip that guarantees the same switching throughput is much higher. I would suggest to put more emphasis that something new is being proposed here adding flow based packet scheduling as a novel item in the charter (in addition to class based, dropping or congestion notification). Using Smart Queue Management seems misleading to me or not enough. Best regards, Luca -- France Telecom R&D - Orange Labs MUSCARIELLO Luca - NMP/TRM 38 - 40, rue du General Leclerc 92794 Issy Les Moulineaux Cedex 9 - France Tel : +33 (0)1 45 29 60 37 http://perso.rd.francetelecom.fr/muscariello On 06/05/2013 09:44 AM, Dave Taht wrote: > > > On Wed, Jun 5, 2013 at 12:09 AM, Rong Pan (ropan) > wrote: > > I understand packet scheduling is important. However, most network > devices today treat these two issues separately: classed-based > scheduling/queueing and drop/mark algorithms. In class-based > queueing, vendors differ in how many classes that they support: 2, > 4 or 8. In drop/mark algorithm, vendors tend to implement both > tail-drop and WRED. > > > I really have a great deal of trouble extrapolating to "vendors" here. > I suppose you mean "big iron vendors", like Juniper, or Allied > Telesis, which haven't weighed in on this list yet. In the CMTS market > there are three players, and while I know what Arris can do, I would > prefer an Arris representative to say. In terms of chipset makers > ethernet support, most don't don't support anything in hardware, > neither do the low end bridge chips, and thus they rely on software to > do either AQM or FQ, in addition to doing useful things like > firewalling or NAT. In servers, home routers, and handhelds, linux > based devices are prevailant and can easily do just about anything. (I > recently had a theorist badger me about doing a particle model for > scheduling packets, because the GPU is effectively free on many common > chipsets. I'm sooo not going there, but he is). Wifi chipsets > generally have no strategy other than 802.11e for managing queues. > Most of the France Telecom's dsl devices run shortest queue first, > with longest queue drop. The list goes on and on... > > > > I think we can continue to treat these two issues separately as it > has been done in practice. > > Scheduling and drop algorithms have long been used in combination in > practice. Examples include the venerable wondershaper, and openwrt's > QoS system, which for most of the last 7 years or so was HFSC + SFQ + > RED. Dozens of other examples exist, but not on the gear you are > familiar with. > > We can prove the values of delay-based AQM and we can also prove > the value of FQ. > > > Sure. Thing is, we've pretty much proven that the two in combination > are better than either, singly. Do you not agree with that assessment? > > The goal of AQM is to give early congestion notification before > the buffer becomes full or the delay gets big. > > > If you want to define AQM as Active Queue LENGTH management, then go > right ahead and try. Actually I was thinking hard about expanding the > scope of this proposed working group to be "Smart Queue Management" - > which neatly not only encompasses FQ and queue length management, but > also extends out to nagging problems like sanely rate limiting ipv6 > ICMP responses to someone probing someone's entire /64 address space, > and dealing with other sorts of attack like that, and problems like > incast. > > Similarly, the problem of dealing with diffserv marking remains, as to > how (or if) it would correctly interact with a SQM system. > > Things like iptables don't seem to have a home within the ietf > presently (?) > > Scheduling algorithms determine bandwidth allocation and weighted > fairness among different classes/flows. We don't have to mandate a > particular scheduling algorithm with a particular AQM scheme. > > > Concur with that very much. I continue to experiment with QFQ+, DRR, > WFQ, SFQ, and other techniques. > > Vendors shall be free to choose their best performance-cost > tradeoff point. I think it would be challenging to force vendors > to adopt a fixed set. > > > Concur. Let the best set of ideas win both here and in the > marketplace! But an end goal should be some effective form of SQM *on > by default*, on everything. > > > Best Regards, > > Rong > > > > From: Jim Gettys > > Date: Tuesday, June 4, 2013 8:47 AM > To: Wesley Eddy > > Cc: Mirja Kuehlewind >, "aqm@ietf.org > " > > > Subject: Re: [aqm] floating a draft charter > > There are a number of things I want to see: > > 1) packet scheduling is as important as drop/mark algorithms: it > is the two together that is such a win with fq_codel. So I, for > one, would be *very* unhappy to see a charter that meant it was > out of scope to discuss and document scheduling and flow queuing > strategies. And packet scheduling can interact with drop mark > algorithms. > > Optimal scheduling depends on where you are in the network (e.g. a > host, versus home router, vs. ISP head end, etc.); documenting > what makes the most sense where would likely help both > implementors of software and hardware and operators of those > networks. I've given some thought to the area, but until we have > running code, it's hand waving, but probably won't be hand waving > in a year or two. > > 2) many mark/drop algorithms can co-exist (and generating > informational RFC's to help customers write RFP's is worth while): > but so far, there is no universal "one size fits all" mark/drop > algorithm that is optimally effective everywhere. Kathy and Van > say a primary goal of CoDel was "first, do no harm", rather than > "optimal under all circumstances". The goal was an algorithm that > never hurt you and would always be safe to have enabled. CoDel's > constants were chosen so that CoDel will work at the edge of the > network with planetary length paths in the face of highly variable > bandwidth. > > But CoDel inside a datacenter (unless you mess with its constants) > is not effective in a timely fashion. > > People have feared tuneable algorithms such as (W)RED can hurt > them, so what we have had for AQM is sometimes not present at all, > or not enabled (even in nodes that it would help greatly). Note > that one of Dave Taht's discoveries was that RED had been broken > in the Linux source tree for 5 years and *no one had noticed*. > > Maybe someday we'll reach nirvana with a drop/mark algorithm that > "just works" everywhere well all the time and can never hurt and > always works perfectly, but that's a lofty goal left for research; > in the meanwhile, helping people understand what to use where in > the network and the tradeoffs is important. > > But if CoDel (as we believe) or some other algorithm really is > "first, do no harm" without tuning, it has great value to be on > by default just to prevent the disaster an operator not enabling > any AQM at all; the world will never be worse than that, and > usually be radically better. Anything that *requires* tuning of > knobs to work is a not a candidate.... > > 3) Some algorithms may be hard to retrofit into existing > platforms. For example, if you can't timestamp a packet, it's > really hard to use CoDel, which is an algorithm easy to implement > from scratch, but could be really hard retrofit in an existing > hardware design. I don't want to discourage people doing things a > lot better than nothing as a stop-gap, but people need to > understand that it may be a second best solution. > > Again, advice as to what/where algorithms may be applicable will > help designers and operators both. We face a 5-10 year transition > here, even if we succeed this time. > > 4) There are "feagtures" or lack of features that today's hardware > does that can/does make life for software very hard. Changes could > make life much easier. > > For example, today's hardware makes it hard to do true head drop > (which hardware assist could make trivial). Some have noticed > that at low bandwidth we may adjust CoDel parameters to work > around the lack of head drop and unaccounted for time in the > driver's ring buffers (which CoDel needs), since Linux's qdisc's > can't see the time in the rings. So there are a set of > recommendations to hardware and OS designers that would help the > state of the network: this simple head drop feature, packet > pacing, making GSO/TSO less evil, and so on, war stories on all > the places we've been working on debloating Linux to guide other > OS developers, etc. > > This is really the other side of queue management: try to > encourage traffic and systems that is less bursty and gentle to > the network, so the AQM/scheduling algorithms don't have to work > as hard. > > Such things should go someplace in the IETF, and if not here, > where? Doing nothing anywhere is what I want to avoid. > > Best regards, > Jim > > > > On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy > wrote: > > On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: > > > > But my point is, we should not only to standardize more and > more AQM > > mechanisms because that's a large research area which maybe > even should be > > homed in the IRTF, but we need to find actually deployment > strategies. Having > > a working group on AQM will already help to make people > aware of the topic > > and maybe think about using AQM. But only standardizing more > and more AQM > > queue, might end up in investing a lot of working that > no-one will ever use. > > > I suggest appending something like this to the charter: > > Many AQM algorithms have been proposed in academic > literature, but > very few are widely implemented and deployed. A goal of the > working > group is to produce recommendations that will actually be > used, and > algorithms that will actually be implemented, deployed in > equipment, > and enabled. Towards these ends, the group actively encourages > participation from operators and implementers, and will > coordinate > with the IETF OPS area and other relevant parts of the IETF and > Internet community. Wider research and evaluation of AQM > mechanisms > shall be coordinated with the IRTF/ICCRG, and significant > participation in this WG from the academic and research > community is > highly desirable, when it is directly relevant to > implementation and > deployment. > > > We will definitely need to get engagement from some of the > operators > that participate in the IETF, and should solicit participation > more > widely outside as well (e.g. advertise to NANOG list, bufferbloat > list, etc.) in order to attract operators and implementers > that aren't > already aware of this activity. > > Would this help to address your concern? > > > -- > Wes Eddy > MTI Systems > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > > > > -- > Dave Täht > > Fixing bufferbloat with cerowrt: > http://www.teklibre.com/cerowrt/subscribe.html > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --------------040906070605000801050000 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit


AFAIK, Chipcos for home gateways (Broadcom, Ikanos, Intel), network processors (Cavium),
but also Cisco, Juniper and Alcatel-Lucent make a difference between class based queuing and
dropping algorithms. The first is implemented everywhere in different ways and I do not think
there would be relevant work to do there (except for HTB in Linux which is really bad in terms
of performance, IMHO).

Dropping algorithms is used frequently as a term to describe AQM, e.g. in CLI based configuration
systems you may find (in commercial Linux based products) SFQ among the dropping algorithms.

It is clearly an abuse of language but common practice.  I unfortunately agree with Dave, and we discussed about
that in Paris, that chipcos do not take care about flow based queue management (typically withing a given class).
Only Linux  does that, but many products make use of Linux only for the slow path while the fast path is managed in hardware.

Deploying QoS (including de-bloating in QoS) based on Linux is soon broken by the trend to distribute packet processing
in dedicated processing units and  chipco ignore completely queue management there.
Anytime I ask whether the chip implements deficit round round robin the answer is "of course!" and in the
end it  turns out to be *modified* deficit round robin, i.e. class based queuing with up to say 64 queues.
Which is not what I am looking for.

One solution would be to move everything to software, but the cost of the chip that guarantees the same switching
throughput is much higher.

I would suggest to put more emphasis that something new is being proposed here adding
flow based packet scheduling as a novel item in the charter (in addition to class based, dropping or
congestion notification).  Using Smart Queue Management seems misleading to me or not enough.

Best regards,
Luca


-- 
France Telecom R&D - Orange Labs
MUSCARIELLO Luca - NMP/TRM
38 - 40, rue du General Leclerc
92794 Issy Les Moulineaux Cedex 9 - France
Tel : +33 (0)1 45 29 60 37
http://perso.rd.francetelecom.fr/muscariello


On 06/05/2013 09:44 AM, Dave Taht wrote:


On Wed, Jun 5, 2013 at 12:09 AM, Rong Pan (ropan) <ropan@cisco.com> wrote:
I understand packet scheduling is important. However, most network devices today treat these two issues separately: classed-based scheduling/queueing and drop/mark algorithms. In class-based queueing, vendors differ in how many classes that they support: 2, 4 or 8. In drop/mark algorithm, vendors tend to implement both tail-drop and WRED. 

I really have a great deal of trouble extrapolating to "vendors" here. I suppose you mean "big iron vendors", like Juniper, or Allied Telesis, which haven't weighed in on this list yet. In the CMTS market there are three players, and while I know what Arris can do, I would prefer an Arris representative to say. In terms of chipset makers ethernet support, most don't don't support anything in hardware, neither do the low end bridge chips, and thus they rely on software to do either AQM or FQ, in addition to doing useful things like firewalling or NAT. In servers, home routers, and handhelds, linux based devices are prevailant and can easily do just about anything. (I recently had a theorist badger me about doing a particle model for scheduling packets, because the GPU is effectively free on many common chipsets. I'm sooo not going there, but he is). Wifi chipsets generally have no strategy other than 802.11e for managing queues. Most of the France Telecom's dsl devices run shortest queue first, with longest queue drop. The list goes on and on...



I think we can continue to treat these two issues separately as it has been done in practice.
 
 Scheduling and drop algorithms have long been used in combination in practice. Examples include the venerable wondershaper, and openwrt's QoS system, which for most of the last 7 years or so was HFSC + SFQ + RED. Dozens of other examples exist, but not on the gear you are familiar with.
 
We can prove the values of delay-based AQM and we can also prove the value of FQ.

Sure. Thing is, we've pretty much proven that the two in combination are better than either, singly. Do you not agree with that assessment?
 
The goal of AQM is to give early congestion notification before the buffer becomes full or the delay gets big.

If you want to define AQM as Active Queue LENGTH management, then go right ahead and try. Actually I was thinking hard about expanding the scope of this proposed working group to be "Smart Queue Management" - which neatly not only encompasses FQ and queue length management, but also extends out to nagging problems like sanely rate limiting ipv6 ICMP responses to someone probing someone's entire /64 address space, and dealing with other sorts of attack like that, and problems like incast.

Similarly, the problem of dealing with diffserv marking remains, as to how (or if) it would correctly interact with a SQM system.
 
Things like iptables don't seem to have a home within the ietf presently (?)

Scheduling algorithms determine bandwidth allocation and weighted fairness among different classes/flows. We don't have to mandate a particular scheduling algorithm with a particular AQM scheme.

Concur with that very much. I continue to experiment with QFQ+, DRR, WFQ, SFQ, and other techniques.
 
Vendors shall be free to choose their best performance-cost tradeoff point. I think it would be challenging to force vendors to adopt a fixed set. 

Concur. Let the best set of ideas win both here and in the marketplace! But an end goal should be some effective form of SQM *on by default*, on everything.


Best Regards,

Rong 



From: Jim Gettys <jg@freedesktop.org>
Date: Tuesday, June 4, 2013 8:47 AM
To: Wesley Eddy <wes@mti-systems.com>
Cc: Mirja Kuehlewind <mirja.kuehlewind@ikr.uni-stuttgart.de>, "aqm@ietf.org" <aqm@ietf.org>

Subject: Re: [aqm] floating a draft charter

There are a number of things I want to see:

1) packet scheduling is as important as drop/mark algorithms: it is the two together that is such a win with fq_codel.  So I, for one, would be *very* unhappy to see a charter that meant it was out of scope to discuss and document scheduling and flow queuing strategies.  And packet scheduling can interact with drop mark algorithms.

Optimal scheduling depends on where you are in the network (e.g. a host, versus home router, vs. ISP head end, etc.); documenting what makes the most sense where would likely help both implementors of software and hardware and operators of those networks.  I've given some thought to the area, but until we have running code, it's hand waving, but probably won't be hand waving in a year or two.

2) many mark/drop algorithms can co-exist (and generating informational RFC's to help customers write RFP's is worth while): but so far, there is no universal "one size fits all" mark/drop algorithm that is optimally effective everywhere.  Kathy and Van say a primary goal of CoDel was "first, do no harm", rather than "optimal under all circumstances".  The goal was an algorithm that never hurt you and would always be safe to have enabled. CoDel's constants were chosen so that CoDel will work at the edge of the network with planetary length paths in the face of highly variable bandwidth. 

But CoDel inside a datacenter (unless you mess with its constants) is not effective in a timely fashion. 

People have feared tuneable algorithms such as (W)RED can hurt them, so what we have had for AQM is sometimes not present at all, or not enabled (even in nodes that it would help greatly).  Note that one of Dave Taht's discoveries was that RED had been broken in the Linux source tree for 5 years and *no one had noticed*.

Maybe someday we'll reach nirvana with a drop/mark algorithm that "just works" everywhere well all the time and can never hurt and always works perfectly, but that's a lofty goal left for research; in the meanwhile, helping people understand what to use where in the network and the tradeoffs is important.

But if CoDel (as we believe) or some other algorithm really is "first, do no harm" without tuning,  it has great value to be on by default just to prevent the disaster an operator not enabling any AQM at all; the world will never be worse than that, and usually be radically better. Anything that *requires* tuning of knobs to work is a not a candidate....

3) Some algorithms may be hard to retrofit into existing platforms.  For example, if you can't timestamp a packet, it's really hard to use CoDel, which is an algorithm easy to implement from scratch, but could be really hard retrofit in an existing hardware design. I don't want to discourage people doing things a lot better than nothing as a stop-gap, but people need to understand that it may be a second best solution.

Again, advice as to what/where algorithms may be applicable will help designers and operators both.  We face a 5-10 year transition here, even if we succeed this time.

4) There are "feagtures" or lack of features that today's hardware does that can/does make life for software very hard. Changes could make life much easier. 

For example, today's hardware makes it hard to do true head drop (which hardware assist could make trivial).  Some have noticed that at low bandwidth we may adjust CoDel parameters to work around the lack of head drop and unaccounted for time in the driver's ring buffers (which CoDel needs), since Linux's qdisc's can't see the time in the rings. So there are a set of recommendations to hardware and OS designers that would help the state of the network: this simple head drop feature, packet pacing, making GSO/TSO less evil, and so on, war stories on all the places we've been working on debloating Linux to guide other OS developers, etc.

This is really the other side of queue management: try to encourage traffic and systems that is less bursty and gentle to the network, so the AQM/scheduling algorithms don't have to work as hard.

Such things should go someplace in the IETF, and if not here, where?  Doing nothing anywhere is what I want to avoid.

Best regards,
    Jim



On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy <wes@mti-systems.com> wrote:
On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote:
>
> But my point is, we should not only to standardize more and more AQM
> mechanisms because that's a large research area which maybe even should be
> homed in the IRTF, but we need to find actually deployment strategies. Having
> a working group on AQM will already help to make people aware of the topic
> and maybe think about using AQM. But only standardizing more and more AQM
> queue, might end up in investing a lot of working that no-one will ever use.


I suggest appending something like this to the charter:

  Many AQM algorithms have been proposed in academic literature, but
  very few are widely implemented and deployed.  A goal of the working
  group is to produce recommendations that will actually be used, and
  algorithms that will actually be implemented, deployed in equipment,
  and enabled.  Towards these ends, the group actively encourages
  participation from operators and implementers, and will coordinate
  with the IETF OPS area and other relevant parts of the IETF and
  Internet community.  Wider research and evaluation of AQM mechanisms
  shall be coordinated with the IRTF/ICCRG, and significant
  participation in this WG from the academic and research community is
  highly desirable, when it is directly relevant to implementation and
  deployment.


We will definitely need to get engagement from some of the operators
that participate in the IETF, and should solicit participation more
widely outside as well (e.g. advertise to NANOG list, bufferbloat
list, etc.) in order to attract operators and implementers that aren't
already aware of this activity.

Would this help to address your concern?


--
Wes Eddy
MTI Systems
_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm




--
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html 

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

--------------040906070605000801050000-- From gettysjim@gmail.com Wed Jun 5 06:19:45 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A9E5421F9A49 for ; Wed, 5 Jun 2013 06:19:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.75 X-Spam-Level: X-Spam-Status: No, score=-1.75 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d5ETHqyc6GJ6 for ; Wed, 5 Jun 2013 06:19:43 -0700 (PDT) Received: from mail-oa0-x22a.google.com (mail-oa0-x22a.google.com [IPv6:2607:f8b0:4003:c02::22a]) by ietfa.amsl.com (Postfix) with ESMTP id 0CD1921F99F8 for ; Wed, 5 Jun 2013 06:19:42 -0700 (PDT) Received: by mail-oa0-f42.google.com with SMTP id i10so1114654oag.29 for ; Wed, 05 Jun 2013 06:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=AK6cZR4Nq2O4RIaUtbrJ2B8uIpvLQWrWXe91nDz7Q4c=; b=HqxB3mDSQaEvxK2X3w5jZSH2WpZJhFUs8XkM8oIu3aGluRuGlvxQlaJGnO3JV6oF+l H/YL06fzE26AJfgNjJfTm2Cx7R6gLFWgja8ltyt/owtgyyruZhdwh8pM2r1hxUC29rkd UOuCxDxaVIvik27mqEYPP9DlWkcl+zhsZJWoP9ZNKsLRgI75LVucnil2Kxo9SJqHHv/o BtW86h/2oGZ5oNRiK+JF2tMRbT4gQFtZ72q5f2o8wSsgaYi52VRoXF9k15FIWNgBCUMO AfcD+ras7urS+8xkNNx9rtyo61SwvrgO0j5HE9WkC0s4RFUAgKs6GLRbZmjUwKZL3GBs bzHw== MIME-Version: 1.0 X-Received: by 10.60.102.230 with SMTP id fr6mr14736375oeb.141.1370438377320; Wed, 05 Jun 2013 06:19:37 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.76.87.129 with HTTP; Wed, 5 Jun 2013 06:19:36 -0700 (PDT) In-Reply-To: <51AEFA86.1070802@orange.com> References: <51AEFA86.1070802@orange.com> Date: Wed, 5 Jun 2013 09:19:36 -0400 X-Google-Sender-Auth: Y_BQWvZZ-LW9K5ppLhrFOnziycI Message-ID: From: Jim Gettys To: Luca MUSCARIELLO Content-Type: multipart/alternative; boundary=089e011830d6ab04d304de680c6f Cc: Wesley Eddy , "Rong Pan \(ropan\)" , Mirja Kuehlewind , Dave Taht , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2013 13:19:45 -0000 --089e011830d6ab04d304de680c6f Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Wed, Jun 5, 2013 at 4:44 AM, Luca MUSCARIELLO < luca.muscariello@orange.com> wrote: > > > AFAIK, Chipcos for home gateways (Broadcom, Ikanos, Intel), network > processors (Cavium), > but also Cisco, Juniper and Alcatel-Lucent make a difference between clas= s > based queuing and > dropping algorithms. The first is implemented everywhere in different way= s > and I do not think > there would be relevant work to do there (except for HTB in Linux which i= s > really bad in terms > of performance, IMHO). > > Dropping algorithms is used frequently as a term to describe AQM, e.g. in > CLI based configuration > systems you may find (in commercial Linux based products) SFQ among the > dropping algorithms. > > It is clearly an abuse of language but common practice. I unfortunately > agree with Dave, and we discussed about > that in Paris, that chipcos do not take care about flow based queue > management (typically withing a given class). > Only Linux does that, but many products make use of Linux only for the > slow path while the fast path is managed in hardware. > > Deploying QoS (including de-bloating in QoS) based on Linux is soon broke= n > by the trend to distribute packet processing > in dedicated processing units and chipco ignore completely queue > management there. > Anytime I ask whether the chip implements deficit round round robin the > answer is "of course!" and in the > end it turns out to be *modified* deficit round robin, i.e. class based > queuing with up to say 64 queues. > Which is not what I am looking for. > > One solution would be to move everything to software, but the cost of the > chip that guarantees the same switching > throughput is much higher. > There are two points I'll react to in this statement: 1) performance: in the edge of the network, the SOC's used are already very fast and more than fast enough to support algorithms such as fq_codel (much faster than one's wireless or broadband connection). The existence proofs are running code in OpenWrt on commodity home router SOC's. And when I asked Eric Dumazet about fq_codel performance, he said I could quote him as it taking 2% of a modern x86 CPU core at 10G speed. This was the point at which I got more than moderately excited about this class of algorithms. CPU's are not what they were in the 1980's and 1990's, and we have to get over that and move on. So if hardware assist is not assisting, we don't have to use it, and it may encourage hardware people to "get it right" the next generation. 2) most application's performance is now bound by latency, and not bandwidth. Your web browsing doesn't get appreciably faster adding bandwidth, but improving the latency, even if there is not competing traffic, does improve performance, not to mention improving everything across the board and for most things getting us out of the impossible to configure QOS morass we're in. As I said, what may be feasible or advisable to do at what point in the network may vary: I don't think this group will *mandate* much at all, but describing what can be used where for best effect is worth a lot, and *possibly* mandating a "first, do no harm" algorithm, even if far from ideal in many circumstances, to prevent no AQM being present at all, which is a headache in today's network. We're going to have a large number of different drop and queuing algorithms over the coming years, I predict. - Jim > I would suggest to put more emphasis that something new is being proposed > here adding > flow based packet scheduling as a novel item in the charter (in addition > to class based, dropping or > congestion notification). Using Smart Queue Management seems misleading > to me or not enough. > > Best regards, > Luca > > > -- > France Telecom R&D - Orange Labs > MUSCARIELLO Luca - NMP/TRM > 38 - 40, rue du General Leclerc > 92794 Issy Les Moulineaux Cedex 9 - France > Tel : +33 (0)1 45 29 60 37http://perso.rd.francetelecom.fr/muscariello > > > > On 06/05/2013 09:44 AM, Dave Taht wrote: > > > > On Wed, Jun 5, 2013 at 12:09 AM, Rong Pan (ropan) wrote= : > >> I understand packet scheduling is important. However, most network >> devices today treat these two issues separately: classed-based >> scheduling/queueing and drop/mark algorithms. In class-based queueing, >> vendors differ in how many classes that they support: 2, 4 or 8. In >> drop/mark algorithm, vendors tend to implement both tail-drop and WRED. >> > > I really have a great deal of trouble extrapolating to "vendors" here. I > suppose you mean "big iron vendors", like Juniper, or Allied Telesis, whi= ch > haven't weighed in on this list yet. In the CMTS market there are three > players, and while I know what Arris can do, I would prefer an Arris > representative to say. In terms of chipset makers ethernet support, most > don't don't support anything in hardware, neither do the low end bridge > chips, and thus they rely on software to do either AQM or FQ, in addition > to doing useful things like firewalling or NAT. In servers, home routers, > and handhelds, linux based devices are prevailant and can easily do just > about anything. (I recently had a theorist badger me about doing a partic= le > model for scheduling packets, because the GPU is effectively free on many > common chipsets. I'm sooo not going there, but he is). Wifi chipsets > generally have no strategy other than 802.11e for managing queues. Most o= f > the France Telecom's dsl devices run shortest queue first, with longest > queue drop. The list goes on and on... > > > >> I think we can continue to treat these two issues separately as it has >> been done in practice. >> > > Scheduling and drop algorithms have long been used in combination in > practice. Examples include the venerable wondershaper, and openwrt's QoS > system, which for most of the last 7 years or so was HFSC + SFQ + RED. > Dozens of other examples exist, but not on the gear you are familiar with= . > > >> We can prove the values of delay-based AQM and we can also prove the >> value of FQ. >> > > Sure. Thing is, we've pretty much proven that the two in combination are > better than either, singly. Do you not agree with that assessment? > > >> The goal of AQM is to give early congestion notification before the >> buffer becomes full or the delay gets big. >> > > If you want to define AQM as Active Queue LENGTH management, then go righ= t > ahead and try. Actually I was thinking hard about expanding the scope of > this proposed working group to be "Smart Queue Management" - which neatly > not only encompasses FQ and queue length management, but also extends out > to nagging problems like sanely rate limiting ipv6 ICMP responses to > someone probing someone's entire /64 address space, and dealing with othe= r > sorts of attack like that, and problems like incast. > > Similarly, the problem of dealing with diffserv marking remains, as to ho= w > (or if) it would correctly interact with a SQM system. > > Things like iptables don't seem to have a home within the ietf presently > (?) > > Scheduling algorithms determine bandwidth allocation and weighted >> fairness among different classes/flows. We don't have to mandate a >> particular scheduling algorithm with a particular AQM scheme. >> > > Concur with that very much. I continue to experiment with QFQ+, DRR, WFQ, > SFQ, and other techniques. > > >> Vendors shall be free to choose their best performance-cost tradeoff >> point. I think it would be challenging to force vendors to adopt a fixed >> set. >> > > Concur. Let the best set of ideas win both here and in the marketplace! > But an end goal should be some effective form of SQM *on by default*, on > everything. > > >> Best Regards, >> >> Rong >> >> >> >> From: Jim Gettys >> Date: Tuesday, June 4, 2013 8:47 AM >> To: Wesley Eddy >> Cc: Mirja Kuehlewind , " >> aqm@ietf.org" >> >> Subject: Re: [aqm] floating a draft charter >> >> There are a number of things I want to see: >> >> 1) packet scheduling is as important as drop/mark algorithms: it is the >> two together that is such a win with fq_codel. So I, for one, would be >> *very* unhappy to see a charter that meant it was out of scope to discus= s >> and document scheduling and flow queuing strategies. And packet schedul= ing >> can interact with drop mark algorithms. >> >> Optimal scheduling depends on where you are in the network (e.g. a >> host, versus home router, vs. ISP head end, etc.); documenting what make= s >> the most sense where would likely help both implementors of software and >> hardware and operators of those networks. I've given some thought to th= e >> area, but until we have running code, it's hand waving, but probably won= 't >> be hand waving in a year or two. >> >> 2) many mark/drop algorithms can co-exist (and generating informational >> RFC's to help customers write RFP's is worth while): but so far, there i= s >> no universal "one size fits all" mark/drop algorithm that is optimally >> effective everywhere. Kathy and Van say a primary goal of CoDel was >> "first, do no harm", rather than "optimal under all circumstances". The >> goal was an algorithm that never hurt you and would always be safe to ha= ve >> enabled. CoDel's constants were chosen so that CoDel will work at the ed= ge >> of the network with planetary length paths in the face of highly variabl= e >> bandwidth. >> >> But CoDel inside a datacenter (unless you mess with its constants) is >> not effective in a timely fashion. >> >> People have feared tuneable algorithms such as (W)RED can hurt them, so >> what we have had for AQM is sometimes not present at all, or not enabled >> (even in nodes that it would help greatly). Note that one of Dave Taht'= s >> discoveries was that RED had been broken in the Linux source tree for 5 >> years and *no one had noticed*. >> >> Maybe someday we'll reach nirvana with a drop/mark algorithm that "just >> works" everywhere well all the time and can never hurt and always works >> perfectly, but that's a lofty goal left for research; in the meanwhile, >> helping people understand what to use where in the network and the >> tradeoffs is important. >> >> But if CoDel (as we believe) or some other algorithm really is "first, >> do no harm" without tuning, it has great value to be on by default just= to >> prevent the disaster an operator not enabling any AQM at all; the world >> will never be worse than that, and usually be radically better. Anything >> that *requires* tuning of knobs to work is a not a candidate.... >> >> 3) Some algorithms may be hard to retrofit into existing platforms. >> For example, if you can't timestamp a packet, it's really hard to use >> CoDel, which is an algorithm easy to implement from scratch, but could b= e >> really hard retrofit in an existing hardware design. I don't want to >> discourage people doing things a lot better than nothing as a stop-gap, = but >> people need to understand that it may be a second best solution. >> >> Again, advice as to what/where algorithms may be applicable will help >> designers and operators both. We face a 5-10 year transition here, even= if >> we succeed this time. >> >> 4) There are "feagtures" or lack of features that today's hardware does >> that can/does make life for software very hard. Changes could make life >> much easier. >> >> For example, today's hardware makes it hard to do true head drop (which >> hardware assist could make trivial). Some have noticed that at low >> bandwidth we may adjust CoDel parameters to work around the lack of head >> drop and unaccounted for time in the driver's ring buffers (which CoDel >> needs), since Linux's qdisc's can't see the time in the rings. So there = are >> a set of recommendations to hardware and OS designers that would help th= e >> state of the network: this simple head drop feature, packet pacing, maki= ng >> GSO/TSO less evil, and so on, war stories on all the places we've been >> working on debloating Linux to guide other OS developers, etc. >> >> This is really the other side of queue management: try to encourage >> traffic and systems that is less bursty and gentle to the network, so th= e >> AQM/scheduling algorithms don't have to work as hard. >> >> Such things should go someplace in the IETF, and if not here, where? >> Doing nothing anywhere is what I want to avoid. >> >> Best regards, >> Jim >> >> >> >> On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy wrote= : >> >>> On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote: >>> > >>> > But my point is, we should not only to standardize more and more AQM >>> > mechanisms because that's a large research area which maybe even >>> should be >>> > homed in the IRTF, but we need to find actually deployment strategies= . >>> Having >>> > a working group on AQM will already help to make people aware of the >>> topic >>> > and maybe think about using AQM. But only standardizing more and more >>> AQM >>> > queue, might end up in investing a lot of working that no-one will >>> ever use. >>> >>> >>> I suggest appending something like this to the charter: >>> >>> Many AQM algorithms have been proposed in academic literature, but >>> very few are widely implemented and deployed. A goal of the working >>> group is to produce recommendations that will actually be used, and >>> algorithms that will actually be implemented, deployed in equipment, >>> and enabled. Towards these ends, the group actively encourages >>> participation from operators and implementers, and will coordinate >>> with the IETF OPS area and other relevant parts of the IETF and >>> Internet community. Wider research and evaluation of AQM mechanisms >>> shall be coordinated with the IRTF/ICCRG, and significant >>> participation in this WG from the academic and research community is >>> highly desirable, when it is directly relevant to implementation and >>> deployment. >>> >>> >>> We will definitely need to get engagement from some of the operators >>> that participate in the IETF, and should solicit participation more >>> widely outside as well (e.g. advertise to NANOG list, bufferbloat >>> list, etc.) in order to attract operators and implementers that aren't >>> already aware of this activity. >>> >>> Would this help to address your concern? >>> >>> >>> -- >>> Wes Eddy >>> MTI Systems >>> _______________________________________________ >>> aqm mailing list >>> aqm@ietf.org >>> https://www.ietf.org/mailman/listinfo/aqm >>> >> >> >> _______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm >> >> > > > -- > Dave T=E4ht > > Fixing bufferbloat with cerowrt: > http://www.teklibre.com/cerowrt/subscribe.html > > _______________________________________________ > aqm mailing listaqm@ietf.orghttps://www.ietf.org/mailman/listinfo/aqm > > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > --089e011830d6ab04d304de680c6f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable


On Wed= , Jun 5, 2013 at 4:44 AM, Luca MUSCARIELLO <luca.muscariello@ora= nge.com> wrote:
=20 =20 =20


AFAIK, Chipcos for home gateways (Broadcom, Ikanos, Intel), network processors (Cavium),
but also Cisco, Juniper and Alcatel-Lucent make a difference between class based queuing and
dropping algorithms. The first is implemented everywhere in different ways and I do not think
there would be relevant work to do there (except for HTB in Linux which is really bad in terms
of performance, IMHO).

Dropping algorithms is used frequently as a term to describe AQM, e.g. in CLI based configuration
systems you may find (in commercial Linux based products) SFQ among the dropping algorithms.

It is clearly an abuse of language but common practice.=A0 I unfortunately agree with Dave, and we discussed about
that in Paris, that chipcos do not take care about flow based queue management (typically withing a given class).
Only Linux=A0 does that, but many products make use of Linux only for the slow path while the fast path is managed in hardware.

Deploying QoS (including de-bloating in QoS) based on Linux is soon broken by the trend to distribute packet processing
in dedicated processing units and=A0 chipco ignore completely queue management there.
Anytime I ask whether the chip implements deficit round round robin the answer is "of course!" and in the
end it=A0 turns out to be *modified* deficit round robin, i.e. class based queuing with up to say 64 queues.
Which is not what I am looking for.

One solution would be to move everything to software, but the cost of the chip that guarantees the same switching
throughput is much higher.

There are two point= s I'll react to in this statement:

=A0 =A0 1)= performance: in the edge of the network, the SOC's used are already ve= ry fast and more than fast enough to support algorithms such as fq_codel (m= uch faster than one's wireless or broadband connection).=A0

The existence proofs are runni= ng code in OpenWrt on commodity home router SOC's. And when I asked Eri= c Dumazet about fq_codel performance, he said I could quote him as it takin= g 2% of a modern x86 CPU core at 10G speed. =A0This was the point at which = I got more than moderately excited about this class of algorithms. CPU'= s are not what they were in the 1980's and 1990's, and we have to g= et over that and move on.

So if hardware assist is not a= ssisting, we don't have to use it, and it may encourage hardware people= to "get it right" the next generation.

=A0 =A02) most application'= ;s performance is now bound by latency, and not bandwidth. =A0Your web brow= sing doesn't get appreciably faster adding bandwidth, but improving the= latency, even if there is not competing traffic, does improve performance,= not to mention improving everything across the board and for most things g= etting us out of the impossible to configure QOS morass we're in.

As I said, what may be feasibl= e or advisable to do at what point in the network may vary: I don't thi= nk this group will *mandate* much at all, but describing what can be used w= here for best effect is worth a lot, and *possibly* mandating a "first= , do no harm" algorithm, even if far from ideal in many circumstances,= to prevent no AQM being present at all, which is a headache in today's= network.

We're going to have a larg= e number of different drop and queuing algorithms over the coming years, I = predict.
=A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= - Jim



I would suggest to put more emphasis that something new is being proposed here adding
flow based packet scheduling as a novel item in the charter (in addition to class based, dropping or
congestion notification).=A0 Using Smart Queue Management seems misleading to me or not enough.

Best regards,
Luca


--=20
France Telecom R&D - Orange Labs
MUSCARIELLO Luca - NMP/TRM
38 - 40, rue du General Leclerc
92794 Issy Les Moulineaux Cedex 9 - France
Tel : +33 (0)1 45 29 60 37
=
http://perso.rd.francetelecom.fr/muscariello


On 06/05/2013 09:44 AM, Dave Taht wrote:
=20

On Wed, Jun 5, 2013 at 12:09 AM, Rong Pan (ropan) <ropan@cisco.com> wrote:
I understand packet scheduling is important. However, most network devices today treat these two issues separately: classed-based scheduling/queueing and drop/mark algorithms.=A0In class-based queueing, vendors differ in how many classes that they support: 2, 4 or 8. In drop/mark algorithm, vendors tend to implement both tail-drop and WRED.=A0

I really have a great deal of trouble extrapolating to "vendors" here. I suppose you mean "big iron vendo= rs", like Juniper, or Allied Telesis, which haven't weighed in on this list yet. In the CMTS market there are three players, and while I know what Arris can do, I would prefer an Arris representative to say. In terms of chipset makers ethernet support, most don't don't support anything in hardware, neither do the low end bridge chips, and thus they rely on software to do either AQM or FQ, in addition to doing useful things like firewalling or NAT. In servers, home routers, and handhelds, linux based devices are prevailant and can easily do just about anything. (I recently had a theorist badger me about doing a particle model for scheduling packets, because the GPU is effectively free on many common chipsets. I'm sooo not going there, but he is). Wifi chipsets generally have no strategy other than 802.11e for managing queues. Most of the France Telecom's dsl devices run shortest queue first, with longest queue drop. The list goes on and on...



I think we can continue to treat these two issues separately as it has been done in practice.
=A0
=A0Scheduling and drop algorithms have long been used in combination in practice. Examples include the venerable wondershaper, and openwrt's QoS system, which for most of the last 7 years or so was HFSC + SFQ + RED. Dozens of other examples exist, but not on the gear you are familiar with.
=A0
We can prove the values of delay-based AQM and we can also prove the value of FQ.

Sure. Thing is, we've pretty much proven that the two in combination are better than either, singly. Do you not agree with that assessment?
=A0
The goal of AQM is to give early congestion notification before the buffer becomes full or the delay gets big.

If you want to define AQM as Active Queue LENGTH management, then go right ahead and try. Actually I was thinking hard about expanding the scope of this proposed working group to be "Smart Queue Management" - which neatly not only encomp= asses FQ and queue length management, but also extends out to nagging problems like sanely rate limiting ipv6 ICMP responses to someone probing someone's entire /64 address space, and dealing with other sorts of attack like that, and problems like incast.

Similarly, the problem of dealing with diffserv marking remains, as to how (or if) it would correctly interact with a SQM system.
=A0
Things like iptables don't seem to have a home within the iet= f presently (?)

Scheduling algorithms determine bandwidth allocation and weighted fairness among different classes/flows. We don't have to mandate a particular scheduling algorithm with a particular AQM scheme.

Concur with that very much. I continue to experiment with QFQ+, DRR, WFQ, SFQ, and other techniques.
=A0
Vendors shall be free to choose their best performance-cost tradeoff point. I think it would be challenging to force vendors to adopt a fixed set.=A0

Concur. Let the best set of ideas win both here and in the marketplace! But an end goal should be some effective form of SQM *on by default*, on everything.


Best Regards,

Rong=A0



From: Jim Gettys <jg@freedesktop.org>
Date: Tuesday, June 4, 2013 8:47 AM
To: Wesley Eddy <wes@mti-systems.com>
Cc: Mirja Kuehlewind <mirja.kuehlewind@ikr.uni-stuttgart.de>= ;, "aqm= @ietf.org" <a= qm@ietf.org>

Subject: Re: [aqm] floating a draft charter

There are a number o= f things I want to see:

1) packet scheduling is as important as drop/mark algorithms: it is the two together that is such a win with fq_codel. =A0So I, for one, would be *very* unhappy to see a charter that meant it was out of scope to discuss and document scheduling and flow queuing strategies. =A0And packet scheduling can interact with drop mark algorithms.

Optimal scheduling depends on where you are in the network (e.g. a host, versus home router, vs. ISP head end, etc.); documenting what makes the most sense where would likely help both implementors of software and hardware and operators of those networks. =A0I've given some thought to the area, but until we have running code, it's hand waving, but probably won't be hand wavin= g in a year or two.

2) many mark/drop algorithms can co-exist (and generating informational RFC's to help customers write RFP's is worth while): but so far, there is n= o universal "one size fits all" mark/drop algorithm that is optimally effective everywhere. =A0Kathy and Van say a primary goal of CoDel was "first, do no harm", rathe= r than "optimal under all circumstances". =A0T= he goal was an algorithm that never hurt you and would always be safe to have enabled. CoDel's constants were chosen so that CoDel will work at the edge of the network with planetary length paths in the face of highly variable bandwidth.=A0

But CoDel inside a datacenter (unless you mess with its constants) is not effective in a timely fashion.=A0

People have feared tuneable algorithms such as (W)RED can hurt them, so what we have had for AQM is sometimes not present at all, or not enabled (even in nodes that it would help greatly). =A0Note that one of Dave Taht's discoveries was that RED had been broken in the Linux source tree for 5 years and *no one had noticed*.

Maybe someday we'= ;ll reach nirvana with a drop/mark algorithm that "just works" everywhere well all the ti= me and can never hurt and always works perfectly, but that's a lofty goal left for research; in the meanwhile, helping people understand what to use where in the network and the tradeoffs is important.

But if CoDel (as we believe) or some other algorithm really is "first, do no harm" without tuning, =A0= it has great value to be on by default just to prevent the disaster an operator not enabling any AQM at all; the world will never be worse than that, and usually be radically better. Anything that *requires* tuning of knobs to work is a not a candidate....

3) Some algorithms m= ay be hard to retrofit into existing platforms. =A0For example, if you can't timestamp a pack= et, it's really hard to use CoDel, which is an algorithm easy to implement from scratch, but could be really hard retrofit in an existing hardware design. I don't want to discourage people doing things a lot better than nothing as a stop-gap, but people need to understand that it may be a second best solution.

Again, advice as to what/where algorithms may be applicable will help designers and operators both. =A0We face a 5-10 year transition here, even if we succeed this time.

4) There are "feagtures" or lack of features that to= day's hardware does that can/does make life for software very hard. Changes could make life much easier.=A0

For example, today&#= 39;s hardware makes it hard to do true head drop (which hardware assist could make trivial). =A0Some have noticed that at low bandwidth we may adjust CoDel parameters to work around the lack of head drop and unaccounted for time in the driver's ring buffers (which CoDel needs)= , since Linux's qdisc's can't see the t= ime in the rings. So there are a set of recommendations to hardware and OS designers that would help the state of the network: this simple head drop feature, packet pacing, making GSO/TSO less evil, and so on, war stories on all the places we've been working on debloating Linux to guide other OS developers, etc.

This is really the other side of queue management: try to encourage traffic and systems that is less bursty and gentle to the network, so the AQM/scheduling algorithms don't have to work as hard.

Such things should g= o someplace in the IETF, and if not here, where? =A0Doing nothing anywhere is what I want to avoid.

Best regards,
=A0 =A0 Jim



On Thu, May 30, 2013 at 12:13 PM, Wesley Eddy <wes@mti-systems.com> wrote:
On 5/30/2013 8:20 AM, Mirja Kuehlewind wrote:
>
> But my point is, we should not only to standardize more and more AQM
> mechanisms because that's a large research area which maybe even should be
> homed in the IRTF, but we need to find actually deployment strategies. Having
> a working group on AQM will already help to make people aware of the topic
> and maybe think about using AQM. But only standardizing more and more AQM
> queue, might end up in investing a lot of working that no-one will ever use.


I suggest appending something like this to the charter:

=A0 Many AQM algorithms have been proposed in academic literature, but
=A0 very few are widely implemented and deployed. =A0A goal of the working
=A0 group is to produce recommendations that will actually be used, and
=A0 algorithms that will actually be implemented, deployed in equipment,
=A0 and enabled. =A0Towards these ends, the group actively encourages
=A0 participation from operators and implementers, and will coordinate
=A0 with the IETF OPS area and other relevant parts of the IETF and
=A0 Internet community. =A0Wider research and evaluation of AQM mechanisms
=A0 shall be coordinated with the IRTF/ICCRG, and significant
=A0 participation in this WG from the academic and research community is
=A0 highly desirable, when it is directly relevant to implementation and
=A0 deployment.


We will definitely need to get engagement from some of the operators
that participate in the IETF, and should solicit participation more
widely outside as well (e.g. advertise to NANOG list, bufferbloat
list, etc.) in order to attract operators and implementers that aren't
already aware of this activity.

Would this help to address your concern?


--
Wes Eddy
MTI Systems
________________________________________= _______
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm<= br>


_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm




--
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/su= bscribe.html 

_______________________________________________
aqm mailing list
aqm@ietf.org
htt=
ps://www.ietf.org/mailman/listinfo/aqm


_______________________________________________
aqm mailing list
aqm@ietf.org
htt= ps://www.ietf.org/mailman/listinfo/aqm


--089e011830d6ab04d304de680c6f-- From luca.muscariello@orange.com Wed Jun 5 07:18:42 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3DB0721F9A66 for ; Wed, 5 Jun 2013 07:18:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.021 X-Spam-Level: X-Spam-Status: No, score=-6.021 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_FR=0.35, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GP5vV25wIN4t for ; Wed, 5 Jun 2013 07:18:37 -0700 (PDT) Received: from p-mail2.rd.orange.com (p-mail2.rd.orange.com [195.101.245.16]) by ietfa.amsl.com (Postfix) with ESMTP id 5CBAA21F9A61 for ; Wed, 5 Jun 2013 07:18:37 -0700 (PDT) Received: from p-mail2.rd.orange.com (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id A2CAD1074004; Wed, 5 Jun 2013 16:18:46 +0200 (CEST) Received: from ftrdsmtp2.rd.francetelecom.fr (unknown [10.192.128.47]) by p-mail2.rd.orange.com (Postfix) with ESMTP id AF7EA1074002; Wed, 5 Jun 2013 16:18:45 +0200 (CEST) Received: from ftrdmel10.rd.francetelecom.fr ([10.192.128.44]) by ftrdsmtp2.rd.francetelecom.fr with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jun 2013 16:18:30 +0200 Received: from [10.193.161.45] ([10.193.161.45]) by ftrdmel10.rd.francetelecom.fr with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jun 2013 16:18:29 +0200 Message-ID: <51AF48B5.8030707@orange.com> Date: Wed, 05 Jun 2013 16:18:29 +0200 From: MUSCARIELLO Luca OLNC/OLN User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130510 Thunderbird/17.0.6 MIME-Version: 1.0 To: Jim Gettys References: <51AEFA86.1070802@orange.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------030407030103050008050409" X-OriginalArrivalTime: 05 Jun 2013 14:18:29.0388 (UTC) FILETIME=[8D301CC0:01CE61F7] Cc: Wesley Eddy , "Rong Pan \(ropan\)" , Mirja Kuehlewind , Dave Taht , "aqm@ietf.org" Subject: Re: [aqm] floating a draft charter X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: MUSCARIELLO Luca OLNC/OLN List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2013 14:18:42 -0000 This is a multi-part message in MIME format. --------------030407030103050008050409 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 06/05/2013 03:19 PM, Jim Gettys wrote: > > On Wed, Jun 5, 2013 at 4:44 AM, Luca MUSCARIELLO > > wrote: > > > > AFAIK, Chipcos for home gateways (Broadcom, Ikanos, Intel), > network processors (Cavium), > but also Cisco, Juniper and Alcatel-Lucent make a difference > between class based queuing and > dropping algorithms. The first is implemented everywhere in > different ways and I do not think > there would be relevant work to do there (except for HTB in Linux > which is really bad in terms > of performance, IMHO). > > Dropping algorithms is used frequently as a term to describe AQM, > e.g. in CLI based configuration > systems you may find (in commercial Linux based products) SFQ > among the dropping algorithms. > > It is clearly an abuse of language but common practice. I > unfortunately agree with Dave, and we discussed about > that in Paris, that chipcos do not take care about flow based > queue management (typically withing a given class). > Only Linux does that, but many products make use of Linux only > for the slow path while the fast path is managed in hardware. > > Deploying QoS (including de-bloating in QoS) based on Linux is > soon broken by the trend to distribute packet processing > in dedicated processing units and chipco ignore completely queue > management there. > Anytime I ask whether the chip implements deficit round round > robin the answer is "of course!" and in the > end it turns out to be *modified* deficit round robin, i.e. class > based queuing with up to say 64 queues. > Which is not what I am looking for. > > One solution would be to move everything to software, but the cost > of the chip that guarantees the same switching > throughput is much higher. > > > There are two points I'll react to in this statement: > > 1) performance: in the edge of the network, the SOC's used are > already very fast and more than fast enough to support algorithms such > as fq_codel (much faster than one's wireless or broadband connection). > > The existence proofs are running code in OpenWrt on commodity home > router SOC's. And when I asked Eric Dumazet about fq_codel > performance, he said I could quote him as it taking 2% of a modern x86 > CPU core at 10G speed. This was the point at which I got more than > moderately excited about this class of algorithms. CPU's are not what > they were in the 1980's and 1990's, and we have to get over that and > move on. For what concerns performance, I am thinking about the following kind of home gateway. A triple play offer (voice, Internet, TV) deployed using a MIPS based gateway. It incorporates 4 Gigabit Ethernet ports, wifi @2.5GHz b/g/n, wifi @5GHz a/n, of course USB, SATA for your local storage. It must support FTTH and xDSL, IPv4/6 plus some software like samba, DLNA. For instance: http://www.ikanos.com/products/communications-processors/fusiv-vx185183/ Packet processing is distributed in hardware to let the central unit take care of other stuff. Do not think to have a multi-core Intel processor there. You can implement your packet scheduling in hardware (micro code, software in the end) but it is *not* an open environment. You code your queuing now, the environment evolves and nobody maintains your code which is broken as the micro-code evolves. So the problem is not really software/hardware but open/closed and re-usability (Linux matters not only because it is open but also because it is used everywhere). > > So if hardware assist is not assisting, we don't have to use it, and > it may encourage hardware people to "get it right" the next generation. > > 2) most application's performance is now bound by latency, and not > bandwidth. Your web browsing doesn't get appreciably faster adding > bandwidth, but improving the latency, even if there is not competing > traffic, does improve performance, not to mention improving everything > across the board and for most things getting us out of the impossible > to configure QOS morass we're in. > > As I said, what may be feasible or advisable to do at what point in > the network may vary: I don't think this group will *mandate* much at > all, but describing what can be used where for best effect is worth a > lot, and *possibly* mandating a "first, do no harm" algorithm, even if > far from ideal in many circumstances, to prevent no AQM being present > at all, which is a headache in today's network. > > We're going to have a large number of different drop and queuing > algorithms over the coming years, I predict. Yes, I hope. But if you want them deployed I suspect we'll wait to have either open hardware or to have *true* FQ in the chip of home gateways. Also because most of the delay is measured in the user's home uplink. Having everything in software is easy and useful but does not cover most of the current market. Best regards Luca --------------030407030103050008050409 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
On 06/05/2013 03:19 PM, Jim Gettys wrote:

On Wed, Jun 5, 2013 at 4:44 AM, Luca MUSCARIELLO <luca.muscariello@orange.com> wrote:


AFAIK, Chipcos for home gateways (Broadcom, Ikanos, Intel), network processors (Cavium),
but also Cisco, Juniper and Alcatel-Lucent make a difference between class based queuing and
dropping algorithms. The first is implemented everywhere in different ways and I do not think
there would be relevant work to do there (except for HTB in Linux which is really bad in terms
of performance, IMHO).

Dropping algorithms is used frequently as a term to describe AQM, e.g. in CLI based configuration
systems you may find (in commercial Linux based products) SFQ among the dropping algorithms.

It is clearly an abuse of language but common practice.  I unfortunately agree with Dave, and we discussed about
that in Paris, that chipcos do not take care about flow based queue management (typically withing a given class).
Only Linux  does that, but many products make use of Linux only for the slow path while the fast path is managed in hardware.

Deploying QoS (including de-bloating in QoS) based on Linux is soon broken by the trend to distribute packet processing
in dedicated processing units and  chipco ignore completely queue management there.
Anytime I ask whether the chip implements deficit round round robin the answer is "of course!" and in the
end it  turns out to be *modified* deficit round robin, i.e. class based queuing with up to say 64 queues.
Which is not what I am looking for.

One solution would be to move everything to software, but the cost of the chip that guarantees the same switching
throughput is much higher.

There are two points I'll react to in this statement:

    1) performance: in the edge of the network, the SOC's used are already very fast and more than fast enough to support algorithms such as fq_codel (much faster than one's wireless or broadband connection). 

The existence proofs are running code in OpenWrt on commodity home router SOC's. And when I asked Eric Dumazet about fq_codel performance, he said I could quote him as it taking 2% of a modern x86 CPU core at 10G speed.  This was the point at which I got more than moderately excited about this class of algorithms. CPU's are not what they were in the 1980's and 1990's, and we have to get over that and move on.

For what concerns performance, I am thinking about the following kind of home gateway.
A triple play offer (voice, Internet, TV)  deployed using a MIPS based gateway.
It incorporates 4 Gigabit Ethernet ports, wifi @2.5GHz b/g/n, wifi @5GHz a/n,
of course USB, SATA for your local storage.
It must support FTTH and xDSL, IPv4/6 plus some software like samba, DLNA.

For instance:
http://www.ikanos.com/products/communications-processors/fusiv-vx185183/

Packet processing is distributed in hardware to let the central unit take care of other stuff.
Do not think to have a multi-core Intel processor there.

You can implement your packet scheduling in hardware (micro code, software in the end)
but it is *not* an open environment. You code your queuing now, the environment evolves
and nobody maintains your code which is broken as the micro-code evolves.

So the problem is not really software/hardware but open/closed and re-usability
(Linux matters not only because it is open but also because it is used everywhere).



So if hardware assist is not assisting, we don't have to use it, and it may encourage hardware people to "get it right" the next generation.

   2) most application's performance is now bound by latency, and not bandwidth.  Your web browsing doesn't get appreciably faster adding bandwidth, but improving the latency, even if there is not competing traffic, does improve performance, not to mention improving everything across the board and for most things getting us out of the impossible to configure QOS morass we're in.

As I said, what may be feasible or advisable to do at what point in the network may vary: I don't think this group will *mandate* much at all, but describing what can be used where for best effect is worth a lot, and *possibly* mandating a "first, do no harm" algorithm, even if far from ideal in many circumstances, to prevent no AQM being present at all, which is a headache in today's network.

We're going to have a large number of different drop and queuing algorithms over the coming years, I predict.

Yes, I hope. But if you want them deployed I suspect we'll wait to have either open hardware
or to have *true* FQ  in the chip of home gateways. Also because most of the delay is measured
in the user's home uplink. Having everything in software is easy and useful but does not cover
most of the current market.

Best regards
Luca

--------------030407030103050008050409-- From dave.taht@gmail.com Mon Jun 17 12:45:30 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8389321F9970 for ; Mon, 17 Jun 2013 12:45:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.475 X-Spam-Level: X-Spam-Status: No, score=-2.475 tagged_above=-999 required=5 tests=[AWL=-0.475, BAYES_00=-2.599, J_CHICKENPOX_23=0.6, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5J19l950s8yT for ; Mon, 17 Jun 2013 12:45:28 -0700 (PDT) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) by ietfa.amsl.com (Postfix) with ESMTP id 3FDB221F973A for ; Mon, 17 Jun 2013 12:45:28 -0700 (PDT) Received: by mail-ie0-f176.google.com with SMTP id ar20so7484447iec.7 for ; Mon, 17 Jun 2013 12:45:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=BLYXZrMFpRXJl2dTd/I0A9KBtkH1Hsm0BZRyoh8ZeGo=; b=jhWyM2f25hEHU77PCf9JpYwPYIhjEJLFIqTfZf3T2c+qRzRgikIHznnyEbCPAIxWN4 FtQ2eD4K1cZ5w4Hjaq/SEushGZB+irBVEmTjbAxrbu2pmN7VoW3HHJt0ni6d9wxQ9aid Rymn9WKUyO4wLzgE11R2WLrRt9nczLMOyDxs9ni+IMshq8e6LQtbink6tCmf1QzCWNgb vuHTSc24IdX9w2Ga3pfbNWtbhg9UDtXWiBGmhtTFl6v43gRz9vt1p+sksVLwxaVm0A1h nKE5gT0jMragKAdx7vv6BthhidXHQuDWOJuFva6yP1mblec+MOcg35jHexI9IIhFxF+C rYCg== MIME-Version: 1.0 X-Received: by 10.50.114.229 with SMTP id jj5mr5635423igb.36.1371498327669; Mon, 17 Jun 2013 12:45:27 -0700 (PDT) Received: by 10.64.45.137 with HTTP; Mon, 17 Jun 2013 12:45:27 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 Jun 2013 12:45:27 -0700 Message-ID: From: Dave Taht To: aqm@ietf.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: [aqm] some comments on draft-ietf-tsvwg-byte-pkt-congest-10.txt X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2013 19:45:30 -0000 grumble. so many mailing lists, so little time, so many fragmented conversations. ---------- Forwarded message ---------- From: Dave Taht Date: Mon, Jun 17, 2013 at 12:40 PM Subject: some comments on draft-ietf-tsvwg-byte-pkt-congest-10.txt To: tsvwg@ietf.org, bloat , iccrg@irtf.org This draft "updates" RFC2309 which has already been obsoleted by one of it's original authors and a replacement draft ( http://tools.ietf.org/html/draft-baker-aqm-recommendation-00 ) is in progress. Be that as it may... This draft starts off on the wrong foot, and proceeds downhill rapidly. I am glad that someone is trying to update BCP for their current RED usage, but in general I think it is incorrect to extrapolate from RED's behavior to other behaviors in many cases, so... " This document provides recommendations of best current practice for dropping or marking packets using any active queue management (AQM) algorithm, such as random early detection (RED), BLUE, pre-congestion notification (PCN), etc. " By excluding DRR, SFQ, SQF, Codel, FQ_Codel, PIE and others, and attempting to generalize from experiences with RED to all AQM technologies, it does its potential readers a disservice. I'd change the first sentence to: " This document provides recommendations of best current practice for dropping or marking packets using the RED active queue management (AQM) algorithm, using packet drop and congestion notification" >From that, it is possible to generalize the following, although the backing argument is suspect, which I'll get into later... " We give three strong recommendations: (1) packet size should be taken into account when transports read and respond to congestion indications, (2) packet size should not be taken into account when network equipment creates congestion signals (marking, dropping), and therefore (3) in the specific case of RED, the byte-mode packet drop variant that drops fewer small packets should not be used. " "This memo updates RFC 2309 to deprecate deliberate preferential treatment of small packets in AQM algorithms." as RFC2309 itself is being obsoleted, we're going around in circles here. Before tackling byte-pkt-congest-10 directly - a couple asides.. ...snip snip... RFC2309 refers to this 1994 paper: "On the Self-Similar Nature of Ethernet Traffic (Extended Version)" I would certainly like the model and analysis of this paper repeated against modern traffic patterns. Has anyone done this? I loved this paper when it came out.... last paragraph in section 2 of RFC2039 I heartily agree with " In short, scheduling algorithms and queue management should be seen as complementary, not as replacements for each other." And if we can agree that AQM =3D Active queue *length* management and can come up with a name for FQ+AQM hybrids that works for people (SQM - smart queue management?) so we know what we're talking about when talking about things, certain bits in section 3 get easier to deal with. Because of the overload on AQM I'm going to use SQM throughout what I write below. As one last general example of problems with rfc2039, some of the references are to protocols so ancient and non-deployed as to render the relevant arguments moot: " voice and video, and also multicast bulk data transport [SRM96]. If no action is taken, such unresponsive flows could lead to a new congestive collapse. In general, all UDP-based streaming applications should incorporate effective congestion avoidance mechanisms. For example, recent research has shown the possibility of incorporating congestion avoidance mechanisms such as Receiver- driven Layered Multicast (RLM) within UDP-based streaming applications such as packet video [McCanne96] [Bolot94]. Further research and development on ways to accomplish congestion avoidance for streaming applications will be very important." It would be nice to have relevance to new stuff like webrtc in a new draft of some sort rather than to non-deployed 20 year old protocols. ...ok back to the pkt-congest draft... " Consensus has emerged over the years concerning the first stage: if queues cannot be measured in time, whether they should be measured in bytes or packets. Section 2.1 of this memo records this consensus in the RFC Series. In summary the choice solely depends on whether the resource is congested by bytes or packets." Measuring queues in time is totally feasible as shown by pie and codel, and implementations on weak hardware such as on mips. Last week I got fq_codel running on a raspberri pi and a beaglebone to no observable hit on cpu usage vs pfifo_fast... I'll do an arduino if that's necessary to make the point, harder. Anybody got a 68020 or slower to play with? So an alternative formulation that makes sense is: " When queues cannot be measured in time, should they be measured in bytes or packets? And dropping the rest of this: " Section 2.1 of this memo records this consensus in the RFC Series. In summary the choice solely depends on whether the resource is congested by bytes or packets." Moving on: " This memo updates [RFC2309] to deprecate deliberate preferential treatment of small packets in AQM algorithms. It recommends that (1) packet size should be taken into account when transports read congestion indications, (2) not when network equipment writes them. This memo also adds to the congestion control principles enumerated in BCP 41 [RFC2914]." s/AQM/the RED/g across the draft Still... to get to the meat of my own complaint with the draft and philosophy expoused within, I'll pull out two example paragraphs and try to make my argument.... "However, at the transport layer, TCP congestion control is a widely deployed protocol that doesn't scale with packet size. To date this hasn't been a significant problem because most TCP implementations have been used with similar packet sizes. But, as we design new congestion control mechanisms, this memo recommends that we should build in scaling with packet size rather than assuming we should follow TCP's example." "Although many control packets happen to be small, the alternative of network equipment favouring all small packets would be dangerous. That would create perverse incentives to split data transfers into smaller packets." packets were (at least originally) "a small group or package of anything", I view the incentives to create larger packets as *far, far, far, far more perverse* than the incentives to create smaller ones. There will always be header overhead, there will always be small signalling packets, and there will always be bulk data that can *always* be broken up into packets of smaller size, that gets in the way of more interactive data if the larger packets aren't broken up into packets of smaller size. In my talks of late, I toss a 64GB memory stick across the room at someone - wow! check out that bandwidth! 64GB/sec!! and then challenge someone to try and read the last byte from the file on it going sequentually) (hint, most USB sticks barely do 8MB/sec and you'll be waiting a looong tim= e) If it were up to me I'd have held the internet's MTU to 574 bytes until everybody was running at greater than 1Mbit, and the ratio between the largest packet and smallest held to less than 10x1 for eternity. Big packets affect latency, badly. They are also subject to much higher potential rates of error. At 1500 bytes, header overhead, even with ipv6, is pretty minimal and few gains can be had by increasing it. Jumbo frames are barely accepted in very few circumstances, etc. So the argument and conclusions about larger packet sizes that permeates this document is the opposite of the argument I'd make, throughout. So, as one example: "6. Security Considerations This memo recommends that queues do not bias drop probability towards small packets as this creates a perverse incentive for transports to break down their flows into tiny segments. One of the benefits of implementing AQM was meant to be to remove this perverse incentive that drop-tail queues gave to small packets." I think the author has got the intended statement wrong ("away from" rather than "towards"?)... and my take on it is that the drop tail behavior towards small packets was indeed very desirable, and should be retained in an SQM, to keep latencies low on overload and create incentives for right-sizing packets in general as per their actual transport needs. "In summary, it says that making drop probability depend on the size of the packets that bits happen to be divided into simply encourages the bits to be divided into smaller packets. " YEA! " Byte-mode drop would therefore irreversibly complicate any attempt to fix the Internet's incentive structures." s/complicate/enhance/ The document refers to things like rfc5690 and I long ago lost hope for ECN= . The document then makes some blanket statements that aren't backed by data (that I'm aware of) Section 6 " In practice, transports cannot all be trusted to respond to congestion. So another reason for recommending that queues do not bias drop probability towards small packets is to avoid the vulnerability to small packet DDoS attacks that would otherwise result. One of the benefits of implementing AQM was meant to be to remove drop-tail's DoS vulnerability to small packets, so we shouldn't add it back again." I am only aware of a few small packet attacks. (most UDP DNS attacks try to do amplification actuallly) A strict DDOS based on small packets can be made effective against drop tail and somewhat effective against RED. Most attacks I've looked at are actually MUCH less effective against a SQM like fq_codel, given the random hash and equal service guarantees. I would certainly like to take a hard look at the tools used to attack and create Robust RED, so everything in section 6 seems highly theoretical and needs proof. Lastly: section 5.2 needs to be thought about in the age of wireless, which is often TXOP-congested rather than either of the two problems identified, and is kind of pressing. "5.2. Bit- & Packet-congestible Network The position is much less clear-cut if the Internet becomes populated by a more even mix of both packet-congestible and bit-congestible resources (see Appendix B.2). This problem is not pressing, because most Internet resources are designed to be bit-congestible before packet processing starts to congest (see Section 1.1)." --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html From Martin.Stiemerling@neclab.eu Thu Jun 20 13:12:09 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 12A4E21E8091 for ; Thu, 20 Jun 2013 13:12:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.334 X-Spam-Level: X-Spam-Status: No, score=-103.334 tagged_above=-999 required=5 tests=[AWL=0.038, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 77mPLj3D2CMC for ; Thu, 20 Jun 2013 13:12:04 -0700 (PDT) Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id D239421E808A for ; Thu, 20 Jun 2013 13:12:03 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 383CC1045ED for ; Thu, 20 Jun 2013 22:11:37 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de) Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AofC5lx6hTpS for ; Thu, 20 Jun 2013 22:11:37 +0200 (CEST) Received: from ENCELADUS.office.hd (enceladus.office.hd [192.168.24.52]) by mailer1.neclab.eu (Postfix) with ESMTP id 1D03B1045EC for ; Thu, 20 Jun 2013 22:11:32 +0200 (CEST) Received: from [10.7.0.105] (10.7.0.105) by skoll.office.hd (192.168.125.11) with Microsoft SMTP Server (TLS) id 14.1.323.3; Thu, 20 Jun 2013 22:09:54 +0200 Message-ID: <51C361F3.6080905@neclab.eu> Date: Thu, 20 Jun 2013 22:11:31 +0200 From: Martin Stiemerling User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130510 Thunderbird/17.0.6 MIME-Version: 1.0 To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.7.0.105] Subject: [aqm] AQM BOF approved for IETF-87 in Berlin X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 20:12:09 -0000 Dear all, The AQM BOF has been approved for the IETF-87 meeting in Berlin. The chairs for the BOF will be announced shortly. Martin -- IETF Transport Area Director martin.stiemerling@neclab.eu NEC Laboratories Europe NEC Europe Limited Registered Office: Athene, Odyssey Business Park, West End Road, London, HA4 6QE, GB Registered in England 2832014 From Martin.Stiemerling@neclab.eu Sun Jun 23 23:34:16 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A3A9321E809C for ; Sun, 23 Jun 2013 23:34:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.343 X-Spam-Level: X-Spam-Status: No, score=-103.343 tagged_above=-999 required=5 tests=[AWL=0.029, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Si05TL1kn5ZF for ; Sun, 23 Jun 2013 23:34:11 -0700 (PDT) Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id 5308221E8092 for ; Sun, 23 Jun 2013 23:34:11 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 8397E10462F for ; Mon, 24 Jun 2013 08:33:46 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de) Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i4nT0-NZk0N9 for ; Mon, 24 Jun 2013 08:33:46 +0200 (CEST) Received: from METHONE.office.hd (methone.office.hd [192.168.24.54]) by mailer1.neclab.eu (Postfix) with ESMTP id 6C83F104558 for ; Mon, 24 Jun 2013 08:33:41 +0200 (CEST) Received: from [10.1.1.190] (10.1.1.190) by skoll.office.hd (192.168.125.11) with Microsoft SMTP Server (TLS) id 14.1.323.3; Mon, 24 Jun 2013 08:33:43 +0200 Message-ID: <51C7E847.3090500@neclab.eu> Date: Mon, 24 Jun 2013 08:33:43 +0200 From: Martin Stiemerling User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130510 Thunderbird/17.0.6 MIME-Version: 1.0 To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.1.1.190] Subject: [aqm] AQM BOF Chairs X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2013 06:34:16 -0000 Dear all, Wes Eddy (wes@mti-systems.com) and Richard Scheffenegger (rs@netapp.com) have agreed to be the chairs of the AQM BOF. Thanks to Wes and Richard! Martin -- IETF Transport Area Director martin.stiemerling@neclab.eu NEC Laboratories Europe NEC Europe Limited Registered Office: Athene, Odyssey Business Park, West End Road, London, HA4 6QE, GB Registered in England 2832014