
From bob.briscoe@bt.com  Thu Dec  5 10:13:18 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16EA01ADF8A for <aqm@ietfa.amsl.com>; Thu,  5 Dec 2013 10:13:18 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.602
X-Spam-Level: 
X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iP67g3Eef2IE for <aqm@ietfa.amsl.com>; Thu,  5 Dec 2013 10:13:13 -0800 (PST)
Received: from hubrelay-by-03.bt.com (hubrelay-by-03.bt.com [62.7.242.139]) by ietfa.amsl.com (Postfix) with ESMTP id A303D1AE046 for <aqm@ietf.org>; Thu,  5 Dec 2013 10:13:13 -0800 (PST)
Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR03-UKBR.bt.com (10.216.161.35) with Microsoft SMTP Server (TLS) id 8.3.327.1; Thu, 5 Dec 2013 18:13:05 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Thu, 5 Dec 2013 18:13:07 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Thu, 5 Dec 2013 18:13:06 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rB5ID6rp004006; Thu, 5 Dec 2013 18:13:06 GMT
Message-ID: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Thu, 5 Dec 2013 18:13:05 +0000
To: Fred Baker <fred@cisco.com>, Gorry Fairhurst <gorry@erg.abdn.ac.uk>
From: Bob Briscoe <bob.briscoe@bt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: AQM IETF list <aqm@ietf.org>
Subject: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Dec 2013 18:13:18 -0000

Fred, Gorry, all,

I promised to suggest text for draft-ietf-aqm-recommendation about 
allowing the AQM's behaviour to be independent for ECN and non-ECN 
packets. In the process, I realised we can't talk about independent 
AQMs for ECN without also including Diffserv.

This gets messy, because I believe a good AQM for BE traffic with and 
without ECN, should remove much if not all the need for Diffserv. But 
we can't ignore Diffserv.

_________________________________________________________________________________________
{In Section 4: add another bullet between recommendations 2 & 3:}

3{New}. It SHOULD be possible to make different instances of an AQM 
algorithm apply to different subsets of packets that share the same 
queue. It SHOULD be possible to classify packets into these subsets 
at least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] 
(or the equivalent of these fields at lower layers).

{Then a new section to expand on this before the current Section 4.3.}
4.3{New}. Independent AQM Instances for ECN and Diffserv

The recommendation to provide a separate instance of the AQM for ECN 
packets goes beyond the assumptions of RFC 3168, which assumed that 
only one instance of an AQM will handle both ECN-capable and 
non-ECN-capable packets.



Bob



________________________________________________________________
Bob Briscoe,                                                  BT  


From fred@cisco.com  Thu Dec  5 11:50:05 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A5BC1AE170 for <aqm@ietfa.amsl.com>; Thu,  5 Dec 2013 11:50:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -114.502
X-Spam-Level: 
X-Spam-Status: No, score=-114.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FOB7UdOe72iA for <aqm@ietfa.amsl.com>; Thu,  5 Dec 2013 11:50:03 -0800 (PST)
Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) by ietfa.amsl.com (Postfix) with ESMTP id 8C0FF1AE142 for <aqm@ietf.org>; Thu,  5 Dec 2013 11:50:03 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2520; q=dns/txt; s=iport; t=1386273000; x=1387482600; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=Z5eSOVCv3aD0jyZKYEEszo6u0AmKZBjLmKiyUGvg3rI=; b=RBmDBLm0gs3Hg1iAlqmL9K9gtjNtjFAd2gBg/mHdoSPumptuWb3yyUPx F2lI3xmL+R8tFzRw9figVGNGWDiKKzyxLnCIr85xhjzR2vbOyuoJVOli9 GtY7DXErQfvXMkOPQvlCGwoUxNke4nBoz/qKQwIgPYi7bbmjcIlOi204B 4=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhsFAGHYoFKtJV2b/2dsb2JhbABZDoJ5OFO4bYEcFnSCJQEBAQMBeQULAgEIRjIlAgQOBQ6HbgbBVxePAAeDIIETA5AxgTGGMoEwkGOCaj+CKg
X-IronPort-AV: E=Sophos;i="4.93,835,1378857600";  d="asc'?scan'208";a="289471610"
Received: from rcdn-core-4.cisco.com ([173.37.93.155]) by rcdn-iport-1.cisco.com with ESMTP; 05 Dec 2013 19:50:00 +0000
Received: from xhc-aln-x08.cisco.com (xhc-aln-x08.cisco.com [173.36.12.82]) by rcdn-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id rB5JnxTn013175 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 5 Dec 2013 19:49:59 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.118]) by xhc-aln-x08.cisco.com ([173.36.12.82]) with mapi id 14.03.0123.003; Thu, 5 Dec 2013 13:49:59 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Gorry Fairhurst <gorry@erg.abdn.ac.uk>
Thread-Topic: Text for aqm-recommendation on independent ECN config
Thread-Index: AQHO8fMu9F2X1bXuSEuErfZnwMRong==
Date: Thu, 5 Dec 2013 19:49:59 +0000
Message-ID: <48668A10-16D1-448D-BBDF-4F395E62B892@cisco.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk>
In-Reply-To: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.123]
Content-Type: multipart/signed; boundary="Apple-Mail=_5336F2E5-A5A5-4499-B5A1-6288D27DAD3D"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: AQM IETF list <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Dec 2013 19:50:05 -0000

--Apple-Mail=_5336F2E5-A5A5-4499-B5A1-6288D27DAD3D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Thanks.

Gorry, Matt Mathis also said he would be putting together some notes.=20

On Dec 5, 2013, at 10:13 AM, Bob Briscoe <bob.briscoe@bt.com>
 wrote:

> Fred, Gorry, all,
>=20
> I promised to suggest text for draft-ietf-aqm-recommendation about =
allowing the AQM's behaviour to be independent for ECN and non-ECN =
packets. In the process, I realised we can't talk about independent AQMs =
for ECN without also including Diffserv.
>=20
> This gets messy, because I believe a good AQM for BE traffic with and =
without ECN, should remove much if not all the need for Diffserv. But we =
can't ignore Diffserv.
>=20
> =
__________________________________________________________________________=
_______________
> {In Section 4: add another bullet between recommendations 2 & 3:}
>=20
> 3{New}. It SHOULD be possible to make different instances of an AQM =
algorithm apply to different subsets of packets that share the same =
queue. It SHOULD be possible to classify packets into these subsets at =
least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or =
the equivalent of these fields at lower layers).
>=20
> {Then a new section to expand on this before the current Section 4.3.}
> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>=20
> The recommendation to provide a separate instance of the AQM for ECN =
packets goes beyond the assumptions of RFC 3168, which assumed that only =
one instance of an AQM will handle both ECN-capable and non-ECN-capable =
packets.
>=20
>=20
>=20
> Bob
>=20
>=20
>=20
> ________________________________________________________________
> Bob Briscoe,                                                  BT =20

------------------------------------------------------
8 issues in virtual infrastructure
http://dcrocker.net/#fallacies


--Apple-Mail=_5336F2E5-A5A5-4499-B5A1-6288D27DAD3D
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSoNjmbjEdbHIsm0MRAjsoAJ9aKmeUFTN5SGO4j+OWmk8OSYzoFACgzjWC
UAWoR4HoCb/UQPqsPug7COs=
=EO4s
-----END PGP SIGNATURE-----

--Apple-Mail=_5336F2E5-A5A5-4499-B5A1-6288D27DAD3D--

From gettysjim@gmail.com  Thu Dec  5 11:50:30 2013
Return-Path: <gettysjim@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C5A9C1AE16A for <aqm@ietfa.amsl.com>; Thu,  5 Dec 2013 11:50:30 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.277
X-Spam-Level: 
X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zBt8EXBiTl2d for <aqm@ietfa.amsl.com>; Thu,  5 Dec 2013 11:50:27 -0800 (PST)
Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) by ietfa.amsl.com (Postfix) with ESMTP id 6AE6E1AE142 for <aqm@ietf.org>; Thu,  5 Dec 2013 11:50:27 -0800 (PST)
Received: by mail-wi0-f174.google.com with SMTP id z2so209081wiv.13 for <aqm@ietf.org>; Thu, 05 Dec 2013 11:50:23 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=VKTKUUOV6NTAeYj7SrZESIHSZLXOFf25i3kMZwHNSXs=; b=bR+s9S6NQGNsUGGAXofQ5u+sKc/Ed+iGOBOrluw22h0yaUqvm2ihWzFcWHcL3tHIE5 K3lNhcLXszVsAyzROSslGujVYDW3Ol90IECh/OkDKdpkoP/1/lyO+2u4R02zKTdYSwQu 6B93UYwmAx9CRlZAMx48O+gAVAsZT1RPggev6ZXEPkB0MJ7W/LQtTpTx38Ba4mQLqi4w VY4GetMazQYPJJnlIP+z0xRTaOjt+Y88s3AvbQ+HWn2QsVeCn5jbp9tz5fnRr+AKkBys /v2BgAZ2UnJ9e86nU5xrtQg6KYjPSH8zSmuFroxz9XAOQ1tdmNuSNdJzoKW83PafrMjn VFiQ==
MIME-Version: 1.0
X-Received: by 10.180.14.134 with SMTP id p6mr13446856wic.6.1386273023175; Thu, 05 Dec 2013 11:50:23 -0800 (PST)
Sender: gettysjim@gmail.com
Received: by 10.227.198.66 with HTTP; Thu, 5 Dec 2013 11:50:23 -0800 (PST)
In-Reply-To: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk>
Date: Thu, 5 Dec 2013 14:50:23 -0500
X-Google-Sender-Auth: S7iRVSpsUmfonm5g5pOM86dOVBQ
Message-ID: <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com>
From: Jim Gettys <jg@freedesktop.org>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: multipart/alternative; boundary=f46d041554181bb5d204ecced70c
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Dec 2013 19:50:31 -0000

--f46d041554181bb5d204ecced70c
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:

> Fred, Gorry, all,
>
> I promised to suggest text for draft-ietf-aqm-recommendation about
> allowing the AQM's behaviour to be independent for ECN and non-ECN packets.
> In the process, I realised we can't talk about independent AQMs for ECN
> without also including Diffserv.
>
> This gets messy, because I believe a good AQM for BE traffic with and
> without ECN, should remove much if not all the need for Diffserv. But we
> can't ignore Diffserv.
>

I agree in principle with what Bob is trying to say here (and is very much
what I've been saying in my blog entry of last summer).

Once you have things under control, the need for Diffserv diminishes
dramatically (but does not go away).

But as Bob notes, there is still a good use for Diffserv: suitably marked
traffic may want to contend for access to the channel differently: your
marked VOIP packets may want to change the priority with which you request
channel access, so that you get more timely access to the medium. This
conserves transmit opportunities, which is often the scarcest resource in
many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
between your VOIP working well, and not working well, on a busy 802.11
network as well as using the channel as efficiently as possible.

Similarly, if you have packets you know are background, it's helpful to
know that to ensure that they never contend for access to the medium but
will always defer to other traffic, and just scavenge available space in
other transmit opportunities where possible.

I'm a bit loathe though to tie the behavior to queues, however; in
particular, best effort traffic may want to be sent in the same aggregate
as higher (or lower) priority traffic, if there is remaining space in the
aggregate.

In short, the mental model we've had that there is a one-to-one model of
hardware and software queues (not to mention flows in a given software
queue) is often incorrect (or at least seriously sub-optimal) in today's
systems (even if the hardware queues "work" properly, which it appears they
do not in 802.11).

So I'm not sure Bob's new section 3 here is how to best to state this (or
to deal with the terminology problem).  Certainly, it may be the same
instance of an AQM algorithm, rather than different instances, for example.
 And "

It SHOULD be possible" is more a pious wish than anything else.  But I
agree in spirit with what Bob's trying to say.
                               - Jim


> ____________________________________________________________
> _____________________________
> {In Section 4: add another bullet between recommendations 2 & 3:}
>
> 3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue.
> 
> 
> It SHOULD be possible to classify packets into these subsets at least by
> ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent
> of these fields at lower layers).
>
> {Then a new section to expand on this before the current Section 4.3.}
> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>
> The recommendation to provide a separate instance of the AQM for ECN
> packets goes beyond the assumptions of RFC 3168, which assumed that only
> one instance of an AQM will handle both ECN-capable and non-ECN-capable
> packets.
>
>
>
> Bob
>
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>

--f46d041554181bb5d204ecced70c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:small"><br=
></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Thu=
, Dec 5, 2013 at 1:13 PM, Bob Briscoe <span dir=3D"ltr">&lt;<a href=3D"mail=
to:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.com</a>&gt;</span> =
wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">Fred, Gorry, all,<br>
<br>
I promised to suggest text for draft-ietf-aqm-recommendation about allowing=
 the AQM&#39;s behaviour to be independent for ECN and non-ECN packets. In =
the process, I realised we can&#39;t talk about independent AQMs for ECN wi=
thout also including Diffserv.<br>

<br>
This gets messy, because I believe a good AQM for BE traffic with and witho=
ut ECN, should remove much if not all the need for Diffserv. But we can&#39=
;t ignore Diffserv.<br></blockquote><div><br></div><div class=3D"gmail_defa=
ult" style=3D"font-size:small">
I agree in principle with what Bob is trying to say here (and is very much =
what I&#39;ve been saying in my blog entry of last summer).</div><div class=
=3D"gmail_default" style=3D"font-size:small"><br></div><div class=3D"gmail_=
default" style=3D"font-size:small">
Once you have things under control, the need for Diffserv diminishes dramat=
ically (but does not go away).</div><div class=3D"gmail_default" style=3D"f=
ont-size:small"><br></div><div class=3D"gmail_default" style=3D"font-size:s=
mall">
But as Bob notes, there is still a good use for Diffserv: suitably marked t=
raffic may want to contend for access to the channel differently: your mark=
ed VOIP packets may want to change the priority with which you request chan=
nel access, so that you get more timely access to the medium. This conserve=
s transmit opportunities, which is often the scarcest resource in many syst=
ems (e.g. 802.11, DOCSIS, etc.). This can be the difference between your VO=
IP working well, and not working well, on a busy 802.11 network as well as =
using the channel as efficiently as possible.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Similarly, if you have packets=
 you know are background, it&#39;s helpful to know that to ensure that they=
 never contend for access to the medium but will always defer to other traf=
fic, and just scavenge available space in other transmit opportunities wher=
e possible.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">I&#39;m a bit loathe though to=
 tie the behavior to queues, however; in particular, best effort traffic ma=
y want to be sent in the same aggregate as higher (or lower) priority traff=
ic, if there is remaining space in the aggregate. =A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">In short, the mental model we&=
#39;ve had that there is a one-to-one model of hardware and software queues=
 (not to mention flows in a given software queue) is often incorrect (or at=
 least seriously sub-optimal) in today&#39;s systems (even if the hardware =
queues &quot;work&quot; properly, which it appears they do not in 802.11).<=
/div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">So I&#39;m not sure Bob&#39;s =
new section 3 here is how to best to state this (or to deal with the termin=
ology problem). =A0Certainly, it may be the same instance of an AQM algorit=
hm, rather than different instances, for example. =A0And &quot;<div class=
=3D"gmail_default" style=3D"display:inline">
</div>It SHOULD be possible&quot; is more a pious wish than anything else. =
=A0But I agree in spirit with what Bob&#39;s trying to say.</div><div class=
=3D"gmail_default" style=3D"font-size:small">=A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0- Jim</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width=
:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-lef=
t:1ex">

<br>
______________________________<u></u>______________________________<u></u>_=
____________________________<br>
{In Section 4: add another bullet between recommendations 2 &amp; 3:}<br>
<br>
3{New}. It SHOULD be possible to make different instances of an AQM algorit=
hm apply to different subsets of packets that share the same queue. <div cl=
ass=3D"gmail_default" style=3D"font-size:small;display:inline"></div><div c=
lass=3D"gmail_default" style=3D"font-size:small;display:inline">
</div>It SHOULD be possible to classify packets into these subsets at least=
 by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equiva=
lent of these fields at lower layers).<br>
<br>
{Then a new section to expand on this before the current Section 4.3.}<br>
4.3{New}. Independent AQM Instances for ECN and Diffserv<br>
<br>
The recommendation to provide a separate instance of the AQM for ECN packet=
s goes beyond the assumptions of RFC 3168, which assumed that only one inst=
ance of an AQM will handle both ECN-capable and non-ECN-capable packets.<br=
>

<br>
<br>
<br>
Bob<br>
<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>_=
___<br>
Bob Briscoe, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BT =A0<br>
______________________________<u></u>_________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/<u></u>listinfo/aqm</a><br>
</blockquote></div><br></div></div>

--f46d041554181bb5d204ecced70c--

From gorry@erg.abdn.ac.uk  Fri Dec  6 02:46:29 2013
Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E92FA1ADDCA for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 02:46:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.202
X-Spam-Level: 
X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DoOWZ1XMP9xP for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 02:46:27 -0800 (PST)
Received: from spey.erg.abdn.ac.uk (spey.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 51BAD1ADEA3 for <aqm@ietf.org>; Fri,  6 Dec 2013 02:46:27 -0800 (PST)
Received: from www.erg.abdn.ac.uk (blake.erg.abdn.ac.uk [139.133.210.30]) by spey.erg.abdn.ac.uk (Postfix) with ESMTPSA id 052472B44A6; Fri,  6 Dec 2013 10:46:21 +0000 (GMT)
Received: from 212.159.18.54 (SquirrelMail authenticated user gorry) by www.erg.abdn.ac.uk with HTTP; Fri, 6 Dec 2013 10:46:21 -0000
Message-ID: <8b37ee098a7edbecfd944e7ce1d37e23.squirrel@www.erg.abdn.ac.uk>
In-Reply-To: <48668A10-16D1-448D-BBDF-4F395E62B892@cisco.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <48668A10-16D1-448D-BBDF-4F395E62B892@cisco.com>
Date: Fri, 6 Dec 2013 10:46:21 -0000
From: gorry@erg.abdn.ac.uk
To: "Fred Baker (fred)" <fred@cisco.com>
User-Agent: SquirrelMail/1.4.22
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Dec 2013 10:46:30 -0000

OK, I'll check through the AQM thread before the end of next week and
summarise what I think has been commented and what we have agreed to
change.

I think the latest discussion on requirements that change diffserv/ECN may
need a strong consensus - so it would be good to understand this.

Gorry

> Thanks.
>
> Gorry, Matt Mathis also said he would be putting together some notes.
>
> On Dec 5, 2013, at 10:13 AM, Bob Briscoe <bob.briscoe@bt.com>
>  wrote:
>
>> Fred, Gorry, all,
>>
>> I promised to suggest text for draft-ietf-aqm-recommendation about
>> allowing the AQM's behaviour to be independent for ECN and non-ECN
>> packets. In the process, I realised we can't talk about independent AQMs
>> for ECN without also including Diffserv.
>>
>> This gets messy, because I believe a good AQM for BE traffic with and
>> without ECN, should remove much if not all the need for Diffserv. But we
>> can't ignore Diffserv.
>>
>> _________________________________________________________________________________________
>> {In Section 4: add another bullet between recommendations 2 & 3:}
>>
>> 3{New}. It SHOULD be possible to make different instances of an AQM
>> algorithm apply to different subsets of packets that share the same
>> queue. It SHOULD be possible to classify packets into these subsets at
>> least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or
>> the equivalent of these fields at lower layers).
>>
>> {Then a new section to expand on this before the current Section 4.3.}
>> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>>
>> The recommendation to provide a separate instance of the AQM for ECN
>> packets goes beyond the assumptions of RFC 3168, which assumed that only
>> one instance of an AQM will handle both ECN-capable and non-ECN-capable
>> packets.
>>
>>
>>
>> Bob
>>
>>
>>
>> ________________________________________________________________
>> Bob Briscoe,                                                  BT
>
> ------------------------------------------------------
> 8 issues in virtual infrastructure
> http://dcrocker.net/#fallacies
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



From dave.c.robinson@alcatel-lucent.com  Fri Dec  6 07:23:06 2013
Return-Path: <dave.c.robinson@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1864B1ADFCA for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 07:23:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.899
X-Spam-Level: 
X-Spam-Status: No, score=-6.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dKnDsniiurcu for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 07:23:02 -0800 (PST)
Received: from ihemail1.lucent.com (ihemail1.lucent.com [135.245.0.33]) by ietfa.amsl.com (Postfix) with ESMTP id 3DBF31AC863 for <aqm@ietf.org>; Fri,  6 Dec 2013 07:23:02 -0800 (PST)
Received: from fr711usmtp1.zeu.alcatel-lucent.com (h135-239-2-122.lucent.com [135.239.2.122]) by ihemail1.lucent.com (8.13.8/IER-o) with ESMTP id rB6FMh26016849 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 6 Dec 2013 09:22:45 -0600 (CST)
Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id rB6FMfVP000887 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 6 Dec 2013 16:22:41 +0100
Received: from FR711WXCHMBA04.zeu.alcatel-lucent.com ([169.254.4.132]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.02.0247.003; Fri, 6 Dec 2013 16:22:41 +0100
From: "Robinson, Dave C (Dave)" <dave.c.robinson@alcatel-lucent.com>
To: Jim Gettys <jg@freedesktop.org>
Thread-Topic: [aqm] Text for aqm-recommendation on independent ECN config
Thread-Index: Ac7ylwDy28ysG4wgQSye96xp0R2A+g==
Date: Fri, 6 Dec 2013 15:22:41 +0000
Message-ID: <1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com>
In-Reply-To: <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-GB
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Content-Type: multipart/alternative; boundary="_000_1419030F827F4132BE5DBC74264CE46Calcatellucentcom_"
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.33
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Dec 2013 15:23:06 -0000

--_000_1419030F827F4132BE5DBC74264CE46Calcatellucentcom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SmltIGV0IGFsLA0KDQpJIGFncmVlIHdpdGggeW91ciBjb21tZW50cyBvbiBEaWZmc2Vydi4gSSdk
IGdvIGZ1cnRoZXIgYW5kIGNoYXJhY3RlcmlzZSBEaWZmc2VydiBhcyBhIGJ1c2luZXNzIHRvb2wg
d2hlcmVhcyBFQ04gYW5kIGdlbmVyYWwgQVFNcyBhcyBuZXR3b3JrIG9wdGltaXNhdGlvbiB0b29s
cy4gQnkgdGhhdCBJIG1lYW4gdGhhdCBEaWZmc2VydiBzaG91bGQgYmUgdXNlZCBieSBhbiBhcHBs
aWNhdGlvbiB0byBzcGVjaWZ5IGEgYnVzaW5lc3MgcG9saWN5IGJlIGl0IGxvdyBsYXRlbmN5IGZv
ciBWb0lQIC8gZ2FtaW5nIG9yIHByZWZlcnJlZCBmb3J3YXJkaW5nIGZvciB5b3VyIHZpZGVvIHNl
cnZpY2UuIEFRTXMgY2FuIGd1ZXNzIGF0IHRoZSB0eXBlcyBvZiB0cmFmZmljLiBGb3IgZXhhbXBs
ZSBhbiBpc29jaHJvbm91cyBmbG93IG9mIHNob3J0IHBhY2tldHMgY291bGQgd2VsbCBiZSBWb0lQ
IHdoaWNoIHdvdWxkIGJlbmVmaXQgZnJvbSBleHBlZGl0ZWQgZm9yd2FyZGluZy4gQnV0IGl0IGhh
cyB0byBiZSBhbiBvcGVyYXRvciBwb2xpY3kgdG8gZGVjaWRlIGJldHdlZW4gdHdvIFZvSVAgc2Vy
dmljZXMuDQoNClNhbWUgYXJndW1lbnQgZm9yIGJ1bGsgdHJhZmZpYyBmcm9tIGRpZmZlcmVudCB2
aWRlbyBzZXJ2aWNlcy4gT25lIGNhbiBiZSBhIHBhcnRuZXIgdHJhZmZpYyBmcm9tIHlvdXIgaW50
ZXJuYWwgQ0ROIG5vZGVzIHdoaWNoIHlvdSB3b3VsZCBsaWtlIHRvIGFjY2VsZXJhdGUgdGhyb3Vn
aCB5b3VyIG5ldHdvcmsuDQoNCkVDTiBpcyBhIGdvb2QgdG9vbHMgdG8gdXNlIHRvIHRlbGwgc291
cmNlcyB0aGUgdGhlIG5ldHdvcmsgaXMgYmVjb21pbmcgb3ZlcmxvYWRlZCBhbmQgaXMga2luZGVy
IHRoYW4gZHJvcHBpbmcgcGFja2V0cy4gQnV0IGl0IGNhbm5vdCBrbm93IHRoZSBidXNpbmVzcyBw
b2xpY3kuIEp1c3QgaGVscCBtYW5hZ2UgY29uZ2VzdGlvbi4NCg0KUmVnYXJkcw0KICAgICAgRGF2
ZQ0KDQpTZW50IGZyb20gbXkgaVBhZA0KDQpPbiA1IERlYyAyMDEzLCBhdCAxOTo1MCwgSmltIEdl
dHR5cyA8amdAZnJlZWRlc2t0b3Aub3JnPG1haWx0bzpqZ0BmcmVlZGVza3RvcC5vcmc+PiB3cm90
ZToNCg0KDQoNCg0KT24gVGh1LCBEZWMgNSwgMjAxMyBhdCAxOjEzIFBNLCBCb2IgQnJpc2NvZSA8
Ym9iLmJyaXNjb2VAYnQuY29tPG1haWx0bzpib2IuYnJpc2NvZUBidC5jb20+PiB3cm90ZToNCkZy
ZWQsIEdvcnJ5LCBhbGwsDQoNCkkgcHJvbWlzZWQgdG8gc3VnZ2VzdCB0ZXh0IGZvciBkcmFmdC1p
ZXRmLWFxbS1yZWNvbW1lbmRhdGlvbiBhYm91dCBhbGxvd2luZyB0aGUgQVFNJ3MgYmVoYXZpb3Vy
IHRvIGJlIGluZGVwZW5kZW50IGZvciBFQ04gYW5kIG5vbi1FQ04gcGFja2V0cy4gSW4gdGhlIHBy
b2Nlc3MsIEkgcmVhbGlzZWQgd2UgY2FuJ3QgdGFsayBhYm91dCBpbmRlcGVuZGVudCBBUU1zIGZv
ciBFQ04gd2l0aG91dCBhbHNvIGluY2x1ZGluZyBEaWZmc2Vydi4NCg0KVGhpcyBnZXRzIG1lc3N5
LCBiZWNhdXNlIEkgYmVsaWV2ZSBhIGdvb2QgQVFNIGZvciBCRSB0cmFmZmljIHdpdGggYW5kIHdp
dGhvdXQgRUNOLCBzaG91bGQgcmVtb3ZlIG11Y2ggaWYgbm90IGFsbCB0aGUgbmVlZCBmb3IgRGlm
ZnNlcnYuIEJ1dCB3ZSBjYW4ndCBpZ25vcmUgRGlmZnNlcnYuDQoNCkkgYWdyZWUgaW4gcHJpbmNp
cGxlIHdpdGggd2hhdCBCb2IgaXMgdHJ5aW5nIHRvIHNheSBoZXJlIChhbmQgaXMgdmVyeSBtdWNo
IHdoYXQgSSd2ZSBiZWVuIHNheWluZyBpbiBteSBibG9nIGVudHJ5IG9mIGxhc3Qgc3VtbWVyKS4N
Cg0KT25jZSB5b3UgaGF2ZSB0aGluZ3MgdW5kZXIgY29udHJvbCwgdGhlIG5lZWQgZm9yIERpZmZz
ZXJ2IGRpbWluaXNoZXMgZHJhbWF0aWNhbGx5IChidXQgZG9lcyBub3QgZ28gYXdheSkuDQoNCkJ1
dCBhcyBCb2Igbm90ZXMsIHRoZXJlIGlzIHN0aWxsIGEgZ29vZCB1c2UgZm9yIERpZmZzZXJ2OiBz
dWl0YWJseSBtYXJrZWQgdHJhZmZpYyBtYXkgd2FudCB0byBjb250ZW5kIGZvciBhY2Nlc3MgdG8g
dGhlIGNoYW5uZWwgZGlmZmVyZW50bHk6IHlvdXIgbWFya2VkIFZPSVAgcGFja2V0cyBtYXkgd2Fu
dCB0byBjaGFuZ2UgdGhlIHByaW9yaXR5IHdpdGggd2hpY2ggeW91IHJlcXVlc3QgY2hhbm5lbCBh
Y2Nlc3MsIHNvIHRoYXQgeW91IGdldCBtb3JlIHRpbWVseSBhY2Nlc3MgdG8gdGhlIG1lZGl1bS4g
VGhpcyBjb25zZXJ2ZXMgdHJhbnNtaXQgb3Bwb3J0dW5pdGllcywgd2hpY2ggaXMgb2Z0ZW4gdGhl
IHNjYXJjZXN0IHJlc291cmNlIGluIG1hbnkgc3lzdGVtcyAoZS5nLiA4MDIuMTEsIERPQ1NJUywg
ZXRjLikuIFRoaXMgY2FuIGJlIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4geW91ciBWT0lQIHdvcmtp
bmcgd2VsbCwgYW5kIG5vdCB3b3JraW5nIHdlbGwsIG9uIGEgYnVzeSA4MDIuMTEgbmV0d29yayBh
cyB3ZWxsIGFzIHVzaW5nIHRoZSBjaGFubmVsIGFzIGVmZmljaWVudGx5IGFzIHBvc3NpYmxlLg0K
DQpTaW1pbGFybHksIGlmIHlvdSBoYXZlIHBhY2tldHMgeW91IGtub3cgYXJlIGJhY2tncm91bmQs
IGl0J3MgaGVscGZ1bCB0byBrbm93IHRoYXQgdG8gZW5zdXJlIHRoYXQgdGhleSBuZXZlciBjb250
ZW5kIGZvciBhY2Nlc3MgdG8gdGhlIG1lZGl1bSBidXQgd2lsbCBhbHdheXMgZGVmZXIgdG8gb3Ro
ZXIgdHJhZmZpYywgYW5kIGp1c3Qgc2NhdmVuZ2UgYXZhaWxhYmxlIHNwYWNlIGluIG90aGVyIHRy
YW5zbWl0IG9wcG9ydHVuaXRpZXMgd2hlcmUgcG9zc2libGUuDQoNCkknbSBhIGJpdCBsb2F0aGUg
dGhvdWdoIHRvIHRpZSB0aGUgYmVoYXZpb3IgdG8gcXVldWVzLCBob3dldmVyOyBpbiBwYXJ0aWN1
bGFyLCBiZXN0IGVmZm9ydCB0cmFmZmljIG1heSB3YW50IHRvIGJlIHNlbnQgaW4gdGhlIHNhbWUg
YWdncmVnYXRlIGFzIGhpZ2hlciAob3IgbG93ZXIpIHByaW9yaXR5IHRyYWZmaWMsIGlmIHRoZXJl
IGlzIHJlbWFpbmluZyBzcGFjZSBpbiB0aGUgYWdncmVnYXRlLg0KDQpJbiBzaG9ydCwgdGhlIG1l
bnRhbCBtb2RlbCB3ZSd2ZSBoYWQgdGhhdCB0aGVyZSBpcyBhIG9uZS10by1vbmUgbW9kZWwgb2Yg
aGFyZHdhcmUgYW5kIHNvZnR3YXJlIHF1ZXVlcyAobm90IHRvIG1lbnRpb24gZmxvd3MgaW4gYSBn
aXZlbiBzb2Z0d2FyZSBxdWV1ZSkgaXMgb2Z0ZW4gaW5jb3JyZWN0IChvciBhdCBsZWFzdCBzZXJp
b3VzbHkgc3ViLW9wdGltYWwpIGluIHRvZGF5J3Mgc3lzdGVtcyAoZXZlbiBpZiB0aGUgaGFyZHdh
cmUgcXVldWVzICJ3b3JrIiBwcm9wZXJseSwgd2hpY2ggaXQgYXBwZWFycyB0aGV5IGRvIG5vdCBp
biA4MDIuMTEpLg0KDQpTbyBJJ20gbm90IHN1cmUgQm9iJ3MgbmV3IHNlY3Rpb24gMyBoZXJlIGlz
IGhvdyB0byBiZXN0IHRvIHN0YXRlIHRoaXMgKG9yIHRvIGRlYWwgd2l0aCB0aGUgdGVybWlub2xv
Z3kgcHJvYmxlbSkuICBDZXJ0YWlubHksIGl0IG1heSBiZSB0aGUgc2FtZSBpbnN0YW5jZSBvZiBh
biBBUU0gYWxnb3JpdGhtLCByYXRoZXIgdGhhbiBkaWZmZXJlbnQgaW5zdGFuY2VzLCBmb3IgZXhh
bXBsZS4gIEFuZCAiDQpJdCBTSE9VTEQgYmUgcG9zc2libGUiIGlzIG1vcmUgYSBwaW91cyB3aXNo
IHRoYW4gYW55dGhpbmcgZWxzZS4gIEJ1dCBJIGFncmVlIGluIHNwaXJpdCB3aXRoIHdoYXQgQm9i
J3MgdHJ5aW5nIHRvIHNheS4NCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtIEppbQ0K
DQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQp7SW4gU2VjdGlvbiA0OiBhZGQg
YW5vdGhlciBidWxsZXQgYmV0d2VlbiByZWNvbW1lbmRhdGlvbnMgMiAmIDM6fQ0KDQoze05ld30u
IEl0IFNIT1VMRCBiZSBwb3NzaWJsZSB0byBtYWtlIGRpZmZlcmVudCBpbnN0YW5jZXMgb2YgYW4g
QVFNIGFsZ29yaXRobSBhcHBseSB0byBkaWZmZXJlbnQgc3Vic2V0cyBvZiBwYWNrZXRzIHRoYXQg
c2hhcmUgdGhlIHNhbWUgcXVldWUuDQpJdCBTSE9VTEQgYmUgcG9zc2libGUgdG8gY2xhc3NpZnkg
cGFja2V0cyBpbnRvIHRoZXNlIHN1YnNldHMgYXQgbGVhc3QgYnkgRUNOIGNvZGVwb2ludCBbUkZD
MzE2OF0gYW5kIERpZmZzZXJ2IGNvZGVwb2ludCBbUkZDMjQ3NF0gKG9yIHRoZSBlcXVpdmFsZW50
IG9mIHRoZXNlIGZpZWxkcyBhdCBsb3dlciBsYXllcnMpLg0KDQp7VGhlbiBhIG5ldyBzZWN0aW9u
IHRvIGV4cGFuZCBvbiB0aGlzIGJlZm9yZSB0aGUgY3VycmVudCBTZWN0aW9uIDQuMy59DQo0LjN7
TmV3fS4gSW5kZXBlbmRlbnQgQVFNIEluc3RhbmNlcyBmb3IgRUNOIGFuZCBEaWZmc2Vydg0KDQpU
aGUgcmVjb21tZW5kYXRpb24gdG8gcHJvdmlkZSBhIHNlcGFyYXRlIGluc3RhbmNlIG9mIHRoZSBB
UU0gZm9yIEVDTiBwYWNrZXRzIGdvZXMgYmV5b25kIHRoZSBhc3N1bXB0aW9ucyBvZiBSRkMgMzE2
OCwgd2hpY2ggYXNzdW1lZCB0aGF0IG9ubHkgb25lIGluc3RhbmNlIG9mIGFuIEFRTSB3aWxsIGhh
bmRsZSBib3RoIEVDTi1jYXBhYmxlIGFuZCBub24tRUNOLWNhcGFibGUgcGFja2V0cy4NCg0KDQoN
CkJvYg0KDQoNCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fXw0KQm9iIEJyaXNjb2UsICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBCVA0KX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX18NCmFxbSBtYWlsaW5nIGxpc3QNCmFxbUBpZXRmLm9yZzxtYWls
dG86YXFtQGlldGYub3JnPg0KaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9h
cW0NCg0K

--_000_1419030F827F4132BE5DBC74264CE46Calcatellucentcom_
Content-Type: text/html; charset="utf-8"
Content-ID: <A72A298AD59CA64F8092F2FFE67CD135@exchange.lucent.com>
Content-Transfer-Encoding: base64

PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i
dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IGRpcj0iYXV0byI+DQo8
ZGl2PkppbSBldCBhbCw8L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2PkkgYWdyZWUgd2l0
aCB5b3VyIGNvbW1lbnRzIG9uIERpZmZzZXJ2LiBJJ2QgZ28gZnVydGhlciBhbmQgY2hhcmFjdGVy
aXNlIERpZmZzZXJ2IGFzIGEgYnVzaW5lc3MgdG9vbCB3aGVyZWFzIEVDTiBhbmQgZ2VuZXJhbCBB
UU1zIGFzIG5ldHdvcmsgb3B0aW1pc2F0aW9uIHRvb2xzLiBCeSB0aGF0IEkgbWVhbiB0aGF0IERp
ZmZzZXJ2IHNob3VsZCBiZSB1c2VkIGJ5IGFuIGFwcGxpY2F0aW9uIHRvIHNwZWNpZnkgYSBidXNp
bmVzcyBwb2xpY3kgYmUNCiBpdCBsb3cgbGF0ZW5jeSBmb3IgVm9JUCAvIGdhbWluZyBvciBwcmVm
ZXJyZWQgZm9yd2FyZGluZyBmb3IgeW91ciB2aWRlbyBzZXJ2aWNlLiBBUU1zIGNhbiBndWVzcyBh
dCB0aGUgdHlwZXMgb2YgdHJhZmZpYy4gRm9yIGV4YW1wbGUgYW4gaXNvY2hyb25vdXMgZmxvdyBv
ZiBzaG9ydCBwYWNrZXRzIGNvdWxkIHdlbGwgYmUgVm9JUCB3aGljaCB3b3VsZCBiZW5lZml0IGZy
b20gZXhwZWRpdGVkIGZvcndhcmRpbmcuIEJ1dCBpdCBoYXMgdG8gYmUgYW4NCiBvcGVyYXRvciBw
b2xpY3kgdG8gZGVjaWRlIGJldHdlZW4gdHdvIFZvSVAgc2VydmljZXMuJm5ic3A7PC9kaXY+DQo8
ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj5TYW1lIGFyZ3VtZW50IGZvciBidWxrIHRyYWZmaWMgZnJv
bSBkaWZmZXJlbnQgdmlkZW8gc2VydmljZXMuIE9uZSBjYW4gYmUgYSBwYXJ0bmVyIHRyYWZmaWMg
ZnJvbSB5b3VyIGludGVybmFsIENETiBub2RlcyB3aGljaCB5b3Ugd291bGQgbGlrZSB0byBhY2Nl
bGVyYXRlIHRocm91Z2ggeW91ciBuZXR3b3JrLjwvZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxk
aXY+RUNOIGlzIGEgZ29vZCB0b29scyB0byB1c2UgdG8gdGVsbCBzb3VyY2VzIHRoZSB0aGUgbmV0
d29yayBpcyBiZWNvbWluZyBvdmVybG9hZGVkIGFuZCBpcyBraW5kZXIgdGhhbiBkcm9wcGluZyBw
YWNrZXRzLiBCdXQgaXQgY2Fubm90IGtub3cgdGhlIGJ1c2luZXNzIHBvbGljeS4gSnVzdCBoZWxw
IG1hbmFnZSBjb25nZXN0aW9uLjwvZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxkaXY+UmVnYXJk
czwvZGl2Pg0KPGRpdj4mbmJzcDsgJm5ic3A7ICZuYnNwOyBEYXZlPGJyPg0KPGJyPg0KU2VudCBm
cm9tIG15IGlQYWQ8L2Rpdj4NCjxkaXY+PGJyPg0KT24gNSBEZWMgMjAxMywgYXQgMTk6NTAsIEpp
bSBHZXR0eXMgJmx0OzxhIGhyZWY9Im1haWx0bzpqZ0BmcmVlZGVza3RvcC5vcmciPmpnQGZyZWVk
ZXNrdG9wLm9yZzwvYT4mZ3Q7IHdyb3RlOjxicj4NCjxicj4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUg
dHlwZT0iY2l0ZSI+DQo8ZGl2Pg0KPGRpdiBkaXI9Imx0ciI+DQo8ZGl2IGNsYXNzPSJnbWFpbF9k
ZWZhdWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsIj48YnI+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9
ImdtYWlsX2V4dHJhIj48YnI+DQo8YnI+DQo8ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gVGh1
LCBEZWMgNSwgMjAxMyBhdCAxOjEzIFBNLCBCb2IgQnJpc2NvZSA8c3BhbiBkaXI9Imx0ciI+DQom
bHQ7PGEgaHJlZj0ibWFpbHRvOmJvYi5icmlzY29lQGJ0LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmJv
Yi5icmlzY29lQGJ0LmNvbTwvYT4mZ3Q7PC9zcGFuPiB3cm90ZTo8YnI+DQo8YmxvY2txdW90ZSBj
bGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVy
LWxlZnQtd2lkdGg6MXB4O2JvcmRlci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVy
LWxlZnQtc3R5bGU6c29saWQ7cGFkZGluZy1sZWZ0OjFleCI+DQpGcmVkLCBHb3JyeSwgYWxsLDxi
cj4NCjxicj4NCkkgcHJvbWlzZWQgdG8gc3VnZ2VzdCB0ZXh0IGZvciBkcmFmdC1pZXRmLWFxbS1y
ZWNvbW1lbmRhdGlvbiBhYm91dCBhbGxvd2luZyB0aGUgQVFNJ3MgYmVoYXZpb3VyIHRvIGJlIGlu
ZGVwZW5kZW50IGZvciBFQ04gYW5kIG5vbi1FQ04gcGFja2V0cy4gSW4gdGhlIHByb2Nlc3MsIEkg
cmVhbGlzZWQgd2UgY2FuJ3QgdGFsayBhYm91dCBpbmRlcGVuZGVudCBBUU1zIGZvciBFQ04gd2l0
aG91dCBhbHNvIGluY2x1ZGluZyBEaWZmc2Vydi48YnI+DQo8YnI+DQpUaGlzIGdldHMgbWVzc3ks
IGJlY2F1c2UgSSBiZWxpZXZlIGEgZ29vZCBBUU0gZm9yIEJFIHRyYWZmaWMgd2l0aCBhbmQgd2l0
aG91dCBFQ04sIHNob3VsZCByZW1vdmUgbXVjaCBpZiBub3QgYWxsIHRoZSBuZWVkIGZvciBEaWZm
c2Vydi4gQnV0IHdlIGNhbid0IGlnbm9yZSBEaWZmc2Vydi48YnI+DQo8L2Jsb2NrcXVvdGU+DQo8
ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImZvbnQt
c2l6ZTpzbWFsbCI+SSBhZ3JlZSBpbiBwcmluY2lwbGUgd2l0aCB3aGF0IEJvYiBpcyB0cnlpbmcg
dG8gc2F5IGhlcmUgKGFuZCBpcyB2ZXJ5IG11Y2ggd2hhdCBJJ3ZlIGJlZW4gc2F5aW5nIGluIG15
IGJsb2cgZW50cnkgb2YgbGFzdCBzdW1tZXIpLjwvZGl2Pg0KPGRpdiBjbGFzcz0iZ21haWxfZGVm
YXVsdCIgc3R5bGU9ImZvbnQtc2l6ZTpzbWFsbCI+PGJyPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSJn
bWFpbF9kZWZhdWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsIj5PbmNlIHlvdSBoYXZlIHRoaW5n
cyB1bmRlciBjb250cm9sLCB0aGUgbmVlZCBmb3IgRGlmZnNlcnYgZGltaW5pc2hlcyBkcmFtYXRp
Y2FsbHkgKGJ1dCBkb2VzIG5vdCBnbyBhd2F5KS48L2Rpdj4NCjxkaXYgY2xhc3M9ImdtYWlsX2Rl
ZmF1bHQiIHN0eWxlPSJmb250LXNpemU6c21hbGwiPjxicj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0i
Z21haWxfZGVmYXVsdCIgc3R5bGU9ImZvbnQtc2l6ZTpzbWFsbCI+QnV0IGFzIEJvYiBub3Rlcywg
dGhlcmUgaXMgc3RpbGwgYSBnb29kIHVzZSBmb3IgRGlmZnNlcnY6IHN1aXRhYmx5IG1hcmtlZCB0
cmFmZmljIG1heSB3YW50IHRvIGNvbnRlbmQgZm9yIGFjY2VzcyB0byB0aGUgY2hhbm5lbCBkaWZm
ZXJlbnRseTogeW91ciBtYXJrZWQgVk9JUCBwYWNrZXRzIG1heSB3YW50IHRvIGNoYW5nZSB0aGUg
cHJpb3JpdHkgd2l0aCB3aGljaA0KIHlvdSByZXF1ZXN0IGNoYW5uZWwgYWNjZXNzLCBzbyB0aGF0
IHlvdSBnZXQgbW9yZSB0aW1lbHkgYWNjZXNzIHRvIHRoZSBtZWRpdW0uIFRoaXMgY29uc2VydmVz
IHRyYW5zbWl0IG9wcG9ydHVuaXRpZXMsIHdoaWNoIGlzIG9mdGVuIHRoZSBzY2FyY2VzdCByZXNv
dXJjZSBpbiBtYW55IHN5c3RlbXMgKGUuZy4gODAyLjExLCBET0NTSVMsIGV0Yy4pLiBUaGlzIGNh
biBiZSB0aGUgZGlmZmVyZW5jZSBiZXR3ZWVuIHlvdXIgVk9JUCB3b3JraW5nIHdlbGwsDQogYW5k
IG5vdCB3b3JraW5nIHdlbGwsIG9uIGEgYnVzeSA4MDIuMTEgbmV0d29yayBhcyB3ZWxsIGFzIHVz
aW5nIHRoZSBjaGFubmVsIGFzIGVmZmljaWVudGx5IGFzIHBvc3NpYmxlLjwvZGl2Pg0KPGRpdiBj
bGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImZvbnQtc2l6ZTpzbWFsbCI+PGJyPg0KPC9kaXY+
DQo8ZGl2IGNsYXNzPSJnbWFpbF9kZWZhdWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsIj5TaW1p
bGFybHksIGlmIHlvdSBoYXZlIHBhY2tldHMgeW91IGtub3cgYXJlIGJhY2tncm91bmQsIGl0J3Mg
aGVscGZ1bCB0byBrbm93IHRoYXQgdG8gZW5zdXJlIHRoYXQgdGhleSBuZXZlciBjb250ZW5kIGZv
ciBhY2Nlc3MgdG8gdGhlIG1lZGl1bSBidXQgd2lsbCBhbHdheXMgZGVmZXIgdG8gb3RoZXIgdHJh
ZmZpYywgYW5kIGp1c3Qgc2NhdmVuZ2UgYXZhaWxhYmxlDQogc3BhY2UgaW4gb3RoZXIgdHJhbnNt
aXQgb3Bwb3J0dW5pdGllcyB3aGVyZSBwb3NzaWJsZS48L2Rpdj4NCjxkaXYgY2xhc3M9ImdtYWls
X2RlZmF1bHQiIHN0eWxlPSJmb250LXNpemU6c21hbGwiPjxicj4NCjwvZGl2Pg0KPGRpdiBjbGFz
cz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImZvbnQtc2l6ZTpzbWFsbCI+SSdtIGEgYml0IGxvYXRo
ZSB0aG91Z2ggdG8gdGllIHRoZSBiZWhhdmlvciB0byBxdWV1ZXMsIGhvd2V2ZXI7IGluIHBhcnRp
Y3VsYXIsIGJlc3QgZWZmb3J0IHRyYWZmaWMgbWF5IHdhbnQgdG8gYmUgc2VudCBpbiB0aGUgc2Ft
ZSBhZ2dyZWdhdGUgYXMgaGlnaGVyIChvciBsb3dlcikgcHJpb3JpdHkgdHJhZmZpYywgaWYgdGhl
cmUgaXMgcmVtYWluaW5nIHNwYWNlDQogaW4gdGhlIGFnZ3JlZ2F0ZS4gJm5ic3A7PC9kaXY+DQo8
ZGl2IGNsYXNzPSJnbWFpbF9kZWZhdWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsIj48YnI+DQo8
L2Rpdj4NCjxkaXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LXNpemU6c21hbGwi
PkluIHNob3J0LCB0aGUgbWVudGFsIG1vZGVsIHdlJ3ZlIGhhZCB0aGF0IHRoZXJlIGlzIGEgb25l
LXRvLW9uZSBtb2RlbCBvZiBoYXJkd2FyZSBhbmQgc29mdHdhcmUgcXVldWVzIChub3QgdG8gbWVu
dGlvbiBmbG93cyBpbiBhIGdpdmVuIHNvZnR3YXJlIHF1ZXVlKSBpcyBvZnRlbiBpbmNvcnJlY3Qg
KG9yIGF0IGxlYXN0IHNlcmlvdXNseSBzdWItb3B0aW1hbCkNCiBpbiB0b2RheSdzIHN5c3RlbXMg
KGV2ZW4gaWYgdGhlIGhhcmR3YXJlIHF1ZXVlcyAmcXVvdDt3b3JrJnF1b3Q7IHByb3Blcmx5LCB3
aGljaCBpdCBhcHBlYXJzIHRoZXkgZG8gbm90IGluIDgwMi4xMSkuPC9kaXY+DQo8ZGl2IGNsYXNz
PSJnbWFpbF9kZWZhdWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsIj48YnI+DQo8L2Rpdj4NCjxk
aXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LXNpemU6c21hbGwiPlNvIEknbSBu
b3Qgc3VyZSBCb2IncyBuZXcgc2VjdGlvbiAzIGhlcmUgaXMgaG93IHRvIGJlc3QgdG8gc3RhdGUg
dGhpcyAob3IgdG8gZGVhbCB3aXRoIHRoZSB0ZXJtaW5vbG9neSBwcm9ibGVtKS4gJm5ic3A7Q2Vy
dGFpbmx5LCBpdCBtYXkgYmUgdGhlIHNhbWUgaW5zdGFuY2Ugb2YgYW4gQVFNIGFsZ29yaXRobSwg
cmF0aGVyIHRoYW4gZGlmZmVyZW50IGluc3RhbmNlcywNCiBmb3IgZXhhbXBsZS4gJm5ic3A7QW5k
ICZxdW90Ow0KPGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImRpc3BsYXk6aW5saW5l
Ij48L2Rpdj4NCkl0IFNIT1VMRCBiZSBwb3NzaWJsZSZxdW90OyBpcyBtb3JlIGEgcGlvdXMgd2lz
aCB0aGFuIGFueXRoaW5nIGVsc2UuICZuYnNwO0J1dCBJIGFncmVlIGluIHNwaXJpdCB3aXRoIHdo
YXQgQm9iJ3MgdHJ5aW5nIHRvIHNheS48L2Rpdj4NCjxkaXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQi
IHN0eWxlPSJmb250LXNpemU6c21hbGwiPiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJz
cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNw
OyAmbmJzcDsgJm5ic3A7ICZuYnNwOy0gSmltPC9kaXY+DQo8ZGl2IGNsYXNzPSJnbWFpbF9kZWZh
dWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsIj48YnI+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIGNs
YXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXIt
bGVmdC13aWR0aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXIt
bGVmdC1zdHlsZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxicj4NCl9fX19fX19fX19fX19f
X19fX19fX19fX19fX19fXzx1PjwvdT5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188dT48
L3U+X19fX19fX19fX19fX19fX19fX19fX19fX19fX188YnI+DQp7SW4gU2VjdGlvbiA0OiBhZGQg
YW5vdGhlciBidWxsZXQgYmV0d2VlbiByZWNvbW1lbmRhdGlvbnMgMiAmYW1wOyAzOn08YnI+DQo8
YnI+DQoze05ld30uIEl0IFNIT1VMRCBiZSBwb3NzaWJsZSB0byBtYWtlIGRpZmZlcmVudCBpbnN0
YW5jZXMgb2YgYW4gQVFNIGFsZ29yaXRobSBhcHBseSB0byBkaWZmZXJlbnQgc3Vic2V0cyBvZiBw
YWNrZXRzIHRoYXQgc2hhcmUgdGhlIHNhbWUgcXVldWUuDQo8ZGl2IGNsYXNzPSJnbWFpbF9kZWZh
dWx0IiBzdHlsZT0iZm9udC1zaXplOnNtYWxsO2Rpc3BsYXk6aW5saW5lIj48L2Rpdj4NCjxkaXYg
Y2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LXNpemU6c21hbGw7ZGlzcGxheTppbmxp
bmUiPjwvZGl2Pg0KSXQgU0hPVUxEIGJlIHBvc3NpYmxlIHRvIGNsYXNzaWZ5IHBhY2tldHMgaW50
byB0aGVzZSBzdWJzZXRzIGF0IGxlYXN0IGJ5IEVDTiBjb2RlcG9pbnQgW1JGQzMxNjhdIGFuZCBE
aWZmc2VydiBjb2RlcG9pbnQgW1JGQzI0NzRdIChvciB0aGUgZXF1aXZhbGVudCBvZiB0aGVzZSBm
aWVsZHMgYXQgbG93ZXIgbGF5ZXJzKS48YnI+DQo8YnI+DQp7VGhlbiBhIG5ldyBzZWN0aW9uIHRv
IGV4cGFuZCBvbiB0aGlzIGJlZm9yZSB0aGUgY3VycmVudCBTZWN0aW9uIDQuMy59PGJyPg0KNC4z
e05ld30uIEluZGVwZW5kZW50IEFRTSBJbnN0YW5jZXMgZm9yIEVDTiBhbmQgRGlmZnNlcnY8YnI+
DQo8YnI+DQpUaGUgcmVjb21tZW5kYXRpb24gdG8gcHJvdmlkZSBhIHNlcGFyYXRlIGluc3RhbmNl
IG9mIHRoZSBBUU0gZm9yIEVDTiBwYWNrZXRzIGdvZXMgYmV5b25kIHRoZSBhc3N1bXB0aW9ucyBv
ZiBSRkMgMzE2OCwgd2hpY2ggYXNzdW1lZCB0aGF0IG9ubHkgb25lIGluc3RhbmNlIG9mIGFuIEFR
TSB3aWxsIGhhbmRsZSBib3RoIEVDTi1jYXBhYmxlIGFuZCBub24tRUNOLWNhcGFibGUgcGFja2V0
cy48YnI+DQo8YnI+DQo8YnI+DQo8YnI+DQpCb2I8YnI+DQo8YnI+DQo8YnI+DQo8YnI+DQpfX19f
X19fX19fX19fX19fX19fX19fX19fX19fX188dT48L3U+X19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fPHU+PC91Pl9fX188YnI+DQpCb2IgQnJpc2NvZSwgJm5ic3A7ICZuYnNwOyAmbmJzcDsg
Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAm
bmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZu
YnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7QlQgJm5ic3A7PGJyPg0KX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fPHU+PC91Pl9fX19fX19fX19fX19fX19fPGJyPg0K
YXFtIG1haWxpbmcgbGlzdDxicj4NCjxhIGhyZWY9Im1haWx0bzphcW1AaWV0Zi5vcmciIHRhcmdl
dD0iX2JsYW5rIj5hcW1AaWV0Zi5vcmc8L2E+PGJyPg0KPGEgaHJlZj0iaHR0cHM6Ly93d3cuaWV0
Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9hcW0iIHRhcmdldD0iX2JsYW5rIj5odHRwczovL3d3dy5p
ZXRmLm9yZy9tYWlsbWFuLzx1PjwvdT5saXN0aW5mby9hcW08L2E+PGJyPg0KPC9ibG9ja3F1b3Rl
Pg0KPC9kaXY+DQo8YnI+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8
L2JvZHk+DQo8L2h0bWw+DQo=

--_000_1419030F827F4132BE5DBC74264CE46Calcatellucentcom_--

From fred@cisco.com  Fri Dec  6 07:51:24 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 45F0A1AE001 for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 07:51:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -109.501
X-Spam-Level: 
X-Spam-Status: No, score=-109.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fxIT3pt6qcUb for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 07:51:19 -0800 (PST)
Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) by ietfa.amsl.com (Postfix) with ESMTP id 23D801ADFC9 for <aqm@ietf.org>; Fri,  6 Dec 2013 07:51:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=15169; q=dns/txt; s=iport; t=1386345075; x=1387554675; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=TWjvRpZJTWYRqPA8tjMGCGS2eieiYR7JZDR9MTuHAzM=; b=A5C0ZncuaVdYq3sca3PBC0dbwaBXCSYg6k/3an+sIc/H0XsTsmkrHcml W08zWarG1USLsKUtixj3+EYEbDIBzwJC4DsW6brtE5kDwDMnw2Tj2n42y QL8o6flQRSyRoyBCFDPIkLnXud64Q96BJQzAQ4j8PslwxQUplwgGmrYZM g=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhQFADTyoVKtJXG9/2dsb2JhbABZgwc4U7kAgSIWdIIlAQEBAwEBAQFrAgEIBQsCAQgYLicLJQIEDgUOh24GDcEBEwSOL10EB4MggRMDkDGBMYYykhOBPYFsgWlB
X-IronPort-AV: E=Sophos;i="4.93,841,1378857600";  d="asc'?scan'208,217";a="4858360"
Received: from rcdn-core2-2.cisco.com ([173.37.113.189]) by alln-iport-2.cisco.com with ESMTP; 06 Dec 2013 15:51:12 +0000
Received: from xhc-rcd-x13.cisco.com (xhc-rcd-x13.cisco.com [173.37.183.87]) by rcdn-core2-2.cisco.com (8.14.5/8.14.5) with ESMTP id rB6FpCsO024467 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 6 Dec 2013 15:51:12 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.118]) by xhc-rcd-x13.cisco.com ([173.37.183.87]) with mapi id 14.03.0123.003; Fri, 6 Dec 2013 09:51:12 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: "Robinson, Dave C (Dave)" <dave.c.robinson@alcatel-lucent.com>
Thread-Topic: [aqm] Text for aqm-recommendation on independent ECN config
Thread-Index: AQHO8pr86/GSYG5zI0mOlgMwqIi0Uw==
Date: Fri, 6 Dec 2013 15:51:11 +0000
Message-ID: <97507B6E-69DA-437C-9730-24CB4403A242@cisco.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com>
In-Reply-To: <1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.123]
Content-Type: multipart/signed; boundary="Apple-Mail=_1C856E67-D80D-4ECF-AA90-42976CA781BD"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Bob Briscoe <bob.briscoe@bt.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Dec 2013 15:51:24 -0000

--Apple-Mail=_1C856E67-D80D-4ECF-AA90-42976CA781BD
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_94E795A3-A1BA-49D4-BD28-98EA42C4ED9B"


--Apple-Mail=_94E795A3-A1BA-49D4-BD28-98EA42C4ED9B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Generally agree. That said, the Conex work has suggested that ECN =
marking or thresholds might operate differently depending on the DSCP. =
As a manufacturer/implementor, my comment is that all things are =
possible to him who doesn't have to do it. However, I "get" that for =
elastic traffic "congestion experienced" might be defined as "delay, or =
projected delay, exceeded some threshold", while for a class of =
real-time traffic it might be defined as "octet throughput rate within =
this class exceeds some threshold".

On Dec 6, 2013, at 7:22 AM, "Robinson, Dave C (Dave)" =
<dave.c.robinson@alcatel-lucent.com> wrote:

> Jim et al,
>=20
> I agree with your comments on Diffserv. I'd go further and =
characterise Diffserv as a business tool whereas ECN and general AQMs as =
network optimisation tools. By that I mean that Diffserv should be used =
by an application to specify a business policy be it low latency for =
VoIP / gaming or preferred forwarding for your video service. AQMs can =
guess at the types of traffic. For example an isochronous flow of short =
packets could well be VoIP which would benefit from expedited =
forwarding. But it has to be an operator policy to decide between two =
VoIP services.=20
>=20
> Same argument for bulk traffic from different video services. One can =
be a partner traffic from your internal CDN nodes which you would like =
to accelerate through your network.
>=20
> ECN is a good tools to use to tell sources the the network is becoming =
overloaded and is kinder than dropping packets. But it cannot know the =
business policy. Just help manage congestion.
>=20
> Regards
>       Dave
>=20
> Sent from my iPad
>=20
> On 5 Dec 2013, at 19:50, Jim Gettys <jg@freedesktop.org> wrote:
>=20
>>=20
>>=20
>>=20
>> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> =
wrote:
>> Fred, Gorry, all,
>>=20
>> I promised to suggest text for draft-ietf-aqm-recommendation about =
allowing the AQM's behaviour to be independent for ECN and non-ECN =
packets. In the process, I realised we can't talk about independent AQMs =
for ECN without also including Diffserv.
>>=20
>> This gets messy, because I believe a good AQM for BE traffic with and =
without ECN, should remove much if not all the need for Diffserv. But we =
can't ignore Diffserv.
>>=20
>> I agree in principle with what Bob is trying to say here (and is very =
much what I've been saying in my blog entry of last summer).
>>=20
>> Once you have things under control, the need for Diffserv diminishes =
dramatically (but does not go away).
>>=20
>> But as Bob notes, there is still a good use for Diffserv: suitably =
marked traffic may want to contend for access to the channel =
differently: your marked VOIP packets may want to change the priority =
with which you request channel access, so that you get more timely =
access to the medium. This conserves transmit opportunities, which is =
often the scarcest resource in many systems (e.g. 802.11, DOCSIS, etc.). =
This can be the difference between your VOIP working well, and not =
working well, on a busy 802.11 network as well as using the channel as =
efficiently as possible.
>>=20
>> Similarly, if you have packets you know are background, it's helpful =
to know that to ensure that they never contend for access to the medium =
but will always defer to other traffic, and just scavenge available =
space in other transmit opportunities where possible.
>>=20
>> I'm a bit loathe though to tie the behavior to queues, however; in =
particular, best effort traffic may want to be sent in the same =
aggregate as higher (or lower) priority traffic, if there is remaining =
space in the aggregate. =20
>>=20
>> In short, the mental model we've had that there is a one-to-one model =
of hardware and software queues (not to mention flows in a given =
software queue) is often incorrect (or at least seriously sub-optimal) =
in today's systems (even if the hardware queues "work" properly, which =
it appears they do not in 802.11).
>>=20
>> So I'm not sure Bob's new section 3 here is how to best to state this =
(or to deal with the terminology problem).  Certainly, it may be the =
same instance of an AQM algorithm, rather than different instances, for =
example.  And " It SHOULD be possible" is more a pious wish than =
anything else.  But I agree in spirit with what Bob's trying to say.
>>                                - Jim
>>=20
>>=20
>> =
__________________________________________________________________________=
_______________
>> {In Section 4: add another bullet between recommendations 2 & 3:}
>>=20
>> 3{New}. It SHOULD be possible to make different instances of an AQM =
algorithm apply to different subsets of packets that share the same =
queue. It SHOULD be possible to classify packets into these subsets at =
least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or =
the equivalent of these fields at lower layers).
>>=20
>> {Then a new section to expand on this before the current Section =
4.3.}
>> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>>=20
>> The recommendation to provide a separate instance of the AQM for ECN =
packets goes beyond the assumptions of RFC 3168, which assumed that only =
one instance of an AQM will handle both ECN-capable and non-ECN-capable =
packets.
>>=20
>>=20
>>=20
>> Bob
>>=20
>>=20
>>=20
>> ________________________________________________________________
>> Bob Briscoe,                                                  BT =20
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>>=20



--Apple-Mail=_94E795A3-A1BA-49D4-BD28-98EA42C4ED9B
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Generally agree. That said, the Conex work has suggested that ECN marking or thresholds might operate differently depending on the DSCP. As a manufacturer/implementor, my comment is that all things are possible to him who doesn't have to do it. However, I "get" that for elastic traffic "congestion experienced" might be defined as "delay, or projected delay, exceeded some threshold", while for a class of real-time traffic it might be defined as "octet throughput rate within this class exceeds some threshold".<div><br><div><div>On Dec 6, 2013, at 7:22 AM, "Robinson, Dave C (Dave)" &lt;<a href="mailto:dave.c.robinson@alcatel-lucent.com">dave.c.robinson@alcatel-lucent.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

<div dir="auto">
<div>Jim et al,</div>
<div><br>
</div>
<div>I agree with your comments on Diffserv. I'd go further and characterise Diffserv as a business tool whereas ECN and general AQMs as network optimisation tools. By that I mean that Diffserv should be used by an application to specify a business policy be
 it low latency for VoIP / gaming or preferred forwarding for your video service. AQMs can guess at the types of traffic. For example an isochronous flow of short packets could well be VoIP which would benefit from expedited forwarding. But it has to be an
 operator policy to decide between two VoIP services.&nbsp;</div>
<div><br>
</div>
<div>Same argument for bulk traffic from different video services. One can be a partner traffic from your internal CDN nodes which you would like to accelerate through your network.</div>
<div><br>
</div>
<div>ECN is a good tools to use to tell sources the the network is becoming overloaded and is kinder than dropping packets. But it cannot know the business policy. Just help manage congestion.</div>
<div><br>
</div>
<div>Regards</div>
<div>&nbsp; &nbsp; &nbsp; Dave<br>
<br>
Sent from my iPad</div>
<div><br>
On 5 Dec 2013, at 19:50, Jim Gettys &lt;<a href="mailto:jg@freedesktop.org">jg@freedesktop.org</a>&gt; wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <span dir="ltr">
&lt;<a href="mailto:bob.briscoe@bt.com" target="_blank">bob.briscoe@bt.com</a>&gt;</span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Fred, Gorry, all,<br>
<br>
I promised to suggest text for draft-ietf-aqm-recommendation about allowing the AQM's behaviour to be independent for ECN and non-ECN packets. In the process, I realised we can't talk about independent AQMs for ECN without also including Diffserv.<br>
<br>
This gets messy, because I believe a good AQM for BE traffic with and without ECN, should remove much if not all the need for Diffserv. But we can't ignore Diffserv.<br>
</blockquote>
<div><br>
</div>
<div class="gmail_default" style="font-size:small">I agree in principle with what Bob is trying to say here (and is very much what I've been saying in my blog entry of last summer).</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">Once you have things under control, the need for Diffserv diminishes dramatically (but does not go away).</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">But as Bob notes, there is still a good use for Diffserv: suitably marked traffic may want to contend for access to the channel differently: your marked VOIP packets may want to change the priority with which
 you request channel access, so that you get more timely access to the medium. This conserves transmit opportunities, which is often the scarcest resource in many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference between your VOIP working well,
 and not working well, on a busy 802.11 network as well as using the channel as efficiently as possible.</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">Similarly, if you have packets you know are background, it's helpful to know that to ensure that they never contend for access to the medium but will always defer to other traffic, and just scavenge available
 space in other transmit opportunities where possible.</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">I'm a bit loathe though to tie the behavior to queues, however; in particular, best effort traffic may want to be sent in the same aggregate as higher (or lower) priority traffic, if there is remaining space
 in the aggregate. &nbsp;</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">In short, the mental model we've had that there is a one-to-one model of hardware and software queues (not to mention flows in a given software queue) is often incorrect (or at least seriously sub-optimal)
 in today's systems (even if the hardware queues "work" properly, which it appears they do not in 802.11).</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">So I'm not sure Bob's new section 3 here is how to best to state this (or to deal with the terminology problem). &nbsp;Certainly, it may be the same instance of an AQM algorithm, rather than different instances,
 for example. &nbsp;And "
<div class="gmail_default" style="display:inline"></div>
It SHOULD be possible" is more a pious wish than anything else. &nbsp;But I agree in spirit with what Bob's trying to say.</div>
<div class="gmail_default" style="font-size:small">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;- Jim</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
______________________________<u></u>______________________________<u></u>_____________________________<br>
{In Section 4: add another bullet between recommendations 2 &amp; 3:}<br>
<br>
3{New}. It SHOULD be possible to make different instances of an AQM algorithm apply to different subsets of packets that share the same queue.
<div class="gmail_default" style="font-size:small;display:inline"></div>
<div class="gmail_default" style="font-size:small;display:inline"></div>
It SHOULD be possible to classify packets into these subsets at least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent of these fields at lower layers).<br>
<br>
{Then a new section to expand on this before the current Section 4.3.}<br>
4.3{New}. Independent AQM Instances for ECN and Diffserv<br>
<br>
The recommendation to provide a separate instance of the AQM for ECN packets goes beyond the assumptions of RFC 3168, which assumed that only one instance of an AQM will handle both ECN-capable and non-ECN-capable packets.<br>
<br>
<br>
<br>
Bob<br>
<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>____<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT &nbsp;<br>
______________________________<u></u>_________________<br>
aqm mailing list<br>
<a href="mailto:aqm@ietf.org" target="_blank">aqm@ietf.org</a><br>
<a href="https://www.ietf.org/mailman/listinfo/aqm" target="_blank">https://www.ietf.org/mailman/<u></u>listinfo/aqm</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>

</blockquote></div><br>
<br></div></body></html>
--Apple-Mail=_94E795A3-A1BA-49D4-BD28-98EA42C4ED9B--

--Apple-Mail=_1C856E67-D80D-4ECF-AA90-42976CA781BD
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSofJtbjEdbHIsm0MRAvF8AJ92WrBp2spaYZzKLONsakCkE2CoMACeMO2G
/sA/Iglq7oOAGDLUMmnno+A=
=VMUX
-----END PGP SIGNATURE-----

--Apple-Mail=_1C856E67-D80D-4ECF-AA90-42976CA781BD--

From gorry@erg.abdn.ac.uk  Fri Dec  6 07:58:12 2013
Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F2AF61AE001 for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 07:58:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.202
X-Spam-Level: 
X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K-Yrl4DwtZk9 for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 07:58:08 -0800 (PST)
Received: from spey.erg.abdn.ac.uk (spey.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 4E4361AE044 for <aqm@ietf.org>; Fri,  6 Dec 2013 07:58:08 -0800 (PST)
Received: by spey.erg.abdn.ac.uk (Postfix, from userid 5001) id 359C52B44D8; Fri,  6 Dec 2013 15:58:04 +0000 (GMT)
Received: from Gorry.local (ra-gorry.erg.abdn.ac.uk [139.133.204.42]) by spey.erg.abdn.ac.uk (Postfix) with ESMTPSA id E2F8E2B44A6; Fri,  6 Dec 2013 15:57:59 +0000 (GMT)
Message-ID: <52A1F407.8070902@erg.abdn.ac.uk>
Date: Fri, 06 Dec 2013 15:57:59 +0000
From: Gorry Fairhurst <gorry@erg.abdn.ac.uk>
Organization: The University of Aberdeen is a charity registered in Scotland,  No SC013683. 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.0.1
MIME-Version: 1.0
To: "Fred Baker (fred)" <fred@cisco.com>,  "Robinson, Dave C (Dave)" <dave.c.robinson@alcatel-lucent.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com> <97507B6E-69DA-437C-9730-24CB4403A242@cisco.com>
In-Reply-To: <97507B6E-69DA-437C-9730-24CB4403A242@cisco.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Bob Briscoe <bob.briscoe@bt.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: gorry@erg.abdn.ac.uk
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Dec 2013 15:58:12 -0000

... and RFC 4774 describes how ECN interpretation can change based on DSCP.

Gorry

On 06/12/2013 15:51, Fred Baker (fred) wrote:
> Generally agree. That said, the Conex work has suggested that ECN
> marking or thresholds might operate differently depending on the DSCP.
> As a manufacturer/implementor, my comment is that all things are
> possible to him who doesn't have to do it. However, I "get" that for
> elastic traffic "congestion experienced" might be defined as "delay, or
> projected delay, exceeded some threshold", while for a class of
> real-time traffic it might be defined as "octet throughput rate within
> this class exceeds some threshold".
>
> On Dec 6, 2013, at 7:22 AM, "Robinson, Dave C (Dave)"
> <dave.c.robinson@alcatel-lucent.com
> <mailto:dave.c.robinson@alcatel-lucent.com>> wrote:
>
>> Jim et al,
>>
>> I agree with your comments on Diffserv. I'd go further and
>> characterise Diffserv as a business tool whereas ECN and general AQMs
>> as network optimisation tools. By that I mean that Diffserv should be
>> used by an application to specify a business policy be it low latency
>> for VoIP / gaming or preferred forwarding for your video service. AQMs
>> can guess at the types of traffic. For example an isochronous flow of
>> short packets could well be VoIP which would benefit from expedited
>> forwarding. But it has to be an operator policy to decide between two
>> VoIP services.
>>
>> Same argument for bulk traffic from different video services. One can
>> be a partner traffic from your internal CDN nodes which you would like
>> to accelerate through your network.
>>
>> ECN is a good tools to use to tell sources the the network is becoming
>> overloaded and is kinder than dropping packets. But it cannot know the
>> business policy. Just help manage congestion.
>>
>> Regards
>>       Dave
>>
>> Sent from my iPad
>>
>> On 5 Dec 2013, at 19:50, Jim Gettys <jg@freedesktop.org
>> <mailto:jg@freedesktop.org>> wrote:
>>
>>>
>>>
>>>
>>> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com
>>> <mailto:bob.briscoe@bt.com>> wrote:
>>>
>>>     Fred, Gorry, all,
>>>
>>>     I promised to suggest text for draft-ietf-aqm-recommendation
>>>     about allowing the AQM's behaviour to be independent for ECN and
>>>     non-ECN packets. In the process, I realised we can't talk about
>>>     independent AQMs for ECN without also including Diffserv.
>>>
>>>     This gets messy, because I believe a good AQM for BE traffic with
>>>     and without ECN, should remove much if not all the need for
>>>     Diffserv. But we can't ignore Diffserv.
>>>
>>>
>>> I agree in principle with what Bob is trying to say here (and is very
>>> much what I've been saying in my blog entry of last summer).
>>>
>>> Once you have things under control, the need for Diffserv diminishes
>>> dramatically (but does not go away).
>>>
>>> But as Bob notes, there is still a good use for Diffserv: suitably
>>> marked traffic may want to contend for access to the channel
>>> differently: your marked VOIP packets may want to change the priority
>>> with which you request channel access, so that you get more timely
>>> access to the medium. This conserves transmit opportunities, which is
>>> often the scarcest resource in many systems (e.g. 802.11, DOCSIS,
>>> etc.). This can be the difference between your VOIP working well, and
>>> not working well, on a busy 802.11 network as well as using the
>>> channel as efficiently as possible.
>>>
>>> Similarly, if you have packets you know are background, it's helpful
>>> to know that to ensure that they never contend for access to the
>>> medium but will always defer to other traffic, and just scavenge
>>> available space in other transmit opportunities where possible.
>>>
>>> I'm a bit loathe though to tie the behavior to queues, however; in
>>> particular, best effort traffic may want to be sent in the same
>>> aggregate as higher (or lower) priority traffic, if there is
>>> remaining space in the aggregate.
>>>
>>> In short, the mental model we've had that there is a one-to-one model
>>> of hardware and software queues (not to mention flows in a given
>>> software queue) is often incorrect (or at least seriously
>>> sub-optimal) in today's systems (even if the hardware queues "work"
>>> properly, which it appears they do not in 802.11).
>>>
>>> So I'm not sure Bob's new section 3 here is how to best to state this
>>> (or to deal with the terminology problem).  Certainly, it may be the
>>> same instance of an AQM algorithm, rather than different instances,
>>> for example.  And "
>>> It SHOULD be possible" is more a pious wish than anything else.  But
>>> I agree in spirit with what Bob's trying to say.
>>>                                - Jim
>>>
>>>
>>>     _____________________________________________________________________________________________
>>>     {In Section 4: add another bullet between recommendations 2 & 3:}
>>>
>>>     3{New}. It SHOULD be possible to make different instances of an
>>>     AQM algorithm apply to different subsets of packets that share
>>>     the same queue.
>>>     It SHOULD be possible to classify packets into these subsets at
>>>     least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474]
>>>     (or the equivalent of these fields at lower layers).
>>>
>>>     {Then a new section to expand on this before the current Section
>>>     4.3.}
>>>     4.3{New}. Independent AQM Instances for ECN and Diffserv
>>>
>>>     The recommendation to provide a separate instance of the AQM for
>>>     ECN packets goes beyond the assumptions of RFC 3168, which
>>>     assumed that only one instance of an AQM will handle both
>>>     ECN-capable and non-ECN-capable packets.
>>>
>>>
>>>
>>>     Bob
>>>
>>>
>>>
>>>     ____________________________________________________________________
>>>     Bob Briscoe,                                                  BT
>>>     _________________________________________________
>>>     aqm mailing list
>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>     https://www.ietf.org/mailman/__listinfo/aqm
>>>     <https://www.ietf.org/mailman/listinfo/aqm>
>>>
>>>
>
>


From davec-b@rogers.com  Fri Dec  6 08:42:06 2013
Return-Path: <davec-b@rogers.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 87ECC1AE348 for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 08:42:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.1
X-Spam-Level: 
X-Spam-Status: No, score=-0.1 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n9b7nHTHTDMw for <aqm@ietfa.amsl.com>; Fri,  6 Dec 2013 08:42:04 -0800 (PST)
Received: from nm23-vm8.access.bullet.mail.bf1.yahoo.com (nm23-vm8.access.bullet.mail.bf1.yahoo.com [216.109.115.167]) by ietfa.amsl.com (Postfix) with SMTP id 995191AE342 for <aqm@ietf.org>; Fri,  6 Dec 2013 08:42:03 -0800 (PST)
Received: from [66.196.81.164] by nm23.access.bullet.mail.bf1.yahoo.com with NNFMP; 06 Dec 2013 16:41:59 -0000
Received: from [98.139.221.157] by tm10.access.bullet.mail.bf1.yahoo.com with NNFMP; 06 Dec 2013 16:41:59 -0000
Received: from [127.0.0.1] by smtp117.sbc.mail.bf1.yahoo.com with NNFMP; 06 Dec 2013 16:41:59 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1386348119; bh=c5pFN7sr+V20cYUJZxZkfoTXIqLdzaq2GZjYyvmFtxY=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:X-Enigmail-Version:Content-Type; b=vBIfWd/acj31ZtkC3g80DLRh8gCkcbRW4LXTcIE5nLmo9/elXjUz8WTiOntREd5SLqPnGo4HaC3QMGBz/79iNO3Q4IPfAKFsuJZ5twiLwm00DbFLObrv8m8UDXOpzB4/YYJv1pr3y9ImAnur14rkdMaicqy9qyNjFatZ6eEEt+k=
X-Yahoo-Newman-Id: 612677.78640.bm@smtp117.sbc.mail.bf1.yahoo.com
X-Yahoo-Newman-Property: ymail-3
X-YMail-OSG: p5guyt4VM1mEUudejGPzX20iJ1rKaAb.GBq8TrhhLyYKB6q MyEhWB6iLDrEYT7HPA_1sb8EtpSruBgD8oYeRC4eq1CvNwb2b17O11IKfmGl GO2Lj416bN6GqBccdIkJdiU1cihCSAmzXXEs8lyBGXBDJyuiPBgnYUobfiaq KmkJ0RZmsdauHzQ5uNH_POzrsKvLouJ78HbAZSo4VSwpPV2ig.Jsru_dE43A XuEqf4R4M2oHy4sfE_OBxedGuWOHtrN1weiwhRU4QB4LcKwQZpVxLTpxmr8X tA_OtitgsQC18o8U85AaBWGz1JwDcqMOB4jDq6EBsKwL2MW1GyEoUjD8GeVS 8CKKj3NGX5kB1uV20D9kDVu72ODpntkTYyYgZcNzaS73eB6W4QXSRZysMF2c LzoFBlit3vaCud.FFXx8Z4H04FWbS62tvFZqyUk7UP5.dzhwykopMRXlJkn2 0MlEX5uVAFx0usRUS9JYE558GFjavg_oPVO9OOY0hI5PIj0FLKS7LExxPKjx 1EqRUBINDaZrgBrveDzWZt9_pID8chqWXybVSU_gkFkPc9nvD_mdOiG3LK1y mSQWXkOI-
X-Yahoo-SMTP: sltvjZWswBCRD.ElTuB1l9j6s9wRYPpuyTNWOE5oEg--
X-Rocket-Received: from [10.111.100.153] (davec-b@74.213.188.46 with ) by smtp117.sbc.mail.bf1.yahoo.com with SMTP; 06 Dec 2013 16:41:59 +0000 UTC
Message-ID: <52A1FE55.9060708@rogers.com>
Date: Fri, 06 Dec 2013 11:41:57 -0500
From: David Collier-Brown <davec-b@rogers.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4
MIME-Version: 1.0
To: aqm@ietf.org
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com>
In-Reply-To: <1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com>
X-Enigmail-Version: 1.6
Content-Type: multipart/alternative; boundary="------------000901010109050609020300"
Subject: [aqm] Semi-off-topic: Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: davecb@spamcop.net
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Dec 2013 16:42:06 -0000

This is a multi-part message in MIME format.
--------------000901010109050609020300
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 12/06/2013 10:22 AM, Robinson, Dave C (Dave) wrote:
> Jim et al,
>
> I agree with your comments on Diffserv. I'd go further and
> characterise Diffserv as a business tool whereas ECN and general AQMs
> as network optimisation tools. By that I mean that Diffserv should be
> used by an application to specify a business policy be it low latency
> for VoIP / gaming or preferred forwarding for your video service. AQMs
> can guess at the types of traffic. For example an isochronous flow of
> short packets could well be VoIP which would benefit from expedited
> forwarding. But it has to be an operator policy to decide between two
> VoIP services. 
>
> Same argument for bulk traffic from different video services. One can
> be a partner traffic from your internal CDN nodes which you would like
> to accelerate through your network.
>
> ECN is a good tools to use to tell sources the the network is becoming
> overloaded and is kinder than dropping packets. But it cannot know the
> business policy. Just help manage congestion.

1) it's also a non-business policy issue: I may want to provide low
latency to a friend or child playing a game at my house, or provide
poorer time/throughput guarantees to a web or ftp download. Yes, this
indeed a niggle (;-))

2) I'd be happier providing a metric to the network's management than an
implementation. 
Let's assume I'm doing VOIP or something else small and low-latency to a
known endpoint, and want to say for this socket, (I want = <100
milliseconds, I'm inducing  load =  0.03 mbits/sec).  The implementation
can turn that into a DIFFSERV option, if it's supported, or something
else if not.   This, IMHO, deserves consideration, but not as part of
this discussion.  Thus "off topic".

--dave


-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain
(416) 223-8968


--------------000901010109050609020300
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">On 12/06/2013 10:22 AM, Robinson, Dave
      C (Dave) wrote:<br>
    </div>
    <blockquote
      cite="mid:1419030F-827F-4132-BE5D-BC74264CE46C@alcatel-lucent.com"
      type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=ISO-8859-1">
      <div>Jim et al,</div>
      <div><br>
      </div>
      <div>I agree with your comments on Diffserv. I'd go further and
        characterise Diffserv as a business tool whereas ECN and general
        AQMs as network optimisation tools. By that I mean that Diffserv
        should be used by an application to specify a business policy be
        it low latency for VoIP / gaming or preferred forwarding for
        your video service. AQMs can guess at the types of traffic. For
        example an isochronous flow of short packets could well be VoIP
        which would benefit from expedited forwarding. But it has to be
        an operator policy to decide between two VoIP services.&nbsp;</div>
      <div><br>
      </div>
      <div>Same argument for bulk traffic from different video services.
        One can be a partner traffic from your internal CDN nodes which
        you would like to accelerate through your network.</div>
      <div><br>
      </div>
      <div>ECN is a good tools to use to tell sources the the network is
        becoming overloaded and is kinder than dropping packets. But it
        cannot know the business policy. Just help manage congestion.</div>
    </blockquote>
    <br>
    1) it's also a non-business policy issue: I may want to provide low
    latency to a friend or child playing a game at my house, or provide
    poorer time/throughput guarantees to a web or ftp download. Yes,
    this indeed a niggle (;-))<br>
    <br>
    2) I'd be happier providing a metric to the network's management
    than an implementation.&nbsp; <br>
    Let's assume I'm doing VOIP or something else small and low-latency
    to a known endpoint, and want to say for this socket, (I want =
    &lt;100 milliseconds, I'm inducing&nbsp; load =&nbsp; 0.03 mbits/sec).&nbsp; The
    implementation can turn that into a DIFFSERV option, if it's
    supported, or something else if not.&nbsp;&nbsp; This, IMHO, deserves
    consideration, but not as part of this discussion.&nbsp; Thus "off
    topic".<br>
    <br>
    --dave<br>
    <br>
    <br>
    <pre class="moz-signature" cols="72">-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
<a class="moz-txt-link-abbreviated" href="mailto:davecb@spamcop.net">davecb@spamcop.net</a>           |                      -- Mark Twain
(416) 223-8968
</pre>
  </body>
</html>

--------------000901010109050609020300--

From bob.briscoe@bt.com  Tue Dec 10 19:04:45 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E851F1AE336 for <aqm@ietfa.amsl.com>; Tue, 10 Dec 2013 19:04:45 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XVAOMjMJnkqI for <aqm@ietfa.amsl.com>; Tue, 10 Dec 2013 19:04:42 -0800 (PST)
Received: from hubrelay-by-03.bt.com (hubrelay-by-03.bt.com [62.7.242.139]) by ietfa.amsl.com (Postfix) with ESMTP id 99C671AE0FF for <aqm@ietf.org>; Tue, 10 Dec 2013 19:04:41 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR03-UKBR.bt.com (10.216.161.35) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 11 Dec 2013 03:04:31 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Wed, 11 Dec 2013 03:04:32 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Wed, 11 Dec 2013 03:04:32 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.108.249])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rBB34Tbg027256;	Wed, 11 Dec 2013 03:04:29 GMT
Message-ID: <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Wed, 11 Dec 2013 03:04:26 +0000
To: Jim Gettys <jg@freedesktop.org>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.g mail.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=====================_312660526==.ALT"
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 03:04:46 -0000

--=====================_312660526==.ALT
Content-Type: text/plain; charset="us-ascii"; format=flowed

Jim,

I'm just checking we're not talking past each other. I'll repeat two 
quotes from each of us, then comment.

>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe 
><<mailto:bob.briscoe@bt.com>bob.briscoe@bt.com> wrote:
>>3{New}. It SHOULD be possible to make different instances of an AQM 
>>algorithm apply to different subsets of packets that share the same queue.
>>It SHOULD be possible to classify packets into these subsets at 
>>least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] 
>>(or the equivalent of these fields at lower layers),

At 19:50 05/12/2013, Jim Gettys wrote:
>"Certainly, it may be the same instance of an AQM algorithm, rather 
>than different instances, for example."

That's true of course, but the case with one AQM handling all packets 
within a queue is the norm. I want to check you're happy with the converse:
1) A set-up more like WRED which was based on Dave Clark's RIO (RED 
with in and out of contract). So we can have WPIE, WCoDel etc where 
the differentiation between aggregates is provided by different AQM 
instances in the same queue, not by different queues with different 
scheduling priorities.
2) Extending this so that AQM differentiation can be between 
ECN-capable and Not-ECN-capable aggregates, not just between Diffserv 
classes (an example being CoDel with a lower 'interval' for 
ECN-capable packets).

I presented the evaluations of this last idea in tsvwg on the final 
Friday of the Vancouver IETF - I don't think you were there. 
<http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf>
This is my primary motivation for this wordsmithing - I'm trying 
allow us to move towards zero signalling delays in CoDel, PIE and RED 
(currently defaults of 200ms, 100ms and 512packets respectively, 
which are not good for dynamics).


Bob

At 19:50 05/12/2013, Jim Gettys wrote:



>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe 
><<mailto:bob.briscoe@bt.com>bob.briscoe@bt.com> wrote:
>Fred, Gorry, all,
>
>I promised to suggest text for draft-ietf-aqm-recommendation about 
>allowing the AQM's behaviour to be independent for ECN and non-ECN 
>packets. In the process, I realised we can't talk about independent 
>AQMs for ECN without also including Diffserv.
>
>This gets messy, because I believe a good AQM for BE traffic with 
>and without ECN, should remove much if not all the need for 
>Diffserv. But we can't ignore Diffserv.
>
>
>I agree in principle with what Bob is trying to say here (and is 
>very much what I've been saying in my blog entry of last summer).
>
>Once you have things under control, the need for Diffserv diminishes 
>dramatically (but does not go away).
>
>But as Bob notes, there is still a good use for Diffserv: suitably 
>marked traffic may want to contend for access to the channel 
>differently: your marked VOIP packets may want to change the 
>priority with which you request channel access, so that you get more 
>timely access to the medium. This conserves transmit opportunities, 
>which is often the scarcest resource in many systems (e.g. 802.11, 
>DOCSIS, etc.). This can be the difference between your VOIP working 
>well, and not working well, on a busy 802.11 network as well as 
>using the channel as efficiently as possible.
>
>Similarly, if you have packets you know are background, it's helpful 
>to know that to ensure that they never contend for access to the 
>medium but will always defer to other traffic, and just scavenge 
>available space in other transmit opportunities where possible.
>
>I'm a bit loathe though to tie the behavior to queues, however; in 
>particular, best effort traffic may want to be sent in the same 
>aggregate as higher (or lower) priority traffic, if there is 
>remaining space in the aggregate.
>
>In short, the mental model we've had that there is a one-to-one 
>model of hardware and software queues (not to mention flows in a 
>given software queue) is often incorrect (or at least seriously 
>sub-optimal) in today's systems (even if the hardware queues "work" 
>properly, which it appears they do not in 802.11).
>
>So I'm not sure Bob's new section 3 here is how to best to state 
>this (or to deal with the terminology problem).  Certainly, it may 
>be the same instance of an AQM algorithm, rather than different 
>instances, for example.  And "
>It SHOULD be possible" is more a pious wish than anything else.  But 
>I agree in spirit with what Bob's trying to say.
>                                - Jim
>
>
>_________________________________________________________________________________________
>{In Section 4: add another bullet between recommendations 2 & 3:}
>
>3{New}. It SHOULD be possible to make different instances of an AQM 
>algorithm apply to different subsets of packets that share the same queue.
>It SHOULD be possible to classify packets into these subsets at 
>least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] 
>(or the equivalent of these fields at lower layers).
>
>{Then a new section to expand on this before the current Section 4.3.}
>4.3{New}. Independent AQM Instances for ECN and Diffserv
>
>The recommendation to provide a separate instance of the AQM for ECN 
>packets goes beyond the assumptions of RFC 3168, which assumed that 
>only one instance of an AQM will handle both ECN-capable and 
>non-ECN-capable packets.
>
>
>
>Bob
>
>
>
>________________________________________________________________
>Bob Briscoe,                                                  BT
>_______________________________________________
>aqm mailing list
><mailto:aqm@ietf.org>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm
>
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 
--=====================_312660526==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
Jim,<br><br>
I'm just checking we're not talking past each other. I'll repeat two
quotes from each of us, then comment.<br><br>
<blockquote type=cite class=cite cite="">On Thu, Dec 5, 2013 at 1:13 PM,
Bob Briscoe
&lt;<a href="mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;
wrote:<br>
<blockquote type=cite class=cite cite="">
<dl>
<dd>3{New}. It SHOULD be possible to make different instances of an AQM
algorithm apply to different subsets of packets that share the same
queue. <br>

<dd>It SHOULD be possible to classify packets into these subsets at least
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers),</blockquote></blockquote>
</dl><br>
At 19:50 05/12/2013, Jim Gettys wrote:<br>
<blockquote type=cite class=cite cite="">&quot;Certainly, it may be the
same instance of an AQM algorithm, rather than different instances, for
example.&quot;</blockquote><br>
That's true of course, but the case with one AQM handling all packets
within a queue is the norm. I want to check you're happy with the
converse: <br>
1) A set-up more like WRED which was based on Dave Clark's RIO (RED with
in and out of contract). So we can have WPIE, WCoDel etc where the
differentiation between aggregates is provided by different AQM instances
in the same queue, not by different queues with different scheduling
priorities.<br>
2) Extending this so that AQM differentiation can be between ECN-capable
and Not-ECN-capable aggregates, not just between Diffserv classes (an
example being CoDel with a lower 'interval' for ECN-capable
packets).<br><br>
I presented the evaluations of this last idea in tsvwg on the final
Friday of the Vancouver IETF - I don't think you were there.
&lt;<a href="http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf" eudora="autourl">
http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf</a>
&gt;<br>
This is my primary motivation for this wordsmithing - I'm trying allow us
to move towards zero signalling delays in CoDel, PIE and RED (currently
defaults of 200ms, 100ms and 512packets respectively, which are not good
for dynamics).<br><br>
<br>
Bob<br><br>
At 19:50 05/12/2013, Jim Gettys wrote:<br><br>
<br><br>
<blockquote type=cite class=cite cite="">On Thu, Dec 5, 2013 at 1:13 PM,
Bob Briscoe
&lt;<a href="mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;
wrote:<br>

<dl>
<dd>Fred, Gorry, all,<br><br>

<dd>I promised to suggest text for draft-ietf-aqm-recommendation about
allowing the AQM's behaviour to be independent for ECN and non-ECN
packets. In the process, I realised we can't talk about independent AQMs
for ECN without also including Diffserv.<br><br>

<dd>This gets messy, because I believe a good AQM for BE traffic with and
without ECN, should remove much if not all the need for Diffserv. But we
can't ignore Diffserv.<br><br>

</dl><br>
I agree in principle with what Bob is trying to say here (and is very
much what I've been saying in my blog entry of last summer).<br><br>
Once you have things under control, the need for Diffserv diminishes
dramatically (but does not go away).<br><br>
But as Bob notes, there is still a good use for Diffserv: suitably marked
traffic may want to contend for access to the channel differently: your
marked VOIP packets may want to change the priority with which you
request channel access, so that you get more timely access to the medium.
This conserves transmit opportunities, which is often the scarcest
resource in many systems (e.g. 802.11, DOCSIS, etc.). This can be the
difference between your VOIP working well, and not working well, on a
busy 802.11 network as well as using the channel as efficiently as
possible.<br><br>
Similarly, if you have packets you know are background, it's helpful to
know that to ensure that they never contend for access to the medium but
will always defer to other traffic, and just scavenge available space in
other transmit opportunities where possible.<br><br>
I'm a bit loathe though to tie the behavior to queues, however; in
particular, best effort traffic may want to be sent in the same aggregate
as higher (or lower) priority traffic, if there is remaining space in the
aggregate.&nbsp; <br><br>
In short, the mental model we've had that there is a one-to-one model of
hardware and software queues (not to mention flows in a given software
queue) is often incorrect (or at least seriously sub-optimal) in today's
systems (even if the hardware queues &quot;work&quot; properly, which it
appears they do not in 802.11).<br><br>
So I'm not sure Bob's new section 3 here is how to best to state this (or
to deal with the terminology problem).&nbsp; Certainly, it may be the
same instance of an AQM algorithm, rather than different instances, for
example.&nbsp; And &quot;<br>
It SHOULD be possible&quot; is more a pious wish than anything
else.&nbsp; But I agree in spirit with what Bob's trying to say.<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
- Jim<br><br>

<dl><br>

<dd>
_________________________________________________________________________________________<br>

<dd>{In Section 4: add another bullet between recommendations 2 &amp;
3:}<br><br>

<dd>3{New}. It SHOULD be possible to make different instances of an AQM
algorithm apply to different subsets of packets that share the same
queue. <br>

<dd>It SHOULD be possible to classify packets into these subsets at least
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers).<br><br>

<dd>{Then a new section to expand on this before the current Section
4.3.}<br>

<dd>4.3{New}. Independent AQM Instances for ECN and Diffserv<br>
<br>

<dd>The recommendation to provide a separate instance of the AQM for ECN
packets goes beyond the assumptions of RFC 3168, which assumed that only
one instance of an AQM will handle both ECN-capable and non-ECN-capable
packets.<br><br>
<br><br>

<dd>Bob<br><br>
<br><br>

<dd>________________________________________________________________<br>

<dd>Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT&nbsp; <br>

<dd>_______________________________________________<br>

<dd>aqm mailing list<br>

<dd><a href="mailto:aqm@ietf.org">aqm@ietf.org</a><br>

<dd><a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dl><br>
_______________________________________________<br>
aqm mailing list<br>
aqm@ietf.org<br>
<a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<x-sigsep><p></x-sigsep>
________________________________________________________________<br>
Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT</body>
</html>

--=====================_312660526==.ALT--

From fred@cisco.com  Tue Dec 10 20:06:56 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ABC031AE14B for <aqm@ietfa.amsl.com>; Tue, 10 Dec 2013 20:06:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -109.501
X-Spam-Level: 
X-Spam-Status: No, score=-109.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EGwI47OuPe8z for <aqm@ietfa.amsl.com>; Tue, 10 Dec 2013 20:06:55 -0800 (PST)
Received: from alln-iport-5.cisco.com (alln-iport-5.cisco.com [173.37.142.92]) by ietfa.amsl.com (Postfix) with ESMTP id F1B541ADF8A for <aqm@ietf.org>; Tue, 10 Dec 2013 20:06:54 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3037; q=dns/txt; s=iport; t=1386734809; x=1387944409; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=66vcpV2a7QUD93pFK7ymNE3J9bcRKq9K0KUZiclJhUs=; b=j2JsnBo4of8ArlJZVHjci045LOPr+2NQ18P46kE1G6Nk21jvo29H8wyL ZbiEJBedQFNS64aFJ965yfeaEWpR1K0gQEuRl936jxVt3VYS6pXJu1CQ+ fUYBT3C7/gFchU7AuX+kbCSNtb+9XWHPg+9FRb+nA1LGVrg7rTiwgse9K 0=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgIFAPrjp1KtJXG//2dsb2JhbABZgweBC7khgSAWdIIlAQEBAwF5BQsCAQgEAUEyJQIEDgUOh24GwVkXjwUHgyGBEwSQMYExhjKSE4Mpgio
X-IronPort-AV: E=Sophos;i="4.93,869,1378857600";  d="asc'?scan'208,217";a="5872475"
Received: from rcdn-core2-4.cisco.com ([173.37.113.191]) by alln-iport-5.cisco.com with ESMTP; 11 Dec 2013 04:06:49 +0000
Received: from xhc-rcd-x08.cisco.com (xhc-rcd-x08.cisco.com [173.37.183.82]) by rcdn-core2-4.cisco.com (8.14.5/8.14.5) with ESMTP id rBB46nGa025495 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 11 Dec 2013 04:06:49 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.86]) by xhc-rcd-x08.cisco.com ([173.37.183.82]) with mapi id 14.03.0123.003; Tue, 10 Dec 2013 22:06:49 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [aqm] Text for aqm-recommendation on independent ECN config
Thread-Index: AQHO9iZpSnc01kXP9Uefsw5FZcogkA==
Date: Wed, 11 Dec 2013 04:06:47 +0000
Message-ID: <EC5D6AA3-31D3-4AA4-B225-96F9D44C6D82@cisco.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk>
In-Reply-To: <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.123]
Content-Type: multipart/signed; boundary="Apple-Mail=_1849B3A2-D1BA-4B4E-A01F-88A5F7E05BF8"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Jim Gettys <jg@freedesktop.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 04:06:56 -0000

--Apple-Mail=_1849B3A2-D1BA-4B4E-A01F-88A5F7E05BF8
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_7BAB2F0C-A954-4C46-B9AE-575A6A3E3FBC"


--Apple-Mail=_7BAB2F0C-A954-4C46-B9AE-575A6A3E3FBC
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Dec 10, 2013, at 7:04 PM, Bob Briscoe <bob.briscoe@bt.com>
 wrote:

> This is my primary motivation for this wordsmithing - I'm trying allow =
us to move towards zero signalling delays in CoDel, PIE and RED =
(currently defaults of 200ms, 100ms and 512packets respectively, which =
are not good for dynamics).

Violent agreement. The target, I should think, is zero statistical =
delay.

--Apple-Mail=_7BAB2F0C-A954-4C46-B9AE-575A6A3E3FBC
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On Dec 10, 2013, at 7:04 PM, Bob Briscoe &lt;<a =
href=3D"mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;</div><div>&n=
bsp;wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><span style=3D"font-family: Helvetica; font-size: medium; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: 2; text-align: =
-webkit-auto; text-indent: 0px; text-transform: none; white-space: =
normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; =
-webkit-text-stroke-width: 0px; display: inline !important; float: none; =
">This is my primary motivation for this wordsmithing - I'm trying allow =
us to move towards zero signalling delays in CoDel, PIE and RED =
(currently defaults of 200ms, 100ms and 512packets respectively, which =
are not good for dynamics).</span><br style=3D"font-family: Helvetica; =
font-size: medium; font-style: normal; font-variant: normal; =
font-weight: normal; letter-spacing: normal; line-height: normal; =
orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: =
none; white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
"></blockquote></div><br>
Violent agreement. The target, I should think, is zero statistical =
delay.</body></html>=

--Apple-Mail=_7BAB2F0C-A954-4C46-B9AE-575A6A3E3FBC--

--Apple-Mail=_1849B3A2-D1BA-4B4E-A01F-88A5F7E05BF8
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSp+TebjEdbHIsm0MRAtgZAKDgnukxW5RTo6qsxi4zPj7Hp5i+IACgjETX
ThimVs8tAEuWM1h4dI+GF4M=
=2Rj6
-----END PGP SIGNATURE-----

--Apple-Mail=_1849B3A2-D1BA-4B4E-A01F-88A5F7E05BF8--

From gettysjim@gmail.com  Wed Dec 11 08:56:00 2013
Return-Path: <gettysjim@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AB3711AE14D for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 08:56:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.277
X-Spam-Level: 
X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fXc-2V8QAG7B for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 08:55:58 -0800 (PST)
Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) by ietfa.amsl.com (Postfix) with ESMTP id CA0761AD694 for <aqm@ietf.org>; Wed, 11 Dec 2013 08:55:57 -0800 (PST)
Received: by mail-wi0-f176.google.com with SMTP id hq4so7355490wib.9 for <aqm@ietf.org>; Wed, 11 Dec 2013 08:55:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=SM1cYaM5A3aI4yRr2hLAYmQGls2YdqJ1eXGYhIiqWnU=; b=skrgyYK8o+zHzeyJnV2fmMt+LuyKMfcHtf/hYuzyj3zj88XlZuPIcnqlLnVRRsNgUG bUZy9hhbA9JD+NLG2iddkDxOcVX493+J9RUmw73O5tcJ2OUxooQEmTPPmpcOfaTTgjV/ x1EKS0MgCWz4i1jWSm5DwB/16p3LIBk+LbXm02erbgwDum5rsRtwm5nftepMxR0cY8YH 0pbt+b4u6IDqOkaefSuJWUv1IIwp1Sla8/QrVGUpMrk7nbBTDBN0vG8wDpci259VbsSZ +hrdIpgKyrXdEuShhn9hLQ517QpBXN5MfZP3rht6Vh8rzQa5CyJithfpnq4h64O1/F/O C+Vg==
MIME-Version: 1.0
X-Received: by 10.180.212.98 with SMTP id nj2mr3793317wic.52.1386780951497; Wed, 11 Dec 2013 08:55:51 -0800 (PST)
Sender: gettysjim@gmail.com
Received: by 10.227.134.74 with HTTP; Wed, 11 Dec 2013 08:55:51 -0800 (PST)
In-Reply-To: <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk>
Date: Wed, 11 Dec 2013 11:55:51 -0500
X-Google-Sender-Auth: atuvErTeTp9DoAL6_WOvhP0VkNM
Message-ID: <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com>
From: Jim Gettys <jg@freedesktop.org>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: multipart/alternative; boundary=001a11c223defed0c704ed451996
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 16:56:00 -0000

--001a11c223defed0c704ed451996
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:

>  Jim,
>
> I'm just checking we're not talking past each other. I'll repeat two
> quotes from each of us, then comment.
>
> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>
>  3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue.
> It SHOULD be possible to classify packets into these subsets at least by
> ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent
> of these fields at lower layers),
>
>
> At 19:50 05/12/2013, Jim Gettys wrote:
>
> "Certainly, it may be the same instance of an AQM algorithm, rather than
> different instances, for example."
>
>
> That's true of course, but the case with one AQM handling all packets
> within a queue is the norm. I want to check you're happy with the converse:
> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED with
> in and out of contract). So we can have WPIE, WCoDel etc where the
> differentiation between aggregates is provided by different AQM instances
> in the same queue, not by different queues with different scheduling
> priorities.
> 2) Extending this so that AQM differentiation can be between ECN-capable
> and Not-ECN-capable aggregates, not just between Diffserv classes (an
> example being CoDel with a lower 'interval' for ECN-capable packets).
>
> I presented the evaluations of this last idea in tsvwg on the final Friday
> of the Vancouver IETF - I don't think you were there. <http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf>
>

Yes, unfortunately I had to leave before the Friday session.


> This is my primary motivation for this wordsmithing - I'm trying allow us
> to move towards zero signalling delays in CoDel, PIE and RED (currently
> defaults of 200ms, 100ms and 512packets respectively, which are not good
> for dynamics).
>

Certainly signalling delays are very important: this is why I'm favorably
inclined to "head mark/drop", as it signals TCP as quickly as possible,
keeping the response of the TCP feedback loop as tight as possible (and
part of why I like CoDel so much for the highly variable bandwidth problem
we face at the edge of the net).

It's *really* important than when the bandwidth drops suddenly that
everyone gets told to slow down quickly (exactly how quickly probably
depends on the propagation change characteristics of the medium), or
packets can pile up in a big way.

How quickly the mark/drop algorithm can figure out that signalling is
appropriate is the *other* piece of getting good dynamics.  Here I don't
doubt that something may be discovered that is better than CoDel in the
slightest.


>
>
> Bob
>

My basic issue is one of terminology: people have talked about "best
effort" queues.  In reality, this is a "class" of service, rather than a
single queue, and when you get into the mental model of BE being a single
queue, (rather than a set of queues) it can lead one astray quickly and
easily.

It's really easy to fall into the idea of a single software queue mapping
to some single hardware supported queue, and that's a cognitive mistake, as
aggregating MACs are showing us; transmit ops are often the scarcest
resource...

Diffserv marking has the potential to give a "hint" to distinguish how
particular flows should be handled (scheduled) in a service class, and as
my previous example shows, that hint may be very useful in channel access
decisions (e.g. voip on 802.11).

But fq_codel teaches the lesson that packet scheduling combined with
keeping TCP sane is a key improvement over handling either problem apart...
In particular, the first packets of new flows/reappearing flows are vastly
more "important" than other packets in terms of the latency cost to users
of that service. Each flow has in essence its own queue in this service
class, and we're using information from that to help schedule the packets
in ways that minimize latency to the user.

So in this case, a single algorithm is acting over a bunch of flows in a
single class of service, and both scheduling packets among the flows, and
signalling TCP flows appropriately when they should "slow down".

So I think you and I are on close to the same page (but have been burned
badly in the past by terminology issues getting in the way).  On HTTP/1.1
we wasted probably > 2 years talking past each other because we didn't have
clear and concise terminology that we all understood the same way.

And I don't claim I have the right terminology for all this stuff, either
(even in this mail).

Which is why I was loathe to suggest exact text...
                           - Jim


>
> At 19:50 05/12/2013, Jim Gettys wrote:
>
>
>
> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>  Fred, Gorry, all,
>
> I promised to suggest text for draft-ietf-aqm-recommendation about
> allowing the AQM's behaviour to be independent for ECN and non-ECN packets.
> In the process, I realised we can't talk about independent AQMs for ECN
> without also including Diffserv.
>
> This gets messy, because I believe a good AQM for BE traffic with and
> without ECN, should remove much if not all the need for Diffserv. But we
> can't ignore Diffserv.
>
>
> I agree in principle with what Bob is trying to say here (and is very much
> what I've been saying in my blog entry of last summer).
>
> Once you have things under control, the need for Diffserv diminishes
> dramatically (but does not go away).
>
> But as Bob notes, there is still a good use for Diffserv: suitably marked
> traffic may want to contend for access to the channel differently: your
> marked VOIP packets may want to change the priority with which you request
> channel access, so that you get more timely access to the medium. This
> conserves transmit opportunities, which is often the scarcest resource in
> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
> between your VOIP working well, and not working well, on a busy 802.11
> network as well as using the channel as efficiently as possible.
>
> Similarly, if you have packets you know are background, it's helpful to
> know that to ensure that they never contend for access to the medium but
> will always defer to other traffic, and just scavenge available space in
> other transmit opportunities where possible.
>
> I'm a bit loathe though to tie the behavior to queues, however; in
> particular, best effort traffic may want to be sent in the same aggregate
> as higher (or lower) priority traffic, if there is remaining space in the
> aggregate.
>
> In short, the mental model we've had that there is a one-to-one model of
> hardware and software queues (not to mention flows in a given software
> queue) is often incorrect (or at least seriously sub-optimal) in today's
> systems (even if the hardware queues "work" properly, which it appears they
> do not in 802.11).
>
> So I'm not sure Bob's new section 3 here is how to best to state this (or
> to deal with the terminology problem).  Certainly, it may be the same
> instance of an AQM algorithm, rather than different instances, for
> example.  And "
> It SHOULD be possible" is more a pious wish than anything else.  But I
> agree in spirit with what Bob's trying to say.
>                                - Jim
>
>
> _________________________________________________________________________________________
> {In Section 4: add another bullet between recommendations 2 & 3:}
>
> 3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue.
> It SHOULD be possible to classify packets into these subsets at least by
> ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent
> of these fields at lower layers).
>
> {Then a new section to expand on this before the current Section 4.3.}
> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>
> The recommendation to provide a separate instance of the AQM for ECN
> packets goes beyond the assumptions of RFC 3168, which assumed that only
> one instance of an AQM will handle both ECN-capable and non-ECN-capable
> packets.
>
>
>
> Bob
>
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>  ________________________________________________________________
> Bob Briscoe,                                                  BT
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>

--001a11c223defed0c704ed451996
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:small"><br=
></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Tue=
, Dec 10, 2013 at 10:04 PM, Bob Briscoe <span dir=3D"ltr">&lt;<a href=3D"ma=
ilto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.com</a>&gt;</span=
> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div>
Jim,<br><br>
I&#39;m just checking we&#39;re not talking past each other. I&#39;ll repea=
t two
quotes from each of us, then comment.<br><br>
<blockquote type=3D"cite"><div class=3D"im">On Thu, Dec 5, 2013 at 1:13 PM,
Bob Briscoe
&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.=
com</a>&gt;
wrote:<br>
</div><blockquote type=3D"cite">
<dl>
<dd><div class=3D"im">3{New}. It SHOULD be possible to make different insta=
nces of an AQM
algorithm apply to different subsets of packets that share the same
queue. <br>

</div></dd><dd>It SHOULD be possible to classify packets into these subsets=
 at least
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers),</dd></dl></blockquote></blockq=
uote><div class=3D"im">
<br>
At 19:50 05/12/2013, Jim Gettys wrote:<br>
<blockquote type=3D"cite">&quot;Certainly, it may be the
same instance of an AQM algorithm, rather than different instances, for
example.&quot;</blockquote><br></div>
That&#39;s true of course, but the case with one AQM handling all packets
within a queue is the norm. I want to check you&#39;re happy with the
converse: <br>
1) A set-up more like WRED which was based on Dave Clark&#39;s RIO (RED wit=
h
in and out of contract). So we can have WPIE, WCoDel etc where the
differentiation between aggregates is provided by different AQM instances
in the same queue, not by different queues with different scheduling
priorities.<br>
2) Extending this so that AQM differentiation can be between ECN-capable
and Not-ECN-capable aggregates, not just between Diffserv classes (an
example being CoDel with a lower &#39;interval&#39; for ECN-capable
packets).<br><br>
I presented the evaluations of this last idea in tsvwg on the final
Friday of the Vancouver IETF - I don&#39;t think you were there.
&lt;<a href=3D"http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20=
.pdf" target=3D"_blank">
http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf</a>
&gt;<br></div></blockquote><div><br></div><div class=3D"gmail_default" styl=
e=3D"font-size:small">Yes, unfortunately I had to leave before the Friday s=
ession.</div><div class=3D"gmail_default" style=3D"font-size:small"></div><=
blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px=
 #ccc solid;padding-left:1ex">
<div>
This is my primary motivation for this wordsmithing - I&#39;m trying allow =
us
to move towards zero signalling delays in CoDel, PIE and RED (currently
defaults of 200ms, 100ms and 512packets respectively, which are not good
for dynamics).<br></div></blockquote><div><br></div><div class=3D"gmail_def=
ault" style=3D"font-size:small">Certainly signalling delays are very import=
ant: this is why I&#39;m favorably inclined to &quot;head mark/drop&quot;, =
as it signals TCP as quickly as possible, keeping the response of the TCP f=
eedback loop as tight as possible (and part of why I like CoDel so much for=
 the highly variable bandwidth problem we face at the edge of the net). =A0=
</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">It&#39;s *really* important th=
an when the bandwidth drops suddenly that everyone gets told to slow down q=
uickly (exactly how quickly probably depends on the propagation change char=
acteristics of the medium), or packets can pile up in a big way.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">How quickly the mark/drop algo=
rithm can figure out that signalling is appropriate is the *other* piece of=
 getting good dynamics. =A0Here I don&#39;t doubt that something may be dis=
covered that is better than CoDel in the slightest.</div>
<div class=3D"gmail_default" style=3D"font-size:small"></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p=
adding-left:1ex"><div><br>
<br>
Bob</div></blockquote><div><br></div><div class=3D"gmail_default" style=3D"=
font-size:small">My basic issue is one of terminology: people have talked a=
bout &quot;best effort&quot; queues. =A0In reality, this is a &quot;class&q=
uot; of service, rather than a single queue, and when you get into the ment=
al model of BE being a single queue, (rather than a set of queues) it can l=
ead one astray quickly and easily.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">It&#39;s really easy to fall i=
nto the idea of a single software queue mapping to some single hardware sup=
ported queue, and that&#39;s a cognitive mistake, as aggregating MACs are s=
howing us; transmit ops are often the scarcest resource...</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Diffserv marking has the poten=
tial to give a &quot;hint&quot; to distinguish how particular flows should =
be handled (scheduled) in a service class, and as my previous example shows=
, that hint may be very useful in channel access decisions (e.g. voip on 80=
2.11).</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">But fq_codel teaches the lesso=
n that packet scheduling combined with keeping TCP sane is a key improvemen=
t over handling either problem apart... In particular, the first packets of=
 new flows/reappearing flows are vastly more &quot;important&quot; than oth=
er packets in terms of the latency cost to users of that service. Each flow=
 has in essence its own queue in this service class, and we&#39;re using in=
formation from that to help schedule the packets in ways that minimize late=
ncy to the user.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">So in this case, a single algo=
rithm is acting over a bunch of flows in a single class of service, and bot=
h scheduling packets among the flows, and signalling TCP flows appropriatel=
y when they should &quot;slow down&quot;.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">So I think you and I are on cl=
ose to the same page (but have been burned badly in the past by terminology=
 issues getting in the way). =A0On HTTP/1.1 we wasted probably &gt; 2 years=
 talking past each other because we didn&#39;t have clear and concise termi=
nology that we all understood the same way.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">And I don&#39;t claim I have t=
he right terminology for all this stuff, either (even in this mail).</div><=
div class=3D"gmail_default" style=3D"font-size:small">
<br></div><div class=3D"gmail_default" style=3D"font-size:small">Which is w=
hy I was loathe to suggest exact text...</div><div class=3D"gmail_default" =
style=3D"font-size:small">=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0- Jim</div><div class=3D"gmail_default" style=3D"font-size:small">
<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bord=
er-left:1px #ccc solid;padding-left:1ex"><div><div><div class=3D"h5"><br><b=
r>
At 19:50 05/12/2013, Jim Gettys wrote:<br><br>
<br><br>
<blockquote type=3D"cite">On Thu, Dec 5, 2013 at 1:13 PM,
Bob Briscoe
&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.=
com</a>&gt;
wrote:<br>

<dl>
<dd>Fred, Gorry, all,<br><br>

</dd><dd>I promised to suggest text for draft-ietf-aqm-recommendation about
allowing the AQM&#39;s behaviour to be independent for ECN and non-ECN
packets. In the process, I realised we can&#39;t talk about independent AQM=
s
for ECN without also including Diffserv.<br><br>

</dd><dd>This gets messy, because I believe a good AQM for BE traffic with =
and
without ECN, should remove much if not all the need for Diffserv. But we
can&#39;t ignore Diffserv.<br><br>

</dd></dl><br>
I agree in principle with what Bob is trying to say here (and is very
much what I&#39;ve been saying in my blog entry of last summer).<br><br>
Once you have things under control, the need for Diffserv diminishes
dramatically (but does not go away).<br><br>
But as Bob notes, there is still a good use for Diffserv: suitably marked
traffic may want to contend for access to the channel differently: your
marked VOIP packets may want to change the priority with which you
request channel access, so that you get more timely access to the medium.
This conserves transmit opportunities, which is often the scarcest
resource in many systems (e.g. 802.11, DOCSIS, etc.). This can be the
difference between your VOIP working well, and not working well, on a
busy 802.11 network as well as using the channel as efficiently as
possible.<br><br>
Similarly, if you have packets you know are background, it&#39;s helpful to
know that to ensure that they never contend for access to the medium but
will always defer to other traffic, and just scavenge available space in
other transmit opportunities where possible.<br><br>
I&#39;m a bit loathe though to tie the behavior to queues, however; in
particular, best effort traffic may want to be sent in the same aggregate
as higher (or lower) priority traffic, if there is remaining space in the
aggregate.=A0 <br><br>
In short, the mental model we&#39;ve had that there is a one-to-one model o=
f
hardware and software queues (not to mention flows in a given software
queue) is often incorrect (or at least seriously sub-optimal) in today&#39;=
s
systems (even if the hardware queues &quot;work&quot; properly, which it
appears they do not in 802.11).<br><br>
So I&#39;m not sure Bob&#39;s new section 3 here is how to best to state th=
is (or
to deal with the terminology problem).=A0 Certainly, it may be the
same instance of an AQM algorithm, rather than different instances, for
example.=A0 And &quot;<br>
It SHOULD be possible&quot; is more a pious wish than anything
else.=A0 But I agree in spirit with what Bob&#39;s trying to say.<br>
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0
- Jim<br><br>

<dl><br>

<dd>
___________________________________________________________________________=
______________<br>

</dd><dd>{In Section 4: add another bullet between recommendations 2 &amp;
3:}<br><br>

</dd><dd>3{New}. It SHOULD be possible to make different instances of an AQ=
M
algorithm apply to different subsets of packets that share the same
queue. <br>

</dd><dd>It SHOULD be possible to classify packets into these subsets at le=
ast
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers).<br><br>

</dd><dd>{Then a new section to expand on this before the current Section
4.3.}<br>

</dd><dd>4.3{New}. Independent AQM Instances for ECN and Diffserv<br>
<br>

</dd><dd>The recommendation to provide a separate instance of the AQM for E=
CN
packets goes beyond the assumptions of RFC 3168, which assumed that only
one instance of an AQM will handle both ECN-capable and non-ECN-capable
packets.<br><br>
<br><br>

</dd><dd>Bob<br><br>
<br><br>

</dd><dd>________________________________________________________________<b=
r>

</dd><dd>Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT=A0 <br>

</dd><dd>_______________________________________________<br>

</dd><dd>aqm mailing list<br>

</dd><dd><a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a>=
<br>

</dd><dd><a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_b=
lank">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dd></dl><br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<u></u></div></div><p><u></u>
________________________________________________________________<br>
Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT</p></div>

<br>_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><br>
<br></blockquote></div><br></div></div>

--001a11c223defed0c704ed451996--

From bob.briscoe@bt.com  Wed Dec 11 11:21:36 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 77F761ADFC7 for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 11:21:36 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wDg2isGqDNHv for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 11:21:32 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 734C71ADFAD for <aqm@ietf.org>; Wed, 11 Dec 2013 11:21:31 -0800 (PST)
Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR68-UKRD.bt.com (10.187.101.23) with Microsoft SMTP Server (TLS) id 8.3.279.1; Wed, 11 Dec 2013 19:21:24 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.279.1; Wed, 11 Dec 2013 19:21:24 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Wed, 11 Dec 2013 19:21:20 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.133.150])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rBBJLGTM000905;	Wed, 11 Dec 2013 19:21:18 GMT
Message-ID: <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Wed, 11 Dec 2013 19:21:14 +0000
To: Jim Gettys <jg@freedesktop.org>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.g mail.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=====================_371269088==.ALT"
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 19:21:36 -0000

--=====================_371269088==.ALT
Content-Type: text/plain; charset="us-ascii"; format=flowed

Jim,

At 16:55 11/12/2013, Jim Gettys wrote:



>On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe 
><<mailto:bob.briscoe@bt.com>bob.briscoe@bt.com> wrote:
>Jim,
>
>I'm just checking we're not talking past each other. I'll repeat two 
>quotes from each of us, then comment.
>
>>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe 
>><<mailto:bob.briscoe@bt.com>bob.briscoe@bt.com> wrote:
>>>3{New}. It SHOULD be possible to make different instances of an 
>>>AQM algorithm apply to different subsets of packets that share the same queue.
>>>It SHOULD be possible to classify packets into these subsets at 
>>>least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] 
>>>(or the equivalent of these fields at lower layers),
>
>At 19:50 05/12/2013, Jim Gettys wrote:
>>"Certainly, it may be the same instance of an AQM algorithm, rather 
>>than different instances, for example."
>
>That's true of course, but the case with one AQM handling all 
>packets within a queue is the norm. I want to check you're happy 
>with the converse:
>1) A set-up more like WRED which was based on Dave Clark's RIO (RED 
>with in and out of contract). So we can have WPIE, WCoDel etc where 
>the differentiation between aggregates is provided by different AQM 
>instances in the same queue, not by different queues with different 
>scheduling priorities.
>2) Extending this so that AQM differentiation can be between 
>ECN-capable and Not-ECN-capable aggregates, not just between 
>Diffserv classes (an example being CoDel with a lower 'interval' for 
>ECN-capable packets).
>
>I presented the evaluations of this last idea in tsvwg on the final 
>Friday of the Vancouver IETF - I don't think you were there. < 
>http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>
>
>Yes, unfortunately I had to leave before the Friday session.
>This is my primary motivation for this wordsmithing - I'm trying 
>allow us to move towards zero signalling delays in CoDel, PIE and 
>RED (currently defaults of 200ms, 100ms and 512packets respectively, 
>which are not good for dynamics).
>
>
>Certainly signalling delays are very important: this is why I'm 
>favorably inclined to "head mark/drop", as it signals TCP as quickly 
>as possible, keeping the response of the TCP feedback loop as tight 
>as possible (and part of why I like CoDel so much for the highly 
>variable bandwidth problem we face at the edge of the net).
>
>It's *really* important than when the bandwidth drops suddenly that 
>everyone gets told to slow down quickly (exactly how quickly 
>probably depends on the propagation change characteristics of the 
>medium), or packets can pile up in a big way.
>
>How quickly the mark/drop algorithm can figure out that signalling 
>is appropriate is the *other* piece of getting good dynamics.  Here 
>I don't doubt that something may be discovered that is better than 
>CoDel in the slightest.

It takes a CoDel instance (within an fq structure) 200ms from its 
queue first passing 'threshold' before it will ever drop the first 
packet (unless the queue hits taildrop before that). So if the RTT is 
20ms, that's 220ms signalling delay. In fq_codel this creates 
considerable self-delay for short flows or r-t apps, which kill their 
own latency before they get any loss signal to tell them to slow 
down. Even for elastic flows, with congestion signals delayed by so 
much, they risk hitting themselves with a huge train of overshoot 
loss. This would be the same for fq_pie, except the number is 100ms + RTT.

Yes, the e2e transport could measure delay growth, but it doesn't 
know whether the delay is coming from a queue that is isolated from 
others or not. So it doesn't want to slow down too quickly in 
response to delay growth in case it gets screwed by other traffic. 
Ie. using delay growth as a signal entails considerable signalling 
delay due to all the uncertainty.

The proposal you missed in tsvwg was to define ECN as an immediate 
signal from the network, 'interval'=0 in CoDel terms, so the host 
always gets congestion signals as fast as possible, and if it needs 
bursts of signals smoothed out, it can do that itself.

The suggested wording ensures all AQM implementations will allow 
operators, vendors and users to configure such a mechanism. But I've 
generalised it from ECN to Diffserv too (because the implementation 
would be no different).


>My basic issue is one of terminology: people have talked about "best 
>effort" queues.  In reality, this is a "class" of service, rather 
>than a single queue, and when you get into the mental model of BE 
>being a single queue, (rather than a set of queues) it can lead one 
>astray quickly and easily.

Yeah, I know this. I suspected we were talking past each other.

I need you to allow the other case into your mind for this 
conversation. The wording is specifically about the case where 
"different subsets of packets ... share the same queue".

We can talk about an fq structure for this another time, but it's a 
really complicated way of doing it. Given simple looks like it could 
work, why get complicated already?

>It's really easy to fall into the idea of a single software queue 
>mapping to some single hardware supported queue, and that's a 
>cognitive mistake, as aggregating MACs are showing us; transmit ops 
>are often the scarcest resource...

It's only a cognitive mistake if one is not aware of all the options. 
I'm fully aware of all the options.

To be specific, a queue into a wireless medium should be configured 
so it holds some 'good queue' in reserve for transmit ops, but the 
queues on top of this that TCP self-inflicts even briefly are not 
'good queues' even if they are isolated from other flows by fq - VJ 
was wrong to generalise the phrase 'good queue' to all bursts of 
queue - it is only necessary to hold back from signalling and allow a 
burst of queue if the only possible signal is a drop. With ECN, you 
don't have this dilemma. This is the key to rapid dynamics.

>Diffserv marking has the potential to give a "hint" to distinguish 
>how particular flows should be handled (scheduled) in a service 
>class, and as my previous example shows, that hint may be very 
>useful in channel access decisions (e.g. voip on 802.11).
>
>But fq_codel teaches the lesson that packet scheduling combined with 
>keeping TCP sane is a key improvement over handling either problem 
>apart... In particular, the first packets of new flows/reappearing 
>flows are vastly more "important" than other packets in terms of the 
>latency cost to users of that service. Each flow has in essence its 
>own queue in this service class, and we're using information from 
>that to help schedule the packets in ways that minimize latency to the user.

I know all this. Please can we keep to the conversation about how to 
avoid the 200ms signalling delay that fq_codel inflicts on each flow 
(and the similar signalling delays that other AQMs inflict).


>So in this case, a single algorithm is acting over a bunch of flows 
>in a single class of service, and both scheduling packets among the 
>flows, and signalling TCP flows appropriately when they should "slow down".

Yup, I know this.


>So I think you and I are on close to the same page (but have been 
>burned badly in the past by terminology issues getting in the 
>way).  On HTTP/1.1 we wasted probably > 2 years talking past each 
>other because we didn't have clear and concise terminology that we 
>all understood the same way.

As I thought, we are talking past each other. We need to be able to 
have a conversation that is not always "Hmm, that's sounds like it 
might be interesting. Can I tell you about fq_codel now?"



Bob


>And I don't claim I have the right terminology for all this stuff, 
>either (even in this mail).
>
>Which is why I was loathe to suggest exact text...
>                            - Jim
>
>
>
>At 19:50 05/12/2013, Jim Gettys wrote:
>
>
>
>>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe 
>><<mailto:bob.briscoe@bt.com>bob.briscoe@bt.com> wrote:
>>Fred, Gorry, all,
>>I promised to suggest text for draft-ietf-aqm-recommendation about 
>>allowing the AQM's behaviour to be independent for ECN and non-ECN 
>>packets. In the process, I realised we can't talk about independent 
>>AQMs for ECN without also including Diffserv.
>>This gets messy, because I believe a good AQM for BE traffic with 
>>and without ECN, should remove much if not all the need for 
>>Diffserv. But we can't ignore Diffserv.
>>
>>
>>I agree in principle with what Bob is trying to say here (and is 
>>very much what I've been saying in my blog entry of last summer).
>>
>>Once you have things under control, the need for Diffserv 
>>diminishes dramatically (but does not go away).
>>
>>But as Bob notes, there is still a good use for Diffserv: suitably 
>>marked traffic may want to contend for access to the channel 
>>differently: your marked VOIP packets may want to change the 
>>priority with which you request channel access, so that you get 
>>more timely access to the medium. This conserves transmit 
>>opportunities, which is often the scarcest resource in many systems 
>>(e.g. 802.11, DOCSIS, etc.). This can be the difference between 
>>your VOIP working well, and not working well, on a busy 802.11 
>>network as well as using the channel as efficiently as possible.
>>
>>Similarly, if you have packets you know are background, it's 
>>helpful to know that to ensure that they never contend for access 
>>to the medium but will always defer to other traffic, and just 
>>scavenge available space in other transmit opportunities where possible.
>>
>>I'm a bit loathe though to tie the behavior to queues, however; in 
>>particular, best effort traffic may want to be sent in the same 
>>aggregate as higher (or lower) priority traffic, if there is 
>>remaining space in the aggregate.
>>
>>In short, the mental model we've had that there is a one-to-one 
>>model of hardware and software queues (not to mention flows in a 
>>given software queue) is often incorrect (or at least seriously 
>>sub-optimal) in today's systems (even if the hardware queues "work" 
>>properly, which it appears they do not in 802.11).
>>
>>So I'm not sure Bob's new section 3 here is how to best to state 
>>this (or to deal with the terminology problem).  Certainly, it may 
>>be the same instance of an AQM algorithm, rather than different 
>>instances, for example.  And "
>>It SHOULD be possible" is more a pious wish than anything 
>>else.  But I agree in spirit with what Bob's trying to say.
>>                                - Jim
>>
>>_________________________________________________________________________________________ 
>>
>>{In Section 4: add another bullet between recommendations 2 & 3:}
>>3{New}. It SHOULD be possible to make different instances of an AQM 
>>algorithm apply to different subsets of packets that share the same queue.
>>It SHOULD be possible to classify packets into these subsets at 
>>least by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] 
>>(or the equivalent of these fields at lower layers).
>>{Then a new section to expand on this before the current Section 4.3.}
>>4.3{New}. Independent AQM Instances for ECN and Diffserv
>>The recommendation to provide a separate instance of the AQM for 
>>ECN packets goes beyond the assumptions of RFC 3168, which assumed 
>>that only one instance of an AQM will handle both ECN-capable and 
>>non-ECN-capable packets.
>>
>>
>>Bob
>>
>>
>>________________________________________________________________
>>Bob Briscoe,                                                  BT
>>_______________________________________________
>>aqm mailing list
>><mailto:aqm@ietf.org>aqm@ietf.org
>>https://www.ietf.org/mailman/listinfo/aqm
>>
>>
>>_______________________________________________
>>aqm mailing list
>><mailto:aqm@ietf.org>aqm@ietf.org
>>https://www.ietf.org/mailman/listinfo/aqm
>
>________________________________________________________________
>Bob Briscoe,                                                  BT
>
>_______________________________________________
>aqm mailing list
><mailto:aqm@ietf.org>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm
>
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 
--=====================_371269088==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
Jim,<br><br>
At 16:55 11/12/2013, Jim Gettys wrote:<br><br>
<br><br>
<blockquote type=cite class=cite cite="">On Tue, Dec 10, 2013 at 10:04
PM, Bob Briscoe
&lt;<a href="mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;
wrote:<br>

<dl>
<dd>Jim,<br><br>

<dd>I'm just checking we're not talking past each other. I'll repeat two
quotes from each of us, then comment.<br><br>
<blockquote type=cite class=cite cite="">
<dd>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe
&lt;<a href="mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;
wrote:<blockquote type=cite class=cite cite="">
<dl>
<dd>3{New}. It SHOULD be possible to make different instances of an AQM
algorithm apply to different subsets of packets that share the same
queue. 
<dd>It SHOULD be possible to classify packets into these subsets at least
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers),</blockquote></blockquote>
</dl><br>

<dd>At 19:50 05/12/2013, Jim Gettys wrote:<br>
<blockquote type=cite class=cite cite="">
<dd>&quot;Certainly, it may be the same instance of an AQM algorithm,
rather than different instances, for example.&quot;</blockquote><br>

<dd>That's true of course, but the case with one AQM handling all packets
within a queue is the norm. I want to check you're happy with the
converse: <br>

<dd>1) A set-up more like WRED which was based on Dave Clark's RIO (RED
with in and out of contract). So we can have WPIE, WCoDel etc where the
differentiation between aggregates is provided by different AQM instances
in the same queue, not by different queues with different scheduling
priorities.<br>

<dd>2) Extending this so that AQM differentiation can be between
ECN-capable and Not-ECN-capable aggregates, not just between Diffserv
classes (an example being CoDel with a lower 'interval' for ECN-capable
packets).<br><br>

<dd>I presented the evaluations of this last idea in tsvwg on the final
Friday of the Vancouver IETF - I don't think you were there. &lt;
<a href="http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf" eudora="autourl">
http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf</a>
&gt;<br>
<br>

</dl><br>
Yes, unfortunately I had to leave before the Friday session.<br>

<dl>
<dd>This is my primary motivation for this wordsmithing - I'm trying
allow us to move towards zero signalling delays in CoDel, PIE and RED
(currently defaults of 200ms, 100ms and 512packets respectively, which
are not good for dynamics).<br><br>

</dl><br>
Certainly signalling delays are very important: this is why I'm favorably
inclined to &quot;head mark/drop&quot;, as it signals TCP as quickly as
possible, keeping the response of the TCP feedback loop as tight as
possible (and part of why I like CoDel so much for the highly variable
bandwidth problem we face at the edge of the net).&nbsp; <br><br>
It's *really* important than when the bandwidth drops suddenly that
everyone gets told to slow down quickly (exactly how quickly probably
depends on the propagation change characteristics of the medium), or
packets can pile up in a big way.<br><br>
How quickly the mark/drop algorithm can figure out that signalling is
appropriate is the *other* piece of getting good dynamics.&nbsp; Here I
don't doubt that something may be discovered that is better than CoDel in
the slightest.</blockquote><br>
It takes a CoDel instance (within an fq structure) 200ms from its queue
first passing 'threshold' before it will ever drop the first packet
(unless the queue hits taildrop before that). So if the RTT is 20ms,
that's 220ms signalling delay. In fq_codel this creates considerable
self-delay for short flows or r-t apps, which kill their own latency
before they get any loss signal to tell them to slow down. Even for
elastic flows, with congestion signals delayed by so much, they risk
hitting themselves with a huge train of overshoot loss. This would be the
same for fq_pie, except the number is 100ms + RTT.<br><br>
Yes, the e2e transport could measure delay growth, but it doesn't know
whether the delay is coming from a queue that is isolated from others or
not. So it doesn't want to slow down too quickly in response to delay
growth in case it gets screwed by other traffic. Ie. using delay growth
as a signal entails considerable signalling delay due to all the
uncertainty.<br><br>
The proposal you missed in tsvwg was to define ECN as an immediate signal
from the network, 'interval'=0 in CoDel terms, so the host always gets
congestion signals as fast as possible, and if it needs bursts of signals
smoothed out, it can do that itself.<br><br>
The suggested wording ensures all AQM implementations will allow
operators, vendors and users to configure such a mechanism. But I've
generalised it from ECN to Diffserv too (because the implementation would
be no different).<br><br>
<br>
<blockquote type=cite class=cite cite="">My basic issue is one of
terminology: people have talked about &quot;best effort&quot;
queues.&nbsp; In reality, this is a &quot;class&quot; of service, rather
than a single queue, and when you get into the mental model of BE being a
single queue, (rather than a set of queues) it can lead one astray
quickly and easily.</blockquote><br>
Yeah, I know this. I suspected we were talking past each other.<br><br>
I need you to allow the other case into your mind for this conversation.
The wording is specifically about the case where &quot;different subsets
of packets ... share the same queue&quot;. <br><br>
We can talk about an fq structure for this another time, but it's a
really complicated way of doing it. Given simple looks like it could
work, why get complicated already?<br><br>
<blockquote type=cite class=cite cite="">It's really easy to fall into
the idea of a single software queue mapping to some single hardware
supported queue, and that's a cognitive mistake, as aggregating MACs are
showing us; transmit ops are often the scarcest
resource...</blockquote><br>
It's only a cognitive mistake if one is not aware of all the options. I'm
fully aware of all the options. <br><br>
To be specific, a queue into a wireless medium should be configured so it
holds some 'good queue' in reserve for transmit ops, but the queues on
top of this that TCP self-inflicts even briefly are not 'good queues'
even if they are isolated from other flows by fq - VJ was wrong to
generalise the phrase 'good queue' to all bursts of queue - it is only
necessary to hold back from signalling and allow a burst of queue if the
only possible signal is a drop. With ECN, you don't have this dilemma.
This is the key to rapid dynamics.<br><br>
<blockquote type=cite class=cite cite="">Diffserv marking has the
potential to give a &quot;hint&quot; to distinguish how particular flows
should be handled (scheduled) in a service class, and as my previous
example shows, that hint may be very useful in channel access decisions
(e.g. voip on 802.11).<br><br>
But fq_codel teaches the lesson that packet scheduling combined with
keeping TCP sane is a key improvement over handling either problem
apart... In particular, the first packets of new flows/reappearing flows
are vastly more &quot;important&quot; than other packets in terms of the
latency cost to users of that service. Each flow has in essence its own
queue in this service class, and we're using information from that to
help schedule the packets in ways that minimize latency to the
user.</blockquote><br>
I know all this. Please can we keep to the conversation about how to
avoid the 200ms signalling delay that fq_codel inflicts on each flow (and
the similar signalling delays that other AQMs inflict).<br><br>
<br>
<blockquote type=cite class=cite cite="">So in this case, a single
algorithm is acting over a bunch of flows in a single class of service,
and both scheduling packets among the flows, and signalling TCP flows
appropriately when they should &quot;slow down&quot;.</blockquote><br>
Yup, I know this.<br><br>
<br>
<blockquote type=cite class=cite cite="">So I think you and I are on
close to the same page (but have been burned badly in the past by
terminology issues getting in the way).&nbsp; On HTTP/1.1 we wasted
probably &gt; 2 years talking past each other because we didn't have
clear and concise terminology that we all understood the same
way.</blockquote><br>
As I thought, we are talking past each other. We need to be able to have
a conversation that is not always &quot;Hmm, that's sounds like it might
be interesting. Can I tell you about fq_codel now?&quot;<br><br>
<br><br>
Bob<br><br>
<br>
<blockquote type=cite class=cite cite="">And I don't claim I have the
right terminology for all this stuff, either (even in this
mail).<br><br>
Which is why I was loathe to suggest exact text...<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
- Jim<br><br>

<dl><br><br>

<dd>At 19:50 05/12/2013, Jim Gettys wrote:<br><br>
<br><br>
<blockquote type=cite class=cite cite="">
<dd>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe
&lt;<a href="mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;
wrote:
<dl>
<dd>Fred, Gorry, all,<br>

<dd>I promised to suggest text for draft-ietf-aqm-recommendation about
allowing the AQM's behaviour to be independent for ECN and non-ECN
packets. In the process, I realised we can't talk about independent AQMs
for ECN without also including Diffserv.<br>

<dd>This gets messy, because I believe a good AQM for BE traffic with and
without ECN, should remove much if not all the need for Diffserv. But we
can't ignore Diffserv.<br><br>
</dl><br>

<dd>I agree in principle with what Bob is trying to say here (and is very
much what I've been saying in my blog entry of last summer).<br><br>

<dd>Once you have things under control, the need for Diffserv diminishes
dramatically (but does not go away).<br><br>

<dd>But as Bob notes, there is still a good use for Diffserv: suitably
marked traffic may want to contend for access to the channel differently:
your marked VOIP packets may want to change the priority with which you
request channel access, so that you get more timely access to the medium.
This conserves transmit opportunities, which is often the scarcest
resource in many systems (e.g. 802.11, DOCSIS, etc.). This can be the
difference between your VOIP working well, and not working well, on a
busy 802.11 network as well as using the channel as efficiently as
possible.<br><br>

<dd>Similarly, if you have packets you know are background, it's helpful
to know that to ensure that they never contend for access to the medium
but will always defer to other traffic, and just scavenge available space
in other transmit opportunities where possible.<br><br>

<dd>I'm a bit loathe though to tie the behavior to queues, however; in
particular, best effort traffic may want to be sent in the same aggregate
as higher (or lower) priority traffic, if there is remaining space in the
aggregate.&nbsp; <br><br>

<dd>In short, the mental model we've had that there is a one-to-one model
of hardware and software queues (not to mention flows in a given software
queue) is often incorrect (or at least seriously sub-optimal) in today's
systems (even if the hardware queues &quot;work&quot; properly, which it
appears they do not in 802.11).<br><br>

<dd>So I'm not sure Bob's new section 3 here is how to best to state this
(or to deal with the terminology problem).&nbsp; Certainly, it may be the
same instance of an AQM algorithm, rather than different instances, for
example.&nbsp; And &quot;<br>

<dd>It SHOULD be possible&quot; is more a pious wish than anything
else.&nbsp; But I agree in spirit with what Bob's trying to say.<br>

<dd>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
- Jim<br><br>
<dl>
<dd>
_________________________________________________________________________________________
<dd>{In Section 4: add another bullet between recommendations 2 &amp;
3:}<br>

<dd>3{New}. It SHOULD be possible to make different instances of an AQM
algorithm apply to different subsets of packets that share the same
queue. 
<dd>It SHOULD be possible to classify packets into these subsets at least
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers).<br>

<dd>{Then a new section to expand on this before the current Section
4.3.}
<dd>4.3{New}. Independent AQM Instances for ECN and Diffserv<br>

<dd>The recommendation to provide a separate instance of the AQM for ECN
packets goes beyond the assumptions of RFC 3168, which assumed that only
one instance of an AQM will handle both ECN-capable and non-ECN-capable
packets.<br>
<br>
<br>

<dd>Bob<br>
<br>
<br>

<dd>________________________________________________________________
<dd>Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT&nbsp; 
<dd>_______________________________________________
<dd>aqm mailing list
<dd><a href="mailto:aqm@ietf.org">aqm@ietf.org</a>
<dd><a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>
</dl><br>

<dd>_______________________________________________<br>

<dd>aqm mailing list<br>

<dd><a href="mailto:aqm@ietf.org">aqm@ietf.org</a><br>

<dd><a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote><br>

<dd>________________________________________________________________<br>

<dd>Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT<br><br>

<dd>_______________________________________________<br>

<dd>aqm mailing list<br>

<dd><a href="mailto:aqm@ietf.org">aqm@ietf.org</a><br>

<dd><a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dl><br>
_______________________________________________<br>
aqm mailing list<br>
aqm@ietf.org<br>
<a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<x-sigsep><p></x-sigsep>
________________________________________________________________<br>
Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT</body>
</html>

--=====================_371269088==.ALT--

From dave.taht@gmail.com  Wed Dec 11 12:01:03 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4EF801AE20D for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 12:01:03 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JU6ZIoHPvOLC for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 12:00:57 -0800 (PST)
Received: from mail-we0-x22a.google.com (mail-we0-x22a.google.com [IPv6:2a00:1450:400c:c03::22a]) by ietfa.amsl.com (Postfix) with ESMTP id 5FA401AE230 for <aqm@ietf.org>; Wed, 11 Dec 2013 12:00:55 -0800 (PST)
Received: by mail-we0-f170.google.com with SMTP id w61so7104875wes.29 for <aqm@ietf.org>; Wed, 11 Dec 2013 12:00:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=oMCaOgzgPoRUruzRRSwY5bx6RMcGnAYQn0mW9w15saI=; b=rVm/iOcDdfH1Zz3osZOMcwhVm+9Zo0h7NxFGdue50KyrvI3Iwwi+rgMWAu+LQSh9RP uIl1w/btljkb48vGJk6nCFshod6GwaFIjEdUwr74t7DaT8t8/kILgy/QTPPfjfaxkomh r6WDIVSpN2BSp+w7QF4GuO9L1EuDzekDVRtO8+Tkm8mhBSgI+sOUYQ8ckwK3GvoLfeGF cv91J9saZ2SDxIpVppr77qSb3HSssxZrcJQQwBatrfuuDjYMxb9qhcIYN+70Tni4zvKB DwP43fhfMFCijw+88EoCNsm5uZxWzoIwtsbZ4rPZqwatZ390SNB83YLprRS3zZhpGWwB z0xQ==
MIME-Version: 1.0
X-Received: by 10.180.105.199 with SMTP id go7mr4395676wib.53.1386792049224; Wed, 11 Dec 2013 12:00:49 -0800 (PST)
Received: by 10.217.123.69 with HTTP; Wed, 11 Dec 2013 12:00:49 -0800 (PST)
In-Reply-To: <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk>
Date: Wed, 11 Dec 2013 12:00:49 -0800
Message-ID: <CAA93jw54uJvhaRZ8OMdM4Rrxs7U=3upAd_gAkVEyXJXqRaaEyg@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 20:01:03 -0000

On Wed, Dec 11, 2013 at 11:21 AM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> Jim,
>
>
> At 16:55 11/12/2013, Jim Gettys wrote:
>
>
>
> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> Jim,
>
> I'm just checking we're not talking past each other. I'll repeat two quot=
es
> from each of us, then comment.
>
> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>
> 3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue=
.
> It SHOULD be possible to classify packets into these subsets at least by =
ECN
> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent o=
f
> these fields at lower layers),
>
>
> At 19:50 05/12/2013, Jim Gettys wrote:
>
> "Certainly, it may be the same instance of an AQM algorithm, rather than
> different instances, for example."
>
>
> That's true of course, but the case with one AQM handling all packets wit=
hin
> a queue is the norm. I want to check you're happy with the converse:
> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED with =
in
> and out of contract). So we can have WPIE, WCoDel etc where the
> differentiation between aggregates is provided by different AQM instances=
 in
> the same queue, not by different queues with different scheduling
> priorities.
> 2) Extending this so that AQM differentiation can be between ECN-capable =
and
> Not-ECN-capable aggregates, not just between Diffserv classes (an example
> being CoDel with a lower 'interval' for ECN-capable packets).
>
> I presented the evaluations of this last idea in tsvwg on the final Frida=
y
> of the Vancouver IETF - I don't think you were there. <
> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>
>
> Yes, unfortunately I had to leave before the Friday session.
> This is my primary motivation for this wordsmithing - I'm trying allow us=
 to
> move towards zero signalling delays in CoDel, PIE and RED (currently
> defaults of 200ms, 100ms and 512packets respectively, which are not good =
for
> dynamics).
>
>
> Certainly signalling delays are very important: this is why I'm favorably
> inclined to "head mark/drop", as it signals TCP as quickly as possible,
> keeping the response of the TCP feedback loop as tight as possible (and p=
art
> of why I like CoDel so much for the highly variable bandwidth problem we
> face at the edge of the net).
>
> It's *really* important than when the bandwidth drops suddenly that every=
one
> gets told to slow down quickly (exactly how quickly probably depends on t=
he
> propagation change characteristics of the medium), or packets can pile up=
 in
> a big way.
>
> How quickly the mark/drop algorithm can figure out that signalling is
> appropriate is the *other* piece of getting good dynamics.  Here I don't
> doubt that something may be discovered that is better than CoDel in the
> slightest.
> It takes a CoDel instance (within an fq structure) 200ms from its queue
> first passing 'threshold' before it will ever drop the first packet (unle=
ss
> the queue hits taildrop before that). So if the RTT is 20ms, that's 220ms
> signalling delay. In fq_codel this creates considerable self-delay for sh=
ort
> flows or r-t apps, which kill their own latency before they get any loss
> signal to tell them to slow down. Even for elastic flows, with congestion
> signals delayed by so much, they risk hitting themselves with a huge trai=
n
> of overshoot loss. This would be the same for fq_pie, except the number i=
s
> 100ms + RTT.

Things have so consistently expressed things this way that I began to
doubt the reality myself. It seems like a large number of folk on this list
don't get it either, so I am going to try an explain in a new way.

Tackling codel first:

the first phase of codel has effectively a "training" period where a link g=
oing
from unloaded to loaded for the first time ever - the very first drop with =
the
default interval will happen in 200ms, yes. IF it stays loaded and  over th=
e
target delay after the first drop/mark, it will then tune to ever
smaller intervals
to approximate an ideal drop rate until the latency on the link drops below=
 the
target. At which point the algorithm saves that rate, and stops doing anyth=
ing
until the next time the target delay is exceeded.

Some keep asserting that that is all there is to codel, saying things like
 "there is a linear increase in drop probability" using the invsqrt mechani=
sm,

*which is true during the training phase*.

After that approximation of ideal drop/mark rate is obtained, the algorithm
goes quiescent until the next time the target delay is consistently exceede=
d,
at which point it schedules the next drop at a little more than the stored
previous drop rate. It then continually seeks around that point up and down=
.

If the delay drops below the target in this phase, the algorithm stops
again and decreases the drop rate again, as it's too high. If the delay
stays above target after the drop for the current value of the interval,
the drop rate increases.

This is an interesting solution to kleinrocks formulation of "power", where
he once said an average of one packet should be in the queue, codel aims
to never have less than one packet in the queue.

And the switch into and out of drop mode going above target is entirely
dependent on the characteristics over time of the flows on the system,
completely nonlinear, and where codel spends 99.999% of it's time on a
loaded link.

As debussy said: "Music is the *space* between the notes".

I wish i had a name for this second "seeking" phase that makes as much sens=
e as
"congestion avoidance".

So asserting that you'll have a 200ms interval on a codled link always is j=
ust
blatantly incorrect. On first boot, yes. On a busy network, never again.[1]

In the event of a link going completely idle, and staying
idle, there is hysteresis built into the code so it will retain that
drop rate for a few
hundred milliseconds (it's 8*interval in some versions of the code,
4 in others), before resetting count to 1 and the resulting estimation
window to interval.

It is certainly possible to come up with a codel variant that more
closely matches the preconditions of the network, and the speeds on
the link.

In the data center case there there are two knobs to set the target
delay and interval to the desired time width of that network. I keep
hoping someone in a datacenter with resources to play will merely set
them, and report the results. I'd suggest target 500us and interval
10ms as numbers most modern x86 server hardware can reliably
accomplish without invoking other scheduling delays from cpu
schedulers, etc. Even lower values are possible, but
these two are fairly close to a data center width, and I hope someone
will try them. [2]

PLEASE? I simply don't have data center resources, yet the experiment
is EASY if you do. (note: ECN is on by default in fq_codel and off by
default in codel.)

:whew: hopefully that explains that.

NEXT, is tackling what has been said about the behavior of fq_codel in this
environment, but to avoid cluttering up the issue, I'll stop here at
getting codel
described right, today. And go back to sigcomm for a while

[1] on fq_codel the values are cached, too, but=85 well=85 I will try to ex=
plain
later. yes, more work is needed in this area.

[2] It might make sense to modify the hysterisis in the data center case

> Yes, the e2e transport could measure delay growth, but it doesn't know
> whether the delay is coming from a queue that is isolated from others or
> not. So it doesn't want to slow down too quickly in response to delay gro=
wth
> in case it gets screwed by other traffic. Ie. using delay growth as a sig=
nal
> entails considerable signalling delay due to all the uncertainty.
>
> The proposal you missed in tsvwg was to define ECN as an immediate signal
> from the network, 'interval'=3D0 in CoDel terms, so the host always gets
> congestion signals as fast as possible, and if it needs bursts of signals
> smoothed out, it can do that itself.
>
> The suggested wording ensures all AQM implementations will allow operator=
s,
> vendors and users to configure such a mechanism. But I've generalised it
> from ECN to Diffserv too (because the implementation would be no differen=
t).
>
>
>
> My basic issue is one of terminology: people have talked about "best effo=
rt"
> queues.  In reality, this is a "class" of service, rather than a single
> queue, and when you get into the mental model of BE being a single queue,
> (rather than a set of queues) it can lead one astray quickly and easily.
>
>
> Yeah, I know this. I suspected we were talking past each other.
>
> I need you to allow the other case into your mind for this conversation. =
The
> wording is specifically about the case where "different subsets of packet=
s
> ... share the same queue".
>
> We can talk about an fq structure for this another time, but it's a reall=
y
> complicated way of doing it. Given simple looks like it could work, why g=
et
> complicated already?
>
>
> It's really easy to fall into the idea of a single software queue mapping=
 to
> some single hardware supported queue, and that's a cognitive mistake, as
> aggregating MACs are showing us; transmit ops are often the scarcest
> resource...
>
>
> It's only a cognitive mistake if one is not aware of all the options. I'm
> fully aware of all the options.
>
> To be specific, a queue into a wireless medium should be configured so it
> holds some 'good queue' in reserve for transmit ops, but the queues on to=
p
> of this that TCP self-inflicts even briefly are not 'good queues' even if
> they are isolated from other flows by fq - VJ was wrong to generalise the
> phrase 'good queue' to all bursts of queue - it is only necessary to hold
> back from signalling and allow a burst of queue if the only possible sign=
al
> is a drop. With ECN, you don't have this dilemma. This is the key to rapi=
d
> dynamics.
>
>
> Diffserv marking has the potential to give a "hint" to distinguish how
> particular flows should be handled (scheduled) in a service class, and as=
 my
> previous example shows, that hint may be very useful in channel access
> decisions (e.g. voip on 802.11).
>
> But fq_codel teaches the lesson that packet scheduling combined with keep=
ing
> TCP sane is a key improvement over handling either problem apart... In
> particular, the first packets of new flows/reappearing flows are vastly m=
ore
> "important" than other packets in terms of the latency cost to users of t=
hat
> service. Each flow has in essence its own queue in this service class, an=
d
> we're using information from that to help schedule the packets in ways th=
at
> minimize latency to the user.
>
>
> I know all this. Please can we keep to the conversation about how to avoi=
d
> the 200ms signalling delay that fq_codel inflicts on each flow (and the
> similar signalling delays that other AQMs inflict).
>
>
>
> So in this case, a single algorithm is acting over a bunch of flows in a
> single class of service, and both scheduling packets among the flows, and
> signalling TCP flows appropriately when they should "slow down".
>
>
> Yup, I know this.
>
>
>
> So I think you and I are on close to the same page (but have been burned
> badly in the past by terminology issues getting in the way).  On HTTP/1.1=
 we
> wasted probably > 2 years talking past each other because we didn't have
> clear and concise terminology that we all understood the same way.
>
>
> As I thought, we are talking past each other. We need to be able to have =
a
> conversation that is not always "Hmm, that's sounds like it might be
> interesting. Can I tell you about fq_codel now?"
>
>
>
> Bob
>
>
>
> And I don't claim I have the right terminology for all this stuff, either
> (even in this mail).
>
> Which is why I was loathe to suggest exact text...
>                            - Jim
>
>
>
> At 19:50 05/12/2013, Jim Gettys wrote:
>
>
>
> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> Fred, Gorry, all,
> I promised to suggest text for draft-ietf-aqm-recommendation about allowi=
ng
> the AQM's behaviour to be independent for ECN and non-ECN packets. In the
> process, I realised we can't talk about independent AQMs for ECN without
> also including Diffserv.
> This gets messy, because I believe a good AQM for BE traffic with and
> without ECN, should remove much if not all the need for Diffserv. But we
> can't ignore Diffserv.
>
>
> I agree in principle with what Bob is trying to say here (and is very muc=
h
> what I've been saying in my blog entry of last summer).
>
> Once you have things under control, the need for Diffserv diminishes
> dramatically (but does not go away).
>
> But as Bob notes, there is still a good use for Diffserv: suitably marked
> traffic may want to contend for access to the channel differently: your
> marked VOIP packets may want to change the priority with which you reques=
t
> channel access, so that you get more timely access to the medium. This
> conserves transmit opportunities, which is often the scarcest resource in
> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference betw=
een
> your VOIP working well, and not working well, on a busy 802.11 network as
> well as using the channel as efficiently as possible.
>
> Similarly, if you have packets you know are background, it's helpful to k=
now
> that to ensure that they never contend for access to the medium but will
> always defer to other traffic, and just scavenge available space in other
> transmit opportunities where possible.
>
> I'm a bit loathe though to tie the behavior to queues, however; in
> particular, best effort traffic may want to be sent in the same aggregate=
 as
> higher (or lower) priority traffic, if there is remaining space in the
> aggregate.
>
> In short, the mental model we've had that there is a one-to-one model of
> hardware and software queues (not to mention flows in a given software
> queue) is often incorrect (or at least seriously sub-optimal) in today's
> systems (even if the hardware queues "work" properly, which it appears th=
ey
> do not in 802.11).
>
> So I'm not sure Bob's new section 3 here is how to best to state this (or=
 to
> deal with the terminology problem).  Certainly, it may be the same instan=
ce
> of an AQM algorithm, rather than different instances, for example.  And "
> It SHOULD be possible" is more a pious wish than anything else.  But I ag=
ree
> in spirit with what Bob's trying to say.
>                                - Jim
>
> _________________________________________________________________________=
________________
> {In Section 4: add another bullet between recommendations 2 & 3:}
> 3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue=
.
> It SHOULD be possible to classify packets into these subsets at least by =
ECN
> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent o=
f
> these fields at lower layers).
> {Then a new section to expand on this before the current Section 4.3.}
> 4.3{New}. Independent AQM Instances for ECN and Diffserv
> The recommendation to provide a separate instance of the AQM for ECN pack=
ets
> goes beyond the assumptions of RFC 3168, which assumed that only one
> instance of an AQM will handle both ECN-capable and non-ECN-capable packe=
ts.
>
>
> Bob
>
>
> ________________________________________________________________ Bob
> Briscoe,                                                  BT
> _______________________________________________ aqm mailing list
> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From ilpo.jarvinen@helsinki.fi  Wed Dec 11 13:11:46 2013
Return-Path: <ilpo.jarvinen@helsinki.fi>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A6351ADBD4 for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 13:11:46 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.903
X-Spam-Level: 
X-Spam-Status: No, score=-3.903 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0SN88gmezqtC for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 13:11:41 -0800 (PST)
Received: from courier.cs.helsinki.fi (courier.cs.helsinki.fi [128.214.9.1]) by ietfa.amsl.com (Postfix) with ESMTP id 030921AD694 for <aqm@ietf.org>; Wed, 11 Dec 2013 13:11:39 -0800 (PST)
Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by mail.cs.helsinki.fi with esmtp; Wed, 11 Dec 2013 23:11:16 +0200 id 0006807C.52A8D4F4.000052D4
Date: Wed, 11 Dec 2013 23:11:16 +0200 (EET)
From: "=?ISO-8859-15?Q?Ilpo_J=E4rvinen?=" <ilpo.jarvinen@helsinki.fi>
X-X-Sender: ijjarvin@melkinpaasi.cs.helsinki.fi
To: Dave Taht <dave.taht@gmail.com>
In-Reply-To: <CAA93jw54uJvhaRZ8OMdM4Rrxs7U=3upAd_gAkVEyXJXqRaaEyg@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1312112215020.12107@melkinpaasi.cs.helsinki.fi>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAA93jw54uJvhaRZ8OMdM4Rrxs7U=3upAd_gAkVEyXJXqRaaEyg@mail.gmail.com>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-CS-Test-DKIM: none
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, Jim Gettys <jg@freedesktop.org>, Bob Briscoe <bob.briscoe@bt.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 21:11:46 -0000

On Wed, 11 Dec 2013, Dave Taht wrote:

> On Wed, Dec 11, 2013 at 11:21 AM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> > Jim,
> >
> > At 16:55 11/12/2013, Jim Gettys wrote:
> > On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> > Jim,
> >
> > I'm just checking we're not talking past each other. I'll repeat two quotes
> > from each of us, then comment.
> >
> > On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> >
> > 3{New}. It SHOULD be possible to make different instances of an AQM
> > algorithm apply to different subsets of packets that share the same queue.
> > It SHOULD be possible to classify packets into these subsets at least by ECN
> > codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent of
> > these fields at lower layers),
> >
> >
> > At 19:50 05/12/2013, Jim Gettys wrote:
> >
> > "Certainly, it may be the same instance of an AQM algorithm, rather than
> > different instances, for example."
> >
> >
> > That's true of course, but the case with one AQM handling all packets within
> > a queue is the norm. I want to check you're happy with the converse:
> > 1) A set-up more like WRED which was based on Dave Clark's RIO (RED with in
> > and out of contract). So we can have WPIE, WCoDel etc where the
> > differentiation between aggregates is provided by different AQM instances in
> > the same queue, not by different queues with different scheduling
> > priorities.
> > 2) Extending this so that AQM differentiation can be between ECN-capable and
> > Not-ECN-capable aggregates, not just between Diffserv classes (an example
> > being CoDel with a lower 'interval' for ECN-capable packets).
> >
> > I presented the evaluations of this last idea in tsvwg on the final Friday
> > of the Vancouver IETF - I don't think you were there. <
> > http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
> >
> >
> > Yes, unfortunately I had to leave before the Friday session.
> > This is my primary motivation for this wordsmithing - I'm trying allow us to
> > move towards zero signalling delays in CoDel, PIE and RED (currently
> > defaults of 200ms, 100ms and 512packets respectively, which are not good for
> > dynamics).
> >
> >
> > Certainly signalling delays are very important: this is why I'm favorably
> > inclined to "head mark/drop", as it signals TCP as quickly as possible,
> > keeping the response of the TCP feedback loop as tight as possible (and part
> > of why I like CoDel so much for the highly variable bandwidth problem we
> > face at the edge of the net).
> >
> > It's *really* important than when the bandwidth drops suddenly that everyone
> > gets told to slow down quickly (exactly how quickly probably depends on the
> > propagation change characteristics of the medium), or packets can pile up in
> > a big way.
> >
> > How quickly the mark/drop algorithm can figure out that signalling is
> > appropriate is the *other* piece of getting good dynamics.  Here I don't
> > doubt that something may be discovered that is better than CoDel in the
> > slightest.
> > It takes a CoDel instance (within an fq structure) 200ms from its queue
> > first passing 'threshold' before it will ever drop the first packet (unless
> > the queue hits taildrop before that). So if the RTT is 20ms, that's 220ms
> > signalling delay. In fq_codel this creates considerable self-delay for short
> > flows or r-t apps, which kill their own latency before they get any loss
> > signal to tell them to slow down. Even for elastic flows, with congestion
> > signals delayed by so much, they risk hitting themselves with a huge train
> > of overshoot loss. This would be the same for fq_pie, except the number is
> > 100ms + RTT.
> 
> Things have so consistently expressed things this way that I began to
> doubt the reality myself. It seems like a large number of folk on this list
> don't get it either, so I am going to try an explain in a new way.
> 
> Tackling codel first:
> 
> the first phase of codel has effectively a "training" period where a link going
> from unloaded to loaded for the first time ever - the very first drop with the
> default interval will happen in 200ms, yes. IF it stays loaded and  over the
> target delay after the first drop/mark, it will then tune to ever
> smaller intervals
> to approximate an ideal drop rate until the latency on the link drops below the
> target. At which point the algorithm saves that rate, and stops doing anything
> until the next time the target delay is exceeded.
> 
> Some keep asserting that that is all there is to codel, saying things like
>  "there is a linear increase in drop probability" using the invsqrt mechanism,
> 
> *which is true during the training phase*.
> 
> After that approximation of ideal drop/mark rate is obtained, the algorithm
> goes quiescent until the next time the target delay is consistently exceeded,
> at which point it schedules the next drop at a little more than the stored
> previous drop rate. It then continually seeks around that point up and down.
> 
> If the delay drops below the target in this phase, the algorithm stops
> again and decreases the drop rate again, as it's too high. If the delay
> stays above target after the drop for the current value of the interval,
> the drop rate increases.
>
> This is an interesting solution to kleinrocks formulation of "power", where
> he once said an average of one packet should be in the queue, codel aims
> to never have less than one packet in the queue.
> 
> And the switch into and out of drop mode going above target is entirely
> dependent on the characteristics over time of the flows on the system,
> completely nonlinear, and where codel spends 99.999% of it's time on a
> loaded link.
> 
> As debussy said: "Music is the *space* between the notes".
> 
> I wish i had a name for this second "seeking" phase that makes as much 
> sense as "congestion avoidance".
> 
> So asserting that you'll have a 200ms interval on a codled link always 
> is just blatantly incorrect. On first boot, yes. On a busy network, 
> never again.[1]

No no. The queue empties after CoDel overshoots the marking probability 
and then CoDel stops and starts from scratch. And yes, CoDel will always 
overshoot for sure because it _controls_ until the queue is in its
control, i.e., below the threshold. How big the overshoot is, of course 
depends.

What this means in terms of TCP:

The network/or you fq-queue (in case of fq_codel if you so want) won't be 
all that busy according to CoDel once CoDel kindly "coddled" the TCPs, 
that's the whole point of CoDel I'm told :-). This is because the queue 
happens to be "under control" only if TCP backs off to below 5ms + RTT 
level of utilization. ...Now remember the effect of beta here. If the 
network remains more than "5ms busy", CoDel thinks that the queue is not 
in control and keeps shooting again and until eventually the network is no 
longer "busy". The worst case beta * (5ms + RTT) is quite small 
utilization and it takes time for a TCP to recover network busyness.

> In the event of a link going completely idle, and staying
> idle, there is hysteresis built into the code so it will retain that
> drop rate for a few
> hundred milliseconds (it's 8*interval in some versions of the code,
> 4 in others), before resetting count to 1 and the resulting estimation
> window to interval.

True, however, it only means that you'll overshoot more or the time was 
too short to retain the trained count in memory (and in that case CoDel 
forgets it like you admit). Or do you think that the magic number applied 
to count on the recall (was it "-2") works for all traffic?


-- 
 i.

From dave.taht@gmail.com  Wed Dec 11 13:56:29 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 700EF1AE0DE for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 13:56:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.7
X-Spam-Level: 
X-Spam-Status: No, score=-1.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E3RwHFtIOP-c for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 13:56:26 -0800 (PST)
Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) by ietfa.amsl.com (Postfix) with ESMTP id C99291AE01E for <aqm@ietf.org>; Wed, 11 Dec 2013 13:56:25 -0800 (PST)
Received: by mail-wg0-f49.google.com with SMTP id x12so6959150wgg.28 for <aqm@ietf.org>; Wed, 11 Dec 2013 13:56:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=4NyFOgUjG7pD3VrqZYGNIK8SmOL2k7+tIhTvyyjbVx4=; b=k3wE+RCA8CBq5fwalefX9bauGQrGmve15GgOOQXkeL4d8ImsE9mOg3NQxiPxx8pxe9 vFspc02gAvtV2Ss3CCPsS/ZlJzDhv1nheySyI2/+UeJBzLLlQ/qeChoejmoST91now1C zbIgOcyp7D/1GIYr1ldLGP6diWmVrBByL2C2k/RuSL3q6ITKHPEAhgEdoEp9qBDgp815 Psn902/02DEKOcXENzw+ZFC+x+5hfcRMf0Gjx2OoqP6t4fP6Xz3LpvdZD0HH3NsIzVi5 Fz8rX+qZXi9mNZqb9M3MyFVm6TsyYywcBWaAlQP+GJ+BlsxqCdw9YPlPFjBBXI3yWGe9 2odQ==
MIME-Version: 1.0
X-Received: by 10.180.160.239 with SMTP id xn15mr4881240wib.17.1386798979688;  Wed, 11 Dec 2013 13:56:19 -0800 (PST)
Received: by 10.217.123.69 with HTTP; Wed, 11 Dec 2013 13:56:19 -0800 (PST)
In-Reply-To: <alpine.DEB.2.02.1312112215020.12107@melkinpaasi.cs.helsinki.fi>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAA93jw54uJvhaRZ8OMdM4Rrxs7U=3upAd_gAkVEyXJXqRaaEyg@mail.gmail.com> <alpine.DEB.2.02.1312112215020.12107@melkinpaasi.cs.helsinki.fi>
Date: Wed, 11 Dec 2013 13:56:19 -0800
Message-ID: <CAA93jw6XDihCm4-f-qHVGoXfeztfzV6omzEMGO50R7NWFekegA@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: =?ISO-8859-1?Q?Ilpo_J=E4rvinen?= <ilpo.jarvinen@helsinki.fi>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, Jim Gettys <jg@freedesktop.org>, Bob Briscoe <bob.briscoe@bt.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 21:56:29 -0000

On Wed, Dec 11, 2013 at 1:11 PM, Ilpo J=E4rvinen
<ilpo.jarvinen@helsinki.fi> wrote:
> On Wed, 11 Dec 2013, Dave Taht wrote:
>
>> On Wed, Dec 11, 2013 at 11:21 AM, Bob Briscoe <bob.briscoe@bt.com> wrote=
:
>> > Jim,
>> >
>> > At 16:55 11/12/2013, Jim Gettys wrote:
>> > On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com> wro=
te:
>> > Jim,
>> >
>> > I'm just checking we're not talking past each other. I'll repeat two q=
uotes
>> > from each of us, then comment.
>> >
>> > On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote=
:
>> >
>> > 3{New}. It SHOULD be possible to make different instances of an AQM
>> > algorithm apply to different subsets of packets that share the same qu=
eue.
>> > It SHOULD be possible to classify packets into these subsets at least =
by ECN
>> > codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalen=
t of
>> > these fields at lower layers),
>> >
>> >
>> > At 19:50 05/12/2013, Jim Gettys wrote:
>> >
>> > "Certainly, it may be the same instance of an AQM algorithm, rather th=
an
>> > different instances, for example."
>> >
>> >
>> > That's true of course, but the case with one AQM handling all packets =
within
>> > a queue is the norm. I want to check you're happy with the converse:
>> > 1) A set-up more like WRED which was based on Dave Clark's RIO (RED wi=
th in
>> > and out of contract). So we can have WPIE, WCoDel etc where the
>> > differentiation between aggregates is provided by different AQM instan=
ces in
>> > the same queue, not by different queues with different scheduling
>> > priorities.
>> > 2) Extending this so that AQM differentiation can be between ECN-capab=
le and
>> > Not-ECN-capable aggregates, not just between Diffserv classes (an exam=
ple
>> > being CoDel with a lower 'interval' for ECN-capable packets).
>> >
>> > I presented the evaluations of this last idea in tsvwg on the final Fr=
iday
>> > of the Vancouver IETF - I don't think you were there. <
>> > http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>> >
>> >
>> > Yes, unfortunately I had to leave before the Friday session.
>> > This is my primary motivation for this wordsmithing - I'm trying allow=
 us to
>> > move towards zero signalling delays in CoDel, PIE and RED (currently
>> > defaults of 200ms, 100ms and 512packets respectively, which are not go=
od for
>> > dynamics).
>> >
>> >
>> > Certainly signalling delays are very important: this is why I'm favora=
bly
>> > inclined to "head mark/drop", as it signals TCP as quickly as possible=
,
>> > keeping the response of the TCP feedback loop as tight as possible (an=
d part
>> > of why I like CoDel so much for the highly variable bandwidth problem =
we
>> > face at the edge of the net).
>> >
>> > It's *really* important than when the bandwidth drops suddenly that ev=
eryone
>> > gets told to slow down quickly (exactly how quickly probably depends o=
n the
>> > propagation change characteristics of the medium), or packets can pile=
 up in
>> > a big way.
>> >
>> > How quickly the mark/drop algorithm can figure out that signalling is
>> > appropriate is the *other* piece of getting good dynamics.  Here I don=
't
>> > doubt that something may be discovered that is better than CoDel in th=
e
>> > slightest.
>> > It takes a CoDel instance (within an fq structure) 200ms from its queu=
e
>> > first passing 'threshold' before it will ever drop the first packet (u=
nless
>> > the queue hits taildrop before that). So if the RTT is 20ms, that's 22=
0ms
>> > signalling delay. In fq_codel this creates considerable self-delay for=
 short
>> > flows or r-t apps, which kill their own latency before they get any lo=
ss
>> > signal to tell them to slow down. Even for elastic flows, with congest=
ion
>> > signals delayed by so much, they risk hitting themselves with a huge t=
rain
>> > of overshoot loss. This would be the same for fq_pie, except the numbe=
r is
>> > 100ms + RTT.
>>
>> Things have so consistently expressed things this way that I began to
>> doubt the reality myself. It seems like a large number of folk on this l=
ist
>> don't get it either, so I am going to try an explain in a new way.
>>
>> Tackling codel first:
>>
>> the first phase of codel has effectively a "training" period where a lin=
k going
>> from unloaded to loaded for the first time ever - the very first drop wi=
th the
>> default interval will happen in 200ms, yes. IF it stays loaded and  over=
 the
>> target delay after the first drop/mark, it will then tune to ever
>> smaller intervals
>> to approximate an ideal drop rate until the latency on the link drops be=
low the
>> target. At which point the algorithm saves that rate, and stops doing an=
ything
>> until the next time the target delay is exceeded.
>>
>> Some keep asserting that that is all there is to codel, saying things li=
ke
>>  "there is a linear increase in drop probability" using the invsqrt mech=
anism,
>>
>> *which is true during the training phase*.
>>
>> After that approximation of ideal drop/mark rate is obtained, the algori=
thm
>> goes quiescent until the next time the target delay is consistently exce=
eded,
>> at which point it schedules the next drop at a little more than the stor=
ed
>> previous drop rate. It then continually seeks around that point up and d=
own.
>>
>> If the delay drops below the target in this phase, the algorithm stops
>> again and decreases the drop rate again, as it's too high. If the delay
>> stays above target after the drop for the current value of the interval,
>> the drop rate increases.
>>
>> This is an interesting solution to kleinrocks formulation of "power", wh=
ere
>> he once said an average of one packet should be in the queue, codel aims
>> to never have less than one packet in the queue.
>>
>> And the switch into and out of drop mode going above target is entirely
>> dependent on the characteristics over time of the flows on the system,
>> completely nonlinear, and where codel spends 99.999% of it's time on a
>> loaded link.
>>
>> As debussy said: "Music is the *space* between the notes".
>>
>> I wish i had a name for this second "seeking" phase that makes as much
>> sense as "congestion avoidance".
>>
>> So asserting that you'll have a 200ms interval on a codled link always
>> is just blatantly incorrect. On first boot, yes. On a busy network,
>> never again.[1]
>
> No no. The queue empties after CoDel overshoots the marking probability
> and then CoDel stops and starts from scratch.

No it doesn't. Usually. When the queue length drops below 5ms, codel
stops dropping packets,
and the queue does not drain to zero.

There have been several variants of the control law so far, with
varying degrees of success
across different ranges of bandwidths.

http://www.pollere.net/CoDel.html documents some of them.

If you would like to back up your assertion with data taken against
the linux codel variant, the more advanced ns2_codel, or fq_codel, please
do so. Patches are available for linux here for the more advanced stuff,
and please be aware of:

http://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmark=
ing_Codel_and_FQ_Codel

One of the biggest problems we've seen with overshoot has been dealing
with TSO and GSO
and GRO offloads, which were not modeled in ns2 - and as of linux 3.12
much of the TSO and GSO
problem were mostly solved (and don't happen in the real world of
routers in the first place, but
only on testbeds not emulating the real world.

Another problem we've seen is scheduling latency in virtualized
environments can exceed
the target. Don't do that.

Hmm=85 what else.

The original paper contained an error that was corrected shortly after it h=
it
dead tree land. The as-deployed-in-linux variant has a tendency to
overshoot in some cases,
and has issues at really high levels of flows, but in the fq_codel
universe it is usually fine.

One problem we know exists in the real world is the 5ms target is
unachievable at
very low (<4mbit) bandwidths in which case we have been increasing the
target delay
for that rate, and there is also a patch under test that eliminates
the maxpacket check
that helps in that case as well.

the largest known fq_codel deployment (free.fr) uses a simple formula
to set the target
at those rates. I don't think it's correct but the packet scheduling
portion hides
the issues here.

No, codel did not fully achieve the parameterless goal, and also was
not targeted at the
data center environments where different rules of physics apply in
it's initial release.

Development continues, but has mostly been focused on the *fq_codel variant=
s
which are superior in every benchmark we've tried against all other comers.

> And yes, CoDel will always
> overshoot for sure because it _controls_ until the queue is in its
> control, i.e., below the threshold. How big the overshoot is, of course
> depends.

I don't understand what you mean by overshoot, it will reduce the
target delay to
less than 5ms.

it can certainly reduce it well below 5ms, but inducing only that much jitt=
er
while still keeping utilization high is a goodness.

I don't have a problem if
people shoot for a higher target rates when using codel alone - pie as
submitted to the lkml has a target of 20ms and a very large estimation
window that I have not yet tested at lower bandwidths.

And nobody uses codel by itself, although given pie's problems I
am thinking of taking a harder thwack at making codel itself better,
abstracting out the hysterisis variable for example.

>
> What this means in terms of TCP:
>
> The network/or you fq-queue (in case of fq_codel if you so want) won't be
> all that busy according to CoDel once CoDel kindly "coddled" the TCPs,
> that's the whole point of CoDel I'm told :-). This is because the queue
> happens to be "under control" only if TCP backs off to below 5ms + RTT
> level of utilization. ...Now remember the effect of beta here. If the
> network remains more than "5ms busy", CoDel thinks that the queue is not
> in control and keeps shooting again and until eventually the network is n=
o
> longer "busy". The worst case beta * (5ms + RTT) is quite small
> utilization and it takes time for a TCP to recover network busyness.

You are trying to explain something to me in RED terms that doesn't
happen in the real world.

packets get smoothed into the "RTT" which serves as inflight storage.

Go take a measurement please. Supplying a packet
capture would help explain whatever you are seeing. tell us
what kernel you are using and follow the guidelines.

>
>> In the event of a link going completely idle, and staying
>> idle, there is hysteresis built into the code so it will retain that
>> drop rate for a few
>> hundred milliseconds (it's 8*interval in some versions of the code,
>> 4 in others), before resetting count to 1 and the resulting estimation
>> window to interval.
>
> True, however, it only means that you'll overshoot more or the time was
> too short to retain the trained count in memory (and in that case CoDel
> forgets it like you admit). Or do you think that the magic number applied
> to count on the recall (was it "-2") works for all traffic?

You are not looking at the current as-shipped code.

>
>
> --
>  i.



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From ilpo.jarvinen@helsinki.fi  Wed Dec 11 14:52:14 2013
Return-Path: <ilpo.jarvinen@helsinki.fi>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3819A1AE0FA for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 14:52:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.903
X-Spam-Level: 
X-Spam-Status: No, score=-3.903 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GC3NKAeNEGoc for <aqm@ietfa.amsl.com>; Wed, 11 Dec 2013 14:52:11 -0800 (PST)
Received: from courier.cs.helsinki.fi (courier.cs.helsinki.fi [128.214.9.1]) by ietfa.amsl.com (Postfix) with ESMTP id 87E831AE204 for <aqm@ietf.org>; Wed, 11 Dec 2013 14:52:09 -0800 (PST)
Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by mail.cs.helsinki.fi with esmtp; Thu, 12 Dec 2013 00:51:31 +0200 id 00068134.52A8EC73.0000039E
Date: Thu, 12 Dec 2013 00:51:31 +0200 (EET)
From: "=?ISO-8859-15?Q?Ilpo_J=E4rvinen?=" <ilpo.jarvinen@helsinki.fi>
X-X-Sender: ijjarvin@melkinpaasi.cs.helsinki.fi
To: Dave Taht <dave.taht@gmail.com>
In-Reply-To: <CAA93jw6XDihCm4-f-qHVGoXfeztfzV6omzEMGO50R7NWFekegA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1312120010440.12107@melkinpaasi.cs.helsinki.fi>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAA93jw54uJvhaRZ8OMdM4Rrxs7U=3upAd_gAkVEyXJXqRaaEyg@mail.gmail.com> <alpine.DEB.2.02.1312112215020.12107@melkinpaasi.cs.helsinki.fi> <CAA93jw6XDihCm4-f-qHVGoXfeztfzV6omzEMGO50R7NWFekegA@mail.gmail.com>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; BOUNDARY="-696250871-96745308-1386801166=:12107"
Content-ID: <alpine.DEB.2.02.1312120033060.19213@melkinpaasi.cs.helsinki.fi>
X-CS-Test-DKIM: none
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, Jim Gettys <jg@freedesktop.org>, Bob Briscoe <bob.briscoe@bt.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Dec 2013 22:52:14 -0000

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

---696250871-96745308-1386801166=:12107
Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-15
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.02.1312120033061.19213@melkinpaasi.cs.helsinki.fi>

On Wed, 11 Dec 2013, Dave Taht wrote:

> On Wed, Dec 11, 2013 at 1:11 PM, Ilpo Järvinen
> <ilpo.jarvinen@helsinki.fi> wrote:
> >
> > What this means in terms of TCP:
> >
> > The network/or you fq-queue (in case of fq_codel if you so want) won't be
> > all that busy according to CoDel once CoDel kindly "coddled" the TCPs,
> > that's the whole point of CoDel I'm told :-). This is because the queue
> > happens to be "under control" only if TCP backs off to below 5ms + RTT
> > level of utilization. ...Now remember the effect of beta here. If the
> > network remains more than "5ms busy", CoDel thinks that the queue is not
> > in control and keeps shooting again and until eventually the network is no
> > longer "busy". The worst case beta * (5ms + RTT) is quite small
> > utilization and it takes time for a TCP to recover network busyness.
> 
> You are trying to explain something to me in RED terms that doesn't
> happen in the real world.

??? This has nothing to with RED. I really don't know where you got that 
one. It's all what TCP is doing, sawtooth etc. (I know Linux uses Cubic 
so no need to point that out).

> packets get smoothed into the "RTT" which serves as inflight storage.

That's exactly the point here. When RTT >> threshold, you take MD beta 
hit, the inflight storage will be affect as well.

> Go take a measurement please. Supplying a packet
> capture would help explain whatever you are seeing. tell us
> what kernel you are using and follow the guidelines.

That particular text snippet didn't come from any particular test, just 
the algorithms (both TCP and CoDel) running in my head.

> >> In the event of a link going completely idle, and staying
> >> idle, there is hysteresis built into the code so it will retain that
> >> drop rate for a few
> >> hundred milliseconds (it's 8*interval in some versions of the code,
> >> 4 in others), before resetting count to 1 and the resulting estimation
> >> window to interval.
> >
> > True, however, it only means that you'll overshoot more or the time was
> > too short to retain the trained count in memory (and in that case CoDel
> > forgets it like you admit). Or do you think that the magic number applied
> > to count on the recall (was it "-2") works for all traffic?
> 
> You are not looking at the current as-shipped code.

Heh, it was somewhat unfair to call me on that one considering how many 
different variants of that particular recall have existed ;-). But you're 
right, it's not exactly what would happen with that variant.


-- 
 i.
---696250871-96745308-1386801166=:12107--

From bob.briscoe@bt.com  Thu Dec 12 05:53:35 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EFD7A1AE2DD for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 05:53:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.602
X-Spam-Level: 
X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7yhTFm0-fyS5 for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 05:53:30 -0800 (PST)
Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id BE5C21AE2E5 for <aqm@ietf.org>; Thu, 12 Dec 2013 05:53:29 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Thu, 12 Dec 2013 13:53:20 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Thu, 12 Dec 2013 13:53:21 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Thu, 12 Dec 2013 13:53:06 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.120.107])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rBCDr3np004004;	Thu, 12 Dec 2013 13:53:03 GMT
Message-ID: <201312121353.rBCDr3np004004@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Thu, 12 Dec 2013 13:53:01 +0000
To: Dave Taht <dave.taht@gmail.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <alpine.DEB.2.02.1312120010440.12107@melkinpaasi.cs.helsink i.fi>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAA93jw54uJvhaRZ8OMdM4Rrxs7U=3upAd_gAkVEyXJXqRaaEyg@mail.gmail.com> <alpine.DEB.2.02.1312112215020.12107@melkinpaasi.cs.helsinki.fi> <CAA93jw6XDihCm4-f-qHVGoXfeztfzV6omzEMGO50R7NWFekegA@mail.gmail.com> <alpine.DEB.2.02.1312120010440.12107@melkinpaasi.cs.helsinki.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Ilpo =?iso-8859-1?Q?J=E4rvinen?= <ilpo.jarvinen@helsinki.fi>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Dec 2013 13:53:35 -0000

Dave,

I was careful to say fq_codel, not just codel.
I was assuming the scenario that fq_codel is most=20
pitched for; a home gateway or modem.

On Wed, 11 Dec 2013, Dave Taht wrote:
> > >> In the event of a link going completely idle, and staying
> > >> idle, there is hysteresis built into the code so it will retain that
> > >> drop rate for a few
> > >> hundred milliseconds (it's 8*interval in some versions of the code,
> > >> 4 in others), before resetting count to 1 and the resulting=
 estimation
> > >> window to interval.

For fq_codel, new queues for new flows are coming=20
and going all the time, with instances of CoDel=20
coming and going as each queue comes and goes...

> > >> [1] on fq_codel the values are cached,=20
> too, but=85 well=85 I will try to explain
> > >> later. yes, more work is needed in this area.

An explanation would be useful because I admit I=20
couldn't unpick the code enough to understand how=20
the learning by one Codel instance in one queue=20
can be picked up by another Codel instance in another queue...

Whatever, passing learning from one queue to=20
another won't help whenever the whole link goes=20
idle. And an access link is idle for the vast=20
majority of the time. The average percent=20
utilisation of a customer's link /during the peak=20
hour/ is in the low single digits (ie 1-3%). The=20
link is constantly idle for periods a lot longer=20
than 8*interval (about a second) even when a user=20
is actively using it, e.g. for browsing, and even=20
streaming is only active in bursts (assuming HAS).

> > >> As debussy said: "Music is the *space* between the notes".

So, in the case that fq_codel is most targeted=20
for, the algorithm will start from scratch at the=20
start of the vast majority of new flows, which=20
means it will delay the first drop signal by=20
200ms. For TCP flows, this means slow-start will=20
tend to cause tail drop and overshoot before=20
fq_codel has hardly started to train. Further,=20
thin flows constantly re-start so they will=20
constantly experiencing this problem when their=20
idle periods are outside CoDel's brief caching time.

In the rarer case that a flow is long enough (or=20
enough other flows are present) for fq_codel to have trained...

> > >> After that approximation of ideal=20
> drop/mark rate is obtained, the algorithm
> > >> goes quiescent until the next time the=20
> target delay is consistently exceeded,
> > >> at which point it schedules the next drop=20
> at a little more than the stored
> > >> previous drop rate. It then continually=20
> seeks around that point up and down.

I believe "consistently exceeded" means "exceeded=20
for 'interval'" (100ms default). So for a long=20
flow, the signalling delay after crossing=20
threshold from below is not 200ms + RTT, but it is:
         100ms + RTT + previous drop delay + a little.
Even if the previous drop delay is tiny, the=20
100ms interval still makes it very sluggish. E.g.=20
if RTT =3D 20ms, signalling delay =3D 120ms =3D 6 round trips. Am I right?

That doesn't matter for a bulk transfer within=20
its own fq_codel queue. But it does matter for a=20
thin stream using TCP where the delay of each=20
message within the stream matters. And it does matter for a r-t stream.

Further, these long flows are rare. So, given the=20
link frequently goes inactive, the learning from=20
one will rarely still be cached when another=20
comes along. Which is why the rate of increase in=20
drop frequency during training is important to=20
get right (that's my other thread with Van &=20
Kathie, which is still unanswered).

___________
I'm not saying Codel / fq_codel is particularly=20
worse than other AQMs in respect of this=20
signalling delay. My point was that all AQMs have=20
to introduce some signalling delay if they use=20
loss as a signal, but not if they use ECN. I'm=20
just responding on CoDel / fq_codel in particular=20
because that's where Jim took this conversation, which you picked up.


Bob

At 22:51 11/12/2013, Ilpo J=E4rvinen wrote:
>On Wed, 11 Dec 2013, Dave Taht wrote:
>
> > On Wed, Dec 11, 2013 at 1:11 PM, Ilpo J=E4rvinen
> > <ilpo.jarvinen@helsinki.fi> wrote:
> > >
> > > What this means in terms of TCP:
> > >
> > > The network/or you fq-queue (in case of fq_codel if you so want) won't=
 be
> > > all that busy according to CoDel once CoDel kindly "coddled" the TCPs,
> > > that's the whole point of CoDel I'm told :-). This is because the=
 queue
> > > happens to be "under control" only if TCP backs off to below 5ms + RTT
> > > level of utilization. ...Now remember the effect of beta here. If the
> > > network remains more than "5ms busy", CoDel thinks that the queue is=
 not
> > > in control and keeps shooting again and=20
> until eventually the network is no
> > > longer "busy". The worst case beta * (5ms + RTT) is quite small
> > > utilization and it takes time for a TCP to recover network busyness.
> >
> > You are trying to explain something to me in RED terms that doesn't
> > happen in the real world.
>
>??? This has nothing to with RED. I really don't know where you got that
>one. It's all what TCP is doing, sawtooth etc. (I know Linux uses Cubic
>so no need to point that out).
>
> > packets get smoothed into the "RTT" which serves as inflight storage.
>
>That's exactly the point here. When RTT >> threshold, you take MD beta
>hit, the inflight storage will be affect as well.
>
> > Go take a measurement please. Supplying a packet
> > capture would help explain whatever you are seeing. tell us
> > what kernel you are using and follow the guidelines.
>
>That particular text snippet didn't come from any particular test, just
>the algorithms (both TCP and CoDel) running in my head.
>
> > >> In the event of a link going completely idle, and staying
> > >> idle, there is hysteresis built into the code so it will retain that
> > >> drop rate for a few
> > >> hundred milliseconds (it's 8*interval in some versions of the code,
> > >> 4 in others), before resetting count to 1 and the resulting=
 estimation
> > >> window to interval.
> > >
> > > True, however, it only means that you'll overshoot more or the time=
 was
> > > too short to retain the trained count in memory (and in that case=
 CoDel
> > > forgets it like you admit). Or do you think that the magic number=
 applied
> > > to count on the recall (was it "-2") works for all traffic?
> >
> > You are not looking at the current as-shipped code.
>
>Heh, it was somewhat unfair to call me on that one considering how many
>different variants of that particular recall have existed ;-). But you're
>right, it's not exactly what would happen with that variant.
>
>
>--
>  i.

________________________________________________________________
Bob Briscoe,                                                  BT=20


From gettysjim@gmail.com  Thu Dec 12 11:35:28 2013
Return-Path: <gettysjim@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E03A51AE416 for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 11:35:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.277
X-Spam-Level: 
X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XHtq_AW72m3F for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 11:35:21 -0800 (PST)
Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) by ietfa.amsl.com (Postfix) with ESMTP id 522C01AE40E for <aqm@ietf.org>; Thu, 12 Dec 2013 11:35:21 -0800 (PST)
Received: by mail-we0-f176.google.com with SMTP id w62so915471wes.7 for <aqm@ietf.org>; Thu, 12 Dec 2013 11:35:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=L62KF7YFNKM87CJBh9+OURpHTRwJtRwFKBHqpPBW97s=; b=HB0VsVlb1k3JQM7aED6+j2BQgScYLHkQGhbAVIAIBMy99f+pNEJnNGQzRV4S256VmI 3sj1/vHlMcwnk929JuzxwjjaXNpdnFIyjQhDr/MdaL/ZupGXunds0bVX3CXVtGP6ql4j lW9tgUhtIgN1PoAqQAqniOzyuGIZEeyGS1BUx0WLXuXRuxPhGdHwtddMbT8+2tgyFiIK bUwaraBmt4K0hbFH8TmXKcP5tDipJipTfF2QCxq/hYwm6JwZL+7xaymszUeO09CQMg0n 15KoQDvWdDLqt4atSvA3yIYtH8VSM9ubxfiWOiPped5OQ957pkdv1xF3x6WblOjbGYOl fx8A==
MIME-Version: 1.0
X-Received: by 10.194.122.99 with SMTP id lr3mr8043002wjb.21.1386876914661; Thu, 12 Dec 2013 11:35:14 -0800 (PST)
Sender: gettysjim@gmail.com
Received: by 10.227.134.74 with HTTP; Thu, 12 Dec 2013 11:35:14 -0800 (PST)
In-Reply-To: <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk>
Date: Thu, 12 Dec 2013 14:35:14 -0500
X-Google-Sender-Auth: oHeAkQW3OuALgeSL2vLfAO5P8sc
Message-ID: <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com>
From: Jim Gettys <jg@freedesktop.org>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: multipart/alternative; boundary=089e011779b5d881ac04ed5b7111
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Dec 2013 19:35:29 -0000

--089e011779b5d881ac04ed5b7111
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:

>  Jim,
>
>
> At 16:55 11/12/2013, Jim Gettys wrote:
>
>
>
> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>  Jim,
>
> I'm just checking we're not talking past each other. I'll repeat two
> quotes from each of us, then comment.
>
>  On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>
>  3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue. It
> SHOULD be possible to classify packets into these subsets at least by ECN
> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent of
> these fields at lower layers),
>
>
> At 19:50 05/12/2013, Jim Gettys wrote:
>
> "Certainly, it may be the same instance of an AQ
> 
> M algorithm, rather than different instances, for example."
>
>
> That's true of course, but the case with one AQM handling all packets
> within a queue is the norm. I want to check you're happy with the converse:
> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED with
> in and out of contract). So we can have WPIE, WCoDel etc where the
> differentiation between aggregates is provided by different AQM instances
> in the same queue, not by different queues with different scheduling
> priorities.
> 2) Extending this so that AQM differentiation can be between ECN-capable
> and Not-ECN-capable aggregates, not just between Diffserv classes (an
> example being CoDel with a lower 'interval' for ECN-capable packets).
>
> I presented the evaluations of this last idea in tsvwg on the final Friday
> of the Vancouver IETF - I don't think you were there. <
> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>
>
> Yes, unfortunately I had to leave before the Friday session.
>  This is my primary motivation for this wordsmithing - I'm trying allow
> us to move towards zero signalling delays in CoDel, PIE and RED (currently
> defaults of 200ms, 100ms and 512packets respectively, which are not good
> for dynamics).
>
>
> Certainly signalling delays are very important: this is why I'm favorably
> inclined to "head mark/drop", as it signals TCP as quickly as possible,
> keeping the response of the TCP feedback loop as tight as possible (and
> part of why I like CoDel so much for the highly variable bandwidth problem
> we face at the edge of the net).
>
> It's *really* important than when the bandwidth drops suddenly that
> everyone gets told to slow down quickly (exactly how quickly probably
> depends on the propagation change characteristics of the medium), or
> packets can pile up in a big way.
>
> How quickly the mark/drop algorithm can figure out that signalling is
> appropriate is the *other* piece of getting good dynamics.  Here I don't
> doubt that something may be discovered that is better than CoDel in the
> slightest.
> It takes a CoDel instance (within an fq structure) 200ms from its queue
> first passing 'threshold' before it will ever drop the first packet (unless
> the queue hits taildrop before that). So if the RTT is 20ms, that's 220ms
> signalling delay.
>

No, again, see Dave's mail, and you are missing the flow scheduling aspect
of this and thinking in terms of a single queue and the usual mark/drop
cases; this is the exact cognitive problem I'm talking about.

The flow scheduling aspect of fq_codel is *more* important than what
mark/drop algorithm decides to signal the TCP's to regulate their servo
systems.  I really wish this algorithm had been called "fs_codel", rather
than "fq_codel", as it is so easy to confuse "fair" with "flow", and
"queue" with "scheduling".

Regulating TCP is absolutely essential to keep TCP "sane" and responsive,
but it isn't the most important part of what is going on.

Here's the case of a new TCP flow on a previously idle link: After the TCP
open handshake, you will have no more than 4 or (if IW10 is active)
packets for that flow in the queue (actually, 3 packets, as presumably at
least one packet is in process of transmission; @ 1Mbps, that first full
size packet takes 13ms).

If another flow starts, it will get preferentially scheduled to the
existing flow(s) until it has also built a queue, at which time it competes
"fairly" with the other flows in the packet scheduling.
This means, in the simple low bandwidth case, that as soon as you start the
second flow, it gets the next possible opportunity to be transmitted in
preference to any flow that has built a queue.

Here's the kicker:

We've just avoid 3 packets worth of "head of line blocking" latency for the
second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved just in
the TCP open, and more in that the first packet from the second flow then
gets scheduled immediately, getting that TCP moving.

Similarly for your voip packet; it saves those 40ms. And your bulk flow
gets its first packet through ASAP as well; for web traffic, that usually
contains the size information or other metadata required for the web
browser to unblock.  And that is for a *single* competing TCP connection.

Now, let's look at today's web: there are many embedded objects in a page.
 Once the base page is downloaded, the web browser (presuming its DNS
lookups are already cached), opens a pile of connections all at once.
 Whether I like it or not, (which I don't, as I tried to prevent it with
pipelining in HTTP/1.1 in the 1990's), you can easily have 5-30 TCP
connections start almost simultaneously. See the connections/page plot on
my web page explaining all this stuff in more detail at:
http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queuing-traditional-aqm-is-not-enough/

Without any TCP flow control ever happening, I can easily get 40 (or even
several hundred packets!) arriving at near line rate (the initial window *
number of embedded objects).  Most of these TCP connections *won't ever
even get out of slow start*; the objects are typically pretty small.  I've
observed transient latency from HOL blocking of > 100ms on 50Mbps cable
service, on some web pages.

Nothing we will/can do at the TCP level can help this situation!!! Nothing,
other than in the long term getting the incentives right so that fewer TCP
connections might be preferable to gaming TCP, as the web already has.

That CoDel may take a while to figure out that it should start marking in
the idle case is really pretty irrelevant; the packets have already arrived
and unless we do better packet scheduling, you are fully stuck.  Dave's
trying to also explain that a number of people's understanding of how CoDel
works has been wrong.

But as I say, *the details of the mark/drop algorithm doesn't matter*.
 Having 3, or 100 packets queued in a FIFO queue will already mean you are
screwed for anything low latency at the bandwidths we all care about.

Once a flow drains to zero, it again gets treated as a new flow.
And what we choose to define a "flow" to be is arbitrary, though the code
today does the usual 5tuple.

In fq_codel this creates considerable self-delay for short flows or r-t
> apps, which kill their own latency before they get any loss signal to tell
> them to slow down.
>

Not likely in the common case.  See Dave's comments and also note that the
initial burst of web page load runs the queue up high immediately.

And as I keep saying, and will say again, the flow queuing decisions
avoiding HOL blocking explained are much more important to dealing with the
latency problems.

So go invent better algorithms that CoDel for drop/mark, that can be
applied to the flow queuing parts of the algorithm, and I'm happy.  Running
code please.




> Even for elastic flows, with congestion signals delayed by so much, they
> risk hitting themselves with a huge train of overshoot loss. This would be
> the same for fq_pie, except the number is 100ms + RTT.
>

When fq_pie exists to test, I'll be happy to see.

I keep saying, and I'll say again: while some mark/drop algorithm needs to
exist, flow scheduling is more important to getting good latency....



>
> Yes, the e2e transport could measure delay growth, but it doesn't know
> whether the delay is coming from a queue that is isolated from others or
> not. So it doesn't want to slow down too quickly in response to delay
> growth in case it gets screwed by other traffic. Ie. using delay growth as
> a signal entails considerable signalling delay due to all the uncertainty.
>
> The proposal you missed in tsvwg was to define ECN as an immediate signal
> from the network, 'interval'=0 in CoDel terms, so the host always gets
> congestion signals as fast as possible, and if it needs bursts of signals
> smoothed out, it can do that itself.
>

Yes, I get that.  I got that a year or more ago. The idea has potential
merit. ECN as it is currently defined is not so useful.

And I wish it would get a different name so we could more easily talk about
the two different things now being called ECN.

I can see having ECN marks on a burst of packets may be helpful in having
the receiver judge things in a highly variable wireless scenario; it may
have additional information about the medium and know that that transient
has gone away, and that it may not be a wise idea to slow the connection at
all.



> The suggested wording ensures all AQM implementations will allow
> operators, vendors and users to configure such a mechanism. But I've
> generalised it from ECN to Diffserv too (because the implementation would
> be no different).
>

As noted before, Diffserv is still interesting, even though the packet
scheduling in fq_codel (or similar algorithms) makes it much less
necessary. There are two aspects to this:

1) higher priority contention to the medium.  If I have a real VOIP packet,
there are ways I can ask for higher priority access to the medium and
reduce the total number of transmit opportunities my traffic requires (and
that is often the scarcest resource on WiFi or Docsis).
2) any hint I can get helps (at the edge) so I can distinguish those
packets from the way the web has been gaming the network.

Even so, for Diffserv to be safely trusted and honored even in your home,
the end user (who is the network operator in this case), will have be able
to know that a device or application is using it and control whether or not
it's honored. Unless it is under the network operator's control (in this
case, you the home user) Diffserv can/will get gamed to uselessness by
application and device manufacturers. Ergo why Dave and I hack on home
routers again: this level of control is not currently present in these
devices, and must be for Diffserv to be useful.



>
>
>
> My basic issue is one of terminology: people have talked about "best
> effort" queues.  In reality, this is a "class" of service, rather than a
> single queue, and when you get into the mental model of BE being a single
> queue, (rather than a set of queues) it can lead one astray quickly and
> easily.
>
>
> Yeah, I know this. I suspected we were talking past each other.
>
> I need you to allow the other case into your mind for this conversation.
> The wording is specifically about the case where "different subsets of
> packets ... share the same queue".
>

And the word "queue" in most people's minds implies ordering, and FIFO
behavior.  This is the terminological issue I'm harping on.  It's also why
I think "bufferbloat" is a better term for our situation than "queue
bloat", which you liked and have harped at me about.  Buffers don't have
such an implication of ordering.

So if you talked about a buffer here, rather than a queue, I'd be a lot
happier. At least in my mind, queues are ordered.


> We can talk about an fq structure for this another time, but it's a really
> complicated way of doing it.
>
Given simple looks like it could work, why get complicated already?
>

Because the flow scheduling is such a win.  You can't solve the whole
problem just with mark/drop algorithms and FIFO queues and get reliably to
decent latencies.

Now, whether Fred's document can/should go into anything like that that
detail is far from clear (arguably not).

I just don't want to further the mythology we can get to decent latencies
at the edge of the network to continue with FIFO queues and an AQM
algorithm, and it's clear that many haven't yet internalized that its flow
scheduling combined with a self tuning adaptive mark/drop algorithm that we
must have, but that the flow scheduling makes the biggest difference.


>
> It's really easy to fall into the idea of a single software queue mapping
> to some single hardware supported queue, and that's a cognitive mistake, as
> aggregating MACs are showing us; transmit ops are often the scarcest
> resource...
>
>
> It's only a cognitive mistake if one is not aware of all the options. I'm
> fully aware of all the options.
>
> To be specific, a queue into a wireless medium should be configured so it
> holds some 'good queue' in reserve for transmit ops, but the queues on top
> of this that TCP self-inflicts even briefly are not 'good queues' even if
> they are isolated from other flows by fq - VJ was wrong to generalise the
> phrase 'good queue' to all bursts of queue - it is only necessary to hold
> back from signalling and allow a burst of queue if the only possible signal
> is a drop. With ECN, you don't have this dilemma. This is the key to rapid
> dynamics.
>

But you also then have to solve the HOL blocking problem, and do so
urgently.  Ergo flow scheduling.... To get to where we need to go, you
have to worry about the order in which each packet is scheduled.


>
> Diffserv marking has the potential to give a "hint" to distinguish how
> particular flows should be handled (scheduled) in a service class, and as
> my previous example shows, that hint may be very useful in channel access
> decisions (e.g. voip on 802.11).
>
>
ECN doesn't help a bit with the head of line blocking problem; it actually
will make it worse with FIFO scheduling. ECN means that you can't even get
the packets out of the way.

Which says you'd better be doing more clever scheduling than a FIFO.

If you want ECN to be usable in the way you want it to be at low
bandwidths, you better become a fan of flow scheduling...

>
> But fq_codel teaches the lesson that packet scheduling combined with
> keeping TCP sane is a key improvement over handling either problem apart...
> In particular, the first packets of new flows/reappearing flows are vastly
> more "important" than other packets in terms of the latency cost to users
> of that service. Each flow has in essence its own queue in this service
> class, and we're using information from that to help schedule the packets
> in ways that minimize latency to the user.
>
>
> I know all this. Please can we keep to the conversation about how to avoid
> the 200ms signalling delay that fq_codel inflicts on each flow (and the
> similar signalling delays that other AQMs inflict).
>

As I said, it's not the big problem we have today at the edge of the
network, and Dave's mail explains your model of CoDel isn't so correct.

It's only very long lived flows where the signalling even matters, and
that's not what we get at the edge of today's network; instead, we get a
mix of a few larger (e.g. video player flows) with the DOS attack that is
web traffic, with some isochronous VOIP and teleconferencing traffic.

>
>
>
> So in this case, a single algorithm is acting over a bunch of flows in a
> single class of service, and both scheduling packets among the flows, and
> signalling TCP flows appropriately when they should "slow down".
>
>
> Yup, I know this.
>
>
>
> So I think you and I are on close to the same page (but have been burned
> badly in the past by terminology issues getting in the way).  On HTTP/1.1
> we wasted probably > 2 years talking past each other because we didn't have
> clear and concise terminology that we all understood the same way.
>
>
> As I thought, we are talking past each other.
>

Yes, in part because I think few have internalized what the web has done to
edge of the network.  It isn't what I had hoped it would be when I was
HTTP/1.1 editor.  Why this outcome occurred is too long a discussion for
this thread.


We need to be able to have a conversation that is not always "Hmm, that's
> sounds like it might be interesting. Can I tell you about fq_codel now?"
>

Running code of other algorithms very welcome. Fq_codel is running code.
 Pie is running code.

Maybe fq_pie will be running code we can test someday.

Even then, I want a low target latency so individual TCP's are kept
responsive, and I need an algorithm that can keep up quickly with the
dynamics of wireless, so most simplistic tests will not be useful (we don't
have such good evaluation tests today).

When it is, and if it is better than fq_codel, I'll be happy.  But the
mark/drop part of the algorithm *isn't* the most important part of the
algorithm. The packet scheduling decisions are....
                               - Jim


>
>
>
> Bob
>
>
>
> And I don't claim I have the right terminology for all this stuff, either
> (even in this mail).
>
> Which is why I was loathe to suggest exact text...
>                            - Jim
>
>
>
> At 19:50 05/12/2013, Jim Gettys wrote:
>
>
>
>  On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote: Fred,
> Gorry, all,
> I promised to suggest text for draft-ietf-aqm-recommendation about
> allowing the AQM's behaviour to be independent for ECN and non-ECN packets.
> In the process, I realised we can't talk about independent AQMs for ECN
> without also including Diffserv.
> This gets messy, because I believe a good AQM for BE traffic with and
> without ECN, should remove much if not all the need for Diffserv. But we
> can't ignore Diffserv.
>
>
> I agree in principle with what Bob is trying to say here (and is very much
> what I've been saying in my blog entry of last summer).
>
> Once you have things under control, the need for Diffserv diminishes
> dramatically (but does not go away).
>
> But as Bob notes, there is still a good use for Diffserv: suitably marked
> traffic may want to contend for access to the channel differently: your
> marked VOIP packets may want to change the priority with which you request
> channel access, so that you get more timely access to the medium. This
> conserves transmit opportunities, which is often the scarcest resource in
> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
> between your VOIP working well, and not working well, on a busy 802.11
> network as well as using the channel as efficiently as possible.
>
> Similarly, if you have packets you know are background, it's helpful to
> know that to ensure that they never contend for access to the medium but
> will always defer to other traffic, and just scavenge available space in
> other transmit opportunities where possible.
>
> I'm a bit loathe though to tie the behavior to queues, however; in
> particular, best effort traffic may want to be sent in the same aggregate
> as higher (or lower) priority traffic, if there is remaining space in the
> aggregate.
>
> In short, the mental model we've had that there is a one-to-one model of
> hardware and software queues (not to mention flows in a given software
> queue) is often incorrect (or at least seriously sub-optimal) in today's
> systems (even if the hardware queues "work" properly, which it appears they
> do not in 802.11).
>
> So I'm not sure Bob's new section 3 here is how to best to state this (or
> to deal with the terminology problem).  Certainly, it may be the same
> instance of an AQM algorithm, rather than different instances, for
> example.  And "
> It SHOULD be possible" is more a pious wish than anything else.  But I
> agree in spirit with what Bob's trying to say.
>                                 - Jim
>
>  _________________________________________________________________________________________
> {In Section 4: add another bullet between recommendations 2 & 3:}
> 3{New}. It SHOULD be possible to make different instances of an AQM
> algorithm apply to different subsets of packets that share the same queue. It
> SHOULD be possible to classify packets into these subsets at least by ECN
> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent of
> these fields at lower layers).
> {Then a new section to expand on this before the current Section 4.3.} 4.3{New}.
> Independent AQM Instances for ECN and Diffserv
> The recommendation to provide a separate instance of the AQM for ECN
> packets goes beyond the assumptions of RFC 3168, which assumed that only
> one instance of an AQM will handle both ECN-capable and non-ECN-capable
> packets.
>
>
> Bob
>
>
> ________________________________________________________________ Bob
> Briscoe,                                                  BT  _______________________________________________
> aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>  ________________________________________________________________
> Bob Briscoe,                                                  BT
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
> I can imagine many other possible algorithms that CoDel for the mark/drop
algorithm; we happen to like CoDel atm for two reasons: 1) it is self
adapting to the line rate, and 2) head mark/drop signals the TCP's as soon
as a decision is made rather than possibly being applied much later (such
as random or tail drop).  We welcome and encourage further essays in the
art.


--089e011779b5d881ac04ed5b7111
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:small"><br=
></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Wed=
, Dec 11, 2013 at 2:21 PM, Bob Briscoe <span dir=3D"ltr">&lt;<a href=3D"mai=
lto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.com</a>&gt;</span>=
 wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div>
Jim,<div class=3D"im"><br><br>
At 16:55 11/12/2013, Jim Gettys wrote:<br><br>
<br><br>
<blockquote type=3D"cite">On Tue, Dec 10, 2013 at 10:04
PM, Bob Briscoe
&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.=
com</a>&gt;
wrote:<br>

<dl>
<dd>Jim,<br><br>

</dd><dd>I&#39;m just checking we&#39;re not talking past each other. I&#39=
;ll repeat two
quotes from each of us, then comment.<br><br>
<blockquote type=3D"cite">
<dd>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe
&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.=
com</a>&gt;
wrote:<blockquote type=3D"cite">
<dl>
<dd>3{New}. It SHOULD be possible to make different instances of an AQM
algorithm apply to different subsets of packets that share the same
queue.=20
</dd><dd>It SHOULD be possible to classify packets into these subsets at le=
ast
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers),</dd></dl></blockquote></dd></b=
lockquote></dd></dl></blockquote>
<br>

<dd>At 19:50 05/12/2013, Jim Gettys wrote:<br>
<blockquote type=3D"cite">
<dd>&quot;Certainly, it may be the same instance of an AQ<div class=3D"gmai=
l_default" style=3D"font-size:small;display:inline"></div>M algorithm,
rather than different instances, for example.&quot;</dd></blockquote><br>

</dd><dd>That&#39;s true of course, but the case with one AQM handling all =
packets
within a queue is the norm. I want to check you&#39;re happy with the
converse: <br>

</dd><dd>1) A set-up more like WRED which was based on Dave Clark&#39;s RIO=
 (RED
with in and out of contract). So we can have WPIE, WCoDel etc where the
differentiation between aggregates is provided by different AQM instances
in the same queue, not by different queues with different scheduling
priorities.<br>

</dd><dd>2) Extending this so that AQM differentiation can be between
ECN-capable and Not-ECN-capable aggregates, not just between Diffserv
classes (an example being CoDel with a lower &#39;interval&#39; for ECN-cap=
able
packets).<br><br>

</dd><dd>I presented the evaluations of this last idea in tsvwg on the fina=
l
Friday of the Vancouver IETF - I don&#39;t think you were there. &lt;
<a href=3D"http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf=
" target=3D"_blank">
http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf</a>
&gt;<br>
<br>

<br>
Yes, unfortunately I had to leave before the Friday session.<br>

<dl>
<dd>This is my primary motivation for this wordsmithing - I&#39;m trying
allow us to move towards zero signalling delays in CoDel, PIE and RED
(currently defaults of 200ms, 100ms and 512packets respectively, which
are not good for dynamics).<br><br>

</dd></dl><br>
Certainly signalling delays are very important: this is why I&#39;m favorab=
ly
inclined to &quot;head mark/drop&quot;, as it signals TCP as quickly as
possible, keeping the response of the TCP feedback loop as tight as
possible (and part of why I like CoDel so much for the highly variable
bandwidth problem we face at the edge of the net).=A0 <br><br>
It&#39;s *really* important than when the bandwidth drops suddenly that
everyone gets told to slow down quickly (exactly how quickly probably
depends on the propagation change characteristics of the medium), or
packets can pile up in a big way.<br><br>
How quickly the mark/drop algorithm can figure out that signalling is
appropriate is the *other* piece of getting good dynamics.=A0 Here I
don&#39;t doubt that something may be discovered that is better than CoDel =
in
the slightest.</dd><br></div>
It takes a CoDel instance (within an fq structure) 200ms from its queue
first passing &#39;threshold&#39; before it will ever drop the first packet
(unless the queue hits taildrop before that). So if the RTT is 20ms,
that&#39;s 220ms signalling delay.</div></blockquote><div><br></div><div cl=
ass=3D"gmail_default" style=3D"font-size:small">No, again, see Dave&#39;s m=
ail, and you are missing the flow scheduling aspect of this and thinking in=
 terms of a single queue and the usual mark/drop cases; this is the exact c=
ognitive problem I&#39;m talking about.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">The flow scheduling aspect of =
fq_codel is *more* important than what mark/drop algorithm decides to signa=
l the TCP&#39;s to regulate their servo systems. =A0I really wish this algo=
rithm had been called &quot;fs_codel&quot;, rather than &quot;fq_codel&quot=
;, as it is so easy to confuse &quot;fair&quot; with &quot;flow&quot;, and =
&quot;queue&quot; with &quot;scheduling&quot;.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Regulating TCP is absolutely e=
ssential to keep TCP &quot;sane&quot; and responsive, but it isn&#39;t the =
most important part of what is going on. =A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Here&#39;s the case of a new T=
CP flow on a previously idle link: After the TCP open handshake, you will h=
ave no more than 4 or (if IW10 is active) packets for that flow in the queu=
e (actually, 3 packets, as presumably at least one packet is in process of =
transmission; @ 1Mbps, that first full size packet takes 13ms).</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">If another flow starts, it wil=
l get preferentially scheduled to the existing flow(s) until it has also bu=
ilt a queue, at which time it competes &quot;fairly&quot; with the other fl=
ows in the packet scheduling.</div>
<div class=3D"gmail_default" style=3D"font-size:small">This means, in the s=
imple low bandwidth case, that as soon as you start the second flow, it get=
s the next possible opportunity to be transmitted in preference to any flow=
 that has built a queue.<br>
</div><div class=3D"gmail_default" style=3D"font-size:small"><br></div><div=
 class=3D"gmail_default" style=3D"font-size:small">Here&#39;s the kicker:</=
div><div class=3D"gmail_default" style=3D"font-size:small"><br></div><div c=
lass=3D"gmail_default" style=3D"font-size:small">
We&#39;ve just avoid 3 packets worth of &quot;head of line blocking&quot; l=
atency for the second flow to &quot;do its thing&quot;. =A0@1Mbs, that is 4=
0ms (3*13ms) saved just in the TCP open, and more in that the first packet =
from the second flow then gets scheduled immediately, getting that TCP movi=
ng.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Similarly for your voip packet=
; it saves those 40ms. And your bulk flow gets its first packet through ASA=
P as well; for web traffic, that usually contains the size information or o=
ther metadata required for the web browser to unblock. =A0And that is for a=
 *single* competing TCP connection.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Now, let&#39;s look at today&#=
39;s web: there are many embedded objects in a page. =A0Once the base page =
is downloaded, the web browser (presuming its DNS lookups are already cache=
d), opens a pile of connections all at once. =A0Whether I like it or not, (=
which I don&#39;t, as I tried to prevent it with pipelining in HTTP/1.1 in =
the 1990&#39;s), you can easily have 5-30 TCP connections start almost simu=
ltaneously. See the connections/page plot on my web page explaining all thi=
s stuff in more detail at:=A0<a href=3D"http://gettys.wordpress.com/2013/07=
/10/low-latency-requires-smart-queuing-traditional-aqm-is-not-enough/">http=
://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queuing-tradi=
tional-aqm-is-not-enough/</a></div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Without any TCP flow control e=
ver happening, I can easily get 40 (or even several hundred packets!) arriv=
ing at near line rate (the initial window * number of embedded objects). =
=A0Most of these TCP connections *won&#39;t ever even get out of slow start=
*; the objects are typically pretty small. =A0I&#39;ve observed transient l=
atency from HOL blocking of &gt; 100ms on 50Mbps cable service, on some web=
 pages. =A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Nothing we will/can do at the =
TCP level can help this situation!!! Nothing, other than in the long term g=
etting the incentives right so that fewer TCP connections might be preferab=
le to gaming TCP, as the web already has.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">That CoDel may take a while to=
 figure out that it should start marking in the idle case is really pretty =
irrelevant; the packets have already arrived and unless we do better packet=
 scheduling, you are fully stuck. =A0Dave&#39;s trying to also explain that=
 a number of people&#39;s understanding of how CoDel works has been wrong. =
=A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">But as I say, *the details of =
the mark/drop algorithm doesn&#39;t matter*. =A0Having 3, or 100 packets qu=
eued in a FIFO queue will already mean you are screwed for anything low lat=
ency at the bandwidths we all care about.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Once a flow drains to zero, it=
 again gets treated as a new flow.</div><div class=3D"gmail_default" style=
=3D"font-size:small">
And what we choose to define a &quot;flow&quot; to be is arbitrary, though =
the code today does the usual 5tuple.</div><div class=3D"gmail_default" sty=
le=3D"font-size:small"><br></div><blockquote class=3D"gmail_quote" style=3D=
"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,2=
04,204);border-left-style:solid;padding-left:1ex">
<div>In fq_codel this creates considerable
self-delay for short flows or r-t apps, which kill their own latency
before they get any loss signal to tell them to slow down.</div></blockquot=
e><div><br></div><div class=3D"gmail_default" style=3D"font-size:small">Not=
 likely in the common case. =A0See Dave&#39;s comments and also note that t=
he initial burst of web page load runs the queue up high immediately.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">And as I keep saying, and will=
 say again, the flow queuing decisions avoiding HOL blocking explained are =
much more important to dealing with the latency problems.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">So go invent better algorithms=
 that CoDel for drop/mark, that can be applied to the flow queuing parts of=
 the algorithm, and I&#39;m happy. =A0Running code please.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div><br><=
/div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border=
-left-style:solid;padding-left:1ex">
<div>Even for
elastic flows, with congestion signals delayed by so much, they risk
hitting themselves with a huge train of overshoot loss. This would be the
same for fq_pie, except the number is 100ms + RTT.<br></div></blockquote><d=
iv><br></div><div><div class=3D"gmail_default" style=3D"font-size:small">Wh=
en fq_pie exists to test, I&#39;ll be happy to see.</div><br></div><div>
<div class=3D"gmail_default" style=3D"font-size:small">I keep saying, and I=
&#39;ll say again: while some mark/drop algorithm needs to exist, flow sche=
duling is more important to getting good latency....</div><br></div><div>
=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-styl=
e:solid;padding-left:1ex"><div><br>
Yes, the e2e transport could measure delay growth, but it doesn&#39;t know
whether the delay is coming from a queue that is isolated from others or
not. So it doesn&#39;t want to slow down too quickly in response to delay
growth in case it gets screwed by other traffic. Ie. using delay growth
as a signal entails considerable signalling delay due to all the
uncertainty.<br><br>
The proposal you missed in tsvwg was to define ECN as an immediate signal
from the network, &#39;interval&#39;=3D0 in CoDel terms, so the host always=
 gets
congestion signals as fast as possible, and if it needs bursts of signals
smoothed out, it can do that itself.<br></div></blockquote><div><br></div><=
div class=3D"gmail_default" style=3D"font-size:small">Yes, I get that. =A0I=
 got that a year or more ago. The idea has potential merit. ECN as it is cu=
rrently defined is not so useful.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">And I wish it would get a diff=
erent name so we could more easily talk about the two different things now =
being called ECN.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">I can see having ECN marks on =
a burst of packets may be helpful in having the receiver judge things in a =
highly variable wireless scenario; it may have additional information about=
 the medium and know that that transient has gone away, and that it may not=
 be a wise idea to slow the connection at all.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small"><br></div><blockquote class=3D=
"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;borde=
r-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><br>
The suggested wording ensures all AQM implementations will allow
operators, vendors and users to configure such a mechanism. But I&#39;ve
generalised it from ECN to Diffserv too (because the implementation would
be no different).</div></blockquote><div><br></div><div class=3D"gmail_defa=
ult" style=3D"font-size:small">As noted before, Diffserv is still interesti=
ng, even though the packet scheduling in fq_codel (or similar algorithms) m=
akes it much less necessary. There are two aspects to this:</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">1) higher priority contention =
to the medium. =A0If I have a real VOIP packet, there are ways I can ask fo=
r higher priority access to the medium and reduce the total number of trans=
mit opportunities my traffic requires (and that is often the scarcest resou=
rce on WiFi or Docsis).</div>
<div class=3D"gmail_default" style=3D"font-size:small">2) any hint I can ge=
t helps (at the edge) so I can distinguish those packets from the way the w=
eb has been gaming the network.</div><div class=3D"gmail_default" style=3D"=
font-size:small">
<br></div><div class=3D"gmail_default" style=3D"font-size:small">Even so, f=
or Diffserv to be safely trusted and honored even in your home, the end use=
r (who is the network operator in this case), will have be able to know tha=
t a device or application is using it and control whether or not it&#39;s h=
onored. Unless it is under the network operator&#39;s control (in this case=
, you the home user) Diffserv can/will get gamed to uselessness by applicat=
ion and device manufacturers. Ergo why Dave and I hack on home routers agai=
n: this level of control is not currently present in these devices, and mus=
t be for Diffserv to be useful.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small"></div><blockquote class=3D"gma=
il_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-le=
ft-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div class=3D"im"><br><br>
<br>
<blockquote type=3D"cite">My basic issue is one of
terminology: people have talked about &quot;best effort&quot;
queues.=A0 In reality, this is a &quot;class&quot; of service, rather
than a single queue, and when you get into the mental model of BE being a
single queue, (rather than a set of queues) it can lead one astray
quickly and easily.</blockquote><br></div>
Yeah, I know this. I suspected we were talking past each other.<br><br>
I need you to allow the other case into your mind for this conversation.
The wording is specifically about the case where &quot;different subsets
of packets ... share the same queue&quot;. <br></div></blockquote><div><br>=
</div><div class=3D"gmail_default" style=3D"font-size:small">And the word &=
quot;queue&quot; in most people&#39;s minds implies ordering, and FIFO beha=
vior. =A0This is the terminological issue I&#39;m harping on. =A0It&#39;s a=
lso why I think &quot;bufferbloat&quot; is a better term for our situation =
than &quot;queue bloat&quot;, which you liked and have harped at me about. =
=A0Buffers don&#39;t have such an implication of ordering.</div>
<div class=3D"gmail_default" style=3D"font-size:small"></div><div class=3D"=
gmail_default" style=3D"font-size:small">So if you talked about a buffer he=
re, rather than a queue, I&#39;d be a lot happier. At least in my mind, que=
ues are ordered.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width=
:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-lef=
t:1ex">
<div><br>
We can talk about an fq structure for this another time, but it&#39;s a
really complicated way of doing it.</div></blockquote><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border=
-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div=
>
Given simple looks like it could
work, why get complicated already?</div></blockquote><div><br></div><div cl=
ass=3D"gmail_default" style=3D"font-size:small">Because the flow scheduling=
 is such a win. =A0You can&#39;t solve the whole problem just with mark/dro=
p algorithms and FIFO queues and get reliably to decent latencies.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Now, whether Fred&#39;s docume=
nt can/should go into anything like that that detail is far from clear (arg=
uably not). =A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">I just don&#39;t want to furth=
er the mythology we can get to decent latencies at the edge of the network =
to continue with FIFO queues and an AQM algorithm, and it&#39;s clear that =
many haven&#39;t yet internalized that its flow scheduling combined with a =
self tuning adaptive mark/drop algorithm that we must have, but that the fl=
ow scheduling makes the biggest difference.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width=
:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-lef=
t:1ex">
<div><div class=3D"im"><br><br>
<blockquote type=3D"cite">It&#39;s really easy to fall into
the idea of a single software queue mapping to some single hardware
supported queue, and that&#39;s a cognitive mistake, as aggregating MACs ar=
e
showing us; transmit ops are often the scarcest
resource...</blockquote><br></div>
It&#39;s only a cognitive mistake if one is not aware of all the options. I=
&#39;m
fully aware of all the options. <br><br>
To be specific, a queue into a wireless medium should be configured so it
holds some &#39;good queue&#39; in reserve for transmit ops, but the queues=
 on
top of this that TCP self-inflicts even briefly are not &#39;good queues&#3=
9;
even if they are isolated from other flows by fq - VJ was wrong to
generalise the phrase &#39;good queue&#39; to all bursts of queue - it is o=
nly
necessary to hold back from signalling and allow a burst of queue if the
only possible signal is a drop. With ECN, you don&#39;t have this dilemma.
This is the key to rapid dynamics.</div></blockquote><div><br></div><div cl=
ass=3D"gmail_default" style=3D"font-size:small">But you also then have to s=
olve the HOL blocking problem, and do so urgently. =A0Ergo flow scheduling.=
... To get to where we need to go, you have to worry about the order in whi=
ch each packet is scheduled.</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width=
:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-lef=
t:1ex">
<div><div class=3D"im"><br><br>
<blockquote type=3D"cite">Diffserv marking has the
potential to give a &quot;hint&quot; to distinguish how particular flows
should be handled (scheduled) in a service class, and as my previous
example shows, that hint may be very useful in channel access decisions
(e.g. voip on 802.11).<br></blockquote></div></div></blockquote><div><br></=
div><div class=3D"gmail_default" style=3D"font-size:small">ECN doesn&#39;t =
help a bit with the head of line blocking problem; it actually will make it=
 worse with FIFO scheduling. ECN means that you can&#39;t even get the pack=
ets out of the way.=A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Which says you&#39;d better be=
 doing more clever scheduling than a FIFO.</div><div class=3D"gmail_default=
" style=3D"font-size:small">
<br></div><div class=3D"gmail_default" style=3D"font-size:small">If you wan=
t ECN to be usable in the way you want it to be at low bandwidths, you bett=
er become a fan of flow scheduling...<br></div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-c=
olor:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div class=3D"im"><blockquote type=3D"cite"><br>
But fq_codel teaches the lesson that packet scheduling combined with
keeping TCP sane is a key improvement over handling either problem
apart... In particular, the first packets of new flows/reappearing flows
are vastly more &quot;important&quot; than other packets in terms of the
latency cost to users of that service. Each flow has in essence its own
queue in this service class, and we&#39;re using information from that to
help schedule the packets in ways that minimize latency to the
user.</blockquote><br></div>
I know all this. Please can we keep to the conversation about how to
avoid the 200ms signalling delay that fq_codel inflicts on each flow (and
the similar signalling delays that other AQMs inflict).</div></blockquote><=
div><br></div><div class=3D"gmail_default" style=3D"font-size:small">As I s=
aid, it&#39;s not the big problem we have today at the edge of the network,=
 and Dave&#39;s mail explains your model of CoDel isn&#39;t so correct.</di=
v>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">It&#39;s only very long lived =
flows where the signalling even matters, and that&#39;s not what we get at =
the edge of today&#39;s network; instead, we get a mix of a few larger (e.g=
. video player flows) with the DOS attack that is web traffic, with some is=
ochronous VOIP and teleconferencing traffic.</div>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div><div class=3D"im"><br><br>
<br>
<blockquote type=3D"cite">So in this case, a single
algorithm is acting over a bunch of flows in a single class of service,
and both scheduling packets among the flows, and signalling TCP flows
appropriately when they should &quot;slow down&quot;.</blockquote><br></div=
>
Yup, I know this.<div class=3D"im"><br><br>
<br>
<blockquote type=3D"cite">So I think you and I are on
close to the same page (but have been burned badly in the past by
terminology issues getting in the way).=A0 On HTTP/1.1 we wasted
probably &gt; 2 years talking past each other because we didn&#39;t have
clear and concise terminology that we all understood the same
way.</blockquote><br></div>
As I thought, we are talking past each other. </div></blockquote><div><br><=
/div><div class=3D"gmail_default" style=3D"font-size:small">Yes, in part be=
cause I think few have internalized what the web has done to edge of the ne=
twork. =A0It isn&#39;t what I had hoped it would be when I was HTTP/1.1 edi=
tor. =A0Why this outcome occurred is too long a discussion for this thread.=
</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small"><br></div><blockquote class=3D=
"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;borde=
r-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>We need to be able to have
a conversation that is not always &quot;Hmm, that&#39;s sounds like it migh=
t
be interesting. Can I tell you about fq_codel now?&quot;</div></blockquote>=
<div><br></div><div class=3D"gmail_default" style=3D"font-size:small">Runni=
ng code of other algorithms very welcome. Fq_codel is running code. =A0Pie =
is running code. =A0</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">Maybe fq_pie will be running c=
ode we can test someday. =A0</div><div class=3D"gmail_default" style=3D"fon=
t-size:small">
<br></div><div class=3D"gmail_default" style=3D"font-size:small">Even then,=
 I want a low target latency so individual TCP&#39;s are kept responsive, a=
nd I need an algorithm that can keep up quickly with the dynamics of wirele=
ss, so most simplistic tests will not be useful (we don&#39;t have such goo=
d evaluation tests today).</div>
<div class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=
=3D"gmail_default" style=3D"font-size:small">When it is, and if it is bette=
r than fq_codel, I&#39;ll be happy. =A0But the mark/drop part of the algori=
thm *isn&#39;t* the most important part of the algorithm. The packet schedu=
ling decisions are....</div>
<div class=3D"gmail_default" style=3D"font-size:small">=A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0- Jim</div><div class=3D"gmail_d=
efault" style=3D"font-size:small"><br></div><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-colo=
r:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span class=3D""><font color=3D"#888888"><br><br>
<br><br>
Bob</font></span><div><div class=3D"h5"><br><br>
<br>
<blockquote type=3D"cite">And I don&#39;t claim I have the
right terminology for all this stuff, either (even in this
mail).<br><br>
Which is why I was loathe to suggest exact text...<br>
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0
- Jim<br><br>

<dl><br><br>

<dd>At 19:50 05/12/2013, Jim Gettys wrote:<br><br>
<br><br>
<blockquote type=3D"cite">
<dd>On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe
&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.=
com</a>&gt;
wrote:
<dl>
<dd>Fred, Gorry, all,<br>

</dd><dd>I promised to suggest text for draft-ietf-aqm-recommendation about
allowing the AQM&#39;s behaviour to be independent for ECN and non-ECN
packets. In the process, I realised we can&#39;t talk about independent AQM=
s
for ECN without also including Diffserv.<br>

</dd><dd>This gets messy, because I believe a good AQM for BE traffic with =
and
without ECN, should remove much if not all the need for Diffserv. But we
can&#39;t ignore Diffserv.<br><br>
</dd></dl></dd></blockquote></dd></dl><br>

<dd>I agree in principle with what Bob is trying to say here (and is very
much what I&#39;ve been saying in my blog entry of last summer).<br><br>

</dd><dd>Once you have things under control, the need for Diffserv diminish=
es
dramatically (but does not go away).<br><br>

</dd><dd>But as Bob notes, there is still a good use for Diffserv: suitably
marked traffic may want to contend for access to the channel differently:
your marked VOIP packets may want to change the priority with which you
request channel access, so that you get more timely access to the medium.
This conserves transmit opportunities, which is often the scarcest
resource in many systems (e.g. 802.11, DOCSIS, etc.). This can be the
difference between your VOIP working well, and not working well, on a
busy 802.11 network as well as using the channel as efficiently as
possible.<br><br>

</dd><dd>Similarly, if you have packets you know are background, it&#39;s h=
elpful
to know that to ensure that they never contend for access to the medium
but will always defer to other traffic, and just scavenge available space
in other transmit opportunities where possible.<br><br>

</dd><dd>I&#39;m a bit loathe though to tie the behavior to queues, however=
; in
particular, best effort traffic may want to be sent in the same aggregate
as higher (or lower) priority traffic, if there is remaining space in the
aggregate.=A0 <br><br>

</dd><dd>In short, the mental model we&#39;ve had that there is a one-to-on=
e model
of hardware and software queues (not to mention flows in a given software
queue) is often incorrect (or at least seriously sub-optimal) in today&#39;=
s
systems (even if the hardware queues &quot;work&quot; properly, which it
appears they do not in 802.11).<br><br>

</dd><dd>So I&#39;m not sure Bob&#39;s new section 3 here is how to best to=
 state this
(or to deal with the terminology problem).=A0 Certainly, it may be the
same instance of an AQM algorithm, rather than different instances, for
example.=A0 And &quot;<br>

</dd><dd>It SHOULD be possible&quot; is more a pious wish than anything
else.=A0 But I agree in spirit with what Bob&#39;s trying to say.<br>

</dd><dd>
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0
- Jim<br><br>
<dl>
<dd>
___________________________________________________________________________=
______________
</dd><dd>{In Section 4: add another bullet between recommendations 2 &amp;
3:}<br>

</dd><dd>3{New}. It SHOULD be possible to make different instances of an AQ=
M
algorithm apply to different subsets of packets that share the same
queue.=20
</dd><dd>It SHOULD be possible to classify packets into these subsets at le=
ast
by ECN codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the
equivalent of these fields at lower layers).<br>

</dd><dd>{Then a new section to expand on this before the current Section
4.3.}
</dd><dd>4.3{New}. Independent AQM Instances for ECN and Diffserv<br>

</dd><dd>The recommendation to provide a separate instance of the AQM for E=
CN
packets goes beyond the assumptions of RFC 3168, which assumed that only
one instance of an AQM will handle both ECN-capable and non-ECN-capable
packets.<br>
<br>
<br>

</dd><dd>Bob<br>
<br>
<br>

</dd><dd>________________________________________________________________
</dd><dd>Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT=A0=20
</dd><dd>_______________________________________________
</dd><dd>aqm mailing list
</dd><dd><a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a>
</dd><dd><a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_b=
lank">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>
</dd></dl></dd><br>

<dd>_______________________________________________<br>

</dd><dd>aqm mailing list<br>

</dd><dd><a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a>=
<br>

</dd><dd><a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_b=
lank">
https://www.ietf.org/mailman/listinfo/aqm</a></dd></blockquote><br>

<dd>________________________________________________________________<br>

</dd><dd>Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT<br><br>

</dd><dd>_______________________________________________<br>

</dd><dd>aqm mailing list<br>

</dd><dd><a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a>=
<br>

</dd><dd><a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_b=
lank">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dd><br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">
https://www.ietf.org/mailman/listinfo/aqm</a>
<u></u><p><u></u>
________________________________________________________________<br>
Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT</p></div></div></div>

<br>_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><br>
<br></blockquote></div><div class=3D"gmail_default" style=3D"font-size:smal=
l">I can imagine many other possible algorithms that CoDel for the mark/dro=
p algorithm; we happen to like CoDel atm for two reasons: 1) it is self ada=
pting to the line rate, and 2) head mark/drop signals the TCP&#39;s as soon=
 as a decision is made rather than possibly being applied much later (such =
as random or tail drop). =A0We welcome and encourage further essays in the =
art.</div>
<div class=3D"gmail_default" style=3D"font-size:small"></div><br></div></di=
v>

--089e011779b5d881ac04ed5b7111--

From dave.taht@gmail.com  Thu Dec 12 14:11:46 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 887591ADF58 for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 14:11:46 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4z3tfefsIb-b for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 14:11:40 -0800 (PST)
Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) by ietfa.amsl.com (Postfix) with ESMTP id CE2FC1AE01E for <aqm@ietf.org>; Thu, 12 Dec 2013 14:11:39 -0800 (PST)
Received: by mail-wi0-f178.google.com with SMTP id bz8so257999wib.11 for <aqm@ietf.org>; Thu, 12 Dec 2013 14:11:33 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=GWCw0ZM+EJ4Tg7qk0tOgJrepC4X2K+YvQGv0mDyskuw=; b=lhbtJZiagM1j8vQFdeDmSDkrX4ZFLTvfji/OZSkfgo6so4Jx2y3XEeCNEz7xAyVcQT fY/0K1yBro4kpgu8iFaWa7x2S1oG09h2w1X6MISnVqzryENQag0uUwW79ZLPwl7s6WCj BFg9/mHk+2UhOGx0fJ8ksuK68p8fKXHgNxi6jt9CuXf4Ce7Q6j3MkIOqB3oyUVupBv7Y qdEXjp8EDTXGCkiTrwK0pMjxDMl5RRg6eGBzYo/WKjXJ2LjyeAxdoMTAZSuhxAR8vXlr hnS4yuqJw3+5HVZ/lH3/XPJS9uZGUYdYHJzuMLAd8b9jaPzK1Ek4yt2ULh9Stx+sySa7 9f3g==
MIME-Version: 1.0
X-Received: by 10.194.77.44 with SMTP id p12mr35008wjw.94.1386886293275; Thu, 12 Dec 2013 14:11:33 -0800 (PST)
Received: by 10.217.123.69 with HTTP; Thu, 12 Dec 2013 14:11:33 -0800 (PST)
In-Reply-To: <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com>
Date: Thu, 12 Dec 2013 14:11:33 -0800
Message-ID: <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: Jim Gettys <jg@freedesktop.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Dec 2013 22:11:46 -0000

I do not have time to track this fork of the discussion right now. For
anyone interested in what I'm doing instead, feel free to read:

http://conferences.sigcomm.org/co-next/2013/program/p133.pdf -
microsecond burst latency issues "bullet trains" - which is something
that was just dramatically improved in the paced fq scheduler and new
TSQ code in linux 3.12

http://conferences.sigcomm.org/co-next/2013/program/p121.pdf - on
limitations of network acceleration

http://conferences.sigcomm.org/co-next/2013/program/p357.pdf -

http://cs.ucsb.edu/~laradeek/Secon13.pdf - nifty new wifi scheduling algo

http://conferences.sigcomm.org/co-next/2013/program/p345.pdf

http://conferences.sigcomm.org/co-next/2013/program/p369.pdf

and start a new thread on those. On the bloat list. where the theory
discussions are more apropos.

but quickly...

Bob, I object to your characterization of users links being busy 1-3%
of the time. That's an average. When they are busy, they are very busy
for short periods, typically 2-16 seconds in the case of web traffic,
then idle for minutes. DASH traffic is busy for 2+ seconds every 10 on
a 20mbit link, and so on, for 1.5 hours or so. Etc. In both cases
congestion exists, and in both cases AQM reaction time measured in
200ms or so is still vastly superior to what happens today, and packet
scheduling masks it to a large extent.

Jim, yes, I was trying to establish the groundwork for ensuring
everyone really understood codel-by-itself before talking about
fq_codel. I'm still not sure that's been established. Anyone here care
to calculate the number of drops on two flows going through codel and
fq_codel, starting with iw10, over a 10mbit 2ms RTT link? And when,
approximately, the queue becomes ideal? And how often the fq_codel
"fast queue" gets used in this case? Gold stars for everyone that gets
it right.

Jim, I'd like you to use larger download speeds than a mbit for your
examples. somewhere between 8 and 20mbit seems appropriate. (IMHO iw10
should not be used on the modern internet on sub 10Mbit links.) The
dynamics change significantly as you get more bandwidth than iw10
abuses.

Anyway=85

After I got as far as describing fq_codel accurately in this thread,
then I'd hoped to be able to tackle the immediate ECN issue, the value
of randomness in pie, and the effectiveness and need for ECN on the
edge as it is currently defined.

and I figured that then I might have written enough to get closer to an rfc=
.

and when I started kibitzing on this thread I thought I was talking to
the DCTCP case which I've spent a few months studying up on, and
looking over alternative ideas there, like

http://conferences.sigcomm.org/co-next/2013/program/p49.pdf
http://conferences.sigcomm.org/co-next/2013/program/p151.pdf

Scalable, Optimal Flow Routing in Datacenters via Local Link Balancing

http://www.irt-systemx.fr/wp-content/uploads/2013/12/AINTEC.ppt

Sigh. I'll

On Thu, Dec 12, 2013 at 11:35 AM, Jim Gettys <jg@freedesktop.org> wrote:
>
>
>
> On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>>
>> Jim,
>>
>>
>> At 16:55 11/12/2013, Jim Gettys wrote:
>>
>>
>>
>> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com> wrote=
:
>> Jim,
>>
>> I'm just checking we're not talking past each other. I'll repeat two
>> quotes from each of us, then comment.
>>
>> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>>
>> 3{New}. It SHOULD be possible to make different instances of an AQM
>> algorithm apply to different subsets of packets that share the same queu=
e.
>> It SHOULD be possible to classify packets into these subsets at least by=
 ECN
>> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent =
of
>> these fields at lower layers),
>>
>>
>> At 19:50 05/12/2013, Jim Gettys wrote:
>>
>> "Certainly, it may be the same instance of an AQ
>> M algorithm, rather than different instances, for example."
>>
>>
>> That's true of course, but the case with one AQM handling all packets
>> within a queue is the norm. I want to check you're happy with the conver=
se:
>> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED with
>> in and out of contract). So we can have WPIE, WCoDel etc where the
>> differentiation between aggregates is provided by different AQM instance=
s in
>> the same queue, not by different queues with different scheduling
>> priorities.
>> 2) Extending this so that AQM differentiation can be between ECN-capable
>> and Not-ECN-capable aggregates, not just between Diffserv classes (an
>> example being CoDel with a lower 'interval' for ECN-capable packets).
>>
>> I presented the evaluations of this last idea in tsvwg on the final Frid=
ay
>> of the Vancouver IETF - I don't think you were there. <
>> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>>
>>
>> Yes, unfortunately I had to leave before the Friday session.
>> This is my primary motivation for this wordsmithing - I'm trying allow u=
s
>> to move towards zero signalling delays in CoDel, PIE and RED (currently
>> defaults of 200ms, 100ms and 512packets respectively, which are not good=
 for
>> dynamics).
>>
>>
>> Certainly signalling delays are very important: this is why I'm favorabl=
y
>> inclined to "head mark/drop", as it signals TCP as quickly as possible,
>> keeping the response of the TCP feedback loop as tight as possible (and =
part
>> of why I like CoDel so much for the highly variable bandwidth problem we
>> face at the edge of the net).
>>
>> It's *really* important than when the bandwidth drops suddenly that
>> everyone gets told to slow down quickly (exactly how quickly probably
>> depends on the propagation change characteristics of the medium), or pac=
kets
>> can pile up in a big way.
>>
>> How quickly the mark/drop algorithm can figure out that signalling is
>> appropriate is the *other* piece of getting good dynamics.  Here I don't
>> doubt that something may be discovered that is better than CoDel in the
>> slightest.
>> It takes a CoDel instance (within an fq structure) 200ms from its queue
>> first passing 'threshold' before it will ever drop the first packet (unl=
ess
>> the queue hits taildrop before that). So if the RTT is 20ms, that's 220m=
s
>> signalling delay.
>
>
> No, again, see Dave's mail, and you are missing the flow scheduling aspec=
t
> of this and thinking in terms of a single queue and the usual mark/drop
> cases; this is the exact cognitive problem I'm talking about.
>
> The flow scheduling aspect of fq_codel is *more* important than what
> mark/drop algorithm decides to signal the TCP's to regulate their servo
> systems.  I really wish this algorithm had been called "fs_codel", rather
> than "fq_codel", as it is so easy to confuse "fair" with "flow", and "que=
ue"
> with "scheduling".
>
> Regulating TCP is absolutely essential to keep TCP "sane" and responsive,
> but it isn't the most important part of what is going on.
>
> Here's the case of a new TCP flow on a previously idle link: After the TC=
P
> open handshake, you will have no more than 4 or (if IW10 is active) packe=
ts
> for that flow in the queue (actually, 3 packets, as presumably at least o=
ne
> packet is in process of transmission; @ 1Mbps, that first full size packe=
t
> takes 13ms).
>
> If another flow starts, it will get preferentially scheduled to the exist=
ing
> flow(s) until it has also built a queue, at which time it competes "fairl=
y"
> with the other flows in the packet scheduling.
> This means, in the simple low bandwidth case, that as soon as you start t=
he
> second flow, it gets the next possible opportunity to be transmitted in
> preference to any flow that has built a queue.
>
> Here's the kicker:
>
> We've just avoid 3 packets worth of "head of line blocking" latency for t=
he
> second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved just i=
n
> the TCP open, and more in that the first packet from the second flow then
> gets scheduled immediately, getting that TCP moving.
>
> Similarly for your voip packet; it saves those 40ms. And your bulk flow g=
ets
> its first packet through ASAP as well; for web traffic, that usually
> contains the size information or other metadata required for the web brow=
ser
> to unblock.  And that is for a *single* competing TCP connection.
>
> Now, let's look at today's web: there are many embedded objects in a page=
.
> Once the base page is downloaded, the web browser (presuming its DNS look=
ups
> are already cached), opens a pile of connections all at once.  Whether I
> like it or not, (which I don't, as I tried to prevent it with pipelining =
in
> HTTP/1.1 in the 1990's), you can easily have 5-30 TCP connections start
> almost simultaneously. See the connections/page plot on my web page
> explaining all this stuff in more detail at:
> http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queuing=
-traditional-aqm-is-not-enough/
>
> Without any TCP flow control ever happening, I can easily get 40 (or even
> several hundred packets!) arriving at near line rate (the initial window =
*
> number of embedded objects).  Most of these TCP connections *won't ever e=
ven
> get out of slow start*; the objects are typically pretty small.  I've
> observed transient latency from HOL blocking of > 100ms on 50Mbps cable
> service, on some web pages.
>
> Nothing we will/can do at the TCP level can help this situation!!! Nothin=
g,
> other than in the long term getting the incentives right so that fewer TC=
P
> connections might be preferable to gaming TCP, as the web already has.
>
> That CoDel may take a while to figure out that it should start marking in
> the idle case is really pretty irrelevant; the packets have already arriv=
ed
> and unless we do better packet scheduling, you are fully stuck.  Dave's
> trying to also explain that a number of people's understanding of how CoD=
el
> works has been wrong.
>
> But as I say, *the details of the mark/drop algorithm doesn't matter*.
> Having 3, or 100 packets queued in a FIFO queue will already mean you are
> screwed for anything low latency at the bandwidths we all care about.
>
> Once a flow drains to zero, it again gets treated as a new flow.
> And what we choose to define a "flow" to be is arbitrary, though the code
> today does the usual 5tuple.
>
>> In fq_codel this creates considerable self-delay for short flows or r-t
>> apps, which kill their own latency before they get any loss signal to te=
ll
>> them to slow down.
>
>
> Not likely in the common case.  See Dave's comments and also note that th=
e
> initial burst of web page load runs the queue up high immediately.
>
> And as I keep saying, and will say again, the flow queuing decisions
> avoiding HOL blocking explained are much more important to dealing with t=
he
> latency problems.
>
> So go invent better algorithms that CoDel for drop/mark, that can be appl=
ied
> to the flow queuing parts of the algorithm, and I'm happy.  Running code
> please.
>
>
>
>>
>> Even for elastic flows, with congestion signals delayed by so much, they
>> risk hitting themselves with a huge train of overshoot loss. This would =
be
>> the same for fq_pie, except the number is 100ms + RTT.
>
>
> When fq_pie exists to test, I'll be happy to see.
>
> I keep saying, and I'll say again: while some mark/drop algorithm needs t=
o
> exist, flow scheduling is more important to getting good latency....
>
>
>>
>>
>> Yes, the e2e transport could measure delay growth, but it doesn't know
>> whether the delay is coming from a queue that is isolated from others or
>> not. So it doesn't want to slow down too quickly in response to delay gr=
owth
>> in case it gets screwed by other traffic. Ie. using delay growth as a si=
gnal
>> entails considerable signalling delay due to all the uncertainty.
>>
>> The proposal you missed in tsvwg was to define ECN as an immediate signa=
l
>> from the network, 'interval'=3D0 in CoDel terms, so the host always gets
>> congestion signals as fast as possible, and if it needs bursts of signal=
s
>> smoothed out, it can do that itself.
>
>
> Yes, I get that.  I got that a year or more ago. The idea has potential
> merit. ECN as it is currently defined is not so useful.
>
> And I wish it would get a different name so we could more easily talk abo=
ut
> the two different things now being called ECN.
>
> I can see having ECN marks on a burst of packets may be helpful in having
> the receiver judge things in a highly variable wireless scenario; it may
> have additional information about the medium and know that that transient
> has gone away, and that it may not be a wise idea to slow the connection =
at
> all.
>
>
>>
>> The suggested wording ensures all AQM implementations will allow
>> operators, vendors and users to configure such a mechanism. But I've
>> generalised it from ECN to Diffserv too (because the implementation woul=
d be
>> no different).
>
>
> As noted before, Diffserv is still interesting, even though the packet
> scheduling in fq_codel (or similar algorithms) makes it much less necessa=
ry.
> There are two aspects to this:
>
> 1) higher priority contention to the medium.  If I have a real VOIP packe=
t,
> there are ways I can ask for higher priority access to the medium and red=
uce
> the total number of transmit opportunities my traffic requires (and that =
is
> often the scarcest resource on WiFi or Docsis).
> 2) any hint I can get helps (at the edge) so I can distinguish those pack=
ets
> from the way the web has been gaming the network.
>
> Even so, for Diffserv to be safely trusted and honored even in your home,
> the end user (who is the network operator in this case), will have be abl=
e
> to know that a device or application is using it and control whether or n=
ot
> it's honored. Unless it is under the network operator's control (in this
> case, you the home user) Diffserv can/will get gamed to uselessness by
> application and device manufacturers. Ergo why Dave and I hack on home
> routers again: this level of control is not currently present in these
> devices, and must be for Diffserv to be useful.
>
>>
>>
>>
>> My basic issue is one of terminology: people have talked about "best
>> effort" queues.  In reality, this is a "class" of service, rather than a
>> single queue, and when you get into the mental model of BE being a singl=
e
>> queue, (rather than a set of queues) it can lead one astray quickly and
>> easily.
>>
>>
>> Yeah, I know this. I suspected we were talking past each other.
>>
>> I need you to allow the other case into your mind for this conversation.
>> The wording is specifically about the case where "different subsets of
>> packets ... share the same queue".
>
>
> And the word "queue" in most people's minds implies ordering, and FIFO
> behavior.  This is the terminological issue I'm harping on.  It's also wh=
y I
> think "bufferbloat" is a better term for our situation than "queue bloat"=
,
> which you liked and have harped at me about.  Buffers don't have such an
> implication of ordering.
> So if you talked about a buffer here, rather than a queue, I'd be a lot
> happier. At least in my mind, queues are ordered.
>
>>
>> We can talk about an fq structure for this another time, but it's a real=
ly
>> complicated way of doing it.
>>
>> Given simple looks like it could work, why get complicated already?
>
>
> Because the flow scheduling is such a win.  You can't solve the whole
> problem just with mark/drop algorithms and FIFO queues and get reliably t=
o
> decent latencies.
>
> Now, whether Fred's document can/should go into anything like that that
> detail is far from clear (arguably not).
>
> I just don't want to further the mythology we can get to decent latencies=
 at
> the edge of the network to continue with FIFO queues and an AQM algorithm=
,
> and it's clear that many haven't yet internalized that its flow schedulin=
g
> combined with a self tuning adaptive mark/drop algorithm that we must hav=
e,
> but that the flow scheduling makes the biggest difference.
>
>>
>>
>> It's really easy to fall into the idea of a single software queue mappin=
g
>> to some single hardware supported queue, and that's a cognitive mistake,=
 as
>> aggregating MACs are showing us; transmit ops are often the scarcest
>> resource...
>>
>>
>> It's only a cognitive mistake if one is not aware of all the options. I'=
m
>> fully aware of all the options.
>>
>> To be specific, a queue into a wireless medium should be configured so i=
t
>> holds some 'good queue' in reserve for transmit ops, but the queues on t=
op
>> of this that TCP self-inflicts even briefly are not 'good queues' even i=
f
>> they are isolated from other flows by fq - VJ was wrong to generalise th=
e
>> phrase 'good queue' to all bursts of queue - it is only necessary to hol=
d
>> back from signalling and allow a burst of queue if the only possible sig=
nal
>> is a drop. With ECN, you don't have this dilemma. This is the key to rap=
id
>> dynamics.
>
>
> But you also then have to solve the HOL blocking problem, and do so
> urgently.  Ergo flow scheduling.... To get to where we need to go, you ha=
ve
> to worry about the order in which each packet is scheduled.
>
>>
>>
>> Diffserv marking has the potential to give a "hint" to distinguish how
>> particular flows should be handled (scheduled) in a service class, and a=
s my
>> previous example shows, that hint may be very useful in channel access
>> decisions (e.g. voip on 802.11).
>
>
> ECN doesn't help a bit with the head of line blocking problem; it actuall=
y
> will make it worse with FIFO scheduling. ECN means that you can't even ge=
t
> the packets out of the way.
>
> Which says you'd better be doing more clever scheduling than a FIFO.
>
> If you want ECN to be usable in the way you want it to be at low bandwidt=
hs,
> you better become a fan of flow scheduling...
>>
>>
>> But fq_codel teaches the lesson that packet scheduling combined with
>> keeping TCP sane is a key improvement over handling either problem apart=
...
>> In particular, the first packets of new flows/reappearing flows are vast=
ly
>> more "important" than other packets in terms of the latency cost to user=
s of
>> that service. Each flow has in essence its own queue in this service cla=
ss,
>> and we're using information from that to help schedule the packets in wa=
ys
>> that minimize latency to the user.
>>
>>
>> I know all this. Please can we keep to the conversation about how to avo=
id
>> the 200ms signalling delay that fq_codel inflicts on each flow (and the
>> similar signalling delays that other AQMs inflict).
>
>
> As I said, it's not the big problem we have today at the edge of the
> network, and Dave's mail explains your model of CoDel isn't so correct.
>
> It's only very long lived flows where the signalling even matters, and
> that's not what we get at the edge of today's network; instead, we get a =
mix
> of a few larger (e.g. video player flows) with the DOS attack that is web
> traffic, with some isochronous VOIP and teleconferencing traffic.
>>
>>
>>
>>
>> So in this case, a single algorithm is acting over a bunch of flows in a
>> single class of service, and both scheduling packets among the flows, an=
d
>> signalling TCP flows appropriately when they should "slow down".
>>
>>
>> Yup, I know this.
>>
>>
>>
>> So I think you and I are on close to the same page (but have been burned
>> badly in the past by terminology issues getting in the way).  On HTTP/1.=
1 we
>> wasted probably > 2 years talking past each other because we didn't have
>> clear and concise terminology that we all understood the same way.
>>
>>
>> As I thought, we are talking past each other.
>
>
> Yes, in part because I think few have internalized what the web has done =
to
> edge of the network.  It isn't what I had hoped it would be when I was
> HTTP/1.1 editor.  Why this outcome occurred is too long a discussion for
> this thread.
>
>
>> We need to be able to have a conversation that is not always "Hmm, that'=
s
>> sounds like it might be interesting. Can I tell you about fq_codel now?"
>
>
> Running code of other algorithms very welcome. Fq_codel is running code.
> Pie is running code.
>
> Maybe fq_pie will be running code we can test someday.
>
> Even then, I want a low target latency so individual TCP's are kept
> responsive, and I need an algorithm that can keep up quickly with the
> dynamics of wireless, so most simplistic tests will not be useful (we don=
't
> have such good evaluation tests today).
>
> When it is, and if it is better than fq_codel, I'll be happy.  But the
> mark/drop part of the algorithm *isn't* the most important part of the
> algorithm. The packet scheduling decisions are....
>                                - Jim
>
>>
>>
>>
>>
>> Bob
>>
>>
>>
>> And I don't claim I have the right terminology for all this stuff, eithe=
r
>> (even in this mail).
>>
>> Which is why I was loathe to suggest exact text...
>>                            - Jim
>>
>>
>>
>> At 19:50 05/12/2013, Jim Gettys wrote:
>>
>>
>>
>> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>> Fred, Gorry, all,
>> I promised to suggest text for draft-ietf-aqm-recommendation about
>> allowing the AQM's behaviour to be independent for ECN and non-ECN packe=
ts.
>> In the process, I realised we can't talk about independent AQMs for ECN
>> without also including Diffserv.
>> This gets messy, because I believe a good AQM for BE traffic with and
>> without ECN, should remove much if not all the need for Diffserv. But we
>> can't ignore Diffserv.
>>
>>
>> I agree in principle with what Bob is trying to say here (and is very mu=
ch
>> what I've been saying in my blog entry of last summer).
>>
>> Once you have things under control, the need for Diffserv diminishes
>> dramatically (but does not go away).
>>
>> But as Bob notes, there is still a good use for Diffserv: suitably marke=
d
>> traffic may want to contend for access to the channel differently: your
>> marked VOIP packets may want to change the priority with which you reque=
st
>> channel access, so that you get more timely access to the medium. This
>> conserves transmit opportunities, which is often the scarcest resource i=
n
>> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference bet=
ween
>> your VOIP working well, and not working well, on a busy 802.11 network a=
s
>> well as using the channel as efficiently as possible.
>>
>> Similarly, if you have packets you know are background, it's helpful to
>> know that to ensure that they never contend for access to the medium but
>> will always defer to other traffic, and just scavenge available space in
>> other transmit opportunities where possible.
>>
>> I'm a bit loathe though to tie the behavior to queues, however; in
>> particular, best effort traffic may want to be sent in the same aggregat=
e as
>> higher (or lower) priority traffic, if there is remaining space in the
>> aggregate.
>>
>> In short, the mental model we've had that there is a one-to-one model of
>> hardware and software queues (not to mention flows in a given software
>> queue) is often incorrect (or at least seriously sub-optimal) in today's
>> systems (even if the hardware queues "work" properly, which it appears t=
hey
>> do not in 802.11).
>>
>> So I'm not sure Bob's new section 3 here is how to best to state this (o=
r
>> to deal with the terminology problem).  Certainly, it may be the same
>> instance of an AQM algorithm, rather than different instances, for examp=
le.
>> And "
>> It SHOULD be possible" is more a pious wish than anything else.  But I
>> agree in spirit with what Bob's trying to say.
>>                                - Jim
>>
>>
>> ________________________________________________________________________=
_________________
>> {In Section 4: add another bullet between recommendations 2 & 3:}
>> 3{New}. It SHOULD be possible to make different instances of an AQM
>> algorithm apply to different subsets of packets that share the same queu=
e.
>> It SHOULD be possible to classify packets into these subsets at least by=
 ECN
>> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent =
of
>> these fields at lower layers).
>> {Then a new section to expand on this before the current Section 4.3.}
>> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>> The recommendation to provide a separate instance of the AQM for ECN
>> packets goes beyond the assumptions of RFC 3168, which assumed that only=
 one
>> instance of an AQM will handle both ECN-capable and non-ECN-capable pack=
ets.
>>
>>
>> Bob
>>
>>
>> ________________________________________________________________ Bob
>> Briscoe,                                                  BT
>> _______________________________________________ aqm mailing list
>> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
>>
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>>
>>
>> ________________________________________________________________
>> Bob Briscoe,                                                  BT
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>>
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>>
>> ________________________________________________________________
>> Bob Briscoe,                                                  BT
>>
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>>
> I can imagine many other possible algorithms that CoDel for the mark/drop
> algorithm; we happen to like CoDel atm for two reasons: 1) it is self
> adapting to the line rate, and 2) head mark/drop signals the TCP's as soo=
n
> as a decision is made rather than possibly being applied much later (such=
 as
> random or tail drop).  We welcome and encourage further essays in the art=
.
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From bob.briscoe@bt.com  Thu Dec 12 16:05:40 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 431F71AE56A for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 16:05:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gl4y-OzTTVol for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 16:05:34 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id 34B2D1AE571 for <aqm@ietf.org>; Thu, 12 Dec 2013 16:05:32 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR65-UKRD.bt.com (10.187.101.20) with Microsoft SMTP Server (TLS) id 14.3.158.1; Fri, 13 Dec 2013 00:05:25 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Fri, 13 Dec 2013 00:05:24 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Fri, 13 Dec 2013 00:05:24 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.123.223])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rBD05LFx005567;	Fri, 13 Dec 2013 00:05:21 GMT
Message-ID: <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Fri, 13 Dec 2013 00:05:18 +0000
To: Dave Taht <dave.taht@gmail.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.g mail.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com> <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Dec 2013 00:05:40 -0000

Dave,

At 22:11 12/12/2013, Dave Taht wrote:
>but quickly...
>
>Bob, I object to your characterization of users links being busy 1-3%
>of the time. That's an average.

I said it was an average. You're repeating and=20
agreeing with what I said, but saying you object to me saying it?

>When they are busy, they are very busy
>for short periods, typically 2-16 seconds in the case of web traffic,
>then idle for minutes. DASH traffic is busy for 2+ seconds every 10 on
>a 20mbit link, and so on, for 1.5 hours or so. Etc.

Yes, again, you're agreeing with me.

The mean for a Web session is towards the low end=20
of the 2-16 seconds range even now. And as we get=20
the other latency-saving advances out there (e.g.=20
removing TCP & TLS handshakes, proper pipelining,=20
and a faster replacement for slow-start without=20
overshoot), then there is potential for Web=20
sessions to drop to 1-2 seconds or less, because=20
they're usually a long way from being bandwidth limited.

You said fq_codel decays out its memory in 800ms.=20
So it will typically have lost all its memory=20
when each Web transfer starts and when each DASH transfer re-starts.

So I think you're agreeing that this 200ms=20
signalling delay will be predominant?

>In both cases
>congestion exists, and in both cases AQM reaction time measured in
>200ms or so is still vastly superior to what happens today, and packet
>scheduling masks it to a large extent.

You're saying it's OK to propose a solution that=20
delays signalling congestion for about 10 typical=20
CDN RTTs,... because it's better than nothing?

That's rich. I have to say this... You're one of=20
the group of protagonists who has persuaded the=20
world to embark on a programme of=20
/implementation/ updates that will take years,=20
and rubbished deploying the AQM that was already=20
implemented (RED), even tho it was already much better than nothing too.

Yes, auto-config for line rate is a nice bell to=20
add to the bicycle. If we are going to embark on=20
new implementations, auto-config for RTT is no less important.

Lack of auto-config for either requires the fixed=20
config to be at the slow end of the range. And=20
both have a similar range of variability (the RTT=20
range is actually wider). So lack of auto-config=20
in either case leads to a similar order of=20
unnecessary delay. Particularly given traffic is=20
predominantly in short sparse bursts.



Bob



>Jim, yes, I was trying to establish the groundwork for ensuring
>everyone really understood codel-by-itself before talking about
>fq_codel. I'm still not sure that's been established. Anyone here care
>to calculate the number of drops on two flows going through codel and
>fq_codel, starting with iw10, over a 10mbit 2ms RTT link? And when,
>approximately, the queue becomes ideal? And how often the fq_codel
>"fast queue" gets used in this case? Gold stars for everyone that gets
>it right.
>
>Jim, I'd like you to use larger download speeds than a mbit for your
>examples. somewhere between 8 and 20mbit seems appropriate. (IMHO iw10
>should not be used on the modern internet on sub 10Mbit links.) The
>dynamics change significantly as you get more bandwidth than iw10
>abuses.
>
>Anyway=85
>
>After I got as far as describing fq_codel accurately in this thread,
>then I'd hoped to be able to tackle the immediate ECN issue, the value
>of randomness in pie, and the effectiveness and need for ECN on the
>edge as it is currently defined.
>
>and I figured that then I might have written enough to get closer to an=
 rfc.
>
>and when I started kibitzing on this thread I thought I was talking to
>the DCTCP case which I've spent a few months studying up on, and
>looking over alternative ideas there, like
>
>http://conferences.sigcomm.org/co-next/2013/program/p49.pdf
>http://conferences.sigcomm.org/co-next/2013/program/p151.pdf
>
>Scalable, Optimal Flow Routing in Datacenters via Local Link Balancing
>
>http://www.irt-systemx.fr/wp-content/uploads/2013/12/AINTEC.ppt
>
>Sigh. I'll
>
>On Thu, Dec 12, 2013 at 11:35 AM, Jim Gettys <jg@freedesktop.org> wrote:
> >
> >
> >
> > On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> >>
> >> Jim,
> >>
> >>
> >> At 16:55 11/12/2013, Jim Gettys wrote:
> >>
> >>
> >>
> >> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com>=
 wrote:
> >> Jim,
> >>
> >> I'm just checking we're not talking past each other. I'll repeat two
> >> quotes from each of us, then comment.
> >>
> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> >>
> >> 3{New}. It SHOULD be possible to make different instances of an AQM
> >> algorithm apply to different subsets of packets that share the same=
 queue.
> >> It SHOULD be possible to classify packets=20
> into these subsets at least by ECN
> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent=
 of
> >> these fields at lower layers),
> >>
> >>
> >> At 19:50 05/12/2013, Jim Gettys wrote:
> >>
> >> "Certainly, it may be the same instance of an AQ
> >> M algorithm, rather than different instances, for example."
> >>
> >>
> >> That's true of course, but the case with one AQM handling all packets
> >> within a queue is the norm. I want to check=20
> you're happy with the converse:
> >> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED=
 with
> >> in and out of contract). So we can have WPIE, WCoDel etc where the
> >> differentiation between aggregates is=20
> provided by different AQM instances in
> >> the same queue, not by different queues with different scheduling
> >> priorities.
> >> 2) Extending this so that AQM differentiation can be between=
 ECN-capable
> >> and Not-ECN-capable aggregates, not just between Diffserv classes (an
> >> example being CoDel with a lower 'interval' for ECN-capable packets).
> >>
> >> I presented the evaluations of this last idea in tsvwg on the final=
 Friday
> >> of the Vancouver IETF - I don't think you were there. <
> >> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
> >>
> >>
> >> Yes, unfortunately I had to leave before the Friday session.
> >> This is my primary motivation for this wordsmithing - I'm trying allow=
 us
> >> to move towards zero signalling delays in CoDel, PIE and RED (currently
> >> defaults of 200ms, 100ms and 512packets=20
> respectively, which are not good for
> >> dynamics).
> >>
> >>
> >> Certainly signalling delays are very important: this is why I'm=
 favorably
> >> inclined to "head mark/drop", as it signals TCP as quickly as possible,
> >> keeping the response of the TCP feedback=20
> loop as tight as possible (and part
> >> of why I like CoDel so much for the highly variable bandwidth problem=
 we
> >> face at the edge of the net).
> >>
> >> It's *really* important than when the bandwidth drops suddenly that
> >> everyone gets told to slow down quickly (exactly how quickly probably
> >> depends on the propagation change=20
> characteristics of the medium), or packets
> >> can pile up in a big way.
> >>
> >> How quickly the mark/drop algorithm can figure out that signalling is
> >> appropriate is the *other* piece of getting good dynamics.  Here I=
 don't
> >> doubt that something may be discovered that is better than CoDel in the
> >> slightest.
> >> It takes a CoDel instance (within an fq structure) 200ms from its queue
> >> first passing 'threshold' before it will=20
> ever drop the first packet (unless
> >> the queue hits taildrop before that). So if the RTT is 20ms, that's=
 220ms
> >> signalling delay.
> >
> >
> > No, again, see Dave's mail, and you are missing the flow scheduling=
 aspect
> > of this and thinking in terms of a single queue and the usual mark/drop
> > cases; this is the exact cognitive problem I'm talking about.
> >
> > The flow scheduling aspect of fq_codel is *more* important than what
> > mark/drop algorithm decides to signal the TCP's to regulate their servo
> > systems.  I really wish this algorithm had been called "fs_codel",=
 rather
> > than "fq_codel", as it is so easy to confuse=20
> "fair" with "flow", and "queue"
> > with "scheduling".
> >
> > Regulating TCP is absolutely essential to keep TCP "sane" and=
 responsive,
> > but it isn't the most important part of what is going on.
> >
> > Here's the case of a new TCP flow on a previously idle link: After the=
 TCP
> > open handshake, you will have no more than 4 or (if IW10 is active)=
 packets
> > for that flow in the queue (actually, 3 packets, as presumably at least=
 one
> > packet is in process of transmission; @ 1Mbps, that first full size=
 packet
> > takes 13ms).
> >
> > If another flow starts, it will get=20
> preferentially scheduled to the existing
> > flow(s) until it has also built a queue, at which time it competes=
 "fairly"
> > with the other flows in the packet scheduling.
> > This means, in the simple low bandwidth case, that as soon as you start=
 the
> > second flow, it gets the next possible opportunity to be transmitted in
> > preference to any flow that has built a queue.
> >
> > Here's the kicker:
> >
> > We've just avoid 3 packets worth of "head of line blocking" latency for=
 the
> > second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved just=
 in
> > the TCP open, and more in that the first packet from the second flow=
 then
> > gets scheduled immediately, getting that TCP moving.
> >
> > Similarly for your voip packet; it saves=20
> those 40ms. And your bulk flow gets
> > its first packet through ASAP as well; for web traffic, that usually
> > contains the size information or other=20
> metadata required for the web browser
> > to unblock.  And that is for a *single* competing TCP connection.
> >
> > Now, let's look at today's web: there are many embedded objects in a=
 page.
> > Once the base page is downloaded, the web=20
> browser (presuming its DNS lookups
> > are already cached), opens a pile of connections all at once.  Whether I
> > like it or not, (which I don't, as I tried to prevent it with pipelining=
 in
> > HTTP/1.1 in the 1990's), you can easily have 5-30 TCP connections start
> > almost simultaneously. See the connections/page plot on my web page
> > explaining all this stuff in more detail at:
> >=20
>=
 http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queuing-t=
raditional-aqm-is-not-enough/
> >
> > Without any TCP flow control ever happening, I can easily get 40 (or=
 even
> > several hundred packets!) arriving at near line rate (the initial window=
 *
> > number of embedded objects).  Most of these=20
> TCP connections *won't ever even
> > get out of slow start*; the objects are typically pretty small.  I've
> > observed transient latency from HOL blocking of > 100ms on 50Mbps cable
> > service, on some web pages.
> >
> > Nothing we will/can do at the TCP level can help this situation!!!=
 Nothing,
> > other than in the long term getting the incentives right so that fewer=
 TCP
> > connections might be preferable to gaming TCP, as the web already has.
> >
> > That CoDel may take a while to figure out that it should start marking=
 in
> > the idle case is really pretty irrelevant; the packets have already=
 arrived
> > and unless we do better packet scheduling, you are fully stuck.  Dave's
> > trying to also explain that a number of people's understanding of how=
 CoDel
> > works has been wrong.
> >
> > But as I say, *the details of the mark/drop algorithm doesn't matter*.
> > Having 3, or 100 packets queued in a FIFO queue will already mean you=
 are
> > screwed for anything low latency at the bandwidths we all care about.
> >
> > Once a flow drains to zero, it again gets treated as a new flow.
> > And what we choose to define a "flow" to be is arbitrary, though the=
 code
> > today does the usual 5tuple.
> >
> >> In fq_codel this creates considerable self-delay for short flows or r-t
> >> apps, which kill their own latency before they get any loss signal to=
 tell
> >> them to slow down.
> >
> >
> > Not likely in the common case.  See Dave's comments and also note that=
 the
> > initial burst of web page load runs the queue up high immediately.
> >
> > And as I keep saying, and will say again, the flow queuing decisions
> > avoiding HOL blocking explained are much more important to dealing with=
 the
> > latency problems.
> >
> > So go invent better algorithms that CoDel for=20
> drop/mark, that can be applied
> > to the flow queuing parts of the algorithm, and I'm happy.  Running code
> > please.
> >
> >
> >
> >>
> >> Even for elastic flows, with congestion signals delayed by so much,=
 they
> >> risk hitting themselves with a huge train of overshoot loss. This would=
 be
> >> the same for fq_pie, except the number is 100ms + RTT.
> >
> >
> > When fq_pie exists to test, I'll be happy to see.
> >
> > I keep saying, and I'll say again: while some mark/drop algorithm needs=
 to
> > exist, flow scheduling is more important to getting good latency....
> >
> >
> >>
> >>
> >> Yes, the e2e transport could measure delay growth, but it doesn't know
> >> whether the delay is coming from a queue that is isolated from others=
 or
> >> not. So it doesn't want to slow down too=20
> quickly in response to delay growth
> >> in case it gets screwed by other traffic.=20
> Ie. using delay growth as a signal
> >> entails considerable signalling delay due to all the uncertainty.
> >>
> >> The proposal you missed in tsvwg was to define ECN as an immediate=
 signal
> >> from the network, 'interval'=3D0 in CoDel terms, so the host always=
 gets
> >> congestion signals as fast as possible, and if it needs bursts of=
 signals
> >> smoothed out, it can do that itself.
> >
> >
> > Yes, I get that.  I got that a year or more ago. The idea has potential
> > merit. ECN as it is currently defined is not so useful.
> >
> > And I wish it would get a different name so we could more easily talk=
 about
> > the two different things now being called ECN.
> >
> > I can see having ECN marks on a burst of packets may be helpful in=
 having
> > the receiver judge things in a highly variable wireless scenario; it may
> > have additional information about the medium and know that that=
 transient
> > has gone away, and that it may not be a wise idea to slow the connection=
 at
> > all.
> >
> >
> >>
> >> The suggested wording ensures all AQM implementations will allow
> >> operators, vendors and users to configure such a mechanism. But I've
> >> generalised it from ECN to Diffserv too=20
> (because the implementation would be
> >> no different).
> >
> >
> > As noted before, Diffserv is still interesting, even though the packet
> > scheduling in fq_codel (or similar=20
> algorithms) makes it much less necessary.
> > There are two aspects to this:
> >
> > 1) higher priority contention to the medium.  If I have a real VOIP=
 packet,
> > there are ways I can ask for higher priority=20
> access to the medium and reduce
> > the total number of transmit opportunities my traffic requires (and that=
 is
> > often the scarcest resource on WiFi or Docsis).
> > 2) any hint I can get helps (at the edge) so=20
> I can distinguish those packets
> > from the way the web has been gaming the network.
> >
> > Even so, for Diffserv to be safely trusted and honored even in your=
 home,
> > the end user (who is the network operator in this case), will have be=
 able
> > to know that a device or application is using it and control whether or=
 not
> > it's honored. Unless it is under the network operator's control (in this
> > case, you the home user) Diffserv can/will get gamed to uselessness by
> > application and device manufacturers. Ergo why Dave and I hack on home
> > routers again: this level of control is not currently present in these
> > devices, and must be for Diffserv to be useful.
> >
> >>
> >>
> >>
> >> My basic issue is one of terminology: people have talked about "best
> >> effort" queues.  In reality, this is a "class" of service, rather than=
 a
> >> single queue, and when you get into the mental model of BE being a=
 single
> >> queue, (rather than a set of queues) it can lead one astray quickly and
> >> easily.
> >>
> >>
> >> Yeah, I know this. I suspected we were talking past each other.
> >>
> >> I need you to allow the other case into your mind for this=
 conversation.
> >> The wording is specifically about the case where "different subsets of
> >> packets ... share the same queue".
> >
> >
> > And the word "queue" in most people's minds implies ordering, and FIFO
> > behavior.  This is the terminological issue=20
> I'm harping on.  It's also why I
> > think "bufferbloat" is a better term for our situation than "queue=
 bloat",
> > which you liked and have harped at me about.  Buffers don't have such an
> > implication of ordering.
> > So if you talked about a buffer here, rather than a queue, I'd be a lot
> > happier. At least in my mind, queues are ordered.
> >
> >>
> >> We can talk about an fq structure for this another time, but it's a=
 really
> >> complicated way of doing it.
> >>
> >> Given simple looks like it could work, why get complicated already?
> >
> >
> > Because the flow scheduling is such a win.  You can't solve the whole
> > problem just with mark/drop algorithms and FIFO queues and get reliably=
 to
> > decent latencies.
> >
> > Now, whether Fred's document can/should go into anything like that that
> > detail is far from clear (arguably not).
> >
> > I just don't want to further the mythology we=20
> can get to decent latencies at
> > the edge of the network to continue with FIFO queues and an AQM=
 algorithm,
> > and it's clear that many haven't yet internalized that its flow=
 scheduling
> > combined with a self tuning adaptive mark/drop algorithm that we must=
 have,
> > but that the flow scheduling makes the biggest difference.
> >
> >>
> >>
> >> It's really easy to fall into the idea of a single software queue=
 mapping
> >> to some single hardware supported queue, and=20
> that's a cognitive mistake, as
> >> aggregating MACs are showing us; transmit ops are often the scarcest
> >> resource...
> >>
> >>
> >> It's only a cognitive mistake if one is not aware of all the options.=
 I'm
> >> fully aware of all the options.
> >>
> >> To be specific, a queue into a wireless medium should be configured so=
 it
> >> holds some 'good queue' in reserve for transmit ops, but the queues on=
 top
> >> of this that TCP self-inflicts even briefly are not 'good queues' even=
 if
> >> they are isolated from other flows by fq - VJ was wrong to generalise=
 the
> >> phrase 'good queue' to all bursts of queue - it is only necessary to=
 hold
> >> back from signalling and allow a burst of=20
> queue if the only possible signal
> >> is a drop. With ECN, you don't have this dilemma. This is the key to=
 rapid
> >> dynamics.
> >
> >
> > But you also then have to solve the HOL blocking problem, and do so
> > urgently.  Ergo flow scheduling.... To get to where we need to go, you=
 have
> > to worry about the order in which each packet is scheduled.
> >
> >>
> >>
> >> Diffserv marking has the potential to give a "hint" to distinguish how
> >> particular flows should be handled=20
> (scheduled) in a service class, and as my
> >> previous example shows, that hint may be very useful in channel access
> >> decisions (e.g. voip on 802.11).
> >
> >
> > ECN doesn't help a bit with the head of line blocking problem; it=
 actually
> > will make it worse with FIFO scheduling. ECN means that you can't even=
 get
> > the packets out of the way.
> >
> > Which says you'd better be doing more clever scheduling than a FIFO.
> >
> > If you want ECN to be usable in the way you=20
> want it to be at low bandwidths,
> > you better become a fan of flow scheduling...
> >>
> >>
> >> But fq_codel teaches the lesson that packet scheduling combined with
> >> keeping TCP sane is a key improvement over=20
> handling either problem apart...
> >> In particular, the first packets of new flows/reappearing flows are=
 vastly
> >> more "important" than other packets in terms=20
> of the latency cost to users of
> >> that service. Each flow has in essence its=20
> own queue in this service class,
> >> and we're using information from that to help schedule the packets in=
 ways
> >> that minimize latency to the user.
> >>
> >>
> >> I know all this. Please can we keep to the conversation about how to=
 avoid
> >> the 200ms signalling delay that fq_codel inflicts on each flow (and the
> >> similar signalling delays that other AQMs inflict).
> >
> >
> > As I said, it's not the big problem we have today at the edge of the
> > network, and Dave's mail explains your model of CoDel isn't so correct.
> >
> > It's only very long lived flows where the signalling even matters, and
> > that's not what we get at the edge of today's=20
> network; instead, we get a mix
> > of a few larger (e.g. video player flows) with the DOS attack that is=
 web
> > traffic, with some isochronous VOIP and teleconferencing traffic.
> >>
> >>
> >>
> >>
> >> So in this case, a single algorithm is acting over a bunch of flows in=
 a
> >> single class of service, and both scheduling packets among the flows,=
 and
> >> signalling TCP flows appropriately when they should "slow down".
> >>
> >>
> >> Yup, I know this.
> >>
> >>
> >>
> >> So I think you and I are on close to the same page (but have been=
 burned
> >> badly in the past by terminology issues=20
> getting in the way).  On HTTP/1.1 we
> >> wasted probably > 2 years talking past each other because we didn't=
 have
> >> clear and concise terminology that we all understood the same way.
> >>
> >>
> >> As I thought, we are talking past each other.
> >
> >
> > Yes, in part because I think few have internalized what the web has done=
 to
> > edge of the network.  It isn't what I had hoped it would be when I was
> > HTTP/1.1 editor.  Why this outcome occurred is too long a discussion for
> > this thread.
> >
> >
> >> We need to be able to have a conversation that is not always "Hmm,=
 that's
> >> sounds like it might be interesting. Can I tell you about fq_codel=
 now?"
> >
> >
> > Running code of other algorithms very welcome. Fq_codel is running code.
> > Pie is running code.
> >
> > Maybe fq_pie will be running code we can test someday.
> >
> > Even then, I want a low target latency so individual TCP's are kept
> > responsive, and I need an algorithm that can keep up quickly with the
> > dynamics of wireless, so most simplistic tests will not be useful (we=
 don't
> > have such good evaluation tests today).
> >
> > When it is, and if it is better than fq_codel, I'll be happy.  But the
> > mark/drop part of the algorithm *isn't* the most important part of the
> > algorithm. The packet scheduling decisions are....
> >                                - Jim
> >
> >>
> >>
> >>
> >>
> >> Bob
> >>
> >>
> >>
> >> And I don't claim I have the right terminology for all this stuff,=
 either
> >> (even in this mail).
> >>
> >> Which is why I was loathe to suggest exact text...
> >>                            - Jim
> >>
> >>
> >>
> >> At 19:50 05/12/2013, Jim Gettys wrote:
> >>
> >>
> >>
> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> >> Fred, Gorry, all,
> >> I promised to suggest text for draft-ietf-aqm-recommendation about
> >> allowing the AQM's behaviour to be=20
> independent for ECN and non-ECN packets.
> >> In the process, I realised we can't talk about independent AQMs for ECN
> >> without also including Diffserv.
> >> This gets messy, because I believe a good AQM for BE traffic with and
> >> without ECN, should remove much if not all the need for Diffserv. But=
 we
> >> can't ignore Diffserv.
> >>
> >>
> >> I agree in principle with what Bob is trying to say here (and is very=
 much
> >> what I've been saying in my blog entry of last summer).
> >>
> >> Once you have things under control, the need for Diffserv diminishes
> >> dramatically (but does not go away).
> >>
> >> But as Bob notes, there is still a good use for Diffserv: suitably=
 marked
> >> traffic may want to contend for access to the channel differently: your
> >> marked VOIP packets may want to change the priority with which you=
 request
> >> channel access, so that you get more timely access to the medium. This
> >> conserves transmit opportunities, which is often the scarcest resource=
 in
> >> many systems (e.g. 802.11, DOCSIS, etc.).=20
> This can be the difference between
> >> your VOIP working well, and not working well, on a busy 802.11 network=
 as
> >> well as using the channel as efficiently as possible.
> >>
> >> Similarly, if you have packets you know are background, it's helpful to
> >> know that to ensure that they never contend for access to the medium=
 but
> >> will always defer to other traffic, and just scavenge available space=
 in
> >> other transmit opportunities where possible.
> >>
> >> I'm a bit loathe though to tie the behavior to queues, however; in
> >> particular, best effort traffic may want to=20
> be sent in the same aggregate as
> >> higher (or lower) priority traffic, if there is remaining space in the
> >> aggregate.
> >>
> >> In short, the mental model we've had that there is a one-to-one model=
 of
> >> hardware and software queues (not to mention flows in a given software
> >> queue) is often incorrect (or at least seriously sub-optimal) in=
 today's
> >> systems (even if the hardware queues "work"=20
> properly, which it appears they
> >> do not in 802.11).
> >>
> >> So I'm not sure Bob's new section 3 here is how to best to state this=
 (or
> >> to deal with the terminology problem).  Certainly, it may be the same
> >> instance of an AQM algorithm, rather than=20
> different instances, for example.
> >> And "
> >> It SHOULD be possible" is more a pious wish than anything else.  But I
> >> agree in spirit with what Bob's trying to say.
> >>                                - Jim
> >>
> >>
> >>=20
>=
 ___________________________________________________________________________=
______________
> >> {In Section 4: add another bullet between recommendations 2 & 3:}
> >> 3{New}. It SHOULD be possible to make different instances of an AQM
> >> algorithm apply to different subsets of packets that share the same=
 queue.
> >> It SHOULD be possible to classify packets=20
> into these subsets at least by ECN
> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivalent=
 of
> >> these fields at lower layers).
> >> {Then a new section to expand on this before the current Section 4.3.}
> >> 4.3{New}. Independent AQM Instances for ECN and Diffserv
> >> The recommendation to provide a separate instance of the AQM for ECN
> >> packets goes beyond the assumptions of RFC=20
> 3168, which assumed that only one
> >> instance of an AQM will handle both=20
> ECN-capable and non-ECN-capable packets.
> >>
> >>
> >> Bob
> >>
> >>
> >> ________________________________________________________________ Bob
> >> Briscoe,                                                  BT
> >> _______________________________________________ aqm mailing list
> >> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
> >>
> >>
> >> _______________________________________________
> >> aqm mailing list
> >> aqm@ietf.org
> >> https://www.ietf.org/mailman/listinfo/aqm
> >>
> >>
> >> ________________________________________________________________
> >> Bob Briscoe,                                                  BT
> >>
> >> _______________________________________________
> >> aqm mailing list
> >> aqm@ietf.org
> >> https://www.ietf.org/mailman/listinfo/aqm
> >>
> >>
> >> _______________________________________________
> >> aqm mailing list
> >> aqm@ietf.org
> >> https://www.ietf.org/mailman/listinfo/aqm
> >>
> >> ________________________________________________________________
> >> Bob Briscoe,                                                  BT
> >>
> >>
> >> _______________________________________________
> >> aqm mailing list
> >> aqm@ietf.org
> >> https://www.ietf.org/mailman/listinfo/aqm
> >>
> > I can imagine many other possible algorithms that CoDel for the=
 mark/drop
> > algorithm; we happen to like CoDel atm for two reasons: 1) it is self
> > adapting to the line rate, and 2) head mark/drop signals the TCP's as=
 soon
> > as a decision is made rather than possibly=20
> being applied much later (such as
> > random or tail drop).  We welcome and encourage further essays in the=
 art.
> >
> >
> > _______________________________________________
> > aqm mailing list
> > aqm@ietf.org
> > https://www.ietf.org/mailman/listinfo/aqm
> >
>
>
>
>--
>Dave T=E4ht
>
>Fixing bufferbloat with cerowrt:=20
>http://www.teklibre.com/cerowrt/subscribe.html

________________________________________________________________
Bob Briscoe,                                                  BT=20


From dave.taht@gmail.com  Thu Dec 12 16:33:20 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 54FBD1AE173 for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 16:33:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id riMkxiYLoCyq for <aqm@ietfa.amsl.com>; Thu, 12 Dec 2013 16:33:14 -0800 (PST)
Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 393FF1AE043 for <aqm@ietf.org>; Thu, 12 Dec 2013 16:33:14 -0800 (PST)
Received: by mail-we0-f175.google.com with SMTP id t60so1213732wes.6 for <aqm@ietf.org>; Thu, 12 Dec 2013 16:33:07 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=zMkui3uldrqDAi0K6NW9ho+r1MPl7HemZd3z6sEdoWw=; b=RfCcX0rqqmWPigAc6k/YSQboUPY4UQL5hOzcE0k32+ZoLN8C9gmWwVczsFWdXPxo2d O6baC2TM0pazapQWNiPZJ4x8j5By5s4qiYHVMc5cQxk9zWnGQSnlUehNTp47CSTg7+bV hu9dGOo+pNqy/grPVMX0LKZci/6nq7pwz5f7UJyVHhPg6UV+BpD+a4LkBgnam797sJHm 03q07PfBGCA53BqWbLuVWILRmp/p7NJyP9CJZbMVJVEoTxfUru7pWvDDCwqIXt8sg4OH Rsg6mPEIzUX54kbX0hveWP+x5kkgpncvY2vMdJeMcaZRsX0xRJ9bJDxS1TcRCXjkiQg0 /chQ==
MIME-Version: 1.0
X-Received: by 10.180.160.239 with SMTP id xn15mr107570wib.17.1386894787441; Thu, 12 Dec 2013 16:33:07 -0800 (PST)
Received: by 10.217.123.69 with HTTP; Thu, 12 Dec 2013 16:33:07 -0800 (PST)
In-Reply-To: <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com> <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com> <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk>
Date: Thu, 12 Dec 2013 16:33:07 -0800
Message-ID: <CAA93jw6FB6wUxwTXKHic5PJ+fo35ZfAKRsVa+xAxE=2U+TCNNQ@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Dec 2013 00:33:20 -0000

 bob, I have had 3% of my braincells free for this conversation for
the last week. I get twitchy whenever I see averages on traffic that
has microbursts - I pointed to one of the papers presented earlier
this week in my last email.

There is a great deal of wonderful work going on to fix the microburst
problems in the ongoing paced tcp work. I'm very encouraged, and also
agree that it is possible to optimize away much of slow start possibly
along the same way you are thinking,

I had hoped that by establishing what exists today, and describing it
properly, and getting everyone on the same page on what exists=85

I could finally get to where I could start to make sense on the ecn
and immediate ecn ideas to work towards reducing congestion still
further in the data center, and along the edge.

and these days, I'm mostly working on wifi. If you can make red work
on wifi, I would love to know how. My take on this list was that we
were trying to standardize on some aqm technologies that exist and are
being deployed today. Bus is leaving the station.

On Thu, Dec 12, 2013 at 4:05 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> Dave,
>
>
> At 22:11 12/12/2013, Dave Taht wrote:
>>
>> but quickly...
>>
>> Bob, I object to your characterization of users links being busy 1-3%
>> of the time. That's an average.
>
>
> I said it was an average. You're repeating and agreeing with what I said,
> but saying you object to me saying it?
>
>
>> When they are busy, they are very busy
>> for short periods, typically 2-16 seconds in the case of web traffic,
>> then idle for minutes. DASH traffic is busy for 2+ seconds every 10 on
>> a 20mbit link, and so on, for 1.5 hours or so. Etc.
>
>
> Yes, again, you're agreeing with me.
>
> The mean for a Web session is towards the low end of the 2-16 seconds ran=
ge
> even now. And as we get the other latency-saving advances out there (e.g.
> removing TCP & TLS handshakes, proper pipelining, and a faster replacemen=
t
> for slow-start without overshoot), then there is potential for Web sessio=
ns
> to drop to 1-2 seconds or less, because they're usually a long way from
> being bandwidth limited.
>
> You said fq_codel decays out its memory in 800ms. So it will typically ha=
ve
> lost all its memory when each Web transfer starts and when each DASH
> transfer re-starts.
>
> So I think you're agreeing that this 200ms signalling delay will be
> predominant?
>
>
>> In both cases
>> congestion exists, and in both cases AQM reaction time measured in
>> 200ms or so is still vastly superior to what happens today, and packet
>> scheduling masks it to a large extent.
>
>
> You're saying it's OK to propose a solution that delays signalling
> congestion for about 10 typical CDN RTTs,... because it's better than
> nothing?
>
> That's rich. I have to say this... You're one of the group of protagonist=
s
> who has persuaded the world to embark on a programme of /implementation/
> updates that will take years, and rubbished deploying the AQM that was
> already implemented (RED), even tho it was already much better than nothi=
ng
> too.
>
> Yes, auto-config for line rate is a nice bell to add to the bicycle. If w=
e
> are going to embark on new implementations, auto-config for RTT is no les=
s
> important.
>
> Lack of auto-config for either requires the fixed config to be at the slo=
w
> end of the range. And both have a similar range of variability (the RTT
> range is actually wider). So lack of auto-config in either case leads to =
a
> similar order of unnecessary delay. Particularly given traffic is
> predominantly in short sparse bursts.
>
>
>
> Bob
>
>
>
>
>> Jim, yes, I was trying to establish the groundwork for ensuring
>> everyone really understood codel-by-itself before talking about
>> fq_codel. I'm still not sure that's been established. Anyone here care
>> to calculate the number of drops on two flows going through codel and
>> fq_codel, starting with iw10, over a 10mbit 2ms RTT link? And when,
>> approximately, the queue becomes ideal? And how often the fq_codel
>> "fast queue" gets used in this case? Gold stars for everyone that gets
>> it right.
>>
>> Jim, I'd like you to use larger download speeds than a mbit for your
>> examples. somewhere between 8 and 20mbit seems appropriate. (IMHO iw10
>> should not be used on the modern internet on sub 10Mbit links.) The
>> dynamics change significantly as you get more bandwidth than iw10
>> abuses.
>>
>> Anyway=85
>>
>> After I got as far as describing fq_codel accurately in this thread,
>> then I'd hoped to be able to tackle the immediate ECN issue, the value
>> of randomness in pie, and the effectiveness and need for ECN on the
>> edge as it is currently defined.
>>
>> and I figured that then I might have written enough to get closer to an
>> rfc.
>>
>> and when I started kibitzing on this thread I thought I was talking to
>> the DCTCP case which I've spent a few months studying up on, and
>> looking over alternative ideas there, like
>>
>> http://conferences.sigcomm.org/co-next/2013/program/p49.pdf
>> http://conferences.sigcomm.org/co-next/2013/program/p151.pdf
>>
>> Scalable, Optimal Flow Routing in Datacenters via Local Link Balancing
>>
>> http://www.irt-systemx.fr/wp-content/uploads/2013/12/AINTEC.ppt
>>
>> Sigh. I'll
>>
>> On Thu, Dec 12, 2013 at 11:35 AM, Jim Gettys <jg@freedesktop.org> wrote:
>> >
>> >
>> >
>> > On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com> wrot=
e:
>> >>
>> >> Jim,
>> >>
>> >>
>> >> At 16:55 11/12/2013, Jim Gettys wrote:
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com>
>> >> wrote:
>> >> Jim,
>> >>
>> >> I'm just checking we're not talking past each other. I'll repeat two
>> >> quotes from each of us, then comment.
>> >>
>> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrot=
e:
>> >>
>> >> 3{New}. It SHOULD be possible to make different instances of an AQM
>> >> algorithm apply to different subsets of packets that share the same
>> >> queue.
>> >> It SHOULD be possible to classify packets into these subsets at least
>> >> by ECN
>> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivale=
nt
>> >> of
>> >> these fields at lower layers),
>> >>
>> >>
>> >> At 19:50 05/12/2013, Jim Gettys wrote:
>> >>
>> >> "Certainly, it may be the same instance of an AQ
>> >> M algorithm, rather than different instances, for example."
>> >>
>> >>
>> >> That's true of course, but the case with one AQM handling all packets
>> >> within a queue is the norm. I want to check you're happy with the
>> >> converse:
>> >> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED
>> >> with
>> >> in and out of contract). So we can have WPIE, WCoDel etc where the
>> >> differentiation between aggregates is provided by different AQM
>> >> instances in
>> >> the same queue, not by different queues with different scheduling
>> >> priorities.
>> >> 2) Extending this so that AQM differentiation can be between
>> >> ECN-capable
>> >> and Not-ECN-capable aggregates, not just between Diffserv classes (an
>> >> example being CoDel with a lower 'interval' for ECN-capable packets).
>> >>
>> >> I presented the evaluations of this last idea in tsvwg on the final
>> >> Friday
>> >> of the Vancouver IETF - I don't think you were there. <
>> >> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>> >>
>> >>
>> >> Yes, unfortunately I had to leave before the Friday session.
>> >> This is my primary motivation for this wordsmithing - I'm trying allo=
w
>> >> us
>> >> to move towards zero signalling delays in CoDel, PIE and RED (current=
ly
>> >> defaults of 200ms, 100ms and 512packets respectively, which are not
>> >> good for
>> >> dynamics).
>> >>
>> >>
>> >> Certainly signalling delays are very important: this is why I'm
>> >> favorably
>> >> inclined to "head mark/drop", as it signals TCP as quickly as possibl=
e,
>> >> keeping the response of the TCP feedback loop as tight as possible (a=
nd
>> >> part
>> >> of why I like CoDel so much for the highly variable bandwidth problem
>> >> we
>> >> face at the edge of the net).
>> >>
>> >> It's *really* important than when the bandwidth drops suddenly that
>> >> everyone gets told to slow down quickly (exactly how quickly probably
>> >> depends on the propagation change characteristics of the medium), or
>> >> packets
>> >> can pile up in a big way.
>> >>
>> >> How quickly the mark/drop algorithm can figure out that signalling is
>> >> appropriate is the *other* piece of getting good dynamics.  Here I
>> >> don't
>> >> doubt that something may be discovered that is better than CoDel in t=
he
>> >> slightest.
>> >> It takes a CoDel instance (within an fq structure) 200ms from its que=
ue
>> >> first passing 'threshold' before it will ever drop the first packet
>> >> (unless
>> >> the queue hits taildrop before that). So if the RTT is 20ms, that's
>> >> 220ms
>> >> signalling delay.
>> >
>> >
>> > No, again, see Dave's mail, and you are missing the flow scheduling
>> > aspect
>> > of this and thinking in terms of a single queue and the usual mark/dro=
p
>> > cases; this is the exact cognitive problem I'm talking about.
>> >
>> > The flow scheduling aspect of fq_codel is *more* important than what
>> > mark/drop algorithm decides to signal the TCP's to regulate their serv=
o
>> > systems.  I really wish this algorithm had been called "fs_codel",
>> > rather
>> > than "fq_codel", as it is so easy to confuse "fair" with "flow", and
>> > "queue"
>> > with "scheduling".
>> >
>> > Regulating TCP is absolutely essential to keep TCP "sane" and
>> > responsive,
>> > but it isn't the most important part of what is going on.
>> >
>> > Here's the case of a new TCP flow on a previously idle link: After the
>> > TCP
>> > open handshake, you will have no more than 4 or (if IW10 is active)
>> > packets
>> > for that flow in the queue (actually, 3 packets, as presumably at leas=
t
>> > one
>> > packet is in process of transmission; @ 1Mbps, that first full size
>> > packet
>> > takes 13ms).
>> >
>> > If another flow starts, it will get preferentially scheduled to the
>> > existing
>> > flow(s) until it has also built a queue, at which time it competes
>> > "fairly"
>> > with the other flows in the packet scheduling.
>> > This means, in the simple low bandwidth case, that as soon as you star=
t
>> > the
>> > second flow, it gets the next possible opportunity to be transmitted i=
n
>> > preference to any flow that has built a queue.
>> >
>> > Here's the kicker:
>> >
>> > We've just avoid 3 packets worth of "head of line blocking" latency fo=
r
>> > the
>> > second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved jus=
t
>> > in
>> > the TCP open, and more in that the first packet from the second flow
>> > then
>> > gets scheduled immediately, getting that TCP moving.
>> >
>> > Similarly for your voip packet; it saves those 40ms. And your bulk flo=
w
>> > gets
>> > its first packet through ASAP as well; for web traffic, that usually
>> > contains the size information or other metadata required for the web
>> > browser
>> > to unblock.  And that is for a *single* competing TCP connection.
>> >
>> > Now, let's look at today's web: there are many embedded objects in a
>> > page.
>> > Once the base page is downloaded, the web browser (presuming its DNS
>> > lookups
>> > are already cached), opens a pile of connections all at once.  Whether=
 I
>> > like it or not, (which I don't, as I tried to prevent it with pipelini=
ng
>> > in
>> > HTTP/1.1 in the 1990's), you can easily have 5-30 TCP connections star=
t
>> > almost simultaneously. See the connections/page plot on my web page
>> > explaining all this stuff in more detail at:
>> >
>> > http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queu=
ing-traditional-aqm-is-not-enough/
>> >
>> > Without any TCP flow control ever happening, I can easily get 40 (or
>> > even
>> > several hundred packets!) arriving at near line rate (the initial wind=
ow
>> > *
>> > number of embedded objects).  Most of these TCP connections *won't eve=
r
>> > even
>> > get out of slow start*; the objects are typically pretty small.  I've
>> > observed transient latency from HOL blocking of > 100ms on 50Mbps cabl=
e
>> > service, on some web pages.
>> >
>> > Nothing we will/can do at the TCP level can help this situation!!!
>> > Nothing,
>> > other than in the long term getting the incentives right so that fewer
>> > TCP
>> > connections might be preferable to gaming TCP, as the web already has.
>> >
>> > That CoDel may take a while to figure out that it should start marking
>> > in
>> > the idle case is really pretty irrelevant; the packets have already
>> > arrived
>> > and unless we do better packet scheduling, you are fully stuck.  Dave'=
s
>> > trying to also explain that a number of people's understanding of how
>> > CoDel
>> > works has been wrong.
>> >
>> > But as I say, *the details of the mark/drop algorithm doesn't matter*.
>> > Having 3, or 100 packets queued in a FIFO queue will already mean you
>> > are
>> > screwed for anything low latency at the bandwidths we all care about.
>> >
>> > Once a flow drains to zero, it again gets treated as a new flow.
>> > And what we choose to define a "flow" to be is arbitrary, though the
>> > code
>> > today does the usual 5tuple.
>> >
>> >> In fq_codel this creates considerable self-delay for short flows or r=
-t
>> >> apps, which kill their own latency before they get any loss signal to
>> >> tell
>> >> them to slow down.
>> >
>> >
>> > Not likely in the common case.  See Dave's comments and also note that
>> > the
>> > initial burst of web page load runs the queue up high immediately.
>> >
>> > And as I keep saying, and will say again, the flow queuing decisions
>> > avoiding HOL blocking explained are much more important to dealing wit=
h
>> > the
>> > latency problems.
>> >
>> > So go invent better algorithms that CoDel for drop/mark, that can be
>> > applied
>> > to the flow queuing parts of the algorithm, and I'm happy.  Running co=
de
>> > please.
>> >
>> >
>> >
>> >>
>> >> Even for elastic flows, with congestion signals delayed by so much,
>> >> they
>> >> risk hitting themselves with a huge train of overshoot loss. This wou=
ld
>> >> be
>> >> the same for fq_pie, except the number is 100ms + RTT.
>> >
>> >
>> > When fq_pie exists to test, I'll be happy to see.
>> >
>> > I keep saying, and I'll say again: while some mark/drop algorithm need=
s
>> > to
>> > exist, flow scheduling is more important to getting good latency....
>> >
>> >
>> >>
>> >>
>> >> Yes, the e2e transport could measure delay growth, but it doesn't kno=
w
>> >> whether the delay is coming from a queue that is isolated from others
>> >> or
>> >> not. So it doesn't want to slow down too quickly in response to delay
>> >> growth
>> >> in case it gets screwed by other traffic. Ie. using delay growth as a
>> >> signal
>> >> entails considerable signalling delay due to all the uncertainty.
>> >>
>> >> The proposal you missed in tsvwg was to define ECN as an immediate
>> >> signal
>> >> from the network, 'interval'=3D0 in CoDel terms, so the host always g=
ets
>> >> congestion signals as fast as possible, and if it needs bursts of
>> >> signals
>> >> smoothed out, it can do that itself.
>> >
>> >
>> > Yes, I get that.  I got that a year or more ago. The idea has potentia=
l
>> > merit. ECN as it is currently defined is not so useful.
>> >
>> > And I wish it would get a different name so we could more easily talk
>> > about
>> > the two different things now being called ECN.
>> >
>> > I can see having ECN marks on a burst of packets may be helpful in
>> > having
>> > the receiver judge things in a highly variable wireless scenario; it m=
ay
>> > have additional information about the medium and know that that
>> > transient
>> > has gone away, and that it may not be a wise idea to slow the connecti=
on
>> > at
>> > all.
>> >
>> >
>> >>
>> >> The suggested wording ensures all AQM implementations will allow
>> >> operators, vendors and users to configure such a mechanism. But I've
>> >> generalised it from ECN to Diffserv too (because the implementation
>> >> would be
>> >> no different).
>> >
>> >
>> > As noted before, Diffserv is still interesting, even though the packet
>> > scheduling in fq_codel (or similar algorithms) makes it much less
>> > necessary.
>> > There are two aspects to this:
>> >
>> > 1) higher priority contention to the medium.  If I have a real VOIP
>> > packet,
>> > there are ways I can ask for higher priority access to the medium and
>> > reduce
>> > the total number of transmit opportunities my traffic requires (and th=
at
>> > is
>> > often the scarcest resource on WiFi or Docsis).
>> > 2) any hint I can get helps (at the edge) so I can distinguish those
>> > packets
>> > from the way the web has been gaming the network.
>> >
>> > Even so, for Diffserv to be safely trusted and honored even in your
>> > home,
>> > the end user (who is the network operator in this case), will have be
>> > able
>> > to know that a device or application is using it and control whether o=
r
>> > not
>> > it's honored. Unless it is under the network operator's control (in th=
is
>> > case, you the home user) Diffserv can/will get gamed to uselessness by
>> > application and device manufacturers. Ergo why Dave and I hack on home
>> > routers again: this level of control is not currently present in these
>> > devices, and must be for Diffserv to be useful.
>> >
>> >>
>> >>
>> >>
>> >> My basic issue is one of terminology: people have talked about "best
>> >> effort" queues.  In reality, this is a "class" of service, rather tha=
n
>> >> a
>> >> single queue, and when you get into the mental model of BE being a
>> >> single
>> >> queue, (rather than a set of queues) it can lead one astray quickly a=
nd
>> >> easily.
>> >>
>> >>
>> >> Yeah, I know this. I suspected we were talking past each other.
>> >>
>> >> I need you to allow the other case into your mind for this
>> >> conversation.
>> >> The wording is specifically about the case where "different subsets o=
f
>> >> packets ... share the same queue".
>> >
>> >
>> > And the word "queue" in most people's minds implies ordering, and FIFO
>> > behavior.  This is the terminological issue I'm harping on.  It's also
>> > why I
>> > think "bufferbloat" is a better term for our situation than "queue
>> > bloat",
>> > which you liked and have harped at me about.  Buffers don't have such =
an
>> > implication of ordering.
>> > So if you talked about a buffer here, rather than a queue, I'd be a lo=
t
>> > happier. At least in my mind, queues are ordered.
>> >
>> >>
>> >> We can talk about an fq structure for this another time, but it's a
>> >> really
>> >> complicated way of doing it.
>> >>
>> >> Given simple looks like it could work, why get complicated already?
>> >
>> >
>> > Because the flow scheduling is such a win.  You can't solve the whole
>> > problem just with mark/drop algorithms and FIFO queues and get reliabl=
y
>> > to
>> > decent latencies.
>> >
>> > Now, whether Fred's document can/should go into anything like that tha=
t
>> > detail is far from clear (arguably not).
>> >
>> > I just don't want to further the mythology we can get to decent
>> > latencies at
>> > the edge of the network to continue with FIFO queues and an AQM
>> > algorithm,
>> > and it's clear that many haven't yet internalized that its flow
>> > scheduling
>> > combined with a self tuning adaptive mark/drop algorithm that we must
>> > have,
>> > but that the flow scheduling makes the biggest difference.
>> >
>> >>
>> >>
>> >> It's really easy to fall into the idea of a single software queue
>> >> mapping
>> >> to some single hardware supported queue, and that's a cognitive
>> >> mistake, as
>> >> aggregating MACs are showing us; transmit ops are often the scarcest
>> >> resource...
>> >>
>> >>
>> >> It's only a cognitive mistake if one is not aware of all the options.
>> >> I'm
>> >> fully aware of all the options.
>> >>
>> >> To be specific, a queue into a wireless medium should be configured s=
o
>> >> it
>> >> holds some 'good queue' in reserve for transmit ops, but the queues o=
n
>> >> top
>> >> of this that TCP self-inflicts even briefly are not 'good queues' eve=
n
>> >> if
>> >> they are isolated from other flows by fq - VJ was wrong to generalise
>> >> the
>> >> phrase 'good queue' to all bursts of queue - it is only necessary to
>> >> hold
>> >> back from signalling and allow a burst of queue if the only possible
>> >> signal
>> >> is a drop. With ECN, you don't have this dilemma. This is the key to
>> >> rapid
>> >> dynamics.
>> >
>> >
>> > But you also then have to solve the HOL blocking problem, and do so
>> > urgently.  Ergo flow scheduling.... To get to where we need to go, you
>> > have
>> > to worry about the order in which each packet is scheduled.
>> >
>> >>
>> >>
>> >> Diffserv marking has the potential to give a "hint" to distinguish ho=
w
>> >> particular flows should be handled (scheduled) in a service class, an=
d
>> >> as my
>> >> previous example shows, that hint may be very useful in channel acces=
s
>> >> decisions (e.g. voip on 802.11).
>> >
>> >
>> > ECN doesn't help a bit with the head of line blocking problem; it
>> > actually
>> > will make it worse with FIFO scheduling. ECN means that you can't even
>> > get
>> > the packets out of the way.
>> >
>> > Which says you'd better be doing more clever scheduling than a FIFO.
>> >
>> > If you want ECN to be usable in the way you want it to be at low
>> > bandwidths,
>> > you better become a fan of flow scheduling...
>> >>
>> >>
>> >> But fq_codel teaches the lesson that packet scheduling combined with
>> >> keeping TCP sane is a key improvement over handling either problem
>> >> apart...
>> >> In particular, the first packets of new flows/reappearing flows are
>> >> vastly
>> >> more "important" than other packets in terms of the latency cost to
>> >> users of
>> >> that service. Each flow has in essence its own queue in this service
>> >> class,
>> >> and we're using information from that to help schedule the packets in
>> >> ways
>> >> that minimize latency to the user.
>> >>
>> >>
>> >> I know all this. Please can we keep to the conversation about how to
>> >> avoid
>> >> the 200ms signalling delay that fq_codel inflicts on each flow (and t=
he
>> >> similar signalling delays that other AQMs inflict).
>> >
>> >
>> > As I said, it's not the big problem we have today at the edge of the
>> > network, and Dave's mail explains your model of CoDel isn't so correct=
.
>> >
>> > It's only very long lived flows where the signalling even matters, and
>> > that's not what we get at the edge of today's network; instead, we get=
 a
>> > mix
>> > of a few larger (e.g. video player flows) with the DOS attack that is
>> > web
>> > traffic, with some isochronous VOIP and teleconferencing traffic.
>> >>
>> >>
>> >>
>> >>
>> >> So in this case, a single algorithm is acting over a bunch of flows i=
n
>> >> a
>> >> single class of service, and both scheduling packets among the flows,
>> >> and
>> >> signalling TCP flows appropriately when they should "slow down".
>> >>
>> >>
>> >> Yup, I know this.
>> >>
>> >>
>> >>
>> >> So I think you and I are on close to the same page (but have been
>> >> burned
>> >> badly in the past by terminology issues getting in the way).  On
>> >> HTTP/1.1 we
>> >> wasted probably > 2 years talking past each other because we didn't
>> >> have
>> >> clear and concise terminology that we all understood the same way.
>> >>
>> >>
>> >> As I thought, we are talking past each other.
>> >
>> >
>> > Yes, in part because I think few have internalized what the web has do=
ne
>> > to
>> > edge of the network.  It isn't what I had hoped it would be when I was
>> > HTTP/1.1 editor.  Why this outcome occurred is too long a discussion f=
or
>> > this thread.
>> >
>> >
>> >> We need to be able to have a conversation that is not always "Hmm,
>> >> that's
>> >> sounds like it might be interesting. Can I tell you about fq_codel
>> >> now?"
>> >
>> >
>> > Running code of other algorithms very welcome. Fq_codel is running cod=
e.
>> > Pie is running code.
>> >
>> > Maybe fq_pie will be running code we can test someday.
>> >
>> > Even then, I want a low target latency so individual TCP's are kept
>> > responsive, and I need an algorithm that can keep up quickly with the
>> > dynamics of wireless, so most simplistic tests will not be useful (we
>> > don't
>> > have such good evaluation tests today).
>> >
>> > When it is, and if it is better than fq_codel, I'll be happy.  But the
>> > mark/drop part of the algorithm *isn't* the most important part of the
>> > algorithm. The packet scheduling decisions are....
>> >                                - Jim
>> >
>> >>
>> >>
>> >>
>> >>
>> >> Bob
>> >>
>> >>
>> >>
>> >> And I don't claim I have the right terminology for all this stuff,
>> >> either
>> >> (even in this mail).
>> >>
>> >> Which is why I was loathe to suggest exact text...
>> >>                            - Jim
>> >>
>> >>
>> >>
>> >> At 19:50 05/12/2013, Jim Gettys wrote:
>> >>
>> >>
>> >>
>> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrot=
e:
>> >> Fred, Gorry, all,
>> >> I promised to suggest text for draft-ietf-aqm-recommendation about
>> >> allowing the AQM's behaviour to be independent for ECN and non-ECN
>> >> packets.
>> >> In the process, I realised we can't talk about independent AQMs for E=
CN
>> >> without also including Diffserv.
>> >> This gets messy, because I believe a good AQM for BE traffic with and
>> >> without ECN, should remove much if not all the need for Diffserv. But
>> >> we
>> >> can't ignore Diffserv.
>> >>
>> >>
>> >> I agree in principle with what Bob is trying to say here (and is very
>> >> much
>> >> what I've been saying in my blog entry of last summer).
>> >>
>> >> Once you have things under control, the need for Diffserv diminishes
>> >> dramatically (but does not go away).
>> >>
>> >> But as Bob notes, there is still a good use for Diffserv: suitably
>> >> marked
>> >> traffic may want to contend for access to the channel differently: yo=
ur
>> >> marked VOIP packets may want to change the priority with which you
>> >> request
>> >> channel access, so that you get more timely access to the medium. Thi=
s
>> >> conserves transmit opportunities, which is often the scarcest resourc=
e
>> >> in
>> >> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
>> >> between
>> >> your VOIP working well, and not working well, on a busy 802.11 networ=
k
>> >> as
>> >> well as using the channel as efficiently as possible.
>> >>
>> >> Similarly, if you have packets you know are background, it's helpful =
to
>> >> know that to ensure that they never contend for access to the medium
>> >> but
>> >> will always defer to other traffic, and just scavenge available space
>> >> in
>> >> other transmit opportunities where possible.
>> >>
>> >> I'm a bit loathe though to tie the behavior to queues, however; in
>> >> particular, best effort traffic may want to be sent in the same
>> >> aggregate as
>> >> higher (or lower) priority traffic, if there is remaining space in th=
e
>> >> aggregate.
>> >>
>> >> In short, the mental model we've had that there is a one-to-one model
>> >> of
>> >> hardware and software queues (not to mention flows in a given softwar=
e
>> >> queue) is often incorrect (or at least seriously sub-optimal) in
>> >> today's
>> >> systems (even if the hardware queues "work" properly, which it appear=
s
>> >> they
>> >> do not in 802.11).
>> >>
>> >> So I'm not sure Bob's new section 3 here is how to best to state this
>> >> (or
>> >> to deal with the terminology problem).  Certainly, it may be the same
>> >> instance of an AQM algorithm, rather than different instances, for
>> >> example.
>> >> And "
>> >> It SHOULD be possible" is more a pious wish than anything else.  But =
I
>> >> agree in spirit with what Bob's trying to say.
>> >>                                - Jim
>> >>
>> >>
>> >>
>> >> _____________________________________________________________________=
____________________
>> >> {In Section 4: add another bullet between recommendations 2 & 3:}
>> >> 3{New}. It SHOULD be possible to make different instances of an AQM
>> >> algorithm apply to different subsets of packets that share the same
>> >> queue.
>> >> It SHOULD be possible to classify packets into these subsets at least
>> >> by ECN
>> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivale=
nt
>> >> of
>> >> these fields at lower layers).
>> >> {Then a new section to expand on this before the current Section 4.3.=
}
>> >> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>> >> The recommendation to provide a separate instance of the AQM for ECN
>> >> packets goes beyond the assumptions of RFC 3168, which assumed that
>> >> only one
>> >> instance of an AQM will handle both ECN-capable and non-ECN-capable
>> >> packets.
>> >>
>> >>
>> >> Bob
>> >>
>> >>
>> >> ________________________________________________________________ Bob
>> >> Briscoe,                                                  BT
>> >> _______________________________________________ aqm mailing list
>> >> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >>
>> >> ________________________________________________________________
>> >> Bob Briscoe,                                                  BT
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >> ________________________________________________________________
>> >> Bob Briscoe,                                                  BT
>> >>
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> > I can imagine many other possible algorithms that CoDel for the
>> > mark/drop
>> > algorithm; we happen to like CoDel atm for two reasons: 1) it is self
>> > adapting to the line rate, and 2) head mark/drop signals the TCP's as
>> > soon
>> > as a decision is made rather than possibly being applied much later
>> > (such as
>> > random or tail drop).  We welcome and encourage further essays in the
>> > art.
>> >
>> >
>> > _______________________________________________
>> > aqm mailing list
>> > aqm@ietf.org
>> > https://www.ietf.org/mailman/listinfo/aqm
>> >
>>
>>
>>
>> --
>> Dave T=E4ht
>>
>> Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From dave.taht@gmail.com  Fri Dec 13 00:15:43 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A51A1AE6C0 for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 00:15:43 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id riPP_mjgGmRJ for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 00:15:28 -0800 (PST)
Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) by ietfa.amsl.com (Postfix) with ESMTP id 760281AE6AA for <aqm@ietf.org>; Fri, 13 Dec 2013 00:15:25 -0800 (PST)
Received: by mail-we0-f182.google.com with SMTP id q59so1558008wes.27 for <aqm@ietf.org>; Fri, 13 Dec 2013 00:15:18 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=dplroO2uGHHlc1Xo2UFshzi07lkeWmgfq2aOk4B4IXY=; b=tDayIAMCpq6gbxmBLPo9KnkFx/QX9bd/tGNZ7SIKSrhGq0euWtuXbFcWqYdkvYyrMy oyZO1FvcMFngVCdqttW2qxl8crOAo152LKN9ljKhfvQHf0vPs+yerervQ/8cEYhox96i x5xhLWV4R7jH9QeDACGopePRi3oOv3PftTgQTMXwxbZqWugIcYIPT8RKxWMy3rNXATBs a/7vxttxEL4Xj6XONU7QZ6DomDZop3ljgqZBwtsMqpoPAvQ9KtrKWxyEHVYhc5V29CoJ kwvysEz+IjaiSD4BAVp3bti+1QBVEYO1Iv80aBO9Cg5cOHM/+wGEt8D3XnISQN/l0nFU AGmQ==
MIME-Version: 1.0
X-Received: by 10.180.160.239 with SMTP id xn15mr1312505wib.17.1386922518750;  Fri, 13 Dec 2013 00:15:18 -0800 (PST)
Received: by 10.217.123.69 with HTTP; Fri, 13 Dec 2013 00:15:18 -0800 (PST)
In-Reply-To: <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com> <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com> <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk>
Date: Fri, 13 Dec 2013 00:15:18 -0800
Message-ID: <CAA93jw7UrTzUrMAkaQOr1dQoGjWD2ktLAn-Uv-oA6x88HpYLfA@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Dec 2013 08:15:43 -0000
X-List-Received-Date: Fri, 13 Dec 2013 08:15:43 -0000

For starters, the codel signaling delay from the onset of continuous
over 5ms delay on packets defaults to target 100ms, not 200ms. I don't
know who started saying 200ms but even I started believing it with the
few brain cells I've had to spare of late. 5x a CDN rtt in a world of
30-60k images sounds about right.

Secondly, codel drops/marks from head, not from tail, so the signal
gets back to the sender in 1/2  the real physical RTT after that,
rather than the tail of a queue that may be out of control at that
point. Much faster than pie.

There has been so much misinformation spread of late on these threads.
I'm hoping we're beginning to make a dent in it? I look forward to
making all this clear on the upcoming RFCs. I think I should stop now,
revisit the rest of this thread and see what else can be cleared up
before even beginning to tackle fq_codel after I get caught up on
sleep.

as for your other comments...

I have always said deploy RED and for that matter DRR, SFQ or SQF
where you can. I distinctly remember polling the crowd at the first
uknof I went to and being sad to discover only about 4% of the room
had (4 people).

I DO hold that red is too hard to configure for ordinary mortals, and
that it doesn't work at all on variable bandwidth links like cable, or
wireless, which happen to be the dominant form of end-user link
nowadays.

As for the hysteresis "problem", in practice it doesn't seem to be
much of a problem. things get well under control before a web page
completes.Same goes for my tests against DASH traffic. I have plenty
of plots and traces of this. Many are on the results webpage for
bufferbloat.net.

as for a good default for interval, a good number IS dependent on your
RTT, and without coupling the ingress and output queues, it's
difficult to determine or even auto tune that. Perhaps with connection
tracking or some other form of coupling, one day. The ACC code from
the gargoyle router project is worth looking at.

I am satisfied that fq_codel can be deployed on fixed rate lines
without any tuning on bandwidths ranging from 4mbit to 1gbit, today,
as it stands. I have done hundreds of thousands of tests to prove
that. Optimizations are helpful for the 3 band system that is what
mostly deployed today, such as smaller quantum's on slow asymmetric
links, and a smaller packet limit on low memory routers.

A larger target is working well on sub 4ms links. I think that could
auto tune better.

A lower target and interval seem right for data center use, but I have
yet to get anyone to run my suite of published tests.

A rate limiter is required to compensate for ISP's lousy
dslam/cmts/gpon head end and CPE at least until this code makes it
onto those devices. Long lead times predominate on this sort of
hardware - We have three years to get DOCSIS 3.1 right, as one
example.

These are second order problems that will be fixed over time. Wifi and
wireless remain problematic, but dents in those problems seem imminent
by next year, and many of the problems aren't aqm or packet scheduling
ones.

SO. damn straight, I'm one of the people pushing for deployment,
notably on boxes that are easy to upgrade and fix as we learn more
about what we should be doing. I'm definitely reluctant to hard code
stuff into big iron or hard to replace firmware as yet. But as matt
mathis said at ietf - what we have is such an improvement over what is
in place today, that it is time to deploy. After almost 3 years of
effort I'm happy to have a few million boxes in place to learn more
from. Aren't you?

We just have a couple billion boxes left to fix. Plenty of time to
tweak things as we go along. If you want RED, or ARED, in linux, it's
been fixed now for 2 years to perform as to the spec. Go for it. If
you could create something to automate RED configuration as I have for
the ceroshaper tool in cerowrt, let me know.

Any time someone has debloating code worth working on... I'm willing
to help. I've been helping on pie, and as you know I've been looking
over your DCTCP experiment carefully, finding and fixing bugs, and
moving the code forward to where it can be compared against a modern
kernel and a modern TCP and modern AQM and packet scheduling systems.




On Thu, Dec 12, 2013 at 4:05 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> Dave,
>
>
> At 22:11 12/12/2013, Dave Taht wrote:
>>
>> but quickly...
>>
>> Bob, I object to your characterization of users links being busy 1-3%
>> of the time. That's an average.
>
>
> I said it was an average. You're repeating and agreeing with what I said,
> but saying you object to me saying it?
>
>
>> When they are busy, they are very busy
>> for short periods, typically 2-16 seconds in the case of web traffic,
>> then idle for minutes. DASH traffic is busy for 2+ seconds every 10 on
>> a 20mbit link, and so on, for 1.5 hours or so. Etc.
>
>
> Yes, again, you're agreeing with me.
>
> The mean for a Web session is towards the low end of the 2-16 seconds ran=
ge
> even now. And as we get the other latency-saving advances out there (e.g.
> removing TCP & TLS handshakes, proper pipelining, and a faster replacemen=
t
> for slow-start without overshoot), then there is potential for Web sessio=
ns
> to drop to 1-2 seconds or less, because they're usually a long way from
> being bandwidth limited.
>
> You said fq_codel decays out its memory in 800ms. So it will typically ha=
ve
> lost all its memory when each Web transfer starts and when each DASH
> transfer re-starts.
>
> So I think you're agreeing that this 200ms signalling delay will be
> predominant?
>
>
>> In both cases
>> congestion exists, and in both cases AQM reaction time measured in
>> 200ms or so is still vastly superior to what happens today, and packet
>> scheduling masks it to a large extent.
>
>
> You're saying it's OK to propose a solution that delays signalling
> congestion for about 10 typical CDN RTTs,... because it's better than
> nothing?
>
> That's rich. I have to say this... You're one of the group of protagonist=
s
> who has persuaded the world to embark on a programme of /implementation/
> updates that will take years, and rubbished deploying the AQM that was
> already implemented (RED), even tho it was already much better than nothi=
ng
> too.
>
> Yes, auto-config for line rate is a nice bell to add to the bicycle. If w=
e
> are going to embark on new implementations, auto-config for RTT is no les=
s
> important.
>
> Lack of auto-config for either requires the fixed config to be at the slo=
w
> end of the range. And both have a similar range of variability (the RTT
> range is actually wider). So lack of auto-config in either case leads to =
a
> similar order of unnecessary delay. Particularly given traffic is
> predominantly in short sparse bursts.
>
>
>
> Bob
>
>
>
>
>> Jim, yes, I was trying to establish the groundwork for ensuring
>> everyone really understood codel-by-itself before talking about
>> fq_codel. I'm still not sure that's been established. Anyone here care
>> to calculate the number of drops on two flows going through codel and
>> fq_codel, starting with iw10, over a 10mbit 2ms RTT link? And when,
>> approximately, the queue becomes ideal? And how often the fq_codel
>> "fast queue" gets used in this case? Gold stars for everyone that gets
>> it right.
>>
>> Jim, I'd like you to use larger download speeds than a mbit for your
>> examples. somewhere between 8 and 20mbit seems appropriate. (IMHO iw10
>> should not be used on the modern internet on sub 10Mbit links.) The
>> dynamics change significantly as you get more bandwidth than iw10
>> abuses.
>>
>> Anyway=85
>>
>> After I got as far as describing fq_codel accurately in this thread,
>> then I'd hoped to be able to tackle the immediate ECN issue, the value
>> of randomness in pie, and the effectiveness and need for ECN on the
>> edge as it is currently defined.
>>
>> and I figured that then I might have written enough to get closer to an
>> rfc.
>>
>> and when I started kibitzing on this thread I thought I was talking to
>> the DCTCP case which I've spent a few months studying up on, and
>> looking over alternative ideas there, like
>>
>> http://conferences.sigcomm.org/co-next/2013/program/p49.pdf
>> http://conferences.sigcomm.org/co-next/2013/program/p151.pdf
>>
>> Scalable, Optimal Flow Routing in Datacenters via Local Link Balancing
>>
>> http://www.irt-systemx.fr/wp-content/uploads/2013/12/AINTEC.ppt
>>
>> Sigh. I'll
>>
>> On Thu, Dec 12, 2013 at 11:35 AM, Jim Gettys <jg@freedesktop.org> wrote:
>> >
>> >
>> >
>> > On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com> wrot=
e:
>> >>
>> >> Jim,
>> >>
>> >>
>> >> At 16:55 11/12/2013, Jim Gettys wrote:
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com>
>> >> wrote:
>> >> Jim,
>> >>
>> >> I'm just checking we're not talking past each other. I'll repeat two
>> >> quotes from each of us, then comment.
>> >>
>> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrot=
e:
>> >>
>> >> 3{New}. It SHOULD be possible to make different instances of an AQM
>> >> algorithm apply to different subsets of packets that share the same
>> >> queue.
>> >> It SHOULD be possible to classify packets into these subsets at least
>> >> by ECN
>> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivale=
nt
>> >> of
>> >> these fields at lower layers),
>> >>
>> >>
>> >> At 19:50 05/12/2013, Jim Gettys wrote:
>> >>
>> >> "Certainly, it may be the same instance of an AQ
>> >> M algorithm, rather than different instances, for example."
>> >>
>> >>
>> >> That's true of course, but the case with one AQM handling all packets
>> >> within a queue is the norm. I want to check you're happy with the
>> >> converse:
>> >> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED
>> >> with
>> >> in and out of contract). So we can have WPIE, WCoDel etc where the
>> >> differentiation between aggregates is provided by different AQM
>> >> instances in
>> >> the same queue, not by different queues with different scheduling
>> >> priorities.
>> >> 2) Extending this so that AQM differentiation can be between
>> >> ECN-capable
>> >> and Not-ECN-capable aggregates, not just between Diffserv classes (an
>> >> example being CoDel with a lower 'interval' for ECN-capable packets).
>> >>
>> >> I presented the evaluations of this last idea in tsvwg on the final
>> >> Friday
>> >> of the Vancouver IETF - I don't think you were there. <
>> >> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>> >>
>> >>
>> >> Yes, unfortunately I had to leave before the Friday session.
>> >> This is my primary motivation for this wordsmithing - I'm trying allo=
w
>> >> us
>> >> to move towards zero signalling delays in CoDel, PIE and RED (current=
ly
>> >> defaults of 200ms, 100ms and 512packets respectively, which are not
>> >> good for
>> >> dynamics).
>> >>
>> >>
>> >> Certainly signalling delays are very important: this is why I'm
>> >> favorably
>> >> inclined to "head mark/drop", as it signals TCP as quickly as possibl=
e,
>> >> keeping the response of the TCP feedback loop as tight as possible (a=
nd
>> >> part
>> >> of why I like CoDel so much for the highly variable bandwidth problem
>> >> we
>> >> face at the edge of the net).
>> >>
>> >> It's *really* important than when the bandwidth drops suddenly that
>> >> everyone gets told to slow down quickly (exactly how quickly probably
>> >> depends on the propagation change characteristics of the medium), or
>> >> packets
>> >> can pile up in a big way.
>> >>
>> >> How quickly the mark/drop algorithm can figure out that signalling is
>> >> appropriate is the *other* piece of getting good dynamics.  Here I
>> >> don't
>> >> doubt that something may be discovered that is better than CoDel in t=
he
>> >> slightest.
>> >> It takes a CoDel instance (within an fq structure) 200ms from its que=
ue
>> >> first passing 'threshold' before it will ever drop the first packet
>> >> (unless
>> >> the queue hits taildrop before that). So if the RTT is 20ms, that's
>> >> 220ms
>> >> signalling delay.
>> >
>> >
>> > No, again, see Dave's mail, and you are missing the flow scheduling
>> > aspect
>> > of this and thinking in terms of a single queue and the usual mark/dro=
p
>> > cases; this is the exact cognitive problem I'm talking about.
>> >
>> > The flow scheduling aspect of fq_codel is *more* important than what
>> > mark/drop algorithm decides to signal the TCP's to regulate their serv=
o
>> > systems.  I really wish this algorithm had been called "fs_codel",
>> > rather
>> > than "fq_codel", as it is so easy to confuse "fair" with "flow", and
>> > "queue"
>> > with "scheduling".
>> >
>> > Regulating TCP is absolutely essential to keep TCP "sane" and
>> > responsive,
>> > but it isn't the most important part of what is going on.
>> >
>> > Here's the case of a new TCP flow on a previously idle link: After the
>> > TCP
>> > open handshake, you will have no more than 4 or (if IW10 is active)
>> > packets
>> > for that flow in the queue (actually, 3 packets, as presumably at leas=
t
>> > one
>> > packet is in process of transmission; @ 1Mbps, that first full size
>> > packet
>> > takes 13ms).
>> >
>> > If another flow starts, it will get preferentially scheduled to the
>> > existing
>> > flow(s) until it has also built a queue, at which time it competes
>> > "fairly"
>> > with the other flows in the packet scheduling.
>> > This means, in the simple low bandwidth case, that as soon as you star=
t
>> > the
>> > second flow, it gets the next possible opportunity to be transmitted i=
n
>> > preference to any flow that has built a queue.
>> >
>> > Here's the kicker:
>> >
>> > We've just avoid 3 packets worth of "head of line blocking" latency fo=
r
>> > the
>> > second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved jus=
t
>> > in
>> > the TCP open, and more in that the first packet from the second flow
>> > then
>> > gets scheduled immediately, getting that TCP moving.
>> >
>> > Similarly for your voip packet; it saves those 40ms. And your bulk flo=
w
>> > gets
>> > its first packet through ASAP as well; for web traffic, that usually
>> > contains the size information or other metadata required for the web
>> > browser
>> > to unblock.  And that is for a *single* competing TCP connection.
>> >
>> > Now, let's look at today's web: there are many embedded objects in a
>> > page.
>> > Once the base page is downloaded, the web browser (presuming its DNS
>> > lookups
>> > are already cached), opens a pile of connections all at once.  Whether=
 I
>> > like it or not, (which I don't, as I tried to prevent it with pipelini=
ng
>> > in
>> > HTTP/1.1 in the 1990's), you can easily have 5-30 TCP connections star=
t
>> > almost simultaneously. See the connections/page plot on my web page
>> > explaining all this stuff in more detail at:
>> >
>> > http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queu=
ing-traditional-aqm-is-not-enough/
>> >
>> > Without any TCP flow control ever happening, I can easily get 40 (or
>> > even
>> > several hundred packets!) arriving at near line rate (the initial wind=
ow
>> > *
>> > number of embedded objects).  Most of these TCP connections *won't eve=
r
>> > even
>> > get out of slow start*; the objects are typically pretty small.  I've
>> > observed transient latency from HOL blocking of > 100ms on 50Mbps cabl=
e
>> > service, on some web pages.
>> >
>> > Nothing we will/can do at the TCP level can help this situation!!!
>> > Nothing,
>> > other than in the long term getting the incentives right so that fewer
>> > TCP
>> > connections might be preferable to gaming TCP, as the web already has.
>> >
>> > That CoDel may take a while to figure out that it should start marking
>> > in
>> > the idle case is really pretty irrelevant; the packets have already
>> > arrived
>> > and unless we do better packet scheduling, you are fully stuck.  Dave'=
s
>> > trying to also explain that a number of people's understanding of how
>> > CoDel
>> > works has been wrong.
>> >
>> > But as I say, *the details of the mark/drop algorithm doesn't matter*.
>> > Having 3, or 100 packets queued in a FIFO queue will already mean you
>> > are
>> > screwed for anything low latency at the bandwidths we all care about.
>> >
>> > Once a flow drains to zero, it again gets treated as a new flow.
>> > And what we choose to define a "flow" to be is arbitrary, though the
>> > code
>> > today does the usual 5tuple.
>> >
>> >> In fq_codel this creates considerable self-delay for short flows or r=
-t
>> >> apps, which kill their own latency before they get any loss signal to
>> >> tell
>> >> them to slow down.
>> >
>> >
>> > Not likely in the common case.  See Dave's comments and also note that
>> > the
>> > initial burst of web page load runs the queue up high immediately.
>> >
>> > And as I keep saying, and will say again, the flow queuing decisions
>> > avoiding HOL blocking explained are much more important to dealing wit=
h
>> > the
>> > latency problems.
>> >
>> > So go invent better algorithms that CoDel for drop/mark, that can be
>> > applied
>> > to the flow queuing parts of the algorithm, and I'm happy.  Running co=
de
>> > please.
>> >
>> >
>> >
>> >>
>> >> Even for elastic flows, with congestion signals delayed by so much,
>> >> they
>> >> risk hitting themselves with a huge train of overshoot loss. This wou=
ld
>> >> be
>> >> the same for fq_pie, except the number is 100ms + RTT.
>> >
>> >
>> > When fq_pie exists to test, I'll be happy to see.
>> >
>> > I keep saying, and I'll say again: while some mark/drop algorithm need=
s
>> > to
>> > exist, flow scheduling is more important to getting good latency....
>> >
>> >
>> >>
>> >>
>> >> Yes, the e2e transport could measure delay growth, but it doesn't kno=
w
>> >> whether the delay is coming from a queue that is isolated from others
>> >> or
>> >> not. So it doesn't want to slow down too quickly in response to delay
>> >> growth
>> >> in case it gets screwed by other traffic. Ie. using delay growth as a
>> >> signal
>> >> entails considerable signalling delay due to all the uncertainty.
>> >>
>> >> The proposal you missed in tsvwg was to define ECN as an immediate
>> >> signal
>> >> from the network, 'interval'=3D0 in CoDel terms, so the host always g=
ets
>> >> congestion signals as fast as possible, and if it needs bursts of
>> >> signals
>> >> smoothed out, it can do that itself.
>> >
>> >
>> > Yes, I get that.  I got that a year or more ago. The idea has potentia=
l
>> > merit. ECN as it is currently defined is not so useful.
>> >
>> > And I wish it would get a different name so we could more easily talk
>> > about
>> > the two different things now being called ECN.
>> >
>> > I can see having ECN marks on a burst of packets may be helpful in
>> > having
>> > the receiver judge things in a highly variable wireless scenario; it m=
ay
>> > have additional information about the medium and know that that
>> > transient
>> > has gone away, and that it may not be a wise idea to slow the connecti=
on
>> > at
>> > all.
>> >
>> >
>> >>
>> >> The suggested wording ensures all AQM implementations will allow
>> >> operators, vendors and users to configure such a mechanism. But I've
>> >> generalised it from ECN to Diffserv too (because the implementation
>> >> would be
>> >> no different).
>> >
>> >
>> > As noted before, Diffserv is still interesting, even though the packet
>> > scheduling in fq_codel (or similar algorithms) makes it much less
>> > necessary.
>> > There are two aspects to this:
>> >
>> > 1) higher priority contention to the medium.  If I have a real VOIP
>> > packet,
>> > there are ways I can ask for higher priority access to the medium and
>> > reduce
>> > the total number of transmit opportunities my traffic requires (and th=
at
>> > is
>> > often the scarcest resource on WiFi or Docsis).
>> > 2) any hint I can get helps (at the edge) so I can distinguish those
>> > packets
>> > from the way the web has been gaming the network.
>> >
>> > Even so, for Diffserv to be safely trusted and honored even in your
>> > home,
>> > the end user (who is the network operator in this case), will have be
>> > able
>> > to know that a device or application is using it and control whether o=
r
>> > not
>> > it's honored. Unless it is under the network operator's control (in th=
is
>> > case, you the home user) Diffserv can/will get gamed to uselessness by
>> > application and device manufacturers. Ergo why Dave and I hack on home
>> > routers again: this level of control is not currently present in these
>> > devices, and must be for Diffserv to be useful.
>> >
>> >>
>> >>
>> >>
>> >> My basic issue is one of terminology: people have talked about "best
>> >> effort" queues.  In reality, this is a "class" of service, rather tha=
n
>> >> a
>> >> single queue, and when you get into the mental model of BE being a
>> >> single
>> >> queue, (rather than a set of queues) it can lead one astray quickly a=
nd
>> >> easily.
>> >>
>> >>
>> >> Yeah, I know this. I suspected we were talking past each other.
>> >>
>> >> I need you to allow the other case into your mind for this
>> >> conversation.
>> >> The wording is specifically about the case where "different subsets o=
f
>> >> packets ... share the same queue".
>> >
>> >
>> > And the word "queue" in most people's minds implies ordering, and FIFO
>> > behavior.  This is the terminological issue I'm harping on.  It's also
>> > why I
>> > think "bufferbloat" is a better term for our situation than "queue
>> > bloat",
>> > which you liked and have harped at me about.  Buffers don't have such =
an
>> > implication of ordering.
>> > So if you talked about a buffer here, rather than a queue, I'd be a lo=
t
>> > happier. At least in my mind, queues are ordered.
>> >
>> >>
>> >> We can talk about an fq structure for this another time, but it's a
>> >> really
>> >> complicated way of doing it.
>> >>
>> >> Given simple looks like it could work, why get complicated already?
>> >
>> >
>> > Because the flow scheduling is such a win.  You can't solve the whole
>> > problem just with mark/drop algorithms and FIFO queues and get reliabl=
y
>> > to
>> > decent latencies.
>> >
>> > Now, whether Fred's document can/should go into anything like that tha=
t
>> > detail is far from clear (arguably not).
>> >
>> > I just don't want to further the mythology we can get to decent
>> > latencies at
>> > the edge of the network to continue with FIFO queues and an AQM
>> > algorithm,
>> > and it's clear that many haven't yet internalized that its flow
>> > scheduling
>> > combined with a self tuning adaptive mark/drop algorithm that we must
>> > have,
>> > but that the flow scheduling makes the biggest difference.
>> >
>> >>
>> >>
>> >> It's really easy to fall into the idea of a single software queue
>> >> mapping
>> >> to some single hardware supported queue, and that's a cognitive
>> >> mistake, as
>> >> aggregating MACs are showing us; transmit ops are often the scarcest
>> >> resource...
>> >>
>> >>
>> >> It's only a cognitive mistake if one is not aware of all the options.
>> >> I'm
>> >> fully aware of all the options.
>> >>
>> >> To be specific, a queue into a wireless medium should be configured s=
o
>> >> it
>> >> holds some 'good queue' in reserve for transmit ops, but the queues o=
n
>> >> top
>> >> of this that TCP self-inflicts even briefly are not 'good queues' eve=
n
>> >> if
>> >> they are isolated from other flows by fq - VJ was wrong to generalise
>> >> the
>> >> phrase 'good queue' to all bursts of queue - it is only necessary to
>> >> hold
>> >> back from signalling and allow a burst of queue if the only possible
>> >> signal
>> >> is a drop. With ECN, you don't have this dilemma. This is the key to
>> >> rapid
>> >> dynamics.
>> >
>> >
>> > But you also then have to solve the HOL blocking problem, and do so
>> > urgently.  Ergo flow scheduling.... To get to where we need to go, you
>> > have
>> > to worry about the order in which each packet is scheduled.
>> >
>> >>
>> >>
>> >> Diffserv marking has the potential to give a "hint" to distinguish ho=
w
>> >> particular flows should be handled (scheduled) in a service class, an=
d
>> >> as my
>> >> previous example shows, that hint may be very useful in channel acces=
s
>> >> decisions (e.g. voip on 802.11).
>> >
>> >
>> > ECN doesn't help a bit with the head of line blocking problem; it
>> > actually
>> > will make it worse with FIFO scheduling. ECN means that you can't even
>> > get
>> > the packets out of the way.
>> >
>> > Which says you'd better be doing more clever scheduling than a FIFO.
>> >
>> > If you want ECN to be usable in the way you want it to be at low
>> > bandwidths,
>> > you better become a fan of flow scheduling...
>> >>
>> >>
>> >> But fq_codel teaches the lesson that packet scheduling combined with
>> >> keeping TCP sane is a key improvement over handling either problem
>> >> apart...
>> >> In particular, the first packets of new flows/reappearing flows are
>> >> vastly
>> >> more "important" than other packets in terms of the latency cost to
>> >> users of
>> >> that service. Each flow has in essence its own queue in this service
>> >> class,
>> >> and we're using information from that to help schedule the packets in
>> >> ways
>> >> that minimize latency to the user.
>> >>
>> >>
>> >> I know all this. Please can we keep to the conversation about how to
>> >> avoid
>> >> the 200ms signalling delay that fq_codel inflicts on each flow (and t=
he
>> >> similar signalling delays that other AQMs inflict).
>> >
>> >
>> > As I said, it's not the big problem we have today at the edge of the
>> > network, and Dave's mail explains your model of CoDel isn't so correct=
.
>> >
>> > It's only very long lived flows where the signalling even matters, and
>> > that's not what we get at the edge of today's network; instead, we get=
 a
>> > mix
>> > of a few larger (e.g. video player flows) with the DOS attack that is
>> > web
>> > traffic, with some isochronous VOIP and teleconferencing traffic.
>> >>
>> >>
>> >>
>> >>
>> >> So in this case, a single algorithm is acting over a bunch of flows i=
n
>> >> a
>> >> single class of service, and both scheduling packets among the flows,
>> >> and
>> >> signalling TCP flows appropriately when they should "slow down".
>> >>
>> >>
>> >> Yup, I know this.
>> >>
>> >>
>> >>
>> >> So I think you and I are on close to the same page (but have been
>> >> burned
>> >> badly in the past by terminology issues getting in the way).  On
>> >> HTTP/1.1 we
>> >> wasted probably > 2 years talking past each other because we didn't
>> >> have
>> >> clear and concise terminology that we all understood the same way.
>> >>
>> >>
>> >> As I thought, we are talking past each other.
>> >
>> >
>> > Yes, in part because I think few have internalized what the web has do=
ne
>> > to
>> > edge of the network.  It isn't what I had hoped it would be when I was
>> > HTTP/1.1 editor.  Why this outcome occurred is too long a discussion f=
or
>> > this thread.
>> >
>> >
>> >> We need to be able to have a conversation that is not always "Hmm,
>> >> that's
>> >> sounds like it might be interesting. Can I tell you about fq_codel
>> >> now?"
>> >
>> >
>> > Running code of other algorithms very welcome. Fq_codel is running cod=
e.
>> > Pie is running code.
>> >
>> > Maybe fq_pie will be running code we can test someday.
>> >
>> > Even then, I want a low target latency so individual TCP's are kept
>> > responsive, and I need an algorithm that can keep up quickly with the
>> > dynamics of wireless, so most simplistic tests will not be useful (we
>> > don't
>> > have such good evaluation tests today).
>> >
>> > When it is, and if it is better than fq_codel, I'll be happy.  But the
>> > mark/drop part of the algorithm *isn't* the most important part of the
>> > algorithm. The packet scheduling decisions are....
>> >                                - Jim
>> >
>> >>
>> >>
>> >>
>> >>
>> >> Bob
>> >>
>> >>
>> >>
>> >> And I don't claim I have the right terminology for all this stuff,
>> >> either
>> >> (even in this mail).
>> >>
>> >> Which is why I was loathe to suggest exact text...
>> >>                            - Jim
>> >>
>> >>
>> >>
>> >> At 19:50 05/12/2013, Jim Gettys wrote:
>> >>
>> >>
>> >>
>> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wrot=
e:
>> >> Fred, Gorry, all,
>> >> I promised to suggest text for draft-ietf-aqm-recommendation about
>> >> allowing the AQM's behaviour to be independent for ECN and non-ECN
>> >> packets.
>> >> In the process, I realised we can't talk about independent AQMs for E=
CN
>> >> without also including Diffserv.
>> >> This gets messy, because I believe a good AQM for BE traffic with and
>> >> without ECN, should remove much if not all the need for Diffserv. But
>> >> we
>> >> can't ignore Diffserv.
>> >>
>> >>
>> >> I agree in principle with what Bob is trying to say here (and is very
>> >> much
>> >> what I've been saying in my blog entry of last summer).
>> >>
>> >> Once you have things under control, the need for Diffserv diminishes
>> >> dramatically (but does not go away).
>> >>
>> >> But as Bob notes, there is still a good use for Diffserv: suitably
>> >> marked
>> >> traffic may want to contend for access to the channel differently: yo=
ur
>> >> marked VOIP packets may want to change the priority with which you
>> >> request
>> >> channel access, so that you get more timely access to the medium. Thi=
s
>> >> conserves transmit opportunities, which is often the scarcest resourc=
e
>> >> in
>> >> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
>> >> between
>> >> your VOIP working well, and not working well, on a busy 802.11 networ=
k
>> >> as
>> >> well as using the channel as efficiently as possible.
>> >>
>> >> Similarly, if you have packets you know are background, it's helpful =
to
>> >> know that to ensure that they never contend for access to the medium
>> >> but
>> >> will always defer to other traffic, and just scavenge available space
>> >> in
>> >> other transmit opportunities where possible.
>> >>
>> >> I'm a bit loathe though to tie the behavior to queues, however; in
>> >> particular, best effort traffic may want to be sent in the same
>> >> aggregate as
>> >> higher (or lower) priority traffic, if there is remaining space in th=
e
>> >> aggregate.
>> >>
>> >> In short, the mental model we've had that there is a one-to-one model
>> >> of
>> >> hardware and software queues (not to mention flows in a given softwar=
e
>> >> queue) is often incorrect (or at least seriously sub-optimal) in
>> >> today's
>> >> systems (even if the hardware queues "work" properly, which it appear=
s
>> >> they
>> >> do not in 802.11).
>> >>
>> >> So I'm not sure Bob's new section 3 here is how to best to state this
>> >> (or
>> >> to deal with the terminology problem).  Certainly, it may be the same
>> >> instance of an AQM algorithm, rather than different instances, for
>> >> example.
>> >> And "
>> >> It SHOULD be possible" is more a pious wish than anything else.  But =
I
>> >> agree in spirit with what Bob's trying to say.
>> >>                                - Jim
>> >>
>> >>
>> >>
>> >> _____________________________________________________________________=
____________________
>> >> {In Section 4: add another bullet between recommendations 2 & 3:}
>> >> 3{New}. It SHOULD be possible to make different instances of an AQM
>> >> algorithm apply to different subsets of packets that share the same
>> >> queue.
>> >> It SHOULD be possible to classify packets into these subsets at least
>> >> by ECN
>> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equivale=
nt
>> >> of
>> >> these fields at lower layers).
>> >> {Then a new section to expand on this before the current Section 4.3.=
}
>> >> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>> >> The recommendation to provide a separate instance of the AQM for ECN
>> >> packets goes beyond the assumptions of RFC 3168, which assumed that
>> >> only one
>> >> instance of an AQM will handle both ECN-capable and non-ECN-capable
>> >> packets.
>> >>
>> >>
>> >> Bob
>> >>
>> >>
>> >> ________________________________________________________________ Bob
>> >> Briscoe,                                                  BT
>> >> _______________________________________________ aqm mailing list
>> >> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >>
>> >> ________________________________________________________________
>> >> Bob Briscoe,                                                  BT
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> >> ________________________________________________________________
>> >> Bob Briscoe,                                                  BT
>> >>
>> >>
>> >> _______________________________________________
>> >> aqm mailing list
>> >> aqm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/aqm
>> >>
>> > I can imagine many other possible algorithms that CoDel for the
>> > mark/drop
>> > algorithm; we happen to like CoDel atm for two reasons: 1) it is self
>> > adapting to the line rate, and 2) head mark/drop signals the TCP's as
>> > soon
>> > as a decision is made rather than possibly being applied much later
>> > (such as
>> > random or tail drop).  We welcome and encourage further essays in the
>> > art.
>> >
>> >
>> > _______________________________________________
>> > aqm mailing list
>> > aqm@ietf.org
>> > https://www.ietf.org/mailman/listinfo/aqm
>> >
>>
>>
>>
>> --
>> Dave T=E4ht
>>
>> Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From dave.taht@gmail.com  Fri Dec 13 00:23:06 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 25E2C1ADFEC for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 00:23:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qlpddc41jzYO for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 00:22:58 -0800 (PST)
Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) by ietfa.amsl.com (Postfix) with ESMTP id ED5891AE149 for <aqm@ietf.org>; Fri, 13 Dec 2013 00:22:57 -0800 (PST)
Received: by mail-wg0-f44.google.com with SMTP id a1so1496349wgh.11 for <aqm@ietf.org>; Fri, 13 Dec 2013 00:22:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=U9yxmkbydqqHZYmf4DrncxxBacgidL2taHpz33CcQVQ=; b=BDST5IicalSdEgdV4J2tx6IqXMheq0plF1A+vBD4Ohc+osQSZ7kw5pgjURqJDyFMB7 kagqjvB0dqjc5gl8uRcc4SR+NMJl8Iy/cTTLFlQmgisHCjVt9lGn40R7Y158Rq4pG1+O mApEteiWWTZ0lRH24BMXSctsfUD7xSzZ5FkKDzkj5bQxUc0VmGWlS2N8KmcGH1xA0p+G rkymRLs/JhGzGumlmmIzY/j6Z07811gsTR8OTsTvj2q/Lzk0Ki4NJBSI96n/jQwWlSD8 dadca+cpmpRFWxsQWWLn3kEtY9ZHcGXenkIeUPWiaNuTGMZppNyQmi9kZ3dktkVunqp5 x1TA==
MIME-Version: 1.0
X-Received: by 10.180.101.197 with SMTP id fi5mr1798655wib.46.1386922971227; Fri, 13 Dec 2013 00:22:51 -0800 (PST)
Received: by 10.217.123.69 with HTTP; Fri, 13 Dec 2013 00:22:51 -0800 (PST)
In-Reply-To: <CAA93jw7UrTzUrMAkaQOr1dQoGjWD2ktLAn-Uv-oA6x88HpYLfA@mail.gmail.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com> <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com> <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk> <CAA93jw7UrTzUrMAkaQOr1dQoGjWD2ktLAn-Uv-oA6x88HpYLfA@mail.gmail.com>
Date: Fri, 13 Dec 2013 00:22:51 -0800
Message-ID: <CAA93jw71X83U_-XNjzhX3+LyL-=xZr3Omi1Fhq6PcEvxZt1tKQ@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Dec 2013 08:23:06 -0000

On Fri, Dec 13, 2013 at 12:15 AM, Dave Taht <dave.taht@gmail.com> wrote:
> For starters, the codel signaling delay from the onset of continuous
> over 5ms delay on packets defaults to target 100ms, not 200ms.

interval 100ms.

it has been a long week.

> I don't
> know who started saying 200ms but even I started believing it with the
> few brain cells I've had to spare of late. 5x a CDN rtt in a world of
> 30-60k images sounds about right.
>
> Secondly, codel drops/marks from head, not from tail, so the signal
> gets back to the sender in 1/2  the real physical RTT after that,
> rather than the tail of a queue that may be out of control at that
> point. Much faster than pie.
>
> There has been so much misinformation spread of late on these threads.
> I'm hoping we're beginning to make a dent in it? I look forward to
> making all this clear on the upcoming RFCs. I think I should stop now,
> revisit the rest of this thread and see what else can be cleared up
> before even beginning to tackle fq_codel after I get caught up on
> sleep.
>
> as for your other comments...
>
> I have always said deploy RED and for that matter DRR, SFQ or SQF
> where you can. I distinctly remember polling the crowd at the first
> uknof I went to and being sad to discover only about 4% of the room
> had (4 people).
>
> I DO hold that red is too hard to configure for ordinary mortals, and
> that it doesn't work at all on variable bandwidth links like cable, or
> wireless, which happen to be the dominant form of end-user link
> nowadays.
>
> As for the hysteresis "problem", in practice it doesn't seem to be
> much of a problem. things get well under control before a web page
> completes.Same goes for my tests against DASH traffic. I have plenty
> of plots and traces of this. Many are on the results webpage for
> bufferbloat.net.
>
> as for a good default for interval, a good number IS dependent on your
> RTT, and without coupling the ingress and output queues, it's
> difficult to determine or even auto tune that. Perhaps with connection
> tracking or some other form of coupling, one day. The ACC code from
> the gargoyle router project is worth looking at.
>
> I am satisfied that fq_codel can be deployed on fixed rate lines
> without any tuning on bandwidths ranging from 4mbit to 1gbit, today,
> as it stands. I have done hundreds of thousands of tests to prove
> that. Optimizations are helpful for the 3 band system that is what
> mostly deployed today, such as smaller quantum's on slow asymmetric
> links, and a smaller packet limit on low memory routers.
>
> A larger target is working well on sub 4ms links. I think that could
> auto tune better.

s/4ms/4mbit/

>
> A lower target and interval seem right for data center use, but I have
> yet to get anyone to run my suite of published tests.
>
> A rate limiter is required to compensate for ISP's lousy
> dslam/cmts/gpon head end and CPE at least until this code makes it
> onto those devices. Long lead times predominate on this sort of
> hardware - We have three years to get DOCSIS 3.1 right, as one
> example.
>
> These are second order problems that will be fixed over time. Wifi and
> wireless remain problematic, but dents in those problems seem imminent
> by next year, and many of the problems aren't aqm or packet scheduling
> ones.
>
> SO. damn straight, I'm one of the people pushing for deployment,
> notably on boxes that are easy to upgrade and fix as we learn more
> about what we should be doing. I'm definitely reluctant to hard code
> stuff into big iron or hard to replace firmware as yet. But as matt
> mathis said at ietf - what we have is such an improvement over what is
> in place today, that it is time to deploy. After almost 3 years of
> effort I'm happy to have a few million boxes in place to learn more
> from. Aren't you?
>
> We just have a couple billion boxes left to fix. Plenty of time to
> tweak things as we go along. If you want RED, or ARED, in linux, it's
> been fixed now for 2 years to perform as to the spec. Go for it. If
> you could create something to automate RED configuration as I have for
> the ceroshaper tool in cerowrt, let me know.
>
> Any time someone has debloating code worth working on... I'm willing
> to help. I've been helping on pie, and as you know I've been looking
> over your DCTCP experiment carefully, finding and fixing bugs, and
> moving the code forward to where it can be compared against a modern
> kernel and a modern TCP and modern AQM and packet scheduling systems.
>
>
>
>
> On Thu, Dec 12, 2013 at 4:05 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>> Dave,
>>
>>
>> At 22:11 12/12/2013, Dave Taht wrote:
>>>
>>> but quickly...
>>>
>>> Bob, I object to your characterization of users links being busy 1-3%
>>> of the time. That's an average.
>>
>>
>> I said it was an average. You're repeating and agreeing with what I said=
,
>> but saying you object to me saying it?
>>
>>
>>> When they are busy, they are very busy
>>> for short periods, typically 2-16 seconds in the case of web traffic,
>>> then idle for minutes. DASH traffic is busy for 2+ seconds every 10 on
>>> a 20mbit link, and so on, for 1.5 hours or so. Etc.
>>
>>
>> Yes, again, you're agreeing with me.
>>
>> The mean for a Web session is towards the low end of the 2-16 seconds ra=
nge
>> even now. And as we get the other latency-saving advances out there (e.g=
.
>> removing TCP & TLS handshakes, proper pipelining, and a faster replaceme=
nt
>> for slow-start without overshoot), then there is potential for Web sessi=
ons
>> to drop to 1-2 seconds or less, because they're usually a long way from
>> being bandwidth limited.
>>
>> You said fq_codel decays out its memory in 800ms. So it will typically h=
ave
>> lost all its memory when each Web transfer starts and when each DASH
>> transfer re-starts.
>>
>> So I think you're agreeing that this 200ms signalling delay will be
>> predominant?
>>
>>
>>> In both cases
>>> congestion exists, and in both cases AQM reaction time measured in
>>> 200ms or so is still vastly superior to what happens today, and packet
>>> scheduling masks it to a large extent.
>>
>>
>> You're saying it's OK to propose a solution that delays signalling
>> congestion for about 10 typical CDN RTTs,... because it's better than
>> nothing?
>>
>> That's rich. I have to say this... You're one of the group of protagonis=
ts
>> who has persuaded the world to embark on a programme of /implementation/
>> updates that will take years, and rubbished deploying the AQM that was
>> already implemented (RED), even tho it was already much better than noth=
ing
>> too.
>>
>> Yes, auto-config for line rate is a nice bell to add to the bicycle. If =
we
>> are going to embark on new implementations, auto-config for RTT is no le=
ss
>> important.
>>
>> Lack of auto-config for either requires the fixed config to be at the sl=
ow
>> end of the range. And both have a similar range of variability (the RTT
>> range is actually wider). So lack of auto-config in either case leads to=
 a
>> similar order of unnecessary delay. Particularly given traffic is
>> predominantly in short sparse bursts.
>>
>>
>>
>> Bob
>>
>>
>>
>>
>>> Jim, yes, I was trying to establish the groundwork for ensuring
>>> everyone really understood codel-by-itself before talking about
>>> fq_codel. I'm still not sure that's been established. Anyone here care
>>> to calculate the number of drops on two flows going through codel and
>>> fq_codel, starting with iw10, over a 10mbit 2ms RTT link? And when,
>>> approximately, the queue becomes ideal? And how often the fq_codel
>>> "fast queue" gets used in this case? Gold stars for everyone that gets
>>> it right.
>>>
>>> Jim, I'd like you to use larger download speeds than a mbit for your
>>> examples. somewhere between 8 and 20mbit seems appropriate. (IMHO iw10
>>> should not be used on the modern internet on sub 10Mbit links.) The
>>> dynamics change significantly as you get more bandwidth than iw10
>>> abuses.
>>>
>>> Anyway=85
>>>
>>> After I got as far as describing fq_codel accurately in this thread,
>>> then I'd hoped to be able to tackle the immediate ECN issue, the value
>>> of randomness in pie, and the effectiveness and need for ECN on the
>>> edge as it is currently defined.
>>>
>>> and I figured that then I might have written enough to get closer to an
>>> rfc.
>>>
>>> and when I started kibitzing on this thread I thought I was talking to
>>> the DCTCP case which I've spent a few months studying up on, and
>>> looking over alternative ideas there, like
>>>
>>> http://conferences.sigcomm.org/co-next/2013/program/p49.pdf
>>> http://conferences.sigcomm.org/co-next/2013/program/p151.pdf
>>>
>>> Scalable, Optimal Flow Routing in Datacenters via Local Link Balancing
>>>
>>> http://www.irt-systemx.fr/wp-content/uploads/2013/12/AINTEC.ppt
>>>
>>> Sigh. I'll
>>>
>>> On Thu, Dec 12, 2013 at 11:35 AM, Jim Gettys <jg@freedesktop.org> wrote=
:
>>> >
>>> >
>>> >
>>> > On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com> wro=
te:
>>> >>
>>> >> Jim,
>>> >>
>>> >>
>>> >> At 16:55 11/12/2013, Jim Gettys wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com>
>>> >> wrote:
>>> >> Jim,
>>> >>
>>> >> I'm just checking we're not talking past each other. I'll repeat two
>>> >> quotes from each of us, then comment.
>>> >>
>>> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wro=
te:
>>> >>
>>> >> 3{New}. It SHOULD be possible to make different instances of an AQM
>>> >> algorithm apply to different subsets of packets that share the same
>>> >> queue.
>>> >> It SHOULD be possible to classify packets into these subsets at leas=
t
>>> >> by ECN
>>> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equival=
ent
>>> >> of
>>> >> these fields at lower layers),
>>> >>
>>> >>
>>> >> At 19:50 05/12/2013, Jim Gettys wrote:
>>> >>
>>> >> "Certainly, it may be the same instance of an AQ
>>> >> M algorithm, rather than different instances, for example."
>>> >>
>>> >>
>>> >> That's true of course, but the case with one AQM handling all packet=
s
>>> >> within a queue is the norm. I want to check you're happy with the
>>> >> converse:
>>> >> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED
>>> >> with
>>> >> in and out of contract). So we can have WPIE, WCoDel etc where the
>>> >> differentiation between aggregates is provided by different AQM
>>> >> instances in
>>> >> the same queue, not by different queues with different scheduling
>>> >> priorities.
>>> >> 2) Extending this so that AQM differentiation can be between
>>> >> ECN-capable
>>> >> and Not-ECN-capable aggregates, not just between Diffserv classes (a=
n
>>> >> example being CoDel with a lower 'interval' for ECN-capable packets)=
.
>>> >>
>>> >> I presented the evaluations of this last idea in tsvwg on the final
>>> >> Friday
>>> >> of the Vancouver IETF - I don't think you were there. <
>>> >> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
>>> >>
>>> >>
>>> >> Yes, unfortunately I had to leave before the Friday session.
>>> >> This is my primary motivation for this wordsmithing - I'm trying all=
ow
>>> >> us
>>> >> to move towards zero signalling delays in CoDel, PIE and RED (curren=
tly
>>> >> defaults of 200ms, 100ms and 512packets respectively, which are not
>>> >> good for
>>> >> dynamics).
>>> >>
>>> >>
>>> >> Certainly signalling delays are very important: this is why I'm
>>> >> favorably
>>> >> inclined to "head mark/drop", as it signals TCP as quickly as possib=
le,
>>> >> keeping the response of the TCP feedback loop as tight as possible (=
and
>>> >> part
>>> >> of why I like CoDel so much for the highly variable bandwidth proble=
m
>>> >> we
>>> >> face at the edge of the net).
>>> >>
>>> >> It's *really* important than when the bandwidth drops suddenly that
>>> >> everyone gets told to slow down quickly (exactly how quickly probabl=
y
>>> >> depends on the propagation change characteristics of the medium), or
>>> >> packets
>>> >> can pile up in a big way.
>>> >>
>>> >> How quickly the mark/drop algorithm can figure out that signalling i=
s
>>> >> appropriate is the *other* piece of getting good dynamics.  Here I
>>> >> don't
>>> >> doubt that something may be discovered that is better than CoDel in =
the
>>> >> slightest.
>>> >> It takes a CoDel instance (within an fq structure) 200ms from its qu=
eue
>>> >> first passing 'threshold' before it will ever drop the first packet
>>> >> (unless
>>> >> the queue hits taildrop before that). So if the RTT is 20ms, that's
>>> >> 220ms
>>> >> signalling delay.
>>> >
>>> >
>>> > No, again, see Dave's mail, and you are missing the flow scheduling
>>> > aspect
>>> > of this and thinking in terms of a single queue and the usual mark/dr=
op
>>> > cases; this is the exact cognitive problem I'm talking about.
>>> >
>>> > The flow scheduling aspect of fq_codel is *more* important than what
>>> > mark/drop algorithm decides to signal the TCP's to regulate their ser=
vo
>>> > systems.  I really wish this algorithm had been called "fs_codel",
>>> > rather
>>> > than "fq_codel", as it is so easy to confuse "fair" with "flow", and
>>> > "queue"
>>> > with "scheduling".
>>> >
>>> > Regulating TCP is absolutely essential to keep TCP "sane" and
>>> > responsive,
>>> > but it isn't the most important part of what is going on.
>>> >
>>> > Here's the case of a new TCP flow on a previously idle link: After th=
e
>>> > TCP
>>> > open handshake, you will have no more than 4 or (if IW10 is active)
>>> > packets
>>> > for that flow in the queue (actually, 3 packets, as presumably at lea=
st
>>> > one
>>> > packet is in process of transmission; @ 1Mbps, that first full size
>>> > packet
>>> > takes 13ms).
>>> >
>>> > If another flow starts, it will get preferentially scheduled to the
>>> > existing
>>> > flow(s) until it has also built a queue, at which time it competes
>>> > "fairly"
>>> > with the other flows in the packet scheduling.
>>> > This means, in the simple low bandwidth case, that as soon as you sta=
rt
>>> > the
>>> > second flow, it gets the next possible opportunity to be transmitted =
in
>>> > preference to any flow that has built a queue.
>>> >
>>> > Here's the kicker:
>>> >
>>> > We've just avoid 3 packets worth of "head of line blocking" latency f=
or
>>> > the
>>> > second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved ju=
st
>>> > in
>>> > the TCP open, and more in that the first packet from the second flow
>>> > then
>>> > gets scheduled immediately, getting that TCP moving.
>>> >
>>> > Similarly for your voip packet; it saves those 40ms. And your bulk fl=
ow
>>> > gets
>>> > its first packet through ASAP as well; for web traffic, that usually
>>> > contains the size information or other metadata required for the web
>>> > browser
>>> > to unblock.  And that is for a *single* competing TCP connection.
>>> >
>>> > Now, let's look at today's web: there are many embedded objects in a
>>> > page.
>>> > Once the base page is downloaded, the web browser (presuming its DNS
>>> > lookups
>>> > are already cached), opens a pile of connections all at once.  Whethe=
r I
>>> > like it or not, (which I don't, as I tried to prevent it with pipelin=
ing
>>> > in
>>> > HTTP/1.1 in the 1990's), you can easily have 5-30 TCP connections sta=
rt
>>> > almost simultaneously. See the connections/page plot on my web page
>>> > explaining all this stuff in more detail at:
>>> >
>>> > http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-que=
uing-traditional-aqm-is-not-enough/
>>> >
>>> > Without any TCP flow control ever happening, I can easily get 40 (or
>>> > even
>>> > several hundred packets!) arriving at near line rate (the initial win=
dow
>>> > *
>>> > number of embedded objects).  Most of these TCP connections *won't ev=
er
>>> > even
>>> > get out of slow start*; the objects are typically pretty small.  I've
>>> > observed transient latency from HOL blocking of > 100ms on 50Mbps cab=
le
>>> > service, on some web pages.
>>> >
>>> > Nothing we will/can do at the TCP level can help this situation!!!
>>> > Nothing,
>>> > other than in the long term getting the incentives right so that fewe=
r
>>> > TCP
>>> > connections might be preferable to gaming TCP, as the web already has=
.
>>> >
>>> > That CoDel may take a while to figure out that it should start markin=
g
>>> > in
>>> > the idle case is really pretty irrelevant; the packets have already
>>> > arrived
>>> > and unless we do better packet scheduling, you are fully stuck.  Dave=
's
>>> > trying to also explain that a number of people's understanding of how
>>> > CoDel
>>> > works has been wrong.
>>> >
>>> > But as I say, *the details of the mark/drop algorithm doesn't matter*=
.
>>> > Having 3, or 100 packets queued in a FIFO queue will already mean you
>>> > are
>>> > screwed for anything low latency at the bandwidths we all care about.
>>> >
>>> > Once a flow drains to zero, it again gets treated as a new flow.
>>> > And what we choose to define a "flow" to be is arbitrary, though the
>>> > code
>>> > today does the usual 5tuple.
>>> >
>>> >> In fq_codel this creates considerable self-delay for short flows or =
r-t
>>> >> apps, which kill their own latency before they get any loss signal t=
o
>>> >> tell
>>> >> them to slow down.
>>> >
>>> >
>>> > Not likely in the common case.  See Dave's comments and also note tha=
t
>>> > the
>>> > initial burst of web page load runs the queue up high immediately.
>>> >
>>> > And as I keep saying, and will say again, the flow queuing decisions
>>> > avoiding HOL blocking explained are much more important to dealing wi=
th
>>> > the
>>> > latency problems.
>>> >
>>> > So go invent better algorithms that CoDel for drop/mark, that can be
>>> > applied
>>> > to the flow queuing parts of the algorithm, and I'm happy.  Running c=
ode
>>> > please.
>>> >
>>> >
>>> >
>>> >>
>>> >> Even for elastic flows, with congestion signals delayed by so much,
>>> >> they
>>> >> risk hitting themselves with a huge train of overshoot loss. This wo=
uld
>>> >> be
>>> >> the same for fq_pie, except the number is 100ms + RTT.
>>> >
>>> >
>>> > When fq_pie exists to test, I'll be happy to see.
>>> >
>>> > I keep saying, and I'll say again: while some mark/drop algorithm nee=
ds
>>> > to
>>> > exist, flow scheduling is more important to getting good latency....
>>> >
>>> >
>>> >>
>>> >>
>>> >> Yes, the e2e transport could measure delay growth, but it doesn't kn=
ow
>>> >> whether the delay is coming from a queue that is isolated from other=
s
>>> >> or
>>> >> not. So it doesn't want to slow down too quickly in response to dela=
y
>>> >> growth
>>> >> in case it gets screwed by other traffic. Ie. using delay growth as =
a
>>> >> signal
>>> >> entails considerable signalling delay due to all the uncertainty.
>>> >>
>>> >> The proposal you missed in tsvwg was to define ECN as an immediate
>>> >> signal
>>> >> from the network, 'interval'=3D0 in CoDel terms, so the host always =
gets
>>> >> congestion signals as fast as possible, and if it needs bursts of
>>> >> signals
>>> >> smoothed out, it can do that itself.
>>> >
>>> >
>>> > Yes, I get that.  I got that a year or more ago. The idea has potenti=
al
>>> > merit. ECN as it is currently defined is not so useful.
>>> >
>>> > And I wish it would get a different name so we could more easily talk
>>> > about
>>> > the two different things now being called ECN.
>>> >
>>> > I can see having ECN marks on a burst of packets may be helpful in
>>> > having
>>> > the receiver judge things in a highly variable wireless scenario; it =
may
>>> > have additional information about the medium and know that that
>>> > transient
>>> > has gone away, and that it may not be a wise idea to slow the connect=
ion
>>> > at
>>> > all.
>>> >
>>> >
>>> >>
>>> >> The suggested wording ensures all AQM implementations will allow
>>> >> operators, vendors and users to configure such a mechanism. But I've
>>> >> generalised it from ECN to Diffserv too (because the implementation
>>> >> would be
>>> >> no different).
>>> >
>>> >
>>> > As noted before, Diffserv is still interesting, even though the packe=
t
>>> > scheduling in fq_codel (or similar algorithms) makes it much less
>>> > necessary.
>>> > There are two aspects to this:
>>> >
>>> > 1) higher priority contention to the medium.  If I have a real VOIP
>>> > packet,
>>> > there are ways I can ask for higher priority access to the medium and
>>> > reduce
>>> > the total number of transmit opportunities my traffic requires (and t=
hat
>>> > is
>>> > often the scarcest resource on WiFi or Docsis).
>>> > 2) any hint I can get helps (at the edge) so I can distinguish those
>>> > packets
>>> > from the way the web has been gaming the network.
>>> >
>>> > Even so, for Diffserv to be safely trusted and honored even in your
>>> > home,
>>> > the end user (who is the network operator in this case), will have be
>>> > able
>>> > to know that a device or application is using it and control whether =
or
>>> > not
>>> > it's honored. Unless it is under the network operator's control (in t=
his
>>> > case, you the home user) Diffserv can/will get gamed to uselessness b=
y
>>> > application and device manufacturers. Ergo why Dave and I hack on hom=
e
>>> > routers again: this level of control is not currently present in thes=
e
>>> > devices, and must be for Diffserv to be useful.
>>> >
>>> >>
>>> >>
>>> >>
>>> >> My basic issue is one of terminology: people have talked about "best
>>> >> effort" queues.  In reality, this is a "class" of service, rather th=
an
>>> >> a
>>> >> single queue, and when you get into the mental model of BE being a
>>> >> single
>>> >> queue, (rather than a set of queues) it can lead one astray quickly =
and
>>> >> easily.
>>> >>
>>> >>
>>> >> Yeah, I know this. I suspected we were talking past each other.
>>> >>
>>> >> I need you to allow the other case into your mind for this
>>> >> conversation.
>>> >> The wording is specifically about the case where "different subsets =
of
>>> >> packets ... share the same queue".
>>> >
>>> >
>>> > And the word "queue" in most people's minds implies ordering, and FIF=
O
>>> > behavior.  This is the terminological issue I'm harping on.  It's als=
o
>>> > why I
>>> > think "bufferbloat" is a better term for our situation than "queue
>>> > bloat",
>>> > which you liked and have harped at me about.  Buffers don't have such=
 an
>>> > implication of ordering.
>>> > So if you talked about a buffer here, rather than a queue, I'd be a l=
ot
>>> > happier. At least in my mind, queues are ordered.
>>> >
>>> >>
>>> >> We can talk about an fq structure for this another time, but it's a
>>> >> really
>>> >> complicated way of doing it.
>>> >>
>>> >> Given simple looks like it could work, why get complicated already?
>>> >
>>> >
>>> > Because the flow scheduling is such a win.  You can't solve the whole
>>> > problem just with mark/drop algorithms and FIFO queues and get reliab=
ly
>>> > to
>>> > decent latencies.
>>> >
>>> > Now, whether Fred's document can/should go into anything like that th=
at
>>> > detail is far from clear (arguably not).
>>> >
>>> > I just don't want to further the mythology we can get to decent
>>> > latencies at
>>> > the edge of the network to continue with FIFO queues and an AQM
>>> > algorithm,
>>> > and it's clear that many haven't yet internalized that its flow
>>> > scheduling
>>> > combined with a self tuning adaptive mark/drop algorithm that we must
>>> > have,
>>> > but that the flow scheduling makes the biggest difference.
>>> >
>>> >>
>>> >>
>>> >> It's really easy to fall into the idea of a single software queue
>>> >> mapping
>>> >> to some single hardware supported queue, and that's a cognitive
>>> >> mistake, as
>>> >> aggregating MACs are showing us; transmit ops are often the scarcest
>>> >> resource...
>>> >>
>>> >>
>>> >> It's only a cognitive mistake if one is not aware of all the options=
.
>>> >> I'm
>>> >> fully aware of all the options.
>>> >>
>>> >> To be specific, a queue into a wireless medium should be configured =
so
>>> >> it
>>> >> holds some 'good queue' in reserve for transmit ops, but the queues =
on
>>> >> top
>>> >> of this that TCP self-inflicts even briefly are not 'good queues' ev=
en
>>> >> if
>>> >> they are isolated from other flows by fq - VJ was wrong to generalis=
e
>>> >> the
>>> >> phrase 'good queue' to all bursts of queue - it is only necessary to
>>> >> hold
>>> >> back from signalling and allow a burst of queue if the only possible
>>> >> signal
>>> >> is a drop. With ECN, you don't have this dilemma. This is the key to
>>> >> rapid
>>> >> dynamics.
>>> >
>>> >
>>> > But you also then have to solve the HOL blocking problem, and do so
>>> > urgently.  Ergo flow scheduling.... To get to where we need to go, yo=
u
>>> > have
>>> > to worry about the order in which each packet is scheduled.
>>> >
>>> >>
>>> >>
>>> >> Diffserv marking has the potential to give a "hint" to distinguish h=
ow
>>> >> particular flows should be handled (scheduled) in a service class, a=
nd
>>> >> as my
>>> >> previous example shows, that hint may be very useful in channel acce=
ss
>>> >> decisions (e.g. voip on 802.11).
>>> >
>>> >
>>> > ECN doesn't help a bit with the head of line blocking problem; it
>>> > actually
>>> > will make it worse with FIFO scheduling. ECN means that you can't eve=
n
>>> > get
>>> > the packets out of the way.
>>> >
>>> > Which says you'd better be doing more clever scheduling than a FIFO.
>>> >
>>> > If you want ECN to be usable in the way you want it to be at low
>>> > bandwidths,
>>> > you better become a fan of flow scheduling...
>>> >>
>>> >>
>>> >> But fq_codel teaches the lesson that packet scheduling combined with
>>> >> keeping TCP sane is a key improvement over handling either problem
>>> >> apart...
>>> >> In particular, the first packets of new flows/reappearing flows are
>>> >> vastly
>>> >> more "important" than other packets in terms of the latency cost to
>>> >> users of
>>> >> that service. Each flow has in essence its own queue in this service
>>> >> class,
>>> >> and we're using information from that to help schedule the packets i=
n
>>> >> ways
>>> >> that minimize latency to the user.
>>> >>
>>> >>
>>> >> I know all this. Please can we keep to the conversation about how to
>>> >> avoid
>>> >> the 200ms signalling delay that fq_codel inflicts on each flow (and =
the
>>> >> similar signalling delays that other AQMs inflict).
>>> >
>>> >
>>> > As I said, it's not the big problem we have today at the edge of the
>>> > network, and Dave's mail explains your model of CoDel isn't so correc=
t.
>>> >
>>> > It's only very long lived flows where the signalling even matters, an=
d
>>> > that's not what we get at the edge of today's network; instead, we ge=
t a
>>> > mix
>>> > of a few larger (e.g. video player flows) with the DOS attack that is
>>> > web
>>> > traffic, with some isochronous VOIP and teleconferencing traffic.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> So in this case, a single algorithm is acting over a bunch of flows =
in
>>> >> a
>>> >> single class of service, and both scheduling packets among the flows=
,
>>> >> and
>>> >> signalling TCP flows appropriately when they should "slow down".
>>> >>
>>> >>
>>> >> Yup, I know this.
>>> >>
>>> >>
>>> >>
>>> >> So I think you and I are on close to the same page (but have been
>>> >> burned
>>> >> badly in the past by terminology issues getting in the way).  On
>>> >> HTTP/1.1 we
>>> >> wasted probably > 2 years talking past each other because we didn't
>>> >> have
>>> >> clear and concise terminology that we all understood the same way.
>>> >>
>>> >>
>>> >> As I thought, we are talking past each other.
>>> >
>>> >
>>> > Yes, in part because I think few have internalized what the web has d=
one
>>> > to
>>> > edge of the network.  It isn't what I had hoped it would be when I wa=
s
>>> > HTTP/1.1 editor.  Why this outcome occurred is too long a discussion =
for
>>> > this thread.
>>> >
>>> >
>>> >> We need to be able to have a conversation that is not always "Hmm,
>>> >> that's
>>> >> sounds like it might be interesting. Can I tell you about fq_codel
>>> >> now?"
>>> >
>>> >
>>> > Running code of other algorithms very welcome. Fq_codel is running co=
de.
>>> > Pie is running code.
>>> >
>>> > Maybe fq_pie will be running code we can test someday.
>>> >
>>> > Even then, I want a low target latency so individual TCP's are kept
>>> > responsive, and I need an algorithm that can keep up quickly with the
>>> > dynamics of wireless, so most simplistic tests will not be useful (we
>>> > don't
>>> > have such good evaluation tests today).
>>> >
>>> > When it is, and if it is better than fq_codel, I'll be happy.  But th=
e
>>> > mark/drop part of the algorithm *isn't* the most important part of th=
e
>>> > algorithm. The packet scheduling decisions are....
>>> >                                - Jim
>>> >
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> Bob
>>> >>
>>> >>
>>> >>
>>> >> And I don't claim I have the right terminology for all this stuff,
>>> >> either
>>> >> (even in this mail).
>>> >>
>>> >> Which is why I was loathe to suggest exact text...
>>> >>                            - Jim
>>> >>
>>> >>
>>> >>
>>> >> At 19:50 05/12/2013, Jim Gettys wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com> wro=
te:
>>> >> Fred, Gorry, all,
>>> >> I promised to suggest text for draft-ietf-aqm-recommendation about
>>> >> allowing the AQM's behaviour to be independent for ECN and non-ECN
>>> >> packets.
>>> >> In the process, I realised we can't talk about independent AQMs for =
ECN
>>> >> without also including Diffserv.
>>> >> This gets messy, because I believe a good AQM for BE traffic with an=
d
>>> >> without ECN, should remove much if not all the need for Diffserv. Bu=
t
>>> >> we
>>> >> can't ignore Diffserv.
>>> >>
>>> >>
>>> >> I agree in principle with what Bob is trying to say here (and is ver=
y
>>> >> much
>>> >> what I've been saying in my blog entry of last summer).
>>> >>
>>> >> Once you have things under control, the need for Diffserv diminishes
>>> >> dramatically (but does not go away).
>>> >>
>>> >> But as Bob notes, there is still a good use for Diffserv: suitably
>>> >> marked
>>> >> traffic may want to contend for access to the channel differently: y=
our
>>> >> marked VOIP packets may want to change the priority with which you
>>> >> request
>>> >> channel access, so that you get more timely access to the medium. Th=
is
>>> >> conserves transmit opportunities, which is often the scarcest resour=
ce
>>> >> in
>>> >> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
>>> >> between
>>> >> your VOIP working well, and not working well, on a busy 802.11 netwo=
rk
>>> >> as
>>> >> well as using the channel as efficiently as possible.
>>> >>
>>> >> Similarly, if you have packets you know are background, it's helpful=
 to
>>> >> know that to ensure that they never contend for access to the medium
>>> >> but
>>> >> will always defer to other traffic, and just scavenge available spac=
e
>>> >> in
>>> >> other transmit opportunities where possible.
>>> >>
>>> >> I'm a bit loathe though to tie the behavior to queues, however; in
>>> >> particular, best effort traffic may want to be sent in the same
>>> >> aggregate as
>>> >> higher (or lower) priority traffic, if there is remaining space in t=
he
>>> >> aggregate.
>>> >>
>>> >> In short, the mental model we've had that there is a one-to-one mode=
l
>>> >> of
>>> >> hardware and software queues (not to mention flows in a given softwa=
re
>>> >> queue) is often incorrect (or at least seriously sub-optimal) in
>>> >> today's
>>> >> systems (even if the hardware queues "work" properly, which it appea=
rs
>>> >> they
>>> >> do not in 802.11).
>>> >>
>>> >> So I'm not sure Bob's new section 3 here is how to best to state thi=
s
>>> >> (or
>>> >> to deal with the terminology problem).  Certainly, it may be the sam=
e
>>> >> instance of an AQM algorithm, rather than different instances, for
>>> >> example.
>>> >> And "
>>> >> It SHOULD be possible" is more a pious wish than anything else.  But=
 I
>>> >> agree in spirit with what Bob's trying to say.
>>> >>                                - Jim
>>> >>
>>> >>
>>> >>
>>> >> ____________________________________________________________________=
_____________________
>>> >> {In Section 4: add another bullet between recommendations 2 & 3:}
>>> >> 3{New}. It SHOULD be possible to make different instances of an AQM
>>> >> algorithm apply to different subsets of packets that share the same
>>> >> queue.
>>> >> It SHOULD be possible to classify packets into these subsets at leas=
t
>>> >> by ECN
>>> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the equival=
ent
>>> >> of
>>> >> these fields at lower layers).
>>> >> {Then a new section to expand on this before the current Section 4.3=
.}
>>> >> 4.3{New}. Independent AQM Instances for ECN and Diffserv
>>> >> The recommendation to provide a separate instance of the AQM for ECN
>>> >> packets goes beyond the assumptions of RFC 3168, which assumed that
>>> >> only one
>>> >> instance of an AQM will handle both ECN-capable and non-ECN-capable
>>> >> packets.
>>> >>
>>> >>
>>> >> Bob
>>> >>
>>> >>
>>> >> ________________________________________________________________ Bob
>>> >> Briscoe,                                                  BT
>>> >> _______________________________________________ aqm mailing list
>>> >> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> aqm mailing list
>>> >> aqm@ietf.org
>>> >> https://www.ietf.org/mailman/listinfo/aqm
>>> >>
>>> >>
>>> >> ________________________________________________________________
>>> >> Bob Briscoe,                                                  BT
>>> >>
>>> >> _______________________________________________
>>> >> aqm mailing list
>>> >> aqm@ietf.org
>>> >> https://www.ietf.org/mailman/listinfo/aqm
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> aqm mailing list
>>> >> aqm@ietf.org
>>> >> https://www.ietf.org/mailman/listinfo/aqm
>>> >>
>>> >> ________________________________________________________________
>>> >> Bob Briscoe,                                                  BT
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> aqm mailing list
>>> >> aqm@ietf.org
>>> >> https://www.ietf.org/mailman/listinfo/aqm
>>> >>
>>> > I can imagine many other possible algorithms that CoDel for the
>>> > mark/drop
>>> > algorithm; we happen to like CoDel atm for two reasons: 1) it is self
>>> > adapting to the line rate, and 2) head mark/drop signals the TCP's as
>>> > soon
>>> > as a decision is made rather than possibly being applied much later
>>> > (such as
>>> > random or tail drop).  We welcome and encourage further essays in the
>>> > art.
>>> >
>>> >
>>> > _______________________________________________
>>> > aqm mailing list
>>> > aqm@ietf.org
>>> > https://www.ietf.org/mailman/listinfo/aqm
>>> >
>>>
>>>
>>>
>>> --
>>> Dave T=E4ht
>>>
>>> Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>
>>
>> ________________________________________________________________
>> Bob Briscoe,                                                  BT
>
>
>
> --
> Dave T=E4ht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscrib=
e.html



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From dhavey@yahoo.com  Fri Dec 13 01:29:58 2013
Return-Path: <dhavey@yahoo.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A20FC1AE08C for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 01:29:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.101
X-Spam-Level: 
X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4e6uBjEKM_1p for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 01:29:51 -0800 (PST)
Received: from nm20.bullet.mail.bf1.yahoo.com (nm20.bullet.mail.bf1.yahoo.com [98.139.212.179]) by ietfa.amsl.com (Postfix) with SMTP id A1BF61A802A for <aqm@ietf.org>; Fri, 13 Dec 2013 01:29:51 -0800 (PST)
Received: from [66.196.81.170] by nm20.bullet.mail.bf1.yahoo.com with NNFMP; 13 Dec 2013 09:29:45 -0000
Received: from [98.139.212.201] by tm16.bullet.mail.bf1.yahoo.com with NNFMP; 13 Dec 2013 09:29:45 -0000
Received: from [127.0.0.1] by omp1010.mail.bf1.yahoo.com with NNFMP; 13 Dec 2013 09:29:45 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 323098.73484.bm@omp1010.mail.bf1.yahoo.com
Received: (qmail 61230 invoked by uid 60001); 13 Dec 2013 09:29:45 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1386926985; bh=jQYMeaLvgw6e6pj5p2krLQ9YLaMvhy3cmimCOn9WXWM=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=v+pg4kgra2hcwWAA/Xs5AZcRc20LfyuQMpyrAaEqOBhtt1ON32a4kmN/Smf0tIrJ/7vgVuASPhS8xAdsuOFrNbwXGLG8hrLLzgR2Y+ZsbBz7c2TTeQQSRMEiXHt6kboIle2wEz4aJQtm6C3IiDBn6QYYovnmvTG5VZ2BVB9k17w=
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=mEMBuNn2cAO5AoZFJq1b01gOSM0HafkYRfx8wetR3lS+o1orMFvj0KHqkY23B0l2zQKb/QTaRdV/mRdMaJr8R346CCiK0hiNwh1ChgqaYuDnbYIQCQGCjLnYgpaHCudkrfnYuEu9Kz1kjUnaZ0faP+v+fs/L67xp8kDMrXHB5ME=;
X-YMail-OSG: k586PcYVM1kbshgjm3BFKY0BEwRTHMMB9FdQ0jSQHOBdQy8 2m4zmWNMEG02CFRjhWGgOiLrFbo3KRj0ulFAfZ5YlQVap.t8Hb5qAMz3vmfu kJlIr15WO35dZilJVDNS2SW2rugz.CG.sIubSi4Ds0m2Dp.WUsH_RxEvjcmp Hzz55fmPeJkPCfm5GUzqhmsNyuk_4ZqiL0DmTzZfyvyvXFJb..sepSXviEB1 sRCgQLH6nYpSp0LYX5VsnMqjZKV4uDSHkAaN0WANEcgvhs56LHiPbZvGqFvp 69xZoLSua9SY3pmMpHJrBiAyvKGYugvR_ygwDjiZQhX6k.A2xbsS.QFtn9g6 3z4IU1xtIXdAzzd_x0Cu6lsiERMJOMTbeHZLrSEkNwcdSKtZ51d4bu_1JuON Ix3z6NA_tWrFCjgxJlX2NL0ItVyOcT.mBXAZqbIOUCRTUq5bl7DejD3tNe8C IRYXyRMzT6kSZJB_X_dRpyBc8RIvrJ5aoD89N6SPMFl.VNCe2IA_djO9QABS nsscOIetCVDBHGWL8ypihRhJklpX.WjlfEgbMTFAQh37wdhpxIdfhLGGD.DH iE.HtFu_U
Received: from [98.182.24.153] by web141605.mail.bf1.yahoo.com via HTTP; Fri, 13 Dec 2013 01:29:45 PST
X-Rocket-MIMEInfo: 002.001, Cgo.IEEgcmF0ZSBsaW1pdGVyIGlzIHJlcXVpcmVkIHRvIGNvbXBlbnNhdGUgZm9yIElTUCdzIGxvdXN5Cj4gZHNsYW0vY210cy9ncG9uIGhlYWQgZW5kIGFuZCBDUEUgYXQgbGVhc3QgdW50aWwgdGhpcyBjb2RlIG1ha2VzIGl0Cj4gb250byB0aG9zZSBkZXZpY2VzLiBMb25nIGxlYWQgdGltZXMgcHJlZG9taW5hdGUgb24gdGhpcyBzb3J0IG9mCj4gaGFyZHdhcmUgLSBXZSBoYXZlIHRocmVlIHllYXJzIHRvIGdldCBET0NTSVMgMy4xIHJpZ2h0LCBhcyBvbmUKPiBleGFtcGxlLgo.ClllYWgsIHRoaXMgc3RpbmsBMAEBAQE-
X-Mailer: YahooMailWebService/0.8.169.609
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com> <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com> <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk> <CAA93jw7UrTzUrMAkaQOr1dQoGjWD2ktLAn-Uv-oA6x88HpYLfA@mail.gmail.com> <CAA93jw71X83U_-XNjzhX3+LyL-=xZr3Omi1Fhq6PcEvxZt1tKQ@mail.gmail.com>
Message-ID: <1386926985.76408.YahooMailNeo@web141605.mail.bf1.yahoo.com>
Date: Fri, 13 Dec 2013 01:29:45 -0800 (PST)
From: Daniel Havey <dhavey@yahoo.com>
To: Dave Taht <dave.taht@gmail.com>, Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAA93jw71X83U_-XNjzhX3+LyL-=xZr3Omi1Fhq6PcEvxZt1tKQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Fred Baker <fred@cisco.com>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: Daniel Havey <dhavey@yahoo.com>
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Dec 2013 09:29:58 -0000

=0A=0A> A rate limiter is required to compensate for ISP's lousy=0A> dslam/=
cmts/gpon head end and CPE at least until this code makes it=0A> onto those=
 devices. Long lead times predominate on this sort of=0A> hardware - We hav=
e three years to get DOCSIS 3.1 right, as one=0A> example.=0A>=0AYeah, this=
 stinks. =A0If I rate limit then I will lose the ability to borrow tokens f=
rom my neighbors when they are not using them.=0A=0A=0A...Daniel

From bob.briscoe@bt.com  Fri Dec 13 03:24:28 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE70E1AE2E8 for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 03:24:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.602
X-Spam-Level: 
X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kxyu-irvGC3m for <aqm@ietfa.amsl.com>; Fri, 13 Dec 2013 03:24:22 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 9CA5D1AE2D8 for <aqm@ietf.org>; Fri, 13 Dec 2013 03:24:20 -0800 (PST)
Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR68-UKRD.bt.com (10.187.101.23) with Microsoft SMTP Server (TLS) id 8.3.279.1; Fri, 13 Dec 2013 11:24:13 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.279.1; Fri, 13 Dec 2013 11:24:13 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Fri, 13 Dec 2013 11:24:05 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.121.100])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rBDBO0Is007562;	Fri, 13 Dec 2013 11:24:01 GMT
Message-ID: <201312131124.rBDBO0Is007562@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Fri, 13 Dec 2013 11:23:56 +0000
To: Dave Taht <dave.taht@gmail.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAA93jw6FB6wUxwTXKHic5PJ+fo35ZfAKRsVa+xAxE=2U+TCNNQ@mail.g mail.com>
References: <201312051813.rB5ID6rp004006@bagheera.jungle.bt.co.uk> <CAGhGL2ACj9rK4JossovPQyq_aMx9kNFc2BXA9BUfj_NyMBdQBA@mail.gmail.com> <201312110304.rBB34Tbg027256@bagheera.jungle.bt.co.uk> <CAGhGL2DBP-EHiRGEShb_=87p4xD2rzrZdGYT3PAmWgGvrZj2_A@mail.gmail.com> <201312111921.rBBJLGTM000905@bagheera.jungle.bt.co.uk> <CAGhGL2D30-uhGZT+suZOjNxtkdGF4O0SUxAvsNS=Ho-3c=H-tg@mail.gmail.com> <CAA93jw5gwtt+cE8f9WJzLyk5JMT_5fYEcdtM6cx1UTV32K0VPA@mail.gmail.com> <201312130005.rBD05LFx005567@bagheera.jungle.bt.co.uk> <CAA93jw6FB6wUxwTXKHic5PJ+fo35ZfAKRsVa+xAxE=2U+TCNNQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, AQM IETF list <aqm@ietf.org>, Jim Gettys <jg@freedesktop.org>, Fred Baker <fred@cisco.com>
Subject: Re: [aqm] Text for aqm-recommendation on independent ECN config
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Dec 2013 11:24:29 -0000

Dave,

I'm not arguing that we shouldn't be trying to=20
improve things. I was reacting to your response=20
that we only need to selectively improve the=20
things that your solution improves.

You also seem to have a selective notion of "what=20
exists today" that excludes reconfig of stuff=20
that's already deployed. Just because you're an=20
implementer, doesn't mean the only valid=20
solutions have to involve implementation. One of=20
the buses that I'm on had already left your station.

However, I'm sure many implementations (of RED)=20
exist that cannot be config'd to do immediate=20
ECN, or they will need firmware update but not=20
hardware etc. And RED doesn't have the=20
auto-config for line-rate. So I'm happy that=20
other buses are leaving the station.

At this stage, all I'm asking of you is to at=20
least make sure that the new implementation you=20
are putting out there can at least be config'd to=20
do immediate ECN. That requires:
* separation of variables into policy config=20
parameters and variables monitoring the queue.
* Then ensuring that multiple instances of the=20
algo can be applied to the same queue, such that=20
they have different policy config parameters, but=20
share the same queue monitoring variables.

________
Sorry, but I'm going to come back to your 2 other=20
posts I see in my inbox - I have 3 major pieces=20
of work with deadlines for this weekend. It=20
sounds like you need some time to work too.



Bob

At 00:33 13/12/2013, Dave Taht wrote:
>  bob, I have had 3% of my braincells free for this conversation for
>the last week. I get twitchy whenever I see averages on traffic that
>has microbursts - I pointed to one of the papers presented earlier
>this week in my last email.
>
>There is a great deal of wonderful work going on to fix the microburst
>problems in the ongoing paced tcp work. I'm very encouraged, and also
>agree that it is possible to optimize away much of slow start possibly
>along the same way you are thinking,
>
>I had hoped that by establishing what exists today, and describing it
>properly, and getting everyone on the same page on what exists=85
>
>I could finally get to where I could start to make sense on the ecn
>and immediate ecn ideas to work towards reducing congestion still
>further in the data center, and along the edge.
>
>and these days, I'm mostly working on wifi. If you can make red work
>on wifi, I would love to know how. My take on this list was that we
>were trying to standardize on some aqm technologies that exist and are
>being deployed today. Bus is leaving the station.
>
>On Thu, Dec 12, 2013 at 4:05 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
> > Dave,
> >
> >
> > At 22:11 12/12/2013, Dave Taht wrote:
> >>
> >> but quickly...
> >>
> >> Bob, I object to your characterization of users links being busy 1-3%
> >> of the time. That's an average.
> >
> >
> > I said it was an average. You're repeating and agreeing with what I=
 said,
> > but saying you object to me saying it?
> >
> >
> >> When they are busy, they are very busy
> >> for short periods, typically 2-16 seconds in the case of web traffic,
> >> then idle for minutes. DASH traffic is busy for 2+ seconds every 10 on
> >> a 20mbit link, and so on, for 1.5 hours or so. Etc.
> >
> >
> > Yes, again, you're agreeing with me.
> >
> > The mean for a Web session is towards the low end of the 2-16 seconds=
 range
> > even now. And as we get the other latency-saving advances out there=
 (e.g.
> > removing TCP & TLS handshakes, proper pipelining, and a faster=
 replacement
> > for slow-start without overshoot), then there is potential for Web=
 sessions
> > to drop to 1-2 seconds or less, because they're usually a long way from
> > being bandwidth limited.
> >
> > You said fq_codel decays out its memory in 800ms. So it will typically=
 have
> > lost all its memory when each Web transfer starts and when each DASH
> > transfer re-starts.
> >
> > So I think you're agreeing that this 200ms signalling delay will be
> > predominant?
> >
> >
> >> In both cases
> >> congestion exists, and in both cases AQM reaction time measured in
> >> 200ms or so is still vastly superior to what happens today, and packet
> >> scheduling masks it to a large extent.
> >
> >
> > You're saying it's OK to propose a solution that delays signalling
> > congestion for about 10 typical CDN RTTs,... because it's better than
> > nothing?
> >
> > That's rich. I have to say this... You're one of the group of=
 protagonists
> > who has persuaded the world to embark on a programme of /implementation/
> > updates that will take years, and rubbished deploying the AQM that was
> > already implemented (RED), even tho it was already much better than=
 nothing
> > too.
> >
> > Yes, auto-config for line rate is a nice bell to add to the bicycle. If=
 we
> > are going to embark on new implementations, auto-config for RTT is no=
 less
> > important.
> >
> > Lack of auto-config for either requires the fixed config to be at the=
 slow
> > end of the range. And both have a similar range of variability (the RTT
> > range is actually wider). So lack of auto-config in either case leads to=
 a
> > similar order of unnecessary delay. Particularly given traffic is
> > predominantly in short sparse bursts.
> >
> >
> >
> > Bob
> >
> >
> >
> >
> >> Jim, yes, I was trying to establish the groundwork for ensuring
> >> everyone really understood codel-by-itself before talking about
> >> fq_codel. I'm still not sure that's been established. Anyone here care
> >> to calculate the number of drops on two flows going through codel and
> >> fq_codel, starting with iw10, over a 10mbit 2ms RTT link? And when,
> >> approximately, the queue becomes ideal? And how often the fq_codel
> >> "fast queue" gets used in this case? Gold stars for everyone that gets
> >> it right.
> >>
> >> Jim, I'd like you to use larger download speeds than a mbit for your
> >> examples. somewhere between 8 and 20mbit seems appropriate. (IMHO iw10
> >> should not be used on the modern internet on sub 10Mbit links.) The
> >> dynamics change significantly as you get more bandwidth than iw10
> >> abuses.
> >>
> >> Anyway=85
> >>
> >> After I got as far as describing fq_codel accurately in this thread,
> >> then I'd hoped to be able to tackle the immediate ECN issue, the value
> >> of randomness in pie, and the effectiveness and need for ECN on the
> >> edge as it is currently defined.
> >>
> >> and I figured that then I might have written enough to get closer to an
> >> rfc.
> >>
> >> and when I started kibitzing on this thread I thought I was talking to
> >> the DCTCP case which I've spent a few months studying up on, and
> >> looking over alternative ideas there, like
> >>
> >> http://conferences.sigcomm.org/co-next/2013/program/p49.pdf
> >> http://conferences.sigcomm.org/co-next/2013/program/p151.pdf
> >>
> >> Scalable, Optimal Flow Routing in Datacenters via Local Link Balancing
> >>
> >> http://www.irt-systemx.fr/wp-content/uploads/2013/12/AINTEC.ppt
> >>
> >> Sigh. I'll
> >>
> >> On Thu, Dec 12, 2013 at 11:35 AM, Jim Gettys <jg@freedesktop.org>=
 wrote:
> >> >
> >> >
> >> >
> >> > On Wed, Dec 11, 2013 at 2:21 PM, Bob Briscoe <bob.briscoe@bt.com>=
 wrote:
> >> >>
> >> >> Jim,
> >> >>
> >> >>
> >> >> At 16:55 11/12/2013, Jim Gettys wrote:
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Dec 10, 2013 at 10:04 PM, Bob Briscoe <bob.briscoe@bt.com>
> >> >> wrote:
> >> >> Jim,
> >> >>
> >> >> I'm just checking we're not talking past each other. I'll repeat two
> >> >> quotes from each of us, then comment.
> >> >>
> >> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com>=
 wrote:
> >> >>
> >> >> 3{New}. It SHOULD be possible to make different instances of an AQM
> >> >> algorithm apply to different subsets of packets that share the same
> >> >> queue.
> >> >> It SHOULD be possible to classify packets into these subsets at=
 least
> >> >> by ECN
> >> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the=
 equivalent
> >> >> of
> >> >> these fields at lower layers),
> >> >>
> >> >>
> >> >> At 19:50 05/12/2013, Jim Gettys wrote:
> >> >>
> >> >> "Certainly, it may be the same instance of an AQ
> >> >> M algorithm, rather than different instances, for example."
> >> >>
> >> >>
> >> >> That's true of course, but the case with one AQM handling all=
 packets
> >> >> within a queue is the norm. I want to check you're happy with the
> >> >> converse:
> >> >> 1) A set-up more like WRED which was based on Dave Clark's RIO (RED
> >> >> with
> >> >> in and out of contract). So we can have WPIE, WCoDel etc where the
> >> >> differentiation between aggregates is provided by different AQM
> >> >> instances in
> >> >> the same queue, not by different queues with different scheduling
> >> >> priorities.
> >> >> 2) Extending this so that AQM differentiation can be between
> >> >> ECN-capable
> >> >> and Not-ECN-capable aggregates, not just between Diffserv classes=
 (an
> >> >> example being CoDel with a lower 'interval' for ECN-capable=
 packets).
> >> >>
> >> >> I presented the evaluations of this last idea in tsvwg on the final
> >> >> Friday
> >> >> of the Vancouver IETF - I don't think you were there. <
> >> >> http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf >
> >> >>
> >> >>
> >> >> Yes, unfortunately I had to leave before the Friday session.
> >> >> This is my primary motivation for this wordsmithing - I'm trying=
 allow
> >> >> us
> >> >> to move towards zero signalling delays in CoDel, PIE and RED=
 (currently
> >> >> defaults of 200ms, 100ms and 512packets respectively, which are not
> >> >> good for
> >> >> dynamics).
> >> >>
> >> >>
> >> >> Certainly signalling delays are very important: this is why I'm
> >> >> favorably
> >> >> inclined to "head mark/drop", as it signals TCP as quickly as=
 possible,
> >> >> keeping the response of the TCP feedback loop as tight as possible=
 (and
> >> >> part
> >> >> of why I like CoDel so much for the highly variable bandwidth=
 problem
> >> >> we
> >> >> face at the edge of the net).
> >> >>
> >> >> It's *really* important than when the bandwidth drops suddenly that
> >> >> everyone gets told to slow down quickly (exactly how quickly=
 probably
> >> >> depends on the propagation change characteristics of the medium), or
> >> >> packets
> >> >> can pile up in a big way.
> >> >>
> >> >> How quickly the mark/drop algorithm can figure out that signalling=
 is
> >> >> appropriate is the *other* piece of getting good dynamics.  Here I
> >> >> don't
> >> >> doubt that something may be discovered that is better than CoDel in=
 the
> >> >> slightest.
> >> >> It takes a CoDel instance (within an fq structure) 200ms from its=
 queue
> >> >> first passing 'threshold' before it will ever drop the first packet
> >> >> (unless
> >> >> the queue hits taildrop before that). So if the RTT is 20ms, that's
> >> >> 220ms
> >> >> signalling delay.
> >> >
> >> >
> >> > No, again, see Dave's mail, and you are missing the flow scheduling
> >> > aspect
> >> > of this and thinking in terms of a single queue and the usual=
 mark/drop
> >> > cases; this is the exact cognitive problem I'm talking about.
> >> >
> >> > The flow scheduling aspect of fq_codel is *more* important than what
> >> > mark/drop algorithm decides to signal the TCP's to regulate their=
 servo
> >> > systems.  I really wish this algorithm had been called "fs_codel",
> >> > rather
> >> > than "fq_codel", as it is so easy to confuse "fair" with "flow", and
> >> > "queue"
> >> > with "scheduling".
> >> >
> >> > Regulating TCP is absolutely essential to keep TCP "sane" and
> >> > responsive,
> >> > but it isn't the most important part of what is going on.
> >> >
> >> > Here's the case of a new TCP flow on a previously idle link: After=
 the
> >> > TCP
> >> > open handshake, you will have no more than 4 or (if IW10 is active)
> >> > packets
> >> > for that flow in the queue (actually, 3 packets, as presumably at=
 least
> >> > one
> >> > packet is in process of transmission; @ 1Mbps, that first full size
> >> > packet
> >> > takes 13ms).
> >> >
> >> > If another flow starts, it will get preferentially scheduled to the
> >> > existing
> >> > flow(s) until it has also built a queue, at which time it competes
> >> > "fairly"
> >> > with the other flows in the packet scheduling.
> >> > This means, in the simple low bandwidth case, that as soon as you=
 start
> >> > the
> >> > second flow, it gets the next possible opportunity to be transmitted=
 in
> >> > preference to any flow that has built a queue.
> >> >
> >> > Here's the kicker:
> >> >
> >> > We've just avoid 3 packets worth of "head of line blocking" latency=
 for
> >> > the
> >> > second flow to "do its thing".  @1Mbs, that is 40ms (3*13ms) saved=
 just
> >> > in
> >> > the TCP open, and more in that the first packet from the second flow
> >> > then
> >> > gets scheduled immediately, getting that TCP moving.
> >> >
> >> > Similarly for your voip packet; it saves those 40ms. And your bulk=
 flow
> >> > gets
> >> > its first packet through ASAP as well; for web traffic, that usually
> >> > contains the size information or other metadata required for the web
> >> > browser
> >> > to unblock.  And that is for a *single* competing TCP connection.
> >> >
> >> > Now, let's look at today's web: there are many embedded objects in a
> >> > page.
> >> > Once the base page is downloaded, the web browser (presuming its DNS
> >> > lookups
> >> > are already cached), opens a pile of connections all at once. =
 Whether I
> >> > like it or not, (which I don't, as I tried to prevent it with=
 pipelining
> >> > in
> >> > HTTP/1.1 in the 1990's), you can easily have 5-30 TCP connections=
 start
> >> > almost simultaneously. See the connections/page plot on my web page
> >> > explaining all this stuff in more detail at:
> >> >
> >> >=20
>=
 http://gettys.wordpress.com/2013/07/10/low-latency-requires-smart-queuing-t=
raditional-aqm-is-not-enough/
> >> >
> >> > Without any TCP flow control ever happening, I can easily get 40 (or
> >> > even
> >> > several hundred packets!) arriving at near line rate (the initial=
 window
> >> > *
> >> > number of embedded objects).  Most of these TCP connections *won't=
 ever
> >> > even
> >> > get out of slow start*; the objects are typically pretty small.  I've
> >> > observed transient latency from HOL blocking of > 100ms on 50Mbps=
 cable
> >> > service, on some web pages.
> >> >
> >> > Nothing we will/can do at the TCP level can help this situation!!!
> >> > Nothing,
> >> > other than in the long term getting the incentives right so that=
 fewer
> >> > TCP
> >> > connections might be preferable to gaming TCP, as the web already=
 has.
> >> >
> >> > That CoDel may take a while to figure out that it should start=
 marking
> >> > in
> >> > the idle case is really pretty irrelevant; the packets have already
> >> > arrived
> >> > and unless we do better packet scheduling, you are fully stuck. =
 Dave's
> >> > trying to also explain that a number of people's understanding of how
> >> > CoDel
> >> > works has been wrong.
> >> >
> >> > But as I say, *the details of the mark/drop algorithm doesn't=
 matter*.
> >> > Having 3, or 100 packets queued in a FIFO queue will already mean you
> >> > are
> >> > screwed for anything low latency at the bandwidths we all care about.
> >> >
> >> > Once a flow drains to zero, it again gets treated as a new flow.
> >> > And what we choose to define a "flow" to be is arbitrary, though the
> >> > code
> >> > today does the usual 5tuple.
> >> >
> >> >> In fq_codel this creates considerable self-delay for short flows or=
 r-t
> >> >> apps, which kill their own latency before they get any loss signal=
 to
> >> >> tell
> >> >> them to slow down.
> >> >
> >> >
> >> > Not likely in the common case.  See Dave's comments and also note=
 that
> >> > the
> >> > initial burst of web page load runs the queue up high immediately.
> >> >
> >> > And as I keep saying, and will say again, the flow queuing decisions
> >> > avoiding HOL blocking explained are much more important to dealing=
 with
> >> > the
> >> > latency problems.
> >> >
> >> > So go invent better algorithms that CoDel for drop/mark, that can be
> >> > applied
> >> > to the flow queuing parts of the algorithm, and I'm happy.  Running=
 code
> >> > please.
> >> >
> >> >
> >> >
> >> >>
> >> >> Even for elastic flows, with congestion signals delayed by so much,
> >> >> they
> >> >> risk hitting themselves with a huge train of overshoot loss. This=
 would
> >> >> be
> >> >> the same for fq_pie, except the number is 100ms + RTT.
> >> >
> >> >
> >> > When fq_pie exists to test, I'll be happy to see.
> >> >
> >> > I keep saying, and I'll say again: while some mark/drop algorithm=
 needs
> >> > to
> >> > exist, flow scheduling is more important to getting good latency....
> >> >
> >> >
> >> >>
> >> >>
> >> >> Yes, the e2e transport could measure delay growth, but it doesn't=
 know
> >> >> whether the delay is coming from a queue that is isolated from=
 others
> >> >> or
> >> >> not. So it doesn't want to slow down too quickly in response to=
 delay
> >> >> growth
> >> >> in case it gets screwed by other traffic. Ie. using delay growth as=
 a
> >> >> signal
> >> >> entails considerable signalling delay due to all the uncertainty.
> >> >>
> >> >> The proposal you missed in tsvwg was to define ECN as an immediate
> >> >> signal
> >> >> from the network, 'interval'=3D0 in CoDel terms, so the host always=
 gets
> >> >> congestion signals as fast as possible, and if it needs bursts of
> >> >> signals
> >> >> smoothed out, it can do that itself.
> >> >
> >> >
> >> > Yes, I get that.  I got that a year or more ago. The idea has=
 potential
> >> > merit. ECN as it is currently defined is not so useful.
> >> >
> >> > And I wish it would get a different name so we could more easily talk
> >> > about
> >> > the two different things now being called ECN.
> >> >
> >> > I can see having ECN marks on a burst of packets may be helpful in
> >> > having
> >> > the receiver judge things in a highly variable wireless scenario; it=
 may
> >> > have additional information about the medium and know that that
> >> > transient
> >> > has gone away, and that it may not be a wise idea to slow the=
 connection
> >> > at
> >> > all.
> >> >
> >> >
> >> >>
> >> >> The suggested wording ensures all AQM implementations will allow
> >> >> operators, vendors and users to configure such a mechanism. But I've
> >> >> generalised it from ECN to Diffserv too (because the implementation
> >> >> would be
> >> >> no different).
> >> >
> >> >
> >> > As noted before, Diffserv is still interesting, even though the=
 packet
> >> > scheduling in fq_codel (or similar algorithms) makes it much less
> >> > necessary.
> >> > There are two aspects to this:
> >> >
> >> > 1) higher priority contention to the medium.  If I have a real VOIP
> >> > packet,
> >> > there are ways I can ask for higher priority access to the medium and
> >> > reduce
> >> > the total number of transmit opportunities my traffic requires (and=
 that
> >> > is
> >> > often the scarcest resource on WiFi or Docsis).
> >> > 2) any hint I can get helps (at the edge) so I can distinguish those
> >> > packets
> >> > from the way the web has been gaming the network.
> >> >
> >> > Even so, for Diffserv to be safely trusted and honored even in your
> >> > home,
> >> > the end user (who is the network operator in this case), will have be
> >> > able
> >> > to know that a device or application is using it and control whether=
 or
> >> > not
> >> > it's honored. Unless it is under the network operator's control (in=
 this
> >> > case, you the home user) Diffserv can/will get gamed to uselessness=
 by
> >> > application and device manufacturers. Ergo why Dave and I hack on=
 home
> >> > routers again: this level of control is not currently present in=
 these
> >> > devices, and must be for Diffserv to be useful.
> >> >
> >> >>
> >> >>
> >> >>
> >> >> My basic issue is one of terminology: people have talked about "best
> >> >> effort" queues.  In reality, this is a "class" of service, rather=
 than
> >> >> a
> >> >> single queue, and when you get into the mental model of BE being a
> >> >> single
> >> >> queue, (rather than a set of queues) it can lead one astray quickly=
 and
> >> >> easily.
> >> >>
> >> >>
> >> >> Yeah, I know this. I suspected we were talking past each other.
> >> >>
> >> >> I need you to allow the other case into your mind for this
> >> >> conversation.
> >> >> The wording is specifically about the case where "different subsets=
 of
> >> >> packets ... share the same queue".
> >> >
> >> >
> >> > And the word "queue" in most people's minds implies ordering, and=
 FIFO
> >> > behavior.  This is the terminological issue I'm harping on.  It's=
 also
> >> > why I
> >> > think "bufferbloat" is a better term for our situation than "queue
> >> > bloat",
> >> > which you liked and have harped at me about.  Buffers don't have such=
 an
> >> > implication of ordering.
> >> > So if you talked about a buffer here, rather than a queue, I'd be a=
 lot
> >> > happier. At least in my mind, queues are ordered.
> >> >
> >> >>
> >> >> We can talk about an fq structure for this another time, but it's a
> >> >> really
> >> >> complicated way of doing it.
> >> >>
> >> >> Given simple looks like it could work, why get complicated already?
> >> >
> >> >
> >> > Because the flow scheduling is such a win.  You can't solve the whole
> >> > problem just with mark/drop algorithms and FIFO queues and get=
 reliably
> >> > to
> >> > decent latencies.
> >> >
> >> > Now, whether Fred's document can/should go into anything like that=
 that
> >> > detail is far from clear (arguably not).
> >> >
> >> > I just don't want to further the mythology we can get to decent
> >> > latencies at
> >> > the edge of the network to continue with FIFO queues and an AQM
> >> > algorithm,
> >> > and it's clear that many haven't yet internalized that its flow
> >> > scheduling
> >> > combined with a self tuning adaptive mark/drop algorithm that we must
> >> > have,
> >> > but that the flow scheduling makes the biggest difference.
> >> >
> >> >>
> >> >>
> >> >> It's really easy to fall into the idea of a single software queue
> >> >> mapping
> >> >> to some single hardware supported queue, and that's a cognitive
> >> >> mistake, as
> >> >> aggregating MACs are showing us; transmit ops are often the scarcest
> >> >> resource...
> >> >>
> >> >>
> >> >> It's only a cognitive mistake if one is not aware of all the=
 options.
> >> >> I'm
> >> >> fully aware of all the options.
> >> >>
> >> >> To be specific, a queue into a wireless medium should be configured=
 so
> >> >> it
> >> >> holds some 'good queue' in reserve for transmit ops, but the queues=
 on
> >> >> top
> >> >> of this that TCP self-inflicts even briefly are not 'good queues'=
 even
> >> >> if
> >> >> they are isolated from other flows by fq - VJ was wrong to=
 generalise
> >> >> the
> >> >> phrase 'good queue' to all bursts of queue - it is only necessary to
> >> >> hold
> >> >> back from signalling and allow a burst of queue if the only possible
> >> >> signal
> >> >> is a drop. With ECN, you don't have this dilemma. This is the key to
> >> >> rapid
> >> >> dynamics.
> >> >
> >> >
> >> > But you also then have to solve the HOL blocking problem, and do so
> >> > urgently.  Ergo flow scheduling.... To get to where we need to go,=
 you
> >> > have
> >> > to worry about the order in which each packet is scheduled.
> >> >
> >> >>
> >> >>
> >> >> Diffserv marking has the potential to give a "hint" to distinguish=
 how
> >> >> particular flows should be handled (scheduled) in a service class,=
 and
> >> >> as my
> >> >> previous example shows, that hint may be very useful in channel=
 access
> >> >> decisions (e.g. voip on 802.11).
> >> >
> >> >
> >> > ECN doesn't help a bit with the head of line blocking problem; it
> >> > actually
> >> > will make it worse with FIFO scheduling. ECN means that you can't=
 even
> >> > get
> >> > the packets out of the way.
> >> >
> >> > Which says you'd better be doing more clever scheduling than a FIFO.
> >> >
> >> > If you want ECN to be usable in the way you want it to be at low
> >> > bandwidths,
> >> > you better become a fan of flow scheduling...
> >> >>
> >> >>
> >> >> But fq_codel teaches the lesson that packet scheduling combined with
> >> >> keeping TCP sane is a key improvement over handling either problem
> >> >> apart...
> >> >> In particular, the first packets of new flows/reappearing flows are
> >> >> vastly
> >> >> more "important" than other packets in terms of the latency cost to
> >> >> users of
> >> >> that service. Each flow has in essence its own queue in this service
> >> >> class,
> >> >> and we're using information from that to help schedule the packets=
 in
> >> >> ways
> >> >> that minimize latency to the user.
> >> >>
> >> >>
> >> >> I know all this. Please can we keep to the conversation about how to
> >> >> avoid
> >> >> the 200ms signalling delay that fq_codel inflicts on each flow (and=
 the
> >> >> similar signalling delays that other AQMs inflict).
> >> >
> >> >
> >> > As I said, it's not the big problem we have today at the edge of the
> >> > network, and Dave's mail explains your model of CoDel isn't so=
 correct.
> >> >
> >> > It's only very long lived flows where the signalling even matters,=
 and
> >> > that's not what we get at the edge of today's network; instead, we=
 get a
> >> > mix
> >> > of a few larger (e.g. video player flows) with the DOS attack that is
> >> > web
> >> > traffic, with some isochronous VOIP and teleconferencing traffic.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> So in this case, a single algorithm is acting over a bunch of flows=
 in
> >> >> a
> >> >> single class of service, and both scheduling packets among the=
 flows,
> >> >> and
> >> >> signalling TCP flows appropriately when they should "slow down".
> >> >>
> >> >>
> >> >> Yup, I know this.
> >> >>
> >> >>
> >> >>
> >> >> So I think you and I are on close to the same page (but have been
> >> >> burned
> >> >> badly in the past by terminology issues getting in the way).  On
> >> >> HTTP/1.1 we
> >> >> wasted probably > 2 years talking past each other because we didn't
> >> >> have
> >> >> clear and concise terminology that we all understood the same way.
> >> >>
> >> >>
> >> >> As I thought, we are talking past each other.
> >> >
> >> >
> >> > Yes, in part because I think few have internalized what the web has=
 done
> >> > to
> >> > edge of the network.  It isn't what I had hoped it would be when I=
 was
> >> > HTTP/1.1 editor.  Why this outcome occurred is too long a discussion=
 for
> >> > this thread.
> >> >
> >> >
> >> >> We need to be able to have a conversation that is not always "Hmm,
> >> >> that's
> >> >> sounds like it might be interesting. Can I tell you about fq_codel
> >> >> now?"
> >> >
> >> >
> >> > Running code of other algorithms very welcome. Fq_codel is running=
 code.
> >> > Pie is running code.
> >> >
> >> > Maybe fq_pie will be running code we can test someday.
> >> >
> >> > Even then, I want a low target latency so individual TCP's are kept
> >> > responsive, and I need an algorithm that can keep up quickly with the
> >> > dynamics of wireless, so most simplistic tests will not be useful (we
> >> > don't
> >> > have such good evaluation tests today).
> >> >
> >> > When it is, and if it is better than fq_codel, I'll be happy.  But=
 the
> >> > mark/drop part of the algorithm *isn't* the most important part of=
 the
> >> > algorithm. The packet scheduling decisions are....
> >> >                                - Jim
> >> >
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Bob
> >> >>
> >> >>
> >> >>
> >> >> And I don't claim I have the right terminology for all this stuff,
> >> >> either
> >> >> (even in this mail).
> >> >>
> >> >> Which is why I was loathe to suggest exact text...
> >> >>                            - Jim
> >> >>
> >> >>
> >> >>
> >> >> At 19:50 05/12/2013, Jim Gettys wrote:
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Dec 5, 2013 at 1:13 PM, Bob Briscoe <bob.briscoe@bt.com>=
 wrote:
> >> >> Fred, Gorry, all,
> >> >> I promised to suggest text for draft-ietf-aqm-recommendation about
> >> >> allowing the AQM's behaviour to be independent for ECN and non-ECN
> >> >> packets.
> >> >> In the process, I realised we can't talk about independent AQMs for=
 ECN
> >> >> without also including Diffserv.
> >> >> This gets messy, because I believe a good AQM for BE traffic with=
 and
> >> >> without ECN, should remove much if not all the need for Diffserv.=
 But
> >> >> we
> >> >> can't ignore Diffserv.
> >> >>
> >> >>
> >> >> I agree in principle with what Bob is trying to say here (and is=
 very
> >> >> much
> >> >> what I've been saying in my blog entry of last summer).
> >> >>
> >> >> Once you have things under control, the need for Diffserv diminishes
> >> >> dramatically (but does not go away).
> >> >>
> >> >> But as Bob notes, there is still a good use for Diffserv: suitably
> >> >> marked
> >> >> traffic may want to contend for access to the channel differently:=
 your
> >> >> marked VOIP packets may want to change the priority with which you
> >> >> request
> >> >> channel access, so that you get more timely access to the medium.=
 This
> >> >> conserves transmit opportunities, which is often the scarcest=
 resource
> >> >> in
> >> >> many systems (e.g. 802.11, DOCSIS, etc.). This can be the difference
> >> >> between
> >> >> your VOIP working well, and not working well, on a busy 802.11=
 network
> >> >> as
> >> >> well as using the channel as efficiently as possible.
> >> >>
> >> >> Similarly, if you have packets you know are background, it's helpful=
 to
> >> >> know that to ensure that they never contend for access to the medium
> >> >> but
> >> >> will always defer to other traffic, and just scavenge available=
 space
> >> >> in
> >> >> other transmit opportunities where possible.
> >> >>
> >> >> I'm a bit loathe though to tie the behavior to queues, however; in
> >> >> particular, best effort traffic may want to be sent in the same
> >> >> aggregate as
> >> >> higher (or lower) priority traffic, if there is remaining space in=
 the
> >> >> aggregate.
> >> >>
> >> >> In short, the mental model we've had that there is a one-to-one=
 model
> >> >> of
> >> >> hardware and software queues (not to mention flows in a given=
 software
> >> >> queue) is often incorrect (or at least seriously sub-optimal) in
> >> >> today's
> >> >> systems (even if the hardware queues "work" properly, which it=
 appears
> >> >> they
> >> >> do not in 802.11).
> >> >>
> >> >> So I'm not sure Bob's new section 3 here is how to best to state=
 this
> >> >> (or
> >> >> to deal with the terminology problem).  Certainly, it may be the=
 same
> >> >> instance of an AQM algorithm, rather than different instances, for
> >> >> example.
> >> >> And "
> >> >> It SHOULD be possible" is more a pious wish than anything else.  But=
 I
> >> >> agree in spirit with what Bob's trying to say.
> >> >>                                - Jim
> >> >>
> >> >>
> >> >>
> >> >>=20
>=
 ___________________________________________________________________________=
______________
> >> >> {In Section 4: add another bullet between recommendations 2 & 3:}
> >> >> 3{New}. It SHOULD be possible to make different instances of an AQM
> >> >> algorithm apply to different subsets of packets that share the same
> >> >> queue.
> >> >> It SHOULD be possible to classify packets into these subsets at=
 least
> >> >> by ECN
> >> >> codepoint [RFC3168] and Diffserv codepoint [RFC2474] (or the=
 equivalent
> >> >> of
> >> >> these fields at lower layers).
> >> >> {Then a new section to expand on this before the current Section=
 4.3.}
> >> >> 4.3{New}. Independent AQM Instances for ECN and Diffserv
> >> >> The recommendation to provide a separate instance of the AQM for ECN
> >> >> packets goes beyond the assumptions of RFC 3168, which assumed that
> >> >> only one
> >> >> instance of an AQM will handle both ECN-capable and non-ECN-capable
> >> >> packets.
> >> >>
> >> >>
> >> >> Bob
> >> >>
> >> >>
> >> >> ________________________________________________________________ Bob
> >> >> Briscoe,                                                  BT
> >> >> _______________________________________________ aqm mailing list
> >> >> aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> aqm mailing list
> >> >> aqm@ietf.org
> >> >> https://www.ietf.org/mailman/listinfo/aqm
> >> >>
> >> >>
> >> >> ________________________________________________________________
> >> >> Bob Briscoe,                                                  BT
> >> >>
> >> >> _______________________________________________
> >> >> aqm mailing list
> >> >> aqm@ietf.org
> >> >> https://www.ietf.org/mailman/listinfo/aqm
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> aqm mailing list
> >> >> aqm@ietf.org
> >> >> https://www.ietf.org/mailman/listinfo/aqm
> >> >>
> >> >> ________________________________________________________________
> >> >> Bob Briscoe,                                                  BT
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> aqm mailing list
> >> >> aqm@ietf.org
> >> >> https://www.ietf.org/mailman/listinfo/aqm
> >> >>
> >> > I can imagine many other possible algorithms that CoDel for the
> >> > mark/drop
> >> > algorithm; we happen to like CoDel atm for two reasons: 1) it is self
> >> > adapting to the line rate, and 2) head mark/drop signals the TCP's as
> >> > soon
> >> > as a decision is made rather than possibly being applied much later
> >> > (such as
> >> > random or tail drop).  We welcome and encourage further essays in the
> >> > art.
> >> >
> >> >
> >> > _______________________________________________
> >> > aqm mailing list
> >> > aqm@ietf.org
> >> > https://www.ietf.org/mailman/listinfo/aqm
> >> >
> >>
> >>
> >>
> >> --
> >> Dave T=E4ht
> >>
> >> Fixing bufferbloat with cerowrt:
> >> http://www.teklibre.com/cerowrt/subscribe.html
> >
> >
> > ________________________________________________________________
> > Bob Briscoe,                                                  BT
>
>
>
>--
>Dave T=E4ht
>
>Fixing bufferbloat with cerowrt:=20
>http://www.teklibre.com/cerowrt/subscribe.html

________________________________________________________________
Bob Briscoe,                                                  BT=20


From naeem.khademi@gmail.com  Sat Dec 14 21:35:17 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 343281AE0DF for <aqm@ietfa.amsl.com>; Sat, 14 Dec 2013 21:35:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gcP2w2H1Gqty for <aqm@ietfa.amsl.com>; Sat, 14 Dec 2013 21:35:12 -0800 (PST)
Received: from mail-vc0-x233.google.com (mail-vc0-x233.google.com [IPv6:2607:f8b0:400c:c03::233]) by ietfa.amsl.com (Postfix) with ESMTP id 029311ADFEE for <aqm@ietf.org>; Sat, 14 Dec 2013 21:35:11 -0800 (PST)
Received: by mail-vc0-f179.google.com with SMTP id ie18so2389774vcb.10 for <aqm@ietf.org>; Sat, 14 Dec 2013 21:35:05 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=XqtLbLNXgFgannlouiwyVB9Ju0EPrsc0IIjFUL8Hjww=; b=v8e7qU0LRBq1YOexufwXPIZP+mZY4fk8Me3lL0C77Bw4ZA4L9s0wDu4C1VGN1Trck8 csExkCzIzxJMuOud52wNtA9OtpEcc0QvLiW4N6licbuUetn+r9Eqd5pLD1RAY3jwefZi BDFOkQMuH3PvNj8hkZtyFEDCTdk8V2HAlYA1Dy+RNt5D/2C5s+czwxjctFslvgMeIUPe KZMXsIuKioNu9Q69+oKwrxwS+oMDkc1RpNdiBDA9BUD2ZTuJi+Dy3dGCIkvZwDu3riDB p6vq9rOcswqhlMyDp/x6A88Vd51QC2uBTQulomvXaA9ivEy24VWvsc7hqct/zFE13ONK ICbA==
MIME-Version: 1.0
X-Received: by 10.52.191.98 with SMTP id gx2mr14309vdc.71.1387085704917; Sat, 14 Dec 2013 21:35:04 -0800 (PST)
Received: by 10.221.5.196 with HTTP; Sat, 14 Dec 2013 21:35:04 -0800 (PST)
Date: Sun, 15 Dec 2013 06:35:04 +0100
Message-ID: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>,  end2end-interest@postel.org
Content-Type: multipart/alternative; boundary=089e0122e75ab6faa004ed8c0e28
Subject: [aqm] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 15 Dec 2013 05:35:17 -0000

--089e0122e75ab6faa004ed8c0e28
Content-Type: text/plain; charset=ISO-8859-1

Hi all

I'm not sure if this has already been covered in any of the other threads,
but looking at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdfand
draft-ietf-aqm-recommendation-00, the question remains: "what is a
good
burst (size) that AQMs should allow?" and/or "how an AQM can have a notion
of the right burst size?".

and how "naturally-occuring bursts" mentioned in
draft-ietf-aqm-recommendation-00
can be defined?



Regards,
Naeem

--089e0122e75ab6faa004ed8c0e28
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all=A0<div><br></div><div>I&#39;m not sure if this has =
already been covered in any of the other threads, but looking at <a href=3D=
"http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf">http://www.=
ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf</a> and=A0draft-ietf-aqm=
-recommendation-00, the question remains: &quot;what is a good burst (size)=
 that AQMs should allow?&quot; and/or &quot;how an AQM can have a notion of=
 the right burst size?&quot;.=A0</div>
<div><br></div><div>and how=A0<font color=3D"#000000">&quot;naturally-occur=
ing bursts&quot; mentioned in=A0</font>draft-ietf-aqm-recommendation-00 can=
 be defined?=A0</div><pre class=3D"" style=3D"font-size:1em;margin-top:0px;=
margin-bottom:0px;color:rgb(0,0,0)">
<br></pre><div><br></div><div>Regards,</div><div>Naeem=A0</div></div>

--089e0122e75ab6faa004ed8c0e28--

From davec-b@rogers.com  Sun Dec 15 06:59:39 2013
Return-Path: <davec-b@rogers.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 006F31AE033 for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 06:59:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.1
X-Spam-Level: 
X-Spam-Status: No, score=-0.1 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NjrIJQOSvoVx for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 06:59:34 -0800 (PST)
Received: from nm18-vm10.access.bullet.mail.bf1.yahoo.com (nm18-vm10.access.bullet.mail.bf1.yahoo.com [216.109.115.89]) by ietfa.amsl.com (Postfix) with SMTP id 143041AD739 for <aqm@ietf.org>; Sun, 15 Dec 2013 06:59:33 -0800 (PST)
Received: from [66.196.81.161] by nm18.access.bullet.mail.bf1.yahoo.com with NNFMP; 15 Dec 2013 14:59:26 -0000
Received: from [98.139.244.53] by tm7.access.bullet.mail.bf1.yahoo.com with NNFMP; 15 Dec 2013 14:59:26 -0000
Received: from [127.0.0.1] by smtp115.sbc.mail.bf1.yahoo.com with NNFMP; 15 Dec 2013 14:59:26 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1387119566; bh=Kn/xEca3Dk64iaziUf9CI8ilYrxkCV2DDpkzwY2oA2M=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:X-Enigmail-Version:Content-Type; b=p2BJuHq3JGgZi8bXJ0UazTwJWHJemGkeHgfkV8OPOXTfJxMGt5uE5JRKYVff2+PT1RAJilKDelgyYSAO5sjSwJLxmPKbcK5LJo68ERvbtdtu3+bOhpBmUKiI99vubPoO/AvZj/x6TqF+Bj1VF6umQ2RUd1ADcndpkALVTR6oVPo=
X-Yahoo-Newman-Id: 864131.47539.bm@smtp115.sbc.mail.bf1.yahoo.com
X-Yahoo-Newman-Property: ymail-3
X-YMail-OSG: RispqZMVM1mEfSDEoYO.3XdnVG1d4JXF5uMBOzFJCfNYZjP SgQXXlJflruqE02FcsgHcgern.9GKlZm9dY2BolnS6cjNdcdQ49NrNM1H1Dz VRk9aBXKggOIEAjif6Fk_7SwFdpxuVKe6gUy6v4iu_0bo4Hvu23kG7XFvtiB jqjKuz.sYV3HFnHZqAZh56tnJlNnSHj07k9YHW2g6II_IUtEMPTL0ue9GDeP r7srw74njjT4kuwHP39Q2k3r7cFNJ2d0fVjhKxzb.eWwJSqqqrE7XN7tDQe3 41Lfj23Kt_SV2OIo_7PYxcywUyBF2l3MuRjhsRKfpm2uGbFZUHcxRlqUmv2e Uj9Nneco46.n58._xl45lJAfLjwy83pZrgx8858c_hU6mbzilALfLnWhbiVk NLvXXKs7u63P7bJT.MWquz_wRM5a_LJdRDpaE5Tgu6TM_onGKEXm.Msnr.VV o9AwGjYiajv2c1w3v6GZ_CSeoBNXvIGgVu6vzCaNHAQsPDYV8.Sc365.A16E yTYDjafcO1_xkB4BztSlGSwZ5ysQIDKCqh_xKjw84VDyFbLyTVixUTJzgDvt 8D0o-
X-Yahoo-SMTP: sltvjZWswBCRD.ElTuB1l9j6s9wRYPpuyTNWOE5oEg--
X-Rocket-Received: from [192.168.0.12] (davec-b@99.225.150.135 with plain [98.139.221.125]) by smtp115.sbc.mail.bf1.yahoo.com with SMTP; 15 Dec 2013 06:59:26 -0800 PST
Message-ID: <52ADC3CD.3010504@rogers.com>
Date: Sun, 15 Dec 2013 09:59:25 -0500
From: David Collier-Brown <davec-b@rogers.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4
MIME-Version: 1.0
To: aqm@ietf.org, naeem.khademi@gmail.com
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
In-Reply-To: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
X-Enigmail-Version: 1.6
Content-Type: multipart/alternative; boundary="------------050006040201000003020705"
Subject: Re: [aqm] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: davecb@spamcop.net
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 15 Dec 2013 14:59:39 -0000

This is a multi-part message in MIME format.
--------------050006040201000003020705
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

It's not been discussed recently (I'm a relatively new lurker) but some
types of burst might be easily classifiable.

In particular, if a video or voice stream requires no less than X KB/S,
has a second's "headroom" in the form of delivered but undisplayed data
and suffers a 1-second outage, then it should be expected to emit a
burst of 2X before settling back down to its previous rate. The AQM
system should permit this, while at the same time defending its stream
of X from bursts from elsewhere that would starve it for a second or more.

In principle, this kind of analysis can be applied to interactive
communications, with latency and user think time taking the place of
throughput and headroom in the logic. For any given stream, a (moving)
average of its needs can be discovered.

Batch transfers are harder to characterize, but at the same time can be
split into "a human is actively waiting for this" and "get it done for
tomorrow" classes (;-))

This kind of "no less than" resource management is something that has
been well-studied, and substantial work (IBM's "goal mode") was done as
long ago as the mainframe era. 

In capacity planning, one often sets guarantees that a database write
process get no less than a certain amount of resources, and then
constructs trend watchers to ensure that the database's own growth in
activity doesn't render the guarantee too small. This is quite ordinary
"state of practice" work for a planner,  and is often applied
holus-bolus to short-run networks like cluster interconnects.

Researchers in queuing-network models will know much more about the
state of the art, and how it can be applied to AQM.

--dave (a capacity planner and user of queuing networks) c-b

On 12/15/2013 12:35 AM, Naeem Khademi wrote:
> Hi all 
>
> I'm not sure if this has already been covered in any of the other
> threads, but looking at
> http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf
> and draft-ietf-aqm-recommendation-00, the question remains: "what is a
> good burst (size) that AQMs should allow?" and/or "how an AQM can have
> a notion of the right burst size?". 
>
> and how "naturally-occuring bursts" mentioned
> in draft-ietf-aqm-recommendation-00 can be defined? 
>
> Regards,
> Naeem 
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm


-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain
(416) 223-8968


--------------050006040201000003020705
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">It's not been discussed recently (I'm a
      relatively new lurker) but some types of burst might be easily
      classifiable.<br>
      <br>
      In particular, if a video or voice stream requires no less than X
      KB/S, has a second's "headroom" in the form of delivered but
      undisplayed data and suffers a 1-second outage, then it should be
      expected to emit a burst of 2X before settling back down to its
      previous rate. The AQM system should permit this, while at the
      same time defending its stream of X from bursts from elsewhere
      that would starve it for a second or more.<br>
      <br>
      In principle, this kind of analysis can be applied to interactive
      communications, with latency and user think time taking the place
      of throughput and headroom in the logic. For any given stream, a
      (moving) average of its needs can be discovered.<br>
      <br>
      Batch transfers are harder to characterize, but at the same time
      can be split into "a human is actively waiting for this" and "get
      it done for tomorrow" classes (;-))<br>
      <br>
      This kind of "no less than" resource management is something that
      has been well-studied, and substantial work (IBM's "goal mode")
      was done as long ago as the mainframe era.&nbsp; <br>
      <br>
      In capacity planning, one often sets guarantees that a database
      write process get no less than a certain amount of resources, and
      then constructs trend watchers to ensure that the database's own
      growth in activity doesn't render the guarantee too small. This is
      quite ordinary "state of practice" work for a planner,&nbsp; and is
      often applied holus-bolus to short-run networks like cluster
      interconnects.<br>
      <br>
      Researchers in queuing-network models will know much more about
      the state of the art, and how it can be applied to AQM. <br>
      <br>
      --dave (a capacity planner and user of queuing networks) c-b<br>
      <br>
      On 12/15/2013 12:35 AM, Naeem Khademi wrote:<br>
    </div>
    <blockquote
cite="mid:CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com"
      type="cite">
      <div dir="ltr">Hi all&nbsp;
        <div><br>
        </div>
        <div>I'm not sure if this has already been covered in any of the
          other threads, but looking at <a moz-do-not-send="true"
            href="http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf">http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf</a>
          and&nbsp;draft-ietf-aqm-recommendation-00, the question remains:
          "what is a good burst (size) that AQMs should allow?" and/or
          "how an AQM can have a notion of the right burst size?".&nbsp;</div>
        <div><br>
        </div>
        <div>and how&nbsp;<font color="#000000">"naturally-occuring bursts"
            mentioned in&nbsp;</font>draft-ietf-aqm-recommendation-00 can be
          defined?&nbsp;</div>
        <pre class="" style="font-size:1em;margin-top:0px;margin-bottom:0px;color:rgb(0,0,0)">
</pre>
        <div><br>
        </div>
        <div>Regards,</div>
        <div>Naeem&nbsp;</div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
aqm mailing list
<a class="moz-txt-link-abbreviated" href="mailto:aqm@ietf.org">aqm@ietf.org</a>
<a class="moz-txt-link-freetext" href="https://www.ietf.org/mailman/listinfo/aqm">https://www.ietf.org/mailman/listinfo/aqm</a>
</pre>
    </blockquote>
    <br>
    <br>
    <pre class="moz-signature" cols="72">-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
<a class="moz-txt-link-abbreviated" href="mailto:davecb@spamcop.net">davecb@spamcop.net</a>           |                      -- Mark Twain
(416) 223-8968
</pre>
  </body>
</html>

--------------050006040201000003020705--

From fred@cisco.com  Sun Dec 15 13:42:17 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 36A831AE007 for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 13:42:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -108.14
X-Spam-Level: 
X-Spam-Status: No, score=-108.14 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.538, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cQ3JFNEGYUl9 for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 13:42:15 -0800 (PST)
Received: from alln-iport-6.cisco.com (alln-iport-6.cisco.com [173.37.142.93]) by ietfa.amsl.com (Postfix) with ESMTP id ED9C51ACC7D for <aqm@ietf.org>; Sun, 15 Dec 2013 13:42:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4711; q=dns/txt; s=iport; t=1387143734; x=1388353334; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=WPN4NGw5/3HLxJ/RXkwU+ljiABUzFakg1BbfqR2wdiI=; b=PIXLfRNTiwIWz6DqU/+Oz+wEUu93ypJI0iVhFSDsOT0AjgDVGmCg5Jjh Xla9R5ZeIFBNA4pjlyZzCJmn/VyIVY0GEed+VyfzY3igskeR6WL1j9MsI Izfr4aMeigPvU26gqYpK3S973Pp9MAueU1BtL8/bQMHwZg9sFGMLh8hRZ 4=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgwFAPcgrlKtJV2Y/2dsb2JhbABZgwo4VbhkgRoWdIIlAQEBAgEBeQULAgEIRiERJQIEDgUJBYdiAwkIDcAfDYcBF40GgTIJWAeDI4ETBJAzgTGER4FrjFqFOoMqgWgBHwQe
X-IronPort-AV: E=Sophos;i="4.95,490,1384300800"; d="asc'?scan'208";a="6914182"
Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by alln-iport-6.cisco.com with ESMTP; 15 Dec 2013 21:42:14 +0000
Received: from xhc-aln-x15.cisco.com (xhc-aln-x15.cisco.com [173.36.12.89]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id rBFLgEY8016434 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 15 Dec 2013 21:42:14 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.86]) by xhc-aln-x15.cisco.com ([173.36.12.89]) with mapi id 14.03.0123.003; Sun, 15 Dec 2013 15:42:14 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Thread-Topic: [aqm] What is a good burst? -- AQM evaluation guidelines
Thread-Index: AQHO+d6DGaO57DEGIE2Y+PGzjUVmZQ==
Date: Sun, 15 Dec 2013 21:42:12 +0000
Message-ID: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com>
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
In-Reply-To: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.114]
Content-Type: multipart/signed; boundary="Apple-Mail=_0D503F3D-FA31-4119-AD12-2565EFEDF94D"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: "<end2end-interest@postel.org>" <end2end-interest@postel.org>, "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [aqm] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 15 Dec 2013 21:42:17 -0000

--Apple-Mail=_0D503F3D-FA31-4119-AD12-2565EFEDF94D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


On Dec 14, 2013, at 9:35 PM, Naeem Khademi <naeem.khademi@gmail.com> =
wrote:

> Hi all=20
>=20
> I'm not sure if this has already been covered in any of the other =
threads, but looking at =
http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf and =
draft-ietf-aqm-recommendation-00, the question remains: "what is a good =
burst (size) that AQMs should allow?" and/or "how an AQM can have a =
notion of the right burst size?".=20
>=20
> and how "naturally-occuring bursts" mentioned in =
draft-ietf-aqm-recommendation-00 can be defined?=20


Imagine, if you will, that you have a host and a network in front of it =
including a first hop switch or router.The host gas a TCP offload =
engine, which is a device that accepts a large chunk of data and sends =
as much of it as it has permission to send as quickly as it can. The =
host has, for sake of argument, a 10 MBPS interface, and everything else =
in the network has interfaces whose rate are measured in gigabits. The =
host gives its TSO one chunk of data, so that can't  be called a "burst" =
- it's one message. The TSO sends data as quickly as it can, but =
presumably does little more than keep the transmission system operating =
without a pause; while it might queue up 45 messages at a crack, there =
is no requirement that it do so, so the term "burst" there doesn't have =
a lot of meaning. And as the data moves through the network, the rate of =
the particular session is absolutely lost in the available capacity. So =
a burst, in the sense of the definition, never happens.

Now, repeat the experiment. However, in this case the host as a gig-E =
interface, and the next interface that its router uses is 10 or 100 =
MBPS. The host and its TSO, and for that matter the router, do exactly =
the same thing. As perceived by the router, data is arriving much more =
quickly than it is leaving, resulting in a temporarily deep queue. If =
the propagation delay through the remainder of the network and the =
destination host are appropriate, acknowledgements could arrive at the =
TSO, soliciting new transmissions, before that queue empties. In that =
case, it is very possible that the queue remains full for a period of =
time. This network event could last for quite some time.

The second is clearly a burst, according to the definition, and I would =
argue that it is naturally occurring.

I imagine you have heard Van and/or Kathy talk about "good queue" vs =
"bad queue". "Good queue" keeps enough traffic in it to fully utilize =
its egress. "Bad queue" also does so, but does so in a manner that also =
materially increases measured latency. This difference is what is behind =
my comment on the objective of a congestion management algorithm (such =
as TCP's but not limited to it) that its objective is to keep the amount =
of data outstanding large enough to maximize its transmission rate =
through the network, but not so large as to materially increase measured =
latency or probability of loss.=20

I would argue that this concept of "Good Queue" is directly relevant to =
the concept of an acceptable burst size. In the first transmission in a =
session, the sender has no information about what it will experience, so =
it behoves it to behave in a manner that is unlikely to create a =
significant amount of "bad queue" - conservatively. But it by definition =
has no numbers by which to quantify that. Hence, we make recommendations =
about the initial window size. After that, I would argue that it should =
continue to behave in a manner that doesn't led to "bad queue", but is =
free to operate in any manner that seeks to keep the amount of data =
outstanding large enough to maximize its transmission rate through the =
network, but not so large as to materially increase measured latency or =
probability of loss. At the point that it sends data in a manner that =
creates a sustained queue, it has exceeded what would be considered a =
useful burst size.

--Apple-Mail=_0D503F3D-FA31-4119-AD12-2565EFEDF94D
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSriIybjEdbHIsm0MRAjAnAKDcPMUZ6cLX7XF/E+AddGlxBtUykACfUozs
mjOeX4kJYvHOF8N9wzSu2Tw=
=nsMl
-----END PGP SIGNATURE-----

--Apple-Mail=_0D503F3D-FA31-4119-AD12-2565EFEDF94D--

From bob.briscoe@bt.com  Sun Dec 15 14:57:50 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8DEC1AE1FA for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 14:57:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.139
X-Spam-Level: 
X-Spam-Status: No, score=-3.139 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.538, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u6HmAK7mEsdw for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 14:57:46 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 82F881AE1D5 for <aqm@ietf.org>; Sun, 15 Dec 2013 14:57:44 -0800 (PST)
Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR67-UKRD.bt.com (10.187.101.22) with Microsoft SMTP Server (TLS) id 8.3.297.1; Sun, 15 Dec 2013 22:57:43 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.279.1; Sun, 15 Dec 2013 22:57:43 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Sun, 15 Dec 2013 22:57:42 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.230.1])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rBFMveZO022075;	Sun, 15 Dec 2013 22:57:40 GMT
Message-ID: <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Sun, 15 Dec 2013 22:57:39 +0000
To: "Fred Baker (fred)" <fred@cisco.com>, Naeem Khademi <naeem.khademi@gmail.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com>
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com> <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: bloat <bloat@lists.bufferbloat.net>, "<end2end-interest@postel.org>" <end2end-interest@postel.org>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] [e2e] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 15 Dec 2013 22:57:51 -0000

Fred,

Jonathan Morton, Michael Scharf & I took Naeem's question to mean 
"What should an AQM assume the size of a good burst is?" whereas I 
think you and David C-B took the question to mean "What should an 
end-system take the size of a good burst to be?".

Naeem, could you clarify which you were asking?


Bob

At 21:42 15/12/2013, Fred Baker (fred) wrote:

>On Dec 14, 2013, at 9:35 PM, Naeem Khademi <naeem.khademi@gmail.com> wrote:
>
> > Hi all
> >
> > I'm not sure if this has already been covered in any of the other 
> threads, but looking at 
> http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf and 
> draft-ietf-aqm-recommendation-00, the question remains: "what is a 
> good burst (size) that AQMs should allow?" and/or "how an AQM can 
> have a notion of the right burst size?".
> >
> > and how "naturally-occuring bursts" mentioned in 
> draft-ietf-aqm-recommendation-00 can be defined?
>
>
>Imagine, if you will, that you have a host and a network in front of 
>it including a first hop switch or router.The host gas a TCP offload 
>engine, which is a device that accepts a large chunk of data and 
>sends as much of it as it has permission to send as quickly as it 
>can. The host has, for sake of argument, a 10 MBPS interface, and 
>everything else in the network has interfaces whose rate are 
>measured in gigabits. The host gives its TSO one chunk of data, so 
>that can't  be called a "burst" - it's one message. The TSO sends 
>data as quickly as it can, but presumably does little more than keep 
>the transmission system operating without a pause; while it might 
>queue up 45 messages at a crack, there is no requirement that it do 
>so, so the term "burst" there doesn't have a lot of meaning. And as 
>the data moves through the network, the rate of the particular 
>session is absolutely lost in the available capacity. So a burst, in 
>the sense of the definition, never happens.
>
>Now, repeat the experiment. However, in this case the host as a 
>gig-E interface, and the next interface that its router uses is 10 
>or 100 MBPS. The host and its TSO, and for that matter the router, 
>do exactly the same thing. As perceived by the router, data is 
>arriving much more quickly than it is leaving, resulting in a 
>temporarily deep queue. If the propagation delay through the 
>remainder of the network and the destination host are appropriate, 
>acknowledgements could arrive at the TSO, soliciting new 
>transmissions, before that queue empties. In that case, it is very 
>possible that the queue remains full for a period of time. This 
>network event could last for quite some time.
>
>The second is clearly a burst, according to the definition, and I 
>would argue that it is naturally occurring.
>
>I imagine you have heard Van and/or Kathy talk about "good queue" vs 
>"bad queue". "Good queue" keeps enough traffic in it to fully 
>utilize its egress. "Bad queue" also does so, but does so in a 
>manner that also materially increases measured latency. This 
>difference is what is behind my comment on the objective of a 
>congestion management algorithm (such as TCP's but not limited to 
>it) that its objective is to keep the amount of data outstanding 
>large enough to maximize its transmission rate through the network, 
>but not so large as to materially increase measured latency or 
>probability of loss.
>
>I would argue that this concept of "Good Queue" is directly relevant 
>to the concept of an acceptable burst size. In the first 
>transmission in a session, the sender has no information about what 
>it will experience, so it behoves it to behave in a manner that is 
>unlikely to create a significant amount of "bad queue" - 
>conservatively. But it by definition has no numbers by which to 
>quantify that. Hence, we make recommendations about the initial 
>window size. After that, I would argue that it should continue to 
>behave in a manner that doesn't led to "bad queue", but is free to 
>operate in any manner that seeks to keep the amount of data 
>outstanding large enough to maximize its transmission rate through 
>the network, but not so large as to materially increase measured 
>latency or probability of loss. At the point that it sends data in a 
>manner that creates a sustained queue, it has exceeded what would be 
>considered a useful burst size.

________________________________________________________________
Bob Briscoe,                                                  BT 


From fred@cisco.com  Sun Dec 15 23:34:56 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F3251AE2BA for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 23:34:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.039
X-Spam-Level: 
X-Spam-Status: No, score=-110.039 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.538, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tMVpMHz_SZLc for <aqm@ietfa.amsl.com>; Sun, 15 Dec 2013 23:34:54 -0800 (PST)
Received: from alln-iport-5.cisco.com (alln-iport-5.cisco.com [173.37.142.92]) by ietfa.amsl.com (Postfix) with ESMTP id 40AF81AE0D9 for <aqm@ietf.org>; Sun, 15 Dec 2013 23:34:54 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1709; q=dns/txt; s=iport; t=1387179294; x=1388388894; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=Z7VTGsCGHQcKC8N1A5grkk3pobGv6SG4Mo2slZ45C90=; b=ZEDvI+3ZQCTBYcjLqZXh2JVndk5oJCKjJmOsFTIW7X8xbB7U106gDXC4 /1keQJASd/hs2Bmcc8zoELZ+D4cZHc3smxKxtzwcSFdwSa+8SE5JNN3R9 H5Kuijua8aH8/DB3eCQsbUTZCH0cAnqIE7AUpOU3wE+GPo4HOF4VnueGw 8=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgMFAKWsrlKtJV2Y/2dsb2JhbABZgwqBDbhlgR4WdIIlAQEBAwF5BQsCAQhGMiUCBA4FDoduCMdLF48ZB4MjgRMEkDOBMYYykhSDKoIq
X-IronPort-AV: E=Sophos;i="4.95,493,1384300800"; d="asc'?scan'208";a="7003326"
Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by alln-iport-5.cisco.com with ESMTP; 16 Dec 2013 07:34:53 +0000
Received: from xhc-rcd-x14.cisco.com (xhc-rcd-x14.cisco.com [173.37.183.88]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id rBG7Yr7v028088 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 16 Dec 2013 07:34:53 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.86]) by xhc-rcd-x14.cisco.com ([173.37.183.88]) with mapi id 14.03.0123.003; Mon, 16 Dec 2013 01:34:52 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [e2e] [aqm] What is a good burst? -- AQM evaluation  guidelines
Thread-Index: AQHO+jFOK/WIn67e5kiRmezjT1upVA==
Date: Mon, 16 Dec 2013 07:34:52 +0000
Message-ID: <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com>
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com> <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk>
In-Reply-To: <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.114]
Content-Type: multipart/signed; boundary="Apple-Mail=_74E31F59-A6A4-49D5-A8B3-725BF1044FF5"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>, Naeem Khademi <naeem.khademi@gmail.com>, "<end2end-interest@postel.org>" <end2end-interest@postel.org>
Subject: Re: [aqm] [e2e] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Dec 2013 07:34:56 -0000

--Apple-Mail=_74E31F59-A6A4-49D5-A8B3-725BF1044FF5
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Dec 15, 2013, at 2:57 PM, Bob Briscoe <bob.briscoe@bt.com>
 wrote:

> Fred,
>=20
> Jonathan Morton, Michael Scharf & I took Naeem's question to mean =
"What should an AQM assume the size of a good burst is?" whereas I think =
you and David C-B took the question to mean "What should an end-system =
take the size of a good burst to be?".

I can't comment on what he means. I took the question as "what should a =
system that is in receipt of what it might consider a 'burst', and more =
especially a 'good burst', to be?"

I don't know that a sending transport (which is to be distinguished from =
the queueing arrangement in that same system) or a receiving system =
*has* a definition of a "good" or "bad" burst. The one is sending data, =
which in the context of y two examples might be a good or bad idea, and =
the other is receiving it. =46rom the receiver's perspective, the data =
either arrived or it didn't; if it arrived, there is no real argument =
for not delivering it to its application...

--Apple-Mail=_74E31F59-A6A4-49D5-A8B3-725BF1044FF5
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSrq0abjEdbHIsm0MRAgMIAJ4lLmSETxkaMAigHrXGpPxbHp0fzACgn/ep
I0xrsI7E7+2EKohbKjOvMCs=
=IglE
-----END PGP SIGNATURE-----

--Apple-Mail=_74E31F59-A6A4-49D5-A8B3-725BF1044FF5--

From naeem.khademi@gmail.com  Mon Dec 16 05:47:38 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EEE511ADF8C for <aqm@ietfa.amsl.com>; Mon, 16 Dec 2013 05:47:37 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Exag6wLaOsEW for <aqm@ietfa.amsl.com>; Mon, 16 Dec 2013 05:47:35 -0800 (PST)
Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) by ietfa.amsl.com (Postfix) with ESMTP id AF7A41ADFC4 for <aqm@ietf.org>; Mon, 16 Dec 2013 05:47:35 -0800 (PST)
Received: by mail-ve0-f178.google.com with SMTP id c14so3371094vea.9 for <aqm@ietf.org>; Mon, 16 Dec 2013 05:47:34 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=A5LQ/cR04oXmE56hrZ9EA/3JIm8ynYOJiTWwih9WTQo=; b=p5bWnLFslFqZ5UPj7hK9/Ps9zvc8gAOhFSK1ljuKQeFVHB05RgymtLwjfX9mOkv7FY 52xTsb1UauE6DJ70/EoTSlB89SfDM7sQ+E54NU2D5SFCxqX1yhdGyC71wp+t163ebq3L ikgLkdg18fyL+jDQS6ndaHx6sWFyqkIkUz7IaICInRReozo/A7vy05Guqhl3czV9GPO3 PPplZX6653tGmWMgteAG/Jx6TAmrA72AtcD+x+x/qq9cloPNne+lVXByHV6thAo4Ww7V /mPqZMVylhDfXjKhNkm55AkaRPwQwwE3TfmUaZmtkgxkmZLhWOlS8hLu2HgmcIaFse1j tjJQ==
MIME-Version: 1.0
X-Received: by 10.52.249.105 with SMTP id yt9mr336365vdc.67.1387201654884; Mon, 16 Dec 2013 05:47:34 -0800 (PST)
Received: by 10.221.5.196 with HTTP; Mon, 16 Dec 2013 05:47:34 -0800 (PST)
In-Reply-To: <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com>
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com> <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com>
Date: Mon, 16 Dec 2013 14:47:34 +0100
Message-ID: <CAEjQQ5XaqZA_0Q8fE05sKOn2_s9SHY5aWr=g3Xa+7b5xwv8zJg@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Content-Type: multipart/alternative; boundary=089e01536864df127f04eda70d14
Cc: bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, "<end2end-interest@postel.org>" <end2end-interest@postel.org>
Subject: Re: [aqm] [e2e] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Dec 2013 13:47:38 -0000

--089e01536864df127f04eda70d14
Content-Type: text/plain; charset=ISO-8859-1

Bob, Fred and all

I'll copy/paste the question here again: "what is a good burst (size) that
AQMs should allow?" and/or "how an AQM can have a notion of the right burst
size?"

So, obviously, as Bob mentioned, I'm concerned about what AQMs should or
shouldn't do. The mission of dealing with packet bursts in addition to the
task of keeping the standing queue very low or minimal is part of an "AQM
evaluation criteria" I envision. While I do agree with all Fred's remarks,
I'm more concerned to have an answer for this, for where AQMs might get
deployed.

An example: when designing my AQM X should I care about 64K TSO-generated
bursts to safely pass without dropping or not?  Does the answer (whatever
it is) also apply to the burst sizes typical of multimedia traffic, etc.?
if the answer is "yes", should an AQM design be actively aware of what
application layer does in terms of sending bursty traffic or not? and to
what extent if yes?

Regards,
Naeem

On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker (fred) <fred@cisco.com> wrote:

>
> On Dec 15, 2013, at 2:57 PM, Bob Briscoe <bob.briscoe@bt.com>
>  wrote:
>
> > Fred,
> >
> > Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What
> should an AQM assume the size of a good burst is?" whereas I think you and
> David C-B took the question to mean "What should an end-system take the
> size of a good burst to be?".
>
> I can't comment on what he means. I took the question as "what should a
> system that is in receipt of what it might consider a 'burst', and more
> especially a 'good burst', to be?"
>
> I don't know that a sending transport (which is to be distinguished from
> the queueing arrangement in that same system) or a receiving system *has* a
> definition of a "good" or "bad" burst. The one is sending data, which in
> the context of y two examples might be a good or bad idea, and the other is
> receiving it. From the receiver's perspective, the data either arrived or
> it didn't; if it arrived, there is no real argument for not delivering it
> to its application...
>

--089e01536864df127f04eda70d14
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Bob, Fred and all=A0<div><br></div><div>I&#39;ll copy/past=
e the question here again:=A0<span style=3D"font-family:arial,sans-serif;fo=
nt-size:13px">&quot;what is a good burst (size) that AQMs should allow?&quo=
t; and/or &quot;how an AQM can have a notion of the right burst size?&quot;=
</span></div>
<div><font face=3D"arial, sans-serif"><br></font></div><div><font face=3D"a=
rial, sans-serif">So, obviously, as Bob mentioned, I&#39;m concerned about =
what AQMs should or shouldn&#39;t do. The mission of dealing with packet bu=
rsts in addition to the task of keeping the standing queue very low or mini=
mal is part of an &quot;AQM evaluation criteria&quot; I envision. While I d=
o agree with all Fred&#39;s remarks, I&#39;m more concerned to have an answ=
er for this, for where AQMs might get deployed. =A0=A0</font></div>
<div><font face=3D"arial, sans-serif"><br></font></div><div><font face=3D"a=
rial, sans-serif">An example:=A0</font><span style=3D"font-family:arial,san=
s-serif;font-size:13px">when designing my AQM X should I care about 64K TSO=
-generated bursts to safely pass without dropping or not? =A0Does the answe=
r (whatever it is) also apply to the burst sizes typical of multimedia traf=
fic, etc.? if the answer is &quot;yes&quot;, should an AQM design be active=
ly aware of what application layer does in terms of sending bursty traffic =
or not? and to what extent if yes?=A0</span><span style=3D"font-family:aria=
l,sans-serif">=A0 =A0</span></div>
<div><span style=3D"font-family:arial,sans-serif"><br></span></div><div sty=
le><span style=3D"font-family:arial,sans-serif">Regards,</span></div><div s=
tyle><span style=3D"font-family:arial,sans-serif">Naeem</span></div><div><d=
iv class=3D"gmail_extra">
<br><div class=3D"gmail_quote">On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker =
(fred) <span dir=3D"ltr">&lt;<a href=3D"mailto:fred@cisco.com" target=3D"_b=
lank">fred@cisco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-co=
lor:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
On Dec 15, 2013, at 2:57 PM, Bob Briscoe &lt;<a href=3D"mailto:bob.briscoe@=
bt.com">bob.briscoe@bt.com</a>&gt;<br>
<div class=3D"im">=A0wrote:<br>
<br>
&gt; Fred,<br>
&gt;<br>
&gt; Jonathan Morton, Michael Scharf &amp; I took Naeem&#39;s question to m=
ean &quot;What should an AQM assume the size of a good burst is?&quot; wher=
eas I think you and David C-B took the question to mean &quot;What should a=
n end-system take the size of a good burst to be?&quot;.<br>

<br>
</div>I can&#39;t comment on what he means. I took the question as &quot;wh=
at should a system that is in receipt of what it might consider a &#39;burs=
t&#39;, and more especially a &#39;good burst&#39;, to be?&quot;<br>
<br>
I don&#39;t know that a sending transport (which is to be distinguished fro=
m the queueing arrangement in that same system) or a receiving system *has*=
 a definition of a &quot;good&quot; or &quot;bad&quot; burst. The one is se=
nding data, which in the context of y two examples might be a good or bad i=
dea, and the other is receiving it. From the receiver&#39;s perspective, th=
e data either arrived or it didn&#39;t; if it arrived, there is no real arg=
ument for not delivering it to its application...<br>

</blockquote></div><br></div></div></div>

--089e01536864df127f04eda70d14--

From naeem.khademi@gmail.com  Mon Dec 16 06:05:58 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 746CE1ADFCE for <aqm@ietfa.amsl.com>; Mon, 16 Dec 2013 06:05:58 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.399
X-Spam-Level: 
X-Spam-Status: No, score=-1.399 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, J_CHICKENPOX_22=0.6, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PzPLAg112GIl for <aqm@ietfa.amsl.com>; Mon, 16 Dec 2013 06:05:57 -0800 (PST)
Received: from mail-vc0-x236.google.com (mail-vc0-x236.google.com [IPv6:2607:f8b0:400c:c03::236]) by ietfa.amsl.com (Postfix) with ESMTP id F3B891AE00E for <aqm@ietf.org>; Mon, 16 Dec 2013 06:05:56 -0800 (PST)
Received: by mail-vc0-f182.google.com with SMTP id lc6so3226014vcb.13 for <aqm@ietf.org>; Mon, 16 Dec 2013 06:05:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=t0pIS0RizA+FRVrt9IV/QJJUI+3IIP5sYTL9axApK0Q=; b=hzmsfZnsSTJpfL72SfDx6osuWbfDLeE5zY80/hkzInqDjYq34ohsgmE+TQr6mj6x6Y X2WBYWmjku9hvrr3f+N3hpxfW+AUhngOkDa9DD66bKZnKTLR6bwrjMN0BLH9qoyF4PL9 mQM2/O1uugHNLDNF4kMVXPFONBtgwW5DAw6/mZVdYKEaqxsS4uNDKPhSAS+G089gmP6q 7OdhUwTOAMS9f+zj2axZToy5ZDKVhGv9LFN6ljHfYrQOzc9zlpLMhVtq5idj8Q0SoPnO i9S/0DazBh4hTYV0Omn5RM6HVEfI8fYxfNy01ISy4ACi7FDItJQg4VVYwYrEdZpENxbZ TnyQ==
MIME-Version: 1.0
X-Received: by 10.58.186.198 with SMTP id fm6mr8472331vec.16.1387202756084; Mon, 16 Dec 2013 06:05:56 -0800 (PST)
Received: by 10.221.5.196 with HTTP; Mon, 16 Dec 2013 06:05:56 -0800 (PST)
In-Reply-To: <CAEjQQ5XaqZA_0Q8fE05sKOn2_s9SHY5aWr=g3Xa+7b5xwv8zJg@mail.gmail.com>
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com> <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> <CAEjQQ5XaqZA_0Q8fE05sKOn2_s9SHY5aWr=g3Xa+7b5xwv8zJg@mail.gmail.com>
Date: Mon, 16 Dec 2013 15:05:56 +0100
Message-ID: <CAEjQQ5WW7NCb5h+JG3f8F19uj96+c_EqpMLarMOtBvyCP2niWA@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Content-Type: multipart/alternative; boundary=047d7b677208820c6e04eda74f9e
Cc: bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, "<end2end-interest@postel.org>" <end2end-interest@postel.org>
Subject: Re: [aqm] [e2e] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Dec 2013 14:05:58 -0000

--047d7b677208820c6e04eda74f9e
Content-Type: text/plain; charset=ISO-8859-1

and to clarify more about what else was discussed, it seems to me some of
us tend to correspond and relate the notion of "good queue" vs. "bad queue"
used by KN+VJ ACM queue paper to my question on "good bursts". While they
likely to be correlated (I have no argument on this now), the notion of
"good burst" goes beyond the "good queue" defined in that paper. Based on
their definition a good queue is a queue that minimizes the standing queue
(or gets rid of it entirely) while allowing a certain amount of (sub-RTT?
typical 100 ms) bursts while avoiding the link to get under-utilized. That
notion (again, I have no argument on its correctness for now) is different
from my question on "good bursts" which means that: once we manage to get
rid of the standing queue, what types/sizes of bursts I should let the AQM
X to protect/handle?

Naeem


On Mon, Dec 16, 2013 at 2:47 PM, Naeem Khademi <naeem.khademi@gmail.com>wrote:

> Bob, Fred and all
>
> I'll copy/paste the question here again: "what is a good burst (size)
> that AQMs should allow?" and/or "how an AQM can have a notion of the right
> burst size?"
>
> So, obviously, as Bob mentioned, I'm concerned about what AQMs should or
> shouldn't do. The mission of dealing with packet bursts in addition to the
> task of keeping the standing queue very low or minimal is part of an "AQM
> evaluation criteria" I envision. While I do agree with all Fred's remarks,
> I'm more concerned to have an answer for this, for where AQMs might get
> deployed.
>
> An example: when designing my AQM X should I care about 64K TSO-generated
> bursts to safely pass without dropping or not?  Does the answer (whatever
> it is) also apply to the burst sizes typical of multimedia traffic, etc.?
> if the answer is "yes", should an AQM design be actively aware of what
> application layer does in terms of sending bursty traffic or not? and to
> what extent if yes?
>
> Regards,
> Naeem
>
> On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker (fred) <fred@cisco.com> wrote:
>
>>
>> On Dec 15, 2013, at 2:57 PM, Bob Briscoe <bob.briscoe@bt.com>
>>  wrote:
>>
>> > Fred,
>> >
>> > Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What
>> should an AQM assume the size of a good burst is?" whereas I think you and
>> David C-B took the question to mean "What should an end-system take the
>> size of a good burst to be?".
>>
>> I can't comment on what he means. I took the question as "what should a
>> system that is in receipt of what it might consider a 'burst', and more
>> especially a 'good burst', to be?"
>>
>> I don't know that a sending transport (which is to be distinguished from
>> the queueing arrangement in that same system) or a receiving system *has* a
>> definition of a "good" or "bad" burst. The one is sending data, which in
>> the context of y two examples might be a good or bad idea, and the other is
>> receiving it. From the receiver's perspective, the data either arrived or
>> it didn't; if it arrived, there is no real argument for not delivering it
>> to its application...
>>
>
>

--047d7b677208820c6e04eda74f9e
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">and to clarify more about what else was discussed, it seem=
s to me some of us tend to correspond and relate the notion of &quot;good q=
ueue&quot; vs. &quot;bad queue&quot; used by KN+VJ ACM queue paper to my qu=
estion on &quot;good bursts&quot;. While they likely to be correlated (I ha=
ve no=A0argument=A0on this now), the notion of &quot;good burst&quot; goes =
beyond the &quot;good queue&quot; defined in that paper. Based on their def=
inition a good queue is a queue that minimizes the standing queue (or gets =
rid of it entirely) while allowing a certain amount of (sub-RTT? typical 10=
0 ms) bursts while avoiding the link to get under-utilized. That notion (ag=
ain, I have no argument on its correctness for now) is different from my qu=
estion on &quot;good bursts&quot; which means that: once we manage to get r=
id of the standing queue, what types/sizes of bursts I should let the AQM X=
 to protect/handle?<div>
<br></div><div style>Naeem =A0<br></div></div><div class=3D"gmail_extra"><b=
r><br><div class=3D"gmail_quote">On Mon, Dec 16, 2013 at 2:47 PM, Naeem Kha=
demi <span dir=3D"ltr">&lt;<a href=3D"mailto:naeem.khademi@gmail.com" targe=
t=3D"_blank">naeem.khademi@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Bob, Fred and all=A0<div><b=
r></div><div>I&#39;ll copy/paste the question here again:=A0<span style=3D"=
font-family:arial,sans-serif;font-size:13px">&quot;what is a good burst (si=
ze) that AQMs should allow?&quot; and/or &quot;how an AQM can have a notion=
 of the right burst size?&quot;</span></div>

<div><font face=3D"arial, sans-serif"><br></font></div><div><font face=3D"a=
rial, sans-serif">So, obviously, as Bob mentioned, I&#39;m concerned about =
what AQMs should or shouldn&#39;t do. The mission of dealing with packet bu=
rsts in addition to the task of keeping the standing queue very low or mini=
mal is part of an &quot;AQM evaluation criteria&quot; I envision. While I d=
o agree with all Fred&#39;s remarks, I&#39;m more concerned to have an answ=
er for this, for where AQMs might get deployed. =A0=A0</font></div>

<div><font face=3D"arial, sans-serif"><br></font></div><div><font face=3D"a=
rial, sans-serif">An example:=A0</font><span style=3D"font-family:arial,san=
s-serif;font-size:13px">when designing my AQM X should I care about 64K TSO=
-generated bursts to safely pass without dropping or not? =A0Does the answe=
r (whatever it is) also apply to the burst sizes typical of multimedia traf=
fic, etc.? if the answer is &quot;yes&quot;, should an AQM design be active=
ly aware of what application layer does in terms of sending bursty traffic =
or not? and to what extent if yes?=A0</span><span style=3D"font-family:aria=
l,sans-serif">=A0 =A0</span></div>

<div><span style=3D"font-family:arial,sans-serif"><br></span></div><div><sp=
an style=3D"font-family:arial,sans-serif">Regards,</span></div><div><span s=
tyle=3D"font-family:arial,sans-serif">Naeem</span></div><div><div class=3D"=
h5">
<div><div class=3D"gmail_extra">
<br><div class=3D"gmail_quote">On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker =
(fred) <span dir=3D"ltr">&lt;<a href=3D"mailto:fred@cisco.com" target=3D"_b=
lank">fred@cisco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-co=
lor:rgb(204,204,204);border-left-style:solid;padding-left:1ex">

<br>
On Dec 15, 2013, at 2:57 PM, Bob Briscoe &lt;<a href=3D"mailto:bob.briscoe@=
bt.com" target=3D"_blank">bob.briscoe@bt.com</a>&gt;<br>
<div>=A0wrote:<br>
<br>
&gt; Fred,<br>
&gt;<br>
&gt; Jonathan Morton, Michael Scharf &amp; I took Naeem&#39;s question to m=
ean &quot;What should an AQM assume the size of a good burst is?&quot; wher=
eas I think you and David C-B took the question to mean &quot;What should a=
n end-system take the size of a good burst to be?&quot;.<br>


<br>
</div>I can&#39;t comment on what he means. I took the question as &quot;wh=
at should a system that is in receipt of what it might consider a &#39;burs=
t&#39;, and more especially a &#39;good burst&#39;, to be?&quot;<br>
<br>
I don&#39;t know that a sending transport (which is to be distinguished fro=
m the queueing arrangement in that same system) or a receiving system *has*=
 a definition of a &quot;good&quot; or &quot;bad&quot; burst. The one is se=
nding data, which in the context of y two examples might be a good or bad i=
dea, and the other is receiving it. From the receiver&#39;s perspective, th=
e data either arrived or it didn&#39;t; if it arrived, there is no real arg=
ument for not delivering it to its application...<br>


</blockquote></div><br></div></div></div></div></div>
</blockquote></div><br></div>

--047d7b677208820c6e04eda74f9e--

From fred@cisco.com  Mon Dec 16 09:30:14 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0EB5A1AE0AA for <aqm@ietfa.amsl.com>; Mon, 16 Dec 2013 09:30:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -109.438
X-Spam-Level: 
X-Spam-Status: No, score=-109.438 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, J_CHICKENPOX_22=0.6, RP_MATCHES_RCVD=-0.538, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4qfubQJEVnHd for <aqm@ietfa.amsl.com>; Mon, 16 Dec 2013 09:30:11 -0800 (PST)
Received: from alln-iport-7.cisco.com (alln-iport-7.cisco.com [173.37.142.94]) by ietfa.amsl.com (Postfix) with ESMTP id 38DE41AE0AE for <aqm@ietf.org>; Mon, 16 Dec 2013 09:30:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=13539; q=dns/txt; s=iport; t=1387215011; x=1388424611; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=k21gh1wJ6qibYymk9hqLMMKhBSyeWiJ8IxkcwELTc84=; b=QTu0BMrnrbGeI3dMTJWmhhPDx7ChnT/PGfRp9o5fw0AAIwl1kKmRDmDp oLyJz0DLyg7Aeo8bb/6BMOdZohzj30/o9JmTcnXKTGrWoQRbvdM4/npK1 WRChNneF0Pw5ceyWuyuZxK41JRG/fyqTHu074qaVYwUV57100VGG0DXZw U=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ak0FAG04r1KtJXG9/2dsb2JhbABZgkZEOFW4coElFnSCJQEBAQMBDlcUBQsCAQgYLiERJQIEDgUOh2IDCQjBJQ2HAReNBoITB4MjgRMBA5AzgTGER4FrjFqFOoMqgio
X-IronPort-AV: E=Sophos;i="4.95,496,1384300800";  d="asc'?scan'208,217";a="7127202"
Received: from rcdn-core2-2.cisco.com ([173.37.113.189]) by alln-iport-7.cisco.com with ESMTP; 16 Dec 2013 17:30:04 +0000
Received: from xhc-rcd-x10.cisco.com (xhc-rcd-x10.cisco.com [173.37.183.84]) by rcdn-core2-2.cisco.com (8.14.5/8.14.5) with ESMTP id rBGHU4BA010476 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 16 Dec 2013 17:30:04 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.86]) by xhc-rcd-x10.cisco.com ([173.37.183.84]) with mapi id 14.03.0123.003; Mon, 16 Dec 2013 11:30:03 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Thread-Topic: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines
Thread-Index: AQHO+oR0UVOioGtzyEG2i2aMc7ccUw==
Date: Mon, 16 Dec 2013 17:30:03 +0000
Message-ID: <84929DB4-4034-4B45-8BAA-D262076E9986@cisco.com>
References: <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com> <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> <CAEjQQ5XaqZA_0Q8fE05sKOn2_s9SHY5aWr=g3Xa+7b5xwv8zJg@mail.gmail.com> <CAEjQQ5WW7NCb5h+JG3f8F19uj96+c_EqpMLarMOtBvyCP2niWA@mail.gmail.com>
In-Reply-To: <CAEjQQ5WW7NCb5h+JG3f8F19uj96+c_EqpMLarMOtBvyCP2niWA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.114]
Content-Type: multipart/signed; boundary="Apple-Mail=_5B40FC2B-5987-41B4-BDDF-D1B901097025"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, "<end2end-interest@postel.org>" <end2end-interest@postel.org>
Subject: Re: [aqm] [e2e] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Dec 2013 17:30:14 -0000

--Apple-Mail=_5B40FC2B-5987-41B4-BDDF-D1B901097025
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_EB4E5CD8-3CF7-422F-85EA-5E7217571769"


--Apple-Mail=_EB4E5CD8-3CF7-422F-85EA-5E7217571769
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


On Dec 16, 2013, at 6:05 AM, Naeem Khademi <naeem.khademi@gmail.com>
 wrote:

> and to clarify more about what else was discussed, it seems to me some =
of us tend to correspond and relate the notion of "good queue" vs. "bad =
queue" used by KN+VJ ACM queue paper to my question on "good bursts". =
While they likely to be correlated (I have no argument on this now), the =
notion of "good burst" goes beyond the "good queue" defined in that =
paper. Based on their definition a good queue is a queue that minimizes =
the standing queue (or gets rid of it entirely) while allowing a certain =
amount of (sub-RTT? typical 100 ms) bursts while avoiding the link to =
get under-utilized. That notion (again, I have no argument on its =
correctness for now) is different from my question on "good bursts" =
which means that: once we manage to get rid of the standing queue, what =
types/sizes of bursts I should let the AQM X to protect/handle?

I think your question has a problem in it.

Going back to my thought experiment, suppose that we have a queuing =
point whose egress speed is X and a sender that is sending data in a CBR =
fashion at (1 + epsilon)*X. In a very formal sense, the entire =
transmission stream is a single burst, and one could imagine it taking =
hundreds or thousands of packets being sent and forwarded before the =
queue built up to a point that AQM would push back. In that case, I =
would expect an "acceptable burst" to be hundreds or thousands of =
packets. If on the other hand you have a new TCP session in slow-start =
that is using an intermediate link that is at the time fully utilized =
and on the cusp of AQM pushing back on it, the new session is very =
likely to tip the balance, and a burst of a few packets might well push =
it over the top.

So to my mind, the question isn't about the size of the burst. It is =
about the rate of onset and the effect of that burst on the latency and =
probably of loss for itself and competing sessions.

And it will never come down to a magic number N in that N is somehow =
"right", N-1 is "better", and N+1 is "over the top." There are no such =
magic numbers.

> Naeem =20
>=20
>=20
> On Mon, Dec 16, 2013 at 2:47 PM, Naeem Khademi =
<naeem.khademi@gmail.com> wrote:
> Bob, Fred and all=20
>=20
> I'll copy/paste the question here again: "what is a good burst (size) =
that AQMs should allow?" and/or "how an AQM can have a notion of the =
right burst size?"
>=20
> So, obviously, as Bob mentioned, I'm concerned about what AQMs should =
or shouldn't do. The mission of dealing with packet bursts in addition =
to the task of keeping the standing queue very low or minimal is part of =
an "AQM evaluation criteria" I envision. While I do agree with all =
Fred's remarks, I'm more concerned to have an answer for this, for where =
AQMs might get deployed.  =20
>=20
> An example: when designing my AQM X should I care about 64K =
TSO-generated bursts to safely pass without dropping or not?  Does the =
answer (whatever it is) also apply to the burst sizes typical of =
multimedia traffic, etc.? if the answer is "yes", should an AQM design =
be actively aware of what application layer does in terms of sending =
bursty traffic or not? and to what extent if yes?   =20
>=20
> Regards,
> Naeem
>=20
> On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker (fred) <fred@cisco.com> =
wrote:
>=20
> On Dec 15, 2013, at 2:57 PM, Bob Briscoe <bob.briscoe@bt.com>
>  wrote:
>=20
> > Fred,
> >
> > Jonathan Morton, Michael Scharf & I took Naeem's question to mean =
"What should an AQM assume the size of a good burst is?" whereas I think =
you and David C-B took the question to mean "What should an end-system =
take the size of a good burst to be?".
>=20
> I can't comment on what he means. I took the question as "what should =
a system that is in receipt of what it might consider a 'burst', and =
more especially a 'good burst', to be?"
>=20
> I don't know that a sending transport (which is to be distinguished =
from the queueing arrangement in that same system) or a receiving system =
*has* a definition of a "good" or "bad" burst. The one is sending data, =
which in the context of y two examples might be a good or bad idea, and =
the other is receiving it. =46rom the receiver's perspective, the data =
either arrived or it didn't; if it arrived, there is no real argument =
for not delivering it to its application...
>=20
>=20

Make things as simple as possible, but not simpler.
Albert Einstein


--Apple-Mail=_EB4E5CD8-3CF7-422F-85EA-5E7217571769
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=iso-8859-1

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Diso-8859-1"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On Dec 16, 2013, at 6:05 AM, Naeem Khademi &lt;<a =
href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;</d=
iv><div>&nbsp;wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><meta =
http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Diso-8859-1"><div dir=3D"ltr">and to clarify more about what =
else was discussed, it seems to me some of us tend to correspond and =
relate the notion of "good queue" vs. "bad queue" used by KN+VJ ACM =
queue paper to my question on "good bursts". While they likely to be =
correlated (I have no&nbsp;argument&nbsp;on this now), the notion of =
"good burst" goes beyond the "good queue" defined in that paper. Based =
on their definition a good queue is a queue that minimizes the standing =
queue (or gets rid of it entirely) while allowing a certain amount of =
(sub-RTT? typical 100 ms) bursts while avoiding the link to get =
under-utilized. That notion (again, I have no argument on its =
correctness for now) is different from my question on "good bursts" =
which means that: once we manage to get rid of the standing queue, what =
types/sizes of bursts I should let the AQM X to =
protect/handle?</div></blockquote><div dir=3D"ltr"><div>
<br></div><div>I think your question has a problem in =
it.</div><div><br></div><div>Going back to my thought experiment, =
suppose that we have a queuing point whose egress speed is X and a =
sender that is sending data in a CBR fashion at (1 + epsilon)*X. In a =
very formal sense, the entire transmission stream is a single burst, and =
one could imagine it taking hundreds or thousands of packets being sent =
and forwarded before the queue built up to a point that AQM would push =
back. In that case, I would expect an "acceptable burst" to be hundreds =
or thousands of packets. If on the other hand you have a new TCP session =
in slow-start that is using an intermediate link that is at the time =
fully utilized and on the cusp of AQM pushing back on it, the new =
session is very likely to tip the balance, and a burst of a few packets =
might well push it over the top.</div><div><br></div><div>So to my mind, =
the question isn't about the size of the burst. It is about the rate of =
onset and the effect of that burst on the latency and probably of loss =
for itself and competing sessions.</div><div><br></div><div>And it will =
never come down to a magic number N in that N is somehow "right", N-1 is =
"better", and N+1 is "over the top." There are no such magic =
numbers.</div><div><br></div></div><blockquote type=3D"cite"><div =
dir=3D"ltr"><div style=3D"">Naeem &nbsp;<br></div></div><div =
class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Mon, Dec 16, =
2013 at 2:47 PM, Naeem Khademi <span dir=3D"ltr">&lt;<a =
href=3D"mailto:naeem.khademi@gmail.com" =
target=3D"_blank">naeem.khademi@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Bob, =
Fred and all&nbsp;<div><br></div><div>I'll copy/paste the question here =
again:&nbsp;<span =
style=3D"font-family:arial,sans-serif;font-size:13px">"what is a good =
burst (size) that AQMs should allow?" and/or "how an AQM can have a =
notion of the right burst size?"</span></div>

<div><font face=3D"arial, sans-serif"><br></font></div><div><font =
face=3D"arial, sans-serif">So, obviously, as Bob mentioned, I'm =
concerned about what AQMs should or shouldn't do. The mission of dealing =
with packet bursts in addition to the task of keeping the standing queue =
very low or minimal is part of an "AQM evaluation criteria" I envision. =
While I do agree with all Fred's remarks, I'm more concerned to have an =
answer for this, for where AQMs might get deployed. =
&nbsp;&nbsp;</font></div>

<div><font face=3D"arial, sans-serif"><br></font></div><div><font =
face=3D"arial, sans-serif">An example:&nbsp;</font><span =
style=3D"font-family:arial,sans-serif;font-size:13px">when designing my =
AQM X should I care about 64K TSO-generated bursts to safely pass =
without dropping or not? &nbsp;Does the answer (whatever it is) also =
apply to the burst sizes typical of multimedia traffic, etc.? if the =
answer is "yes", should an AQM design be actively aware of what =
application layer does in terms of sending bursty traffic or not? and to =
what extent if yes?&nbsp;</span><span =
style=3D"font-family:arial,sans-serif">&nbsp; &nbsp;</span></div>

<div><span =
style=3D"font-family:arial,sans-serif"><br></span></div><div><span =
style=3D"font-family:arial,sans-serif">Regards,</span></div><div><span =
style=3D"font-family:arial,sans-serif">Naeem</span></div><div><div =
class=3D"h5">
<div><div class=3D"gmail_extra">
<br><div class=3D"gmail_quote">On Mon, Dec 16, 2013 at 8:34 AM, Fred =
Baker (fred) <span dir=3D"ltr">&lt;<a href=3D"mailto:fred@cisco.com" =
target=3D"_blank">fred@cisco.com</a>&gt;</span> wrote:<br><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left=
-style:solid;padding-left:1ex">

<br>
On Dec 15, 2013, at 2:57 PM, Bob Briscoe &lt;<a =
href=3D"mailto:bob.briscoe@bt.com" =
target=3D"_blank">bob.briscoe@bt.com</a>&gt;<br>
<div>&nbsp;wrote:<br>
<br>
&gt; Fred,<br>
&gt;<br>
&gt; Jonathan Morton, Michael Scharf &amp; I took Naeem's question to =
mean "What should an AQM assume the size of a good burst is?" whereas I =
think you and David C-B took the question to mean "What should an =
end-system take the size of a good burst to be?".<br>


<br>
</div>I can't comment on what he means. I took the question as "what =
should a system that is in receipt of what it might consider a 'burst', =
and more especially a 'good burst', to be?"<br>
<br>
I don't know that a sending transport (which is to be distinguished from =
the queueing arrangement in that same system) or a receiving system =
*has* a definition of a "good" or "bad" burst. The one is sending data, =
which in the context of y two examples might be a good or bad idea, and =
the other is receiving it. =46rom the receiver's perspective, the data =
either arrived or it didn't; if it arrived, there is no real argument =
for not delivering it to its application...<br>


</blockquote></div><br></div></div></div></div></div>
</blockquote></div><br></div>
</blockquote></div><br><div apple-content-edited=3D"true">
<span class=3D"Apple-style-span" style=3D"border-collapse: separate; =
color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: 2; text-align: =
auto; text-indent: 0px; text-transform: none; white-space: normal; =
widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; =
-webkit-border-vertical-spacing: 0px; =
-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
auto; -webkit-text-stroke-width: 0px; "><span class=3D"Apple-style-span" =
style=3D"font-family: sans-serif; font-size: 12px; line-height: 18px; =
"><ul style=3D"line-height: 1.5em; list-style-type: square; margin-top: =
0.3em; margin-right: 0px; margin-bottom: 0px; margin-left: 1.5em; =
padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: =
0px; list-style-image: =
url(http://en.wikiquote.org/skins-1.5/monobook/bullet.gif); "><li =
style=3D"margin-bottom: 0.1em; ">Make things as simple as possible, but =
not simpler.</li><div>Albert Einstein</div></ul></span></span>
</div>
<br></body></html>=

--Apple-Mail=_EB4E5CD8-3CF7-422F-85EA-5E7217571769--

--Apple-Mail=_5B40FC2B-5987-41B4-BDDF-D1B901097025
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSrziZbjEdbHIsm0MRAgd9AKDckCgnYG1DGKMkJ0emyiRQ3OzuDgCfWc7Y
YncxnOjtRhXPHgi6A0oifIM=
=Em55
-----END PGP SIGNATURE-----

--Apple-Mail=_5B40FC2B-5987-41B4-BDDF-D1B901097025--

From curtis@ipv6.occnc.com  Tue Dec 17 10:10:41 2013
Return-Path: <curtis@ipv6.occnc.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61C721AE0A3 for <aqm@ietfa.amsl.com>; Tue, 17 Dec 2013 10:10:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.437
X-Spam-Level: 
X-Spam-Status: No, score=-2.437 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.538, SPF_HELO_FAIL=0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7VHxUo6nPAVm for <aqm@ietfa.amsl.com>; Tue, 17 Dec 2013 10:10:38 -0800 (PST)
Received: from gateway0.orleans.occnc.com (gateway0.orleans.occnc.com [IPv6:2001:470:88e6:1::135]) by ietfa.amsl.com (Postfix) with ESMTP id 77D371AE085 for <aqm@ietf.org>; Tue, 17 Dec 2013 10:10:37 -0800 (PST)
Received: from harbor1.ipv6.occnc.com (harbor1.orleans.occnc.com [IPv6:2001:470:88e6:1::137]) (authenticated bits=128) by gateway0.ipv6.occnc.com (8.14.7/8.14.7) with ESMTP id rBFIuuea043478; Sun, 15 Dec 2013 13:57:02 -0500 (EST) (envelope-from curtis@ipv6.occnc.com)
Message-Id: <201312151857.rBFIuuea043478@gateway0.ipv6.occnc.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
From: Curtis Villamizar <curtis@ipv6.occnc.com>
In-reply-to: Your message of "Sun, 15 Dec 2013 06:35:04 +0100." <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
Date: Sun, 15 Dec 2013 13:56:56 -0500
Cc: end2end-interest@postel.org, "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [aqm] What is a good burst? -- AQM evaluation guidelines
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: curtis@ipv6.occnc.com
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 17 Dec 2013 18:10:41 -0000

In message <CAEjQQ5XPiqL9ywD3zXGKvUb_FoWmgJ3Zq_g_b8ONfHieHaRWzg@mail.gmail.com>
Naeem Khademi writes:
 
> Hi all
>  
> I'm not sure if this has already been covered in any of the other
> threads, but looking at
> http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdfand
> draft-ietf-aqm-recommendation-00, the question remains: "what is a
> good burst (size) that AQMs should allow?" and/or "how an AQM can have
> a notion of the right burst size?".
>  
> and how "naturally-occuring bursts" mentioned in
> draft-ietf-aqm-recommendation-00 can be defined?
>  
> Regards,
> Naeem


It is probably best not to try to define "naturally-occuring bursts"
since these are dependent on the type of traffic on the Internet, or
the target network.  This will vary with the type of target network
and will vary as services evolve on the Internet.  Therefore it may
not be definable beyond the words that make up this term and it may be
a disservice to try to define it.

If the draft is to attempt to define a target for burst tolerance (not
implying that leaky bucket is used even though it shares that term),
then the definition should be in terms of end results and it should
not be specific to any particular type of service.  For all services,
and perhaps most important is fairness among flows.  Beyond that the
criteria for "good end results" differs by traffic type.  For example,
for non-interactive bulk transfer high goodput and fairness are
desireable.  On the other side of the continuum, For interactive
bounded delay is important, though throughput is still important as is
fairness.

<slightly-off-topic>

The whole fairness thing is a sticky point and goes beyond AQM alone.
Today fairness on the Internet and most networks relies on the good
behavior of end-system protocols such as TCP.  There are
hyper-aggressive TCP variants and also real time applications that
don't reduce load when loss is detected.  In order for there to be
enforceable fairness something like SFQ or some variant such as
cascaded SFQ is needed to better isolate flows.  SFQ just breaks the
queue up into groups of flows and many unlucky ones will end up in a
queue with a flow behaving badly.  In cascaded SFQ if any specific
queue SFQ is growing, then that queue is broken down further.  Depth
and/or total number of queue in cascaded SFQ is generally bounded but
the end result is to very well isolated poorly behaving flows with far
less queues than would be needed for one queue per flow.

No one AFAIK has tried to allow flows to pick a type of queue (small
queue vs deep).  For SFQ or cascaded SFQ it might be best if before
loss occurs high variation in delay and/or growing delay causes the
delay sensitive application to back off.  This way if fairness is
acheived, "bounded delay" may be better acheived without forcing a
small queue.  This may also be related but slightly off topic.

</slightly-off-topic>

Back to AQM.  Some forms of AQM do better at spcific criteria or do
better at finding a good tradeoff among the commonly cited criteria:
fairness, bulk transfer goodput, bounded delay.  Where the tradeoffs
should be set is maybe not a good thing for the AQM WG to try to
define as the optimal point will depend on perspective - on what mix
of services are being used and on which services are of greater
importance to a specific individual or organization.

So briefly, my answer is: as a WG, I don't think we want to go there.
If we do go there at all, then we should define "good AQM" in terms of
acheving a "good" tradeoff between fairness, bulk transfer goodput,
and bounded delay.  IMHO sometimes vague is better.

Curtis
