
From bob.briscoe@bt.com  Mon Nov  4 14:04:20 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B258721E8102; Mon,  4 Nov 2013 14:04:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.402
X-Spam-Level: 
X-Spam-Status: No, score=-3.402 tagged_above=-999 required=5 tests=[AWL=0.197,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NievnbU1iCCr; Mon,  4 Nov 2013 14:04:15 -0800 (PST)
Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id 4349821E80DB; Mon,  4 Nov 2013 14:03:54 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Mon, 4 Nov 2013 22:03:48 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 4 Nov 2013 22:03:50 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Mon, 4 Nov 2013 22:03:50 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.157.6])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA4M3lo0026458; Mon, 4 Nov 2013 22:03:48 GMT
Message-ID: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Mon, 4 Nov 2013 22:03:45 +0000
To: tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
From: Bob Briscoe <bob.briscoe@bt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org
Subject: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2013 22:04:20 -0000

Folks,

Pls respond if you support this being adopted as a work-group item in 
the IETF transport services w-g (tsvwg). The WG chairs need 
visibility of interest.
Even better, if you're willing to read / comment / review / implement

Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP
<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>

Abstract

    The purpose of this document is to guide the design of congestion
    notification in any lower layer or tunnelling protocol that
    encapsulates IP.  The aim is for explicit congestion signals to
    propagate consistently from lower layer protocols into IP.  Then the
    IP internetwork layer can act as a portability layer to carry
    congestion notification from non-IP-aware congested nodes up to the
    transport layer (L4).  Following these guidelines should assure
    interworking between new lower layer congestion notification
    mechanisms, whether specified by the IETF or other standards bodies.


[Cross-posting tsvwg & aqm, just in case]


Bob Briscoe,
also for co-authors Pat Thaler and John Kaippallimalil


________________________________________________________________
Bob Briscoe,                                                  BT 


From fred@cisco.com  Mon Nov  4 14:39:17 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AFAF521E80E1; Mon,  4 Nov 2013 14:39:16 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.419
X-Spam-Level: 
X-Spam-Status: No, score=-110.419 tagged_above=-999 required=5 tests=[AWL=-0.047, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NiN8pGH+rODs; Mon,  4 Nov 2013 14:39:11 -0800 (PST)
Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) by ietfa.amsl.com (Postfix) with ESMTP id DA25821E809F; Mon,  4 Nov 2013 14:39:10 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2874; q=dns/txt; s=iport; t=1383604751; x=1384814351; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=yR0YuyUT4sViaubdlhZfpFBlglGNt8cLShXpsuI9QUQ=; b=kntTcXNCapxBafXiCWLkp01QzyWlgNnl7qfnBSNicsYj+phl6NE+k/ng x6mZDnbiy9de8UPp7i8zhFUI0wnAcPomu6e898D/n+C6tLM50DpgvidZl qgb6txBGJSj9kfYu5IUdKE1FBNG3Kb0HYIeLOn48h8Lmfazb8aC3/ioq/ E=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AiQFAFcheFKtJXG//2dsb2JhbABZgwc4U74tgRKBJxZ0giUBAQEDAQEBASRHCwULAgEIRicLJQIEDgUOh20GDb5Gj1gHgyCBDgOQLoEwhiyBL5BagyaCKg
X-IronPort-AV: E=Sophos;i="4.93,635,1378857600";  d="asc'?scan'208";a="280408696"
Received: from rcdn-core2-4.cisco.com ([173.37.113.191]) by rcdn-iport-1.cisco.com with ESMTP; 04 Nov 2013 22:39:10 +0000
Received: from xhc-rcd-x04.cisco.com (xhc-rcd-x04.cisco.com [173.37.183.78]) by rcdn-core2-4.cisco.com (8.14.5/8.14.5) with ESMTP id rA4MdADP006852 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 4 Nov 2013 22:39:10 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-rcd-x04.cisco.com ([fe80::200:5efe:173.37.183.34%12]) with mapi id 14.03.0123.003; Mon, 4 Nov 2013 16:39:10 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO2a6s10fjb0a1UEK5OS95UGUypw==
Date: Mon, 4 Nov 2013 22:39:09 +0000
Message-ID: <AB7A26DF-2432-4320-B048-69070D31C7D6@cisco.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
In-Reply-To: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.21.88.9]
Content-Type: multipart/signed; boundary="Apple-Mail=_AD0D3F69-F2EF-4DCE-BA6B-C70B0DDB695C"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, "<draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2013 22:39:17 -0000

--Apple-Mail=_AD0D3F69-F2EF-4DCE-BA6B-C70B0DDB695C
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Nov 4, 2013, at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:

> Folks,
>=20
> Pls respond if you support this being adopted as a work-group item in =
the IETF transport services w-g (tsvwg). The WG chairs need visibility =
of interest.

And don't respond if we don't support it? :-)

I have a very basic question. For IP/IP tunnels, such as GRE, the ECN =
flags are defined, and I would imagine that RFC 2983 could be =
interpreted as saying what one does with flags (when you move the TOS =
octet, move 8 bits, whether in->out or out->in). For non-IP tunnels, =
with the exception of MPLS, the IETF doesn't define the header.

How does one add ECN to a protocol we don't maintain? If we maintain it, =
are we already done?

> Even better, if you're willing to read / comment / review / implement
>=20
> Guidelines for Adding Congestion Notification to Protocols that =
Encapsulate IP
> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>=20
> Abstract
>=20
>   The purpose of this document is to guide the design of congestion
>   notification in any lower layer or tunnelling protocol that
>   encapsulates IP.  The aim is for explicit congestion signals to
>   propagate consistently from lower layer protocols into IP.  Then the
>   IP internetwork layer can act as a portability layer to carry
>   congestion notification from non-IP-aware congested nodes up to the
>   transport layer (L4).  Following these guidelines should assure
>   interworking between new lower layer congestion notification
>   mechanisms, whether specified by the IETF or other standards bodies.
>=20
>=20
> [Cross-posting tsvwg & aqm, just in case]
>=20
>=20
> Bob Briscoe,
> also for co-authors Pat Thaler and John Kaippallimalil
>=20
>=20
> ________________________________________________________________
> Bob Briscoe,                                                  BT=20
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

------------------------------------------------------
8 issues in virtual infrastructure
http://dcrocker.net/#fallacies


--Apple-Mail=_AD0D3F69-F2EF-4DCE-BA6B-C70B0DDB695C
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSeCIKbjEdbHIsm0MRAuWJAJwPStWpPNbsiubCQAKy/cm6paiIVACgzoCa
IrBfQAS0x17h3ML0ddxCBc4=
=VdNX
-----END PGP SIGNATURE-----

--Apple-Mail=_AD0D3F69-F2EF-4DCE-BA6B-C70B0DDB695C--

From bob.briscoe@bt.com  Mon Nov  4 14:45:08 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 605C121E80E1; Mon,  4 Nov 2013 14:45:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.297
X-Spam-Level: 
X-Spam-Status: No, score=-3.297 tagged_above=-999 required=5 tests=[AWL=0.075,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N1XYvv8IamQO; Mon,  4 Nov 2013 14:44:58 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id BE2F921E80AF; Mon,  4 Nov 2013 14:44:56 -0800 (PST)
Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR67-UKRD.bt.com (10.187.101.22) with Microsoft SMTP Server (TLS) id 8.3.297.1; Mon, 4 Nov 2013 22:44:54 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Mon, 4 Nov 2013 22:44:52 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Mon, 4 Nov 2013 22:44:52 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.157.6])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA4MinXo026739; Mon, 4 Nov 2013 22:44:50 GMT
Message-ID: <201311042244.rA4MinXo026739@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Mon, 4 Nov 2013 22:44:48 +0000
To: "Fred Baker (fred)" <fred@cisco.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <AB7A26DF-2432-4320-B048-69070D31C7D6@cisco.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <AB7A26DF-2432-4320-B048-69070D31C7D6@cisco.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, "<draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2013 22:45:08 -0000

Fred,

At 22:39 04/11/2013, Fred Baker (fred) wrote:

>On Nov 4, 2013, at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:
>
> > Folks,
> >
> > Pls respond if you support this being adopted as a work-group 
> item in the IETF transport services w-g (tsvwg). The WG chairs need 
> visibility of interest.
>
>And don't respond if we don't support it? :-)
>
>I have a very basic question. For IP/IP tunnels, such as GRE, the 
>ECN flags are defined, and I would imagine that RFC 2983 could be 
>interpreted as saying what one does with flags (when you move the 
>TOS octet, move 8 bits, whether in->out or out->in). For non-IP 
>tunnels, with the exception of MPLS, the IETF doesn't define the header.
>
>How does one add ECN to a protocol we don't maintain? If we maintain 
>it, are we already done?

The idea is to issue liaison statements to make the relevant stds 
bodies aware of this. E.g.:
- to 3GPP for GTP,
- to IEEE for 802.

The IETF maintains a number of the relevant tunnelling protocols it 
mentions, e.g.
L2TP, GRE, PPTP.



________________________________________________________________
Bob Briscoe,                                                  BT 


From andrewmcgr@google.com  Mon Nov  4 15:04:08 2013
Return-Path: <andrewmcgr@google.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 615BD21E8217 for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 15:04:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.084
X-Spam-Level: 
X-Spam-Status: No, score=-0.084 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_HTML_USL_OBFU=1.666, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cyGVCT5s8YVI for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 15:04:07 -0800 (PST)
Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) by ietfa.amsl.com (Postfix) with ESMTP id 38CA021E81C6 for <aqm@ietf.org>; Mon,  4 Nov 2013 15:04:02 -0800 (PST)
Received: by mail-qa0-f48.google.com with SMTP id w8so634156qac.14 for <aqm@ietf.org>; Mon, 04 Nov 2013 15:04:01 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UgKzNxyw1zcSX2Bv5kGlsO5Fr76M4Ynb/z4RYYwowjc=; b=ZfZ4b0Pb2aOp/FO6O4173sXLhg9F/sU2FUlOS3WmlMxlwyUqJxKqR5cwhEiJ3B+ANO 1kmzEa1DktpwwyIrwBnRrJXuIvOalInzeQedsh3Ckz2FDr4RocJwVaRnkJPdGf7OCPpS WB1sj2xmPrnaC37SrPh79c4ymOh0hoOo2rm2bcI7uN6lrgVGMF/cyVGtjry5DMcP04d6 /QR5tZy5eSQo6EJiYNb7KxdRkjkyM+CIh0rrA4Zqh1aN7YhbXQJk4EUrXwPsg0lTp83V FnaVEISYzPwufdBmDH9wAz5nzp5hXVXCZMcscdCHISgoCoBBq6VNyqZiS2jH87kBAmDf hCzQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=UgKzNxyw1zcSX2Bv5kGlsO5Fr76M4Ynb/z4RYYwowjc=; b=Vkin2uNSdRjYRArQLgJXz+4tsM0KEsugcQM1dGb57TkpcO+qAalqa/ohlKR0xEkRti ooAZQfQYpEJL2J9Vu8vOLWPy7T1RibPJlj+FmYWo6kKJWrp9Yk0jnivpLbY5YGK2if8i BCJzbYnuiox82vO9c+hwKIxdpAtB/frQFq8r5nwAN1bCXbwBWyHyqaq+LRRyULCBqCtG 3Qcs+XKkcdcJpGBoWqGbuayCF0+6swnFIo3vfTiPy+sKhz5raAczUR9fMjeoERxSYQ1E JS8MkfYYvMaLLKKw98YkWlQfxyXmTArtGl6r7M23b5P9MubDn3HtWuaYx/Zkf7fkp6pA XZcg==
X-Gm-Message-State: ALoCoQnou5EGibyNaQarpVePYllOiEb3x7mZaPeGgHKom3a1ilJofQg4pwhVxgc09V3Iz9HGUEa8qGbzz6G9AFOqgkY+KiIoC0gduv+H1XnhVEa+YviyXUt2oTsmz726hoYfFUk/mnqB+44cNJSVsY8ukcXlbbhm6SClt5z5pQAN89ygR3z3mdB2v1P7Q6/bBfmmKRZu1ual
MIME-Version: 1.0
X-Received: by 10.224.22.75 with SMTP id m11mr25705540qab.27.1383606241488; Mon, 04 Nov 2013 15:04:01 -0800 (PST)
Received: by 10.224.197.198 with HTTP; Mon, 4 Nov 2013 15:04:01 -0800 (PST)
In-Reply-To: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
Date: Tue, 5 Nov 2013 10:04:01 +1100
Message-ID: <CAPRuP3kRsoPP2e9+vBjn1xiZ1UWJpc8Ds5VhsggKWa2suj_19Q@mail.gmail.com>
From: Andrew Mcgregor <andrewmcgr@google.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: multipart/alternative; boundary=047d7bdc78cc888b7b04ea61eed3
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2013 23:04:08 -0000

--047d7bdc78cc888b7b04ea61eed3
Content-Type: text/plain; charset=ISO-8859-1

This seems like valuable work.  One question is, can we put in scope
notification that losses are NOT due to congestion?

Willing to comment and review, at least.


On 5 November 2013 09:03, Bob Briscoe <bob.briscoe@bt.com> wrote:

> Folks,
>
> Pls respond if you support this being adopted as a work-group item in the
> IETF transport services w-g (tsvwg). The WG chairs need visibility of
> interest.
> Even better, if you're willing to read / comment / review / implement
>
> Guidelines for Adding Congestion Notification to Protocols that
> Encapsulate IP
> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>
> Abstract
>
>    The purpose of this document is to guide the design of congestion
>    notification in any lower layer or tunnelling protocol that
>    encapsulates IP.  The aim is for explicit congestion signals to
>    propagate consistently from lower layer protocols into IP.  Then the
>    IP internetwork layer can act as a portability layer to carry
>    congestion notification from non-IP-aware congested nodes up to the
>    transport layer (L4).  Following these guidelines should assure
>    interworking between new lower layer congestion notification
>    mechanisms, whether specified by the IETF or other standards bodies.
>
>
> [Cross-posting tsvwg & aqm, just in case]
>
>
> Bob Briscoe,
> also for co-authors Pat Thaler and John Kaippallimalil
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



-- 
Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128

--047d7bdc78cc888b7b04ea61eed3
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">This seems like valuable work. =A0One question is, can we =
put in scope notification that losses are NOT due to congestion?<div><br></=
div><div>Willing to comment and review, at least.</div></div><div class=3D"=
gmail_extra">
<br><br><div class=3D"gmail_quote">On 5 November 2013 09:03, Bob Briscoe <s=
pan dir=3D"ltr">&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank"=
>bob.briscoe@bt.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quo=
te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=
>
Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.<br>
Even better, if you&#39;re willing to read / comment / review / implement<b=
r>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP<br>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">http://tools.ietf.org/html/<u></u>draft-briscoe-=
tsvwg-ecn-encap-<u></u>guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
=A0 =A0The purpose of this document is to guide the design of congestion<br=
>
=A0 =A0notification in any lower layer or tunnelling protocol that<br>
=A0 =A0encapsulates IP. =A0The aim is for explicit congestion signals to<br=
>
=A0 =A0propagate consistently from lower layer protocols into IP. =A0Then t=
he<br>
=A0 =A0IP internetwork layer can act as a portability layer to carry<br>
=A0 =A0congestion notification from non-IP-aware congested nodes up to the<=
br>
=A0 =A0transport layer (L4). =A0Following these guidelines should assure<br=
>
=A0 =A0interworking between new lower layer congestion notification<br>
=A0 =A0mechanisms, whether specified by the IETF or other standards bodies.=
<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>_=
___<br>
Bob Briscoe, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BT <br>
______________________________<u></u>_________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/<u></u>listinfo/aqm</a><br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div dir=3D"=
ltr"><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:sm=
all;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-co=
lor:rgb(213,15,37);padding-top:2px;margin-top:2px">Andrew McGregor=A0|</spa=
n><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small=
;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color=
:rgb(51,105,232);padding-top:2px;margin-top:2px">=A0SRE=A0|</span><span sty=
le=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-heigh=
t:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,153,=
57);padding-top:2px;margin-top:2px">=A0<a href=3D"mailto:andrewmcgr@google.=
com" target=3D"_blank">andrewmcgr@google.com</a>=A0|</span><span style=3D"c=
olor:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em=
;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);p=
adding-top:2px;margin-top:2px">=A0+61 4 8143 7128</span><br>
</div>
</div>

--047d7bdc78cc888b7b04ea61eed3--

From mattmathis@google.com  Mon Nov  4 15:16:32 2013
Return-Path: <mattmathis@google.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8117321E8270 for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 15:16:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.779
X-Spam-Level: 
X-Spam-Status: No, score=-0.779 tagged_above=-999 required=5 tests=[AWL=-0.695, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_HTML_USL_OBFU=1.666, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yJ1zoCoOtO5c for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 15:16:31 -0800 (PST)
Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) by ietfa.amsl.com (Postfix) with ESMTP id 990C421E824E for <aqm@ietf.org>; Mon,  4 Nov 2013 15:16:31 -0800 (PST)
Received: by mail-ie0-f176.google.com with SMTP id u16so13957570iet.35 for <aqm@ietf.org>; Mon, 04 Nov 2013 15:16:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K97CLgUclEVmjrZo3fPN6I5oOKAiBcHfb/31J0lsIMk=; b=GEfL50c5gsiTmFf37WJPaxNE3fIS7xzDVdIsYOxh5BO8ugcpAlD+dnhQeyQJyDWUMK xMCByhaKRr+o8vwiLnw78pwi/Po6Isn3IU5W65POZbrdqPTzIFy0h3NnVklGee3cqOCt 2otNmKC52nfZMsPdXOYyMnbikEiUYnuSarNwBboOWuiGy/NcovWEYK2GIVn8y+mI/E8G XKXf+1Z0F6Bu96ZTNP7SaXYBOa7cwN8mJZImnbBtjGaiCLGYEzXkzEkmzY3CDS4nGcKX 3yefHCGA7yHFBb/OKOH1Th0VcRsBxsOt1L+xwd854LX2NuV+auARHF1oIyLDr7ckV4Xh igNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=K97CLgUclEVmjrZo3fPN6I5oOKAiBcHfb/31J0lsIMk=; b=QLdd0nwXQk4u/YFrcTRqgJzSm5FhfZ3hbO3A2e2meaomtFt141JqAQygbo1ZOS8TTf +c7Lx/LcZPCWMG7fseIVlRhtErOR++x+EHz56vS4yl4fLqCnDUyc7jY6RjVE+c8UzWq8 cfU8Hk7CxMOlBymkuAwW56cDAScnfAjIIepa+eZNKs+dY9q7eenl+PEA/TNmUf7KI5Xg GddhDN/7TKbRz1xixkYX2cWNMl+AXJv7g1v6RLSaR1He+UPgLxt/1T2GDsgPjpDKUDs2 lq/AFe2OpAY8UGg0KlJvcZ4vc3AxVRbWbP+v4CrRYMha7WwUBdLQRhYT7Hiz3Apn2250 7JTg==
X-Gm-Message-State: ALoCoQl+g5AfHnuGP5+3C8V460N85DarfQYeDJBR6gkBWQqbNF3306KDDluR5SuHOz0CMyKfNDxjXWODkroxB0WIElkIoEMwJNnEztn8IPOnb5XqVZ9nGm18r+H26S7XZk6shLt+ka/VcrGyGyazrUPspHTXooBMomR+liKFCV0ALyvjBnPJa60N2TrpbPGZM/ZNA3E7I9NZ
MIME-Version: 1.0
X-Received: by 10.43.158.10 with SMTP id ls10mr11482691icc.20.1383606990999; Mon, 04 Nov 2013 15:16:30 -0800 (PST)
Received: by 10.64.243.66 with HTTP; Mon, 4 Nov 2013 15:16:30 -0800 (PST)
In-Reply-To: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
Date: Mon, 4 Nov 2013 15:16:30 -0800
Message-ID: <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
From: Matt Mathis <mattmathis@google.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: multipart/alternative; boundary=001a11c1ead6352a2604ea621b57
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2013 23:16:32 -0000

--001a11c1ead6352a2604ea621b57
Content-Type: text/plain; charset=ISO-8859-1

I think this is valuable work.  Having a single document that describes the
requirements and general principles will save future tunnel
inventor/implementers from rediscovering the same bugs

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our
services to speak in defiance of unjust governments.   We treat privacy and
security as matters of life and death, because for some users, they are.


On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com> wrote:

> Folks,
>
> Pls respond if you support this being adopted as a work-group item in the
> IETF transport services w-g (tsvwg). The WG chairs need visibility of
> interest.
> Even better, if you're willing to read / comment / review / implement
>
> Guidelines for Adding Congestion Notification to Protocols that
> Encapsulate IP
> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>
> Abstract
>
>    The purpose of this document is to guide the design of congestion
>    notification in any lower layer or tunnelling protocol that
>    encapsulates IP.  The aim is for explicit congestion signals to
>    propagate consistently from lower layer protocols into IP.  Then the
>    IP internetwork layer can act as a portability layer to carry
>    congestion notification from non-IP-aware congested nodes up to the
>    transport layer (L4).  Following these guidelines should assure
>    interworking between new lower layer congestion notification
>    mechanisms, whether specified by the IETF or other standards bodies.
>
>
> [Cross-posting tsvwg & aqm, just in case]
>
>
> Bob Briscoe,
> also for co-authors Pat Thaler and John Kaippallimalil
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>

--001a11c1ead6352a2604ea621b57
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I think this is valuable work. =A0Having a single document=
 that describes the requirements and general principles will save future tu=
nnel inventor/implementers from rediscovering the same bugs<br><div class=
=3D"gmail_extra">
<br clear=3D"all"><div><div dir=3D"ltr"><div>Thanks,</div>--MM--<br>The bes=
t way to predict the future is to create it. =A0- Alan Kay<br><br>Privacy m=
atters! =A0We know from recent events that people are using our services to=
 speak in defiance of unjust governments. =A0 We treat privacy and security=
 as matters of life and death, because for some users, they are.</div>
</div>
<br><br><div class=3D"gmail_quote">On Mon, Nov 4, 2013 at 2:03 PM, Bob Bris=
coe <span dir=3D"ltr">&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_=
blank">bob.briscoe@bt.com</a>&gt;</span> wrote:<br><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex">
Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.<br>
Even better, if you&#39;re willing to read / comment / review / implement<b=
r>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP<br>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">http://tools.ietf.org/html/<u></u>draft-briscoe-=
tsvwg-ecn-encap-<u></u>guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
=A0 =A0The purpose of this document is to guide the design of congestion<br=
>
=A0 =A0notification in any lower layer or tunnelling protocol that<br>
=A0 =A0encapsulates IP. =A0The aim is for explicit congestion signals to<br=
>
=A0 =A0propagate consistently from lower layer protocols into IP. =A0Then t=
he<br>
=A0 =A0IP internetwork layer can act as a portability layer to carry<br>
=A0 =A0congestion notification from non-IP-aware congested nodes up to the<=
br>
=A0 =A0transport layer (L4). =A0Following these guidelines should assure<br=
>
=A0 =A0interworking between new lower layer congestion notification<br>
=A0 =A0mechanisms, whether specified by the IETF or other standards bodies.=
<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>_=
___<br>
Bob Briscoe, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BT <br>
______________________________<u></u>_________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/<u></u>listinfo/aqm</a><br>
</blockquote></div><br></div></div>

--001a11c1ead6352a2604ea621b57--

From g.white@CableLabs.com  Mon Nov  4 15:20:58 2013
Return-Path: <g.white@CableLabs.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0BD2B11E821A for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 15:20:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.235
X-Spam-Level: 
X-Spam-Status: No, score=-0.235 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IgzvCdlof183 for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 15:20:50 -0800 (PST)
Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id DDA8811E8200 for <aqm@ietf.org>; Mon,  4 Nov 2013 15:20:49 -0800 (PST)
Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id rA4NKls9013011; Mon, 4 Nov 2013 16:20:47 -0700
Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Mon, 04 Nov 2013 16:20:46 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com)
Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Mon, 4 Nov 2013 16:20:46 -0700
From: Greg White <g.white@CableLabs.com>
To: Aaron Wood <woody77@gmail.com>, bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: [Bloat] [aqm] DOCSIS 3.1 support for AQM
Thread-Index: AQHO1hLGwCMHKHWaMUWxuDtUWCG/VZoPMUmAgAZ6x4A=
Date: Mon, 4 Nov 2013 23:20:46 +0000
Message-ID: <CE9D7773.2152C%g.white@cablelabs.com>
In-Reply-To: <CALQXh-NBWoeCOt_CS65c3NKM3p_cs19a+BjBUi+qzb2j8UF7=w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.5.0.27]
Content-Type: multipart/alternative; boundary="_000_CE9D77732152Cgwhitecablelabscom_"
MIME-Version: 1.0
X-Approved: ondar
Subject: Re: [aqm] [Bloat]  DOCSIS 3.1 support for AQM
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2013 23:20:58 -0000

--_000_CE9D77732152Cgwhitecablelabscom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Rong will cover this at a high-level during the IETF AQM session tomorrow. =
 Her slides are posted:

?www.ietf.org/proceedings/88/slides/slides-88-aqm-1.pdf<http://www.ietf.org=
/proceedings/88/slides/slides-88-aqm-1.pdf>

I plan to do a follow-up to the paper you linked to below to give some of t=
he details.  Should be ready before IETF89.

-Greg

From: Aaron Wood <woody77@gmail.com<mailto:woody77@gmail.com>>
Date: Thursday, October 31, 2013 at 7:23 AM
To: bloat <bloat@lists.bufferbloat.net<mailto:bloat@lists.bufferbloat.net>>
Subject: Re: [Bloat] [aqm] DOCSIS 3.1 support for AQM


Thanks for the information. I'd be interested in why you have chosen
PIE, e.g., instead of sfq-CoDel. Any pointers to evaluation
reports/results? Last time I saw a presentation on this it seemed
that CoDel was performing quite well.

I think this cablelabs report makes the argument for PIE:

http://www.cablelabs.com/downloads/pubs/Active_Queue_Management_Algorithms_=
DOCSIS_3_0.pdf

Mostly in that in the heavy traffic scenarios, PIE outperforms sfq_codel, a=
nd in general is a tad bit better than codel, with a simpler implementation=
 (I think).  Although I think I take issue with the "heavy traffic" model, =
but I'm guessing (hoping) that it's based on surveys of customer traffic.  =
60-110 upstream flows seems like a lot.  But it's based around a heavy use =
of BitTorrent, so maybe that's reasonable for some people.

But in all other cases, sfq really blows the doors off of the others.

-Aaron

--_000_CE9D77732152Cgwhitecablelabscom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <5A9D96C8D6D39B4FA8B99E077EBA7FB8@cablelabs.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-fami=
ly: Calibri, sans-serif;">
<div>Rong will cover this at a high-level during the IETF AQM session tomor=
row. &nbsp;Her slides are posted:</div>
<div><br>
</div>
<div><a href=3D"http://www.ietf.org/proceedings/88/slides/slides-88-aqm-1.p=
df">?www.ietf.org/proceedings/88/slides/slides-88-aqm-1.pdf</a></div>
<div><br>
</div>
<div>I plan to do a follow-up to the paper you linked to below to give some=
 of the details. &nbsp;Should be ready before IETF89.</div>
<div><br>
</div>
<div>-Greg</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div style=3D"font-family:Calibri; font-size:11pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>Aaron Wood &lt;<a href=3D"mai=
lto:woody77@gmail.com">woody77@gmail.com</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Thursday, October 31, 2013 at=
 7:23 AM<br>
<span style=3D"font-weight:bold">To: </span>bloat &lt;<a href=3D"mailto:blo=
at@lists.bufferbloat.net">bloat@lists.bufferbloat.net</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span>Re: [Bloat] [aqm] DOCSIS 3=
.1 support for AQM<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir=3D"ltr"><br>
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
Thanks for the information. I'd be interested in why you have chosen<br>
PIE, e.g., instead of sfq-CoDel. Any pointers to evaluation<br>
reports/results? Last time I saw a presentation on this it seemed<br>
that CoDel was performing quite well.<br>
</blockquote>
<div><br>
</div>
<div>I think this cablelabs report makes the argument for PIE:</div>
<div><br>
</div>
<div><a href=3D"http://www.cablelabs.com/downloads/pubs/Active_Queue_Manage=
ment_Algorithms_DOCSIS_3_0.pdf">http://www.cablelabs.com/downloads/pubs/Act=
ive_Queue_Management_Algorithms_DOCSIS_3_0.pdf</a></div>
<div><br>
</div>
<div>Mostly in that in the heavy traffic scenarios, PIE outperforms sfq_cod=
el, and in general is a tad bit better than codel, with a simpler implement=
ation (I think). &nbsp;Although I think I take issue with the &quot;heavy t=
raffic&quot; model, but I'm guessing (hoping) that
 it's based on surveys of customer traffic. &nbsp;60-110 upstream flows see=
ms like a lot. &nbsp;But it's based around a heavy use of BitTorrent, so ma=
ybe that's reasonable for some people.</div>
<div><br>
</div>
<div>But in all other cases, sfq really blows the doors off of the others.<=
/div>
<div><br>
</div>
<div>-Aaron</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>

--_000_CE9D77732152Cgwhitecablelabscom_--

From bob.briscoe@bt.com  Mon Nov  4 16:00:42 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C1E3A11E82CA; Mon,  4 Nov 2013 16:00:42 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.808
X-Spam-Level: 
X-Spam-Status: No, score=-2.808 tagged_above=-999 required=5 tests=[AWL=-0.436, BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GCIK6iTaFY6S; Mon,  4 Nov 2013 16:00:37 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id 871D511E8105; Mon,  4 Nov 2013 16:00:33 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR66-UKRD.bt.com (10.187.101.21) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 5 Nov 2013 00:00:31 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Tue, 5 Nov 2013 00:00:31 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Tue, 5 Nov 2013 00:00:30 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.157.6])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA500RMV027031; Tue, 5 Nov 2013 00:00:27 GMT
Message-ID: <201311050000.rA500RMV027031@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 5 Nov 2013 00:00:26 +0000
To: Andrew Mcgregor <andrewmcgr@google.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAPRuP3kRsoPP2e9+vBjn1xiZ1UWJpc8Ds5VhsggKWa2suj_19Q@mail.g mail.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAPRuP3kRsoPP2e9+vBjn1xiZ1UWJpc8Ds5VhsggKWa2suj_19Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=====================_490245263==.ALT"
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 00:00:42 -0000

--=====================_490245263==.ALT
Content-Type: text/plain; charset="us-ascii"; format=flowed

Andrew,

Thx for supporting words and offer of help.

At 23:04 04/11/2013, Andrew Mcgregor wrote:
>This seems like valuable work.  One question is, can we put in scope 
>notification that losses are NOT due to congestion?

Eyooow. That really does not belong in this scope.

This is about the interface between lower layers and the ECN field in 
the IP header. We can't really talk about new semantics in lower 
layers that aren't even defined in ECN at the IP layer.

Naively, one could imagine some text treating the ECN field as 4 
opaque codepoints, and propagating  a similar 4 opaque codepoints 
from each lower layer protocol. Then if something new were defined in 
IP-ECN like a non-congestive loss codepoint, it could also be defined 
in lower layers...

But that's naive, because each lower layer has it's own constraints 
and ways of handling concepts like "ECN-capable transport", none of 
which easily translates to a simple mapping of codepoints.

Explicit signalling of non-congestive loss is a research problem, because...
* Clearly a link cannot signal that a loss was not due to congestion 
using the lost frame itself (there aren't any bits :)
* So the link would have to signal in another frame that didn't get corrupted.
   - That frame may go to another process that didn't see any loss; 
then what does the process do?
   - or the link has to use DPI to signal only in frames going to the 
same process as the lost frame.
   - that means the link has to understand all transport protocols

There's other hard problems
* A transmission loss doesn't always mean there is no congestion loss
* At my last count, there were at least nine different reasons for 
packet loss. If we had 4 bits spare in each lost packet... :)



Bob


>Willing to comment and review, at least.
>
>
>On 5 November 2013 09:03, Bob Briscoe 
><<mailto:bob.briscoe@bt.com>bob.briscoe@bt.com> wrote:
>Folks,
>
>Pls respond if you support this being adopted as a work-group item 
>in the IETF transport services w-g (tsvwg). The WG chairs need 
>visibility of interest.
>Even better, if you're willing to read / comment / review / implement
>
>Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP
><<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>
>Abstract
>
>    The purpose of this document is to guide the design of congestion
>    notification in any lower layer or tunnelling protocol that
>    encapsulates IP.  The aim is for explicit congestion signals to
>    propagate consistently from lower layer protocols into IP.  Then the
>    IP internetwork layer can act as a portability layer to carry
>    congestion notification from non-IP-aware congested nodes up to the
>    transport layer (L4).  Following these guidelines should assure
>    interworking between new lower layer congestion notification
>    mechanisms, whether specified by the IETF or other standards bodies.
>
>
>[Cross-posting tsvwg & aqm, just in case]
>
>
>Bob Briscoe,
>also for co-authors Pat Thaler and John Kaippallimalil
>
>
>________________________________________________________________
>Bob Briscoe,                                                  BT
>_______________________________________________
>aqm mailing list
><mailto:aqm@ietf.org>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm
>
>
>
>
>--
>Andrew McGregor | SRE | 
><mailto:andrewmcgr@google.com>andrewmcgr@google.com | +61 4 8143 7128
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 
--=====================_490245263==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
Andrew,<br><br>
Thx for supporting words and offer of help.<br><br>
At 23:04 04/11/2013, Andrew Mcgregor wrote:<br>
<blockquote type=cite class=cite cite="">This seems like valuable
work.&nbsp; One question is, can we put in scope notification that losses
are NOT due to congestion?</blockquote><br>
Eyooow. That really does not belong in this scope. <br><br>
This is about the interface between lower layers and the ECN field in the
IP header. We can't really talk about new semantics in lower layers that
aren't even defined in ECN at the IP layer.<br><br>
Naively, one could imagine some text treating the ECN field as 4 opaque
codepoints, and propagating&nbsp; a similar 4 opaque codepoints from each
lower layer protocol. Then if something new were defined in IP-ECN like a
non-congestive loss codepoint, it could also be defined in lower
layers...<br><br>
But that's naive, because each lower layer has it's own constraints and
ways of handling concepts like &quot;ECN-capable transport&quot;, none of
which easily translates to a simple mapping of codepoints.<br><br>
Explicit signalling of non-congestive loss is a research problem,
because...<br>
* Clearly a link cannot signal that a loss was not due to congestion
using the lost frame itself (there aren't any bits :)<br>
* So the link would have to signal in another frame that didn't get
corrupted.<br>
&nbsp; - That frame may go to another process that didn't see any loss;
then what does the process do?<br>
&nbsp; - or the link has to use DPI to signal only in frames going to the
same process as the lost frame.<br>
&nbsp; - that means the link has to understand all transport
protocols<br><br>
There's other hard problems<br>
* A transmission loss doesn't always mean there is no congestion
loss<br>
* At my last count, there were at least nine different reasons for packet
loss. If we had 4 bits spare in each lost packet... :)<br><br>
<br><br>
Bob<br><br>
<br>
<blockquote type=cite class=cite cite="">Willing to comment and review,
at least.<br><br>
<br>
On 5 November 2013 09:03, Bob Briscoe
&lt;<a href="mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;
wrote:<br>

<dl>
<dd>Folks,<br><br>

<dd>Pls respond if you support this being adopted as a work-group item in
the IETF transport services w-g (tsvwg). The WG chairs need visibility of
interest.<br>

<dd>Even better, if you're willing to read / comment / review /
implement<br><br>

<dd>Guidelines for Adding Congestion Notification to Protocols that
Encapsulate IP<br>

<dd>
&lt;<a href="http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines">
http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines</a>
&gt;<br><br>

<dd>Abstract<br><br>

<dd>&nbsp;&nbsp; The purpose of this document is to guide the design of
congestion<br>

<dd>&nbsp;&nbsp; notification in any lower layer or tunnelling protocol
that<br>

<dd>&nbsp;&nbsp; encapsulates IP.&nbsp; The aim is for explicit
congestion signals to<br>

<dd>&nbsp;&nbsp; propagate consistently from lower layer protocols into
IP.&nbsp; Then the<br>

<dd>&nbsp;&nbsp; IP internetwork layer can act as a portability layer to
carry<br>

<dd>&nbsp;&nbsp; congestion notification from non-IP-aware congested
nodes up to the<br>

<dd>&nbsp;&nbsp; transport layer (L4).&nbsp; Following these guidelines
should assure<br>

<dd>&nbsp;&nbsp; interworking between new lower layer congestion
notification<br>

<dd>&nbsp;&nbsp; mechanisms, whether specified by the IETF or other
standards bodies.<br><br>
<br>

<dd>[Cross-posting tsvwg &amp; aqm, just in case]<br><br>
<br>

<dd>Bob Briscoe,<br>

<dd>also for co-authors Pat Thaler and John Kaippallimalil<br><br>
<br>

<dd>________________________________________________________________<br>

<dd>Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT <br>

<dd>_______________________________________________<br>

<dd>aqm mailing list<br>

<dd><a href="mailto:aqm@ietf.org">aqm@ietf.org</a><br>

<dd><a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dl><br><br>
<br>
-- <br>
Andrew McGregor | SRE |
<a href="mailto:andrewmcgr@google.com">andrewmcgr@google.com</a> | +61 4
8143 7128<br>
_______________________________________________<br>
aqm mailing list<br>
aqm@ietf.org<br>
<a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<x-sigsep><p></x-sigsep>
________________________________________________________________<br>
Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT</body>
</html>

--=====================_490245263==.ALT--

From michawe@ifi.uio.no  Mon Nov  4 16:02:54 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01B2321E8318; Mon,  4 Nov 2013 16:02:54 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -101.652
X-Spam-Level: 
X-Spam-Status: No, score=-101.652 tagged_above=-999 required=5 tests=[AWL=-0.947, BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_HTML_USL_OBFU=1.666, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G-3it9XUWY6O; Mon,  4 Nov 2013 16:02:49 -0800 (PST)
Received: from mail-out5.uio.no (mail-out5.uio.no [IPv6:2001:700:100:10::17]) by ietfa.amsl.com (Postfix) with ESMTP id 6F02B21E833A; Mon,  4 Nov 2013 16:02:44 -0800 (PST)
Received: from mail-mx4.uio.no ([129.240.10.45]) by mail-out5.uio.no with esmtp (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1VdU6S-0001CB-W1; Tue, 05 Nov 2013 01:02:40 +0100
Received: from dhcp-9c73.meeting.ietf.org ([31.133.156.115]) by mail-mx4.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from <michawe@ifi.uio.no>) id 1VdU6R-0004gn-NB; Tue, 05 Nov 2013 01:02:40 +0100
Content-Type: multipart/alternative; boundary="Apple-Mail=_CDB986EB-EC31-41E4-8F39-FF55DFD9A5CD"
Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\))
From: Michael Welzl <michawe@ifi.uio.no>
In-Reply-To: <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
Date: Mon, 4 Nov 2013 16:02:32 -0800
Message-Id: <0D37F652-8259-4A03-AA1F-4889221D84BD@ifi.uio.no>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
To: Bob Briscoe <bob.briscoe@bt.com>
X-Mailer: Apple Mail (2.1510)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 4 msgs/h 1 sum rcpts/h 9 sum msgs/h 5 total rcpts 9380 max rcpts/h 40 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: F87095E641CC2FDC768DC1C3634DD0D2BDEA6645
X-UiO-SPAM-Test: remote_host: 31.133.156.115 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 21 max/h 8 blacklist 0 greylist 0 ratelimit 0
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 00:02:54 -0000

--Apple-Mail=_CDB986EB-EC31-41E4-8F39-FF55DFD9A5CD
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1

+1


On 4. nov. 2013, at 15:16, Matt Mathis <mattmathis@google.com> wrote:

> I think this is valuable work.  Having a single document that =
describes the requirements and general principles will save future =
tunnel inventor/implementers from rediscovering the same bugs
>=20
> Thanks,
> --MM--
> The best way to predict the future is to create it.  - Alan Kay
>=20
> Privacy matters!  We know from recent events that people are using our =
services to speak in defiance of unjust governments.   We treat privacy =
and security as matters of life and death, because for some users, they =
are.
>=20
>=20
> On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com> =
wrote:
> Folks,
>=20
> Pls respond if you support this being adopted as a work-group item in =
the IETF transport services w-g (tsvwg). The WG chairs need visibility =
of interest.
> Even better, if you're willing to read / comment / review / implement
>=20
> Guidelines for Adding Congestion Notification to Protocols that =
Encapsulate IP
> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>=20
> Abstract
>=20
>    The purpose of this document is to guide the design of congestion
>    notification in any lower layer or tunnelling protocol that
>    encapsulates IP.  The aim is for explicit congestion signals to
>    propagate consistently from lower layer protocols into IP.  Then =
the
>    IP internetwork layer can act as a portability layer to carry
>    congestion notification from non-IP-aware congested nodes up to the
>    transport layer (L4).  Following these guidelines should assure
>    interworking between new lower layer congestion notification
>    mechanisms, whether specified by the IETF or other standards =
bodies.
>=20
>=20
> [Cross-posting tsvwg & aqm, just in case]
>=20
>=20
> Bob Briscoe,
> also for co-authors Pat Thaler and John Kaippallimalil
>=20
>=20
> ________________________________________________________________
> Bob Briscoe,                                                  BT=20
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>=20
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm


--Apple-Mail=_CDB986EB-EC31-41E4-8F39-FF55DFD9A5CD
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
	charset=iso-8859-1

<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">+1<div><br></div><div><br><div><div>On 4. nov. 2013, at 15:16, Matt Mathis &lt;<a href="mailto:mattmathis@google.com">mattmathis@google.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">I think this is valuable work. &nbsp;Having a single document that describes the requirements and general principles will save future tunnel inventor/implementers from rediscovering the same bugs<br><div class="gmail_extra">
<br clear="all"><div><div dir="ltr"><div>Thanks,</div>--MM--<br>The best way to predict the future is to create it. &nbsp;- Alan Kay<br><br>Privacy matters! &nbsp;We know from recent events that people are using our services to speak in defiance of unjust governments. &nbsp; We treat privacy and security as matters of life and death, because for some users, they are.</div>
</div>
<br><br><div class="gmail_quote">On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <span dir="ltr">&lt;<a href="mailto:bob.briscoe@bt.com" target="_blank">bob.briscoe@bt.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the IETF transport services w-g (tsvwg). The WG chairs need visibility of interest.<br>
Even better, if you're willing to read / comment / review / implement<br>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP<br>
&lt;<a href="http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines" target="_blank">http://tools.ietf.org/html/<u></u>draft-briscoe-tsvwg-ecn-encap-<u></u>guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
&nbsp; &nbsp;The purpose of this document is to guide the design of congestion<br>
&nbsp; &nbsp;notification in any lower layer or tunnelling protocol that<br>
&nbsp; &nbsp;encapsulates IP. &nbsp;The aim is for explicit congestion signals to<br>
&nbsp; &nbsp;propagate consistently from lower layer protocols into IP. &nbsp;Then the<br>
&nbsp; &nbsp;IP internetwork layer can act as a portability layer to carry<br>
&nbsp; &nbsp;congestion notification from non-IP-aware congested nodes up to the<br>
&nbsp; &nbsp;transport layer (L4). &nbsp;Following these guidelines should assure<br>
&nbsp; &nbsp;interworking between new lower layer congestion notification<br>
&nbsp; &nbsp;mechanisms, whether specified by the IETF or other standards bodies.<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>____<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT <br>
______________________________<u></u>_________________<br>
aqm mailing list<br>
<a href="mailto:aqm@ietf.org" target="_blank">aqm@ietf.org</a><br>
<a href="https://www.ietf.org/mailman/listinfo/aqm" target="_blank">https://www.ietf.org/mailman/<u></u>listinfo/aqm</a><br>
</blockquote></div><br></div></div>
_______________________________________________<br>aqm mailing list<br><a href="mailto:aqm@ietf.org">aqm@ietf.org</a><br>https://www.ietf.org/mailman/listinfo/aqm<br></blockquote></div><br></div></body></html>
--Apple-Mail=_CDB986EB-EC31-41E4-8F39-FF55DFD9A5CD--

From rs@netapp.com  Mon Nov  4 16:49:08 2013
Return-Path: <rs@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9F43721E836B; Mon,  4 Nov 2013 16:49:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.288
X-Spam-Level: 
X-Spam-Status: No, score=-5.288 tagged_above=-999 required=5 tests=[AWL=-2.917, BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3TOxQXRXWICE; Mon,  4 Nov 2013 16:49:04 -0800 (PST)
Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 89E6F11E82FF; Mon,  4 Nov 2013 16:49:02 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,636,1378882800"; d="scan'208,217";a="69544373"
Received: from vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) by mx11-out.netapp.com with ESMTP; 04 Nov 2013 16:48:44 -0800
Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.86]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.03.0123.003; Mon, 4 Nov 2013 16:48:44 -0800
From: "Scheffenegger, Richard" <rs@netapp.com>
To: Matt Mathis <mattmathis@google.com>, Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel	protocols?
Thread-Index: AQHO2bPschNRr0CscEy8XBGE9jPQ05oVzbZA
Date: Tue, 5 Nov 2013 00:48:43 +0000
Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25E65C8E@SACEXCMBX02-PRD.hq.netapp.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
In-Reply-To: <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
Accept-Language: de-AT, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.104.60.117]
Content-Type: multipart/alternative; boundary="_000_012C3117EDDB3C4781FD802A8C27DD4F25E65C8ESACEXCMBX02PRDh_"
MIME-Version: 1.0
Cc: "draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel	protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 00:49:08 -0000

--_000_012C3117EDDB3C4781FD802A8C27DD4F25E65C8ESACEXCMBX02PRDh_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

(aqm chair hat off)

I can fully support what Matt just stated!

I have read an earlier version (-01 iirc), I'll try to scan over this versi=
on in the coming days.


Richard Scheffenegger


From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of Matt =
Mathis
Sent: Montag, 04. November 2013 15:17
To: Bob Briscoe
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg IETF lis=
t; AQM IETF list
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunne=
l protocols?

I think this is valuable work.  Having a single document that describes the=
 requirements and general principles will save future tunnel inventor/imple=
menters from rediscovering the same bugs

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our serv=
ices to speak in defiance of unjust governments.   We treat privacy and sec=
urity as matters of life and death, because for some users, they are.

On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com<mailto:bob.=
briscoe@bt.com>> wrote:
Folks,

Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.
Even better, if you're willing to read / comment / review / implement

Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP
<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>

Abstract

   The purpose of this document is to guide the design of congestion
   notification in any lower layer or tunnelling protocol that
   encapsulates IP.  The aim is for explicit congestion signals to
   propagate consistently from lower layer protocols into IP.  Then the
   IP internetwork layer can act as a portability layer to carry
   congestion notification from non-IP-aware congested nodes up to the
   transport layer (L4).  Following these guidelines should assure
   interworking between new lower layer congestion notification
   mechanisms, whether specified by the IETF or other standards bodies.


[Cross-posting tsvwg & aqm, just in case]


Bob Briscoe,
also for co-authors Pat Thaler and John Kaippallimalil


________________________________________________________________
Bob Briscoe,                                                  BT
_______________________________________________
aqm mailing list
aqm@ietf.org<mailto:aqm@ietf.org>
https://www.ietf.org/mailman/listinfo/aqm


--_000_012C3117EDDB3C4781FD802A8C27DD4F25E65C8ESACEXCMBX02PRDh_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";
	mso-fareast-language:EN-US;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"DE-AT" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D">(aqm chair=
 hat off)<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp=
;</o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I can full=
y support what Matt just stated!
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp=
;</o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I have rea=
d an earlier version (-01 iirc), I&#8217;ll try to scan over this version i=
n the coming days.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp=
;</o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:11.0pt;font-=
family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp=
;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;;color:#1F497D">Richard Scheffenegger</sp=
an><span style=3D"font-size:10.0pt;font-family:&quot;Calibri&quot;,&quot;sa=
ns-serif&quot;;color:#1F497D"><br>
<br>
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span><=
/p>
<div style=3D"border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm =
4.0pt">
<div>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm =
0cm 0cm">
<p class=3D"MsoNormal"><b><span lang=3D"EN-US" style=3D"font-size:10.0pt;fo=
nt-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span></b><span =
lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&qu=
ot;sans-serif&quot;"> aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org]
<b>On Behalf Of </b>Matt Mathis<br>
<b>Sent:</b> Montag, 04. November 2013 15:17<br>
<b>To:</b> Bob Briscoe<br>
<b>Cc:</b> draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg I=
ETF list; AQM IETF list<br>
<b>Subject:</b> Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 o=
r tunnel protocols?<o:p></o:p></span></p>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<p class=3D"MsoNormal">I think this is valuable work. &nbsp;Having a single=
 document that describes the requirements and general principles will save =
future tunnel inventor/implementers from rediscovering the same bugs<o:p></=
o:p></p>
<div>
<p class=3D"MsoNormal"><br clear=3D"all">
<o:p></o:p></p>
<div>
<div>
<div>
<p class=3D"MsoNormal">Thanks,<o:p></o:p></p>
</div>
<p class=3D"MsoNormal">--MM--<br>
The best way to predict the future is to create it. &nbsp;- Alan Kay<br>
<br>
Privacy matters! &nbsp;We know from recent events that people are using our=
 services to speak in defiance of unjust governments. &nbsp; We treat priva=
cy and security as matters of life and death, because for some users, they =
are.<o:p></o:p></p>
</div>
</div>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><o:p>&nbsp;</o:p></p>
<div>
<p class=3D"MsoNormal">On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe &lt;<a h=
ref=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.com</a>&=
gt; wrote:<o:p></o:p></p>
<p class=3D"MsoNormal">Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.<br>
Even better, if you're willing to read / comment / review / implement<br>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP<br>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">http://tools.ietf.org/html/draft-briscoe-tsvwg-e=
cn-encap-guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
&nbsp; &nbsp;The purpose of this document is to guide the design of congest=
ion<br>
&nbsp; &nbsp;notification in any lower layer or tunnelling protocol that<br=
>
&nbsp; &nbsp;encapsulates IP. &nbsp;The aim is for explicit congestion sign=
als to<br>
&nbsp; &nbsp;propagate consistently from lower layer protocols into IP. &nb=
sp;Then the<br>
&nbsp; &nbsp;IP internetwork layer can act as a portability layer to carry<=
br>
&nbsp; &nbsp;congestion notification from non-IP-aware congested nodes up t=
o the<br>
&nbsp; &nbsp;transport layer (L4). &nbsp;Following these guidelines should =
assure<br>
&nbsp; &nbsp;interworking between new lower layer congestion notification<b=
r>
&nbsp; &nbsp;mechanisms, whether specified by the IETF or other standards b=
odies.<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
<br>
________________________________________________________________<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;=
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbs=
p; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT <br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><o:p></o:p></p>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>

--_000_012C3117EDDB3C4781FD802A8C27DD4F25E65C8ESACEXCMBX02PRDh_--

From weixinpeng@huawei.com  Mon Nov  4 17:08:54 2013
Return-Path: <weixinpeng@huawei.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A86211E8316; Mon,  4 Nov 2013 17:08:54 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.598
X-Spam-Level: 
X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=0.001,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id McFbWub1Eir8; Mon,  4 Nov 2013 17:08:49 -0800 (PST)
Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id 0E08611E8300; Mon,  4 Nov 2013 17:08:48 -0800 (PST)
Received: from 172.18.7.190 (EHLO lhreml204-edg.china.huawei.com) ([172.18.7.190]) by lhrrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id AZV94252; Tue, 05 Nov 2013 01:08:47 +0000 (GMT)
Received: from LHREML405-HUB.china.huawei.com (10.201.5.242) by lhreml204-edg.china.huawei.com (172.18.7.223) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 5 Nov 2013 01:08:09 +0000
Received: from NKGEML403-HUB.china.huawei.com (10.98.56.34) by lhreml405-hub.china.huawei.com (10.201.5.242) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 5 Nov 2013 01:08:44 +0000
Received: from NKGEML507-MBX.china.huawei.com ([169.254.5.4]) by nkgeml403-hub.china.huawei.com ([10.98.56.34]) with mapi id 14.03.0158.001; Tue, 5 Nov 2013 09:08:39 +0800
From: Weixinpeng <weixinpeng@huawei.com>
To: "tsvwg@ietf.org" <tsvwg@ietf.org>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO2ar1vFFB2XF4BkqGsTlMgx3rlZoV0TJQ
Date: Tue, 5 Nov 2013 01:08:38 +0000
Message-ID: <C5C3BB522B1DDF478AA09545169155B43DA963DB@nkgeml507-mbx.china.huawei.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.111.76.176]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 01:08:54 -0000

I look through the whole document and I really think it's a good idea to pr=
opagate congestion signals through different networks,
it will be good to deal with the congestion situation especially when there=
 are lower layer en route.



> >Date: Mon, 04 Nov 2013 22:03:45 +0000
> >To: "tsvwg IETF list" <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
> >From: Bob Briscoe <bob.briscoe@bt.com>
> >Subject: Who supports tsvwg adoption of adding ECN to L2 or tunnel
> protocols?
> >Cc: John Kaippallimalil <John.Kaippallimalil@huawei.com>, Pat Thaler
> ><pthaler@broadcom.com>,
> >draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org
> >
> >Folks,
> >
> >Pls respond if you support this being adopted as a work-group item in
> >the IETF transport services w-g (tsvwg). The WG chairs need visibility
> >of interest.
> >Even better, if you're willing to read / comment / review / implement
> >
> >Guidelines for Adding Congestion Notification to Protocols that
> >Encapsulate IP
> ><http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
> >
> >Abstract
> >
> >    The purpose of this document is to guide the design of congestion
> >    notification in any lower layer or tunnelling protocol that
> >    encapsulates IP.  The aim is for explicit congestion signals to
> >    propagate consistently from lower layer protocols into IP.  Then the
> >    IP internetwork layer can act as a portability layer to carry
> >    congestion notification from non-IP-aware congested nodes up to the
> >    transport layer (L4).  Following these guidelines should assure
> >    interworking between new lower layer congestion notification
> >    mechanisms, whether specified by the IETF or other standards bodies.
> >
> >
> >[Cross-posting tsvwg & aqm, just in case]
> >
> >
> >Bob Briscoe,
> >also for co-authors Pat Thaler and John Kaippallimalil
> >
> >__________________________________________________________
> ______
> >Bob Briscoe,                                                  BT
>=20
> ___________________________________________________________
> _____
> Bob Briscoe,                                                  BT


From ropan@cisco.com  Mon Nov  4 17:14:15 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A345A11E8316; Mon,  4 Nov 2013 17:14:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.485
X-Spam-Level: 
X-Spam-Status: No, score=-10.485 tagged_above=-999 required=5 tests=[AWL=-0.114, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e14X9T1KkYFx; Mon,  4 Nov 2013 17:13:59 -0800 (PST)
Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by ietfa.amsl.com (Postfix) with ESMTP id 7C64411E80F5; Mon,  4 Nov 2013 17:13:58 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=13274; q=dns/txt; s=iport; t=1383614038; x=1384823638; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=djI+f2n0G0atlRwP00qQx2nEnqbW86hpaTvWMeMKBYQ=; b=jhareXT9+xwKLhhXG4vkY9rNH7Em/CW0pJgLZ/WxZguGMKpYZNWct37e Q9bcp/wZyu+14Uk+5Xsd7FwxiP94ZQMJRnqUowqumj4XNhcac8c5Kr3H0 tdhp9hEZvHNYTZnO7lhLpNLwOo7vR+aKsRDc+Qlp/NX/Htk3PyKpgbVNO o=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AqMGAKlFeFKtJXG+/2dsb2JhbABZgkNEOFO2coc1gRKBJxZ0giUBAQEEAQEBKkELEgEIEQMBAQEoLgsUCQgCBAENBRmHaA2+O480Ew0EBgGELgOYCoEvkFqDJoIq
X-IronPort-AV: E=Sophos;i="4.93,636,1378857600";  d="scan'208,217";a="280667393"
Received: from rcdn-core2-3.cisco.com ([173.37.113.190]) by rcdn-iport-6.cisco.com with ESMTP; 05 Nov 2013 01:13:58 +0000
Received: from xhc-rcd-x06.cisco.com (xhc-rcd-x06.cisco.com [173.37.183.80]) by rcdn-core2-3.cisco.com (8.14.5/8.14.5) with ESMTP id rA51Dvu7028463 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 5 Nov 2013 01:13:57 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.185]) by xhc-rcd-x06.cisco.com ([173.37.183.80]) with mapi id 14.03.0123.003; Mon, 4 Nov 2013 19:13:57 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: "Scheffenegger, Richard" <rs@netapp.com>, Matt Mathis <mattmathis@google.com>, Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO2cRM86J8yspfZkulRcpRHYSx/A==
Date: Tue, 5 Nov 2013 01:13:57 +0000
Message-ID: <CE9D85DC.53C7A%ropan@cisco.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E65C8E@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [10.21.122.56]
Content-Type: multipart/alternative; boundary="_000_CE9D85DC53C7Aropanciscocom_"
MIME-Version: 1.0
Cc: "draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 01:14:16 -0000

--_000_CE9D85DC53C7Aropanciscocom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

I agree. This would be valuable work=85

Rong

From: <Scheffenegger>, Richard <rs@netapp.com<mailto:rs@netapp.com>>
Date: Monday, November 4, 2013 5:48 PM
To: Matt Mathis <mattmathis@google.com<mailto:mattmathis@google.com>>, Bob =
Briscoe <bob.briscoe@bt.com<mailto:bob.briscoe@bt.com>>
Cc: "draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org<mailto:draft-b=
riscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn=
-encap-guidelines@tools.ietf.org<mailto:draft-briscoe-tsvwg-ecn-encap-guide=
lines@tools.ietf.org>>, tsvwg IETF list <tsvwg@ietf.org<mailto:tsvwg@ietf.o=
rg>>, AQM IETF list <aqm@ietf.org<mailto:aqm@ietf.org>>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunne=
l protocols?

(aqm chair hat off)

I can fully support what Matt just stated!

I have read an earlier version (-01 iirc), I=92ll try to scan over this ver=
sion in the coming days.


Richard Scheffenegger


From: aqm-bounces@ietf.org<mailto:aqm-bounces@ietf.org> [mailto:aqm-bounces=
@ietf.org] On Behalf Of Matt Mathis
Sent: Montag, 04. November 2013 15:17
To: Bob Briscoe
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org<mailto:draft-br=
iscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>; tsvwg IETF list; AQM IETF=
 list
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunne=
l protocols?

I think this is valuable work.  Having a single document that describes the=
 requirements and general principles will save future tunnel inventor/imple=
menters from rediscovering the same bugs

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our serv=
ices to speak in defiance of unjust governments.   We treat privacy and sec=
urity as matters of life and death, because for some users, they are.

On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com<mailto:bob.=
briscoe@bt.com>> wrote:
Folks,

Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.
Even better, if you're willing to read / comment / review / implement

Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP
<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>

Abstract

   The purpose of this document is to guide the design of congestion
   notification in any lower layer or tunnelling protocol that
   encapsulates IP.  The aim is for explicit congestion signals to
   propagate consistently from lower layer protocols into IP.  Then the
   IP internetwork layer can act as a portability layer to carry
   congestion notification from non-IP-aware congested nodes up to the
   transport layer (L4).  Following these guidelines should assure
   interworking between new lower layer congestion notification
   mechanisms, whether specified by the IETF or other standards bodies.


[Cross-posting tsvwg & aqm, just in case]


Bob Briscoe,
also for co-authors Pat Thaler and John Kaippallimalil


________________________________________________________________
Bob Briscoe,                                                  BT
_______________________________________________
aqm mailing list
aqm@ietf.org<mailto:aqm@ietf.org>
https://www.ietf.org/mailman/listinfo/aqm


--_000_CE9D85DC53C7Aropanciscocom_
Content-Type: text/html; charset="Windows-1252"
Content-ID: <5A5E8ED82900434E87D4C72ECB6B4D3A@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1=
252">
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-fami=
ly: Calibri, sans-serif; ">
<div>I agree. This would be valuable work=85</div>
<div><br>
</div>
<div>Rong</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div style=3D"font-family:Calibri; font-size:14pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>&lt;Scheffenegger&gt;, Richar=
d &lt;<a href=3D"mailto:rs@netapp.com">rs@netapp.com</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Monday, November 4, 2013 5:48=
 PM<br>
<span style=3D"font-weight:bold">To: </span>Matt Mathis &lt;<a href=3D"mail=
to:mattmathis@google.com">mattmathis@google.com</a>&gt;, Bob Briscoe &lt;<a=
 href=3D"mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt;<br>
<span style=3D"font-weight:bold">Cc: </span>&quot;<a href=3D"mailto:draft-b=
riscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org">draft-briscoe-tsvwg-ecn-e=
ncap-guidelines@tools.ietf.org</a>&quot; &lt;<a href=3D"mailto:draft-brisco=
e-tsvwg-ecn-encap-guidelines@tools.ietf.org">draft-briscoe-tsvwg-ecn-encap-=
guidelines@tools.ietf.org</a>&gt;,
 tsvwg IETF list &lt;<a href=3D"mailto:tsvwg@ietf.org">tsvwg@ietf.org</a>&g=
t;, AQM IETF list &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<=
br>
<span style=3D"font-weight:bold">Subject: </span>Re: [aqm] Who supports tsv=
wg adoption of adding ECN to L2 or tunnel protocols?<br>
</div>
<div><br>
</div>
<div xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micro=
soft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" x=
mlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:/=
/www.w3.org/TR/REC-html40">
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";
	mso-fareast-language:EN-US;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
<div lang=3D"DE-AT" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">(aqm chair hat off)=
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p>&nbsp;</o:p></=
span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">I can fully support=
 what Matt just stated!
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p>&nbsp;</o:p></=
span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">I have read an earl=
ier version (-01 iirc), I=92ll try to scan over this version in the coming =
days.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p>&nbsp;</o:p></=
span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size: 11pt; color=
: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p>&nbsp;</o:p></=
span></p>
<p class=3D"MsoNormal"><span style=3D"font-size: 11pt; color: rgb(31, 73, 1=
25); font-family: Calibri, sans-serif; ">Richard Scheffenegger</span><span =
style=3D"font-size: 10pt; color: rgb(31, 73, 125); font-family: Calibri, sa=
ns-serif; "><br>
<br>
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size: 11pt; color: rgb(31, 73, 1=
25); font-family: Calibri, sans-serif; "><o:p>&nbsp;</o:p></span></p>
<div style=3D"border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm =
4.0pt">
<div>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm =
0cm 0cm">
<p class=3D"MsoNormal"><b><span lang=3D"EN-US" style=3D"font-size: 10pt; fo=
nt-family: Tahoma, sans-serif; ">From:</span></b><span lang=3D"EN-US" style=
=3D"font-size: 10pt; font-family: Tahoma, sans-serif; ">
<a href=3D"mailto:aqm-bounces@ietf.org">aqm-bounces@ietf.org</a> [<a href=
=3D"mailto:aqm-bounces@ietf.org">mailto:aqm-bounces@ietf.org</a>]
<b>On Behalf Of </b>Matt Mathis<br>
<b>Sent:</b> Montag, 04. November 2013 15:17<br>
<b>To:</b> Bob Briscoe<br>
<b>Cc:</b> <a href=3D"mailto:draft-briscoe-tsvwg-ecn-encap-guidelines@tools=
.ietf.org">
draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org</a>; tsvwg IETF lis=
t; AQM IETF list<br>
<b>Subject:</b> Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 o=
r tunnel protocols?<o:p></o:p></span></p>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<p class=3D"MsoNormal">I think this is valuable work. &nbsp;Having a single=
 document that describes the requirements and general principles will save =
future tunnel inventor/implementers from rediscovering the same bugs<o:p></=
o:p></p>
<div>
<p class=3D"MsoNormal"><br clear=3D"all">
<o:p></o:p></p>
<div>
<div>
<div>
<p class=3D"MsoNormal">Thanks,<o:p></o:p></p>
</div>
<p class=3D"MsoNormal">--MM--<br>
The best way to predict the future is to create it. &nbsp;- Alan Kay<br>
<br>
Privacy matters! &nbsp;We know from recent events that people are using our=
 services to speak in defiance of unjust governments. &nbsp; We treat priva=
cy and security as matters of life and death, because for some users, they =
are.<o:p></o:p></p>
</div>
</div>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><o:p>&nbsp;</o:p></p>
<div>
<p class=3D"MsoNormal">On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe &lt;<a h=
ref=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.com</a>&=
gt; wrote:<o:p></o:p></p>
<p class=3D"MsoNormal">Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.<br>
Even better, if you're willing to read / comment / review / implement<br>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP<br>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">http://tools.ietf.org/html/draft-briscoe-tsvwg-e=
cn-encap-guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
&nbsp; &nbsp;The purpose of this document is to guide the design of congest=
ion<br>
&nbsp; &nbsp;notification in any lower layer or tunnelling protocol that<br=
>
&nbsp; &nbsp;encapsulates IP. &nbsp;The aim is for explicit congestion sign=
als to<br>
&nbsp; &nbsp;propagate consistently from lower layer protocols into IP. &nb=
sp;Then the<br>
&nbsp; &nbsp;IP internetwork layer can act as a portability layer to carry<=
br>
&nbsp; &nbsp;congestion notification from non-IP-aware congested nodes up t=
o the<br>
&nbsp; &nbsp;transport layer (L4). &nbsp;Following these guidelines should =
assure<br>
&nbsp; &nbsp;interworking between new lower layer congestion notification<b=
r>
&nbsp; &nbsp;mechanisms, whether specified by the IETF or other standards b=
odies.<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
<br>
________________________________________________________________<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;=
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbs=
p; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT <br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><o:p></o:p></p>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>

--_000_CE9D85DC53C7Aropanciscocom_--

From bob.briscoe@bt.com  Mon Nov  4 18:56:42 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 22EE111E8267 for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 18:56:42 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.278
X-Spam-Level: 
X-Spam-Status: No, score=-3.278 tagged_above=-999 required=5 tests=[AWL=0.093,  BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nIDTnqIWNM8K for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 18:56:36 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 78C6621E80AC for <aqm@ietf.org>; Mon,  4 Nov 2013 18:56:31 -0800 (PST)
Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR67-UKRD.bt.com (10.187.101.22) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 5 Nov 2013 02:56:30 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 5 Nov 2013 02:56:30 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Tue, 5 Nov 2013 02:56:29 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.170.252])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA52uPeG027636; Tue, 5 Nov 2013 02:56:26 GMT
Message-ID: <201311050256.rA52uPeG027636@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 5 Nov 2013 02:56:24 +0000
To: <curtis@ipv6.occnc.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <201310141707.r9EH7wC7068478@gateway1.orleans.occnc.com>
References: <Your message of "Fri, 11 Oct 2013 23:18:05 EDT." <12845.1381547885@sandelman.ca> <201310141707.r9EH7wC7068478@gateway1.orleans.occnc.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=====================_500802318==.ALT"
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Michael Richardson <mcr+ietf@sandelman.ca>, "iccrg@irtf.org" <iccrg@irtf.org>, "Akhtar, Shahid \(Shahid\)" <shahid.akhtar@alcatel-lucent.com>, "Eggert, Lars" <lars@netapp.com>, bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 02:56:42 -0000

--=====================_500802318==.ALT
Content-Type: text/plain; charset="us-ascii"; format=flowed

Curtis & all,

At 17:07 14/10/2013, Curtis Villamizar wrote:
>In enterprise and data center there is also very good control over
>what equipment is used and how it is used.  However, clue density
>decreases exponentially farther from the core and approaches zero in
>some data centers and in some enterprise IT departments.  This is also
>true of the part of service provider organizations that pick stuff for
>consumer access.  Where clue density is lowest, you'll find ignorance
>of AQM and buffer issues and lack of consideration for buffering and
>AQM when picking equipment for deployment and configuring it.

We (BT) use WRED extensively at the edge routers into our global 
enterprise MPLS network (see the 
<http://www2.bt.com/btPortal/application;JSESSIONID_btPortalWebApp=Jq1gl1Qz8sbdOQjvnRTGVGo1IfBs47bTe6Amy69S3bYcsw5A99sp!6984303?namespace=pns_catalogue&origin=mb_navigator_right_cd_editorial.jsp&event=link.print.editorial&PorS=products&contentType=techinicalspec&contentitemid=editorial/tech_spec/mpls_ts.xml&productDetail=products/mpls.xml>BT 
MPLS technical 
<http://www2.bt.com/btPortal/application;JSESSIONID_btPortalWebApp=Jq1gl1Qz8sbdOQjvnRTGVGo1IfBs47bTe6Amy69S3bYcsw5A99sp!6984303?namespace=pns_catalogue&origin=mb_navigator_right_cd_editorial.jsp&event=link.print.editorial&PorS=products&contentType=techinicalspec&contentitemid=editorial/tech_spec/mpls_ts.xml&productDetail=products/mpls.xml>specification).


Bob




________________________________________________________________
Bob Briscoe,                                                  BT 
--=====================_500802318==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
Curtis &amp; all,<br><br>
At 17:07 14/10/2013, Curtis Villamizar wrote:<br>
<blockquote type=cite class=cite cite="">In enterprise and data center
there is also very good control over<br>
what equipment is used and how it is used.&nbsp; However, clue
density<br>
decreases exponentially farther from the core and approaches zero in<br>
some data centers and in some enterprise IT departments.&nbsp; This is
also<br>
true of the part of service provider organizations that pick stuff
for<br>
consumer access.&nbsp; Where clue density is lowest, you'll find
ignorance<br>
of AQM and buffer issues and lack of consideration for buffering and<br>
AQM when picking equipment for deployment and configuring
it.</blockquote><br>
We (BT) use WRED extensively at the edge routers into our global
enterprise MPLS network (see the
<a href="http://www2.bt.com/btPortal/application;JSESSIONID_btPortalWebApp=Jq1gl1Qz8sbdOQjvnRTGVGo1IfBs47bTe6Amy69S3bYcsw5A99sp!6984303?namespace=pns_catalogue&amp;origin=mb_navigator_right_cd_editorial.jsp&amp;event=link.print.editorial&amp;PorS=products&amp;contentType=techinicalspec&amp;contentitemid=editorial/tech_spec/mpls_ts.xml&amp;productDetail=products/mpls.xml">
BT MPLS technical
</a>
<a href="http://www2.bt.com/btPortal/application;JSESSIONID_btPortalWebApp=Jq1gl1Qz8sbdOQjvnRTGVGo1IfBs47bTe6Amy69S3bYcsw5A99sp!6984303?namespace=pns_catalogue&amp;origin=mb_navigator_right_cd_editorial.jsp&amp;event=link.print.editorial&amp;PorS=products&amp;contentType=techinicalspec&amp;contentitemid=editorial/tech_spec/mpls_ts.xml&amp;productDetail=products/mpls.xml">
specification</a>).<br><br>
<br>
Bob<br><br>
<br><br>
<x-sigsep><p></x-sigsep>
________________________________________________________________<br>
Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT</body>
</html>

--=====================_500802318==.ALT--

From menth@uni-tuebingen.de  Mon Nov  4 23:36:32 2013
Return-Path: <menth@uni-tuebingen.de>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 192FA21E8087 for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 23:36:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.021
X-Spam-Level: 
X-Spam-Status: No, score=-6.021 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZnqVrf8qRjer for <aqm@ietfa.amsl.com>; Mon,  4 Nov 2013 23:36:25 -0800 (PST)
Received: from mx09.uni-tuebingen.de (mx09.uni-tuebingen.de [134.2.3.2]) by ietfa.amsl.com (Postfix) with ESMTP id 6383721F9D5F for <aqm@ietf.org>; Mon,  4 Nov 2013 23:36:22 -0800 (PST)
Received: from [134.2.11.131] (chaos.Informatik.Uni-Tuebingen.De [134.2.11.131]) (authenticated bits=0) by mx09.uni-tuebingen.de (8.14.3/8.14.3) with ESMTP id rA57aJgw023898 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=FAIL) for <aqm@ietf.org>; Tue, 5 Nov 2013 08:36:19 +0100
Message-ID: <52789FF5.3030907@uni-tuebingen.de>
Date: Tue, 05 Nov 2013 08:36:21 +0100
From: Michael Menth <menth@uni-tuebingen.de>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8
MIME-Version: 1.0
To: aqm@ietf.org
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
In-Reply-To: <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
Content-Type: multipart/alternative; boundary="------------040803080401020203000301"
X-AntiVirus: checked by Avira MailGate (version: 3.2.1.26; AVE: 8.2.12.136; VDF: 7.11.110.236; host: mx09); id=25033-JlXNOw
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 07:36:32 -0000

This is a multi-part message in MIME format.
--------------040803080401020203000301
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

+1

We need such guidelines for consistent congestion management.

Best wishes,

Michael

Am 05.11.2013 00:16, schrieb Matt Mathis:
> I think this is valuable work.  Having a single document that
> describes the requirements and general principles will save future
> tunnel inventor/implementers from rediscovering the same bugs
>
> Thanks,
> --MM--
> The best way to predict the future is to create it.  - Alan Kay
>
> Privacy matters!  We know from recent events that people are using our
> services to speak in defiance of unjust governments.   We treat
> privacy and security as matters of life and death, because for some
> users, they are.
>
>
> On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
> <mailto:bob.briscoe@bt.com>> wrote:
>
>     Folks,
>
>     Pls respond if you support this being adopted as a work-group item
>     in the IETF transport services w-g (tsvwg). The WG chairs need
>     visibility of interest.
>     Even better, if you're willing to read / comment / review / implement
>
>     Guidelines for Adding Congestion Notification to Protocols that
>     Encapsulate IP
>     <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>
>     Abstract
>
>        The purpose of this document is to guide the design of congestion
>        notification in any lower layer or tunnelling protocol that
>        encapsulates IP.  The aim is for explicit congestion signals to
>        propagate consistently from lower layer protocols into IP.
>      Then the
>        IP internetwork layer can act as a portability layer to carry
>        congestion notification from non-IP-aware congested nodes up to the
>        transport layer (L4).  Following these guidelines should assure
>        interworking between new lower layer congestion notification
>        mechanisms, whether specified by the IETF or other standards
>     bodies.
>
>
>     [Cross-posting tsvwg & aqm, just in case]
>
>
>     Bob Briscoe,
>     also for co-authors Pat Thaler and John Kaippallimalil
>
>
>     ________________________________________________________________
>     Bob Briscoe,                                                  BT
>     _______________________________________________
>     aqm mailing list
>     aqm@ietf.org <mailto:aqm@ietf.org>
>     https://www.ietf.org/mailman/listinfo/aqm
>
>
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

-- 
Prof. Dr. habil. Michael Menth
University of Tuebingen
Faculty of Science
Department of Computer Science
Chair of Communication Networks
Sand 13, 72076 Tuebingen, Germany
phone: (+49)-7071/29-70505
fax: (+49)-7071/29-5220
mailto:menth@uni-tuebingen.de
http://kn.inf.uni-tuebingen.de


--------------040803080401020203000301
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    +1<br>
    <br>
    We need such guidelines for consistent congestion management.<br>
    <br>
    Best wishes,<br>
    <br>
    Michael<br>
    <br>
    <div class="moz-cite-prefix">Am 05.11.2013 00:16, schrieb Matt
      Mathis:<br>
    </div>
    <blockquote
cite="mid:CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com"
      type="cite">
      <div dir="ltr">I think this is valuable work. &nbsp;Having a single
        document that describes the requirements and general principles
        will save future tunnel inventor/implementers from rediscovering
        the same bugs<br>
        <div class="gmail_extra">
          <br clear="all">
          <div>
            <div dir="ltr">
              <div>Thanks,</div>
              --MM--<br>
              The best way to predict the future is to create it. &nbsp;-
              Alan Kay<br>
              <br>
              Privacy matters! &nbsp;We know from recent events that people
              are using our services to speak in defiance of unjust
              governments. &nbsp; We treat privacy and security as matters of
              life and death, because for some users, they are.</div>
          </div>
          <br>
          <br>
          <div class="gmail_quote">On Mon, Nov 4, 2013 at 2:03 PM, Bob
            Briscoe <span dir="ltr">&lt;<a moz-do-not-send="true"
                href="mailto:bob.briscoe@bt.com" target="_blank">bob.briscoe@bt.com</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              Folks,<br>
              <br>
              Pls respond if you support this being adopted as a
              work-group item in the IETF transport services w-g
              (tsvwg). The WG chairs need visibility of interest.<br>
              Even better, if you're willing to read / comment / review
              / implement<br>
              <br>
              Guidelines for Adding Congestion Notification to Protocols
              that Encapsulate IP<br>
              &lt;<a moz-do-not-send="true"
href="http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines"
                target="_blank">http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines</a>&gt;<br>
              <br>
              Abstract<br>
              <br>
              &nbsp; &nbsp;The purpose of this document is to guide the design of
              congestion<br>
              &nbsp; &nbsp;notification in any lower layer or tunnelling protocol
              that<br>
              &nbsp; &nbsp;encapsulates IP. &nbsp;The aim is for explicit congestion
              signals to<br>
              &nbsp; &nbsp;propagate consistently from lower layer protocols into
              IP. &nbsp;Then the<br>
              &nbsp; &nbsp;IP internetwork layer can act as a portability layer to
              carry<br>
              &nbsp; &nbsp;congestion notification from non-IP-aware congested
              nodes up to the<br>
              &nbsp; &nbsp;transport layer (L4). &nbsp;Following these guidelines
              should assure<br>
              &nbsp; &nbsp;interworking between new lower layer congestion
              notification<br>
              &nbsp; &nbsp;mechanisms, whether specified by the IETF or other
              standards bodies.<br>
              <br>
              <br>
              [Cross-posting tsvwg &amp; aqm, just in case]<br>
              <br>
              <br>
              Bob Briscoe,<br>
              also for co-authors Pat Thaler and John Kaippallimalil<br>
              <br>
              <br>
              ________________________________________________________________<br>
              Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
              &nbsp; &nbsp;BT <br>
              _______________________________________________<br>
              aqm mailing list<br>
              <a moz-do-not-send="true" href="mailto:aqm@ietf.org"
                target="_blank">aqm@ietf.org</a><br>
              <a moz-do-not-send="true"
                href="https://www.ietf.org/mailman/listinfo/aqm"
                target="_blank">https://www.ietf.org/mailman/listinfo/aqm</a><br>
            </blockquote>
          </div>
          <br>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
aqm mailing list
<a class="moz-txt-link-abbreviated" href="mailto:aqm@ietf.org">aqm@ietf.org</a>
<a class="moz-txt-link-freetext" href="https://www.ietf.org/mailman/listinfo/aqm">https://www.ietf.org/mailman/listinfo/aqm</a>
</pre>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Prof. Dr. habil. Michael Menth
University of Tuebingen
Faculty of Science
Department of Computer Science
Chair of Communication Networks
Sand 13, 72076 Tuebingen, Germany
phone: (+49)-7071/29-70505
fax: (+49)-7071/29-5220
<a class="moz-txt-link-freetext" href="mailto:menth@uni-tuebingen.de">mailto:menth@uni-tuebingen.de</a>
<a class="moz-txt-link-freetext" href="http://kn.inf.uni-tuebingen.de">http://kn.inf.uni-tuebingen.de</a>
</pre>
  </body>
</html>

--------------040803080401020203000301--

From lei.zhu@huawei.com  Mon Nov  4 23:50:05 2013
Return-Path: <lei.zhu@huawei.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79A1611E822B; Mon,  4 Nov 2013 23:49:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.214
X-Spam-Level: 
X-Spam-Status: No, score=-4.214 tagged_above=-999 required=5 tests=[AWL=-2.384, BAYES_00=-2.599, CN_BODY_35=0.339, MIME_BASE64_TEXT=1.753, MIME_CHARSET_FARAWAY=2.45, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6uW8eiAwBeaH; Mon,  4 Nov 2013 23:49:37 -0800 (PST)
Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id 9E4AB11E81AB; Mon,  4 Nov 2013 23:49:36 -0800 (PST)
Received: from 172.18.7.190 (EHLO lhreml204-edg.china.huawei.com) ([172.18.7.190]) by lhrrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id AZX01520; Tue, 05 Nov 2013 07:49:28 +0000 (GMT)
Received: from LHREML402-HUB.china.huawei.com (10.201.5.241) by lhreml204-edg.china.huawei.com (172.18.7.223) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 5 Nov 2013 07:48:52 +0000
Received: from nkgeml405-hub.china.huawei.com (10.98.56.36) by lhreml402-hub.china.huawei.com (10.201.5.241) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 5 Nov 2013 07:49:27 +0000
Received: from NKGEML512-MBS.china.huawei.com ([169.254.8.193]) by nkgeml405-hub.china.huawei.com ([10.98.56.36]) with mapi id 14.03.0158.001; Tue, 5 Nov 2013 15:49:17 +0800
From: "Zhulei (A)" <lei.zhu@huawei.com>
To: Bob Briscoe <bob.briscoe@bt.com>, tsvwg IETF list <tsvwg@ietf.org>, "AQM IETF list" <aqm@ietf.org>
Thread-Topic: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO2andeDdoGsWyNkSS9zl+hZ4GQJoWPPww
Date: Tue, 5 Nov 2013 07:49:16 +0000
Message-ID: <470F27D1263A1B4EB73491ED995DE6252585DA0A@NKGEML512-MBS.china.huawei.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
In-Reply-To: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.111.65.68]
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, "draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 07:50:06 -0000

SGkgYWxsLA0KDQpUaGlzIGRvY3VtZW50IHJlbWluZHMgb2YgdGhpbmtpbmcgb2YgbG93ZXIgbGF5
ZXIgY29uZ2VzdGlvbiBhbmQgcXVldWluZyBtZWNoYW5pc21zIGluIG5ldHdvcmssIHNvbWV0aW1l
cyB3aGljaCB3b3VsZCBsZXZlcmFnZSBsYXJnZSBjb3ZlcmFnZS4gVGhlIG1vZGVzIHRvIGd1aWRl
IHNlZW0gYWxsIHZhbGlkYXRlZCBhbmQgZXZlbiBoZWxwZnVsIGZvciBteSBzY2VuYXJpb3Mgb2Yg
c29tZSBJUC1pbi1JUCBjYXNlIGFzIHdoYXQgaXMgc2F5aW5nIGluIGRyYWZ0LiBTbywgSSB3b3Vs
ZCBiZSBnb29kIGZvciBhZG9wdGlvbiBvZiB0aGlzIGRyYWZ0IGFzIGEgd29yayBpdGVtLCBhbmQs
IHdvdWxkIGxpa2UgdG8gcHJvbW90ZSBpbiB0aGlzIGRpcmVjdGlvbiBieSBzb21ldGhpbmcgZmVl
ZGJhY2sgdG8gV0cuDQoNCk15IGFsd2F5cyB0aG91Z2h0IHRoYXQgdGhlIGVuY29kaW5nIG9mIEVD
TiBtYXJraW5nLCBhcyBpbiBzb21lIGNhc2UgdGhlIGluZ3Jlc3Mgb2YgYSB0dW5uZWwgbWF5IG5l
ZWQgdGhlIHNwYWNlKHMpIGZvciBmb2xsb3dpbmcgbGF5ZXIzIGVudGl0aWVzIHRvIGZpbGwgdGhl
IEVDVCBhbmQgbG9jYXRpb24gaW5mb3JtYXRpb24gKGF0IGxlYXNlIGUuZy4gSVAgYWRkcmVzcyku
IE9idmlvdXNseSwgaW5mb3JtYXRpb24gb2YgbG9jYXRpb24gd291bGQgYmUgb25seSBoZWxwZnVs
IHRvIG5ldHdvcmsgb3BlcmF0b3IsIGJ1dCB0aGUgZW5kIHBvaW50IGhhbmRsaW5nIEVDTiBtYXJr
aW5nLiANCg0KQmVzdCByZWdhcmRzLA0KWmh1IExlaQ0KTW9iaWxlOiArODYtMTM5MTAxNTcwMjAN
Cg0KLS0tLS3Tyrz+1K28/i0tLS0tDQq3orz+yMs6IGFxbS1ib3VuY2VzQGlldGYub3JnIFttYWls
dG86YXFtLWJvdW5jZXNAaWV0Zi5vcmddILT6se0gQm9iIEJyaXNjb2UNCreiy83KsbzkOiAyMDEz
xOoxMdTCNcjVIDY6MDQNCsrVvP7IyzogdHN2d2cgSUVURiBsaXN0OyBBUU0gSUVURiBsaXN0DQqz
rcvNOiBQYXQgVGhhbGVyOyBKb2huIEthaXBwYWxsaW1hbGlsOyBkcmFmdC1icmlzY29lLXRzdndn
LWVjbi1lbmNhcC1ndWlkZWxpbmVzQHRvb2xzLmlldGYub3JnDQrW98ziOiBbYXFtXSBXaG8gc3Vw
cG9ydHMgdHN2d2cgYWRvcHRpb24gb2YgYWRkaW5nIEVDTiB0byBMMiBvciB0dW5uZWwgcHJvdG9j
b2xzPw0KDQpGb2xrcywNCg0KUGxzIHJlc3BvbmQgaWYgeW91IHN1cHBvcnQgdGhpcyBiZWluZyBh
ZG9wdGVkIGFzIGEgd29yay1ncm91cCBpdGVtIGluIHRoZSBJRVRGIHRyYW5zcG9ydCBzZXJ2aWNl
cyB3LWcgKHRzdndnKS4gVGhlIFdHIGNoYWlycyBuZWVkIHZpc2liaWxpdHkgb2YgaW50ZXJlc3Qu
DQpFdmVuIGJldHRlciwgaWYgeW91J3JlIHdpbGxpbmcgdG8gcmVhZCAvIGNvbW1lbnQgLyByZXZp
ZXcgLyBpbXBsZW1lbnQNCg0KR3VpZGVsaW5lcyBmb3IgQWRkaW5nIENvbmdlc3Rpb24gTm90aWZp
Y2F0aW9uIHRvIFByb3RvY29scyB0aGF0IEVuY2Fwc3VsYXRlIElQIDxodHRwOi8vdG9vbHMuaWV0
Zi5vcmcvaHRtbC9kcmFmdC1icmlzY29lLXRzdndnLWVjbi1lbmNhcC1ndWlkZWxpbmVzPg0KDQpB
YnN0cmFjdA0KDQogICAgVGhlIHB1cnBvc2Ugb2YgdGhpcyBkb2N1bWVudCBpcyB0byBndWlkZSB0
aGUgZGVzaWduIG9mIGNvbmdlc3Rpb24NCiAgICBub3RpZmljYXRpb24gaW4gYW55IGxvd2VyIGxh
eWVyIG9yIHR1bm5lbGxpbmcgcHJvdG9jb2wgdGhhdA0KICAgIGVuY2Fwc3VsYXRlcyBJUC4gIFRo
ZSBhaW0gaXMgZm9yIGV4cGxpY2l0IGNvbmdlc3Rpb24gc2lnbmFscyB0bw0KICAgIHByb3BhZ2F0
ZSBjb25zaXN0ZW50bHkgZnJvbSBsb3dlciBsYXllciBwcm90b2NvbHMgaW50byBJUC4gIFRoZW4g
dGhlDQogICAgSVAgaW50ZXJuZXR3b3JrIGxheWVyIGNhbiBhY3QgYXMgYSBwb3J0YWJpbGl0eSBs
YXllciB0byBjYXJyeQ0KICAgIGNvbmdlc3Rpb24gbm90aWZpY2F0aW9uIGZyb20gbm9uLUlQLWF3
YXJlIGNvbmdlc3RlZCBub2RlcyB1cCB0byB0aGUNCiAgICB0cmFuc3BvcnQgbGF5ZXIgKEw0KS4g
IEZvbGxvd2luZyB0aGVzZSBndWlkZWxpbmVzIHNob3VsZCBhc3N1cmUNCiAgICBpbnRlcndvcmtp
bmcgYmV0d2VlbiBuZXcgbG93ZXIgbGF5ZXIgY29uZ2VzdGlvbiBub3RpZmljYXRpb24NCiAgICBt
ZWNoYW5pc21zLCB3aGV0aGVyIHNwZWNpZmllZCBieSB0aGUgSUVURiBvciBvdGhlciBzdGFuZGFy
ZHMgYm9kaWVzLg0KDQoNCltDcm9zcy1wb3N0aW5nIHRzdndnICYgYXFtLCBqdXN0IGluIGNhc2Vd
DQoNCg0KQm9iIEJyaXNjb2UsDQphbHNvIGZvciBjby1hdXRob3JzIFBhdCBUaGFsZXIgYW5kIEpv
aG4gS2FpcHBhbGxpbWFsaWwNCg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpCb2IgQnJpc2NvZSwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEJUIA0KDQpfX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KYXFtIG1haWxpbmcgbGlzdA0KYXFt
QGlldGYub3JnDQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2FxbQ0K

From p.ohanlon@gmail.com  Tue Nov  5 01:42:38 2013
Return-Path: <p.ohanlon@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 659C411E8173; Tue,  5 Nov 2013 01:42:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.569
X-Spam-Level: 
X-Spam-Status: No, score=-2.569 tagged_above=-999 required=5 tests=[AWL=0.030,  BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cWh9oWBB6Lck; Tue,  5 Nov 2013 01:42:37 -0800 (PST)
Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) by ietfa.amsl.com (Postfix) with ESMTP id CB4A711E80FC; Tue,  5 Nov 2013 01:42:36 -0800 (PST)
Received: by mail-we0-f172.google.com with SMTP id q58so3178044wes.17 for <multiple recipients>; Tue, 05 Nov 2013 01:42:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=0pJKqr3cnG25z/G+GY90KwNzIg5EyP2cLdmbbJDC2sw=; b=rENxcGDPlCcZv/wn9NNEk0YmPzvHiUJz9mvfV2b10nhdlhjo02+s0aU9lOX+KgeS2t ooV7irdfPbUrd4l4iwkGDzLa/9QQFOlvsELSeS8xWkzyln1S3XJu9HKIQ5RZNCDNpN86 8p6MpS/sX6zWDSJcuecSo7atxsizMAKWnR8NnI+YuqOSEos3Ollx9AZMVkYlQOP1i8kt T3kqd1zt9Vh2nbsCgGqcHVeehSBmDq0gXO2B156zz7zhi42IA3XJWNnlde1rGcTlgpSb AyBV2Pru6U3XfVgyhOMO6C2kvvAlh9b7cyvJQimDWl+ViyFCwgQJJEqd4xPkpShPh6SV tVqA==
X-Received: by 10.180.39.34 with SMTP id m2mr16179787wik.26.1383644555208; Tue, 05 Nov 2013 01:42:35 -0800 (PST)
Received: from black.lan ([149.241.129.101]) by mx.google.com with ESMTPSA id b7sm12452434wiz.8.2013.11.05.01.42.32 for <multiple recipients> (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 05 Nov 2013 01:42:34 -0800 (PST)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Piers O'Hanlon <p.ohanlon@gmail.com>
In-Reply-To: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
Date: Tue, 5 Nov 2013 09:42:31 +0000
Content-Transfer-Encoding: quoted-printable
Message-Id: <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>
To: Bob Briscoe <bob.briscoe@bt.com>
X-Mailer: Apple Mail (2.1283)
X-Mailman-Approved-At: Tue, 05 Nov 2013 08:51:56 -0800
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 09:42:38 -0000

Hi Bob,

I took a brief look at the draft and it's clearly useful work.

One thing that could do with clarification in the Introduction is that =
ECN - by itself - doesn't necessarily lead to low loss and delay - it =
should be made clear that it reflects the marking approach of the =
underlying scheme/AQM.

Piers

On 4 Nov 2013, at 22:03, Bob Briscoe wrote:

> Folks,
>=20
> Pls respond if you support this being adopted as a work-group item in =
the IETF transport services w-g (tsvwg). The WG chairs need visibility =
of interest.
> Even better, if you're willing to read / comment / review / implement
>=20
> Guidelines for Adding Congestion Notification to Protocols that =
Encapsulate IP
> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>=20
> Abstract
>=20
>   The purpose of this document is to guide the design of congestion
>   notification in any lower layer or tunnelling protocol that
>   encapsulates IP.  The aim is for explicit congestion signals to
>   propagate consistently from lower layer protocols into IP.  Then the
>   IP internetwork layer can act as a portability layer to carry
>   congestion notification from non-IP-aware congested nodes up to the
>   transport layer (L4).  Following these guidelines should assure
>   interworking between new lower layer congestion notification
>   mechanisms, whether specified by the IETF or other standards bodies.
>=20
>=20
> [Cross-posting tsvwg & aqm, just in case]
>=20
>=20
> Bob Briscoe,
> also for co-authors Pat Thaler and John Kaippallimalil
>=20
>=20
> ________________________________________________________________
> Bob Briscoe,                                                  BT=20


From Dirk.Kutscher@neclab.eu  Tue Nov  5 01:50:39 2013
Return-Path: <Dirk.Kutscher@neclab.eu>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FFC111E81A5; Tue,  5 Nov 2013 01:50:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.2
X-Spam-Level: 
X-Spam-Status: No, score=-3.2 tagged_above=-999 required=5 tests=[AWL=0.400, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F9WpsJYx1RJS; Tue,  5 Nov 2013 01:50:25 -0800 (PST)
Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id 1CD1711E8186; Tue,  5 Nov 2013 01:50:21 -0800 (PST)
Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 43768105F1A; Tue,  5 Nov 2013 10:44:14 +0100 (CET)
X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de)
Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AA3b28iFxMZx; Tue,  5 Nov 2013 10:44:14 +0100 (CET)
Received: from METHONE.office.hd (methone.office.hd [192.168.24.54]) by mailer1.neclab.eu (Postfix) with ESMTP id 27E19105F17; Tue,  5 Nov 2013 10:43:49 +0100 (CET)
Received: from PALLENE.office.hd ([169.254.1.11]) by METHONE.office.hd ([192.168.24.54]) with mapi id 14.01.0323.003; Tue, 5 Nov 2013 10:49:34 +0100
From: Dirk Kutscher <Dirk.Kutscher@neclab.eu>
To: Piers O'Hanlon <p.ohanlon@gmail.com>, Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO2gtteH36tQ1iVEKiOjJC7yWNSZoWYy5w
Date: Tue, 5 Nov 2013 09:49:34 +0000
Message-ID: <82AB329A76E2484D934BBCA77E9F52496383930E@PALLENE.office.hd>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com>
In-Reply-To: <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com>
Accept-Language: de-DE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.1.99.203]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailman-Approved-At: Tue, 05 Nov 2013 08:51:56 -0800
Cc: "draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or	tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 09:50:39 -0000

Hi,

As earlier indicated, I'd like to see this document being adopted as a tsvw=
g work item.

I'd be happy to help with reviewing.

Best regards,
Dirk




> -----Original Message-----
> From: tsvwg-bounces@ietf.org [mailto:tsvwg-bounces@ietf.org] On Behalf
> Of Piers O'Hanlon
> Sent: Dienstag, 5. November 2013 10:43
> To: Bob Briscoe
> Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg IETF l=
ist;
> AQM IETF list
> Subject: Re: [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or
> tunnel protocols?
>=20
> Hi Bob,
>=20
> I took a brief look at the draft and it's clearly useful work.
>=20
> One thing that could do with clarification in the Introduction is that EC=
N - by
> itself - doesn't necessarily lead to low loss and delay - it should be ma=
de clear
> that it reflects the marking approach of the underlying scheme/AQM.
>=20
> Piers
>=20
> On 4 Nov 2013, at 22:03, Bob Briscoe wrote:
>=20
> > Folks,
> >
> > Pls respond if you support this being adopted as a work-group item in t=
he
> IETF transport services w-g (tsvwg). The WG chairs need visibility of int=
erest.
> > Even better, if you're willing to read / comment / review / implement
> >
> > Guidelines for Adding Congestion Notification to Protocols that
> > Encapsulate IP
> > <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
> >
> > Abstract
> >
> >   The purpose of this document is to guide the design of congestion
> >   notification in any lower layer or tunnelling protocol that
> >   encapsulates IP.  The aim is for explicit congestion signals to
> >   propagate consistently from lower layer protocols into IP.  Then the
> >   IP internetwork layer can act as a portability layer to carry
> >   congestion notification from non-IP-aware congested nodes up to the
> >   transport layer (L4).  Following these guidelines should assure
> >   interworking between new lower layer congestion notification
> >   mechanisms, whether specified by the IETF or other standards bodies.
> >
> >
> > [Cross-posting tsvwg & aqm, just in case]
> >
> >
> > Bob Briscoe,
> > also for co-authors Pat Thaler and John Kaippallimalil
> >
> >
> >
> __________________________________________________________
> ______
> > Bob Briscoe,                                                  BT


From philip.eardley@bt.com  Tue Nov  5 10:32:48 2013
Return-Path: <philip.eardley@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 85A9F11E81FD; Tue,  5 Nov 2013 10:32:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.936
X-Spam-Level: 
X-Spam-Status: No, score=-102.936 tagged_above=-999 required=5 tests=[AWL=-0.165, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8xi-l2I7hLbw; Tue,  5 Nov 2013 10:32:44 -0800 (PST)
Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 9471C21E809E; Tue,  5 Nov 2013 10:32:39 -0800 (PST)
Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Tue, 5 Nov 2013 18:32:37 +0000
Received: from EMV67-UKRD.domain1.systemhost.net ([169.254.2.147]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Tue, 5 Nov 2013 18:32:37 +0000
From: <philip.eardley@bt.com>
To: <rs@netapp.com>, <mattmathis@google.com>, <bob.briscoe@bt.com>
Date: Tue, 5 Nov 2013 18:32:22 +0000
Thread-Topic: [tsvwg] [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel	protocols?
Thread-Index: AQHO2bPschNRr0CscEy8XBGE9jPQ05oVzbZAgAEpnkM=
Message-ID: <A2E337CDB7BC4145B018B9BEE8EB3E0D3FFA6F97ED@EMV67-UKRD.domain1.systemhost.net>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>, <012C3117EDDB3C4781FD802A8C27DD4F25E65C8E@SACEXCMBX02-PRD.hq.netapp.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E65C8E@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US, en-GB
Content-Language: en-GB
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US, en-GB
Content-Type: multipart/alternative; boundary="_000_A2E337CDB7BC4145B018B9BEE8EB3E0D3FFA6F97EDEMV67UKRDdoma_"
MIME-Version: 1.0
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg@ietf.org, aqm@ietf.org
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2	or tunnel	protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 18:32:48 -0000

--_000_A2E337CDB7BC4145B018B9BEE8EB3E0D3FFA6F97EDEMV67UKRDdoma_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

agree with these comments
best wishes
phil

________________________________
From: tsvwg-bounces@ietf.org [tsvwg-bounces@ietf.org] On Behalf Of Scheffen=
egger, Richard [rs@netapp.com]
Sent: 05 November 2013 00:48
To: Matt Mathis; Briscoe,RJ,Bob,TUB8 R
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg IETF lis=
t; AQM IETF list
Subject: Re: [tsvwg] [aqm] Who supports tsvwg adoption of adding ECN to L2 =
or tunnel protocols?

(aqm chair hat off)

I can fully support what Matt just stated!

I have read an earlier version (-01 iirc), I=92ll try to scan over this ver=
sion in the coming days.


Richard Scheffenegger


From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of Matt =
Mathis
Sent: Montag, 04. November 2013 15:17
To: Bob Briscoe
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg IETF lis=
t; AQM IETF list
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunne=
l protocols?

I think this is valuable work.  Having a single document that describes the=
 requirements and general principles will save future tunnel inventor/imple=
menters from rediscovering the same bugs

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our serv=
ices to speak in defiance of unjust governments.   We treat privacy and sec=
urity as matters of life and death, because for some users, they are.

On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com<mailto:bob.=
briscoe@bt.com>> wrote:
Folks,

Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.
Even better, if you're willing to read / comment / review / implement

Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP
<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>

Abstract

   The purpose of this document is to guide the design of congestion
   notification in any lower layer or tunnelling protocol that
   encapsulates IP.  The aim is for explicit congestion signals to
   propagate consistently from lower layer protocols into IP.  Then the
   IP internetwork layer can act as a portability layer to carry
   congestion notification from non-IP-aware congested nodes up to the
   transport layer (L4).  Following these guidelines should assure
   interworking between new lower layer congestion notification
   mechanisms, whether specified by the IETF or other standards bodies.


[Cross-posting tsvwg & aqm, just in case]


Bob Briscoe,
also for co-authors Pat Thaler and John Kaippallimalil


________________________________________________________________
Bob Briscoe,                                                  BT
_______________________________________________
aqm mailing list
aqm@ietf.org<mailto:aqm@ietf.org>
https://www.ietf.org/mailman/listinfo/aqm


--_000_A2E337CDB7BC4145B018B9BEE8EB3E0D3FFA6F97EDEMV67UKRDdoma_
Content-Type: text/html; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

<html dir=3D"ltr"><head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1=
252">
<style>@font-face {
	font-family: Calibri;
}
@font-face {
	font-family: Tahoma;
}
@page WordSection1 {margin: 70.85pt 70.85pt 2.0cm 70.85pt; }
P.MsoNormal {
	MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman","serif"; FONT-SIZE: 12=
pt
}
LI.MsoNormal {
	MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman","serif"; FONT-SIZE: 12=
pt
}
DIV.MsoNormal {
	MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman","serif"; FONT-SIZE: 12=
pt
}
A:link {
	COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
	COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
	COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
	COLOR: purple; TEXT-DECORATION: underline
}
SPAN.EmailStyle17 {
	FONT-FAMILY: "Calibri","sans-serif"; COLOR: #1f497d
}
.MsoChpDefault {
	FONT-FAMILY: "Calibri","sans-serif"
}
DIV.WordSection1 {
=09
}
</style>
<meta name=3D"GENERATOR" content=3D"MSHTML 9.00.8112.16443">
<style id=3D"owaTempEditStyle"></style><style title=3D"owaParaStyle"><!--P =
{
	MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
--></style>
</head>
<body lang=3D"DE-AT" vlink=3D"purple" link=3D"blue" ocsi=3D"x">
<div style=3D"FONT-FAMILY: Tahoma; DIRECTION: ltr; COLOR: #000000; FONT-SIZ=
E: x-small">
<div>agree with these comments</div>
<div><font face=3D"tahoma">best wishes</font></div>
<div><font face=3D"tahoma">phil</font></div>
<div dir=3D"ltr"><font color=3D"#000000" size=3D"2" face=3D"Tahoma"></font>=
&nbsp;</div>
<div style=3D"DIRECTION: ltr" id=3D"divRpF878526">
<hr tabindex=3D"-1">
<font color=3D"#000000" size=3D"2" face=3D"Tahoma"><b>From:</b> tsvwg-bounc=
es@ietf.org [tsvwg-bounces@ietf.org] On Behalf Of Scheffenegger, Richard [r=
s@netapp.com]<br>
<b>Sent:</b> 05 November 2013 00:48<br>
<b>To:</b> Matt Mathis; Briscoe,RJ,Bob,TUB8 R<br>
<b>Cc:</b> draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg I=
ETF list; AQM IETF list<br>
<b>Subject:</b> Re: [tsvwg] [aqm] Who supports tsvwg adoption of adding ECN=
 to L2 or tunnel protocols?<br>
</font><br>
</div>
<div></div>
<div>
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US">(aqm chair hat off)</span><=
/p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US"></span>&nbsp;</p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US">I can fully support what Ma=
tt just stated!
</span></p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US"></span>&nbsp;</p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US">I have read an earlier vers=
ion (-01 iirc), I=92ll try to scan over this version in the coming days.</s=
pan></p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US"></span>&nbsp;</p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt" lang=3D"EN-US"></span>&nbsp;</p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt">Richard Scheffenegger</span><span style=3D=
"FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10pt"><br>
<br>
</span></p>
<p class=3D"MsoNormal"><span style=3D"FONT-FAMILY: 'Calibri','sans-serif'; =
COLOR: #1f497d; FONT-SIZE: 11pt"></span>&nbsp;</p>
<div style=3D"BORDER-BOTTOM: medium none; BORDER-LEFT: blue 1.5pt solid; PA=
DDING-BOTTOM: 0cm; PADDING-LEFT: 4pt; PADDING-RIGHT: 0cm; BORDER-TOP: mediu=
m none; BORDER-RIGHT: medium none; PADDING-TOP: 0cm">
<div>
<div style=3D"BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING=
-BOTTOM: 0cm; PADDING-LEFT: 0cm; PADDING-RIGHT: 0cm; BORDER-TOP: #b5c4df 1p=
t solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<p class=3D"MsoNormal"><b><span style=3D"FONT-FAMILY: 'Tahoma','sans-serif'=
; FONT-SIZE: 10pt" lang=3D"EN-US">From:</span></b><span style=3D"FONT-FAMIL=
Y: 'Tahoma','sans-serif'; FONT-SIZE: 10pt" lang=3D"EN-US"> aqm-bounces@ietf=
.org [mailto:aqm-bounces@ietf.org]
<b>On Behalf Of </b>Matt Mathis<br>
<b>Sent:</b> Montag, 04. November 2013 15:17<br>
<b>To:</b> Bob Briscoe<br>
<b>Cc:</b> draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org; tsvwg I=
ETF list; AQM IETF list<br>
<b>Subject:</b> Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 o=
r tunnel protocols?</span></p>
</div>
</div>
<p class=3D"MsoNormal">&nbsp;</p>
<div>
<p class=3D"MsoNormal">I think this is valuable work. &nbsp;Having a single=
 document that describes the requirements and general principles will save =
future tunnel inventor/implementers from rediscovering the same bugs</p>
<div>
<p class=3D"MsoNormal"><br clear=3D"all">
</p>
<div>
<div>
<div>
<p class=3D"MsoNormal">Thanks,</p>
</div>
<p class=3D"MsoNormal">--MM--<br>
The best way to predict the future is to create it. &nbsp;- Alan Kay<br>
<br>
Privacy matters! &nbsp;We know from recent events that people are using our=
 services to speak in defiance of unjust governments. &nbsp; We treat priva=
cy and security as matters of life and death, because for some users, they =
are.</p>
</div>
</div>
<p style=3D"MARGIN-BOTTOM: 12pt" class=3D"MsoNormal">&nbsp;</p>
<div>
<p class=3D"MsoNormal">On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe &lt;<a h=
ref=3D"mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt; wrote:</p>
<p class=3D"MsoNormal">Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.<br>
Even better, if you're willing to read / comment / review / implement<br>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP<br>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">http://tools.ietf.org/html/draft-briscoe-tsvwg-e=
cn-encap-guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
&nbsp; &nbsp;The purpose of this document is to guide the design of congest=
ion<br>
&nbsp; &nbsp;notification in any lower layer or tunnelling protocol that<br=
>
&nbsp; &nbsp;encapsulates IP. &nbsp;The aim is for explicit congestion sign=
als to<br>
&nbsp; &nbsp;propagate consistently from lower layer protocols into IP. &nb=
sp;Then the<br>
&nbsp; &nbsp;IP internetwork layer can act as a portability layer to carry<=
br>
&nbsp; &nbsp;congestion notification from non-IP-aware congested nodes up t=
o the<br>
&nbsp; &nbsp;transport layer (L4). &nbsp;Following these guidelines should =
assure<br>
&nbsp; &nbsp;interworking between new lower layer congestion notification<b=
r>
&nbsp; &nbsp;mechanisms, whether specified by the IETF or other standards b=
odies.<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
<br>
________________________________________________________________<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;=
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbs=
p; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT <br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a></p>
</div>
<p class=3D"MsoNormal">&nbsp;</p>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>

--_000_A2E337CDB7BC4145B018B9BEE8EB3E0D3FFA6F97EDEMV67UKRDdoma_--

From touch@isi.edu  Tue Nov  5 10:39:00 2013
Return-Path: <touch@isi.edu>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 75FE321E809E for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 10:39:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.199
X-Spam-Level: 
X-Spam-Status: No, score=-103.199 tagged_above=-999 required=5 tests=[AWL=-0.827, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SpELT1V7FLUq for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 10:38:56 -0800 (PST)
Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id D45A111E80DE for <aqm@ietf.org>; Tue,  5 Nov 2013 10:38:39 -0800 (PST)
Received: from [128.9.160.166] (abc.isi.edu [128.9.160.166]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id rA5IcAAZ011239 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 5 Nov 2013 10:38:10 -0800 (PST)
Message-ID: <52793B87.4040102@isi.edu>
Date: Tue, 05 Nov 2013 10:40:07 -0800
From: Joe Touch <touch@isi.edu>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0
MIME-Version: 1.0
To: Michael Menth <menth@uni-tuebingen.de>, aqm@ietf.org
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk>	<CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de>
In-Reply-To: <52789FF5.3030907@uni-tuebingen.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 18:39:00 -0000

IMO, these guidelines ought to come out in a single recommendation for 
tunnels; we had a draft of that in INTAREA but insufficient momentum.

Piecemeal recommendations are likely to be ignored/lost.

Joe

On 11/4/2013 11:36 PM, Michael Menth wrote:
> +1
>
> We need such guidelines for consistent congestion management.
>
> Best wishes,
>
> Michael
>
> Am 05.11.2013 00:16, schrieb Matt Mathis:
>> I think this is valuable work.  Having a single document that
>> describes the requirements and general principles will save future
>> tunnel inventor/implementers from rediscovering the same bugs
>>
>> Thanks,
>> --MM--
>> The best way to predict the future is to create it.  - Alan Kay
>>
>> Privacy matters!  We know from recent events that people are using our
>> services to speak in defiance of unjust governments.   We treat
>> privacy and security as matters of life and death, because for some
>> users, they are.
>>
>>
>> On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>> <mailto:bob.briscoe@bt.com>> wrote:
>>
>>     Folks,
>>
>>     Pls respond if you support this being adopted as a work-group item
>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>     visibility of interest.
>>     Even better, if you're willing to read / comment / review / implement
>>
>>     Guidelines for Adding Congestion Notification to Protocols that
>>     Encapsulate IP
>>     <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>
>>     Abstract
>>
>>        The purpose of this document is to guide the design of congestion
>>        notification in any lower layer or tunnelling protocol that
>>        encapsulates IP.  The aim is for explicit congestion signals to
>>        propagate consistently from lower layer protocols into IP.
>>      Then the
>>        IP internetwork layer can act as a portability layer to carry
>>        congestion notification from non-IP-aware congested nodes up to the
>>        transport layer (L4).  Following these guidelines should assure
>>        interworking between new lower layer congestion notification
>>        mechanisms, whether specified by the IETF or other standards
>>     bodies.
>>
>>
>>     [Cross-posting tsvwg & aqm, just in case]
>>
>>
>>     Bob Briscoe,
>>     also for co-authors Pat Thaler and John Kaippallimalil
>>
>>
>>     ________________________________________________________________
>>     Bob Briscoe,    BT
>>     _______________________________________________
>>     aqm mailing list
>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>     https://www.ietf.org/mailman/listinfo/aqm
>>
>>
>>
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>
> --
> Prof. Dr. habil. Michael Menth
> University of Tuebingen
> Faculty of Science
> Department of Computer Science
> Chair of Communication Networks
> Sand 13, 72076 Tuebingen, Germany
> phone: (+49)-7071/29-70505
> fax: (+49)-7071/29-5220
> mailto:menth@uni-tuebingen.de
> http://kn.inf.uni-tuebingen.de
>
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>

From bob.briscoe@bt.com  Tue Nov  5 10:41:50 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8F2311E81CB; Tue,  5 Nov 2013 10:41:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.285
X-Spam-Level: 
X-Spam-Status: No, score=-3.285 tagged_above=-999 required=5 tests=[AWL=0.087,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GXIYLV-8kJ7f; Tue,  5 Nov 2013 10:41:43 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 3F1FC11E81EB; Tue,  5 Nov 2013 10:41:39 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR67-UKRD.bt.com (10.187.101.22) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 5 Nov 2013 18:41:34 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Tue, 5 Nov 2013 18:41:34 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Tue, 5 Nov 2013 18:41:33 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.124.253])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA5IfUeP030224; Tue, 5 Nov 2013 18:41:31 GMT
Message-ID: <201311051841.rA5IfUeP030224@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 5 Nov 2013 18:41:30 +0000
To: Piers O'Hanlon <p.ohanlon@gmail.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 18:41:50 -0000

Piers,

I tried to state exactly how ECN can benefit (2nd para of intro 
below), rather than make overblown claims. You seem to be saying I 
didn't succeed? I can't see how to do any better. Suggestions?

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
ECN removes nearly all congestion loss and it cuts delays for two 
main reasons: i) it avoids the delay when recovering from congestion 
losses, which particularly benefits small flows, making their 
completion time predictably short [RFC2884] and ii) as ECN is used 
more widely by end-systems, it will gradually remove the need to 
configure a degree of delay into buffers before they start to notify 
congestion (the cause of bufferbloat). The latter delay is because 
drop involves a trade-off between sending a timely signal and trying 
to avoid impairment, whereas ECN is solely a signal not an 
impairment, so there is no harm triggering it earlier.
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

PS. Thx for supporting words.

Bob

At 09:42 05/11/2013, Piers O'Hanlon wrote:
>Hi Bob,
>
>I took a brief look at the draft and it's clearly useful work.
>
>One thing that could do with clarification in the Introduction is 
>that ECN - by itself - doesn't necessarily lead to low loss and 
>delay - it should be made clear that it reflects the marking 
>approach of the underlying scheme/AQM.
>
>Piers
>
>On 4 Nov 2013, at 22:03, Bob Briscoe wrote:
>
> > Folks,
> >
> > Pls respond if you support this being adopted as a work-group 
> item in the IETF transport services w-g (tsvwg). The WG chairs need 
> visibility of interest.
> > Even better, if you're willing to read / comment / review / implement
> >
> > Guidelines for Adding Congestion Notification to Protocols that 
> Encapsulate IP
> > <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
> >
> > Abstract
> >
> >   The purpose of this document is to guide the design of congestion
> >   notification in any lower layer or tunnelling protocol that
> >   encapsulates IP.  The aim is for explicit congestion signals to
> >   propagate consistently from lower layer protocols into IP.  Then the
> >   IP internetwork layer can act as a portability layer to carry
> >   congestion notification from non-IP-aware congested nodes up to the
> >   transport layer (L4).  Following these guidelines should assure
> >   interworking between new lower layer congestion notification
> >   mechanisms, whether specified by the IETF or other standards bodies.
> >
> >
> > [Cross-posting tsvwg & aqm, just in case]
> >
> >
> > Bob Briscoe,
> > also for co-authors Pat Thaler and John Kaippallimalil
> >
> >
> > ________________________________________________________________
> > Bob Briscoe,                                                  BT
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 


From bob.briscoe@bt.com  Tue Nov  5 10:59:31 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE83111E81DE; Tue,  5 Nov 2013 10:59:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.288
X-Spam-Level: 
X-Spam-Status: No, score=-3.288 tagged_above=-999 required=5 tests=[AWL=0.084,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DTQa1v0+7xqr; Tue,  5 Nov 2013 10:59:26 -0800 (PST)
Received: from hubrelay-by-03.bt.com (hubrelay-by-03.bt.com [62.7.242.139]) by ietfa.amsl.com (Postfix) with ESMTP id B6B3021E811A; Tue,  5 Nov 2013 10:59:25 -0800 (PST)
Received: from EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) by EVMHR03-UKBR.bt.com (10.216.161.35) with Microsoft SMTP Server (TLS) id 8.3.327.1; Tue, 5 Nov 2013 18:59:20 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 5 Nov 2013 18:59:23 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Tue, 5 Nov 2013 18:59:21 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.124.253])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA5IxJvZ030310; Tue, 5 Nov 2013 18:59:20 GMT
Message-ID: <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 5 Nov 2013 18:59:18 +0000
To: Joe Touch <touch@isi.edu>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <52793B87.4040102@isi.edu>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: aqm@ietf.org, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 18:59:31 -0000

Joe,

I envisage that a very brief standards track doc that explicitly 
UPDATES the relevant IETF tunnel specs will be written, and it will 
refer to this doc for rationale.

See Appendix A (outstanding items), which I have also highlighted 
when presenting each time:

    2.  Consider whether an IETF Standard Track doc will be needed to
        Update the IP-in-IP protocols listed in Section 4.1--at least
        those that the IETF controls--and which Area it should sit under.

Does that address your concern?


Bob

At 18:40 05/11/2013, Joe Touch wrote:
>IMO, these guidelines ought to come out in a single recommendation 
>for tunnels; we had a draft of that in INTAREA but insufficient momentum.
>
>Piecemeal recommendations are likely to be ignored/lost.
>
>Joe
>
>On 11/4/2013 11:36 PM, Michael Menth wrote:
>>+1
>>
>>We need such guidelines for consistent congestion management.
>>
>>Best wishes,
>>
>>Michael
>>
>>Am 05.11.2013 00:16, schrieb Matt Mathis:
>>>I think this is valuable work.  Having a single document that
>>>describes the requirements and general principles will save future
>>>tunnel inventor/implementers from rediscovering the same bugs
>>>
>>>Thanks,
>>>--MM--
>>>The best way to predict the future is to create it.  - Alan Kay
>>>
>>>Privacy matters!  We know from recent events that people are using our
>>>services to speak in defiance of unjust governments.   We treat
>>>privacy and security as matters of life and death, because for some
>>>users, they are.
>>>
>>>
>>>On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>>><mailto:bob.briscoe@bt.com>> wrote:
>>>
>>>     Folks,
>>>
>>>     Pls respond if you support this being adopted as a work-group item
>>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>>     visibility of interest.
>>>     Even better, if you're willing to read / comment / review / implement
>>>
>>>     Guidelines for Adding Congestion Notification to Protocols that
>>>     Encapsulate IP
>>>     <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>>
>>>     Abstract
>>>
>>>        The purpose of this document is to guide the design of congestion
>>>        notification in any lower layer or tunnelling protocol that
>>>        encapsulates IP.  The aim is for explicit congestion signals to
>>>        propagate consistently from lower layer protocols into IP.
>>>      Then the
>>>        IP internetwork layer can act as a portability layer to carry
>>>        congestion notification from non-IP-aware congested nodes up to the
>>>        transport layer (L4).  Following these guidelines should assure
>>>        interworking between new lower layer congestion notification
>>>        mechanisms, whether specified by the IETF or other standards
>>>     bodies.
>>>
>>>
>>>     [Cross-posting tsvwg & aqm, just in case]
>>>
>>>
>>>     Bob Briscoe,
>>>     also for co-authors Pat Thaler and John Kaippallimalil
>>>
>>>
>>>     ________________________________________________________________
>>>     Bob Briscoe,    BT
>>>     _______________________________________________
>>>     aqm mailing list
>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>     https://www.ietf.org/mailman/listinfo/aqm
>>>
>>>
>>>
>>>
>>>_______________________________________________
>>>aqm mailing list
>>>aqm@ietf.org
>>>https://www.ietf.org/mailman/listinfo/aqm
>>
>>--
>>Prof. Dr. habil. Michael Menth
>>University of Tuebingen
>>Faculty of Science
>>Department of Computer Science
>>Chair of Communication Networks
>>Sand 13, 72076 Tuebingen, Germany
>>phone: (+49)-7071/29-70505
>>fax: (+49)-7071/29-5220
>>mailto:menth@uni-tuebingen.de
>>http://kn.inf.uni-tuebingen.de
>>
>>
>>
>>_______________________________________________
>>aqm mailing list
>>aqm@ietf.org
>>https://www.ietf.org/mailman/listinfo/aqm
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 


From suresh.krishnan@ericsson.com  Tue Nov  5 11:23:08 2013
Return-Path: <suresh.krishnan@ericsson.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 677C421E8096; Tue,  5 Nov 2013 11:23:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.35
X-Spam-Level: 
X-Spam-Status: No, score=-102.35 tagged_above=-999 required=5 tests=[AWL=0.022, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VAYL2mrUL3o9; Tue,  5 Nov 2013 11:23:03 -0800 (PST)
Received: from usevmg20.ericsson.net (usevmg20.ericsson.net [198.24.6.45]) by ietfa.amsl.com (Postfix) with ESMTP id 441D611E8159; Tue,  5 Nov 2013 11:22:18 -0800 (PST)
X-AuditID: c618062d-b7fd88e00000439f-b2-527945683726
Received: from EUSAAHC006.ericsson.se (Unknown_Domain [147.117.188.90]) by usevmg20.ericsson.net (Symantec Mail Security) with SMTP id A4.85.17311.86549725; Tue,  5 Nov 2013 20:22:16 +0100 (CET)
Received: from EUSAAMB108.ericsson.se ([147.117.188.125]) by EUSAAHC006.ericsson.se ([147.117.188.90]) with mapi id 14.02.0328.009; Tue, 5 Nov 2013 14:22:16 -0500
From: Suresh Krishnan <suresh.krishnan@ericsson.com>
To: Bob Briscoe <bob.briscoe@bt.com>, Joe Touch <touch@isi.edu>
Thread-Topic: [tsvwg] [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO2lkgxGg9YUDRb0SNPOHwSlpU15oXAb0A
Date: Tue, 5 Nov 2013 19:22:15 +0000
Message-ID: <E87B771635882B4BA20096B589152EF628440AA2@eusaamb108.ericsson.se>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu> <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk>
In-Reply-To: <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [147.117.188.134]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrLLMWRmVeSWpSXmKPExsUyuXRPlG6Ga2WQQVujjcWafZIW09d/YbRY 92cui8WxN3fZHFg82r5MZvJYsuQnk8fu91tZApijuGxSUnMyy1KL9O0SuDLOd25iLpilUbGl /xdbA+Na+S5GTg4JAROJky+OsUPYYhIX7q1n62Lk4hASOMIoce/kYWYIZxmjxKZPL8Gq2IA6 Nuz8zARiiwjYS5ze8oMFxGYWcJa4/mMlWI2wQKLEgXNfWCBqkiR+b77JDGEbSWx8/BnMZhFQ kThwfgZYDa+Ar8Tmxf2MEMuamSQaJn4AS3ACDd24fy/YMkag876fWsMEsUxc4taT+UwQZwtI LNlznhnCFpV4+fgfK4StLLHkyX6o43QkFuz+xAZha0ssW/iaGWKxoMTJmU9YJjCKzUIydhaS lllIWmYhaVnAyLKKkaO0OLUsN93IYBMjMIqOSbDp7mDc89LyEKM0B4uSOO+Xt85BQgLpiSWp 2ampBalF8UWlOanFhxiZODilGhi162Na7v38vcHJtn96+MWV1QcSHnlG9mxyPaPyzvLhndbY V1PkBe9zJSdtFSteLvxfTDtFaL6qQkKeo+Ae7iv3lpVJTrFmEu+76c3xZp7nuX0ajh87E0UC nr7dfO6y4vVQw5/lCl1+/4oCvm0QFos4dOfu6w/eqYvanaYcvbjm5pwPh8xqTwYqsRRnJBpq MRcVJwIAnY6z+3ACAAA=
X-Mailman-Approved-At: Tue, 05 Nov 2013 11:26:42 -0800
Cc: "aqm@ietf.org" <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 19:23:08 -0000

I do support this work going forward. I do understand Joe's position/desire=
 and while it would be preferable to have a comprehensive document,  I thin=
k draft-briscoe is useful by itself. FYI, Joe's draft can be found here

http://tools.ietf.org/html/draft-ietf-intarea-tunnels-00

Thanks
Suresh
=20
> -----Original Message-----
> From: tsvwg-bounces@ietf.org [mailto:tsvwg-bounces@ietf.org] On Behalf
> Of Bob Briscoe
> Sent: November-05-13 1:59 PM
> To: Joe Touch
> Cc: aqm@ietf.org; tsvwg IETF list
> Subject: Re: [tsvwg] [aqm] Who supports tsvwg adoption of adding ECN to L=
2
> or tunnel protocols?
>=20
> Joe,
>=20
> I envisage that a very brief standards track doc that explicitly UPDATES =
the
> relevant IETF tunnel specs will be written, and it will refer to this doc=
 for
> rationale.
>=20
> See Appendix A (outstanding items), which I have also highlighted when
> presenting each time:
>=20
>     2.  Consider whether an IETF Standard Track doc will be needed to
>         Update the IP-in-IP protocols listed in Section 4.1--at least
>         those that the IETF controls--and which Area it should sit under.
>=20
> Does that address your concern?
>=20
>=20
> Bob
>=20
> At 18:40 05/11/2013, Joe Touch wrote:
> >IMO, these guidelines ought to come out in a single recommendation for
> >tunnels; we had a draft of that in INTAREA but insufficient momentum.
> >
> >Piecemeal recommendations are likely to be ignored/lost.
> >
> >Joe
> >
> >On 11/4/2013 11:36 PM, Michael Menth wrote:
> >>+1
> >>
> >>We need such guidelines for consistent congestion management.
> >>
> >>Best wishes,
> >>
> >>Michael
> >>
> >>Am 05.11.2013 00:16, schrieb Matt Mathis:
> >>>I think this is valuable work.  Having a single document that
> >>>describes the requirements and general principles will save future
> >>>tunnel inventor/implementers from rediscovering the same bugs
> >>>
> >>>Thanks,
> >>>--MM--
> >>>The best way to predict the future is to create it.  - Alan Kay
> >>>
> >>>Privacy matters!  We know from recent events that people are using our
> >>>services to speak in defiance of unjust governments.   We treat
> >>>privacy and security as matters of life and death, because for some
> >>>users, they are.
> >>>
> >>>
> >>>On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
> >>><mailto:bob.briscoe@bt.com>> wrote:
> >>>
> >>>     Folks,
> >>>
> >>>     Pls respond if you support this being adopted as a work-group ite=
m
> >>>     in the IETF transport services w-g (tsvwg). The WG chairs need
> >>>     visibility of interest.
> >>>     Even better, if you're willing to read / comment / review /
> >>> implement
> >>>
> >>>     Guidelines for Adding Congestion Notification to Protocols that
> >>>     Encapsulate IP
> >>>
> >>> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines
> >>> >
> >>>
> >>>     Abstract
> >>>
> >>>        The purpose of this document is to guide the design of congest=
ion
> >>>        notification in any lower layer or tunnelling protocol that
> >>>        encapsulates IP.  The aim is for explicit congestion signals t=
o
> >>>        propagate consistently from lower layer protocols into IP.
> >>>      Then the
> >>>        IP internetwork layer can act as a portability layer to carry
> >>>        congestion notification from non-IP-aware congested nodes up t=
o
> the
> >>>        transport layer (L4).  Following these guidelines should assur=
e
> >>>        interworking between new lower layer congestion notification
> >>>        mechanisms, whether specified by the IETF or other standards
> >>>     bodies.
> >>>
> >>>
> >>>     [Cross-posting tsvwg & aqm, just in case]
> >>>
> >>>
> >>>     Bob Briscoe,
> >>>     also for co-authors Pat Thaler and John Kaippallimalil
> >>>
> >>>
> >>>
> __________________________________________________________
> ______
> >>>     Bob Briscoe,    BT
> >>>     _______________________________________________
> >>>     aqm mailing list
> >>>     aqm@ietf.org <mailto:aqm@ietf.org>
> >>>     https://www.ietf.org/mailman/listinfo/aqm
> >>>
> >>>
> >>>
> >>>
> >>>_______________________________________________
> >>>aqm mailing list
> >>>aqm@ietf.org
> >>>https://www.ietf.org/mailman/listinfo/aqm
> >>
> >>--
> >>Prof. Dr. habil. Michael Menth
> >>University of Tuebingen
> >>Faculty of Science
> >>Department of Computer Science
> >>Chair of Communication Networks
> >>Sand 13, 72076 Tuebingen, Germany
> >>phone: (+49)-7071/29-70505
> >>fax: (+49)-7071/29-5220
> >>mailto:menth@uni-tuebingen.de
> >>http://kn.inf.uni-tuebingen.de
> >>
> >>
> >>
> >>_______________________________________________
> >>aqm mailing list
> >>aqm@ietf.org
> >>https://www.ietf.org/mailman/listinfo/aqm
> >_______________________________________________
> >aqm mailing list
> >aqm@ietf.org
> >https://www.ietf.org/mailman/listinfo/aqm
>=20
> __________________________________________________________
> ______
> Bob Briscoe,                                                  BT


From touch@isi.edu  Tue Nov  5 12:49:06 2013
Return-Path: <touch@isi.edu>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7195D21E8093; Tue,  5 Nov 2013 12:49:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.177
X-Spam-Level: 
X-Spam-Status: No, score=-103.177 tagged_above=-999 required=5 tests=[AWL=-0.805, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QYquuis4mhyN; Tue,  5 Nov 2013 12:49:02 -0800 (PST)
Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id 3390811E8175; Tue,  5 Nov 2013 12:49:02 -0800 (PST)
Received: from [128.9.160.166] (abc.isi.edu [128.9.160.166]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id rA5KmDEp021494 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 5 Nov 2013 12:48:14 -0800 (PST)
Message-ID: <52795A03.9010804@isi.edu>
Date: Tue, 05 Nov 2013 12:50:11 -0800
From: Joe Touch <touch@isi.edu>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0
MIME-Version: 1.0
To: Bob Briscoe <bob.briscoe@bt.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu> <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk>
In-Reply-To: <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Cc: aqm@ietf.org, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 20:49:06 -0000

On 11/5/2013 10:59 AM, Bob Briscoe wrote:
> Joe,
>
> I envisage that a very brief standards track doc that explicitly UPDATES
> the relevant IETF tunnel specs will be written, and it will refer to
> this doc for rationale.

Tunnels need to handle ingress/egress translation of all signals in the 
header. This is no different.

My concern is that putting these recommendations in separate places 
gives an opportunity for different groups to have different 
interpretations of that sort of translation, and that's a bad thing IMO.

Joe


>
> See Appendix A (outstanding items), which I have also highlighted when
> presenting each time:
>
>     2.  Consider whether an IETF Standard Track doc will be needed to
>         Update the IP-in-IP protocols listed in Section 4.1--at least
>         those that the IETF controls--and which Area it should sit under.
>
> Does that address your concern?
>
>
> Bob
>
> At 18:40 05/11/2013, Joe Touch wrote:
>> IMO, these guidelines ought to come out in a single recommendation for
>> tunnels; we had a draft of that in INTAREA but insufficient momentum.
>>
>> Piecemeal recommendations are likely to be ignored/lost.
>>
>> Joe
>>
>> On 11/4/2013 11:36 PM, Michael Menth wrote:
>>> +1
>>>
>>> We need such guidelines for consistent congestion management.
>>>
>>> Best wishes,
>>>
>>> Michael
>>>
>>> Am 05.11.2013 00:16, schrieb Matt Mathis:
>>>> I think this is valuable work.  Having a single document that
>>>> describes the requirements and general principles will save future
>>>> tunnel inventor/implementers from rediscovering the same bugs
>>>>
>>>> Thanks,
>>>> --MM--
>>>> The best way to predict the future is to create it.  - Alan Kay
>>>>
>>>> Privacy matters!  We know from recent events that people are using our
>>>> services to speak in defiance of unjust governments.   We treat
>>>> privacy and security as matters of life and death, because for some
>>>> users, they are.
>>>>
>>>>
>>>> On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>>>> <mailto:bob.briscoe@bt.com>> wrote:
>>>>
>>>>     Folks,
>>>>
>>>>     Pls respond if you support this being adopted as a work-group item
>>>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>>>     visibility of interest.
>>>>     Even better, if you're willing to read / comment / review /
>>>> implement
>>>>
>>>>     Guidelines for Adding Congestion Notification to Protocols that
>>>>     Encapsulate IP
>>>>
>>>> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>>>
>>>>     Abstract
>>>>
>>>>        The purpose of this document is to guide the design of
>>>> congestion
>>>>        notification in any lower layer or tunnelling protocol that
>>>>        encapsulates IP.  The aim is for explicit congestion signals to
>>>>        propagate consistently from lower layer protocols into IP.
>>>>      Then the
>>>>        IP internetwork layer can act as a portability layer to carry
>>>>        congestion notification from non-IP-aware congested nodes up
>>>> to the
>>>>        transport layer (L4).  Following these guidelines should assure
>>>>        interworking between new lower layer congestion notification
>>>>        mechanisms, whether specified by the IETF or other standards
>>>>     bodies.
>>>>
>>>>
>>>>     [Cross-posting tsvwg & aqm, just in case]
>>>>
>>>>
>>>>     Bob Briscoe,
>>>>     also for co-authors Pat Thaler and John Kaippallimalil
>>>>
>>>>
>>>>     ________________________________________________________________
>>>>     Bob Briscoe,    BT
>>>>     _______________________________________________
>>>>     aqm mailing list
>>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>>     https://www.ietf.org/mailman/listinfo/aqm
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> aqm mailing list
>>>> aqm@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/aqm
>>>
>>> --
>>> Prof. Dr. habil. Michael Menth
>>> University of Tuebingen
>>> Faculty of Science
>>> Department of Computer Science
>>> Chair of Communication Networks
>>> Sand 13, 72076 Tuebingen, Germany
>>> phone: (+49)-7071/29-70505
>>> fax: (+49)-7071/29-5220
>>> mailto:menth@uni-tuebingen.de
>>> http://kn.inf.uni-tuebingen.de
>>>
>>>
>>>
>>> _______________________________________________
>>> aqm mailing list
>>> aqm@ietf.org
>>> https://www.ietf.org/mailman/listinfo/aqm
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT

From andrewmcgr@google.com  Tue Nov  5 14:59:53 2013
Return-Path: <andrewmcgr@google.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A9E5921E80E3 for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 14:59:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.39
X-Spam-Level: 
X-Spam-Status: No, score=-1.39 tagged_above=-999 required=5 tests=[AWL=0.360,  BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e-HpctNPhPbJ for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 14:59:52 -0800 (PST)
Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) by ietfa.amsl.com (Postfix) with ESMTP id 7C90021E80BD for <aqm@ietf.org>; Tue,  5 Nov 2013 14:59:52 -0800 (PST)
Received: by mail-qa0-f43.google.com with SMTP id cm18so1528754qab.16 for <aqm@ietf.org>; Tue, 05 Nov 2013 14:59:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Lf/5G9teQxJ988SAafneSc1NNFqkvD0m7fJmlBGuul0=; b=fowY2i8ff1U6odbN0fhgy08Ln3NP4i+OCl2yRDiCKOykD0YAZWe3lsiJIdOvEjD2gb trHgDkqJa8nCJZHK4KOI3BDkMPVcTZRRLlwT67KOhocCaIQuw/ZaIPmFQlw7ZmYL/zEF UCk7iSyiLx7Dsy1M3ECrrKnNKYo1A/dhk0JNfYC8G9mRqroS/vdmWWcSrjhulUA3cH6r sjm2hVcPcehkCDJ/uyNZHQ9/OmjY1L00/+MZlHOjGbn66HMoJfyPrEBeau2DBvQ4PR6H vbO+Uu2WdsLEUM2KAaFzXwT3710S+wkSCWD/mmj50ROaeMyNO913U7R2reyZ9Ur1jjU+ qlVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Lf/5G9teQxJ988SAafneSc1NNFqkvD0m7fJmlBGuul0=; b=P6QTELIGirK7onkTxV7EDT1hF9JmDeugeDGPale14tlaXGsqcfn8uWtFT2r1ex0J5e NaJSXM4HbcH2RHontyWTg+WtRyEPk45A9eXJwCAqHCwz4jm7LsCggTCfcawXvu3vLzEO r+RRzU2kuHem+bx4oVnLlvKf3756jWw6UNgqLnxz0UFlb3g93dmhof2Ix6xJrQZwXaOz BzVeDJnwuyRwHdn9CSpFKATNhw3yK8q1Wm1UCCDlexSmzqlqPc98fFkvtt07LuN+93+v kSwVHbAVDyNOB+PLoF4354h+t91R/+BIOnOQ9jnOlCAtIbJrx+XxseWxqRrdZxYIaFJv c8Kw==
X-Gm-Message-State: ALoCoQlWAJXIjMFKDoifS1+tNI0FW9kk0+uF/tyQJZdmpCpIgGzt8Rz13wgX4zHUofn2Sr0YL2300m4Qp0JcadnFe1m75MPpxPnV4IWrtVgbXqBNHCkLwQN8oL85X1QuDBi4doxd83PvgqHJuH1L5ATCIjHsiVwk0/x8ye7j0e5EwmpN8E9hKO74aqUjqelDQYmZnS3yDrha
MIME-Version: 1.0
X-Received: by 10.224.11.7 with SMTP id r7mr1227819qar.91.1383692391960; Tue, 05 Nov 2013 14:59:51 -0800 (PST)
Received: by 10.224.197.198 with HTTP; Tue, 5 Nov 2013 14:59:51 -0800 (PST)
In-Reply-To: <201311050000.rA500RMV027031@bagheera.jungle.bt.co.uk>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAPRuP3kRsoPP2e9+vBjn1xiZ1UWJpc8Ds5VhsggKWa2suj_19Q@mail.gmail.com> <201311050000.rA500RMV027031@bagheera.jungle.bt.co.uk>
Date: Wed, 6 Nov 2013 09:59:51 +1100
Message-ID: <CAPRuP3kXx-xJC7kmaa0pfdMNQUuKXsf+ZLJ=WooE57Gg6A4=Ew@mail.gmail.com>
From: Andrew Mcgregor <andrewmcgr@google.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Content-Type: multipart/alternative; boundary=001a11c2c12680725d04ea75fd14
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 22:59:53 -0000

--001a11c2c12680725d04ea75fd14
Content-Type: text/plain; charset=ISO-8859-1

That's about what I thought the answer would be... but it's worth having
that in the record :-)


On 5 November 2013 11:00, Bob Briscoe <bob.briscoe@bt.com> wrote:

>  Andrew,
>
> Thx for supporting words and offer of help.
>
>
> At 23:04 04/11/2013, Andrew Mcgregor wrote:
>
> This seems like valuable work.  One question is, can we put in scope
> notification that losses are NOT due to congestion?
>
>
> Eyooow. That really does not belong in this scope.
>
> This is about the interface between lower layers and the ECN field in the
> IP header. We can't really talk about new semantics in lower layers that
> aren't even defined in ECN at the IP layer.
>
> Naively, one could imagine some text treating the ECN field as 4 opaque
> codepoints, and propagating  a similar 4 opaque codepoints from each lower
> layer protocol. Then if something new were defined in IP-ECN like a
> non-congestive loss codepoint, it could also be defined in lower layers...
>
> But that's naive, because each lower layer has it's own constraints and
> ways of handling concepts like "ECN-capable transport", none of which
> easily translates to a simple mapping of codepoints.
>
> Explicit signalling of non-congestive loss is a research problem,
> because...
> * Clearly a link cannot signal that a loss was not due to congestion using
> the lost frame itself (there aren't any bits :)
> * So the link would have to signal in another frame that didn't get
> corrupted.
>   - That frame may go to another process that didn't see any loss; then
> what does the process do?
>   - or the link has to use DPI to signal only in frames going to the same
> process as the lost frame.
>   - that means the link has to understand all transport protocols
>
> There's other hard problems
> * A transmission loss doesn't always mean there is no congestion loss
> * At my last count, there were at least nine different reasons for packet
> loss. If we had 4 bits spare in each lost packet... :)
>
>
>
> Bob
>
>
>
> Willing to comment and review, at least.
>
>
> On 5 November 2013 09:03, Bob Briscoe <bob.briscoe@bt.com> wrote:
>  Folks,
>
> Pls respond if you support this being adopted as a work-group item in the
> IETF transport services w-g (tsvwg). The WG chairs need visibility of
> interest.
> Even better, if you're willing to read / comment / review / implement
>
> Guidelines for Adding Congestion Notification to Protocols that
> Encapsulate IP
>  < http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines >
>
> Abstract
>
>    The purpose of this document is to guide the design of congestion
>    notification in any lower layer or tunnelling protocol that
>    encapsulates IP.  The aim is for explicit congestion signals to
>    propagate consistently from lower layer protocols into IP.  Then the
>    IP internetwork layer can act as a portability layer to carry
>    congestion notification from non-IP-aware congested nodes up to the
>    transport layer (L4).  Following these guidelines should assure
>    interworking between new lower layer congestion notification
>    mechanisms, whether specified by the IETF or other standards bodies.
>
>
> [Cross-posting tsvwg & aqm, just in case]
>
>
> Bob Briscoe,
> also for co-authors Pat Thaler and John Kaippallimalil
>
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>
>
>
> --
> Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
>  https://www.ietf.org/mailman/listinfo/aqm
>
>  ________________________________________________________________
> Bob Briscoe,                                                  BT
>



-- 
Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128

--001a11c2c12680725d04ea75fd14
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">That&#39;s about what I thought the answer would be... but=
 it&#39;s worth having that in the record :-)</div><div class=3D"gmail_extr=
a"><br><br><div class=3D"gmail_quote">On 5 November 2013 11:00, Bob Briscoe=
 <span dir=3D"ltr">&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_bla=
nk">bob.briscoe@bt.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div>
Andrew,<br><br>
Thx for supporting words and offer of help.<div class=3D"im"><br><br>
At 23:04 04/11/2013, Andrew Mcgregor wrote:<br>
<blockquote type=3D"cite">This seems like valuable
work.=A0 One question is, can we put in scope notification that losses
are NOT due to congestion?</blockquote><br></div>
Eyooow. That really does not belong in this scope. <br><br>
This is about the interface between lower layers and the ECN field in the
IP header. We can&#39;t really talk about new semantics in lower layers tha=
t
aren&#39;t even defined in ECN at the IP layer.<br><br>
Naively, one could imagine some text treating the ECN field as 4 opaque
codepoints, and propagating=A0 a similar 4 opaque codepoints from each
lower layer protocol. Then if something new were defined in IP-ECN like a
non-congestive loss codepoint, it could also be defined in lower
layers...<br><br>
But that&#39;s naive, because each lower layer has it&#39;s own constraints=
 and
ways of handling concepts like &quot;ECN-capable transport&quot;, none of
which easily translates to a simple mapping of codepoints.<br><br>
Explicit signalling of non-congestive loss is a research problem,
because...<br>
* Clearly a link cannot signal that a loss was not due to congestion
using the lost frame itself (there aren&#39;t any bits :)<br>
* So the link would have to signal in another frame that didn&#39;t get
corrupted.<br>
=A0 - That frame may go to another process that didn&#39;t see any loss;
then what does the process do?<br>
=A0 - or the link has to use DPI to signal only in frames going to the
same process as the lost frame.<br>
=A0 - that means the link has to understand all transport
protocols<br><br>
There&#39;s other hard problems<br>
* A transmission loss doesn&#39;t always mean there is no congestion
loss<br>
* At my last count, there were at least nine different reasons for packet
loss. If we had 4 bits spare in each lost packet... :)<br><br>
<br><br>
Bob<div><div class=3D"h5"><br><br>
<br>
<blockquote type=3D"cite">Willing to comment and review,
at least.<br><br>
<br>
On 5 November 2013 09:03, Bob Briscoe
&lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blank">bob.briscoe@bt.=
com</a>&gt;
wrote:<br>

<dl>
<dd>Folks,<br><br>

</dd><dd>Pls respond if you support this being adopted as a work-group item=
 in
the IETF transport services w-g (tsvwg). The WG chairs need visibility of
interest.<br>

</dd><dd>Even better, if you&#39;re willing to read / comment / review /
implement<br><br>

</dd><dd>Guidelines for Adding Congestion Notification to Protocols that
Encapsulate IP<br>

</dd><dd>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">
http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines</a>
&gt;<br><br>

</dd><dd>Abstract<br><br>

</dd><dd>=A0=A0 The purpose of this document is to guide the design of
congestion<br>

</dd><dd>=A0=A0 notification in any lower layer or tunnelling protocol
that<br>

</dd><dd>=A0=A0 encapsulates IP.=A0 The aim is for explicit
congestion signals to<br>

</dd><dd>=A0=A0 propagate consistently from lower layer protocols into
IP.=A0 Then the<br>

</dd><dd>=A0=A0 IP internetwork layer can act as a portability layer to
carry<br>

</dd><dd>=A0=A0 congestion notification from non-IP-aware congested
nodes up to the<br>

</dd><dd>=A0=A0 transport layer (L4).=A0 Following these guidelines
should assure<br>

</dd><dd>=A0=A0 interworking between new lower layer congestion
notification<br>

</dd><dd>=A0=A0 mechanisms, whether specified by the IETF or other
standards bodies.<br><br>
<br>

</dd><dd>[Cross-posting tsvwg &amp; aqm, just in case]<br><br>
<br>

</dd><dd>Bob Briscoe,<br>

</dd><dd>also for co-authors Pat Thaler and John Kaippallimalil<br><br>
<br>

</dd><dd>________________________________________________________________<b=
r>

</dd><dd>Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT <br>

</dd><dd>_______________________________________________<br>

</dd><dd>aqm mailing list<br>

</dd><dd><a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a>=
<br>

</dd><dd><a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_b=
lank">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dd></dl><br><br>
<br>
-- <br>
Andrew McGregor | SRE |
<a href=3D"mailto:andrewmcgr@google.com" target=3D"_blank">andrewmcgr@googl=
e.com</a> | +61 4
8143 7128<br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<u></u></div></div><p><u></u>
________________________________________________________________<br>
Bob
Briscoe,=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0
BT</p></div>

</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div dir=3D"=
ltr"><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:sm=
all;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-co=
lor:rgb(213,15,37);padding-top:2px;margin-top:2px">Andrew McGregor=A0|</spa=
n><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small=
;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color=
:rgb(51,105,232);padding-top:2px;margin-top:2px">=A0SRE=A0|</span><span sty=
le=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-heigh=
t:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,153,=
57);padding-top:2px;margin-top:2px">=A0<a href=3D"mailto:andrewmcgr@google.=
com" target=3D"_blank">andrewmcgr@google.com</a>=A0|</span><span style=3D"c=
olor:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em=
;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);p=
adding-top:2px;margin-top:2px">=A0+61 4 8143 7128</span><br>
</div>
</div>

--001a11c2c12680725d04ea75fd14--

From rs@netapp.com  Tue Nov  5 15:02:25 2013
Return-Path: <rs@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DEB511E8185 for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 15:02:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.453
X-Spam-Level: 
X-Spam-Status: No, score=-9.453 tagged_above=-999 required=5 tests=[AWL=1.146,  BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2WQ7-X7nnk77 for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 15:02:20 -0800 (PST)
Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) by ietfa.amsl.com (Postfix) with ESMTP id 7A41111E814F for <aqm@ietf.org>; Tue,  5 Nov 2013 15:02:20 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,642,1378882800"; d="scan'208";a="110554596"
Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx12-out.netapp.com with ESMTP; 05 Nov 2013 15:02:20 -0800
Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.86]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Tue, 5 Nov 2013 15:02:20 -0800
From: "Scheffenegger, Richard" <rs@netapp.com>
To: "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: Ac7aemDyYVICOG/TQKCzUJnt0vYtGg==
Date: Tue, 5 Nov 2013 23:02:18 +0000
Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: de-AT, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.104.60.115]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "Fred Baker \(fred\) \(fred@cisco.com\)" <fred@cisco.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 23:02:25 -0000

Hi,

this is just a simple remainder that the WG will convene also this Friday;

As the mechanism update presentations took quite a bit longer than we had e=
xpected - and we appreciate the discussion these presentations have sparked=
 - we agreed with Fred to have his presentation as the first agenda point o=
n Friday; the second agenda point will be a presentation around AQM evaluat=
ion process and criteria.


The Room will be Regency B (3. Floor), and we will start at 12:30.


Best regards,

Richard Scheffenegger


From rs@netapp.com  Tue Nov  5 15:04:53 2013
Return-Path: <rs@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C86A921E80E3 for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 15:04:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.548
X-Spam-Level: 
X-Spam-Status: No, score=-9.548 tagged_above=-999 required=5 tests=[AWL=1.050,  BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z76s-gsKv5Qc for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 15:04:48 -0800 (PST)
Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) by ietfa.amsl.com (Postfix) with ESMTP id AF2B721E80CA for <aqm@ietf.org>; Tue,  5 Nov 2013 15:04:48 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,642,1378882800";  d="scan'208,217";a="110555506"
Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx12-out.netapp.com with ESMTP; 05 Nov 2013 15:04:48 -0800
Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.86]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Tue, 5 Nov 2013 15:04:48 -0800
From: "Scheffenegger, Richard" <rs@netapp.com>
To: "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: ECN use statement
Thread-Index: Ac7aez2kx5olm7OoQX2X5d/R+TcByQ==
Date: Tue, 5 Nov 2013 23:04:47 +0000
Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25E6A876@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: de-AT, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.104.60.115]
Content-Type: multipart/alternative; boundary="_000_012C3117EDDB3C4781FD802A8C27DD4F25E6A876SACEXCMBX02PRDh_"
MIME-Version: 1.0
Subject: [aqm] ECN use statement
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 23:04:53 -0000

--_000_012C3117EDDB3C4781FD802A8C27DD4F25E6A876SACEXCMBX02PRDh_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

In today's discussion it became apparent, that we might need a stronger sta=
tement around the expected use of the IP ECN signaling, and encourage futur=
e implementers to support this, and also agree around the way how ECN shoul=
d be used.

I would like to encourage discussion around that interaction here on list..=
.

Richard Scheffenegger



--_000_012C3117EDDB3C4781FD802A8C27DD4F25E6A876SACEXCMBX02PRDh_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;">
<div>Hi,</div>
<div>&nbsp;</div>
<div>In today&#8217;s discussion it became apparent, that we might need a s=
tronger statement around the expected use of the IP ECN signaling, and enco=
urage future implementers to support this, and also agree around the way ho=
w ECN should be used.</div>
<div>&nbsp;</div>
<div>I would like to encourage discussion around that interaction here on l=
ist&#8230;</div>
<div>&nbsp;</div>
<div>Richard Scheffenegger<br>

</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_012C3117EDDB3C4781FD802A8C27DD4F25E6A876SACEXCMBX02PRDh_--

From fred@cisco.com  Tue Nov  5 15:30:46 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1632C21F8531 for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 15:30:46 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.348
X-Spam-Level: 
X-Spam-Status: No, score=-110.348 tagged_above=-999 required=5 tests=[AWL=0.251, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aMRuMZ0cu5Ed for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 15:30:40 -0800 (PST)
Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) by ietfa.amsl.com (Postfix) with ESMTP id 5EE3021F9BC2 for <aqm@ietf.org>; Tue,  5 Nov 2013 15:30:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1995; q=dns/txt; s=iport; t=1383694237; x=1384903837; h=from:to:subject:date:message-id:mime-version; bh=Ryv1OT4r21TpxjhBxkdHuDp+9awTtmS7cj430h672h4=; b=EKbbGZvqLNKHDHSthkrk5d+CTfOkR8AFH+fsAq3Gnk06/inNIg6P2LVR Gj9R3ej6Hr90/mdRSPHkODbqRyoq/+DcSgJxoLi668E3RRCPSncnFCxWH smHF6aHFtJEGGM4XPUY1r6tR4m5Tol16i7SzEuMvNJKPYUxte7bIAy6Af w=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhQFAM5+eVKtJXG+/2dsb2JhbABZgweBC8BtFnSCJQEBAQQdbgEgYCcEEw6Hc50loU2TAIEPA5AugTCGLJIJgyaCKg
X-IronPort-AV: E=Sophos;i="4.93,642,1378857600";  d="asc'?scan'208";a="280964167"
Received: from rcdn-core2-3.cisco.com ([173.37.113.190]) by rcdn-iport-1.cisco.com with ESMTP; 05 Nov 2013 23:30:10 +0000
Received: from xhc-aln-x11.cisco.com (xhc-aln-x11.cisco.com [173.36.12.85]) by rcdn-core2-3.cisco.com (8.14.5/8.14.5) with ESMTP id rA5NUAti009659 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for <aqm@ietf.org>; Tue, 5 Nov 2013 23:30:10 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x11.cisco.com ([173.36.12.85]) with mapi id 14.03.0123.003; Tue, 5 Nov 2013 17:30:10 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: AQM IETF list <aqm@ietf.org>
Thread-Topic: Recommendations Draft discussion
Thread-Index: AQHO2n73JkSlcyICNkqxiw8Q+x6a7g==
Date: Tue, 5 Nov 2013 23:30:09 +0000
Message-ID: <664F9AFD-9D16-449D-9BEC-0326BD87AB4B@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.21.75.25]
Content-Type: multipart/signed; boundary="Apple-Mail=_8101C671-A10A-4000-A5CD-6C80E7B22D74"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Subject: [aqm] Recommendations Draft discussion
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 23:30:46 -0000

--Apple-Mail=_8101C671-A10A-4000-A5CD-6C80E7B22D74
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On Nov 5, 2013, at 3:02 PM, "Scheffenegger, Richard" <rs@netapp.com> =
wrote:
> As the mechanism update presentations took quite a bit longer than we =
had expected - and we appreciate the discussion these presentations have =
sparked - we agreed with Fred to have his presentation as the first =
agenda point on Friday; the second agenda point will be a presentation =
around AQM evaluation process and criteria.

What Gorry and I will try to achieve Friday is the high bandwidth =
discussion necessary to put the recommendations of the draft to bed. =
There has been list discussion of them, and we have attempted to address =
the points raised in the current version of the draft.=20

If you will not be present Friday and have issues or concerns with the =
draft in any way, and especially with its recommendations, I would =
appreciate it if you could post them to the list *prior* to the meeting, =
so that they can be noted and discussed by those present. It would help =
Gorry and I if other people would do the same in addition to raising =
them at the mike, the point again being discussion and (with any luck) =
the achievement of consensus.

What we'd like to believe is that we can do a quick spin on the draft =
next week, and the chairs can call for closure when that is posted.

--Apple-Mail=_8101C671-A10A-4000-A5CD-6C80E7B22D74
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSeX9/bjEdbHIsm0MRAsgGAKCHeri5w5n+icC+i5UgZXPQMUZ1MgCfa0yV
y4ASQBq/LCH5yaf1PnyQGik=
=urzA
-----END PGP SIGNATURE-----

--Apple-Mail=_8101C671-A10A-4000-A5CD-6C80E7B22D74--

From naeem.khademi@gmail.com  Tue Nov  5 17:31:08 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D12B521F9C8E for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 17:31:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.372
X-Spam-Level: 
X-Spam-Status: No, score=-2.372 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6Q9BnQwMsnD7 for <aqm@ietfa.amsl.com>; Tue,  5 Nov 2013 17:31:07 -0800 (PST)
Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) by ietfa.amsl.com (Postfix) with ESMTP id BD7E321F9B0E for <aqm@ietf.org>; Tue,  5 Nov 2013 17:31:07 -0800 (PST)
Received: by mail-vc0-f172.google.com with SMTP id ks9so6170318vcb.17 for <aqm@ietf.org>; Tue, 05 Nov 2013 17:31:07 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jDxJi7qiGRTixShVNoc3zqpDd3AaQvlkpkQAFQertSA=; b=dgxiwc4F9+yjyjsaOa2Aq1p8vHlZTtxf1ERg12MuW61NcCanHGrd+bHXvR5Im1kYwy G9vAp3StLvtPy8vPTQmQN7ZmO9lh/+BkNYdi7OQ+NtJbsZrbJGbYeeEeSP71V7BSmSus CAS72QUVxXvQGSRNIDlW85scS+cjKvXCf8FiBIMnVcJN8tfTsZR9SaLnjqlnHr668hSI 4cD51C4vXccvSzhDznnsvF9Au6hbU9Cy0wn9U/7LDhEFBYolP+9tyV9JPAcz0zgH4OOH GOjam9kf4bG83FBbd/Z5zxBYULqk9SE1sx0Cto2vBGcobd/MQPtU9pe4KZZ14idl9qAu B+LQ==
MIME-Version: 1.0
X-Received: by 10.58.29.37 with SMTP id g5mr282571veh.38.1383701466967; Tue, 05 Nov 2013 17:31:06 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Tue, 5 Nov 2013 17:31:06 -0800 (PST)
Date: Tue, 5 Nov 2013 17:31:06 -0800
Message-ID: <CAEjQQ5UdObE5nh8mc-34zCT1K+ZRJJgpBBFmi=ErAtb_-Y2uBw@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>,  "iccrg@irtf.org" <iccrg@irtf.org>
Content-Type: multipart/alternative; boundary=047d7b6786fe69f60f04ea781a96
Subject: [aqm] New AQM Kids on the block -- experimental evaluation of CoDel and PIE (and ARED)
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 01:31:08 -0000

--047d7b6786fe69f60f04ea781a96
Content-Type: text/plain; charset=ISO-8859-1

Hi

This might be of interest of some of AQMers:

The slides of ICCRG talk at IETF88 comparing the performance of CoDel, PIE
and ARED is available at:
http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf

A detailed technical report with more experimental results can be found at:
http://urn.nb.no/URN:NBN:no-38868

Please don't hesitate to provide feedback.

Regards,
Naeem

--047d7b6786fe69f60f04ea781a96
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><font face=3D"arial, helvetica, sans-serif">Hi=A0</fo=
nt></div><font face=3D"arial, helvetica, sans-serif"><div><font face=3D"ari=
al, helvetica, sans-serif"><br></font></div><div><font face=3D"arial, helve=
tica, sans-serif">This might be of interest of some of AQMers:</font></div>
<div><font face=3D"arial, helvetica, sans-serif"><br></font></div>The slide=
s of ICCRG talk at IETF88 comparing the performance of CoDel, PIE and ARED =
is available at:=A0<a href=3D"http://www.ietf.org/proceedings/88/slides/sli=
des-88-iccrg-4.pdf">http://www.ietf.org/proceedings/88/slides/slides-88-icc=
rg-4.pdf</a></font><div>
<font face=3D"arial, helvetica, sans-serif"><br></font></div><div><font fac=
e=3D"arial, helvetica, sans-serif">A detailed technical report with more ex=
perimental results can be found at:=A0<span style=3D"font-size:13px"><a hre=
f=3D"http://urn.nb.no/URN:NBN:no-38868">http://urn.nb.no/URN:NBN:no-38868</=
a></span></font></div>
<div><font face=3D"arial, helvetica, sans-serif"><span style=3D"font-size:1=
3px"><br></span></font></div><div><font face=3D"arial, helvetica, sans-seri=
f">Please don&#39;t=A0hesitate=A0to provide feedback.=A0</font></div><div><=
font face=3D"arial, helvetica, sans-serif"><br>
</font></div><div><font face=3D"arial, helvetica, sans-serif">Regards,</fon=
t></div><div><font face=3D"arial, helvetica, sans-serif">Naeem</font></div>=
<div><font face=3D"arial, helvetica, sans-serif"><br></font></div><div><br>=
</div>
</div>

--047d7b6786fe69f60f04ea781a96--

From shahid.akhtar@alcatel-lucent.com  Wed Nov  6 12:38:36 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C93021E8193 for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 12:38:36 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.698
X-Spam-Level: 
X-Spam-Status: No, score=-9.698 tagged_above=-999 required=5 tests=[AWL=0.901,  BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9FlYXNj-cHuM for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 12:38:30 -0800 (PST)
Received: from ihemail2.lucent.com (ihemail2.lucent.com [135.245.0.35]) by ietfa.amsl.com (Postfix) with ESMTP id 8902A11E812B for <aqm@ietf.org>; Wed,  6 Nov 2013 12:38:30 -0800 (PST)
Received: from us70tusmtp2.zam.alcatel-lucent.com (h135-5-2-64.lucent.com [135.5.2.64]) by ihemail2.lucent.com (8.13.8/IER-o) with ESMTP id rA6KcPYp001765 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Wed, 6 Nov 2013 14:38:26 -0600 (CST)
Received: from US70UWXCHHUB01.zam.alcatel-lucent.com (us70uwxchhub01.zam.alcatel-lucent.com [135.5.2.48]) by us70tusmtp2.zam.alcatel-lucent.com (GMO) with ESMTP id rA6KcPkf024872 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 6 Nov 2013 15:38:25 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70UWXCHHUB01.zam.alcatel-lucent.com ([135.5.2.48]) with mapi id 14.02.0247.003; Wed, 6 Nov 2013 15:38:25 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: "Scheffenegger, Richard" <rs@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: Ac7aemDyYVICOG/TQKCzUJnt0vYtGgAtZkow
Date: Wed, 6 Nov 2013 20:38:24 +0000
Message-ID: <C0611F522A6FA9498A044C36073E49657E5FED7D@US70UWXCHMBA01.zam.alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.5.27.18]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.35
Cc: "Fred Baker \(fred\) \(fred@cisco.com\)" <fred@cisco.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 20:38:36 -0000

Is a document on AQM evaluation and criteria available before the meeting?

Thanks.=20

-----Original Message-----
From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of Schef=
fenegger, Richard
Sent: Tuesday, November 05, 2013 5:02 PM
To: aqm@ietf.org
Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi (naeemk@ifi.uio.no)
Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B

Hi,

this is just a simple remainder that the WG will convene also this Friday;

As the mechanism update presentations took quite a bit longer than we had e=
xpected - and we appreciate the discussion these presentations have sparked=
 - we agreed with Fred to have his presentation as the first agenda point o=
n Friday; the second agenda point will be a presentation around AQM evaluat=
ion process and criteria.


The Room will be Regency B (3. Floor), and we will start at 12:30.


Best regards,

Richard Scheffenegger

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

From David.Ros@telecom-bretagne.eu  Wed Nov  6 13:49:36 2013
Return-Path: <David.Ros@telecom-bretagne.eu>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16D6B21E808A for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 13:49:36 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.136
X-Spam-Level: 
X-Spam-Status: No, score=-2.136 tagged_above=-999 required=5 tests=[AWL=-0.114, BAYES_00=-2.599, HELO_EQ_FR=0.35, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K26k6Q13c2ar for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 13:49:31 -0800 (PST)
Received: from zproxy220.enst-bretagne.fr (zproxy220.enst-bretagne.fr [192.108.117.9]) by ietfa.amsl.com (Postfix) with ESMTP id 9B4D721E811C for <aqm@ietf.org>; Wed,  6 Nov 2013 13:49:31 -0800 (PST)
Received: from localhost (localhost [127.0.0.1]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTP id 26C2250013; Wed,  6 Nov 2013 22:49:30 +0100 (CET)
X-Virus-Scanned: amavisd-new at zproxy220.enst-bretagne.fr
Received: from zproxy220.enst-bretagne.fr ([127.0.0.1]) by localhost (zproxy220.enst-bretagne.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hux9FZ9+b6LR; Wed,  6 Nov 2013 22:49:29 +0100 (CET)
Received: from dhcp-9869.meeting.ietf.org (dhcp-9869.meeting.ietf.org [31.133.152.105]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTPSA id 3B13F50012; Wed,  6 Nov 2013 22:49:26 +0100 (CET)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1816\))
From: David Ros <David.Ros@telecom-bretagne.eu>
In-Reply-To: <C0611F522A6FA9498A044C36073E49657E5FED7D@US70UWXCHMBA01.zam.alcatel-lucent.com>
Date: Wed, 6 Nov 2013 13:49:24 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <8A99C0E8-29CA-46C6-90CC-DFCC842C51E5@telecom-bretagne.eu>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FED7D@US70UWXCHMBA01.zam.alcatel-lucent.com>
To: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
X-Mailer: Apple Mail (2.1816)
Cc: "Scheffenegger, Richard" <rs@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>, "Fred Baker \(fred\) \(fred@cisco.com\)" <fred@cisco.com>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 21:49:36 -0000

On 6 nov. 2013, at 12:38, Akhtar, Shahid (Shahid) =
<shahid.akhtar@alcatel-lucent.com> wrote:

> Is a document on AQM evaluation and criteria available before the =
meeting?
>=20

Hi,

Not yet, sorry (couldn't make it in time before the deadline), but we're =
planning to submit it in the coming few weeks.

David


> Thanks.=20
>=20
> -----Original Message-----
> From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of =
Scheffenegger, Richard
> Sent: Tuesday, November 05, 2013 5:02 PM
> To: aqm@ietf.org
> Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi =
(naeemk@ifi.uio.no)
> Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
>=20
> Hi,
>=20
> this is just a simple remainder that the WG will convene also this =
Friday;
>=20
> As the mechanism update presentations took quite a bit longer than we =
had expected - and we appreciate the discussion these presentations have =
sparked - we agreed with Fred to have his presentation as the first =
agenda point on Friday; the second agenda point will be a presentation =
around AQM evaluation process and criteria.
>=20
>=20
> The Room will be Regency B (3. Floor), and we will start at 12:30.
>=20
>=20
> Best regards,
>=20
> Richard Scheffenegger
>=20
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
David ROS
http://www.rennes.enst-bretagne.fr/~dros/

We've heard that a million monkeys at a million keyboards could produce =
the Complete Works of Shakespeare; now, thanks to the Internet, we know =
this is not true. -- Robert Wilenski






From bob.briscoe@bt.com  Wed Nov  6 14:50:51 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 840AE21E811F; Wed,  6 Nov 2013 14:50:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.288
X-Spam-Level: 
X-Spam-Status: No, score=-3.288 tagged_above=-999 required=5 tests=[AWL=0.084,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g2ExicdKjX-g; Wed,  6 Nov 2013 14:50:46 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 47B1321E809E; Wed,  6 Nov 2013 14:50:28 -0800 (PST)
Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR68-UKRD.bt.com (10.187.101.23) with Microsoft SMTP Server (TLS) id 8.3.279.1; Wed, 6 Nov 2013 22:50:20 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Wed, 6 Nov 2013 22:50:20 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Wed, 6 Nov 2013 22:50:19 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.16.29])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA6MoGY0003371; Wed, 6 Nov 2013 22:50:17 GMT
Message-ID: <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Wed, 6 Nov 2013 22:50:14 +0000
To: Joe Touch <touch@isi.edu>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <52795A03.9010804@isi.edu>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu> <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk> <52795A03.9010804@isi.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: aqm@ietf.org, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 22:50:51 -0000

Joe,

The approach of a BCP just about ECN doesn't preclude any of the 
following options for stds track docs:
a) one RFC updating all tunnel specs about ECN
b) one RFC updating all tunnel specs about ECN and Diffserv (the two 
fields that propagate up as well as down).
c) one RFC updating all tunnel specs about everything
d) one RFC per each tunnel spec to wrap up all updates around at the time

These choices are for the relevant ADs and WGs to make.
Writing a BCP in TSV gives the raw info that any of these approaches can use.



Bob

At 20:50 05/11/2013, Joe Touch wrote:


>On 11/5/2013 10:59 AM, Bob Briscoe wrote:
>>Joe,
>>
>>I envisage that a very brief standards track doc that explicitly UPDATES
>>the relevant IETF tunnel specs will be written, and it will refer to
>>this doc for rationale.
>
>Tunnels need to handle ingress/egress translation of all signals in 
>the header. This is no different.
>
>My concern is that putting these recommendations in separate places 
>gives an opportunity for different groups to have different 
>interpretations of that sort of translation, and that's a bad thing IMO.
>
>Joe
>
>
>>
>>See Appendix A (outstanding items), which I have also highlighted when
>>presenting each time:
>>
>>     2.  Consider whether an IETF Standard Track doc will be needed to
>>         Update the IP-in-IP protocols listed in Section 4.1--at least
>>         those that the IETF controls--and which Area it should sit under.
>>
>>Does that address your concern?
>>
>>
>>Bob
>>
>>At 18:40 05/11/2013, Joe Touch wrote:
>>>IMO, these guidelines ought to come out in a single recommendation for
>>>tunnels; we had a draft of that in INTAREA but insufficient momentum.
>>>
>>>Piecemeal recommendations are likely to be ignored/lost.
>>>
>>>Joe
>>>
>>>On 11/4/2013 11:36 PM, Michael Menth wrote:
>>>>+1
>>>>
>>>>We need such guidelines for consistent congestion management.
>>>>
>>>>Best wishes,
>>>>
>>>>Michael
>>>>
>>>>Am 05.11.2013 00:16, schrieb Matt Mathis:
>>>>>I think this is valuable work.  Having a single document that
>>>>>describes the requirements and general principles will save future
>>>>>tunnel inventor/implementers from rediscovering the same bugs
>>>>>
>>>>>Thanks,
>>>>>--MM--
>>>>>The best way to predict the future is to create it.  - Alan Kay
>>>>>
>>>>>Privacy matters!  We know from recent events that people are using our
>>>>>services to speak in defiance of unjust governments.   We treat
>>>>>privacy and security as matters of life and death, because for some
>>>>>users, they are.
>>>>>
>>>>>
>>>>>On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>>>>><mailto:bob.briscoe@bt.com>> wrote:
>>>>>
>>>>>     Folks,
>>>>>
>>>>>     Pls respond if you support this being adopted as a work-group item
>>>>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>>>>     visibility of interest.
>>>>>     Even better, if you're willing to read / comment / review /
>>>>>implement
>>>>>
>>>>>     Guidelines for Adding Congestion Notification to Protocols that
>>>>>     Encapsulate IP
>>>>>
>>>>><http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>>>>
>>>>>     Abstract
>>>>>
>>>>>        The purpose of this document is to guide the design of
>>>>>congestion
>>>>>        notification in any lower layer or tunnelling protocol that
>>>>>        encapsulates IP.  The aim is for explicit congestion signals to
>>>>>        propagate consistently from lower layer protocols into IP.
>>>>>      Then the
>>>>>        IP internetwork layer can act as a portability layer to carry
>>>>>        congestion notification from non-IP-aware congested nodes up
>>>>>to the
>>>>>        transport layer (L4).  Following these guidelines should assure
>>>>>        interworking between new lower layer congestion notification
>>>>>        mechanisms, whether specified by the IETF or other standards
>>>>>     bodies.
>>>>>
>>>>>
>>>>>     [Cross-posting tsvwg & aqm, just in case]
>>>>>
>>>>>
>>>>>     Bob Briscoe,
>>>>>     also for co-authors Pat Thaler and John Kaippallimalil
>>>>>
>>>>>
>>>>>     ________________________________________________________________
>>>>>     Bob Briscoe,    BT
>>>>>     _______________________________________________
>>>>>     aqm mailing list
>>>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>>>     https://www.ietf.org/mailman/listinfo/aqm
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>aqm mailing list
>>>>>aqm@ietf.org
>>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>>
>>>>--
>>>>Prof. Dr. habil. Michael Menth
>>>>University of Tuebingen
>>>>Faculty of Science
>>>>Department of Computer Science
>>>>Chair of Communication Networks
>>>>Sand 13, 72076 Tuebingen, Germany
>>>>phone: (+49)-7071/29-70505
>>>>fax: (+49)-7071/29-5220
>>>>mailto:menth@uni-tuebingen.de
>>>>http://kn.inf.uni-tuebingen.de
>>>>
>>>>
>>>>
>>>>_______________________________________________
>>>>aqm mailing list
>>>>aqm@ietf.org
>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>_______________________________________________
>>>aqm mailing list
>>>aqm@ietf.org
>>>https://www.ietf.org/mailman/listinfo/aqm
>>
>>________________________________________________________________
>>Bob Briscoe,                                                  BT
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 


From touch@isi.edu  Wed Nov  6 15:00:51 2013
Return-Path: <touch@isi.edu>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5BAE911E8156; Wed,  6 Nov 2013 15:00:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.157
X-Spam-Level: 
X-Spam-Status: No, score=-103.157 tagged_above=-999 required=5 tests=[AWL=-0.785, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L6bzKPZG1XIX; Wed,  6 Nov 2013 15:00:33 -0800 (PST)
Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id 32F2411E8190; Wed,  6 Nov 2013 15:00:05 -0800 (PST)
Received: from [128.9.160.166] (abc.isi.edu [128.9.160.166]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id rA6MxdN0027097 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 6 Nov 2013 14:59:39 -0800 (PST)
Message-ID: <527ACA56.8010902@isi.edu>
Date: Wed, 06 Nov 2013 15:01:42 -0800
From: Joe Touch <touch@isi.edu>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0
MIME-Version: 1.0
To: Bob Briscoe <bob.briscoe@bt.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu> <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk> <52795A03.9010804@isi.edu> <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
In-Reply-To: <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Cc: aqm@ietf.org, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 23:00:51 -0000

Hi, Bob,

IMO, the desciption of how IP header info (which is what I presume 
you're suggesting) to impact L2s (which include tunnels) belongs in a 
tunnel doc, which AFAICT belongs in INTAREA. The tunnel landscape is 
already quite complex.

I have no idea why TSV would be involved in this except if you're 
talking about TCP ECH markings, but IMO the relationship of the TCP mark 
to the IP mark might belong in TSV.

However, I do not think TSV should be promoting having signals "jump" 
layers, i.e., from TCP ECN to its impact on tunnels or L2.

Joe

On 11/6/2013 2:50 PM, Bob Briscoe wrote:
> Joe,
>
> The approach of a BCP just about ECN doesn't preclude any of the
> following options for stds track docs:
> a) one RFC updating all tunnel specs about ECN
> b) one RFC updating all tunnel specs about ECN and Diffserv (the two
> fields that propagate up as well as down).
> c) one RFC updating all tunnel specs about everything
> d) one RFC per each tunnel spec to wrap up all updates around at the time
>
> These choices are for the relevant ADs and WGs to make.
> Writing a BCP in TSV gives the raw info that any of these approaches can
> use.
>
>
>
> Bob
>
> At 20:50 05/11/2013, Joe Touch wrote:
>
>
>> On 11/5/2013 10:59 AM, Bob Briscoe wrote:
>>> Joe,
>>>
>>> I envisage that a very brief standards track doc that explicitly UPDATES
>>> the relevant IETF tunnel specs will be written, and it will refer to
>>> this doc for rationale.
>>
>> Tunnels need to handle ingress/egress translation of all signals in
>> the header. This is no different.
>>
>> My concern is that putting these recommendations in separate places
>> gives an opportunity for different groups to have different
>> interpretations of that sort of translation, and that's a bad thing IMO.
>>
>> Joe
>>
>>
>>>
>>> See Appendix A (outstanding items), which I have also highlighted when
>>> presenting each time:
>>>
>>>     2.  Consider whether an IETF Standard Track doc will be needed to
>>>         Update the IP-in-IP protocols listed in Section 4.1--at least
>>>         those that the IETF controls--and which Area it should sit
>>> under.
>>>
>>> Does that address your concern?
>>>
>>>
>>> Bob
>>>
>>> At 18:40 05/11/2013, Joe Touch wrote:
>>>> IMO, these guidelines ought to come out in a single recommendation for
>>>> tunnels; we had a draft of that in INTAREA but insufficient momentum.
>>>>
>>>> Piecemeal recommendations are likely to be ignored/lost.
>>>>
>>>> Joe
>>>>
>>>> On 11/4/2013 11:36 PM, Michael Menth wrote:
>>>>> +1
>>>>>
>>>>> We need such guidelines for consistent congestion management.
>>>>>
>>>>> Best wishes,
>>>>>
>>>>> Michael
>>>>>
>>>>> Am 05.11.2013 00:16, schrieb Matt Mathis:
>>>>>> I think this is valuable work.  Having a single document that
>>>>>> describes the requirements and general principles will save future
>>>>>> tunnel inventor/implementers from rediscovering the same bugs
>>>>>>
>>>>>> Thanks,
>>>>>> --MM--
>>>>>> The best way to predict the future is to create it.  - Alan Kay
>>>>>>
>>>>>> Privacy matters!  We know from recent events that people are using
>>>>>> our
>>>>>> services to speak in defiance of unjust governments.   We treat
>>>>>> privacy and security as matters of life and death, because for some
>>>>>> users, they are.
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>>>>>> <mailto:bob.briscoe@bt.com>> wrote:
>>>>>>
>>>>>>     Folks,
>>>>>>
>>>>>>     Pls respond if you support this being adopted as a work-group
>>>>>> item
>>>>>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>>>>>     visibility of interest.
>>>>>>     Even better, if you're willing to read / comment / review /
>>>>>> implement
>>>>>>
>>>>>>     Guidelines for Adding Congestion Notification to Protocols that
>>>>>>     Encapsulate IP
>>>>>>
>>>>>> <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>>>>>
>>>>>>     Abstract
>>>>>>
>>>>>>        The purpose of this document is to guide the design of
>>>>>> congestion
>>>>>>        notification in any lower layer or tunnelling protocol that
>>>>>>        encapsulates IP.  The aim is for explicit congestion
>>>>>> signals to
>>>>>>        propagate consistently from lower layer protocols into IP.
>>>>>>      Then the
>>>>>>        IP internetwork layer can act as a portability layer to carry
>>>>>>        congestion notification from non-IP-aware congested nodes up
>>>>>> to the
>>>>>>        transport layer (L4).  Following these guidelines should
>>>>>> assure
>>>>>>        interworking between new lower layer congestion notification
>>>>>>        mechanisms, whether specified by the IETF or other standards
>>>>>>     bodies.
>>>>>>
>>>>>>
>>>>>>     [Cross-posting tsvwg & aqm, just in case]
>>>>>>
>>>>>>
>>>>>>     Bob Briscoe,
>>>>>>     also for co-authors Pat Thaler and John Kaippallimalil
>>>>>>
>>>>>>
>>>>>>     ________________________________________________________________
>>>>>>     Bob Briscoe,    BT
>>>>>>     _______________________________________________
>>>>>>     aqm mailing list
>>>>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>>>>     https://www.ietf.org/mailman/listinfo/aqm
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> aqm mailing list
>>>>>> aqm@ietf.org
>>>>>> https://www.ietf.org/mailman/listinfo/aqm
>>>>>
>>>>> --
>>>>> Prof. Dr. habil. Michael Menth
>>>>> University of Tuebingen
>>>>> Faculty of Science
>>>>> Department of Computer Science
>>>>> Chair of Communication Networks
>>>>> Sand 13, 72076 Tuebingen, Germany
>>>>> phone: (+49)-7071/29-70505
>>>>> fax: (+49)-7071/29-5220
>>>>> mailto:menth@uni-tuebingen.de
>>>>> http://kn.inf.uni-tuebingen.de
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> aqm mailing list
>>>>> aqm@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo/aqm
>>>> _______________________________________________
>>>> aqm mailing list
>>>> aqm@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/aqm
>>>
>>> ________________________________________________________________
>>> Bob Briscoe,                                                  BT
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT

From gorry@erg.abdn.ac.uk  Wed Nov  6 15:01:05 2013
Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3326011E8205 for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 15:01:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.352
X-Spam-Level: 
X-Spam-Status: No, score=-106.352 tagged_above=-999 required=5 tests=[AWL=0.020, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZZvulPJqs5U9 for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 15:01:00 -0800 (PST)
Received: from spey.erg.abdn.ac.uk (spey.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 88F7C11E8156 for <aqm@ietf.org>; Wed,  6 Nov 2013 15:00:52 -0800 (PST)
Received: from www.erg.abdn.ac.uk (blake.erg.abdn.ac.uk [139.133.210.30]) by spey.erg.abdn.ac.uk (Postfix) with ESMTPSA id 64B892B43B3; Wed,  6 Nov 2013 23:00:51 +0000 (GMT)
Received: from 31.133.152.246 (SquirrelMail authenticated user gorry) by www.erg.abdn.ac.uk with HTTP; Wed, 6 Nov 2013 23:00:51 -0000
Message-ID: <46f2001e920e849b0051562b5fcf12fa.squirrel@www.erg.abdn.ac.uk>
In-Reply-To: <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu> <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk> <52795A03.9010804@isi.edu> <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
Date: Wed, 6 Nov 2013 23:00:51 -0000
From: gorry@erg.abdn.ac.uk
To: "Bob Briscoe" <bob.briscoe@bt.com>
User-Agent: SquirrelMail/1.4.22
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
Cc: aqm@ietf.org, Joe Touch <touch@isi.edu>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 23:01:05 -0000

This message flow about ECN belongs in TSVWG - please do NOT cross post to
the AQM list.

If you are interested in the end-to-end usage of ECN or rules for tunnels,
etc then please do join the TSVWG list.

Gorry

---

> Joe,
>
> The approach of a BCP just about ECN doesn't preclude any of the
> following options for stds track docs:
> a) one RFC updating all tunnel specs about ECN
> b) one RFC updating all tunnel specs about ECN and Diffserv (the two
> fields that propagate up as well as down).
> c) one RFC updating all tunnel specs about everything
> d) one RFC per each tunnel spec to wrap up all updates around at the time
>
> These choices are for the relevant ADs and WGs to make.
> Writing a BCP in TSV gives the raw info that any of these approaches can
> use.
>
>
>
> Bob
>
> At 20:50 05/11/2013, Joe Touch wrote:
>
>
>>On 11/5/2013 10:59 AM, Bob Briscoe wrote:
>>>Joe,
>>>
>>>I envisage that a very brief standards track doc that explicitly UPDATES
>>>the relevant IETF tunnel specs will be written, and it will refer to
>>>this doc for rationale.
>>
>>Tunnels need to handle ingress/egress translation of all signals in
>>the header. This is no different.
>>
>>My concern is that putting these recommendations in separate places
>>gives an opportunity for different groups to have different
>>interpretations of that sort of translation, and that's a bad thing IMO.
>>
>>Joe
>>
>>
>>>
>>>See Appendix A (outstanding items), which I have also highlighted when
>>>presenting each time:
>>>
>>>     2.  Consider whether an IETF Standard Track doc will be needed to
>>>         Update the IP-in-IP protocols listed in Section 4.1--at least
>>>         those that the IETF controls--and which Area it should sit
>>> under.
>>>
>>>Does that address your concern?
>>>
>>>
>>>Bob
>>>
>>>At 18:40 05/11/2013, Joe Touch wrote:
>>>>IMO, these guidelines ought to come out in a single recommendation for
>>>>tunnels; we had a draft of that in INTAREA but insufficient momentum.
>>>>
>>>>Piecemeal recommendations are likely to be ignored/lost.
>>>>
>>>>Joe
>>>>
>>>>On 11/4/2013 11:36 PM, Michael Menth wrote:
>>>>>+1
>>>>>
>>>>>We need such guidelines for consistent congestion management.
>>>>>
>>>>>Best wishes,
>>>>>
>>>>>Michael
>>>>>
>>>>>Am 05.11.2013 00:16, schrieb Matt Mathis:
>>>>>>I think this is valuable work.  Having a single document that
>>>>>>describes the requirements and general principles will save future
>>>>>>tunnel inventor/implementers from rediscovering the same bugs
>>>>>>
>>>>>>Thanks,
>>>>>>--MM--
>>>>>>The best way to predict the future is to create it.  - Alan Kay
>>>>>>
>>>>>>Privacy matters!  We know from recent events that people are using
>>>>>> our
>>>>>>services to speak in defiance of unjust governments.   We treat
>>>>>>privacy and security as matters of life and death, because for some
>>>>>>users, they are.
>>>>>>
>>>>>>
>>>>>>On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>>>>>><mailto:bob.briscoe@bt.com>> wrote:
>>>>>>
>>>>>>     Folks,
>>>>>>
>>>>>>     Pls respond if you support this being adopted as a work-group
>>>>>> item
>>>>>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>>>>>     visibility of interest.
>>>>>>     Even better, if you're willing to read / comment / review /
>>>>>>implement
>>>>>>
>>>>>>     Guidelines for Adding Congestion Notification to Protocols that
>>>>>>     Encapsulate IP
>>>>>>
>>>>>><http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>>>>>
>>>>>>     Abstract
>>>>>>
>>>>>>        The purpose of this document is to guide the design of
>>>>>>congestion
>>>>>>        notification in any lower layer or tunnelling protocol that
>>>>>>        encapsulates IP.  The aim is for explicit congestion signals
>>>>>> to
>>>>>>        propagate consistently from lower layer protocols into IP.
>>>>>>      Then the
>>>>>>        IP internetwork layer can act as a portability layer to carry
>>>>>>        congestion notification from non-IP-aware congested nodes up
>>>>>>to the
>>>>>>        transport layer (L4).  Following these guidelines should
>>>>>> assure
>>>>>>        interworking between new lower layer congestion notification
>>>>>>        mechanisms, whether specified by the IETF or other standards
>>>>>>     bodies.
>>>>>>
>>>>>>
>>>>>>     [Cross-posting tsvwg & aqm, just in case]
>>>>>>
>>>>>>
>>>>>>     Bob Briscoe,
>>>>>>     also for co-authors Pat Thaler and John Kaippallimalil
>>>>>>
>>>>>>
>>>>>>     ________________________________________________________________
>>>>>>     Bob Briscoe,    BT
>>>>>>     _______________________________________________
>>>>>>     aqm mailing list
>>>>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>>>>     https://www.ietf.org/mailman/listinfo/aqm
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>_______________________________________________
>>>>>>aqm mailing list
>>>>>>aqm@ietf.org
>>>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>>>
>>>>>--
>>>>>Prof. Dr. habil. Michael Menth
>>>>>University of Tuebingen
>>>>>Faculty of Science
>>>>>Department of Computer Science
>>>>>Chair of Communication Networks
>>>>>Sand 13, 72076 Tuebingen, Germany
>>>>>phone: (+49)-7071/29-70505
>>>>>fax: (+49)-7071/29-5220
>>>>>mailto:menth@uni-tuebingen.de
>>>>>http://kn.inf.uni-tuebingen.de
>>>>>
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>aqm mailing list
>>>>>aqm@ietf.org
>>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>>_______________________________________________
>>>>aqm mailing list
>>>>aqm@ietf.org
>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>
>>>________________________________________________________________
>>>Bob Briscoe,                                                  BT
>>_______________________________________________
>>aqm mailing list
>>aqm@ietf.org
>>https://www.ietf.org/mailman/listinfo/aqm
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>


From gorry@erg.abdn.ac.uk  Wed Nov  6 15:01:06 2013
Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2439811E8205 for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 15:01:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.352
X-Spam-Level: 
X-Spam-Status: No, score=-106.352 tagged_above=-999 required=5 tests=[AWL=0.020, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OHn3X9WDkP0u for <aqm@ietfa.amsl.com>; Wed,  6 Nov 2013 15:01:05 -0800 (PST)
Received: from spey.erg.abdn.ac.uk (spey.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 4174B11E81D4 for <aqm@ietf.org>; Wed,  6 Nov 2013 15:00:53 -0800 (PST)
Received: from www.erg.abdn.ac.uk (blake.erg.abdn.ac.uk [139.133.210.30]) by spey.erg.abdn.ac.uk (Postfix) with ESMTPSA id 72EB62B44A6; Wed,  6 Nov 2013 23:00:52 +0000 (GMT)
Received: from 31.133.152.246 (SquirrelMail authenticated user gorry) by www.erg.abdn.ac.uk with HTTP; Wed, 6 Nov 2013 23:00:52 -0000
Message-ID: <2f314effc3048bb7072e30e38d7ab63a.squirrel@www.erg.abdn.ac.uk>
In-Reply-To: <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <52789FF5.3030907@uni-tuebingen.de> <52793B87.4040102@isi.edu> <201311051859.rA5IxJvZ030310@bagheera.jungle.bt.co.uk> <52795A03.9010804@isi.edu> <201311062250.rA6MoGY0003371@bagheera.jungle.bt.co.uk>
Date: Wed, 6 Nov 2013 23:00:52 -0000
From: gorry@erg.abdn.ac.uk
To: "Bob Briscoe" <bob.briscoe@bt.com>
User-Agent: SquirrelMail/1.4.22
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
Cc: aqm@ietf.org, Joe Touch <touch@isi.edu>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2013 23:01:06 -0000

This message flow about ECN belongs in TSVWG - please do NOT cross post to
the AQM list.

If you are interested in the end-to-end usage of ECN or rules for tunnels,
etc then please do join the TSVWG list.

Gorry

---

> Joe,
>
> The approach of a BCP just about ECN doesn't preclude any of the
> following options for stds track docs:
> a) one RFC updating all tunnel specs about ECN
> b) one RFC updating all tunnel specs about ECN and Diffserv (the two
> fields that propagate up as well as down).
> c) one RFC updating all tunnel specs about everything
> d) one RFC per each tunnel spec to wrap up all updates around at the time
>
> These choices are for the relevant ADs and WGs to make.
> Writing a BCP in TSV gives the raw info that any of these approaches can
> use.
>
>
>
> Bob
>
> At 20:50 05/11/2013, Joe Touch wrote:
>
>
>>On 11/5/2013 10:59 AM, Bob Briscoe wrote:
>>>Joe,
>>>
>>>I envisage that a very brief standards track doc that explicitly UPDATES
>>>the relevant IETF tunnel specs will be written, and it will refer to
>>>this doc for rationale.
>>
>>Tunnels need to handle ingress/egress translation of all signals in
>>the header. This is no different.
>>
>>My concern is that putting these recommendations in separate places
>>gives an opportunity for different groups to have different
>>interpretations of that sort of translation, and that's a bad thing IMO.
>>
>>Joe
>>
>>
>>>
>>>See Appendix A (outstanding items), which I have also highlighted when
>>>presenting each time:
>>>
>>>     2.  Consider whether an IETF Standard Track doc will be needed to
>>>         Update the IP-in-IP protocols listed in Section 4.1--at least
>>>         those that the IETF controls--and which Area it should sit
>>> under.
>>>
>>>Does that address your concern?
>>>
>>>
>>>Bob
>>>
>>>At 18:40 05/11/2013, Joe Touch wrote:
>>>>IMO, these guidelines ought to come out in a single recommendation for
>>>>tunnels; we had a draft of that in INTAREA but insufficient momentum.
>>>>
>>>>Piecemeal recommendations are likely to be ignored/lost.
>>>>
>>>>Joe
>>>>
>>>>On 11/4/2013 11:36 PM, Michael Menth wrote:
>>>>>+1
>>>>>
>>>>>We need such guidelines for consistent congestion management.
>>>>>
>>>>>Best wishes,
>>>>>
>>>>>Michael
>>>>>
>>>>>Am 05.11.2013 00:16, schrieb Matt Mathis:
>>>>>>I think this is valuable work.  Having a single document that
>>>>>>describes the requirements and general principles will save future
>>>>>>tunnel inventor/implementers from rediscovering the same bugs
>>>>>>
>>>>>>Thanks,
>>>>>>--MM--
>>>>>>The best way to predict the future is to create it.  - Alan Kay
>>>>>>
>>>>>>Privacy matters!  We know from recent events that people are using
>>>>>> our
>>>>>>services to speak in defiance of unjust governments.   We treat
>>>>>>privacy and security as matters of life and death, because for some
>>>>>>users, they are.
>>>>>>
>>>>>>
>>>>>>On Mon, Nov 4, 2013 at 2:03 PM, Bob Briscoe <bob.briscoe@bt.com
>>>>>><mailto:bob.briscoe@bt.com>> wrote:
>>>>>>
>>>>>>     Folks,
>>>>>>
>>>>>>     Pls respond if you support this being adopted as a work-group
>>>>>> item
>>>>>>     in the IETF transport services w-g (tsvwg). The WG chairs need
>>>>>>     visibility of interest.
>>>>>>     Even better, if you're willing to read / comment / review /
>>>>>>implement
>>>>>>
>>>>>>     Guidelines for Adding Congestion Notification to Protocols that
>>>>>>     Encapsulate IP
>>>>>>
>>>>>><http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>>>>>>
>>>>>>     Abstract
>>>>>>
>>>>>>        The purpose of this document is to guide the design of
>>>>>>congestion
>>>>>>        notification in any lower layer or tunnelling protocol that
>>>>>>        encapsulates IP.  The aim is for explicit congestion signals
>>>>>> to
>>>>>>        propagate consistently from lower layer protocols into IP.
>>>>>>      Then the
>>>>>>        IP internetwork layer can act as a portability layer to carry
>>>>>>        congestion notification from non-IP-aware congested nodes up
>>>>>>to the
>>>>>>        transport layer (L4).  Following these guidelines should
>>>>>> assure
>>>>>>        interworking between new lower layer congestion notification
>>>>>>        mechanisms, whether specified by the IETF or other standards
>>>>>>     bodies.
>>>>>>
>>>>>>
>>>>>>     [Cross-posting tsvwg & aqm, just in case]
>>>>>>
>>>>>>
>>>>>>     Bob Briscoe,
>>>>>>     also for co-authors Pat Thaler and John Kaippallimalil
>>>>>>
>>>>>>
>>>>>>     ________________________________________________________________
>>>>>>     Bob Briscoe,    BT
>>>>>>     _______________________________________________
>>>>>>     aqm mailing list
>>>>>>     aqm@ietf.org <mailto:aqm@ietf.org>
>>>>>>     https://www.ietf.org/mailman/listinfo/aqm
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>_______________________________________________
>>>>>>aqm mailing list
>>>>>>aqm@ietf.org
>>>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>>>
>>>>>--
>>>>>Prof. Dr. habil. Michael Menth
>>>>>University of Tuebingen
>>>>>Faculty of Science
>>>>>Department of Computer Science
>>>>>Chair of Communication Networks
>>>>>Sand 13, 72076 Tuebingen, Germany
>>>>>phone: (+49)-7071/29-70505
>>>>>fax: (+49)-7071/29-5220
>>>>>mailto:menth@uni-tuebingen.de
>>>>>http://kn.inf.uni-tuebingen.de
>>>>>
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>aqm mailing list
>>>>>aqm@ietf.org
>>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>>_______________________________________________
>>>>aqm mailing list
>>>>aqm@ietf.org
>>>>https://www.ietf.org/mailman/listinfo/aqm
>>>
>>>________________________________________________________________
>>>Bob Briscoe,                                                  BT
>>_______________________________________________
>>aqm mailing list
>>aqm@ietf.org
>>https://www.ietf.org/mailman/listinfo/aqm
>
> ________________________________________________________________
> Bob Briscoe,                                                  BT
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>


From p.ohanlon@gmail.com  Thu Nov  7 05:26:49 2013
Return-Path: <p.ohanlon@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C183211E8259; Thu,  7 Nov 2013 05:26:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.461
X-Spam-Level: 
X-Spam-Status: No, score=-2.461 tagged_above=-999 required=5 tests=[AWL=-0.089, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LD36NPhhnXv1; Thu,  7 Nov 2013 05:26:49 -0800 (PST)
Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) by ietfa.amsl.com (Postfix) with ESMTP id F12FB11E8255; Thu,  7 Nov 2013 05:26:40 -0800 (PST)
Received: by mail-we0-f178.google.com with SMTP id q59so530960wes.9 for <multiple recipients>; Thu, 07 Nov 2013 05:26:40 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=dkvGq4ZGYFJ4ermv6u0gFf1qF4dUttQSRuppSTA5ufM=; b=iEH1Jqahpm2Tb2WzmFOyJoukuQsk+TVAMfggM0CZOPgQnWfd7gfiTvza4d9DJVf0m4 21qnxWa9dyLNqaScM+Q6yappPjoi/FSPy6kguon0VYdsKwtIfihgtzkc9ddRvsc1jrGR kul+hl/yifsaH/ovhMLDvBpHz+oX2rwwrZ/ffFlaj7ZYh7NN9iG8mGqIt0sB8rOKo53J Z4oi3bPgOw5Hlib04vCU2BwCtdJsEUKOg/jtQSjaDmiTL74hzIZjeAeqENDNavAx7bN1 tJUTd9PNRfiNpV7SA9pE9I2T3Pa1h9m7jOLtL7wOhG7tIw7/PQJy9ZM0RoODMldT15LB TYqA==
X-Received: by 10.194.1.162 with SMTP id 2mr48481wjn.91.1383830799927; Thu, 07 Nov 2013 05:26:39 -0800 (PST)
Received: from [149.241.130.95] ([149.241.130.95]) by mx.google.com with ESMTPSA id y11sm6881221wie.7.2013.11.07.05.26.37 for <multiple recipients> (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 07 Nov 2013 05:26:38 -0800 (PST)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Piers O'Hanlon <p.ohanlon@gmail.com>
In-Reply-To: <201311051841.rA5IfUeP030224@bagheera.jungle.bt.co.uk>
Date: Thu, 7 Nov 2013 13:26:36 +0000
Content-Transfer-Encoding: quoted-printable
Message-Id: <6DB095B2-E77B-457F-81D2-5E9CC940871B@gmail.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com> <201311051841.rA5IfUeP030224@bagheera.jungle.bt.co.uk>
To: Bob Briscoe <bob.briscoe@bt.com>
X-Mailer: Apple Mail (2.1283)
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 13:26:49 -0000

Bob,

I was thinking that it would be good to add the following =
clarifications:

Given a suitable marking scheme ECN provides for the removal of nearly =
all congestion loss and it reduces delays for two main reasons:=20
  i) It avoids the delay when recovering from congestion losses, which =
particularly benefits small flows, making their=20
      completion time predictably short [RFC2884]
  ii) As ECN is used more widely by end-systems, it will gradually =
remove the need to configure a degree of delay
      into buffers before they start to notify congestion (the cause of =
bufferbloat).=20
The latter delay is because a drop involves a trade-off between sending =
a timely signal and trying to avoid impairment, whereas ECN is solely a =
signal not an impairment, so there is no harm triggering it earlier.

Cheers,

Piers

On 5 Nov 2013, at 18:41, Bob Briscoe wrote:

> Piers,
>=20
> I tried to state exactly how ECN can benefit (2nd para of intro =
below), rather than make overblown claims. You seem to be saying I =
didn't succeed? I can't see how to do any better. Suggestions?
>=20
> =
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\=
/\/\/\/\/\/\/\
> ECN removes nearly all congestion loss and it cuts delays for two main =
reasons: i) it avoids the delay when recovering from congestion losses, =
which particularly benefits small flows, making their completion time =
predictably short [RFC2884] and ii) as ECN is used more widely by =
end-systems, it will gradually remove the need to configure a degree of =
delay into buffers before they start to notify congestion (the cause of =
bufferbloat). The latter delay is because drop involves a trade-off =
between sending a timely signal and trying to avoid impairment, whereas =
ECN is solely a signal not an impairment, so there is no harm triggering =
it earlier.
> =
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\=
/\/\/\/\/\/\/\
>=20
> PS. Thx for supporting words.
>=20
> Bob
>=20
> At 09:42 05/11/2013, Piers O'Hanlon wrote:
>> Hi Bob,
>>=20
>> I took a brief look at the draft and it's clearly useful work.
>>=20
>> One thing that could do with clarification in the Introduction is =
that ECN - by itself - doesn't necessarily lead to low loss and delay - =
it should be made clear that it reflects the marking approach of the =
underlying scheme/AQM.
>>=20
>> Piers
>>=20
>> On 4 Nov 2013, at 22:03, Bob Briscoe wrote:
>>=20
>> > Folks,
>> >
>> > Pls respond if you support this being adopted as a work-group item =
in the IETF transport services w-g (tsvwg). The WG chairs need =
visibility of interest.
>> > Even better, if you're willing to read / comment / review / =
implement
>> >
>> > Guidelines for Adding Congestion Notification to Protocols that =
Encapsulate IP
>> > =
<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
>> >
>> > Abstract
>> >
>> >   The purpose of this document is to guide the design of congestion
>> >   notification in any lower layer or tunnelling protocol that
>> >   encapsulates IP.  The aim is for explicit congestion signals to
>> >   propagate consistently from lower layer protocols into IP.  Then =
the
>> >   IP internetwork layer can act as a portability layer to carry
>> >   congestion notification from non-IP-aware congested nodes up to =
the
>> >   transport layer (L4).  Following these guidelines should assure
>> >   interworking between new lower layer congestion notification
>> >   mechanisms, whether specified by the IETF or other standards =
bodies.
>> >
>> >
>> > [Cross-posting tsvwg & aqm, just in case]
>> >
>> >
>> > Bob Briscoe,
>> > also for co-authors Pat Thaler and John Kaippallimalil
>> >
>> >
>> > ________________________________________________________________
>> > Bob Briscoe,                                                  BT
>>=20
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>=20
> ________________________________________________________________
> Bob Briscoe,                                                  BT=20


From shahid.akhtar@alcatel-lucent.com  Thu Nov  7 09:00:09 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 48B5C11E81E4 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 09:00:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.149
X-Spam-Level: 
X-Spam-Status: No, score=-10.149 tagged_above=-999 required=5 tests=[AWL=0.450, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nfkap6Yxcy6Z for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 09:00:03 -0800 (PST)
Received: from ihemail3.lucent.com (ihemail3.lucent.com [135.245.0.37]) by ietfa.amsl.com (Postfix) with ESMTP id 894AA11E819E for <aqm@ietf.org>; Thu,  7 Nov 2013 09:00:02 -0800 (PST)
Received: from us70tusmtp2.zam.alcatel-lucent.com (h135-5-2-64.lucent.com [135.5.2.64]) by ihemail3.lucent.com (8.13.8/IER-o) with ESMTP id rA7Gxsf0016608 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Thu, 7 Nov 2013 10:59:55 -0600 (CST)
Received: from US70UWXCHHUB02.zam.alcatel-lucent.com (us70uwxchhub02.zam.alcatel-lucent.com [135.5.2.49]) by us70tusmtp2.zam.alcatel-lucent.com (GMO) with ESMTP id rA7Gxr5a003043 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Nov 2013 11:59:53 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70UWXCHHUB02.zam.alcatel-lucent.com ([135.5.2.49]) with mapi id 14.02.0247.003; Thu, 7 Nov 2013 11:59:53 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: "Scheffenegger, Richard" <rs@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: Ac7aemDyYVICOG/TQKCzUJnt0vYtGgAxY8ew
Date: Thu, 7 Nov 2013 16:59:52 +0000
Message-ID: <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [135.5.27.18]
Content-Type: multipart/mixed; boundary="_002_C0611F522A6FA9498A044C36073E49657E5FF524US70UWXCHMBA01z_"
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.37
Cc: "Fred Baker \(fred\) \(fred@cisco.com\)" <fred@cisco.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 17:00:10 -0000

--_002_C0611F522A6FA9498A044C36073E49657E5FF524US70UWXCHMBA01z_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi All,

Had some comments on Fred's document. I have added the comments as track ch=
anges in a word document to easily see them. I used the 02 version.=20

Thanks.=20

-----Original Message-----
From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of Schef=
fenegger, Richard
Sent: Tuesday, November 05, 2013 5:02 PM
To: aqm@ietf.org
Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi (naeemk@ifi.uio.no)
Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B

Hi,

this is just a simple remainder that the WG will convene also this Friday;

As the mechanism update presentations took quite a bit longer than we had e=
xpected - and we appreciate the discussion these presentations have sparked=
 - we agreed with Fred to have his presentation as the first agenda point o=
n Friday; the second agenda point will be a presentation around AQM evaluat=
ion process and criteria.


The Room will be Regency B (3. Floor), and we will start at 12:30.


Best regards,

Richard Scheffenegger

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

--_002_C0611F522A6FA9498A044C36073E49657E5FF524US70UWXCHMBA01z_
Content-Type: application/msword;
	name="draft-baker-aqm-recommendation-02.doc"
Content-Description: draft-baker-aqm-recommendation-02.doc
Content-Disposition: attachment;
	filename="draft-baker-aqm-recommendation-02.doc"; size=176640;
	creation-date="Wed, 06 Nov 2013 20:49:29 GMT";
	modification-date="Thu, 07 Nov 2013 16:58:54 GMT"
Content-Transfer-Encoding: base64

0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAADAAAAVAEAAAAAAAAA
EAAAVgEAAAEAAAD+////AAAAAFEBAABSAQAAUwEAAP//////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////s
pcEAKWAJBAAA8BK/AAAAAAAAEAAAAAAABgAA7tIAAA4AYmpiavPq8+oAAAAAAAAAAAAAAAAAAAAA
AAAJBBYANDoBAJGAAACRgAAA7soAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD//w8AAAAA
AAAAAAD//w8AAAAAAAAAAAD//w8AAAAAAAAAAAAAAAAAAAAAAKQAAAAAAEQEAAAAAAAARAQAAEQE
AAAAAAAARAQAAAAAAABEBAAAAAAAAEQEAAAAAAAARAQAABQAAAAAAAAAAAAAAFgEAAAAAAAAQAYB
AAAAAABABgEAAAAAAEAGAQAAAAAAQAYBACwAAABsBgEAbAEAAFgEAAAAAAAAYDABAAoCAADkBwEA
AAAAAOQHAQAAAAAA5AcBAAAAAADkBwEAAAAAAOQHAQAAAAAAvwgBAAAAAAC/CAEAAAAAAL8IAQAA
AAAAwy8BAAIAAADFLwEAAAAAAMUvAQAAAAAAxS8BAAAAAADFLwEAAAAAAMUvAQAAAAAAxS8BACQA
AABqMgEAaAIAANI0AQC+AAAA6S8BABUAAAAAAAAAAAAAAAAAAAAAAAAARAQAAAAAAACEDAEAAAAA
AAAAAAAAAAAAAAAAAAAAAAC/CAEAAAAAAL8IAQAAAAAAhAwBAAAAAACEDAEAAAAAAOkvAQAAAAAA
AAAAAAAAAABEBAAAAAAAAEQEAAAAAAAA5AcBAAAAAAAAAAAAAAAAAOQHAQDbAAAA/i8BACYAAADM
KwEAAAAAAMwrAQAAAAAAzCsBAAAAAACEDAEAlgcAAEQEAAAAAAAA5AcBAAAAAABEBAAAAAAAAOQH
AQAAAAAAwy8BAAAAAAAAAAAAAAAAAMwrAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAhAwBAAAAAADDLwEAAAAAAAAAAAAAAAAAzCsBAAAAAADMKwEA
HgAAAKsuAQAYAAAARAQAAAAAAABEBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA9y4BAAAAAADkBwEAAAAAANgHAQAMAAAAoKRspNrb
zgEAAAAAAAAAAEAGAQAAAAAAGhQBADIXAADDLgEACAAAAAAAAAAAAAAAwy8BAAAAAAAkMAEAPAAA
AGAwAQAAAAAAyy4BACwAAACQNQEAAAAAAEwrAQAiAAAAkDUBABAAAAD3LgEAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAJA1AQAAAAAAAAAAAAAAAABEBAAAAAAAAPcuAQDMAAAAvwgBAAIBAADBCQEAuAAAAMwr
AQAAAAAAeQoBAJQAAAANCwEAdwEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAvwgB
AAAAAAC/CAEAAAAAAL8IAQAAAAAA6S8BAAAAAADpLwEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAbisBAF4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAL8IAQAA
AAAAvwgBAAAAAAC/CAEAAAAAAGAwAQAAAAAAhAwBAAAAAACEDAEAAAAAAIQMAQAAAAAAhAwBAAAA
AAAAAAAAAAAAAFgEAAAAAAAAWAQAAAAAAABYBAAARI4AAJySAACkcwAAWAQAAAAAAABYBAAAAAAA
AFgEAAAAAAAAnJIAAAAAAABYBAAAAAAAAFgEAAAAAAAAWAQAAAAAAABEBAAAAAAAAEQEAAAAAAAA
RAQAAAAAAABEBAAAAAAAAEQEAAAAAAAARAQAAAAAAAD/////AAAAAAIADAEAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0NDQ1O
ZXR3b3JrIFdvcmtpbmcgR3JvdXAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IEYuIEJha2VyLCBFZC4NSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBDaXNjbyBTeXN0ZW1zDU9ic29sZXRlczogMjMwOSAoaWYgYXBwcm92
ZWQpICAgICAgICAgICAgICAgICAgICAgICAgICBHLiBGYWlyaHVyc3QsIEVkLg1JbnRlbmRlZCBz
dGF0dXM6IEJlc3QgQ3VycmVudCBQcmFjdGljZSAgICAgICAgICAgIFVuaXZlcnNpdHkgb2YgQWJl
cmRlZW4NRXhwaXJlczogSmFudWFyeSAxMCwgMjAxNCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBKdWx5IDA5LCAyMDEzDQ0NICAgICAgICAgSUVURiBSZWNvbW1lbmRhdGlvbnMgUmVn
YXJkaW5nIEFjdGl2ZSBRdWV1ZSBNYW5hZ2VtZW50DSAgICAgICAgICAgICAgICAgICBkcmFmdC1i
YWtlci1hcW0tcmVjb21tZW5kYXRpb24tMDINDUFic3RyYWN0DQ0gICBUaGlzIG1lbW8gcHJlc2Vu
dHMgcmVjb21tZW5kYXRpb25zIHRvIHRoZSBJbnRlcm5ldCBjb21tdW5pdHkNICAgY29uY2Vybmlu
ZyBtZWFzdXJlcyB0byBpbXByb3ZlIGFuZCBwcmVzZXJ2ZSBJbnRlcm5ldCBwZXJmb3JtYW5jZS4g
IEl0DSAgIHByZXNlbnRzIGEgc3Ryb25nIHJlY29tbWVuZGF0aW9uIGZvciB0ZXN0aW5nLCBzdGFu
ZGFyZGl6YXRpb24sIGFuZA0gICB3aWRlc3ByZWFkIGRlcGxveW1lbnQgb2YgYWN0aXZlIHF1ZXVl
IG1hbmFnZW1lbnQgKEFRTSkgaW4gbmV0d29yaw0gICBkZXZpY2VzLCB0byBpbXByb3ZlIHRoZSBw
ZXJmb3JtYW5jZSBvZiB0b2RheSdzIEludGVybmV0LiAgSXQgYWxzbw0gICB1cmdlcyBhIGNvbmNl
cnRlZCBlZmZvcnQgb2YgcmVzZWFyY2gsIG1lYXN1cmVtZW50LCBhbmQgdWx0aW1hdGUNICAgZGVw
bG95bWVudCBvZiBBUU0gbWVjaGFuaXNtcyB0byBwcm90ZWN0IHRoZSBJbnRlcm5ldCBmcm9tIGZs
b3dzIHRoYXQNICAgYXJlIG5vdCBzdWZmaWNpZW50bHkgcmVzcG9uc2l2ZSB0byBjb25nZXN0aW9u
IG5vdGlmaWNhdGlvbi4NDSAgIFRoZSBub3RlIGxhcmdlbHkgcmVwZWF0cyB0aGUgcmVjb21tZW5k
YXRpb25zIG9mIFJGQyAyMzA5LCB1cGRhdGVkDSAgIGFmdGVyIGZpZnRlZW4geWVhcnMgb2YgZXhw
ZXJpZW5jZSBhbmQgbmV3IHJlc2VhcmNoLg0NU3RhdHVzIG9mIFRoaXMgTWVtbw0NICAgVGhpcyBJ
bnRlcm5ldC1EcmFmdCBpcyBzdWJtaXR0ZWQgaW4gZnVsbCBjb25mb3JtYW5jZSB3aXRoIHRoZQ0g
ICBwcm92aXNpb25zIG9mIEJDUCA3OCBhbmQgQkNQIDc5Lg0NICAgSW50ZXJuZXQtRHJhZnRzIGFy
ZSB3b3JraW5nIGRvY3VtZW50cyBvZiB0aGUgSW50ZXJuZXQgRW5naW5lZXJpbmcNICAgVGFzayBG
b3JjZSAoSUVURikuICBOb3RlIHRoYXQgb3RoZXIgZ3JvdXBzIG1heSBhbHNvIGRpc3RyaWJ1dGUN
ICAgd29ya2luZyBkb2N1bWVudHMgYXMgSW50ZXJuZXQtRHJhZnRzLiAgVGhlIGxpc3Qgb2YgY3Vy
cmVudCBJbnRlcm5ldC0NICAgRHJhZnRzIGlzIGF0IGh0dHA6Ly9kYXRhdHJhY2tlci5pZXRmLm9y
Zy9kcmFmdHMvY3VycmVudC8uDQ0gICBJbnRlcm5ldC1EcmFmdHMgYXJlIGRyYWZ0IGRvY3VtZW50
cyB2YWxpZCBmb3IgYSBtYXhpbXVtIG9mIHNpeCBtb250aHMNICAgYW5kIG1heSBiZSB1cGRhdGVk
LCByZXBsYWNlZCwgb3Igb2Jzb2xldGVkIGJ5IG90aGVyIGRvY3VtZW50cyBhdCBhbnkNICAgdGlt
ZS4gIEl0IGlzIGluYXBwcm9wcmlhdGUgdG8gdXNlIEludGVybmV0LURyYWZ0cyBhcyByZWZlcmVu
Y2UNICAgbWF0ZXJpYWwgb3IgdG8gY2l0ZSB0aGVtIG90aGVyIHRoYW4gYXMgIndvcmsgaW4gcHJv
Z3Jlc3MuIg0NICAgVGhpcyBJbnRlcm5ldC1EcmFmdCB3aWxsIGV4cGlyZSBvbiBKYW51YXJ5IDEw
LCAyMDE0Lg0NQ29weXJpZ2h0IE5vdGljZQ0NICAgQ29weXJpZ2h0IChjKSAyMDEzIElFVEYgVHJ1
c3QgYW5kIHRoZSBwZXJzb25zIGlkZW50aWZpZWQgYXMgdGhlDSAgIGRvY3VtZW50IGF1dGhvcnMu
ICBBbGwgcmlnaHRzIHJlc2VydmVkLg0NICAgVGhpcyBkb2N1bWVudCBpcyBzdWJqZWN0IHRvIEJD
UCA3OCBhbmQgdGhlIElFVEYgVHJ1c3QncyBMZWdhbA0gICBQcm92aXNpb25zIFJlbGF0aW5nIHRv
IElFVEYgRG9jdW1lbnRzDQ0NDUJha2VyICYgRmFpcmh1cnN0ICAgICAgIEV4cGlyZXMgSmFudWFy
eSAxMCwgMjAxNCAgICAgICAgICAgICAgICBbUGFnZSAxXQ0MDUludGVybmV0LURyYWZ0ICAgQWN0
aXZlIFF1ZXVlIE1hbmFnZW1lbnQgUmVjb21tZW5kYXRpb25zICAgICAgIEp1bHkgMjAxMw0NDSAg
IChodHRwOi8vdHJ1c3RlZS5pZXRmLm9yZy9saWNlbnNlLWluZm8pIGluIGVmZmVjdCBvbiB0aGUg
ZGF0ZSBvZg0gICBwdWJsaWNhdGlvbiBvZiB0aGlzIGRvY3VtZW50LiAgUGxlYXNlIHJldmlldyB0
aGVzZSBkb2N1bWVudHMNICAgY2FyZWZ1bGx5LCBhcyB0aGV5IGRlc2NyaWJlIHlvdXIgcmlnaHRz
IGFuZCByZXN0cmljdGlvbnMgd2l0aCByZXNwZWN0DSAgIHRvIHRoaXMgZG9jdW1lbnQuICBDb2Rl
IENvbXBvbmVudHMgZXh0cmFjdGVkIGZyb20gdGhpcyBkb2N1bWVudCBtdXN0DSAgIGluY2x1ZGUg
U2ltcGxpZmllZCBCU0QgTGljZW5zZSB0ZXh0IGFzIGRlc2NyaWJlZCBpbiBTZWN0aW9uIDQuZSBv
Zg0gICB0aGUgVHJ1c3QgTGVnYWwgUHJvdmlzaW9ucyBhbmQgYXJlIHByb3ZpZGVkIHdpdGhvdXQg
d2FycmFudHkgYXMNICAgZGVzY3JpYmVkIGluIHRoZSBTaW1wbGlmaWVkIEJTRCBMaWNlbnNlLg0N
VGFibGUgb2YgQ29udGVudHMNDSAgIDEuICBJbnRyb2R1Y3Rpb24gIC4gLiAuIC4gLiAuIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICAgMg0gICAgIDEuMS4gIFJlcXVpcmVtZW50
cyBMYW5ndWFnZSAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgIDQNICAgMi4g
IFRoZSBOZWVkIEZvciBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCAgLiAuIC4gLiAuIC4gLiAuIC4g
LiAuIC4gICA0DSAgIDMuICBNYW5hZ2luZyBBZ2dyZXNzaXZlIEZsb3dzIC4gLiAuIC4gLiAuIC4g
LiAuIC4gLiAuIC4gLiAuIC4gLiAuICAgOA0gICA0LiAgQ29uY2x1c2lvbnMgYW5kIFJlY29tbWVu
ZGF0aW9ucyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMTANICAgICA0LjEuICBPcGVy
YXRpb25hbCBkZXBsb3ltZW50cyBTSE9VTEQgIHVzZSBBUU0gcHJvY2VkdXJlcyAgLiAuIC4gIDEx
DSAgICAgNC4yLiAgU2lnbmFsaW5nIHRvIHRoZSB0cmFuc3BvcnQgZW5kcG9pbnRzICAuIC4gLiAu
IC4gLiAuIC4gLiAuICAxMQ0gICAgICAgNC4yLjEuICBBUU0gYW5kIEVDTiAuIC4gLiAuIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMTINICAgICA0LjMuICBBUU0gYWxnb3JpdGht
cyBkZXBsb3llZCBTSE9VTEQgTk9UIHJlcXVpcmUgb3BlcmF0aW9uYWwNICAgICAgICAgICB0dW5p
bmcgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDEz
DSAgICAgNC40LiAgQVFNIGFsZ29yaXRobXMgU0hPVUxEIHJlc3BvbmQgdG8gbWVhc3VyZWQgY29u
Z2VzdGlvbiwgbm90DSAgICAgICAgICAgYXBwbGljYXRpb24gcHJvZmlsZXMuIC4gLiAuIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICAxMw0gICAgIDQuNS4gIEFRTSBhbGdvcml0aG1zIFNI
T1VMRCBOT1QgYmUgZGVwZW5kZW50IG9uIHNwZWNpZmljDSAgICAgICAgICAgdHJhbnNwb3J0IHBy
b3RvY29sIGJlaGF2aW91cnMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICAxNA0gICAgIDQu
Ni4gIEludGVyYWN0aW9ucyB3aXRoIGNvbmdlc3Rpb24gY29udHJvbCBhbGdvcml0aG1zID8/Pz8g
IC4gLiAgMTQNICAgICA0LjcuICBUaGUgbmVlZCBmb3IgZnVydGhlciByZXNlYXJjaCAuIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuIC4gIDE1DSAgIDUuICBJQU5BIENvbnNpZGVyYXRpb25zIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICAxNg0gICA2LiAgU2VjdXJpdHkg
Q29uc2lkZXJhdGlvbnMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMTYN
ICAgNy4gIFByaXZhY3kgQ29uc2lkZXJhdGlvbnMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g
LiAuIC4gLiAuIC4gIDE2DSAgIDguICBBY2tub3dsZWRnZW1lbnRzICAuIC4gLiAuIC4gLiAuIC4g
LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICAxNg0gICA5LiAgUmVmZXJlbmNlcyAgLiAuIC4g
LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMTcNICAgICA5LjEu
ICBOb3JtYXRpdmUgUmVmZXJlbmNlcyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu
IC4gIDE3DSAgICAgOS4yLiAgSW5mb3JtYXRpdmUgUmVmZXJlbmNlcyAgLiAuIC4gLiAuIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuICAxNw0gICBBcHBlbmRpeCBBLiAgQ2hhbmdlIExvZyAuIC4gLiAu
IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMjANICAgQXV0aG9ycycgQWRkcmVz
c2VzICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDIwDQ0x
LiAgSW50cm9kdWN0aW9uDQ0NDQ0NDQ0NDQ0NDQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAgRXhw
aXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAgICAgICAgICAgIFtQYWdlIDJdDQwNSW50ZXJuZXQt
RHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAgSnVs
eSAyMDEzDQ0NICAgVGhlIEludGVybmV0IHByb3RvY29sIGFyY2hpdGVjdHVyZSBpcyBiYXNlZCBv
biBhIGNvbm5lY3Rpb25sZXNzIGVuZC0NICAgdG8tZW5kIHBhY2tldCBzZXJ2aWNlIHVzaW5nIHRo
ZSBJbnRlcm5ldCBQcm90b2NvbCwgd2hldGhlciBJUHY0DSAgIFtSRkMwNzkxXSBvciBJUHY2IFtS
RkMyNDYwXS4gIFRoZSBhZHZhbnRhZ2VzIG9mIGl0cyBjb25uZWN0aW9ubGVzcw0gICBkZXNpZ246
IGZsZXhpYmlsaXR5IGFuZCByb2J1c3RuZXNzLCBoYXZlIGJlZW4gYW1wbHkgZGVtb25zdHJhdGVk
Lg0gICBIb3dldmVyLCB0aGVzZSBhZHZhbnRhZ2VzIGFyZSBub3Qgd2l0aG91dCBjb3N0OiBjYXJl
ZnVsIGRlc2lnbiBpcw0gICByZXF1aXJlZCB0byBwcm92aWRlIGdvb2Qgc2VydmljZSB1bmRlciBo
ZWF2eSBsb2FkLiAgSW4gZmFjdCwgbGFjayBvZg0gICBhdHRlbnRpb24gdG8gdGhlIGR5bmFtaWNz
IG9mIHBhY2tldCBmb3J3YXJkaW5nIGNhbiByZXN1bHQgaW4gc2V2ZXJlDSAgIHNlcnZpY2UgZGVn
cmFkYXRpb24gb3IgIkludGVybmV0IG1lbHRkb3duIi4gIFRoaXMgcGhlbm9tZW5vbiB3YXMNICAg
Zmlyc3Qgb2JzZXJ2ZWQgZHVyaW5nIHRoZSBlYXJseSBncm93dGggcGhhc2Ugb2YgdGhlIEludGVy
bmV0IG9mIHRoZQ0gICBtaWQgMTk4MHMgW1JGQzA4OTZdW1JGQzA5NzBdLCBhbmQgaXMgdGVjaG5p
Y2FsbHkgY2FsbGVkICJjb25nZXN0aXZlDSAgIGNvbGxhcHNlIi4NDSAgIFRoZSBvcmlnaW5hbCBm
aXggZm9yIEludGVybmV0IG1lbHRkb3duIHdhcyBwcm92aWRlZCBieSBWYW4gSmFjb2JzZW4uDSAg
IEJlZ2lubmluZyBpbiAxOTg2LCBKYWNvYnNlbiBkZXZlbG9wZWQgdGhlIGNvbmdlc3Rpb24gYXZv
aWRhbmNlDSAgIG1lY2hhbmlzbXMgdGhhdCBhcmUgbm93IHJlcXVpcmVkIGluIFRDUCBpbXBsZW1l
bnRhdGlvbnMgW0phY29ic29uODhdDSAgIFtSRkMxMTIyXS4gIFRoZXNlIG1lY2hhbmlzbXMgb3Bl
cmF0ZSBpbiBJbnRlcm5ldCBob3N0cyB0byBjYXVzZSBUQ1ANICAgY29ubmVjdGlvbnMgdG8gImJh
Y2sgb2ZmIiBkdXJpbmcgY29uZ2VzdGlvbi4gIFdlIHNheSB0aGF0IFRDUCBmbG93cw0gICBhcmUg
InJlc3BvbnNpdmUiIHRvIGNvbmdlc3Rpb24gc2lnbmFscyAoaS5lLiwgbWFya2VkIG9yIGRyb3Bw
ZWQNICAgcGFja2V0cykgZnJvbSB0aGUgbmV0d29yay4gIEl0IGlzIHByaW1hcmlseSB0aGVzZSBU
Q1AgY29uZ2VzdGlvbg0gICBhdm9pZGFuY2UgYWxnb3JpdGhtcyB0aGF0IHByZXZlbnQgdGhlIGNv
bmdlc3RpdmUgY29sbGFwc2Ugb2YgdG9kYXkncw0gICBJbnRlcm5ldC4NDSAgIEhvd2V2ZXIsIHRo
YXQgaXMgbm90IHRoZSBlbmQgb2YgdGhlIHN0b3J5LiAgQ29uc2lkZXJhYmxlIHJlc2VhcmNoIGhh
cw0gICBiZWVuIGRvbmUgb24gSW50ZXJuZXQgZHluYW1pY3Mgc2luY2UgMTk4OCwgYW5kIHRoZSBJ
bnRlcm5ldCBoYXMNICAgZ3Jvd24uICBJdCBoYXMgYmVjb21lIGNsZWFyIHRoYXQgdGhlIFRDUCBj
b25nZXN0aW9uIGF2b2lkYW5jZQ0gICBtZWNoYW5pc21zIFtSRkM1NjgxXSwgd2hpbGUgbmVjZXNz
YXJ5IGFuZCBwb3dlcmZ1bCwgYXJlIG5vdA0gICBzdWZmaWNpZW50IHRvIHByb3ZpZGUgZ29vZCBz
ZXJ2aWNlIGluIGFsbCBjaXJjdW1zdGFuY2VzLiAgQmFzaWNhbGx5LA0gICB0aGVyZSBpcyBhIGxp
bWl0IHRvIGhvdyBtdWNoIGNvbnRyb2wgY2FuIGJlIGFjY29tcGxpc2hlZCBmcm9tIHRoZQ0gICBl
ZGdlcyBvZiB0aGUgbmV0d29yay4gIFNvbWUgbWVjaGFuaXNtcyBhcmUgbmVlZGVkIGluIHRoZSBu
ZXR3b3JrDSAgIGRldmljZXMgdG8gY29tcGxlbWVudCB0aGUgZW5kcG9pbnQgY29uZ2VzdGlvbiBh
dm9pZGFuY2UgbWVjaGFuaXNtcy4NICAgVGhlc2UgbWVjaGFuaXNtcyBtYXkgYmUgaW1wbGVtZW50
ZWQgaW4gbmV0d29yayBkZXZpY2VzIHRoYXQgaW5jbHVkZQ0gICByb3V0ZXJzLCBzd2l0Y2hlcywg
YW5kIG90aGVyIG5ldHdvcmsgbWlkZGxlYm94ZXMuDQ0gICBJdCBpcyB1c2VmdWwgdG8gZGlzdGlu
Z3Vpc2ggYmV0d2VlbiB0d28gY2xhc3NlcyBvZiBhbGdvcml0aG1zIHJlbGF0ZWQNICAgdG8gY29u
Z2VzdGlvbiBjb250cm9sOiAicXVldWUgbWFuYWdlbWVudCIgdmVyc3VzICJzY2hlZHVsaW5nIg0g
ICBhbGdvcml0aG1zLiAgVG8gYSByb3VnaCBhcHByb3hpbWF0aW9uLCBxdWV1ZSBtYW5hZ2VtZW50
IGFsZ29yaXRobXMNICAgbWFuYWdlIHRoZSBsZW5ndGggb2YgcGFja2V0IHF1ZXVlcyBieSBtYXJr
aW5nIG9yIGRyb3BwaW5nIHBhY2tldHMNICAgd2hlbiBuZWNlc3Nhcnkgb3IgYXBwcm9wcmlhdGUs
IHdoaWxlIHNjaGVkdWxpbmcgYWxnb3JpdGhtcyBkZXRlcm1pbmUNICAgd2hpY2ggcGFja2V0IHRv
IHNlbmQgbmV4dCBhbmQgYXJlIHVzZWQgcHJpbWFyaWx5IHRvIG1hbmFnZSB0aGUNICAgYWxsb2Nh
dGlvbiBvZiBiYW5kd2lkdGggYW1vbmcgZmxvd3MuICBXaGlsZSB0aGVzZSB0d28gQVFNIG1lY2hh
bmlzbXMNICAgYXJlIGNsb3NlbHkgcmVsYXRlZCwgdGhleSBhZGRyZXNzIGRpZmZlcmVudCBwZXJm
b3JtYW5jZSBpc3N1ZXMuDQ0gICBUaGlzIG1lbW8gaGlnaGxpZ2h0cyB0d28gcGVyZm9ybWFuY2Ug
aXNzdWVzOg0NICAgVGhlIGZpcnN0IGlzc3VlIGlzIHRoZSBuZWVkIGZvciBhbiBhZHZhbmNlZCBm
b3JtIG9mIHF1ZXVlIG1hbmFnZW1lbnQNICAgdGhhdCB3ZSBjYWxsICJhY3RpdmUgcXVldWUgbWFu
YWdlbWVudC4iICBTZWN0aW9uIDIgc3VtbWFyaXplcyB0aGUNICAgYmVuZWZpdHMgdGhhdCBhY3Rp
dmUgcXVldWUgbWFuYWdlbWVudCBjYW4gYnJpbmcuICBBIG51bWJlciBvZiBBY3RpdmUNICAgUXVl
dWUgTWFuYWdlbWVudCAoQVFNKSBwcm9jZWR1cmVzIGFyZSBkZXNjcmliZWQgaW4gdGhlIGxpdGVy
YXR1cmUsDQ0NDUJha2VyICYgRmFpcmh1cnN0ICAgICAgIEV4cGlyZXMgSmFudWFyeSAxMCwgMjAx
NCAgICAgICAgICAgICAgICBbUGFnZSAzXQ0MDUludGVybmV0LURyYWZ0ICAgQWN0aXZlIFF1ZXVl
IE1hbmFnZW1lbnQgUmVjb21tZW5kYXRpb25zICAgICAgIEp1bHkgMjAxMw0NDSAgIHdpdGggZGlm
ZmVyZW50IGNoYXJhY3RlcmlzdGljcy4gIFRoaXMgZG9jdW1lbnQgZG9lcyBub3QgcmVjb21tZW5k
IGFueQ0gICBvZiB0aGVtIGluIHBhcnRpY3VsYXIsIGJ1dCBkb2VzIG1ha2UgcmVjb21tZW5kYXRp
b25zIHRoYXQgaWRlYWxseQ0gICB3b3VsZCBhZmZlY3QgdGhlIGNob2ljZSBvZiBwcm9jZWR1cmUg
dXNlZCBpbiBhIGdpdmVuIGltcGxlbWVudGF0aW9uLg0NICAgVGhlIHNlY29uZCBpc3N1ZSwgZGlz
Y3Vzc2VkIGluIFNlY3Rpb24gMyBvZiB0aGlzIG1lbW8sIGlzIHRoZQ0gICBwb3RlbnRpYWwgZm9y
IGZ1dHVyZSBjb25nZXN0aXZlIGNvbGxhcHNlIG9mIHRoZSBJbnRlcm5ldCBkdWUgdG8gZmxvd3MN
ICAgdGhhdCBhcmUgdW5yZXNwb25zaXZlLCBvciBub3Qgc3VmZmljaWVudGx5IHJlc3BvbnNpdmUs
IHRvIGNvbmdlc3Rpb24NICAgaW5kaWNhdGlvbnMuICBVbmZvcnR1bmF0ZWx5LCB0aGVyZSBpcyBu
byBjb25zZW5zdXMgc29sdXRpb24gdG8NICAgY29udHJvbGxpbmcgY29uZ2VzdGlvbiBjYXVzZWQg
Ynkgc3VjaCBhZ2dyZXNzaXZlIGZsb3dzOyBzaWduaWZpY2FudA0gICByZXNlYXJjaCBhbmQgZW5n
aW5lZXJpbmcgd2lsbCBiZSByZXF1aXJlZCBiZWZvcmUgYW55IHNvbHV0aW9uIHdpbGwgYmUNICAg
YXZhaWxhYmxlLiAgSXQgaXMgaW1wZXJhdGl2ZSB0aGF0IHRoaXMgd29yayBiZSBlbmVyZ2V0aWNh
bGx5IHB1cnN1ZWQsDSAgIHRvIGVuc3VyZSB0aGUgZnV0dXJlIHN0YWJpbGl0eSBvZiB0aGUgSW50
ZXJuZXQuDQ0gICBTZWN0aW9uIDQgY29uY2x1ZGVzIHRoZSBtZW1vIHdpdGggYSBzZXQgb2YgcmVj
b21tZW5kYXRpb25zIHRvIHRoZQ0gICBJbnRlcm5ldCBjb21tdW5pdHkgY29uY2VybmluZyB0aGVz
ZSB0b3BpY3MuDQ0gICBUaGUgZGlzY3Vzc2lvbiBpbiB0aGlzIG1lbW8gYXBwbGllcyB0byAiYmVz
dC1lZmZvcnQiIHRyYWZmaWMsIHdoaWNoDSAgIGlzIHRvIHNheSwgdHJhZmZpYyBnZW5lcmF0ZWQg
YnkgYXBwbGljYXRpb25zIHRoYXQgYWNjZXB0IHRoZQ0gICBvY2Nhc2lvbmFsIGxvc3MsIGR1cGxp
Y2F0aW9uLCBvciByZW9yZGVyaW5nIG9mIHRyYWZmaWMgaW4gZmxpZ2h0LiAgSXQNICAgYWxzbyBh
cHBsaWVzIHRvIG90aGVyIHRyYWZmaWMsIHN1Y2ggYXMgcmVhbC10aW1lIHRyYWZmaWMgdGhhdCBj
YW4NICAgYWRhcHQgaXRzIHNlbmRpbmcgcmF0ZSB0byByZWR1Y2UgbG9zcyBhbmQvb3IgZGVsYXku
ICBJdCBpcyBtb3N0DSAgIGVmZmVjdGl2ZSwgd2hlbiB0aGUgYWRhcHRpb24gb2NjdXJzIG9uIHRp
bWUgc2NhbGVzIG9mIGEgc2luZ2xlIFJUVCBvcg0gICBhIHNtYWxsIG51bWJlciBvZiBSVFRzLCBm
b3IgZWxhc3RpYyB0cmFmZmljIFtSRkMxNjMzXS4NDSAgIFtSRkMyMzA5XSByZXN1bHRlZCBmcm9t
IHBhc3QgZGlzY3Vzc2lvbnMgb2YgZW5kLXRvLWVuZCBwZXJmb3JtYW5jZSwNICAgSW50ZXJuZXQg
Y29uZ2VzdGlvbiwgYW5kIFJFRCBpbiB0aGUgRW5kLXRvLUVuZCBSZXNlYXJjaCBHcm91cCBvZiB0
aGUNICAgSW50ZXJuZXQgUmVzZWFyY2ggVGFzayBGb3JjZSAoSVJURikuICBUaGlzIHVwZGF0ZSBy
ZXN1bHRzIGZyb20NICAgZXhwZXJpZW5jZSB3aXRoIHRoaXMgYW5kIG90aGVyIGFsZ29yaXRobXMs
IGFuZCB0aGUgQWN0aXZlIFF1ZXVlDSAgIE1hbmFnZW1lbnQgZGlzY3Vzc2lvbiB3aXRoaW4gdGhl
IElFVEYuDQ0xLjEuICBSZXF1aXJlbWVudHMgTGFuZ3VhZ2UNDSAgIFRoZSBrZXkgd29yZHMgIk1V
U1QiLCAiTVVTVCBOT1QiLCAiUkVRVUlSRUQiLCAiU0hBTEwiLCAiU0hBTEwgTk9UIiwNICAgIlNI
T1VMRCIsICJTSE9VTEQgTk9UIiwgIlJFQ09NTUVOREVEIiwgIk1BWSIsIGFuZCAiT1BUSU9OQUwi
IGluIHRoaXMNICAgZG9jdW1lbnQgYXJlIHRvIGJlIGludGVycHJldGVkIGFzIGRlc2NyaWJlZCBp
biBbUkZDMjExOV0uDQ0yLiAgVGhlIE5lZWQgRm9yIEFjdGl2ZSBRdWV1ZSBNYW5hZ2VtZW50DQ0g
ICBUaGUgdHJhZGl0aW9uYWwgdGVjaG5pcXVlIGZvciBtYW5hZ2luZyB0aGUgcXVldWUgbGVuZ3Ro
IGluIGEgbmV0d29yaw0gICBkZXZpY2UgaXMgdG8gc2V0IGEgbWF4aW11bSBsZW5ndGggKGluIHRl
cm1zIG9mIHBhY2tldHMpIGZvciBlYWNoDSAgIHF1ZXVlLCBhY2NlcHQgcGFja2V0cyBmb3IgdGhl
IHF1ZXVlIHVudGlsIHRoZSBtYXhpbXVtIGxlbmd0aCBpcw0gICByZWFjaGVkLCB0aGVuIHJlamVj
dCAoZHJvcCkgc3Vic2VxdWVudCBpbmNvbWluZyBwYWNrZXRzIHVudGlsIHRoZQ0gICBxdWV1ZSBk
ZWNyZWFzZXMgYmVjYXVzZSBhIHBhY2tldCBmcm9tIHRoZSBxdWV1ZSBoYXMgYmVlbiB0cmFuc21p
dHRlZC4NICAgVGhpcyB0ZWNobmlxdWUgaXMga25vd24gYXMgInRhaWwgZHJvcCIsIHNpbmNlIHRo
ZSBwYWNrZXQgdGhhdCBhcnJpdmVkDSAgIG1vc3QgcmVjZW50bHkgKGkuZS4sIHRoZSBvbmUgb24g
dGhlIHRhaWwgb2YgdGhlIHF1ZXVlKSBpcyBkcm9wcGVkDSAgIHdoZW4gdGhlIHF1ZXVlIGlzIGZ1
bGwuICBUaGlzIG1ldGhvZCBoYXMgc2VydmVkIHRoZSBJbnRlcm5ldCB3ZWxsIGZvcg0gICB5ZWFy
cywgYnV0IGl0IGhhcyB0d28gaW1wb3J0YW50IGRyYXdiYWNrcy4NDQ0NDUJha2VyICYgRmFpcmh1
cnN0ICAgICAgIEV4cGlyZXMgSmFudWFyeSAxMCwgMjAxNCAgICAgICAgICAgICAgICBbUGFnZSA0
XQ0MDUludGVybmV0LURyYWZ0ICAgQWN0aXZlIFF1ZXVlIE1hbmFnZW1lbnQgUmVjb21tZW5kYXRp
b25zICAgICAgIEp1bHkgMjAxMw0NDSAgIDEuICBMb2NrLU91dA0NICAgICAgIEluIHNvbWUgc2l0
dWF0aW9ucyB0YWlsIGRyb3AgYWxsb3dzIGEgc2luZ2xlIGNvbm5lY3Rpb24gb3IgYSBmZXcNICAg
ICAgIGZsb3dzIHRvIG1vbm9wb2xpemUgcXVldWUgc3BhY2UsIHByZXZlbnRpbmcgb3RoZXIgY29u
bmVjdGlvbnMNICAgICAgIGZyb20gZ2V0dGluZyByb29tIGluIHRoZSBxdWV1ZS4gIFRoaXMgImxv
Y2stb3V0IiBwaGVub21lbm9uIGlzDSAgICAgICBvZnRlbiB0aGUgcmVzdWx0IG9mIHN5bmNocm9u
aXphdGlvbiBvciBvdGhlciB0aW1pbmcgZWZmZWN0cy4NDSAgIDIuICBGdWxsIFF1ZXVlcw0NICAg
ICAgIFRoZSB0YWlsIGRyb3AgZGlzY2lwbGluZSBhbGxvd3MgcXVldWVzIHRvIG1haW50YWluIGEg
ZnVsbCAob3IsDSAgICAgICBhbG1vc3QgZnVsbCkgc3RhdHVzIGZvciBsb25nIHBlcmlvZHMgb2Yg
dGltZSwgc2luY2UgdGFpbCBkcm9wDSAgICAgICBzaWduYWxzIGNvbmdlc3Rpb24gKHZpYSBhIHBh
Y2tldCBkcm9wKSBvbmx5IHdoZW4gdGhlIHF1ZXVlIGhhcw0gICAgICAgYmVjb21lIGZ1bGwuICBJ
dCBpcyBpbXBvcnRhbnQgdG8gcmVkdWNlIHRoZSBzdGVhZHktc3RhdGUgcXVldWUNICAgICAgIHNp
emUsIGFuZCB0aGlzIGlzIHBlcmhhcHMgdGhlIG1vc3QgaW1wb3J0YW50IGdvYWwgZm9yIHF1ZXVl
DSAgICAgICBtYW5hZ2VtZW50Lg0NICAgICAgIFRoZSBuYWl2ZSBhc3N1bXB0aW9uIG1pZ2h0IGJl
IHRoYXQgdGhlcmUgaXMgYSBzaW1wbGUgdHJhZGVvZmYNICAgICAgIGJldHdlZW4gZGVsYXkgYW5k
IHRocm91Z2hwdXQsIGFuZCB0aGF0IHRoZSByZWNvbW1lbmRhdGlvbiB0aGF0DSAgICAgICBxdWV1
ZXMgYmUgbWFpbnRhaW5lZCBpbiBhICJub24tZnVsbCIgc3RhdGUgZXNzZW50aWFsbHkgdHJhbnNs
YXRlcw0gICAgICAgdG8gYSByZWNvbW1lbmRhdGlvbiB0aGF0IGxvdyBlbmQtdG8tZW5kIGRlbGF5
IGlzIG1vcmUgaW1wb3J0YW50DSAgICAgICB0aGFuIGhpZ2ggdGhyb3VnaHB1dC4gIEhvd2V2ZXIs
IHRoaXMgZG9lcyBub3QgdGFrZSBpbnRvIGFjY291bnQNICAgICAgIHRoZSBjcml0aWNhbCByb2xl
IHRoYXQgcGFja2V0IGJ1cnN0cyBwbGF5IGluIEludGVybmV0DSAgICAgICBwZXJmb3JtYW5jZS4g
IEV2ZW4gdGhvdWdoIFRDUCBjb25zdHJhaW5zIHRoZSBjb25nZXN0aW9uIHdpbmRvdyBvZg0gICAg
ICAgYSBmbG93LCBwYWNrZXRzIG9mdGVuIGFycml2ZSBhdCBuZXR3b3JrIGRldmljZXMgaW4gYnVy
c3RzDSAgICAgICBbTGVsYW5kOTRdLiAgSWYgdGhlIHF1ZXVlIGlzIGZ1bGwgb3IgYWxtb3N0IGZ1
bGwsIGFuIGFycml2aW5nDSAgICAgICBidXJzdCB3aWxsIGNhdXNlIG11bHRpcGxlIHBhY2tldHMg
dG8gYmUgZHJvcHBlZC4gIFRoaXMgY2FuIHJlc3VsdA0gICAgICAgaW4gYSBnbG9iYWwgc3luY2hy
b25pemF0aW9uIG9mIGZsb3dzIHRocm90dGxpbmcgYmFjaywgZm9sbG93ZWQgYnkNICAgICAgIGEg
c3VzdGFpbmVkIHBlcmlvZCBvZiBsb3dlcmVkIGxpbmsgdXRpbGl6YXRpb24sIHJlZHVjaW5nIG92
ZXJhbGwNICAgICAgIHRocm91Z2hwdXQuDQ0gICAgICAgVGhlIHBvaW50IG9mIGJ1ZmZlcmluZyBp
biB0aGUgbmV0d29yayBpcyB0byBhYnNvcmIgZGF0YSBidXJzdHMNICAgICAgIGFuZCB0byB0cmFu
c21pdCB0aGVtIGR1cmluZyB0aGUgKGhvcGVmdWxseSkgZW5zdWluZyBidXJzdHMgb2YNICAgICAg
IHNpbGVuY2UuICBUaGlzIGlzIGVzc2VudGlhbCB0byBwZXJtaXQgdGhlIHRyYW5zbWlzc2lvbiBv
ZiBidXJzdHkNICAgICAgIGRhdGEuICBOb3JtYWxseSB3ZSB3b3VsZCBsaWtlIHRvIGhhdmUgbWFs
bCBxdWV1ZXMgaW4gbmV0d29yaw0gICAgICAgZGV2aWNlczogd2l0aCBzdWZmaWNpZW50IHF1ZXVl
IGNhcGFjaXR5IHRvIGFic29yYiB0aGUgYnVyc3RzLg0gICAgICAgVGhlIGNvdW50ZXItaW50dWl0
aXZlIHJlc3VsdCBpcyB0aGF0IG1haW50YWluaW5nIG5vcm1hbGx5LXNtYWxsDSAgICAgICBxdWV1
ZXMgY2FuIHJlc3VsdCBpbiBoaWdoZXIgdGhyb3VnaHB1dCBhcyB3ZWxsIGFzIGxvd2VyIGVuZC10
by0NICAgICAgIGVuZCBkZWxheS4gIEluIHNob3J0LCBxdWV1ZSBsaW1pdHMgc2hvdWxkIG5vdCBy
ZWZsZWN0IHRoZSBzdGVhZHkNICAgICAgIHN0YXRlIHF1ZXVlcyB3ZSB3YW50IHRvIGJlIG1haW50
YWluZWQgaW4gdGhlIG5ldHdvcms7IGluc3RlYWQsDSAgICAgICB0aGV5IHNob3VsZCByZWZsZWN0
IHRoZSBzaXplIG9mIGJ1cnN0cyB0aGF0IGEgbmV0d29yayBkZXZpY2UNICAgICAgIG5lZWRzIHRv
IGFic29yYi4NDSAgIEJlc2lkZXMgdGFpbCBkcm9wLCB0d28gYWx0ZXJuYXRpdmUgcXVldWUgZGlz
Y2lwbGluZXMgdGhhdCBjYW4gYmUNICAgYXBwbGllZCB3aGVuIGEgcXVldWUgYmVjb21lcyBmdWxs
IGFyZSAicmFuZG9tIGRyb3Agb24gZnVsbCIgb3IgImRyb3ANICAgZnJvbnQgb24gZnVsbCIuICBV
bmRlciB0aGUgcmFuZG9tIGRyb3Agb24gZnVsbCBkaXNjaXBsaW5lLCBhIG5ldHdvcmsNICAgZGV2
aWNlIGRyb3BzIGEgcmFuZG9tbHkgc2VsZWN0ZWQgcGFja2V0IGZyb20gdGhlIHF1ZXVlICh3aGlj
aCBjYW4gYmUNICAgYW4gZXhwZW5zaXZlIG9wZXJhdGlvbiwgc2luY2UgaXQgbmFpdmVseSByZXF1
aXJlcyBhbiBPKE4pIHdhbGsNICAgdGhyb3VnaCB0aGUgcGFja2V0IHF1ZXVlKSB3aGVuIHRoZSBx
dWV1ZSBpcyBmdWxsIGFuZCBhIG5ldyBwYWNrZXQNDQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAg
RXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAgICAgICAgICAgIFtQYWdlIDVdDQwNSW50ZXJu
ZXQtRHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAg
SnVseSAyMDEzDQ0NICAgYXJyaXZlcy4gIFVuZGVyIHRoZSAiZHJvcCBmcm9udCBvbiBmdWxsIiBk
aXNjaXBsaW5lIFtMYWtzaG1hbjk2XSwgdGhlDSAgIG5ldHdvcmsgZGV2aWNlIGRyb3BzIHRoZSBw
YWNrZXQgYXQgdGhlIGZyb250IG9mIHRoZSBxdWV1ZSB3aGVuIHRoZQ0gICBxdWV1ZSBpcyBmdWxs
IGFuZCBhIG5ldyBwYWNrZXQgYXJyaXZlcy4gIEJvdGggb2YgdGhlc2Ugc29sdmUgdGhlDSAgIGxv
Y2stb3V0IHByb2JsZW0sIGJ1dCBuZWl0aGVyIHNvbHZlcyB0aGUgZnVsbC1xdWV1ZXMgcHJvYmxl
bQ0gICBkZXNjcmliZWQgYWJvdmUuDQ0gICBXZSBrbm93IGluIGdlbmVyYWwgaG93IHRvIHNvbHZl
IHRoZSBmdWxsLXF1ZXVlcyBwcm9ibGVtIGZvcg0gICAicmVzcG9uc2l2ZSIgZmxvd3MsIGkuZS4s
IHRob3NlIGZsb3dzIHRoYXQgdGhyb3R0bGUgYmFjayBpbiByZXNwb25zZQ0gICB0byBjb25nZXN0
aW9uIG5vdGlmaWNhdGlvbi4gIEluIHRoZSBjdXJyZW50IEludGVybmV0LCBkcm9wcGVkIHBhY2tl
dHMNICAgcHJvdmlkZSBhIGNyaXRpY2FsIG1lY2hhbmlzbSBpbmRpY2F0aW5nIGNvbmdlc3Rpb24g
bm90aWZpY2F0aW9uIHRvDSAgIGhvc3RzLiAgVGhlIHNvbHV0aW9uIHRvIHRoZSBmdWxsLXF1ZXVl
cyBwcm9ibGVtIGlzIGZvciBuZXR3b3JrDSAgIGRldmljZXMgdG8gZHJvcCBwYWNrZXRzIGJlZm9y
ZSBhIHF1ZXVlIGJlY29tZXMgZnVsbCwgc28gdGhhdCBob3N0cw0gICBjYW4gcmVzcG9uZCB0byBj
b25nZXN0aW9uIGJlZm9yZSBidWZmZXJzIG92ZXJmbG93LiAgV2UgY2FsbCBzdWNoIGENICAgcHJv
YWN0aXZlIGFwcHJvYWNoIEFRTS4gIEJ5IGRyb3BwaW5nIHBhY2tldHMgYmVmb3JlIGJ1ZmZlcnMg
b3ZlcmZsb3csDSAgIEFRTSBhbGxvd3MgbmV0d29yayBkZXZpY2VzIHRvIGNvbnRyb2wgd2hlbiBh
bmQgaG93IG1hbnkgcGFja2V0cyB0bw0gICBkcm9wLg0NICAgSW4gc3VtbWFyeSwgYW4gYWN0aXZl
IHF1ZXVlIG1hbmFnZW1lbnQgbWVjaGFuaXNtIGNhbiBwcm92aWRlIHRoZQ0gICBmb2xsb3dpbmcg
YWR2YW50YWdlcyBmb3IgcmVzcG9uc2l2ZSBmbG93cy4NDSAgIDEuICBSZWR1Y2UgbnVtYmVyIG9m
IHBhY2tldHMgZHJvcHBlZCBpbiBuZXR3b3JrIGRldmljZXMNDSAgICAgICBQYWNrZXQgYnVyc3Rz
IGFyZSBhbiB1bmF2b2lkYWJsZSBhc3BlY3Qgb2YgcGFja2V0IG5ldHdvcmtzDSAgICAgICBbV2ls
bGluZ2VyOTVdLiAgSWYgYWxsIHRoZSBxdWV1ZSBzcGFjZSBpbiBhIG5ldHdvcmsgZGV2aWNlIGlz
DSAgICAgICBhbHJlYWR5IGNvbW1pdHRlZCB0byAic3RlYWR5IHN0YXRlIiB0cmFmZmljIG9yIGlm
IHRoZSBidWZmZXINICAgICAgIHNwYWNlIGlzIGluYWRlcXVhdGUsIHRoZW4gdGhlIG5ldHdvcmsg
ZGV2aWNlIHdpbGwgaGF2ZSBubyBhYmlsaXR5DSAgICAgICB0byBidWZmZXIgYnVyc3RzLiAgQnkg
a2VlcGluZyB0aGUgYXZlcmFnZSBxdWV1ZSBzaXplIHNtYWxsLCBBUU0NICAgICAgIHdpbGwgcHJv
dmlkZSBncmVhdGVyIGNhcGFjaXR5IHRvIGFic29yYiBuYXR1cmFsbHktb2NjdXJyaW5nDSAgICAg
ICBidXJzdHMgd2l0aG91dCBkcm9wcGluZyBwYWNrZXRzLg0NICAgICAgIEZ1cnRoZXJtb3JlLCB3
aXRob3V0IEFRTSwgbW9yZSBwYWNrZXRzIHdpbGwgYmUgZHJvcHBlZCB3aGVuIGENICAgICAgIHF1
ZXVlIGRvZXMgb3ZlcmZsb3cuICBUaGlzIGlzIHVuZGVzaXJhYmxlIGZvciBzZXZlcmFsIHJlYXNv
bnMuDSAgICAgICBGaXJzdCwgd2l0aCBhIHNoYXJlZCBxdWV1ZSBhbmQgdGhlIHRhaWwgZHJvcCBk
aXNjaXBsaW5lLCB0aGlzIGNhbg0gICAgICAgcmVzdWx0IGluIHVubmVjZXNzYXJ5IGdsb2JhbCBz
eW5jaHJvbml6YXRpb24gb2YgZmxvd3MsIHJlc3VsdGluZw0gICAgICAgaW4gbG93ZXJlZCBhdmVy
YWdlIGxpbmsgdXRpbGl6YXRpb24sIGFuZCBoZW5jZSBsb3dlcmVkIG5ldHdvcmsNICAgICAgIHRo
cm91Z2hwdXQuICBTZWNvbmQsIHVubmVjZXNzYXJ5IHBhY2tldCBkcm9wcyByZXByZXNlbnQgYQ0g
ICAgICAgcG9zc2libGUgd2FzdGUgb2YgbmV0d29yayBjYXBhY2l0eSBvbiB0aGUgcGF0aCBiZWZv
cmUgdGhlIGRyb3ANICAgICAgIHBvaW50Lg0NICAgICAgIFdoaWxlIEFRTSBjYW4gbWFuYWdlIHF1
ZXVlIGxlbmd0aHMgYW5kIHJlZHVjZSBlbmQtdG8tZW5kIGxhdGVuY3kNICAgICAgIGV2ZW4gaW4g
dGhlIGFic2VuY2Ugb2YgZW5kLXRvLWVuZCBjb25nZXN0aW9uIGNvbnRyb2wsIGl0IHdpbGwgYmUN
ICAgICAgIGFibGUgdG8gcmVkdWNlIHBhY2tldCBkcm9wcyBvbmx5IGluIGFuIGVudmlyb25tZW50
IHRoYXQgY29udGludWVzDSAgICAgICB0byBiZSBkb21pbmF0ZWQgYnkgZW5kLXRvLWVuZCBjb25n
ZXN0aW9uIGNvbnRyb2wuDQ0gICAyLiAgUHJvdmlkZSBhIGxvd2VyLWRlbGF5IGludGVyYWN0aXZl
IHNlcnZpY2UNDQ0NDQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEw
LCAyMDE0ICAgICAgICAgICAgICAgIFtQYWdlIDZdDQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUg
UXVldWUgTWFuYWdlbWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAgSnVseSAyMDEzDQ0NICAgICAg
IEJ5IGtlZXBpbmcgYSBzbWFsbCBhdmVyYWdlIHF1ZXVlIHNpemUsIEFRTSB3aWxsIHJlZHVjZSB0
aGUgZGVsYXlzDSAgICAgICBleHBlcmllbmNlZCBieSBmbG93cy4gIFRoaXMgaXMgcGFydGljdWxh
cmx5IGltcG9ydGFudCBmb3INICAgICAgIGludGVyYWN0aXZlIGFwcGxpY2F0aW9ucyBzdWNoIGFz
IHNob3J0IFdlYiB0cmFuc2ZlcnMsIFRlbG5ldA0gICAgICAgdHJhZmZpYywgb3IgaW50ZXJhY3Rp
dmUgYXVkaW8tdmlkZW8gc2Vzc2lvbnMsIHdob3NlIHN1YmplY3RpdmUNICAgICAgIChhbmQgb2Jq
ZWN0aXZlKSBwZXJmb3JtYW5jZSBpcyBiZXR0ZXIgd2hlbiB0aGUgZW5kLXRvLWVuZCBkZWxheQ0g
ICAgICAgaXMgbG93Lg0NICAgMy4gIEF2b2lkIGxvY2stb3V0IGJlaGF2aW9yDQ0gICAgICAgQVFN
IGNhbiBwcmV2ZW50IGxvY2stb3V0IGJlaGF2aW9yIGJ5IGVuc3VyaW5nIHRoYXQgdGhlcmUgd2ls
bA0gICAgICAgYWxtb3N0IGFsd2F5cyBiZSBhIGJ1ZmZlciBhdmFpbGFibGUgZm9yIGFuIGluY29t
aW5nIHBhY2tldC4gIEZvcg0gICAgICAgdGhlIHNhbWUgcmVhc29uLCBBUU0gY2FuIHByZXZlbnQg
YSBiaWFzIGFnYWluc3QgbG93IGNhcGFjaXR5LCBidXQNICAgICAgIGhpZ2hseSBidXJzdHksIGZs
b3dzLg0NICAgICAgIExvY2stb3V0IGlzIHVuZGVzaXJhYmxlIGJlY2F1c2UgaXQgY29uc3RpdHV0
ZXMgYSBncm9zcyB1bmZhaXJuZXNzDSAgICAgICBhbW9uZyBncm91cHMgb2YgZmxvd3MuICBIb3dl
dmVyLCB3ZSBzdG9wIHNob3J0IG9mIGNhbGxpbmcgdGhpcw0gICAgICAgYmVuZWZpdCAiaW5jcmVh
c2VkIGZhaXJuZXNzIiwgYmVjYXVzZSBnZW5lcmFsIGZhaXJuZXNzIGFtb25nDSAgICAgICBmbG93
cyByZXF1aXJlcyBwZXItZmxvdyBzdGF0ZSwgd2hpY2ggaXMgbm90IHByb3ZpZGVkIGJ5IHF1ZXVl
DSAgICAgICBtYW5hZ2VtZW50LiAgRm9yIGV4YW1wbGUsIGluIGEgbmV0d29yayBkZXZpY2UgdXNp
bmcgQVFNIHdpdGggb25seQ0gICAgICAgRklGTyBzY2hlZHVsaW5nLCB0d28gVENQIGZsb3dzIG1h
eSByZWNlaXZlIHZlcnkgZGlmZmVyZW50IHNoYXJlDSAgICAgICBvZiB0aGUgbmV0d29yayBjYXBh
Y2l0eSBzaW1wbHkgYmVjYXVzZSB0aGV5IGhhdmUgZGlmZmVyZW50IHJvdW5kLQ0gICAgICAgdHJp
cCB0aW1lcyBbRmxveWQ5MV0sIGFuZCBhIGZsb3cgdGhhdCBkb2VzIG5vdCB1c2UgY29uZ2VzdGlv
bg0gICAgICAgY29udHJvbCBtYXkgcmVjZWl2ZSBtb3JlIGNhcGFjaXR5IHRoYW4gYSBmbG93IHRo
YXQgZG9lcy4gIEZvcg0gICAgICAgZXhhbXBsZSwgYSByb3V0ZXIgbWF5IG1haW50YWluIHBlci1m
bG93IHN0YXRlIHRvIGFjaGlldmUgZ2VuZXJhbA0gICAgICAgZmFpcm5lc3MgYnkgYSBwZXItZmxv
dyBzY2hlZHVsaW5nIGFsZ29yaXRobSBzdWNoIGFzIEZhaXIgUXVldWVpbmcNICAgICAgIChGUSkg
W0RlbWVyczkwXSwgb3IgYSBDbGFzcy1CYXNlZCBRdWV1ZSBzY2hlZHVsaW5nIGFsZ29yaXRobSBz
dWNoDSAgICAgICBhcyBDQlEgW0Zsb3lkOTVdLg0NICAgICAgIEluIGNvbnRyYXN0LCBBUU0gaXMg
bmVlZGVkIGV2ZW4gZm9yIG5ldHdvcmsgZGV2aWNlcyB0aGF0IHVzZSBwZXItDSAgICAgICBmbG93
IHNjaGVkdWxpbmcgYWxnb3JpdGhtcyBzdWNoIGFzIEZRIG9yIGNsYXNzLWJhc2VkIHNjaGVkdWxp
bmcNICAgICAgIGFsZ29yaXRobXMsIHN1Y2ggYXMgQ0JRLiAgVGhpcyBpcyBiZWNhdXNlIHBlci1m
bG93IHNjaGVkdWxpbmcNICAgICAgIGFsZ29yaXRobXMgYnkgdGhlbXNlbHZlcyBkbyBub3QgY29u
dHJvbCB0aGUgb3ZlcmFsbCBxdWV1ZSBzaXplIG9yDSAgICAgICB0aGUgc2l6ZSBvZiBpbmRpdmlk
dWFsIHF1ZXVlcy4gIEFRTSBpcyBuZWVkZWQgdG8gY29udHJvbCB0aGUNICAgICAgIG92ZXJhbGwg
YXZlcmFnZSBxdWV1ZSBzaXplcywgc28gdGhhdCBhcnJpdmluZyBidXJzdHMgY2FuIGJlDSAgICAg
ICBhY2NvbW1vZGF0ZWQgd2l0aG91dCBkcm9wcGluZyBwYWNrZXRzLiAgSW4gYWRkaXRpb24sIEFR
TSBzaG91bGQNICAgICAgIGJlIHVzZWQgdG8gY29udHJvbCB0aGUgcXVldWUgc2l6ZSBmb3IgZWFj
aCBpbmRpdmlkdWFsIGZsb3cgb3INICAgICAgIGNsYXNzLCBzbyB0aGF0IHRoZXkgZG8gbm90IGV4
cGVyaWVuY2UgdW5uZWNlc3NhcmlseSBoaWdoIGRlbGF5Lg0gICAgICAgVGhlcmVmb3JlLCBBUU0g
c2hvdWxkIGJlIGFwcGxpZWQgYWNyb3NzIHRoZSBjbGFzc2VzIG9yIGZsb3dzIGFzDSAgICAgICB3
ZWxsIGFzIHdpdGhpbiBlYWNoIGNsYXNzIG9yIGZsb3cuDQ0gICAgICAgSW4gc2hvcnQsIHNjaGVk
dWxpbmcgYWxnb3JpdGhtcyBhbmQgcXVldWUgbWFuYWdlbWVudCBzaG91bGQgYmUNICAgICAgIHNl
ZW4gYXMgY29tcGxlbWVudGFyeSwgbm90IGFzIHJlcGxhY2VtZW50cyBmb3IgZWFjaCBvdGhlci4N
DQ0NDQ0NDQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0
ICAgICAgICAgICAgICAgIFtQYWdlIDddDQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUgUXVldWUg
TWFuYWdlbWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAgSnVseSAyMDEzDQ0NMy4gIE1hbmFnaW5n
IEFnZ3Jlc3NpdmUgRmxvd3MNDSAgIE9uZSBvZiB0aGUga2V5cyB0byB0aGUgc3VjY2VzcyBvZiB0
aGUgSW50ZXJuZXQgaGFzIGJlZW4gdGhlDSAgIGNvbmdlc3Rpb24gYXZvaWRhbmNlIG1lY2hhbmlz
bXMgb2YgVENQLiAgQmVjYXVzZSBUQ1AgImJhY2tzIG9mZiINICAgZHVyaW5nIGNvbmdlc3Rpb24s
IGEgbGFyZ2UgbnVtYmVyIG9mIFRDUCBjb25uZWN0aW9ucyBjYW4gc2hhcmUgYQ0gICBzaW5nbGUs
IGNvbmdlc3RlZCBsaW5rIGluIHN1Y2ggYSB3YXkgdGhhdCBsaW5rIGJhbmR3aWR0aCBpcyBzaGFy
ZWQNICAgcmVhc29uYWJseSBlcXVpdGFibHkgYW1vbmcgc2ltaWxhcmx5IHNpdHVhdGVkIGZsb3dz
LiAgVGhlIGVxdWl0YWJsZQ0gICBzaGFyaW5nIG9mIGJhbmR3aWR0aCBhbW9uZyBmbG93cyBkZXBl
bmRzIG9uIGFsbCBmbG93cyBydW5uaW5nDSAgIGNvbXBhdGlibGUgY29uZ2VzdGlvbiBhdm9pZGFu
Y2UgYWxnb3JpdGhtcywgaS5lLiwgbWV0aG9kcyBjb25mb3JtYW50DSAgIHdpdGggdGhlIGN1cnJl
bnQgVENQIHNwZWNpZmljYXRpb24gW1JGQzU2ODFdLg0NICAgV2UgY2FsbCBhIGZsb3cgIlRDUC1m
cmllbmRseSIgd2hlbiBpdCBoYXMgYSBjb25nZXN0aW9uIHJlc3BvbnNlIHRoYXQNICAgYXBwcm94
aW1hdGVzIHRoZSBhdmVyYWdlIHJlc3BvbnNlIGV4cGVjdGVkIG9mIGEgVENQIGZsb3cuICBPbmUN
ICAgZXhhbXBsZSBtZXRob2Qgb2YgYSBUQ1AtZnJpZW5kbHkgc2NoZW1lIGlzIHRoZSBUQ1AtRnJp
ZW5kbHkgUmF0ZQ0gICBDb250cm9sIGFsZ29yaXRobSBbUkZDNTM0OF0uICBJbiB0aGlzIGRvY3Vt
ZW50LCB0aGUgdGVybSBpcyB1c2VkIG1vcmUNICAgZ2VuZXJhbGx5IHRvIGRlc2NyaWJlIHRoaXMg
YW5kIG90aGVyIGFsZ29yaXRobXMgdGhhdCBtZWV0IHRoZXNlDSAgIGdvYWxzLg0NICAgSXQgaXMg
Y29udmVuaWVudCB0byBkaXZpZGUgZmxvd3MgaW50byB0aHJlZSBjbGFzc2VzOiAoMSkgVENQIEZy
aWVuZGx5DSAgIGZsb3dzLCAoMikgdW5yZXNwb25zaXZlIGZsb3dzLCBpLmUuLCBmbG93cyB0aGF0
IGRvIG5vdCBzbG93IGRvd24gd2hlbg0gICBjb25nZXN0aW9uIG9jY3VycywgYW5kICgzKSBmbG93
cyB0aGF0IGFyZSByZXNwb25zaXZlIGJ1dCBhcmUgbm90IFRDUC0NICAgZnJpZW5kbHkuICBUaGUg
bGFzdCB0d28gY2xhc3NlcyBjb250YWluIG1vcmUgYWdncmVzc2l2ZSBmbG93cyB0aGF0DSAgIHBv
c2Ugc2lnbmlmaWNhbnQgdGhyZWF0cyB0byBJbnRlcm5ldCBwZXJmb3JtYW5jZSwgd2hpY2ggd2Ug
d2lsbCBub3cNICAgZGlzY3Vzcy4NDSAgIDEuICBUQ1AtRnJpZW5kbHkgZmxvd3MNDSAgICAgICBB
IFRDUC1mcmllbmRseSBmbG93IHJlc3BvbmRzIHRvIGNvbmdlc3Rpb24gbm90aWZpY2F0aW9uIHdp
dGhpbiBhDSAgICAgICBzbWFsbCBudW1iZXIgb2YgcGF0aCBSb3VuZCBUcmlwIFRpbWVzIChSVFQp
LCBhbmQgaW4gc3RlYWR5LXN0YXRlDSAgICAgICBpdCB1c2VzIG5vIG1vcmUgY2FwYWNpdHkgdGhh
biBhIGNvbmZvcm1hbnQgVENQIHJ1bm5pbmcgdW5kZXINICAgICAgIGNvbXBhcmFibGUgY29uZGl0
aW9ucyAoZHJvcCByYXRlLCBSVFQsIE1UVSwgZXRjLikuICBUaGlzIGlzDSAgICAgICBkZXNjcmli
ZWQgaW4gdGhlIHJlbWFpbmRlciBvZiB0aGUgZG9jdW1lbnQuDQ0gICAyLiAgTm9uLVJlc3BvbnNp
dmUgRmxvd3MNDSAgICAgICBUaGUgVXNlciBEYXRhZ3JhbSBQcm90b2NvbCAoVURQKSBbUkZDMDc2
OF0gcHJvdmlkZXMgYSBtaW5pbWFsLA0gICAgICAgYmVzdC1lZmZvcnQgdHJhbnNwb3J0IHRvIGFw
cGxpY2F0aW9ucyBhbmQgdXBwZXItbGF5ZXIgcHJvdG9jb2xzDSAgICAgICAoYm90aCBzaW1wbHkg
Y2FsbGVkICJhcHBsaWNhdGlvbnMiIGluIHRoZSByZW1haW5kZXIgb2YgdGhpcw0gICAgICAgZG9j
dW1lbnQpIGFuZCBkb2VzIG5vdCBpdHNlbGYgcHJvdmlkZSBtZWNoYW5pc21zIHRvIHByZXZlbnQN
ICAgICAgIGNvbmdlc3Rpb24gY29sbGFwc2UgYW5kIGVzdGFibGlzaCBhIGRlZ3JlZSBvZiBmYWly
bmVzcyBbUkZDNTQwNV0uDQ0gICAgICAgVGhlcmUgaXMgYSBncm93aW5nIHNldCBvZiBVRFAtYmFz
ZWQgYXBwbGljYXRpb25zIHdob3NlIGNvbmdlc3Rpb24NICAgICAgIGF2b2lkYW5jZSBhbGdvcml0
aG1zIGFyZSBpbmFkZXF1YXRlIG9yIG5vbmV4aXN0ZW50IChpLmUsIGEgZmxvdw0gICAgICAgdGhh
dCBkb2VzIG5vdCB0aHJvdHRsZSBpdHMgc2VuZGluZyByYXRlIHdoZW4gaXQgZXhwZXJpZW5jZXMN
ICAgICAgIGNvbmdlc3Rpb24pLiAgRXhhbXBsZXMgaW5jbHVkZSBzb21lIFVEUCBzdHJlYW1pbmcg
YXBwbGljYXRpb25zDSAgICAgICBmb3IgcGFja2V0IHZvaWNlIGFuZCB2aWRlbywgYW5kIHNvbWUg
bXVsdGljYXN0IGJ1bGsgZGF0YQ0gICAgICAgdHJhbnNwb3J0LiAgSWYgbm8gYWN0aW9uIGlzIHRh
a2VuLCBzdWNoIHVucmVzcG9uc2l2ZSBmbG93cyBjb3VsZA0gICAgICAgbGVhZCB0byBhIG5ldyBj
b25nZXN0aXZlIGNvbGxhcHNlIFtSRkMyMzA5XS4NDQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAg
RXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAgICAgICAgICAgIFtQYWdlIDhdDQwNSW50ZXJu
ZXQtRHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAg
SnVseSAyMDEzDQ0NICAgICAgIEluIGdlbmVyYWwsIFVEUC1iYXNlZCBhcHBsaWNhdGlvbnMgbmVl
ZCB0byBpbmNvcnBvcmF0ZSBlZmZlY3RpdmUNICAgICAgIGNvbmdlc3Rpb24gYXZvaWRhbmNlIG1l
Y2hhbmlzbXMgW1JGQzU0MDVdLiAgRnVydGhlciByZXNlYXJjaCBhbmQNICAgICAgIGRldmVsb3Bt
ZW50IG9mIHdheXMgdG8gYWNjb21wbGlzaCBjb25nZXN0aW9uIGF2b2lkYW5jZSBmb3INICAgICAg
IHByZXNlbnRseSB1bnJlc3BvbnNpdmUgYXBwbGljYXRpb25zIGNvbnRpbnVlIHRvIGJlDSAgICAg
ICBpbXBvcnRhbnQuTmV0d29yayBkZXZpY2VzIG5lZWQgdG8gYmUgYWJsZSB0byBwcm90ZWN0IHRo
ZW1zZWx2ZXMNICAgICAgIGFnYWluc3QgdW5yZXNwb25zaXZlIGZsb3dzLCBhbmQgbWVjaGFuaXNt
cyB0byBhY2NvbXBsaXNoIHRoaXMNICAgICAgIG11c3QgYmUgZGV2ZWxvcGVkIGFuZCBkZXBsb3ll
ZC4gIERlcGxveW1lbnQgb2Ygc3VjaCBtZWNoYW5pc21zDSAgICAgICB3b3VsZCBwcm92aWRlIGFu
IGluY2VudGl2ZSBmb3IgYWxsIGFwcGxpY2F0aW9ucyB0byBiZWNvbWUNICAgICAgIHJlc3BvbnNp
dmUgYnkgZWl0aGVyIHVzaW5nIGEgY29uZ2VzdGlvbi1jb250cm9sbGVkIHRyYW5zcG9ydA0gICAg
ICAgKGUuZy4gVENQLCBTQ1RQLCBEQ0NQKSBvciBieSBpbmNvcnBvcmF0aW5nIHRoZWlyIG93biBj
b25nZXN0aW9uDSAgICAgICBjb250cm9sIGluIHRoZSBhcHBsaWNhdGlvbi4gIFtSRkM1NDA1XS4N
DSAgIDMuICBOb24tVENQLWZyaWVuZGx5IFRyYW5zcG9ydCBQcm90b2NvbHMNDSAgICAgICBBIHNl
Y29uZCB0aHJlYXQgaXMgcG9zZWQgYnkgdHJhbnNwb3J0IHByb3RvY29sIGltcGxlbWVudGF0aW9u
cw0gICAgICAgdGhhdCBhcmUgcmVzcG9uc2l2ZSB0byBjb25nZXN0aW9uLCBidXQsIGVpdGhlciBk
ZWxpYmVyYXRlbHkgb3INICAgICAgIHRocm91Z2ggZmF1bHR5IGltcGxlbWVudGF0aW9uLCBhcmUg
bm90IFRDUC1mcmllbmRseS4gIFN1Y2gNICAgICAgIGFwcGxpY2F0aW9ucyBtYXkgZ2FpbiBhbiB1
bmZhaXIgc2hhcmUgb2YgdGhlIGF2YWlsYWJsZSBuZXR3b3JrDSAgICAgICBjYXBhY2l0eS4NDSAg
ICAgICBGb3IgZXhhbXBsZSwgdGhlIHBvcHVsYXJpdHkgb2YgdGhlIEludGVybmV0IGhhcyBjYXVz
ZWQgYQ0gICAgICAgcHJvbGlmZXJhdGlvbiBpbiB0aGUgbnVtYmVyIG9mIFRDUCBpbXBsZW1lbnRh
dGlvbnMuICBTb21lIG9mDSAgICAgICB0aGVzZSBtYXkgZmFpbCB0byBpbXBsZW1lbnQgdGhlIFRD
UCBjb25nZXN0aW9uIGF2b2lkYW5jZQ0gICAgICAgbWVjaGFuaXNtcyBjb3JyZWN0bHkgYmVjYXVz
ZSBvZiBwb29yIGltcGxlbWVudGF0aW9uLiAgT3RoZXJzIG1heQ0gICAgICAgZGVsaWJlcmF0ZWx5
IGJlIGltcGxlbWVudGVkIHdpdGggY29uZ2VzdGlvbiBhdm9pZGFuY2UgYWxnb3JpdGhtcw0gICAg
ICAgdGhhdCBhcmUgbW9yZSBhZ2dyZXNzaXZlIGluIHRoZWlyIHVzZSBvZiBjYXBhY2l0eSB0aGFu
IG90aGVyIFRDUA0gICAgICAgaW1wbGVtZW50YXRpb25zOyB0aGlzIHdvdWxkIGFsbG93IGEgdmVu
ZG9yIHRvIGNsYWltIHRvIGhhdmUgYQ0gICAgICAgImZhc3RlciBUQ1AiLiAgVGhlIGxvZ2ljYWwg
Y29uc2VxdWVuY2Ugb2Ygc3VjaCBpbXBsZW1lbnRhdGlvbnMNICAgICAgIHdvdWxkIGJlIGEgc3Bp
cmFsIG9mIGluY3JlYXNpbmdseSBhZ2dyZXNzaXZlIFRDUCBpbXBsZW1lbnRhdGlvbnMsDSAgICAg
ICBsZWFkaW5nIGJhY2sgdG8gdGhlIHBvaW50IHdoZXJlIHRoZXJlIGlzIGVmZmVjdGl2ZWx5IG5v
DSAgICAgICBjb25nZXN0aW9uIGF2b2lkYW5jZSBhbmQgdGhlIEludGVybmV0IGlzIGNocm9uaWNh
bGx5IGNvbmdlc3RlZC4NDSAgICAgICBBbm90aGVyIGV4YW1wbGUgY291bGQgYmUgYW4gUlRQL1VE
UCB2aWRlbyBmbG93IHRoYXQgdXNlcyBhbg0gICAgICAgYWRhcHRpdmUgY29kZWMsIGJ1dCByZXNw
b25kcyBpbmNvbXBsZXRlbHkgdG8gaW5kaWNhdGlvbnMgb2YNICAgICAgIGNvbmdlc3Rpb24gb3Ig
b3ZlciByZXNwb25kcyBvdmVyIGFuIGV4Y2Vzc2l2ZWx5IGxvbmcgdGltZSBwZXJpb2QuDSAgICAg
ICBTdWNoIGZsb3dzIGFyZSB1bmxpa2VseSB0byBiZSByZXNwb25zaXZlIHRvIGNvbmdlc3Rpb24g
c2lnbmFscyBpbg0gICAgICAgYSB0aW1lIGZyYW1lIGNvbXBhcmFibGUgdG8gYSBzbWFsbCBudW1i
ZXIgb2YgZW5kLXRvLWVuZA0gICAgICAgdHJhbnNtaXNzaW9uIGRlbGF5cy4gIEhvd2V2ZXIsIG92
ZXIgYSBsb25nZXIgdGltZXNjYWxlLCBwZXJoYXBzDSAgICAgICBzZWNvbmRzIGluIGR1cmF0aW9u
LCB0aGV5IGNvdWxkIG1vZGVyYXRlIHRoZWlyIHNwZWVkLCBvciBpbmNyZWFzZQ0gICAgICAgdGhl
aXIgc3BlZWQgaWYgdGhleSBkZXRlcm1pbmUgY2FwYWNpdHkgdG8gYmUgYXZhaWxhYmxlLg0NICAg
ICAgIFR1bm5lbGVkIHRyYWZmaWMgYWdncmVnYXRlcyBvZiBtdWx0aXBsZSAoc2hvcnQpIFRDUCBm
bG93cyBjYW4gYmUNICAgICAgIG1vcmUgYWdncmVzc2l2ZSB0aGFuIHN0YW5kYXJkIGJ1bGsgVENQ
LiAgQXBwbGljYXRpb25zIChlLmcuIHdlYg0gICAgICAgYnJvd3NlcnMgYW5kIHBlZXItdG8tcGVl
ciBmaWxlLXNoYXJpbmcpIGhhdmUgZXhwbG9pdGVkIHRoaXMgYnkNICAgICAgIG9wZW5pbmcgbXVs
dGlwbGUgY29ubmVjdGlvbnMgdG8gdGhlIHNhbWUgZW5kcG9pbnQuDQ0gICBUaGUgcHJvamVjdGVk
IGluY3JlYXNlIGluIHRoZSBmcmFjdGlvbiBvZiB0b3RhbCBJbnRlcm5ldCB0cmFmZmljIGZvcg0g
ICBtb3JlIGFnZ3Jlc3NpdmUgZmxvd3MgaW4gY2xhc3NlcyAyIGFuZCAzIGNsZWFybHkgcG9zZXMg
YSB0aHJlYXQgdG8NDQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEw
LCAyMDE0ICAgICAgICAgICAgICAgIFtQYWdlIDldDQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUg
UXVldWUgTWFuYWdlbWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAgSnVseSAyMDEzDQ0NICAgZnV0
dXJlIEludGVybmV0IHN0YWJpbGl0eS4gIFRoZXJlIGlzIGFuIHVyZ2VudCBuZWVkIGZvciBtZWFz
dXJlbWVudHMNICAgb2YgY3VycmVudCBjb25kaXRpb25zIGFuZCBmb3IgZnVydGhlciByZXNlYXJj
aCBpbnRvIHRoZSB3YXlzIG9mDSAgIG1hbmFnaW5nIHN1Y2ggZmxvd3MuICBUaGlzIHJhaXNlcyBt
YW55IGRpZmZpY3VsdCBpc3N1ZXMgaW4NICAgaWRlbnRpZnlpbmcgYW5kIGlzb2xhdGluZyB1bnJl
c3BvbnNpdmUgb3Igbm9uLVRDUC1mcmllbmRseSBmbG93cyBhdA0gICBhbiBhY2NlcHRhYmxlIG92
ZXJoZWFkIGNvc3QuICBGaW5hbGx5LCB0aGVyZSBpcyBhcyB5ZXQgbGl0dGxlDSAgIG1lYXN1cmVt
ZW50IG9yIHNpbXVsYXRpb24gZXZpZGVuY2UgYXZhaWxhYmxlIGFib3V0IHRoZSByYXRlIGF0IHdo
aWNoDSAgIHRoZXNlIHRocmVhdHMgYXJlIGxpa2VseSB0byBiZSByZWFsaXplZCwgb3IgYWJvdXQg
dGhlIGV4cGVjdGVkDSAgIGJlbmVmaXQgb2YgYWxnb3JpdGhtcyBmb3IgbWFuYWdpbmcgc3VjaCBm
bG93cy4NDSAgIEFub3RoZXIgdG9waWMgcmVxdWlyaW5nIGNvbnNpZGVyYXRpb24gaXMgdGhlIGFw
cHJvcHJpYXRlIGdyYW51bGFyaXR5DSAgIG9mIGEgImZsb3ciIHdoZW4gY29uc2lkZXJpbmcgYSBx
dWV1ZSBtYW5hZ2VtZW50IG1ldGhvZC4gIFRoZXJlIGFyZSBhDSAgIGZldyAibmF0dXJhbCIgYW5z
d2VyczogMSkgYSB0cmFuc3BvcnQgKGUuZy4gVENQIG9yIFVEUCkgZmxvdyAoc291cmNlDSAgIGFk
ZHJlc3MvcG9ydCwgZGVzdGluYXRpb24gYWRkcmVzcy9wb3J0LCBEU0NQKTsgMikgYSBzb3VyY2Uv
DSAgIGRlc3RpbmF0aW9uIGhvc3QgcGFpciAoSVAgYWRkcmVzc2VzLCBEU0NQKTsgMykgYSBnaXZl
biBzb3VyY2UgaG9zdCBvcg0gICBhIGdpdmVuIGRlc3RpbmF0aW9uIGhvc3QuICBXZSBzdWdnZXN0
IHRoYXQgdGhlIHNvdXJjZS9kZXN0aW5hdGlvbg0gICBob3N0IHBhaXIgZ2l2ZXMgdGhlIG1vc3Qg
YXBwcm9wcmlhdGUgZ3JhbnVsYXJpdHkgaW4gbWFueQ0gICBjaXJjdW1zdGFuY2VzLiAgSG93ZXZl
ciwgaXQgaXMgcG9zc2libGUgdGhhdCBkaWZmZXJlbnQgdmVuZG9ycy8NICAgcHJvdmlkZXJzIGNv
dWxkIHNldCBkaWZmZXJlbnQgZ3JhbnVsYXJpdGllcyBmb3IgZGVmaW5pbmcgYSBmbG93IChhcyBh
DSAgIHdheSBvZiAiZGlzdGluZ3Vpc2hpbmciIHRoZW1zZWx2ZXMgZnJvbSBvbmUgYW5vdGhlciks
IG9yIHRoYXQNICAgZGlmZmVyZW50IGdyYW51bGFyaXRpZXMgY291bGQgYmUgY2hvc2VuIGZvciBk
aWZmZXJlbnQgcGxhY2VzIGluIHRoZQ0gICBuZXR3b3JrLiAgSXQgbWF5IGJlIHRoZSBjYXNlIHRo
YXQgdGhlIGdyYW51bGFyaXR5IGlzIGxlc3MgaW1wb3J0YW50DSAgIHRoYW4gdGhlIGZhY3QgdGhh
dCBhIG5ldHdvcmsgZGV2aWNlIG5lZWRzIHRvIGJlIGFibGUgdG8gZGVhbCB3aXRoDSAgIG1vcmUg
dW5yZXNwb25zaXZlIGZsb3dzIGF0ICpzb21lKiBncmFudWxhcml0eS4gIFRoZSBncmFudWxhcml0
eSBvZg0gICBmbG93cyBmb3IgY29uZ2VzdGlvbiBtYW5hZ2VtZW50IGlzLCBhdCBsZWFzdCBpbiBw
YXJ0LCBhIHF1ZXN0aW9uIG9mDSAgIHBvbGljeSB0aGF0IG5lZWRzIHRvIGJlIGFkZHJlc3NlZCBp
biB0aGUgd2lkZXIgSUVURiBjb21tdW5pdHkuDQ00LiAgQ29uY2x1c2lvbnMgYW5kIFJlY29tbWVu
ZGF0aW9ucw0NICAgVGhlIElSVEYsIGluIHB1Ymxpc2hpbmcgW1JGQzIzMDldLCBhbmQgdGhlIElF
VEYgaW4gc3Vic2VxdWVudA0gICBkaXNjdXNzaW9uLCBoYXMgZGV2ZWxvcGVkIGEgc2V0IG9mIHNw
ZWNpZmljIHJlY29tbWVuZGF0aW9ucyByZWdhcmRpbmcNICAgdGhlIGltcGxlbWVudGF0aW9uIGFu
ZCBvcGVyYXRpb25hbCB1c2Ugb2YgQVFNIHByb2NlZHVyZXMuICBUaGlzDSAgIGRvY3VtZW50IHVw
ZGF0ZXMgdGhlc2UgdG8gaW5jbHVkZToNDSAgIDEuICBOZXR3b3JrIGRldmljZXMgU0hPVUxEIGlt
cGxlbWVudCBzb21lIEFRTSBtZWNoYW5pc20gdG8gbWFuYWdlDSAgICAgICBxdWV1ZSBsZW5ndGhz
LCByZWR1Y2UgZW5kLXRvLWVuZCBsYXRlbmN5LCBhbmQgYXZvaWQgbG9jay1vdXQNICAgICAgIHBo
ZW5vbWVuYSB3aXRoaW4gdGhlIEludGVybmV0Lg0NICAgMi4gIERlcGxveWVkIEFRTSBhbGdvcml0
aG1zIFNIT1VMRCBzdXBwb3J0IEV4cGxpY2l0IENvbmdlc3Rpb24NICAgICAgIE5vdGlmaWNhdGlv
biAoRUNOKSBhcyB3ZWxsIGFzIGxvc3MgdG8gc2lnbmFsIGNvbmdlc3Rpb24gdG8NICAgICAgIGVu
ZHBvaW50cy4NDSAgIDMuICBUaGUgYWxnb3JpdGhtcyB0aGF0IHRoZSBJRVRGIHJlY29tbWVuZHMg
U0hPVUxEIE5PVCByZXF1aXJlDSAgICAgICBvcGVyYXRpb25hbCAoZXNwZWNpYWxseSBtYW51YWwp
IGNvbmZpZ3VyYXRpb24gb3IgdHVuaW5nLg1Tb21lIHR1bmluZyBtYXkgYmUgcmVxdWlyZWQgb3Ig
aW1wbGljaXRseSBhc3N1bWVkIGZvciB2aXJ0dWFsbHkgYWxsIEFRTXMgliBwbGVhc2Ugc2VlIG15
IGNvbW1lbnQgbGF0ZXIuIA0NICAgNC4gIEFRTSBhbGdvcml0aG1zIFNIT1VMRCByZXNwb25kIHRv
IG1lYXN1cmVkIGNvbmdlc3Rpb24sIG5vdA0gICAgICAgYXBwbGljYXRpb24gcHJvZmlsZXMuDQ0N
DQ0NQmFrZXIgJiBGYWlyaHVyc3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAg
ICAgICAgICAgW1BhZ2UgMTBdDQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdl
bWVudCBSZWNvbW1lbmRhdGlvbnMgICAgICAgSnVseSAyMDEzDQ0NICAgNS4gIEFRTSBhbGdvcml0
aG1zIFNIT1VMRCBOT1QgaW50ZXJwcmV0IHNwZWNpZmljIHRyYW5zcG9ydCBwcm90b2NvbA0gICAg
ICAgYmVoYXZpb3Vycy4NDSAgIDYuICBUcmFuc3BvcnQgcHJvdG9jb2wgY29uZ2VzdGlvbiBjb250
cm9sIGFsZ29yaXRobXMgU0hPVUxEIG1heGltaXplDSAgICAgICB0aGVpciB1c2Ugb2YgYXZhaWxh
YmxlIGNhcGFjaXR5ICh3aGVuIHRoZXJlIGlzIGRhdGEgdG8gc2VuZCkNICAgICAgIHdpdGhvdXQg
aW5jdXJyaW5nIHVuZHVlIGxvc3Mgb3IgdW5kdWUgcm91bmQgdHJpcCBkZWxheS4NDSAgIDcuICBS
ZXNlYXJjaCwgZW5naW5lZXJpbmcsIGFuZCBtZWFzdXJlbWVudCBlZmZvcnRzIGFyZSBuZWVkZWQN
ICAgICAgIHJlZ2FyZGluZyB0aGUgZGVzaWduIG9mIG1lY2hhbmlzbXMgdG8gZGVhbCB3aXRoIGZs
b3dzIHRoYXQgYXJlDSAgICAgICB1bnJlc3BvbnNpdmUgdG8gY29uZ2VzdGlvbiBub3RpZmljYXRp
b24gb3IgYXJlIHJlc3BvbnNpdmUsIGJ1dA0gICAgICAgYXJlIG1vcmUgYWdncmVzc2l2ZSB0aGFu
IHByZXNlbnQgVENQLg0NCURvIHdlIHdhbnQgdG8gbWFrZSBhIHN1Z2dlc3Rpb24gb24gaG93IHRv
IGNvbmZpZ3VyZSBidWZmZXIgc2l6ZXMgd2l0aCBlYWNoIHR5cGUgb2YgQVFNIGhlcmUgKGUuZy4g
MnhCRFAgZXRjKSBvciBzaW1wbHkgc3RhdGUgdGhhdCByZXNlYXJjaCBzaG91bGQgYmUgY29uZHVj
dGVkIG9uIHRoZSBiZXN0IGJ1ZmZlciBzaXplcyB0byB1c2Ugd2l0aCBBUU0uDQ0gICBUaGVzZSBy
ZWNvbW1lbmRhdGlvbnMgYXJlIGV4cHJlc3NlZCB1c2luZyB0aGUgd29yZCAiU0hPVUxEIi4gIFRo
aXMgaXMNICAgaW4gcmVjb2duaXRpb24gdGhhdCB0aGVyZSBtYXkgYmUgdXNlIGNhc2VzIHRoYXQg
aGF2ZSBub3QgYmVlbg0gICBlbnZpc2FnZWQgaW4gdGhpcyBkb2N1bWVudCBpbiB3aGljaCB0aGUg
cmVjb21tZW5kYXRpb24gZG9lcyBub3QNICAgYXBwbHkuICBIb3dldmVyLCBjYXJlIHNob3VsZCBi
ZSB0YWtlbiBpbiBjb25jbHVkaW5nIHRoYXQgb25lJ3MgdXNlDSAgIGNhc2UgZmFsbHMgaW4gdGhh
dCBjYXRlZ29yeTsgZHVyaW5nIHRoZSBsaWZlIG9mIHRoZSBJbnRlcm5ldCwgc3VjaA0gICB1c2Ug
Y2FzZXMgaGF2ZSBiZWVuIHJhcmVseSBpZiBldmVyIG9ic2VydmVkIGFuZCByZXBvcnRlZCBvbi4g
IFRvIHRoZQ0gICBjb250cmFyeSwgYXZhaWxhYmxlIHJlc2VhcmNoIFtQYXBhZ2lhbm5ha2ldIHNh
eXMgdGhhdCBldmVuIGhpZ2ggc3BlZWQNICAgbGlua3MgaW4gbmV0d29yayBjb3JlcyB0aGF0IGFy
ZSBub3JtYWxseSB2ZXJ5IHN0YWJsZSBpbiBkZXB0aCBhbmQNICAgYmVoYXZpb3IgZXhwZXJpZW5j
ZSBvY2Nhc2lvbmFsIGlzc3VlcyB0aGF0IG5lZWQgbW9kZXJhdGlvbi4NDTQuMS4gIE9wZXJhdGlv
bmFsIGRlcGxveW1lbnRzIFNIT1VMRCB1c2UgQVFNIHByb2NlZHVyZXMNDSAgIEluIHNob3J0LCBB
UU0gcHJvY2VkdXJlcyBhcmUgZGVzaWduZWQgdG8gbWluaW1pemUgZGVsYXkgaW5kdWNlZCBpbg0g
ICB0aGUgbmV0d29yayBieSBxdWV1ZXMgdGhhdCBoYXZlIGZpbGxlZCBhcyBhIHJlc3VsdCBvZiBo
b3N0IGJlaGF2aW9yLg0gICBNYXJraW5nIGFuZCBsb3NzIGJlaGF2aW9ycyBwcm92aWRlIGEgc2ln
bmFsIHRoYXQgYnVmZmVycyBpbiBuZXR3b3JrDSAgIGRldmljZXMgYXJlIGJlY29taW5nIHVubmVj
ZXNzYXJpbHkgZnVsbCwgYW5kIHRoYXQgdGhlIHNlbmRlciB3b3VsZCBkbw0gICB3ZWxsIHRvIG1v
ZGVyYXRlIGl0cyBiZWhhdmlvci4NDTQuMi4gIFNpZ25hbGluZyB0byB0aGUgdHJhbnNwb3J0IGVu
ZHBvaW50cw0NICAgVGhlcmUgYXJlIGEgbnVtYmVyIG9mIHdheXMgYSBuZXR3b3JrIGRldmljZSBt
YXkgc2lnbmFsIHRvIHRoZSBlbmQNICAgcG9pbnQgdGhhdCB0aGUgbmV0d29yayBpcyBiZWNvbWlu
ZyBjb25nZXN0ZWQgYW5kIHRyaWdnZXIgYSByZWR1Y3Rpb24NICAgaW4gcmF0ZS4gIFRoZSBzaWdu
YWxsaW5nIG1ldGhvZHMgaW5jbHVkZToNDSAgIG8gIERlbGF5aW5nIGRhdGEgc2VnbWVudHMgaW4g
ZmxpZ2h0LCBzdWNoIGFzIGluIGEgcXVldWUuDQ0gICBvICBEcm9wcGluZyB0cmFmZmljIGluIHRy
YW5zaXQuDQ0gICBvICBNYXJraW5nIHRyYWZmaWMsIHN1Y2ggYXMgdXNpbmcgRXhwbGljaXQgQ29u
Z2VzdGlvbg0gICAgICBDb250cm9sW1JGQzMxNjhdIFtSRkM0MzAxXSBbUkZDNDc3NF0gW1JGQzYw
NDBdIFtSRkM2Njc5XS4NDSAgIFRoZSB1c2Ugb2Ygc2NoZWR1bGluZyBtZWNoYW5pc21zLCBzdWNo
IGFzIHByaW9yaXR5IHF1ZXVpbmcsIGNsYXNzZnVsDSAgIHF1ZXVpbmcsIGFuZCBmYWlyIHF1ZXVp
bmcsIGlzIG9mdGVuIGVmZmVjdGl2ZSBpbiBuZXR3b3JrcyB0byBoZWxwIGENICAgbmV0d29yayBz
ZXJ2ZSB0aGUgbmVlZHMgb2YgYSByYW5nZSBvZiBhcHBsaWNhdGlvbnMuICBOZXR3b3JrDSAgIG9w
ZXJhdG9ycyBjYW4gdXNlIHRoZXNlIG1ldGhvZHMgdG8gbWFuYWdlIHRyYWZmaWMgcGFzc2luZyBh
IGNob2tlDSAgIHBvaW50LiAgVGhpcyBpcyBkaXNjdXNzZWQgaW4gW1JGQzI0NzRdIGFuZCBbUkZD
MjQ3NV0uDQ0NDUJha2VyICYgRmFpcmh1cnN0ICAgICAgIEV4cGlyZXMgSmFudWFyeSAxMCwgMjAx
NCAgICAgICAgICAgICAgIFtQYWdlIDExXQ0MDUludGVybmV0LURyYWZ0ICAgQWN0aXZlIFF1ZXVl
IE1hbmFnZW1lbnQgUmVjb21tZW5kYXRpb25zICAgICAgIEp1bHkgMjAxMw0NDSAgIEluY3JlYXNl
ZCBuZXR3b3JrIGxhdGVuY3kgY2FuIGJlIHVzZWQgYXMgYW4gaW1wbGljaXQgc2lnbmFsIG9mDSAg
IGNvbmdlc3Rpb24uICBFLmcuLCBpbiBUQ1AgYWRkaXRpb25hbCBkZWxheSBjYW4gYWZmZWN0IEFD
SyBDbG9ja2luZw0gICBhbmQgaGFzIHRoZSByZXN1bHQgb2YgcmVkdWNpbmcgdGhlIHJhdGUgb2Yg
dHJhbnNtaXNzaW9uIG9mIG5ldyBkYXRhLg0gICBJbiBSVFAsIHRoaXMgaW1wYWN0cyB0aGUgUlRD
UC1yZXBvcnRlZCBSVFQgYW5kIGNhbiB0cmlnZ2VyIGEgc2VuZGVyDSAgIHRvIGFkanVzdCBpdHMg
cmF0ZS4gIEZvciBleGFtcGxlLCBMRURCQVQgW1JGQzY4MTddIGFzc3VtZXMgZGVsYXkgYXMgYQ0g
ICBwcmltYXJ5IHNpZ25hbCBvZiBjb25nZXN0aW9uLg0NICAgSXQgaXMgZXNzZW50aWFsIHRoYXQg
YWxsIEludGVybmV0IGhvc3RzIHJlc3BvbmQgdG8gbG9zcyBbUkZDNTY4MV0sDSAgIFtSRkM1NDA1
XVtSRkMyOTYwXVtSRkM0MzQwXS4gIFBhY2tldCBkcm9wcGluZyBieSBuZXR3b3JrIGRldmljZXMg
dGhhdA0gICBhcmUgdW5kZXIgbG9hZCBoYXMgdHdvIGVmZmVjdHM6IEl0IHByb3RlY3RzIHRoZSBu
ZXR3b3JrLCB3aGljaCBpcyB0aGUNICAgcHJpbWFyeSByZWFzb24gdGhhdCBuZXR3b3JrIGRldmlj
ZXMgZHJvcCBwYWNrZXRzLiAgVGhlIGRldGVjdGlvbiBvZg0gICBsb3NzIGFsc28gcHJvdmlkZXMg
YSBzaWduYWwgdG8gYSByZWxpYWJsZSB0cmFuc3BvcnQgKGUuZy4gVENQLCBTQ1RQKQ0gICB0aGF0
IHRoZXJlIGlzIHBvdGVudGlhbCBjb25nZXN0aW9uIHVzaW5nIGEgcHJhZ21hdGljIGhldXJpc3Rp
YzsgIndoZW4NICAgdGhlIG5ldHdvcmsgZGlzY2FyZHMgYSBtZXNzYWdlIGluIGZsaWdodCwgaXQg
bWF5IGltcGx5IHRoZSBwcmVzZW5jZQ0gICBvZiBmYXVsdHkgZXF1aXBtZW50IG9yIG1lZGlhIGlu
IGEgcGF0aCwgYW5kIGl0IG1heSBpbXBseSB0aGUgcHJlc2VuY2UNICAgb2YgY29uZ2VzdGlvbi4g
IFRvIGJlIGNvbnNlcnZhdGl2ZSB0cmFuc3BvcnQgbXVzdCB0aGUgbGF0dGVyLiINICAgVW5yZWxp
YWJsZSB0cmFuc3BvcnRzIChlLmcuIHVzaW5nIFVEUCkgbmVlZCB0byBzaW1pbGFybHkgcmVhY3Qg
dG8NICAgbG9zcyBbUkZDNTQwNV0NDSAgIE5ldHdvcmsgZGV2aWNlcyBTSE9VTEQgdXNlIHVzZSBh
biBBUU0gYWxnb3JpdGhtIHRvIGRldGVybWluZSB3aGljaA0gICBwYWNrZXRzIGFyZSBlZmZlY3Rl
ZCBieSBjb25nZXN0aW9uLg0NICAgTG9zcyBhbHNvIGhhcyBhbiBlZmZlY3Qgb24gdGhlIGVmZmlj
aWVuY3kgb2YgYSBmbG93IGFuZCBjYW4NICAgc2lnbmlmaWNhbnRseSBpbXBhY3Qgc29tZSBjbGFz
c2VzIG9mIGFwcGxpY2F0aW9uLiAgSW4gcmVsaWFibGUNICAgdHJhbnNwb3J0cyB0aGUgZHJvcHBl
ZCBkYXRhIG11c3QgYmUgcmV0cmFuc21pdHRlZC4gIFdoaWxlIG90aGVyDSAgIGFwcGxpY2F0aW9u
cy90cmFuc3BvcnRzIG1heSBhZGFwdCB0byB0aGUgYWJzZW5jZSBvZiB0aGUgZGF0YSwgdGhpcw0g
ICBzdGlsbCBpbXBsaWVzIGluZWZmaWNpZW50IHVzZSBvZiBhdmFpbGFibGUgY2FwYWNpdHkgYW5k
IHRoZSBkcm9wcGVkDSAgIHRyYWZmaWMgY2FuIGFmZmVjdCBvdGhlciBmbG93cy4gIEhlbmNlLCBs
b3NzIGlzIG5vdCBlbnRpcmVseQ0gICBwb3NpdGl2ZTsgaXQgaXMgYSBuZWNlc3NhcnkgZXZpbC4N
DTQuMi4xLiAgQVFNIGFuZCBFQ04NDSAgIEV4cGxpY2l0IENvbmdlc3Rpb24gTm90aWZpY2F0aW9u
IChFQ04pIFtSRkM0MzAxXSBbUkZDNDc3NF0gW1JGQzYwNDBdDSAgIFtSRkM2Njc5XS4gaXMgYSBu
ZXR3b3JrLWxheWVyIGZ1bmN0aW9uIHRoYXQgYWxsb3dzIGEgdHJhbnNwb3J0IHRvDSAgIHJlY2Vp
dmUgbmV0d29yayBjb25nZXN0aW9uIGluZm9ybWF0aW9uIGZyb20gYSBuZXR3b3JrIGRldmljZSB3
aXRob3V0DSAgIGluY3VycmluZyB0aGUgdW5pbnRlbmRlZCBjb25zZXF1ZW5jZXMgb2YgbG9zcy4g
IEVDTiBpbmNsdWRlcyBib3RoDSAgIHRyYW5zcG9ydCBhbmQgZnVuY3Rpb25zIGltcGxlbWVudGVk
IGluIG5ldHdvcmsgZGV2aWNlcywgdGhlIGxhdHRlcg0gICByZWx5IHVwb24gdXNpbmcgQVFNLg0N
ICAgQ29uZ2VzdGlvbiBmb3IgRUNOLWNhcGFibGUgdHJhbnNwb3J0cyBpcyBpbnN0ZWFkIHNpZ25h
bGxlZCBieSBhDSAgIG5ldHdvcmsgZGV2aWNlIHNldHRpbmcgdGhlICJDb25nZXN0aW9uIEV4cGVy
aWVuY2VkIChDRSkiIGNvZGVwb2ludCBpbg0gICB0aGUgSVAgaGVhZGVyLiAgVGhpcyBjb2RlcG9p
bnQgaXMgbm90ZWQgYnkgdGhlIHJlbW90ZSByZWNlaXZpbmcgZW5kDSAgIHBvaW50IGFuZCBzaWdu
YWxsZWQgYmFjayB0byB0aGUgc2VuZGVyIHVzaW5nIGEgdHJhbnNwb3J0IHByb3RvY29sDSAgIG1l
Y2hhbmlzbSwgYWxsb3dpbmcgdGhlIHNlbmRlciB0byB0cmlnZ2VyIHRpbWVseSBjb25nZXN0aW9u
IGNvbnRyb2wuDSAgIFRoZSBkZWNpc2lvbiB0byBzZXQgdGhlIENFIGNvZGVwb2ludCByZXF1aXJl
cyBhbiBBUU0gYWxnb3JpdGhtDSAgIGNvbmZpZ3VyZWQgd2l0aCBhIHRocmVzaG9sZC4gIE5vbi1F
Q04gY2FwYWJsZSBmbG93cyAodGhlIGRlZmF1bHQpIGFyZQ0gICBkcm9wcGVkIHVuZGVyIGNvbmdl
c3Rpb24uDQ0NDQ1CYWtlciAmIEZhaXJodXJzdCAgICAgICBFeHBpcmVzIEphbnVhcnkgMTAsIDIw
MTQgICAgICAgICAgICAgICBbUGFnZSAxMl0NDA1JbnRlcm5ldC1EcmFmdCAgIEFjdGl2ZSBRdWV1
ZSBNYW5hZ2VtZW50IFJlY29tbWVuZGF0aW9ucyAgICAgICBKdWx5IDIwMTMNDQ0gICBOZXR3b3Jr
IGRldmljZXMgU0hPVUxEIHVzZSBhbiBBUU0gYWxnb3JpdGhtIHRoYXQgbWFya3MgRUNOLWNhcGFi
bGUNICAgdHJhZmZpYyB3aGVuIG1ha2luZyBkZWNpc2lvbnMgYWJvdXQgdGhlIHJlc3BvbnNlIHRv
IGNvbmdlc3Rpb24uDSAgIE5ldHdvcmsgZGV2aWNlcyBuZWVkIHRvIGltcGxlbWVudCB0aGlzIG1l
dGhvZCBieSBtYXJraW5nIEVDTi1jYXBhYmxlDSAgIHRyYWZmaWMgb3IgYnkgZHJvcHBpbmcgbm9u
LUVDTi1jYXBhYmxlIHRyYWZmaWMuDQ0gICBTYWZlIGRlcGxveW1lbnQgb2YgRUNOIHJlcXVpcmVz
IHRoYXQgbmV0d29yayBkZXZpY2VzIGRyb3AgZXhjZXNzaXZlDSAgIHRyYWZmaWMsIGV2ZW4gd2hl
biBtYXJrZWQgYXMgb3JpZ2luYXRpbmcgZnJvbSBhbiBFQ04gY2FwYWJsZQ0gICB0cmFuc3BvcnQu
ICBUaGlzIGlzIG5lY2Vzc2FyeSBiZWNhdXNlICgxKSBBIG5vbi1jb25mb3JtYW50LCBicm9rZW4g
b3INICAgbWFsaWNpb3VzIHJlY2VpdmVyIGNvdWxkIGNvbmNlYWwgYW4gRUNOIG1hcmssIGFuZCBu
b3QgcmVwb3J0IHRoaXMgdG8NICAgdGhlIHNlbmRlciAoMikgQSBub24tY29uZm9ybWFudCwgYnJv
a2VuIG9yIG1hbGljaW91cyBzZW5kZXIgY291bGQNICAgaWdub3JlIGEgcmVwb3J0ZWQgRUNOIG1h
cmssIGFzIGl0IGNvdWxkIGlnbm9yZSBhIGxvc3Mgd2l0aG91dCB1c2luZw0gICBFQ04gKDMpIEEg
bWFsZnVuY3Rpb25pbmcgb3Igbm9uLWNvbmZvcm1pbmcgbmV0d29yayBkZXZpY2VzIG1heQ0gICBz
aW1pbGFybHkgImhpZGUiIGFuZCBFQ04gbWFyay4gIEluIG5vcm1hbCBvcGVyYXRpb24gc3VjaCBj
YXNlcyBzaG91bGQNICAgYmUgdmVyeSB1bmNvbW1vbi4NDSAgIE5ldHdvcmsgZGV2aWNlcyBTSE9V
TEQgdXNlIGFuIGFsZ29yaXRobSB0byBkcm9wIGV4Y2Vzc2l2ZSB0cmFmZmljLA0gICBldmVuIHdo
ZW4gbWFya2VkIGFzIG9yaWdpbmF0aW5nIGZyb20gYW4gRUNOIGNhcGFibGUgdHJhbnNwb3J0Lg0N
NC4zLiAgQVFNIGFsZ29yaXRobXMgZGVwbG95ZWQgU0hPVUxEIE5PVCByZXF1aXJlIG9wZXJhdGlv
bmFsIHR1bmluZw0NICAgQSBudW1iZXIgb2YgYWxnb3JpdGhtcyBoYXZlIGJlZW4gcHJvcG9zZWQu
ICBNYW55IHJlcXVpcmUgc29tZSBmb3JtIG9mDSAgIHR1bmluZyBvciBpbml0aWFsIGNvbmRpdGlv
bi4gIFRoaXMgY2FuIG1ha2UgdGhlbSBkaWZmaWN1bHQgdG8gdXNlDSAgIG9wZXJhdGlvbmFsbHku
ICBIZW5jZSwgc2VsZi10dW5pbmcgYWxnb3JpdGhtcyBhcmUgdG8gYmUgcHJlZmVycmVkLg0gICBU
aGUgYWxnb3JpdGhtcyB0aGF0IHRoZSBJRVRGIHJlY29tbWVuZHMgU0hPVUxEIE5PVCByZXF1aXJl
DSAgIG9wZXJhdGlvbmFsIChlc3BlY2lhbGx5IG1hbnVhbCkgY29uZmlndXJhdGlvbiBvciB0dW5p
bmcuDQ1NYXkgYmUgYmV0dGVyIHRvIHN0YXRlIHRoYXQgdHVuaW5nIHNob3VsZCBiZSBtaW5pbWl6
ZWQuIEZvciB0aGUgc2Vjb25kIHNlbnRlbmNlIJNUaGUgYWxnb3JpdGhtcyB0aGF0IHRoZSBJRVRG
IHJlY29tbWVuZHMgc2hvdWxkIG1pbmltaXplIHR1bmluZyBvciBjb25maWd1cmF0aW9ucyBjaGFu
Z2VzIGZvciBzcGVjaWZpYyB0cmFmZmljIG9yIG5ldHdvcmsgY29uZGl0aW9uc5QNDUkgd291bGQg
YXJndWUgdGhhdCBhbGwgQVFNcyB3aWxsIGxpa2VseSByZXF1aXJlIG9yIGFzc3VtZSBzb21lIHR5
cGUgb2YgY29uZmlndXJhdGlvbi90dW5pbmcuIA0NRm9yIGV4YW1wbGUsIGlmIHdlIHRha2UgQ29k
ZWw6DQ1Gb3Igc21hbGwgdGhpbiBsaW5rcywgc3VjaCBhcyAxLTEwTWJzIERTTCwgdGhlIDVtcyB0
YXJnZXQgd291bGQgaW5jcmVhc2UgcGFja2V0IGxvc3Mgc2lnbmlmaWNhbnRseSBhbmQgYXQgMk1i
cHMsIGEgc2luZ2xlIE1UVSB0aW1lIG1heSBldmVuIGV4Y2VlZCB0aGUgNW1zIHRhcmdldC4gDQ1J
ZiB0aGUgYXZlcmFnZSBSVFQgb2YgYWxsIGZsb3dzIGdvaW5nIHRocm91Z2ggYSBsaW5rIGlzIG1v
cmUgdGhhbiA1MDBtcywgZS5nLiBmb3Igc2F0ZWxsaXRlLCB0aGVuIHRoZSAxMDBtcyBpbnRlcnZh
bCB3b3VsZCBwcmVtYXR1cmVseSBkcm9wIHBhY2tldHMgYmVmb3JlIHRoZSBzb3VyY2VzIGhhdmUg
aGFkIGEgY2hhbmNlIHRvIHJlZHVjZSB0aGVpciBzZW5kaW5nIHJhdGUuIE9yIGlmIHRoZSBhdmVy
YWdlIFJUVCBpcyB2ZXJ5IGxvdyCWIGUuZy4gMTBtcyCWIHN1Y2ggYXMgZm9yIGZsb3dzIGJldHdl
ZW4gZGF0YS1jZW50ZXIgZWxlbWVudHMsIHRoZW4gMTAwbXMgaW50ZXJ2YWwgbWF5IGJlIHNsb3cg
dG8gc2lnbmFsIGNvbmdlc3Rpb24gYmFjayB0byB0aGUgc291cmNlcyBhbmQgc2lnbmlmaWNhbnQg
cGFja2V0IGxvc3MgbWF5IGhhdmUgb2NjdXJyZWQgYmVmb3JlIHN1Y2ggc2lnbmFsaW5nLg0NDTQu
NC4gIEFRTSBhbGdvcml0aG1zIFNIT1VMRCByZXNwb25kIHRvIG1lYXN1cmVkIGNvbmdlc3Rpb24s
IG5vdA0gICAgICBhcHBsaWNhdGlvbiBwcm9maWxlcy4NDSAgIE5vdCBhbGwgYXBwbGljYXRpb25z
IHRyYW5zbWl0IHBhY2tldHMgb2YgdGhlIHNhbWUgc2l6ZS4gIEFsdGhvdWdoDSAgIGFwcGxpY2F0
aW9ucyBtYXkgYmUgY2hhcmFjdGVyaXNlZCBieSBwYXJ0aWN1bGFyIHByb2ZpbGVzIG9mIHBhY2tl
dA0gICBzaXplIHRoaXMgc2hvdWxkIG5vdCBiZSB1c2VkIGFzIHRoZSBiYXNpcyBmb3IgQVFNLiAg
T3RoZXIgbWV0aG9kcw0gICBleGlzdCwgZS5nLiBEaWZmZXJlbnRpYXRlZCBTZXJ2aWNlcyBxdWV1
ZWluZywgUHJlLUNvbmdlc3Rpb24NICAgTm90aWZpY2F0aW9uIChQQ04pIFtSRkM1NTU5XSwgdGhh
dCBjYW4gYmUgdXNlZCB0byBkaWZmZXJlbnRpYXRlIGFuZA0gICBwb2xpY2UgY2xhc3NlcyBvZiBh
cHBsaWNhdGlvbi4gIE5ldHdvcmsgZGV2aWNlcyBtYXkgY29tYmluZSBBUU0gd2l0aA0gICB0aGVz
ZSB0cmFmZmljIGNsYXNzaWZpY2F0aW9uIG1lY2hhbmlzbXMgYW5kIHBlcmZvcm0gQVFNIG9ubHkg
b24NICAgc3BlY2lmaWMgcXVldWVzIHdpdGhpbiBhIG5ldHdvcmsgZGV2aWNlLg0NICAgQW4gQVFN
IGFsZ29yaXRobSBzaG91bGQgbm90IGRlbGliZXJhdGVseSB0cnkgdG8gcHJlanVkaWNlIHRoZSBz
aXplIG9mDSAgIHBhY2tldCB0aGF0IHBlcmZvcm1zIGJlc3QgKGkuZS4gcHJlZmVyZW50aWFsbHkg
ZHJvcC9tYXJrIGJhc2VkIG9ubHkNICAgb24gcGFja2V0IHNpemUpLiAgUHJvY2VkdXJlcyBmb3Ig
c2VsZWN0aW5nIHBhY2tldHMgdG8gbWFyay9kcm9wDSAgIFNIT1VMRCBvYnNlcnZlIGFjdHVhbCBv
ciBwcm9qZWN0ZWQgdGltZSBhIHBhY2tldCBpcyBpbiBhIHF1ZXVlIChieXRlcw0gICBhdCBhIHJh
dGUgYmVpbmcgYW4gYW5hbG9nIHRvIHRpbWUpLiAgV2hlbiBhbiBBUU0gYWxnb3JpdGhtIGRlY2lk
ZXMNICAgd2hldGhlciB0byBkcm9wIChvciBtYXJrKSBhIHBhY2tldCwgaXQgaXMgUkVDT01NRU5E
RUQgdGhhdCB0aGUgc2l6ZQ0gICBvZiB0aGUgcGFydGljdWxhciBwYWNrZXQgc2hvdWxkIG5vdCBi
ZSB0YWtlbiBpbnRvIGFjY291bnQgW0J5dGUtcGt0XS4NDSAgIEFwcGxpY2F0aW9ucyAob3IgdHJh
bnNwb3J0cykgZ2VuZXJhbGx5IGtub3cgd2hhdCBwYWNrZXQgc2l6ZSB0aGV5IGFyZQ0gICB1c2lu
ZyBhbmQgY2FuIGhlbmNlIG1ha2UgdGhlaXIganVkZ2VtZW50cyBhYm91dCB3aGV0aGVyIHRvIHVz
ZSBzbWFsbA0NDQ1CYWtlciAmIEZhaXJodXJzdCAgICAgICBFeHBpcmVzIEphbnVhcnkgMTAsIDIw
MTQgICAgICAgICAgICAgICBbUGFnZSAxM10NDA1JbnRlcm5ldC1EcmFmdCAgIEFjdGl2ZSBRdWV1
ZSBNYW5hZ2VtZW50IFJlY29tbWVuZGF0aW9ucyAgICAgICBKdWx5IDIwMTMNDSAgIG9yIGxhcmdl
IHBhY2tldHMgYmFzZWQgb24gdGhlIGRhdGEgdGhleSB3aXNoIHRvIHNlbmQgYW5kIHRoZSBleHBl
Y3RlZA0gICBpbXBhY3Qgb24gdGhlIGRlbGF5IG9yIHRocm91Z2hwdXQsIG9yIG90aGVyIHBlcmZv
cm1hbmNlIHBhcmFtZXRlci4NICAgV2hlbiBhIHRyYW5zcG9ydCBvciBhcHBsaWNhdGlvbiByZXNw
b25kcyB0byBhIGRyb3BwZWQgb3IgbWFya2VkDSAgIHBhY2tldCwgdGhlIHNpemUgb2YgdGhlIHJh
dGUgcmVkdWN0aW9uIHNob3VsZCBiZSBwcm9wb3J0aW9uYXRlIHRvIHRoZQ0gICBzaXplIG9mIHRo
ZSBwYWNrZXQgdGhhdCB3YXMgc2VudCBbQnl0ZS1wa3RdLg0NNC41LiAgQVFNIGFsZ29yaXRobXMg
U0hPVUxEIE5PVCBiZSBkZXBlbmRlbnQgb24gc3BlY2lmaWMgdHJhbnNwb3J0DSAgICAgIHByb3Rv
Y29sIGJlaGF2aW91cnMNDSAgIEluIGRlcGxveWluZyBBUU0sIG5ldHdvcmsgZGV2aWNlcyBuZWVk
IHRvIHN1cHBvcnQgYSByYW5nZSBvZiBJbnRlcm5ldA0gICB0cmFmZmljIGFuZCBTSE9VTEQgTk9U
IG1ha2UgaW1wbGljaXQgYXNzdW1wdGlvbnMgYWJvdXQgdGhlDSAgIGNoYXJhY3RlcmlzdGljcyBk
ZXNpcmVkIGJ5IHRoZSBzZXQgdHJhbnNwb3J0cy9hcHBsaWNhdGlvbnMgdGhlDSAgIG5ldHdvcmsg
c3VwcG9ydHMuICBUaGF0IGlzLCBBUU0gbWV0aG9kcyBzaG91bGQgYmUgb3BhcXVlIHRvIHRoZQ0g
ICBjaG9pY2Ugb2YgdHJhbnNwb3J0IGFuZCBhcHBsaWNhdGlvbi4NDSAgIEFRTSBhbGdvcml0aG1z
IGFyZSBvZnRlbiBldmFsdWF0ZWQgYnkgY29uc2lkZXJpbmcgVENQIFtSRkMwNzkzXSB3aXRoDSAg
IGEgbGltaXRlZCBudW1iZXIgb2YgYXBwbGljYXRpb25zLiAgQWx0aG91Z2ggVENQIGlzIHRoZSBw
cmVkb21pbmFudA0gICB0cmFuc3BvcnQgaW4gdGhlIEludGVybmV0IHRvZGF5LiAgVGhpcyBpcyBu
byBsb25nZXIgcmVwcmVzZW50cyBhDSAgIHN1ZmZpY2llbnQgc2VsZWN0aW9uIG9mIHRyYWZmaWMg
Zm9yIHZlcmlmaWNhdGlvbi4gIFRoZXJlIGlzDSAgIHNpZ25pZmljYW50IHVzZSBvZiBVRFAgW1JG
QzA3NjhdIGluIHZvaWNlIGFuZCB2aWRlbyBzZXJ2aWNlcywgYW5kDSAgIHNvbWUgYXBwbGljYXRp
b25zIGZpbmQgdXRpbGl0eSBpbiBTQ1RQIFtSRkM0OTYwXSBhbmQgRENDUCBbUkZDNDM0MF0uDSAg
IEhlbmNlLCBBUU0gYWxnb3JpdGhtcyBzaG91bGQgYWxzbyBkZW1vbnN0cmF0ZSBvcGVyYXRpb24g
d2l0aA0gICB0cmFuc3BvcnRzIG90aGVyIHRoYW4gVENQIGFuZCBuZWVkIHRvIGNvbnNpZGVyIGEg
dmFyaWV0eSBvZg0gICBhcHBsaWNhdGlvbnMuICBBUU0gYWxnb3JpdGhtcyBhbHNvIG5lZWQgdG8g
Y29uc2lkZXIgdXNlIG9mIHR1bm5lbA0gICBlbmNhcHN1bGF0aW9ucywgd2hpY2ggbWF5IGNhcnJ5
IHRyYWZmaWMgYWdncmVnYXRlcy4NDSAgIEFRTSBhbGdvcml0aG1zIFNIT1VMRCBOT1QgdGFyZ2V0
IG9yIGRlcml2ZSBpbXBsaWNpdCBhc3N1bXB0aW9ucyBhYm91dA0gICB0aGUgY2hhcmFjdGVyaXN0
aWNzIGRlc2lyZWQgYnkgc3BlY2lmaWMgdHJhbnNwb3J0cy9hcHBsaWNhdGlvbnMuDSAgIFRyYW5z
cG9ydHMgYW5kIGFwcGxpY2F0aW9ucyBuZWVkIHRvIHJlc3BvbmQgdG8gdGhlIGNvbmdlc3Rpb24g
c2lnbmFscw0gICBwcm92aWRlZCBieSBBUU0gKGkuZS4gZHJvcHBpbmcgb3IgRUNOLW1hcmtpbmcp
IGluIGEgdGltZWx5IG1hbm5lcg0gICAod2l0aGluIGEgZmV3IFJUVCBhdCB0aGUgbGF0ZXN0KS4N
DTQuNi4gIEludGVyYWN0aW9ucyB3aXRoIGNvbmdlc3Rpb24gY29udHJvbCBhbGdvcml0aG1zID8/
Pz8NDSAgIEFwcGxpY2F0aW9ucyBhbmQgdHJhbnNwb3J0cyBuZWVkIHRvIHJlYWN0IHRvIHJlY2Vp
dmVkIGltcGxpY2l0IG9yDSAgIGV4cGxpY2l0IHNpZ25hbHMgdGhhdCBpbmRpY2F0ZSB0aGUgcHJl
c2VuY2Ugb2YgY29uZ2VzdGlvbi4NDSAgIFdoZW4gc3BlYWtpbmcgb2YgVENQIHBlcmZvcm1hbmNl
LCB0aGUgdGVybXMgImtuZWUiIGFuZCAiY2xpZmYiIGFyZWENICAgZGVmaW5lZCBieSBbSmFpbjk0
XS4gIFRoZXkgcmVzcGVjdGl2ZWx5IHJlZmVyIHRvIHRoZSBtaW5pbXVtDSAgIGNvbmdlc3Rpb24g
d2luZG93IHRoYXQgbWF4aW1pc2VzIHRocm91Z2hwdXQgYW5kIHRoZSBtYXhpbXVtDSAgIGNvbmdl
c3Rpb24gd2luZG93IHRoYXQgYXZvaWRzIGxvc3MuICBBbiBhcHBsaWNhdGlvbiB0aGF0IHRyYW5z
bWl0cyBhdA0gICB0aGUgcmF0ZSBkZXRlcm1pbmVkIGJ5IHRoaXMgd2luZG93IGhhcyB0aGUgZWZm
ZWN0IG9mIG1heGltaXppbmcgdGhlDSAgIHJhdGUgb3IgdGhyb3VnaHB1dC4gIEZvciB0aGUgc2Vu
ZGVyLCBleGNlZWRpbmcgdGhlIGNsaWZmIGlzDSAgIGluZWZmZWN0aXZlLCBhcyBpdCAoYnkgZGVm
aW5pdGlvbikgaW5kdWNlcyBsb3NzOyBvcGVyYXRpbmcgYXQgYSBwb2ludA0gICBjbG9zZSB0byB0
aGUgY2xpZmYgaGFzIGEgbmVnYXRpdmUgaW1wYWN0IG9uIG90aGVyIHRyYWZmaWMgYW5kDSAgIGFw
cGxpY2F0aW9ucywgdHJpZ2dlcmluZyBvcGVyYXRvciBhY3Rpdml0aWVzLCBzdWNoIGFzIHRob3Nl
IGRpc2N1c3NlZA0gICBpbiBbUkZDNjA1N10uICBPcGVyYXRpbmcgYmVsb3cgdGhlIGtuZWUgcmVk
dWNlcyB0aGUgdGhyb3VnaHB1dCwgc2luY2UNICAgdGhlIHNlbmRlciBmYWlscyB0byB1c2UgYXZh
aWxhYmxlIG5ldHdvcmsgY2FwYWNpdHkuDQ0NDUJha2VyICYgRmFpcmh1cnN0ICAgICAgIEV4cGly
ZXMgSmFudWFyeSAxMCwgMjAxNCAgICAgICAgICAgICAgIFtQYWdlIDE0XQ0MDUludGVybmV0LURy
YWZ0ICAgQWN0aXZlIFF1ZXVlIE1hbmFnZW1lbnQgUmVjb21tZW5kYXRpb25zICAgICAgIEp1bHkg
MjAxMw0NDSAgIElmIHRoZSBvYmplY3RpdmUgaXMgdG8gZGVsaXZlciBkYXRhIGZyb20gaXRzIHNv
dXJjZSB0byBpdHMgcmVjaXBpZW50DSAgIGluIHRoZSBsZWFzdCBwb3NzaWJsZSB0aW1lLCBhcyBh
IHJlc3VsdCwgdGhlIGJlaGF2aW9yIG9mIGFueSBlbGFzdGljDSAgIHRyYW5zcG9ydCBjb25nZXN0
aW9uIGNvbnRyb2wgYWxnb3JpdGhtIHNob3VsZCBzZWVrIHRvIHVzZSBhbg0gICBlZmZlY3RpdmUg
d2luZG93IGF0IG9yIGFib3ZlIHRoZSBrbmVlIGFuZCB3ZWxsIGJlbG93IHRoZSBjbGlmZi4NICAg
Q2hvaWNlIG9mIGFuIGFwcHJvcHJpYXRlIHJhdGUgY2FuIHNpZ25pZmljYW50bHkgaW1wYWN0IHRo
ZSBsb3NzIGFuZA0gICBkZWxheSBleHBlcmllbmNlZCBub3Qgb25seSBieSBhIGZsb3cgYnV0IGJ5
IG90aGVyIGZsb3dzIHRoYXQgc2hhcmUNICAgdGhlIHNhbWUgcXVldWUuDQ0gICBTb21lIGFwcGxp
Y2F0aW9ucyBtYXkgc2VuZCBsZXNzIHRoYW4gcGVybWl0dGVkIGJ5IHRoZSBjb25nZXN0aW9uDSAg
IGNvbnRyb2wgd2luZG93IChvciByYXRlKS4gIEV4YW1wbGVzIGluY2x1ZGUgbXVsdGltZWRpYSBj
b2RlY3MgdGhhdA0gICBzdHJlYW0gYXQgc29tZSBuYXR1cmFsIHJhdGUgKG9yIHNldCBvZiByYXRl
cykgb3IgYW4gYXBwbGljYXRpb24gdGhhdA0gICBpcyBuYXR1cmFsbHkgaW50ZXJhY3RpdmUgKGUu
Zy4gc29tZSB3ZWIgYXBwbGljYXRpb25zLCBnYW1pbmcsDSAgIHRyYW5zYWN0aW9uLWJhc2VkIHBy
b3RvY29scykuICBTdWNoIGFwcGxpY2F0aW9ucyBtYXkgbm90IHdpc2ggdG8NICAgbWF4aW1pc2Ug
dGhyb3VnaHB1dCwgYnV0IG1heSBhbHNvIGRlc2lyZSBhIGxvd2VyIGxvc3MgcmF0ZSBvciBib3Vu
ZGVkDSAgIGRlbGF5Lg0NICAgVHJhbnNwb3J0IHByb3RvY29scyBhbmQgYXBwbGljYXRpb25zIG5l
ZWQgdGltZWx5IHNpZ25hbHMgb2YNICAgY29uZ2VzdGlvbi4gIFRoZSB0aW1lIHRha2VuIHRvIGRl
dGVjdCBhbmQgcmVzcG9uZCB0byBjb25nZXN0aW9uIGlzDSAgIGFzc2lzdGVkIGJ5IG5ldHdvcmsg
ZGV2aWNlcyBub3QgZHJvcHBpbmcgbG9uZyBydW5zIG9mIHBhY2tldHMgZnJvbQ0gICB0aGUgc2Ft
ZSBmbG93LiAgSXQgaXMgZGlmZmljdWx0IHRvIGRldGVjdCB0YWlsIGxvc3NlcyBhdCBhIGhpZ2hl
cg0gICBsYXllciBhbmQgbWF5IHNvbWV0aW1lcyByZXF1aXJlIHRpbWVycyBvciBwcm9iZXMgdG8g
ZGV0ZWN0IGFuZA0gICByZXNwb25kIHRvIHN1Y2ggbG9zcy4NDSAgIFRoZSBjb3JyZWN0IG9wZXJh
dGlvbiBvZiBhbiBBUU0tZW5hYmxlZCBuZXR3b3JrIGRldmljZSBNVVNUIE5PVCByZWx5DSAgIHVw
b24gc3BlY2lmaWMgdHJhbnNwb3J0IHJlc3BvbnNlcyB0byBjb25nZXN0aW9uIHNpZ25hbHMuDQ00
LjcuICBUaGUgbmVlZCBmb3IgZnVydGhlciByZXNlYXJjaA0NICAgVGhlIHNlY29uZCByZWNvbW1l
bmRhdGlvbiBvZiBbUkZDMjMwOV0gY2FsbGVkIGZvciBmdXJ0aGVyIHJlc2VhcmNoIGluDSAgIHRo
ZSBpbnRlcmFjdGlvbiBiZXR3ZWVuIG5ldHdvcmsgcXVldWVzIGFuZCBob3N0IGFwcGxpY2F0aW9u
cywgYW5kIHRoZQ0gICBtZWFucyBvZiBzaWduYWxpbmcgYmV0d2VlbiB0aGVtLiAgVGhpcyByZXNl
YXJjaCBoYXMgb2NjdXJyZWQsIGFuZCB3ZQ0gICBhcyBhIGNvbW11bml0eSBoYXZlIGxlYXJuZWQg
YSBsb3QuICBIb3dldmVyLCB3ZSBhcmUgbm90IGRvbmUuDQ0gICBXZSBoYXZlIGxlYXJuZWQgdGhh
dCB0aGUgcHJvYmxlbXMgb2YgY29uZ2VzdGlvbiwgbGF0ZW5jeSBhbmQgYnVmZmVyLQ0gICBzaXpp
bmcgaGF2ZSBub3QgZ29uZSBhd2F5LCBhbmQgYXJlIGJlY29taW5nIG1vcmUgaW1wb3J0YW50IHRv
IG1hbnkNICAgdXNlcnMuICBBIG51bWJlciBvZiBzZWxmLXR1bmluZyBBUU0gYWxnb3JpdGhtcyBo
YXZlIGJlIGZvdW5kIHRoYXQNICAgb2ZmZXIgc2lnbmlmaWNhbnQgYWR2YW50YWdlcyBmb3IgZGVw
bG95ZWQgbmV0d29ya3MuICBUaGVyZSBpcyBhbHNvDSAgIHJlbmV3ZWQgaW50ZXJlc3QgaW4gZGVw
bG95aW5nIEFRTSBhbmQgdGhlIHBvdGVudGlhbCBvZiBFQ04uDQ0gICBBbiBvYnZpb3VzIGV4YW1w
bGUgb2YgZnVydGhlciByZXNlYXJjaCBpbiAyMDEzIGlzIHRoZSBuZWVkIHRvDSAgIGNvbnNpZGVy
IHRoZSB1c2Ugb2YgTWFwL1JlZHVjZSBhcHBsaWNhdGlvbnMgaW4gZGF0YSBjZW50ZXJzOyBkbyB3
ZQ0gICBuZWVkIHRvIGV4dGVuZCBvdXIgdGF4b25vbXkgb2YgVENQL1NDVFAgc2Vzc2lvbnMgdG8g
aW5jbHVkZSBub3Qgb25seQ0gICAibWljZSIgYW5kICJlbGVwaGFudHMiLCBidXQgImxlbW1pbmdz
Ij8gICJMZW1taW5ncyIgYXJlIGZsYXNoIGNyb3dkcw0gICBvZiAibWljZSIgdGhhdCB0aGUgbmV0
d29yayBpbmFkdmVydGVudGx5IHRyaWVzIHRvIHNpZ25hbCB0byBhcyBpZg0gICB0aGV5IHdlcmUg
ZWxlcGhhbnQgZmxvd3MsIHJlc3VsdGluZyBpbiBoZWFkIG9mIGxpbmUgYmxvY2tpbmcgaW4gZGF0
YQ0gICBjZW50ZXIgYXBwbGljYXRpb25zLg0NU3VjaCByZXNlYXJjaCBzaG91bGQgYWxzbyBmb2N1
cyBvbiBpbXByb3ZpbmcgZW5kIHVzZXIgUW9FIGZyb20gQVFNcyByYXRoZXIgdGhhbiBuZXR3b3Jr
IHJlbGF0ZWQgbWV0cmljcyBvbmx5LiBPZnRlbiBhIHNpZ25pZmljYW50IGNoYW5nZSBpbiBhIG5l
dHdvcmsgbWV0cmljIG1heSBvbmx5IG1ha2UgYSBtaW5pbWFsIGNoYW5nZSBpbiBlbmQtdXNlciBR
b0UgYW5kIHRodXMgdGhlIHZhbHVlIG9mIHN1Y2ggY2hhbmdlIG1heSBiZSBtaW5pbWFsLg0NICAg
RXhhbXBsZXMgb2Ygb3RoZXIgcmVxdWlyZWQgcmVzZWFyY2ggaW5jbHVkZToNDQ0NQmFrZXIgJiBG
YWlyaHVyc3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAgICAgICAgICAgW1Bh
Z2UgMTVdDQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCBSZWNvbW1l
bmRhdGlvbnMgICAgICAgSnVseSAyMDEzDQ0gICBvICBSZXNlYXJjaCBpbnRvIG5ldyBBUU0gYW5k
IHNjaGVkdWxpbmcgYWxnb3JpdGhtcy4NDSAgIG8gIFJlc2VhcmNoIGludG8gdGhlIHVzZSBvZiBh
bmQgZGVwbG95bWVudCBvZiBFQ04gYWxvbmdzaWRlIEFRTS4NDSAgIG8gIFRvb2xzIGZvciBlbmFi
bGluZyBBUU0gKGFuZCBFQ04pIGRlcGxveW1lbnQgYW5kIG1lYXN1cmluZyB0aGUNICAgICAgcGVy
Zm9ybWFuY2UuDQ0gICBvICBNZXRob2RzIGZvciBtaXRpZ2F0aW5nIHRoZSBpbXBhY3Qgb2Ygbm9u
LWNvbmZvcm1hbnQgYW5kIG1hbGljaW91cw0gICAgICBmbG93cy4NDU1ldGhvZHMgb3IgY29uZmln
dXJhdGlvbnMgdGhhdCBsZXZlcmFnZSBkZXBsb3llZCBBUU1zIHN1Y2ggYXMgUkVEL1dSRUQgdG8g
cmVkdWNlIGRlbGF5cyBhbmQgbG9ja291dCBmb3IgdHlwaWNhbCB0cmFmZmljIHdoaWNoIHJlcXVp
cmUgbWluaW1hbCBlZmZvcnQgb3IgdHVuaW5nIGZyb20gdGhlIG9wZXJhdG9yLiANDSAgIEhlbmNl
LCB0aGlzIGRvY3VtZW50IHRoZXJlZm9yZSByZWl0ZXJhdGVzIHRoZSBjYWxsIG9mIFJGQyAyMzA5
OiB3ZQ0gICBuZWVkIGNvbnRpbnVpbmcgcmVzZWFyY2ggYXMgYXBwbGljYXRpb25zIGRldmVsb3Au
DQ01LiAgSUFOQSBDb25zaWRlcmF0aW9ucw0NICAgVGhpcyBtZW1vIGFza3MgdGhlIElBTkEgZm9y
IG5vIG5ldyBwYXJhbWV0ZXJzLg0NNi4gIFNlY3VyaXR5IENvbnNpZGVyYXRpb25zDQ0gICBXaGls
ZSBzZWN1cml0eSBpcyBhIHZlcnkgaW1wb3J0YW50IGlzc3VlLCBpdCBpcyBsYXJnZWx5IG9ydGhv
Z29uYWwgdG8NICAgdGhlIHBlcmZvcm1hbmNlIGlzc3VlcyBkaXNjdXNzZWQgaW4gdGhpcyBtZW1v
LiAgV2Ugbm90ZSwgaG93ZXZlciwNICAgdGhhdCBkZW5pYWwtb2Ytc2VydmljZSBhdHRhY2tzIG1h
eSBjcmVhdGUgdW5yZXNwb25zaXZlIHRyYWZmaWMgZmxvd3MNICAgdGhhdCBtYXkgYmUgaW5kaXN0
aW5ndWlzaGFibGUgZnJvbSBvdGhlciBmbG93cyAoZS5nLiB0dW5uZWxzIGNhcnJ5aW5nDSAgIGFn
Z3JlZ2F0ZXMgb2Ygc2hvcnQgZmxvd3MsIGhpZ2gtcmF0ZSBpc29jaHJvbm91cyBhcHBsaWNhdGlv
bnMpLiAgVGhpcw0gICB0aHJlYXQgZXhpc3RzIGFsc28gaW4gbmV0d29yayBkZXZpY2VzIHRoYXQg
ZG8gbm90IGRlcGxveSBBUU0sIGJ1dA0gICB3aGVuIEFRTSBpcyBkZXBsb3llZCBjb3VsZCBiZSB1
c2VkIHRvIGRlZ3JhZGUgdGhlIGJlbmVmaXQgb2YgdGhlIG5ldw0gICBtZXRob2QuICBUaGUgcmVj
b21tZW5kYXRpb25zIHN1cHBvcnQgb25nb2luZyByZXNlYXJjaCBhcHBsaWNhYmxlIHRvDSAgIHN1
Y2ggYXR0YWNrcy4NDTcuICBQcml2YWN5IENvbnNpZGVyYXRpb25zDQ0gICBUaGlzIGRvY3VtZW50
LCBieSBpdHNlbGYsIHByZXNlbnRzIG5vIG5ldyBwcml2YWN5IGlzc3Vlcy4NDTguICBBY2tub3ds
ZWRnZW1lbnRzDQ0gICBUaGUgb3JpZ2luYWwgcmVjb21tZW5kYXRpb24gaW4gW1JGQzIzMDldIHdh
cyB3cml0dGVuIGJ5IHRoZSBFbmQtdG8tDSAgIEVuZCBSZXNlYXJjaCBHcm91cCwgd2hpY2ggaXMg
dG8gc2F5IEJvYiBCcmFkZW4sIERhdmUgQ2xhcmssIEpvbg0gICBDcm93Y3JvZnQsIEJydWNlIERh
dmllLCBTdGV2ZSBEZWVyaW5nLCBEZWJvcmFoIEVzdHJpbiwgU2FsbHkgRmxveWQsDSAgIFZhbiBK
YWNvYnNvbiwgR3JlZyBNaW5zaGFsbCwgQ3JhaWcgUGFydHJpZGdlLCBMYXJyeSBQZXRlcnNvbiwg
S0sNICAgUmFtYWtyaXNobmFuLCBTY290dCBTaGVua2VyLCBKb2huIFdyb2NsYXdza2ksIGFuZCBM
aXhpYSBaaGFuZy4gIFRoaXMNICAgaXMgYW4gZWRpdGVkIHZlcnNpb24gb2YgdGhhdCBkb2N1bWVu
dCwgd2l0aCBtdWNoIG9mIGl0cyB0ZXh0IGFuZA0gICBhcmd1bWVudHMgdW5jaGFuZ2VkLg0NICAg
VGhlIG5lZWQgZm9yIGFuIHVwZGF0ZWQgZG9jdW1lbnQgd2FzIGFncmVlZCB0byBpbiB0aGUgdHN2
YXJlYSBtZWV0aW5nDSAgIGF0IElFVEYgODYuICBUaGlzIGRvY3VtZW50IHdhcyByZXZpZXdlZCBv
biB0aGUgYXFtQGlldGYub3JnIGxpc3QuDSAgIENvbW1lbnRzIGNhbWUgZnJvbSBDb2xpbiBQZXJr
aW5zLCBSaWNoYXJkIFNjaGVmZmVuZWdnZXIsIGFuZCBEYXZlDSAgIFRhaHQuDQ0NDQ1CYWtlciAm
IEZhaXJodXJzdCAgICAgICBFeHBpcmVzIEphbnVhcnkgMTAsIDIwMTQgICAgICAgICAgICAgICBb
UGFnZSAxNl0NDA1JbnRlcm5ldC1EcmFmdCAgIEFjdGl2ZSBRdWV1ZSBNYW5hZ2VtZW50IFJlY29t
bWVuZGF0aW9ucyAgICAgICBKdWx5IDIwMTMNDSAgIEdvcnJ5IEZhaXJodXJzdCB3YXMgaW4gcGFy
dCBzdXBwb3J0ZWQgYnkgdGhlIEV1cm9wZWFuIENvbW11bml0eSB1bmRlcg0gICBpdHMgU2V2ZW50
aCBGcmFtZXdvcmsgUHJvZ3JhbW1lIHRocm91Z2ggdGhlIFJlZHVjaW5nIEludGVybmV0DSAgIFRy
YW5zcG9ydCBMYXRlbmN5IChSSVRFKSBwcm9qZWN0IChJQ1QtMzE3NzAwKS4NDTkuICBSZWZlcmVu
Y2VzDQ05LjEuICBOb3JtYXRpdmUgUmVmZXJlbmNlcw0NICAgW0J5dGUtcGt0XQ0gICAgICAgICAg
ICAgIEludGVybmV0IEVuZ2luZWVyaW5nIFRhc2sgRm9yY2UsIFdvcmsgaW4gUHJvZ3Jlc3MsICJC
eXRlDSAgICAgICAgICAgICAgYW5kIFBhY2tldCBDb25nZXN0aW9uIE5vdGlmaWNhdGlvbiAoZHJh
ZnQtaWV0Zi10c3Z3Zy1ieXRlLQ0gICAgICAgICAgICAgIHBrdC1jb25nZXN0KSIsIEp1bHkgMjAx
My4NDSAgIFtSRkMyMTE5XSAgQnJhZG5lciwgUy4sICJLZXkgd29yZHMgZm9yIHVzZSBpbiBSRkNz
IHRvIEluZGljYXRlDSAgICAgICAgICAgICAgUmVxdWlyZW1lbnQgTGV2ZWxzIiwgQkNQIDE0LCBS
RkMgMjExOSwgTWFyY2ggMTk5Ny4NDSAgIFtSRkMzMTY4XSAgUmFtYWtyaXNobmFuLCBLLiwgRmxv
eWQsIFMuLCBhbmQgRC4gQmxhY2ssICJUaGUgQWRkaXRpb24NICAgICAgICAgICAgICBvZiBFeHBs
aWNpdCBDb25nZXN0aW9uIE5vdGlmaWNhdGlvbiAoRUNOKSB0byBJUCIsIFJGQw0gICAgICAgICAg
ICAgIDMxNjgsIFNlcHRlbWJlciAyMDAxLg0NICAgW1JGQzQzMDFdICBLZW50LCBTLiBhbmQgSy4g
U2VvLCAiU2VjdXJpdHkgQXJjaGl0ZWN0dXJlIGZvciB0aGUNICAgICAgICAgICAgICBJbnRlcm5l
dCBQcm90b2NvbCIsIFJGQyA0MzAxLCBEZWNlbWJlciAyMDA1Lg0NICAgW1JGQzQ3NzRdICBGbG95
ZCwgUy4sICJTcGVjaWZ5aW5nIEFsdGVybmF0ZSBTZW1hbnRpY3MgZm9yIHRoZQ0gICAgICAgICAg
ICAgIEV4cGxpY2l0IENvbmdlc3Rpb24gTm90aWZpY2F0aW9uIChFQ04pIEZpZWxkIiwgQkNQIDEy
NCwNICAgICAgICAgICAgICBSRkMgNDc3NCwgTm92ZW1iZXIgMjAwNi4NDSAgIFtSRkM1NDA1XSAg
RWdnZXJ0LCBMLiBhbmQgRy4gRmFpcmh1cnN0LCAiVW5pY2FzdCBVRFAgVXNhZ2UgR3VpZGVsaW5l
cw0gICAgICAgICAgICAgIGZvciBBcHBsaWNhdGlvbiBEZXNpZ25lcnMiLCBCQ1AgMTQ1LCBSRkMg
NTQwNSwgTm92ZW1iZXINICAgICAgICAgICAgICAyMDA4Lg0NICAgW1JGQzU2ODFdICBBbGxtYW4s
IE0uLCBQYXhzb24sIFYuLCBhbmQgRS4gQmxhbnRvbiwgIlRDUCBDb25nZXN0aW9uDSAgICAgICAg
ICAgICAgQ29udHJvbCIsIFJGQyA1NjgxLCBTZXB0ZW1iZXIgMjAwOS4NDSAgIFtSRkM2MDQwXSAg
QnJpc2NvZSwgQi4sICJUdW5uZWxsaW5nIG9mIEV4cGxpY2l0IENvbmdlc3Rpb24NICAgICAgICAg
ICAgICBOb3RpZmljYXRpb24iLCBSRkMgNjA0MCwgTm92ZW1iZXIgMjAxMC4NDSAgIFtSRkM2Njc5
XSAgV2VzdGVybHVuZCwgTS4sIEpvaGFuc3NvbiwgSS4sIFBlcmtpbnMsIEMuLCBPJ0hhbmxvbiwg
UC4sDSAgICAgICAgICAgICAgYW5kIEsuIENhcmxiZXJnLCAiRXhwbGljaXQgQ29uZ2VzdGlvbiBO
b3RpZmljYXRpb24gKEVDTikNICAgICAgICAgICAgICBmb3IgUlRQIG92ZXIgVURQIiwgUkZDIDY2
NzksIEF1Z3VzdCAyMDEyLg0NOS4yLiAgSW5mb3JtYXRpdmUgUmVmZXJlbmNlcw0NICAgW0RlbWVy
czkwXQ0gICAgICAgICAgICAgIERlbWVycywgQS4sIEtlc2hhdiwgUy4sIGFuZCBTLiBTaGVua2Vy
LCAiQW5hbHlzaXMgYW5kDSAgICAgICAgICAgICAgU2ltdWxhdGlvbiBvZiBhIEZhaXIgUXVldWVp
bmcgQWxnb3JpdGhtLCBJbnRlcm5ldHdvcmtpbmc6DSAgICAgICAgICAgICAgUmVzZWFyY2ggYW5k
IEV4cGVyaWVuY2UiLCBTSUdDT01NIFN5bXBvc2l1bSBwcm9jZWVkaW5ncyBvbg0gICAgICAgICAg
ICAgIENvbW11bmljYXRpb25zIGFyY2hpdGVjdHVyZXMgYW5kIHByb3RvY29scyAsIDE5OTAuDQ0N
DUJha2VyICYgRmFpcmh1cnN0ICAgICAgIEV4cGlyZXMgSmFudWFyeSAxMCwgMjAxNCAgICAgICAg
ICAgICAgIFtQYWdlIDE3XQ0MDUludGVybmV0LURyYWZ0ICAgQWN0aXZlIFF1ZXVlIE1hbmFnZW1l
bnQgUmVjb21tZW5kYXRpb25zICAgICAgIEp1bHkgMjAxMw0NICAgW0Zsb3lkOTFdICBGbG95ZCwg
Uy4sICJDb25uZWN0aW9ucyB3aXRoIE11bHRpcGxlIENvbmdlc3RlZCBHYXRld2F5cw0gICAgICAg
ICAgICAgIGluIFBhY2tldC1Td2l0Y2hlZCBOZXR3b3JrcyBQYXJ0IDE6IE9uZS13YXkgVHJhZmZp
Yy4iLA0gICAgICAgICAgICAgIENvbXB1dGVyIENvbW11bmljYXRpb25zIFJldmlldyAsIE9jdG9i
ZXIgMTk5MS4NDSAgIFtGbG95ZDk1XSAgRmxveWQsIFMuIGFuZCBWLiBKYWNvYnNvbiwgIkxpbmst
c2hhcmluZyBhbmQgUmVzb3VyY2UNICAgICAgICAgICAgICBNYW5hZ2VtZW50IE1vZGVscyBmb3Ig
UGFja2V0IE5ldHdvcmtzIiwgSUVFRS9BQ00NICAgICAgICAgICAgICBUcmFuc2FjdGlvbnMgb24g
TmV0d29ya2luZyAsIEF1Z3VzdCAxOTk1Lg0NICAgW0phY29ic29uODhdDSAgICAgICAgICAgICAg
SmFjb2Jzb24sIFYuLCAiQ29uZ2VzdGlvbiBBdm9pZGFuY2UgYW5kIENvbnRyb2wiLCBTSUdDT01N
DSAgICAgICAgICAgICAgU3ltcG9zaXVtIHByb2NlZWRpbmdzIG9uIENvbW11bmljYXRpb25zIGFy
Y2hpdGVjdHVyZXMgYW5kDSAgICAgICAgICAgICAgcHJvdG9jb2xzICwgQXVndXN0IDE5ODguDQ0g
ICBbSmFpbjk0XSAgIEphaW4sIFJhai4sIFJhbWFrcmlzaG5hbiwgS0suLCBhbmQgQ2hpdS4gRGFo
LU1pbmcsDSAgICAgICAgICAgICAgIkNvbmdlc3Rpb24gYXZvaWRhbmNlIHNjaGVtZSBmb3IgY29t
cHV0ZXIgbmV0d29ya3MiLCBVUw0gICAgICAgICAgICAgIFBhdGVudCBPZmZpY2UgNTM3NzMyNywg
RGVjZW1iZXIgMTk5NC4NDSAgIFtMYWtzaG1hbjk2XQ0gICAgICAgICAgICAgIExha3NobWFuLCBU
Vi4sIE5laWRoYXJkdCwgQS4sIGFuZCBULiBPdHQsICJUaGUgRHJvcCBGcm9tDSAgICAgICAgICAg
ICAgRnJvbnQgU3RyYXRlZ3kgaW4gVENQIE92ZXIgQVRNIGFuZCBJdHMgSW50ZXJ3b3JraW5nIHdp
dGgNICAgICAgICAgICAgICBPdGhlciBDb250cm9sIEZlYXR1cmVzIiwgSUVFRSBJbmZvY29tbSAs
IDE5OTYuDQ0gICBbTGVsYW5kOTRdDSAgICAgICAgICAgICAgTGVsYW5kLCBXLiwgVGFxcXUsIE0u
LCBXaWxsaW5nZXIsIFcuLCBhbmQgRC4gV2lsc29uLCAiT24NICAgICAgICAgICAgICB0aGUgU2Vs
Zi1TaW1pbGFyIE5hdHVyZSBvZiBFdGhlcm5ldCBUcmFmZmljIChFeHRlbmRlZA0gICAgICAgICAg
ICAgIFZlcnNpb24pIiwgSUVFRS9BQ00gVHJhbnNhY3Rpb25zIG9uIE5ldHdvcmtpbmcgLCBGZWJy
dWFyeQ0gICAgICAgICAgICAgIDE5OTQuDQ0gICBbUGFwYWdpYW5uYWtpXQ0gICAgICAgICAgICAg
IFNwcmludCBBVEwsIEtBSVNULCBVbml2ZXJzaXR5IG9mIE1pbm5lc290YSwgU3ByaW50IEFUTCwN
ICAgICAgICAgICAgICBJbnRlbCBSZXNlYXJjaCwgIkFuYWx5c2lzIG9mIFBvaW50LVRvLVBvaW50
IFBhY2tldCBEZWxheQ0gICAgICAgICAgICAgIEluIGFuIE9wZXJhdGlvbmFsIE5ldHdvcmsiLCBJ
RUVFIEluZm9jb20gMjAwNCwgTWFyY2ggMjAwNCwNICAgICAgICAgICAgICA8aHR0cDovL3d3dy5p
ZWVlLWluZm9jb20ub3JnLzIwMDQvUGFwZXJzLzM3XzQuUERGPi4NDSAgIFtSRkMwNzY4XSAgUG9z
dGVsLCBKLiwgIlVzZXIgRGF0YWdyYW0gUHJvdG9jb2wiLCBTVEQgNiwgUkZDIDc2OCwNICAgICAg
ICAgICAgICBBdWd1c3QgMTk4MC4NDSAgIFtSRkMwNzkxXSAgUG9zdGVsLCBKLiwgIkludGVybmV0
IFByb3RvY29sIiwgU1REIDUsIFJGQyA3OTEsIFNlcHRlbWJlcg0gICAgICAgICAgICAgIDE5ODEu
DQ0gICBbUkZDMDc5M10gIFBvc3RlbCwgSi4sICJUcmFuc21pc3Npb24gQ29udHJvbCBQcm90b2Nv
bCIsIFNURCA3LCBSRkMNICAgICAgICAgICAgICA3OTMsIFNlcHRlbWJlciAxOTgxLg0NICAgW1JG
QzA4OTZdICBOYWdsZSwgSi4sICJDb25nZXN0aW9uIGNvbnRyb2wgaW4gSVAvVENQIGludGVybmV0
d29ya3MiLA0gICAgICAgICAgICAgIFJGQyA4OTYsIEphbnVhcnkgMTk4NC4NDSAgIFtSRkMwOTcw
XSAgTmFnbGUsIEouLCAiT24gcGFja2V0IHN3aXRjaGVzIHdpdGggaW5maW5pdGUgc3RvcmFnZSIs
IFJGQw0gICAgICAgICAgICAgIDk3MCwgRGVjZW1iZXIgMTk4NS4NDQ0NQmFrZXIgJiBGYWlyaHVy
c3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAgICAgICAgICAgW1BhZ2UgMThd
DQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCBSZWNvbW1lbmRhdGlv
bnMgICAgICAgSnVseSAyMDEzDQ0gICBbUkZDMTEyMl0gIEJyYWRlbiwgUi4sICJSZXF1aXJlbWVu
dHMgZm9yIEludGVybmV0IEhvc3RzIC0NICAgICAgICAgICAgICBDb21tdW5pY2F0aW9uIExheWVy
cyIsIFNURCAzLCBSRkMgMTEyMiwgT2N0b2JlciAxOTg5Lg0NICAgW1JGQzE2MzNdICBCcmFkZW4s
IEIuLCBDbGFyaywgRC4sIGFuZCBTLiBTaGVua2VyLCAiSW50ZWdyYXRlZA0gICAgICAgICAgICAg
IFNlcnZpY2VzIGluIHRoZSBJbnRlcm5ldCBBcmNoaXRlY3R1cmU6IGFuIE92ZXJ2aWV3IiwgUkZD
DSAgICAgICAgICAgICAgMTYzMywgSnVuZSAxOTk0Lg0NICAgW1JGQzIzMDldICBCcmFkZW4sIEIu
LCBDbGFyaywgRC4sIENyb3djcm9mdCwgSi4sIERhdmllLCBCLiwgRGVlcmluZywNICAgICAgICAg
ICAgICBTLiwgRXN0cmluLCBELiwgRmxveWQsIFMuLCBKYWNvYnNvbiwgVi4sIE1pbnNoYWxsLCBH
LiwNICAgICAgICAgICAgICBQYXJ0cmlkZ2UsIEMuLCBQZXRlcnNvbiwgTC4sIFJhbWFrcmlzaG5h
biwgSy4sIFNoZW5rZXIsDSAgICAgICAgICAgICAgUy4sIFdyb2NsYXdza2ksIEouLCBhbmQgTC4g
WmhhbmcsICJSZWNvbW1lbmRhdGlvbnMgb24NICAgICAgICAgICAgICBRdWV1ZSBNYW5hZ2VtZW50
IGFuZCBDb25nZXN0aW9uIEF2b2lkYW5jZSBpbiB0aGUNICAgICAgICAgICAgICBJbnRlcm5ldCIs
IFJGQyAyMzA5LCBBcHJpbCAxOTk4Lg0NICAgW1JGQzI0NjBdICBEZWVyaW5nLCBTLiBhbmQgUi4g
SGluZGVuLCAiSW50ZXJuZXQgUHJvdG9jb2wsIFZlcnNpb24gNg0gICAgICAgICAgICAgIChJUHY2
KSBTcGVjaWZpY2F0aW9uIiwgUkZDIDI0NjAsIERlY2VtYmVyIDE5OTguDQ0gICBbUkZDMjQ3NF0g
IE5pY2hvbHMsIEsuLCBCbGFrZSwgUy4sIEJha2VyLCBGLiwgYW5kIEQuIEJsYWNrLA0gICAgICAg
ICAgICAgICJEZWZpbml0aW9uIG9mIHRoZSBEaWZmZXJlbnRpYXRlZCBTZXJ2aWNlcyBGaWVsZCAo
RFMNICAgICAgICAgICAgICBGaWVsZCkgaW4gdGhlIElQdjQgYW5kIElQdjYgSGVhZGVycyIsIFJG
QyAyNDc0LCBEZWNlbWJlcg0gICAgICAgICAgICAgIDE5OTguDQ0gICBbUkZDMjQ3NV0gIEJsYWtl
LCBTLiwgQmxhY2ssIEQuLCBDYXJsc29uLCBNLiwgRGF2aWVzLCBFLiwgV2FuZywgWi4sDSAgICAg
ICAgICAgICAgYW5kIFcuIFdlaXNzLCAiQW4gQXJjaGl0ZWN0dXJlIGZvciBEaWZmZXJlbnRpYXRl
ZA0gICAgICAgICAgICAgIFNlcnZpY2VzIiwgUkZDIDI0NzUsIERlY2VtYmVyIDE5OTguDQ0gICBb
UkZDMjk2MF0gIFN0ZXdhcnQsIFIuLCBYaWUsIFEuLCBNb3JuZWF1bHQsIEsuLCBTaGFycCwgQy4s
DSAgICAgICAgICAgICAgU2Nod2FyemJhdWVyLCBILiwgVGF5bG9yLCBULiwgUnl0aW5hLCBJLiwg
S2FsbGEsIE0uLCAgICAgICAgICAgICAgWmhhbmcsIEwuLCBhbmQgVi4gUGF4c29uLCAiU3RyZWFt
IENvbnRyb2wgVHJhbnNtaXNzaW9uDSAgICAgICAgICAgICAgUHJvdG9jb2wiLCBSRkMgMjk2MCwg
T2N0b2JlciAyMDAwLg0NICAgW1JGQzQzNDBdICBLb2hsZXIsIEUuLCBIYW5kbGV5LCBNLiwgYW5k
IFMuIEZsb3lkLCAiRGF0YWdyYW0NICAgICAgICAgICAgICBDb25nZXN0aW9uIENvbnRyb2wgUHJv
dG9jb2wgKERDQ1ApIiwgUkZDIDQzNDAsIE1hcmNoIDIwMDYuDQ0gICBbUkZDNDk2MF0gIFN0ZXdh
cnQsIFIuLCAiU3RyZWFtIENvbnRyb2wgVHJhbnNtaXNzaW9uIFByb3RvY29sIiwgUkZDDSAgICAg
ICAgICAgICAgNDk2MCwgU2VwdGVtYmVyIDIwMDcuDQ0gICBbUkZDNTM0OF0gIEZsb3lkLCBTLiwg
SGFuZGxleSwgTS4sIFBhZGh5ZSwgSi4sIGFuZCBKLiBXaWRtZXIsICJUQ1ANICAgICAgICAgICAg
ICBGcmllbmRseSBSYXRlIENvbnRyb2wgKFRGUkMpOiBQcm90b2NvbCBTcGVjaWZpY2F0aW9uIiwg
UkZDDSAgICAgICAgICAgICAgNTM0OCwgU2VwdGVtYmVyIDIwMDguDQ0gICBbUkZDNTU1OV0gIEVh
cmRsZXksIFAuLCAiUHJlLUNvbmdlc3Rpb24gTm90aWZpY2F0aW9uIChQQ04pDSAgICAgICAgICAg
ICAgQXJjaGl0ZWN0dXJlIiwgUkZDIDU1NTksIEp1bmUgMjAwOS4NDSAgIFtSRkM1NjgxXSAgQWxs
bWFuLCBNLiwgUGF4c29uLCBWLiwgYW5kIEUuIEJsYW50b24sICJUQ1AgQ29uZ2VzdGlvbg0gICAg
ICAgICAgICAgIENvbnRyb2wiLCBSRkMgNTY4MSwgU2VwdGVtYmVyIDIwMDkuDQ0NDQ0NQmFrZXIg
JiBGYWlyaHVyc3QgICAgICAgRXhwaXJlcyBKYW51YXJ5IDEwLCAyMDE0ICAgICAgICAgICAgICAg
W1BhZ2UgMTldDQwNSW50ZXJuZXQtRHJhZnQgICBBY3RpdmUgUXVldWUgTWFuYWdlbWVudCBSZWNv
bW1lbmRhdGlvbnMgICAgICAgSnVseSAyMDEzDQ0gICBbUkZDNjA1N10gIEJhc3RpYW4sIEMuLCBL
bGllYmVyLCBULiwgTGl2aW5nb29kLCBKLiwgTWlsbHMsIEouLCBhbmQgUi4NICAgICAgICAgICAg
ICBXb3VuZHksICJDb21jYXN0J3MgUHJvdG9jb2wtQWdub3N0aWMgQ29uZ2VzdGlvbiBNYW5hZ2Vt
ZW50DSAgICAgICAgICAgICAgU3lzdGVtIiwgUkZDIDYwNTcsIERlY2VtYmVyIDIwMTAuDQ0gICBb
UkZDNjgxN10gIFNoYWx1bm92LCBTLiwgSGF6ZWwsIEcuLCBJeWVuZ2FyLCBKLiwgYW5kIE0uIEt1
ZWhsZXdpbmQsDSAgICAgICAgICAgICAgIkxvdyBFeHRyYSBEZWxheSBCYWNrZ3JvdW5kIFRyYW5z
cG9ydCAoTEVEQkFUKSIsIFJGQyA2ODE3LA0gICAgICAgICAgICAgIERlY2VtYmVyIDIwMTIuDQ0g
ICBbV2lsbGluZ2VyOTVdDSAgICAgICAgICAgICAgV2lsbGluZ2VyLCBXLiwgVGFxcXUsIE0uLCBT
aGVybWFuLCBSLiwgV2lsc29uLCBELiwgYW5kIFYuDSAgICAgICAgICAgICAgSmFjb2Jzb24sICJT
ZWxmLVNpbWlsYXJpdHkgVGhyb3VnaCBIaWdoLVZhcmlhYmlsaXR5Og0gICAgICAgICAgICAgIFN0
YXRpc3RpY2FsIEFuYWx5c2lzIG9mIEV0aGVybmV0IExBTiBUcmFmZmljIGF0IHRoZSBTb3VyY2UN
ICAgICAgICAgICAgICBMZXZlbCIsIFNJR0NPTU0gU3ltcG9zaXVtIHByb2NlZWRpbmdzIG9uIENv
bW11bmljYXRpb25zDSAgICAgICAgICAgICAgYXJjaGl0ZWN0dXJlcyBhbmQgcHJvdG9jb2xzICwg
QXVndXN0IDE5OTUuDQ1BcHBlbmRpeCBBLiAgQ2hhbmdlIExvZw0NICAgSW5pdGlhbCBWZXJzaW9u
OiAgTWFyY2ggMjAxMw0NICAgTWlub3IgdXBoZSBhbGdvcml0aG1zIHRoYXQgdGhlIElFVEYgcmVj
b21tZW5kcyBTSE9VTEQgTk9UIHJlcXVpcmUNICAgb3BlcmF0aW9uYWwgKGVzcGVjaWFsbHkgbWFu
dWFsKSBjb25maWd1cmF0aW9uIG9yIHR1bmluZ2RhdGU6DSAgICAgIEFwcmlsIDIwMTMNDSAgIC0w
MjsgTWFqb3Igc3VyZ2VyeS4gIFRoaXMgZHJhZnQgaXMgZm9yIGRpc2N1c3Npb24gYXQgSUVURi04
NyBhbmQNICAgZXhwZWN0ZWQgdG8gYmUgZnVydGhlciB1cGRhdGVkLg0gICAgICBKdWx5IDIwMTMN
DUF1dGhvcnMnIEFkZHJlc3Nlcw0NICAgRnJlZCBCYWtlciAoZWRpdG9yKQ0gICBDaXNjbyBTeXN0
ZW1zICAgU2FudGEgQmFyYmFyYSwgQ2FsaWZvcm5pYSAgOTMxMTcNICAgVVNBDSAgIEVtYWlsOiBm
cmVkQGNpc2NvLmNvbQ0NICAgR29kcmVkIEZhaXJodXJzdCAoZWRpdG9yKQ0gICBVbml2ZXJzaXR5
IG9mIEFiZXJkZWVuDSAgIFNjaG9vbCBvZiBFbmdpbmVlcmluZw0gICBGcmFzZXIgTm9ibGUgQnVp
bGRpbmcNICAgQWJlcmRlZW4sIFNjb3RsYW5kICBBQjI0IDNVRSAgIFVLDSAgIEVtYWlsOiBnb3Jy
eUBlcmcuYWJkbi5hYy51aw0gICBVUkk6ICAgaHR0cDovL3d3dy5lcmcuYWJkbi5hYy51aw0NDQ0N
DUJha2VyICYgRmFpcmh1cnN0ICAgICAgIEV4cGlyZXMgSmFudWFyeSAxMCwgMjAxNCAgICAgICAg
ICAgICAgIFtQYWdlIDIwXQ0NAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAGAACgEAAA7hAAAPclAABFJgAAjzAAAN0wAACnOwAA9TsAAJpFAADsRQAAmU8AAORP
AADiVQAAFlYAAO9ZAABBWgAAcWUAAL9lAADibQAA420AAPxuAAD9bgAA/m4AAAJvAAAJbwAAD28A
ABlvAAAvbwAARm8AAEdvAABIbwAAY28AAGVvAABmbwAAFXAAAGNwAACFcQAAhnEAAIByAACBcgAA
+e757vnu+e757vnu+e757vnu+dj5zsS6sMSwprCcsJK6i/nu+XX5awAAAAAAAAAAAAATAQiBBEgB
AAVoYjobhxZow3D0ACsACIEVaMNw9AAWaLYdNgAXaFUAzABjSAEAZGgAAAAAZGgAAAAAZGjbMxtn
DBVow3D0ABZoth02AAATAQiBBEgBAAVo2TMbZxZoqhmhABMBCIEESAEABWhYMxtnFmi2HTYAEwEI
gQRIAQAFaBo0G2cWaM8zrQATAQiBBEgBAAVoVzMbZxZoth02ABMBCIEESAEABWhZMxtnFmi2HTYA
EwEIgQRIAQAFaNgzG2cWaKoZoQATAQiBBEgBAAVoVzMbZxZow3D0ACsACIEVaMNw9AAWaLYdNgAX
aHNMlABjSAEAZGgAAAAAZGgAAAAAZGhiOhuHFBVo1yGBABZow3D0AG1IDARzSAwEAAwVaMNw9AAW
aMNw9AAoAAYAAAEIAAACCAAAAwgAAAQIAABNCAAAlggAAN8IAAAoCQAAcQkAAHIJAABzCQAAswkA
AOgJAADpCQAA8gkAAPMJAAAzCgAAfAoAAMIKAAAHCwAATAsAAI8LAADXCwAAFgwAABcMAABcDAAA
kwwAAJQMAACoDAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TX
IYEAAB0ABgAA7tIAAP0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQE
AAEBAagMAACpDAAA6gwAAA4NAAAPDQAAVA0AAJYNAADeDQAAGw4AABwOAABlDgAArQ4AAO8OAAAt
DwAALg8AAGYPAABnDwAAeA8AAHkPAAC8DwAA5w8AAOgPAAApEAAAUhAAAFMQAABUEAAAVRAAAJ4Q
AACgEAAA6RAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGB
AAAd6RAAAOoQAADrEAAALhEAAG4RAAC3EQAA/xEAAEUSAACIEgAAtBIAALUSAADHEgAAyBIAABET
AABaEwAAoxMAAOwTAAA1FAAAfhQAAMcUAAAQFQAAUhUAAJsVAADgFQAAKRYAAGcWAACwFgAA+RYA
AEIXAACLFwAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEA
AB2LFwAA1BcAAB0YAABmGAAArxgAAPgYAABBGQAAihkAANMZAADUGQAA5RkAAOYZAADnGQAA6BkA
AOkZAADqGQAA6xkAAOwZAADtGQAA7hkAAO8ZAADwGQAA8RkAAPIZAADzGQAAPBoAAD4aAACHGgAA
iBoAAIkaAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAA
HYkaAADRGgAAFBsAAFobAACfGwAA5BsAACwcAABzHAAAtxwAAP4cAABFHQAAUx0AAFQdAACcHQAA
3h0AACYeAABtHgAAtB4AAPceAAA7HwAAgx8AAJAfAACRHwAA2h8AAB0gAABeIAAAnSAAAOUgAAAq
IQAAbiEAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAd
biEAALUhAAD8IQAAMSIAADIiAAB7IgAAvCIAAAIjAABHIwAAjyMAANEjAAAZJAAAXCQAAF0kAACN
JAAAjiQAANYkAAAbJQAAYyUAAKklAACqJQAAqyUAAKwlAAD1JQAA9yUAAEAmAABBJgAAQiYAAIsm
AADQJgAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB3Q
JgAAGCcAABknAABaJwAAoycAAOsnAAAtKAAAdCgAAL0oAAAGKQAAOSkAADopAAB/KQAArikAAK8p
AAD2KQAANioAAH8qAADEKgAABysAAFArAACKKwAAiysAANIrAAAaLAAAXCwAAJ8sAADJLAAAyiwA
AOYsAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAHeYs
AADnLAAALi0AAHYtAACzLQAAtC0AAN0tAADeLQAAJi4AAGouAACtLgAA8i4AADsvAACELwAAyS8A
ABIwAABAMAAAQTAAAEIwAABDMAAARDAAAI0wAACPMAAA2DAAANkwAADaMAAA6jAAAOswAAAzMQAA
eDEAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAdeDEA
AL4xAAACMgAAAzIAABYyAAAXMgAAXTIAAKIyAADoMgAALjMAAHEzAACEMwAAhTMAAMozAAAQNAAA
WTQAAKA0AADnNAAAJDUAAG01AACuNQAA8zUAADw2AACFNgAAzTYAAOA2AADhNgAAJzcAAGw3AAC0
NwAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB20NwAA
+DcAAD04AACEOAAAyzgAABM5AABZOQAAnTkAALU5AAC2OQAA+jkAAEI6AACKOgAA0joAABQ7AABZ
OwAAWjsAAFs7AABcOwAApTsAAKc7AADwOwAA8TsAAPI7AAA7PAAAgTwAAMU8AAAFPQAAGT0AABo9
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAHRo9AABZ
PQAAoT0AAOo9AAAwPgAAcj4AALg+AAD+PgAARz8AAI0/AACWPwAAlz8AANs/AAAJQAAACkAAAEVA
AABGQAAAiEAAAM1AAAARQQAAWkEAAKFBAADkQQAADEIAAA1CAABSQgAAmEIAAOFCAAApQwAAb0MA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAdb0MAALBD
AAD2QwAABEQAAAVEAABNRAAAlUQAAN5EAAAXRQAAGEUAAElFAABKRQAAS0UAAExFAABNRQAATkUA
AE9FAACYRQAAmkUAAONFAADkRQAA5UUAAC5GAABvRgAAs0YAAPlGAABARwAAT0cAAFBHAABvRwAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB1vRwAAcEcA
ALVHAAD9RwAARkgAAGNIAABkSAAArUgAAPNIAAA3SQAAfEkAAMVJAAAMSgAAVUoAAJpKAADfSgAA
J0sAAHBLAAC5SwAA0ksAANNLAAAcTAAAY0wAAKhMAADxTAAANU0AAHhNAAC/TQAABE4AAEtOAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAHUtOAACSTgAA
vE4AAL1OAAADTwAARU8AAEZPAABHTwAASE8AAElPAABKTwAAS08AAExPAABNTwAATk8AAJdPAACZ
TwAA4k8AAONPAADkTwAAAlAAAANQAABCUAAAhlAAAMpQAAAQUQAAV1EAAJhRAADgUQAAEVIAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAdEVIAABJSAABa
UgAAnFIAAOBSAAApUwAAbFMAAHZTAAB3UwAAwFMAAAlUAABSVAAAmFQAAN9UAADrVAAA7FQAAAZV
AAAHVQAAT1UAAJdVAADbVQAAHlYAAFFWAABSVgAAblYAAG9WAAC1VgAA/FYAAD9XAACCVwAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB2CVwAAy1cAAMxX
AAAVWAAAXFgAAJ9YAADlWAAAJVkAAG1ZAAChWQAAolkAAKNZAACkWQAA7VkAAO9ZAAA4WgAAOVoA
ADpaAACCWgAAyloAAAxbAABGWwAAjVsAANJbAAAYXAAAWVwAAJ1cAADkXAAAE10AABRdAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAHRRdAABAXQAAQV0A
AIddAADNXQAAD14AAFVeAABmXgAAZ14AAKdeAADrXgAAK18AAHNfAAC7XwAAA2AAAEhgAACOYAAA
12AAABZhAABdYQAAXmEAAKFhAADkYQAALWIAAHZiAAC1YgAA/GIAAEVjAACEYwAAhWMAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAdhWMAAM1jAAAUZAAA
WmQAAJRkAACVZAAA3WQAACNlAAAkZQAAJWUAACZlAABvZQAAcWUAALplAAC7ZQAAvGUAAARmAABH
ZgAAhWYAAMxmAAANZwAAVWcAAJdnAADJZwAAymcAABJoAABaaAAAomgAAOFoAAAqaQAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB0qaQAAb2kAAKtpAADu
aQAAN2oAAHhqAAC/agAABmsAAEtrAACRawAA2GsAABpsAAAbbAAAP2wAAEBsAACBbAAAymwAAA1t
AAAzbQAANG0AAHltAAC9bQAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAFb1tAADjbQAA5G0AACZu
AABobgAAem4AAHtuAAC9bgAA/W4AAGZvAABnbwAAqG8AAMVvAAC5AAAAAAAAAAAAAAAAtAAAAAAA
AAAAAAAAALQAAAAAAAAAAAAAAAC0AAAAAAAAAAAAAAAAtAAAAAAAAAAAAAAAALQAAAAAAAAAAAAA
AAC0AAAAAAAAAAAAAAAAtAAAAAAAAAAAAAAAAGwAAAAAAAAAAAAAAAC0AAAAAAAAAAAAAAAAtAAA
AAAAAAAAAAAAALQAAAAAAAAAAAAAAAAAAAAAAAAAAABHDwBDJAFFxoAAAAEAVzMbZwAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGdk
1yGBAAAEDwBnZNchgQBGDwBFxoAAAAEAVzMbZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGdk1yGBAAAMxW8AAMZvAADHbwAAyG8A
AMlvAADKbwAAE3AAABVwAABecAAAX3AAAGBwAACncAAAunAAALtwAAADcQAAR3EAAIZxAACHcQAA
yHEAAA5yAABUcgAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAtAAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAARg8ARcaAAAABAFozG2cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABnZNchgQAABA8AZ2TXIYEAABRUcgAAgXIAAIJyAABMcwAA
TXMAAJZzAADXcwAAGnQAAGB0AACmdAAA7nQAADd1AAB8dQAAu3UAALx1AAD0dQAA9XUAADt2AACD
dgAAynYAABN3AAD6AAAAAAAAAAAAAAAAsgAAAAAAAAAAAAAAALIAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABH
DwBDJAFFxoAAAAEAYjobhwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAGdk1yGBAAAEDwBnZNchgQAAFIFyAADJcgAA1nIAAN9yAABE
cwAAS3MAAExzAACTegAA4XoAAFd/AABbfwAARIUAAJKFAAD0igAA9YoAAC2LAABHiwAAy4sAAP2L
AAAHjAAAFIwAACGMAABEjAAARYwAAEuMAABMjAAAT4wAAGCMAABqjAAAdYwAAK+MAAD16/Xh19DJ
vsmoyb7JnpSKgHZsdmJ2YnZYYlhiTlgAABMBCIEESAEABWjFMxtnFmgjBKIAEwEIgQRIAQAFaMQz
G2cWaCMEogATAQiBBEgBAAVozTMbZxZoIwSiABMBCIEESAEABWjQMxtnFmiqGaEAEwEIgQRIAQAF
aMwzG2cWaCMEogATAQiBBEgBAAVoVzobhxZotkF2ABMBCIEESAEABWhWOhuHFmi2QXYAEwEIgQRI
AQAFaMszG2cWaCMEogATAQiBBEgBAAVoYjMbZxZow3D0ACsACIEVaMNw9AAWaMNw9AAXaLYdNgBj
SAEAZGgAAAAAZGgAAAAAZGhgMxtnFBVo1yGBABZow3D0AG1IDARzSAwEAAwVaMNw9AAWaMNw9AAA
DBVow3D0ABZoc0yUAAATAQiBBEgBAAVoZDobhxZoc0yUABMBCIEESAEABWhjOhuHFmhzTJQAEwEI
gQRIAQAFaLo6G4cWaMocsgATAQiBBEgBAAVoYjobhxZoc0yUAAAeE3cAADV3AAA2dwAAYXcAAGJ3
AACndwAA73cAABx4AAAdeAAAWXgAAFp4AAB9eAAAfngAALd4AAD3eAAA+HgAAEB5AACHeQAAx3kA
AAx6AABFegAARnoAAEd6AABIegAAkXoAAJN6AADcegAA3XoAAN56AAAgewAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB0gewAAZnsAAK57AAD1ewAAPnwA
AF98AABgfAAApnwAAO98AAA4fQAAf30AAMd9AAAQfgAAV34AAKB+AADifgAAJ38AADl/AAA6fwAA
gH8AAKd/AACofwAA538AACmAAABsgAAAsoAAAPmAAAA5gQAAXoEAAF+BAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAHV+BAABzgQAAdIEAALyBAAABggAA
SYIAAI6CAADUggAA7IIAAO2CAAAwgwAAeYMAAMCDAAAFhAAATYQAAI+EAADYhAAA9YQAAPaEAAD3
hAAA+IQAAPmEAABChQAARIUAAI2FAACOhQAAj4UAANWFAAAYhgAAYIYAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAdYIYAAJOGAACUhgAA24YAABuHAABk
hwAArIcAAPGHAAA4iAAAeogAAMOIAADYiAAA2YgAAB+JAABhiQAAYokAAKaJAACniQAA8IkAADWK
AAB7igAAuYoAAPWKAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAABb1igAA9ooAAMyLAADNiwAAK4wAACyM
AABLjAAAtwAAAAAAAAAAAAAAAG8AAAAAAAAAAAAAAABvAAAAAAAAAAAAAAAAbwAAAAAAAAAAAAAA
AG8AAAAAAAAAAAAAAABvAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARw8AQyQBRcaAAAABAMwzG2cAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABnZNch
gQAARw8AQyQBRcaAAAABAMszG2cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABnZNchgQAABkuMAABMjAAA8IwAAPGMAAC2jgAAtwAA
AAAAAAAAAAAAAGwAAAAAAAAAAAAAAAAkAAAAAAAAAAAAAAAAbAAAAAAAAAAAAAAAAABHDwBDJAFF
xoAAAAEAxjMbZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAGdk1yGBAEsPAAomAAtGAQBDJAFFxoAAAAEAzzMbZwAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGdkqhmh
AABHDwBDJAFFxoAAAAEAxDMbZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGdk1yGBAAAEr4wAALOMAADTjAAA2IwAAPCMAADxjAAA
8owAADiNAAA6jQAAPo0AAEKNAABMjQAAlo0AAJeNAADGjQAAyI0AAMmNAADKjQAA6o0AAPaNAAC0
jgAAtY4AALaOAAC3jgAA9pMAAEOUAADwngAAPp8AAImkAACKpAAA4KgAAOGoAADiqAAA46gAAPeo
AAD16/Xr4fXh9df11+HN4fXD9cO5w6/hqKGQoZCheqFwZlxmAAAAAAAAAAAAABMBCIEESAEABWjY
MxtnFmiqGaEAEwEIgQRIAQAFaNUzG2cWaKoZoQATAQiBBEgBAAVo1TMbZxZow3D0ACsACIEVaMNw
9AAWaMttDAAXaM8zrQBjSAEAZGgAAAAAZGgAAAAAZGgdNBtnIRVo1yGBABZow3D0AInKBwEBAPgy
G2eDKgFtSAwEc0gMBAwVaMNw9AAWaMNw9AAADBVow3D0ABZoIwSiAAATAQiBBEgBAAVozzMbZxZo
qhmhABMBCIEESAEABWjPMxtnFmgjBKIAEwEIgQRIAQAFaMkzG2cWaCMEogATAQiBBEgBAAVo3TMb
ZxZoVQDMABMBCIEESAEABWjLMxtnFmgjBKIAEwEIgQRIAQAFaMYzG2cWaCMEogATAQiBBEgBAAVo
xTMbZxZoIwSiABMBCIEESAEABWjOMxtnFmgjBKIAACK2jgAAt44AALiOAAD4jgAAFI8AABWPAABa
jwAAoI8AAOWPAAAlkAAAbJAAALSQAAD3kAAAI5EAACSRAABtkQAAtJEAAPeRAABAkgAAhpIAAM2S
AAC3AAAAAAAAAAAAAAAAsgAAAAAAAAAAAAAAALIAAAAAAAAAAAAAAACyAAAAAAAAAAAAAAAAsgAA
AAAAAAAAAAAAALIAAAAAAAAAAAAAAACyAAAAAAAAAAAAAAAAsgAAAAAAAAAAAAAAALIAAAAAAAAA
AAAAAACyAAAAAAAAAAAAAAAAsgAAAAAAAAAAAAAAALIAAAAAAAAAAAAAAACyAAAAAAAAAAAAAAAA
sgAAAAAAAAAAAAAAALIAAAAAAAAAAAAAAACyAAAAAAAAAAAAAAAAsgAAAAAAAAAAAAAAALIAAAAA
AAAAAAAAAACyAAAAAAAAAAAAAAAAsgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAA
Rw8AQyQBRcaAAAABAMczG2cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABnZNchgQAAFM2SAAAWkwAAF5MAAGCTAACokwAAqZMAAKqT
AACrkwAA9JMAAPaTAAA+lAAAP5QAAECUAACJlAAAz5QAABKVAABblQAAi5UAAIyVAADPlQAA6ZUA
AOqVAAAzlgAAcZYAALOWAAD2lgAAHpcAAB+XAABnlwAArZcAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAAAdrZcAAPGXAAAwmAAAdZgAAL2YAAD9mAAAPJkA
AIGZAAC4mQAAuZkAAAKaAABGmgAAj5oAANSaAAD5mgAA+poAADWbAAA2mwAAe5sAALmbAAC6mwAA
AZwAAEGcAACAnAAAyZwAABCdAABPnQAAmJ0AANmdAAAingAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAAAAAAABA8AZ2TXIYEAAB0ingAAa54AAKKeAACjngAApJ4AAKWeAADungAA
8J4AADmfAAA6nwAAO58AAIOfAADLnwAAC6AAAE6gAACVoAAA26AAAO6gAADvoAAAM6EAAHmhAADB
oQAAAqIAAEaiAACPogAAmaIAAJqiAADZogAAH6MAAGWjAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAAAAAAAEDwBnZNchgQAAHWWjAACqowAA7KMAAAWkAAAGpAAATqQAAIqkAACL
pAAAr6QAALCkAAD5pAAAQqUAAIqlAADLpQAAzKUAABSmAABapgAAn6YAAOWmAAAkpwAAJacAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAAtAAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEYPAEXGgAAAAQCu
MxtnAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAZ2TXIYEAAAQPAGdk1yGBAAAUJacAAGanAACspwAA9KcAADyoAACBqAAAyagAAOGo
AADiqAAA4KkAAOGpAAARqgAAEqoAABOqAAAUqgAAXaoAAF+qAACnqgAAqKoAAKmqAADgqgAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAALIAAAAAAAAAAAAAAACyAAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARw8AQyQBRcaAAAABANUz
G2cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAABnZNchgQAABA8AZ2TXIYEAABT3qAAA/KgAAFCpAADeqQAA36kAAOCpAABfqgAArKoA
ANKrAADUqwAA2qsAANurAADfqwAA7asAAO6rAADxqwAA86sAAB6sAAA4rAAAS6wAAFKsAABirAAA
hawAAIasAACHrAAAELMAAF2zAABVuwAAorsAAPTDAABBxAAAWMkAAPXr4dfQybjJrqSarpCuhpCu
fHKQ62iQYcm4ybjJuMkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwVaMNw9AAW
aMttDAAAEwEIgQRIAQAFaN8zG2cWaFUAzAATAQiBBEgBAAVo4DMbZxZoVQDMABMBCIEESAEABWjJ
MxtnFmgjBKIAEwEIgQRIAQAFaNQzG2cWaKoZoQATAQiBBEgBAAVo0zMbZxZoqhmhABMBCIEESAEA
BWjSMxtnFmiqGaEAEwEIgQRIAQAFaLAzG2cWaCMEogATAQiBBEgBAAVoyDMbZxZoIwSiACEVaNch
gQAWaMNw9ACJygcBAQD4MhtngyoBbUgMBHNIDAQMFWjDcPQAFmjDcPQAAAwVaMNw9AAWaKoZoQAA
EwEIgQRIAQAFaNgzG2cWaKoZoQATAQiBBEgBAAVo1zMbZxZoqhmhABMBCIEESAEABWjVMxtnFmiq
GaEAEwEIgQRIAQAFaFk6G4cWaLZBdgAAH+CqAADhqgAAJasAACarAABqqwAAfasAAH6rAADGqwAA
06sAANSrAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAtAAAAAAAAAAAAAAAAGwA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAABHDwBDJAFFxoAAAAEAyDMbZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGdk1yGBAEYPAEXGgAAAAQDIMxtn
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAZ2TXIYEAAAQPAGdk1yGBAAAJ1KsAAIesAACIrAAAzqwAAAOtAAAErQAAHK0AAB2tAABP
rQAAUK0AAGytAABtrQAAtq0AAPutAABDrgAAjK4AANWuAAAarwAAYq8AAKmvAACtAAAAAAAAAAAA
AAAAqAAAAAAAAAAAAAAAAKgAAAAAAAAAAAAAAACoAAAAAAAAAAAAAAAAqAAAAAAAAAAAAAAAAKgA
AAAAAAAAAAAAAACoAAAAAAAAAAAAAAAAqAAAAAAAAAAAAAAAAKgAAAAAAAAAAAAAAACoAAAAAAAA
AAAAAAAAqAAAAAAAAAAAAAAAAKgAAAAAAAAAAAAAAACoAAAAAAAAAAAAAAAAqAAAAAAAAAAAAAAA
AKgAAAAAAAAAAAAAAACoAAAAAAAAAAAAAAAAqAAAAAAAAAAAAAAAAKgAAAAAAAAAAAAAAACoAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAQPAGdk1yGBAFIPAEMkAUXGgAAAAQDIMxtnAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZ2QjBKIA
b8YHAQEAyDMbZ2QmAQ+E0AJehNACABOprwAAuq8AALuvAADWrwAA168AABSwAAAVsAAAKrAAACuw
AABysAAAtbAAAPywAABAsQAAiLEAAMyxAADksQAA5bEAAC6yAABzsgAAuLIAAMGyAADCsgAAw7IA
AMSyAADFsgAADrMAABCzAABYswAAWbMAAFqzAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAAAAAAAEDwBnZNchgQAAHVqzAACjswAA5LMAABa0AAAXtAAAJrQAACe0AABCtAAAQ7QA
AFG0AACYtAAA4bQAAAm1AAAKtQAATLUAAI21AACOtQAA1bUAABm2AAA9tgAAPrYAAIC2AAC7tgAA
vLYAAP22AABDtwAAarcAAGu3AAC0twAA+rcAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAAAAAAAAQPAGdk1yGBAAAd+rcAAA64AAAPuAAAVbgAAIe4AACIuAAAxrgAAPy4AAD9uAAA
RbkAAIy5AADEuQAAxbkAAOK5AADjuQAA8bkAADW6AAB9ugAAxroAAAe7AAAIuwAACbsAAAq7AABT
uwAAVbsAAJ27AACeuwAAn7sAAOa7AAArvAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAAAAAAABA8AZ2TXIYEAAB0rvAAAaLwAAGm8AACtvAAA7LwAACS9AAAlvQAANb0AAH29AADF
vQAA7L0AAO29AAAuvgAAdL4AAKi+AACpvgAAub4AAAC/AABHvwAAhL8AAIW/AACTvwAA2r8AAB7A
AABmwAAAesAAAHvAAACNwAAA08AAABrBAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAAAAAAAEDwBnZNchgQAAHRrBAABjwQAApcEAAKbBAADqwQAABcIAAAbCAABPwgAAY8IAAGTC
AACqwgAAzcIAAM7CAAAVwwAAOsMAADvDAACEwwAApsMAAKfDAACowwAAqcMAAPLDAAD0wwAAPMQA
AD3EAAA+xAAAe8QAAL/EAADAxAAAAcUAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAA
AAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAAAAAAAAQPAGdk1yGBAAAdAcUAAEjFAABnxQAAaMUAALDFAAD1xQAAO8YAAH/GAAC+xgAA7cYA
AO7GAAA1xwAAc8cAAHTHAACzxwAA9scAAD3IAABRyAAAUsgAAJnIAADZyAAAC8kAAAzJAABKyQAA
jMkAANHJAAACygAAA8oAAELKAACLygAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAAAAAAABA8AZ2TXIYEAAB1YyQAAmskAAOXMAAAyzQAAfNEAANHRAAA60gAAX9IAAO7SAADu59bn
xefu5wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAIRVo1yGBABZow3D0AInKBwEBAPgyG2eDKgFtSBAEc0gQBCEVaNchgQAWaMNw
9ACJygcBAQD4MhtngyoBbUgMBHNIDAQMFWjDcPQAFmjDcPQAACEVaNchgQAWaMNw9ACJygcBAQD4
MhtngyoBbUgHBHNIBwQACIvKAACMygAA08oAAPfKAAD4ygAAPssAAIfLAACrywAArMsAAOrLAAAc
zAAAHcwAAGPMAACVzAAAlswAAJfMAACYzAAAmcwAAJrMAADjzAAA5cwAAC3NAAAuzQAAL80AAHjN
AADBzQAA8c0AAPLNAAA5zgAAgs4AAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6
AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAA
AAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
AAAAAAQPAGdk1yGBAAAdgs4AAJ/OAACgzgAAsc4AAPnOAAA8zwAAhc8AAMvPAAAE0AAABdAAAB3Q
AAAe0AAAPtAAAD/QAACE0AAAxNAAANXQAADW0AAAGtEAAD3RAABN0QAATtEAAGHRAABi0QAAedEA
AInRAACt0QAAs9EAALTRAADM0QAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAA
AAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoA
AAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAA
AAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAA
APoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAA
AAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAA
AAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAAA
AAAABA8AZ2TXIYEAAB3M0QAAzdEAAM7RAADr0QAABdIAAB7SAAA30gAAVtIAAFvSAABc0gAAe9IA
AJ/SAACg0gAAodIAAKLSAACj0gAApNIAAO3SAADu0gAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAA
AAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAA
AAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAA
AAAAAAD6AAAAAAAAAAAAAAAA+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAA
+gAAAAAAAAAAAAAAAPoAAAAAAAAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAEDwBnZNchgQAAEjIAMZBoATpw1yGBAB+wgi4gsMZBIbCBBCKwgQQjkKAFJJCgBSWwAAAXsMQC
GLDEAgyQxAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAIYCEQASAAEAnAAPAAQAAAADAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEwAAEDx/wIATAAMAAAAAAAAAAAABgBOAG8AcgBtAGEA
bAAAAAIAAAAkAENKFgBPSgQAUUoEAF9IAQRhShgAbUgJBG5IEQRzSAkEdEgRBAAAAAAAAAAAAAAA
AAAAAAAAAEQAQQDy/6EARAAMAQAAAAAAAAAAFgBEAGUAZgBhAHUAbAB0ACAAUABhAHIAYQBnAHIA
YQBwAGgAIABGAG8AbgB0AAAAAABWAGkA8/+zAFYADAUAAAAAAAAAAAwAVABhAGIAbABlACAATgBv
AHIAbQBhAGwAAAAgADpWCwAX9gMAADTWBgABBQMAADTWBgABCgNsAGH2AwAAAgALAAAAKABrAPT/
wQAoAAAFAAAAAAAAAAAHAE4AbwAgAEwAaQBzAHQAAAACAAwAAAAAAEQAWkABAPIARAAMBAAAVnjR
AAAACgBQAGwAYQBpAG4AIABUAGUAeAB0AAAAAgAPABQAQ0oUAE9KBQBRSgUAXkoFAGFKFABIAJlA
AQACAUgADAUAANchgQAAAAwAQgBhAGwAbABvAG8AbgAgAFQAZQB4AHQAAAACABAAFABDShAAT0oG
AFFKBgBeSgYAYUoQAAAAAADuygAABgAAOgEAAAD/////AAAAAAEAAAACAAAAAwAAAAQAAABNAAAA
lgAAAN8AAAAoAQAAcQEAAHIBAABzAQAAswEAAOgBAADpAQAA8gEAAPMBAAAzAgAAfAIAAMICAAAH
AwAATAMAAI8DAADXAwAAFgQAABcEAABcBAAAkwQAAJQEAACoBAAAqQQAAOoEAAAOBQAADwUAAFQF
AACWBQAA3gUAABsGAAAcBgAAZQYAAK0GAADvBgAALQcAAC4HAABmBwAAZwcAAHgHAAB5BwAAvAcA
AOcHAADoBwAAKQgAAFIIAABTCAAAVAgAAFUIAACeCAAAoAgAAOkIAADqCAAA6wgAAC4JAABuCQAA
twkAAP8JAABFCgAAiAoAALQKAAC1CgAAxwoAAMgKAAARCwAAWgsAAKMLAADsCwAANQwAAH4MAADH
DAAAEA0AAFINAACbDQAA4A0AACkOAABnDgAAsA4AAPkOAABCDwAAiw8AANQPAAAdEAAAZhAAAK8Q
AAD4EAAAQREAAIoRAADTEQAA1BEAAOURAADmEQAA5xEAAOgRAADpEQAA6hEAAOsRAADsEQAA7REA
AO4RAADvEQAA8BEAAPERAADyEQAA8xEAADwSAAA+EgAAhxIAAIgSAACJEgAA0RIAABQTAABaEwAA
nxMAAOQTAAAsFAAAcxQAALcUAAD+FAAARRUAAFMVAABUFQAAnBUAAN4VAAAmFgAAbRYAALQWAAD3
FgAAOxcAAIMXAACQFwAAkRcAANoXAAAdGAAAXhgAAJ0YAADlGAAAKhkAAG4ZAAC1GQAA/BkAADEa
AAAyGgAAexoAALwaAAACGwAARxsAAI8bAADRGwAAGRwAAFwcAABdHAAAjRwAAI4cAADWHAAAGx0A
AGMdAACpHQAAqh0AAKsdAACsHQAA9R0AAPcdAABAHgAAQR4AAEIeAACLHgAA0B4AABgfAAAZHwAA
Wh8AAKMfAADrHwAALSAAAHQgAAC9IAAABiEAADkhAAA6IQAAfyEAAK4hAACvIQAA9iEAADYiAAB/
IgAAxCIAAAcjAABQIwAAiiMAAIsjAADSIwAAGiQAAFwkAACfJAAAySQAAMokAADmJAAA5yQAAC4l
AAB2JQAAsyUAALQlAADdJQAA3iUAACYmAABqJgAArSYAAPImAAA7JwAAhCcAAMknAAASKAAAQCgA
AEEoAABCKAAAQygAAEQoAACNKAAAjygAANgoAADZKAAA2igAAOooAADrKAAAMykAAHgpAAC+KQAA
AioAAAMqAAAWKgAAFyoAAF0qAACiKgAA6CoAAC4rAABxKwAAhCsAAIUrAADKKwAAECwAAFksAACg
LAAA5ywAACQtAABtLQAAri0AAPMtAAA8LgAAhS4AAM0uAADgLgAA4S4AACcvAABsLwAAtC8AAPgv
AAA9MAAAhDAAAMswAAATMQAAWTEAAJ0xAAC1MQAAtjEAAPoxAABCMgAAijIAANIyAAAUMwAAWTMA
AFozAABbMwAAXDMAAKUzAACnMwAA8DMAAPEzAADyMwAAOzQAAIE0AADFNAAABTUAABk1AAAaNQAA
WTUAAKE1AADqNQAAMDYAAHI2AAC4NgAA/jYAAEc3AACNNwAAljcAAJc3AADbNwAACTgAAAo4AABF
OAAARjgAAIg4AADNOAAAETkAAFo5AAChOQAA5DkAAAw6AAANOgAAUjoAAJg6AADhOgAAKTsAAG87
AACwOwAA9jsAAAQ8AAAFPAAATTwAAJU8AADePAAAFz0AABg9AABJPQAASj0AAEs9AABMPQAATT0A
AE49AABPPQAAmD0AAJo9AADjPQAA5D0AAOU9AAAuPgAAbz4AALM+AAD5PgAAQD8AAE8/AABQPwAA
bz8AAHA/AAC1PwAA/T8AAEZAAABjQAAAZEAAAK1AAADzQAAAN0EAAHxBAADFQQAADEIAAFVCAACa
QgAA30IAACdDAABwQwAAuUMAANJDAADTQwAAHEQAAGNEAACoRAAA8UQAADVFAAB4RQAAv0UAAARG
AABLRgAAkkYAALxGAAC9RgAAA0cAAEVHAABGRwAAR0cAAEhHAABJRwAASkcAAEtHAABMRwAATUcA
AE5HAACXRwAAmUcAAOJHAADjRwAA5EcAAAJIAAADSAAAQkgAAIZIAADKSAAAEEkAAFdJAACYSQAA
4EkAABFKAAASSgAAWkoAAJxKAADgSgAAKUsAAGxLAAB2SwAAd0sAAMBLAAAJTAAAUkwAAJhMAADf
TAAA60wAAOxMAAAGTQAAB00AAE9NAACXTQAA200AAB5OAABRTgAAUk4AAG5OAABvTgAAtU4AAPxO
AAA/TwAAgk8AAMtPAADMTwAAFVAAAFxQAACfUAAA5VAAACVRAABtUQAAoVEAAKJRAACjUQAApFEA
AO1RAADvUQAAOFIAADlSAAA6UgAAglIAAMpSAAAMUwAARlMAAI1TAADSUwAAGFQAAFlUAACdVAAA
5FQAABNVAAAUVQAAQFUAAEFVAACHVQAAzVUAAA9WAABVVgAAZlYAAGdWAACnVgAA61YAACtXAABz
VwAAu1cAAANYAABIWAAAjlgAANdYAAAWWQAAXVkAAF5ZAAChWQAA5FkAAC1aAAB2WgAAtVoAAPxa
AABFWwAAhFsAAIVbAADNWwAAFFwAAFpcAACUXAAAlVwAAN1cAAAjXQAAJF0AACVdAAAmXQAAb10A
AHFdAAC6XQAAu10AALxdAAAEXgAAR14AAIVeAADMXgAADV8AAFVfAACXXwAAyV8AAMpfAAASYAAA
WmAAAKJgAADhYAAAKmEAAG9hAACrYQAA7mEAADdiAAB4YgAAv2IAAAZjAABLYwAAkWMAANhjAAAa
ZAAAG2QAAD9kAABAZAAAgWQAAMpkAAANZQAAM2UAADRlAAB5ZQAAvWUAAONlAADkZQAAJmYAAGhm
AAB6ZgAAe2YAAL1mAAD9ZgAAZmcAAGdnAACoZwAAxWcAAMZnAADHZwAAyGcAAMlnAADKZwAAE2gA
ABVoAABeaAAAX2gAAGBoAACnaAAAumgAALtoAAADaQAAR2kAAIZpAACHaQAAyGkAAA5qAABUagAA
gWoAAIJqAABMawAATWsAAJZrAADXawAAGmwAAGBsAACmbAAA7mwAADdtAAB8bQAAu20AALxtAAD0
bQAA9W0AADtuAACDbgAAym4AABNvAAA1bwAANm8AAGFvAABibwAAp28AAO9vAAAccAAAHXAAAFlw
AABacAAAfXAAAH5wAAC3cAAA93AAAPhwAABAcQAAh3EAAMdxAAAMcgAARXIAAEZyAABHcgAASHIA
AJFyAACTcgAA3HIAAN1yAADecgAAIHMAAGZzAACucwAA9XMAAD50AABfdAAAYHQAAKZ0AADvdAAA
OHUAAH91AADHdQAAEHYAAFd2AACgdgAA4nYAACd3AAA5dwAAOncAAIB3AACndwAAqHcAAOd3AAAp
eAAAbHgAALJ4AAD5eAAAOXkAAF55AABfeQAAc3kAAHR5AAC8eQAAAXoAAEl6AACOegAA1HoAAOx6
AADtegAAMHsAAHl7AADAewAABXwAAE18AACPfAAA2HwAAPV8AAD2fAAA93wAAPh8AAD5fAAAQn0A
AER9AACNfQAAjn0AAI99AADVfQAAGH4AAGB+AACTfgAAlH4AANt+AAAbfwAAZH8AAKx/AADxfwAA
OIAAAHqAAADDgAAA2IAAANmAAAAfgQAAYYEAAGKBAACmgQAAp4EAAPCBAAA1ggAAe4IAALmCAAD1
ggAA9oIAAMyDAADNgwAAK4QAACyEAABLhAAATIQAAPCEAADxhAAAtoYAALeGAAC4hgAA+IYAABSH
AAAVhwAAWocAAKCHAADlhwAAJYgAAGyIAAC0iAAA94gAACOJAAAkiQAAbYkAALSJAAD3iQAAQIoA
AIaKAADNigAAFosAABeLAABgiwAAqIsAAKmLAACqiwAAq4sAAPSLAAD2iwAAPowAAD+MAABAjAAA
QYwAAIqMAADQjAAAE40AAFyNAACMjQAAjY0AANCNAADqjQAA640AADSOAAByjgAAtI4AAPeOAAAf
jwAAII8AAGiPAACujwAA8o8AADGQAAB2kAAAvpAAAP6QAAA9kQAAgpEAALmRAAC6kQAAA5IAAEeS
AACQkgAA1ZIAAPqSAAD7kgAANpMAADeTAAB8kwAAupMAALuTAAAClAAAQpQAAIGUAADKlAAAEZUA
AFCVAACZlQAA2pUAACOWAABslgAAo5YAAKSWAACllgAAppYAAO+WAADxlgAAOpcAADuXAAA8lwAA
hJcAAMyXAAAMmAAAT5gAAJaYAADcmAAA75gAAPCYAAA0mQAAepkAAMKZAAADmgAAR5oAAJCaAACa
mgAAm5oAANqaAAAgmwAAZpsAAKubAADtmwAABpwAAE6cAACKnAAAi5wAAIycAACwnAAAsZwAAPqc
AABDnQAAi50AAMydAADNnQAAFZ4AAFueAACgngAA5p4AACWfAAAmnwAAZ58AAK2fAAD1nwAAPaAA
AIKgAADhoAAA4qAAAOChAADhoQAA4qEAABKiAAATogAAFKIAABWiAABeogAAYKIAAKmiAACqogAA
q6IAAOKiAADjogAAJ6MAACijAABsowAAxqMAANOjAADUowAAh6QAAIikAADOpAAA0KQAAAWlAAAG
pQAAHqUAAB+lAABRpQAAUqUAAG6lAABvpQAAuKUAAP2lAABFpgAAjqYAANemAAAcpwAAZKcAAKun
AAC8pwAAvacAANinAADZpwAAFqgAABeoAAAsqAAALagAAHSoAAC3qAAA/qgAAEKpAACKqQAAzqkA
AOapAADnqQAAMKoAAHWqAAC6qgAAw6oAAMSqAADFqgAAxqoAAMeqAAAQqwAAEqsAAFurAABcqwAA
XasAAKarAADnqwAAGawAABqsAAAprAAAKqwAAEWsAABGrAAAVKwAAJusAADkrAAADK0AAA2tAABP
rQAAkK0AAJGtAADYrQAAHK4AAECuAABBrgAAg64AAL6uAAC/rgAAAK8AAEavAABtrwAAbq8AALev
AAD9rwAAEbAAABKwAABYsAAAirAAAIuwAADJsAAA/7AAAACxAABIsQAAj7EAAMexAADIsQAA5bEA
AOaxAAD0sQAAOLIAAICyAADJsgAACrMAAAuzAAAMswAADbMAAFazAABYswAAobMAAKKzAACjswAA
6rMAAC+0AABstAAAbbQAALG0AADwtAAAKLUAACm1AAA5tQAAgbUAAMm1AADwtQAA8bUAADK2AAB4
tgAArLYAAK22AAC9tgAABLcAAEu3AACItwAAibcAAJe3AADetwAAIrgAAGq4AAB+uAAAf7gAAJG4
AADXuAAAHrkAAGe5AACpuQAAqrkAAO65AAAJugAACroAAFO6AABnugAAaLoAAK66AADRugAA0roA
ABm7AAA+uwAAP7sAAIi7AACquwAAq7sAAKy7AACtuwAA9rsAAPi7AABBvAAAQrwAAEO8AACAvAAA
xLwAAMW8AAAGvQAATb0AAGy9AABtvQAAtb0AAPq9AABAvgAAhL4AAMO+AADyvgAA874AADq/AAB4
vwAAeb8AALi/AAD7vwAAQsAAAFbAAABXwAAAnsAAAN7AAAAQwQAAEcEAAE/BAACSwQAA18EAAAjC
AAAJwgAASMIAAJHCAACSwgAA2cIAAP3CAAD+wgAARMMAAI3DAACxwwAAssMAAPDDAAAixAAAI8QA
AGnEAACbxAAAnMQAAJ3EAACexAAAn8QAAKDEAADpxAAA68QAADTFAAA1xQAANsUAAH/FAADIxQAA
+MUAAPnFAABAxgAAicYAAKbGAACnxgAAuMYAAADHAABDxwAAjMcAANLHAAALyAAADMgAACTIAAAl
yAAARcgAAEbIAACLyAAAy8gAANzIAADdyAAAIckAAETJAABUyQAAVckAAGjJAABpyQAAgMkAAJHJ
AAC1yQAAvMkAAL3JAADWyQAA18kAANjJAAD1yQAAD8oAACjKAABBygAAYcoAAGfKAABoygAAh8oA
AKvKAACsygAArcoAAK7KAACvygAAsMoAAPDKAACYAAAADzAAAAAAAAABAAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAgAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAgA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAEACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAABAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAEACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAABAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAQAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAEACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAABAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAEACYgAEgDzAAAAAAAAAA
gAAAAIAAAAAAAAAAABAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAQAJiAASAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAEACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
AACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAAAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAIAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AAAQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAEACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAABAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAAAQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAAACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAAAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAAAAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAAAAAAAEAAAACAAAAAwAAAAQAAABNAAAAlgAA
AN8AAAAoAQAAcQEAAHIBAABzAQAAswEAAOgBAADpAQAA8gEAAPMBAAAzAgAAfAIAAMICAAAHAwAA
TAMAAI8DAADXAwAAFgQAABcEAABcBAAAkwQAAJQEAACoBAAAqQQAAOoEAAAOBQAADwUAAFQFAACW
BQAA3gUAABsGAAAcBgAAZQYAAK0GAADvBgAALQcAAC4HAABmBwAAZwcAAHgHAAB5BwAAvAcAAOcH
AADoBwAAKQgAAFIIAABTCAAAVAgAAFUIAACeCAAAoAgAAOkIAADqCAAA6wgAAC4JAABuCQAAtwkA
AP8JAABFCgAAiAoAALQKAAC1CgAAxwoAAMgKAAARCwAAWgsAAKMLAADsCwAANQwAAH4MAADHDAAA
EA0AAFINAACbDQAA4A0AACkOAABnDgAAsA4AAPkOAABCDwAAiw8AANQPAAAdEAAAZhAAAK8QAAD4
EAAAQREAAIoRAADTEQAA1BEAAOURAADmEQAA5xEAAOgRAADpEQAA6hEAAOsRAADsEQAA7REAAO4R
AADvEQAA8BEAAPERAADyEQAA8xEAADwSAAA+EgAAhxIAAIgSAACJEgAA0RIAABQTAABaEwAAnxMA
AOQTAAAsFAAAcxQAALcUAAD+FAAARRUAAFMVAABUFQAAnBUAAN4VAAAmFgAAbRYAALQWAAD3FgAA
OxcAAIMXAACQFwAAkRcAANoXAAAdGAAAXhgAAJ0YAADlGAAAKhkAAG4ZAAC1GQAA/BkAADEaAAAy
GgAAexoAALwaAAACGwAARxsAAI8bAADRGwAAGRwAAFwcAABdHAAAjRwAAI4cAADWHAAAGx0AAGMd
AACpHQAAqh0AAKsdAACsHQAA9R0AAPcdAABAHgAAQR4AAEIeAACLHgAA0B4AABgfAAAZHwAAWh8A
AKMfAADrHwAALSAAAHQgAAC9IAAABiEAADkhAAA6IQAAfyEAAK4hAACvIQAA9iEAADYiAAB/IgAA
xCIAAAcjAABQIwAAiiMAAIsjAADSIwAAGiQAAFwkAACfJAAAySQAAMokAADmJAAA5yQAAC4lAAB2
JQAAsyUAALQlAADdJQAA3iUAACYmAABqJgAArSYAAPImAAA7JwAAhCcAAMknAAASKAAAQCgAAEEo
AABCKAAAQygAAEQoAACNKAAAjygAANgoAADZKAAA2igAAOooAADrKAAAMykAAHgpAAC+KQAAAioA
AAMqAAAWKgAAFyoAAF0qAACiKgAA6CoAAC4rAABxKwAAhCsAAIUrAADKKwAAECwAAFksAACgLAAA
5ywAACQtAABtLQAAri0AAPMtAAA8LgAAhS4AAM0uAADgLgAA4S4AACcvAABsLwAAtC8AAPgvAAA9
MAAAhDAAAMswAAATMQAAWTEAAJ0xAAC1MQAAtjEAAPoxAABCMgAAijIAANIyAAAUMwAAWTMAAFoz
AABbMwAAXDMAAKUzAACnMwAA8DMAAPEzAADyMwAAOzQAAIE0AADFNAAABTUAABk1AAAaNQAAWTUA
AKE1AADqNQAAMDYAAHI2AAC4NgAA/jYAAEc3AACNNwAAljcAAJc3AADbNwAACTgAAAo4AABFOAAA
RjgAAIg4AADNOAAAETkAAFo5AAChOQAA5DkAAAw6AAANOgAAUjoAAJg6AADhOgAAKTsAAG87AACw
OwAA9jsAAAQ8AAAFPAAATTwAAJU8AADePAAAFz0AABg9AABJPQAASj0AAEs9AABMPQAATT0AAE49
AABPPQAAmD0AAJo9AADjPQAA5D0AAOU9AAAuPgAAbz4AALM+AAD5PgAAQD8AAE8/AABQPwAAbz8A
AHA/AAC1PwAA/T8AAEZAAABjQAAAZEAAAK1AAADzQAAAN0EAAHxBAADFQQAADEIAAFVCAACaQgAA
30IAACdDAABwQwAAuUMAANJDAADTQwAAHEQAAGNEAACoRAAA8UQAADVFAAB4RQAAv0UAAARGAABL
RgAAkkYAALxGAAC9RgAAA0cAAEVHAABGRwAAR0cAAEhHAABJRwAASkcAAEtHAABMRwAATUcAAE5H
AACXRwAAmUcAAOJHAADjRwAA5EcAAAJIAAADSAAAQkgAAIZIAADKSAAAEEkAAFdJAACYSQAA4EkA
ABFKAAASSgAAWkoAAJxKAADgSgAAKUsAAGxLAAB2SwAAd0sAAMBLAAAJTAAAUkwAAJhMAADfTAAA
60wAAOxMAAAGTQAAB00AAE9NAACXTQAA200AAB5OAABRTgAAUk4AAG5OAABvTgAAtU4AAPxOAAA/
TwAAgk8AAMtPAADMTwAAFVAAAFxQAACfUAAA5VAAACVRAABtUQAAoVEAAKJRAACjUQAApFEAAO1R
AADvUQAAOFIAADlSAAA6UgAAglIAAMpSAAAMUwAARlMAAI1TAADSUwAAGFQAAFlUAACdVAAA5FQA
ABNVAAAUVQAAQFUAAEFVAACHVQAAzVUAAA9WAABVVgAAZlYAAGdWAACnVgAA61YAACtXAABzVwAA
u1cAAANYAABIWAAAjlgAANdYAAAWWQAAXVkAAF5ZAAChWQAA5FkAAC1aAAB2WgAAtVoAAPxaAABF
WwAAhFsAAIVbAADNWwAAFFwAAFpcAACUXAAAlVwAAN1cAAAjXQAAJF0AACVdAAAmXQAAb10AAHFd
AAC6XQAAu10AALxdAAAEXgAAR14AAIVeAADMXgAADV8AAFVfAACXXwAAyV8AAMpfAAASYAAAWmAA
AKJgAADhYAAAKmEAAG9hAACrYQAA7mEAADdiAAB4YgAAv2IAAAZjAABLYwAAkWMAANhjAAAaZAAA
G2QAAD9kAABAZAAAgWQAAMpkAAANZQAAM2UAADRlAAB5ZQAAvWUAAONlAADkZQAAJmYAAGhmAAB6
ZgAAe2YAAL1mAAD9ZgAAZmcAAGdnAACoZwAAxWcAAMZnAADHZwAAyGcAAMlnAADKZwAAE2gAABVo
AABeaAAAX2gAAGBoAACnaAAAumgAALtoAAADaQAAR2kAAIZpAACHaQAAyGkAAA5qAABUagAAgWoA
AIJqAABMawAATWsAAJZrAADXawAAGmwAAGBsAACmbAAA7mwAADdtAAB8bQAAu20AALxtAAD0bQAA
9W0AADtuAACDbgAAym4AABNvAAA1bwAANm8AAGFvAABibwAAp28AAO9vAAAccAAAHXAAAFlwAABa
cAAAfXAAAH5wAAC3cAAA93AAAPhwAABAcQAAh3EAAMdxAAAMcgAARXIAAEZyAABHcgAASHIAAJFy
AACTcgAA3HIAAN1yAADecgAAIHMAAGZzAACucwAA9XMAAD50AABfdAAAYHQAAKZ0AADvdAAAOHUA
AH91AADHdQAAEHYAAFd2AACgdgAA4nYAACd3AAA5dwAAOncAAIB3AACndwAAqHcAAOd3AAApeAAA
bHgAALJ4AAD5eAAAOXkAAF55AABfeQAAc3kAAHR5AAC8eQAAAXoAAEl6AACOegAA1HoAAOx6AADt
egAAMHsAAHl7AADAewAABXwAAE18AACPfAAA2HwAAPV8AAD2fAAA93wAAPh8AAD5fAAAQn0AAER9
AACNfQAAjn0AAI99AADVfQAAGH4AAGB+AACTfgAAlH4AANt+AAAbfwAAZH8AAKx/AADxfwAAOIAA
AHqAAADDgAAA2IAAANmAAAAfgQAAYYEAAGKBAACmgQAAp4EAAPCBAAA1ggAAe4IAALmCAAD1ggAA
9oIAAMyDAADNgwAAK4QAACyEAABLhAAATIQAAPCEAADxhAAAtoYAALeGAAC4hgAA+IYAABSHAAAV
hwAAWocAAKCHAADlhwAAJYgAAGyIAAC0iAAA94gAACOJAAAkiQAAbYkAALSJAAD3iQAAQIoAAIaK
AADNigAAFosAABeLAABgiwAAqIsAAKmLAACqiwAAq4sAAPSLAAD2iwAAPowAAD+MAABAjAAAiYwA
AM+MAAASjQAAW40AAIuNAACMjQAAz40AAOmNAADqjQAAM44AAHGOAACzjgAA9o4AAB6PAAAfjwAA
Z48AAK2PAADxjwAAMJAAAHWQAAC9kAAA/ZAAADyRAACBkQAAuJEAALmRAAACkgAARpIAAI+SAADU
kgAA+ZIAAPqSAAA1kwAANpMAAHuTAAC5kwAAupMAAAGUAABBlAAAgJQAAMmUAAAQlQAAT5UAAJiV
AADZlQAAIpYAAGuWAACilgAAo5YAAKSWAACllgAA7pYAAPCWAAA5lwAAOpcAADuXAACDlwAAy5cA
AAuYAABOmAAAlZgAANuYAADumAAA75gAADOZAAB5mQAAwZkAAAKaAABGmgAAj5oAAJmaAACamgAA
2ZoAAB+bAABlmwAAqpsAAOybAAAFnAAABpwAAE6cAACKnAAAi5wAAK+cAACwnAAA+ZwAAEKdAACK
nQAAy50AAMydAAAUngAAWp4AAJ+eAADlngAAJJ8AACWfAABmnwAArJ8AAPSfAAA8oAAAgaAAAMmg
AADhoAAA4qAAAOChAADhoQAAEaIAABKiAAATogAAFKIAAF2iAABfogAAp6IAAKiiAACpogAA4KIA
AOGiAAAlowAAJqMAAGqjAAB9owAAfqMAAMajAADTowAA1KMAAIekAACIpAAAzqQAAAOlAAAEpQAA
HKUAAB2lAABPpQAAUKUAAGylAABtpQAAtqUAAPulAABDpgAAjKYAANWmAAAapwAAYqcAAKmnAAC6
pwAAu6cAANanAADXpwAAFKgAABWoAAAqqAAAK6gAAHKoAAC1qAAA/KgAAECpAACIqQAAzKkAAOSp
AADlqQAALqoAAHOqAAC4qgAAwaoAAMKqAADDqgAAxKoAAMWqAAAOqwAAEKsAAFirAABZqwAAWqsA
AKOrAADkqwAAFqwAABesAAAmrAAAJ6wAAPDKAACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIgBmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgAGYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIgA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAkAGYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAJABmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAZgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAA
AACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAA
AIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAA
gAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACA
AAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAA
AAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAA
AAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAA
AAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAA
AAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAA
AAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAA
AACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAA
AIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAA
gACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACA
AJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAA
mAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACY
AAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgA
AAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAA
AA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAA
DzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAP
MAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8w
AAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAA
AAAAAAAAgAAAAIAAAAAAAAAAAIAAmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACAAJgAAAAPMAAA
AAAAAACAAAAAgAAAAAAAAAAAgACYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAIABmAAAAA8wAAAA
AAAAAIAAAACAAAAAAAAAAACQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAkACYAAAADzAAAAAA
AAAAgAAAAIAAAAAAAAAAAJABmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACQAJgAAAAPMAAAAAAA
AACAAAAAgAAAAAAAAAAAkAGYAAAADzAAAAAAAAAAgAAAAIAAAAAAAAAAAJAAmAAAAA8wAAAAAAAA
AIAAAACAAAAAAAAAAACQAJgAAAAPMAAAAAAAAACAAAAAgAAAAAAAAAAAkACYAAAADzAAAAAAAAAA
gAAAAIAAAAAAAAAAAJABmAAAAA8wAAAAAAAAAIAAAACAAAAAAAAAAACQAJgAAAAPMAAAAAAAAACA
AAAAgAAAAAAAAAAAkAGd+SUwAAAAAAAAAAABAAAAIQAAAAAAAAAAAIABnfklMAAAAAAAAAAAAQAA
ACAAAAAAAAAAAACAAZ35JTAAAAAAAAAAAAEAAAAfAAAAAAAAAAAAgAGd+SUwAAAAAAAAAAABAAAA
HgAAAAAAAAAAAIABnfklMAAAAAAAAAAAAQAAAB0AAAAAAAAAAACAAZ35JTAAAAAAAAAAAAEAAAAc
AAAAAAAAAAAAgAGd+SUwAAAAAAAAAAABAAAAGwAAAAAAAAAAAIABnfklMAAAAAAAAAAAAQAAABoA
AAAAAAAAAACAAZ35JTAAAAAAAAAAAAEAAAAZAAAAAAAAAAAAgAGd+SUwAAAAAAAAAAABAAAAGAAA
AAAAAAAAAAABnfklMAAAAAAAAAAAAQAAABcAAAAAAAAAAAAAAZ35JTAAAAAAAAAAAAEAAAAWAAAA
AAAAAAAAAAGd+SUwAAAAAAAAAAABAAAAFQAAAAAAAAAAAAABnfklMAAAAAAAAAAAAQAAABQAAAAA
AAAAAAAAAZ35JTAAAAAAAAAAAAEAAAATAAAAAAAAAAAAAAGd+SUwAAAAAAAAAAABAAAAEgAAAAAA
AAAAAAABnfklMAAAAAAAAAAAAQAAABEAAAAAAAAAAAAAAZ35JTAAAAAAAAAAAAEAAAAQAAAAAAAA
AAAAAAGd+SUwAAAAAAAAAAABAAAADwAAAAAAAAAAAAABnfklMAAAAAAAAAAAAQAAAA4AAAAAAAAA
AAAAAZ35JTAAAAAAAAAAAAEAAAANAAAAAAAAAAAAAAGd+SUwAAAAAAAAAAABAAAADAAAAAAAAAAA
AAABnfklMAAAAAAAAAAAAQAAAAsAAAAAAAAAAAAAAZ35JTAAAAAAAAAAAAEAAAAKAAAAAAAAAAAA
AAGd+SUwAAAAAAAAAAABAAAACQAAAAAAAAAAAAABnfklMAAAAAAAAAAAAQAAAAgAAAAAAAAAAAAA
AZ35JTAAAAAAAAAAAAEAAAAGAAAAAAAAAAAAAAGd+SUwAAAAAAAAAAABAAAABQAAAAAAAAAAAAAB
nfklMAAAAAAAAAAAAQAAAAQAAAAAAAAAAAAAAZ35JTAAAAAAAAAAAAIAAAACAAAAAAAAAAAAAAGY
AAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAIABnfklMAAQAAAAAAAAAQAAAAAAAAAAAAAAAACAAZ35
JTDmAQAAAAAAAAEAAABFAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAARAAAAAAAAAAAAIABnfkl
MOYBAAAAAAAAAQAAAEMAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAABCAAAAAAAAAAAAgAGd+SUw
5gEAAAAAAAABAAAAQQAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAAEAAAAAAAAAAAACAAZ35JTDm
AQAAAAAAAAEAAAA/AAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAPgAAAAAAAAAAAIABnfklMOYB
AAAAAAAAAQAAAD0AAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAA8AAAAAAAAAAAAgAGd+SUw5gEA
AAAAAAABAAAAOwAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAADoAAAAAAAAAAACAAZ35JTDmAQAA
AAAAAAEAAAA5AAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAOAAAAAAAAAAAAIABnfklMOYBAAAA
AAAAAQAAADcAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAA2AAAAAAAAAAAAgAGd+SUw5gEAAAAA
AAABAAAANQAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAADQAAAAAAAAAAACAAZ35JTDmAQAAAAAA
AAEAAAAzAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAMgAAAAAAAAAAAIABnfklMOYBAAAAAAAA
AQAAADEAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAwAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAAB
AAAALwAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAAC4AAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEA
AAAtAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAALAAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAA
ACsAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAqAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAA
KQAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAACgAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAn
AAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAJgAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAACUA
AAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAkAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAIwAA
AAAAAAAAAIABnfklMOYBAAAAAAAAAQAAACIAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAhAAAA
AAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAIAAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAAB8AAAAA
AAAAAACAAZ35JTDmAQAAAAAAAAEAAAAeAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAHQAAAAAA
AAAAAIABnfklMOYBAAAAAAAAAQAAABwAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAbAAAAAAAA
AAAAgAGd+SUw5gEAAAAAAAABAAAAGgAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAABkAAAAAAAAA
AACAAZ35JTDmAQAAAAAAAAEAAAAYAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAFwAAAAAAAAAA
AIABnfklMOYBAAAAAAAAAQAAABYAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAVAAAAAAAAAAAA
gAGd+SUw5gEAAAAAAAABAAAAFAAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAABMAAAAAAAAAAACA
AZ35JTDmAQAAAAAAAAEAAAASAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAAEQAAAAAAAAAAAIAB
nfklMOYBAAAAAAAAAQAAABAAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAPAAAAAAAAAAAAgAGd
+SUw5gEAAAAAAAABAAAADgAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAAA0AAAAAAAAAAACAAZ35
JTDmAQAAAAAAAAEAAAAMAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAACwAAAAAAAAAAAIABnfkl
MOYBAAAAAAAAAQAAAAoAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAEAAAAJAAAAAAAAAAAAgAGd+SUw
5gEAAAAAAAABAAAACAAAAAAAAAAAAIABnfklMOYBAAAAAAAAAQAAAAcAAAAAAAAAAACAAZ35JTDm
AQAAAAAAAAEAAAAGAAAAAAAAAAAAgAGd+SUw5gEAAAAAAAABAAAABQAAAAAAAAAAAIABnfklMOYB
AAAAAAAAAQAAAAQAAAAAAAAAAACAAZ35JTDmAQAAAAAAAAIAAAACAAAAAAAAAAAAgAGYAAAAAAAA
AAAAAAAAgAAAAIAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAGUAAAAAAAAAAACAAZ35JTAqAgAA
AAAAAAEAAABkAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAYwAAAAAAAAAAAIABnfklMCoCAAAA
AAAAAQAAAGIAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABhAAAAAAAAAAAAgAGd+SUwKgIAAAAA
AAABAAAAYAAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAF8AAAAAAAAAAACAAZ35JTAqAgAAAAAA
AAEAAABeAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAXQAAAAAAAAAAAIABnfklMCoCAAAAAAAA
AQAAAFwAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABbAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAAB
AAAAWgAAAAAAAAAAAIABnfklMCoSAAAAAAAAAQAAAFkAAAAAAAAAAACIAZ35JTAqAgAAAAAAAAEA
AABYAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAVwAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAA
AFYAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABVAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAA
VAAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAFMAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABS
AAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAUQAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAFAA
AAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABPAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAATgAA
AAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAE0AAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABMAAAA
AAAAAAAAgAGd+SUwKgIAAAAAAAABAAAASwAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAEoAAAAA
AAAAAACAAZ35JTAqAgAAAAAAAAEAAABJAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAASAAAAAAA
AAAAAIABnfklMCoCAAAAAAAAAQAAAEcAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABGAAAAAAAA
AAAAgAGd+SUwKgIAAAAAAAABAAAARQAAAAAAAAAAAIABnfklMCoSAAAAAAAAAQAAAEQAAAAAAAAA
AACQAZ35JTAqEgAAAAAAAAEAAABEAAAAAAAAAAAAkAGYAAAADzAAAAAAAAABAAAARAAAAAAAAAAA
AIABnfklMCoCAAAAAAAAAQAAAEMAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAABCAAAAAAAAAAAA
gAGd+SUwKgIAAAAAAAABAAAAQQAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAEAAAAAAAAAAAACA
AZ35JTAqAgAAAAAAAAEAAAA/AAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAPgAAAAAAAAAAAIAB
nfklMCoCAAAAAAAAAQAAAD0AAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAA8AAAAAAAAAAAAgAGd
+SUwKgIAAAAAAAABAAAAOwAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAADoAAAAAAAAAAACAAZ35
JTAqAgAAAAAAAAEAAAA5AAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAOAAAAAAAAAAAAIABnfkl
MCoCAAAAAAAAAQAAADcAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAA2AAAAAAAAAAAAgAGd+SUw
KgIAAAAAAAABAAAANQAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAADQAAAAAAAAAAACAAZ35JTAq
AgAAAAAAAAEAAAAzAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAMgAAAAAAAAAAAIABnfklMCoC
AAAAAAAAAQAAADEAAAAAAAAAAACQAZ35JTAqAgAAAAAAAAEAAAAxAAAAAAAAAAAAkAGd+SUwKjIA
AAAAAAABAAAAMQAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAADAAAAAAAAAAAACAAZ35JTAqAgAA
AAAAAAEAAAAvAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAALgAAAAAAAAAAAIABnfklMCoCAAAA
AAAAAQAAAC0AAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAsAAAAAAAAAAAAgAGd+SUwKgIAAAAA
AAABAAAAKwAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAACoAAAAAAAAAAACAAZ35JTAqAgAAAAAA
AAEAAAApAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAKAAAAAAAAAAAAIABnfklMCoCAAAAAAAA
AQAAACcAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAmAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAAB
AAAAJQAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAACQAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEA
AAAjAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAIgAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAA
ACEAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAgAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAA
HwAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAB4AAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAd
AAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAHAAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAABsA
AAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAaAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAGQAA
AAAAAAAAAIABnfklMCoCAAAAAAAAAQAAABgAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAXAAAA
AAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAFgAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAABUAAAAA
AAAAAACAAZ35JTAqAgAAAAAAAAEAAAAUAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAAEwAAAAAA
AAAAAIABnfklMCoCAAAAAAAAAQAAABIAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAARAAAAAAAA
AAAAgAGd+SUwKgIAAAAAAAABAAAAEAAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAA8AAAAAAAAA
AACAAZ35JTAqAgAAAAAAAAEAAAAOAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAADQAAAAAAAAAA
AIABnfklMCoCAAAAAAAAAQAAAAwAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAALAAAAAAAAAAAA
gAGd+SUwKgIAAAAAAAABAAAACgAAAAAAAAAAAIABnfklMCoCAAAAAAAAAQAAAAkAAAAAAAAAAACA
AZ35JTAqAgAAAAAAAAEAAAAIAAAAAAAAAAAAgAGd+SUwKgIAAAAAAAABAAAABwAAAAAAAAAAAIAB
nfklMCoCAAAAAAAAAQAAAAYAAAAAAAAAAACAAZ35JTAqAgAAAAAAAAEAAAAFAAAAAAAAAAAAgAGd
+SUwKgIAAAAAAAABAAAABAAAAAAAAAAAAIABnfklMCoCAAAAAAAAAgAAAAIAAAAAAAAAAACAAZgA
AAAAAAAAAAAAAACAAAAAgAAAAAAAAAAAgAGd+SUwlwMAAAAAAAABAAAABwAAAAAAAAAAAIABnfkl
MJcDAAAAAAAAAQAAAAYAAAAAAAAAAACAAZ35JTCXAwAAAAAAAAEAAAAFAAAAAAAAAAAAgAGd+SUw
lwMAAAAAAAABAAAABAAAAAAAAAAAAIABnfklMJcDAAAAAAAAAgAAAAIAAAAAAAAAAACAAZgAAAAA
AAAAAAAAAACAAAAAgAAAAAAAAAAAgAGd+SUwABAAAAAAAAABAAAAAAAAAAAAAAAAAKIHAAYAAIFy
AACvjAAA96gAAFjJAADu0gAAagAAAIIAAACJAAAAkAAAAJkAAAAABgAAqAwAAOkQAACLFwAAiRoA
AG4hAADQJgAA5iwAAHgxAAC0NwAAGj0AAG9DAABvRwAAS04AABFSAACCVwAAFF0AAIVjAAAqaQAA
vW0AAMVvAABUcgAAE3cAACB7AABfgQAAYIYAAPWKAABLjAAAto4AAM2SAACtlwAAIp4AAGWjAAAl
pwAA4KoAANSrAACprwAAWrMAAPq3AAArvAAAGsEAAAHFAACLygAAgs4AAMzRAADu0gAAawAAAG0A
AABuAAAAbwAAAHAAAABxAAAAcgAAAHMAAAB0AAAAdQAAAHYAAAB3AAAAeAAAAHkAAAB6AAAAewAA
AHwAAAB9AAAAfgAAAH8AAACAAAAAgQAAAIMAAACEAAAAhQAAAIYAAACHAAAAiAAAAIoAAACLAAAA
jAAAAI0AAACOAAAAjwAAAJEAAACSAAAAkwAAAJQAAACVAAAAlgAAAJcAAACYAAAAmgAAAJsAAACc
AAAAAAYAAO7SAABsAAAADwAA8DgAAAAAAAbwGAAAAAIIAAACAAAAAQAAAAEAAAABAAAAAgAAAEAA
HvEQAAAA//8AAAAA/wCAgIAA9wAAEAAPAALwkgAAABAACPAIAAAAAQAAAAEEAAAPAAPwMAAAAA8A
BPAoAAAAAQAJ8BAAAAAAAAAAAAAAAAAAAAAAAAAAAgAK8AgAAAAABAAABQAAAA8ABPBCAAAAEgAK
8AgAAAABBAAAAA4AAFMAC/AeAAAAvwEAABAAywEAAAAA/wEAAAgABAMJAAAAPwMBAAEAAAAR8AQA
AAABAAAA//8SAAAABgAkyaViEAABAPx9WA0GACXJpWIQAAEA1E7rCwYAJsmlYhAAAQDcfloNBgAn
yaViEQABAJS74AsGACjJpWIRAAEA9NfgCwYAKcmlYhAAAQDMxhoABgAqyaViEQABACwvGgAGACvJ
pWIQAAEApE/fCwYALMmlYhAAAQA0dFoNBgAtyaViEQABAExLVA0GAC7JpWIRAAEAJDUXAAYAL8ml
YhAAAQB0yhsABgAwyaViEQABAOQ0FwAGADHJpWIRAAEAlB/nCwYAMsmlYhAAAQAMyRoABgAzyaVi
EQABAMzFGgAGADTJpWIRAAEATMUaAAYANcmlYhEAAQAMxhoAJrEAAB6yAACyuAAAsrgAAMC4AACh
vQAAob0AAKvAAAD4yQAA+MkAAAbKAAASygAAEsoAABzKAAArygAAK8oAADLKAAA4ygAA8MoAAAAA
AAABAAEAAAABAAMAAAACAAIAAAACAAQAAAACAAUAAAACAAYAAAACAAcAAAABAAkAAAACAAgAAAAC
AAoAAAACAAwAAAACAAsAAAACAA0AAAACABAAAAACAA4AAAACAA8AAAACABEAAAACACixAAAosgAA
vLgAAMm4AADJuAAApr0AAKa9AACzwAAAAsoAAA7KAAAOygAAGMoAACfKAAAnygAAMcoAADfKAABA
ygAAQMoAAPDKAAAAAAAAAQAAAAMAAQACAAAABAAAAAUAAAAGAAAABwAAAAkAAQAIAAAACgAAAAwA
AQALAAAADQAAAA8AAQAQAAEADgAAABEAAAAEAAAAOAAAAAwAAAAqgHVybjpzY2hlbWFzLW1pY3Jv
c29mdC1jb206b2ZmaWNlOnNtYXJ0dGFncwSAQ2l0eQCAPQAAAA4AAAAqgHVybjpzY2hlbWFzLW1p
Y3Jvc29mdC1jb206b2ZmaWNlOnNtYXJ0dGFncwmAUGxhY2VOYW1lAIA9AAAADwAAACqAdXJuOnNj
aGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6c21hcnR0YWdzCYBQbGFjZVR5cGUAgDkAAAASAAAA
KoB1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTpzbWFydHRhZ3MFgHBsYWNlAIAMAAAB
HANbDwAAAAASAAAAAAASAAAAAAASAAAAAAAPAAAAAAAOAAAAAAASAAAAAAAMAAAAAAASAAAAAAAS
AAAAAAAPAAAAAAAOAAAAAAASAAAAAAAPAAAAAAAOAAAAAAASAAAAAAAOAAAAAAAOAAAAAAAPAAAA
AAAAAAAA0AAAANkAAACJBgAAkgYAAF0IAABmCAAAqQgAAK4IAADJCAAA2AgAAIUOAACPDgAA+xEA
AAQSAAAkGgAALxoAALQdAAC9HQAAAB4AAAUeAAAgHgAALx4AAGUjAABpIwAATCgAAFUoAACYKAAA
nSgAALgoAADHKAAAZDMAAG0zAACwMwAAtTMAANAzAADfMwAAVz0AAGA9AACjPQAAqD0AAMM9AADS
PQAAZ0MAAG9DAABWRwAAX0cAAKJHAACnRwAAwkcAANFHAABQUAAAU1AAAKxRAAC1UQAA+FEAAP1R
AAAYUgAAJ1IAAE1TAABeUwAALl0AADddAAB6XQAAf10AAJpdAACpXQAAQWcAAEVnAADSZwAA22cA
AB5oAAAjaAAAPmgAAE1oAACuaAAAuGgAANZqAADZagAAD20AABttAAAAcAAACnAAADdxAAA/cQAA
UHIAAFlyAACccgAAoXIAALxyAADLcgAAIXsAACp7AABsewAAdXsAAJF7AACaewAAzXsAANZ7AABr
fAAAdHwAAAF9AAAKfQAATX0AAFJ9AABtfQAAfH0AAOSDAADogwAARIQAAEmEAABxhwAAfocAAAyI
AAAUiAAAEIsAABOLAACCiwAAjIsAALOLAAC8iwAA/4sAAASMAAAfjAAALowAAIWNAACIjQAA3o0A
AOiNAABblAAAZJQAAK2WAAC2lgAA+ZYAAP6WAAAZlwAAKJcAAEmaAABRmgAAIaEAACWhAAAcogAA
JaIAAGiiAABtogAAiKIAAJeiAAAFpAAACaQAALioAADBqAAA1qgAAN2oAADnqAAA7agAABKpAAAa
qQAAQ6kAAE+pAABXqQAAXqkAAGWpAABvqQAAdakAAHqpAAAeqgAAJaoAAKCqAACtqgAAu6oAAL+q
AADNqgAA1qoAABmrAAAeqwAAOasAAEirAABdqwAAYqsAAGOrAABsqwAAvKsAAMWrAADQrAAA36wA
AO+sAADyrAAAGK0AAB+tAAA7rQAAP60AAJytAACorQAAXK4AAF+uAAB5rwAAf68AAIuvAACUrwAA
HbAAACOwAAApsAAAL7AAAKSwAACusAAAC7EAABWxAABasQAAYrEAAAuyAAARsgAAHrIAACWyAABY
sgAAYLIAABKzAAAbswAAXrMAAGOzAAB+swAAjbMAAAe2AAATtgAAJLYAACe2AADWtgAA37YAAHO3
AAB7twAArbcAALK3AAC4twAAwbcAAH+4AACLuAAASbkAAFC5AAC0uQAAurkAABS6AAAaugAAcroA
AHi6AACxuwAAursAAP27AAACvAAAHbwAACy8AADsvAAA87wAAI29AACWvQAAp70AAK69AADCvQAA
yL0AAOe9AADvvQAAIL4AACy+AAAyvgAAOb4AAE2+AABXvgAA/L4AAAO/AAAPvwAAFb8AADDBAAA5
wQAAWMEAAIzBAACswQAAssEAAB7DAAAkwwAAMcMAADfDAAC6wwAAwcMAACvEAAAxxAAAN8QAAD3E
AACixAAAq8QAAO7EAADzxAAADsUAAB3FAABKxQAAUcUAAFfFAABgxQAAAMYAAAjGAAAZxgAAIMYA
AC3GAAA3xgAAv8YAAMjGAADOxgAA08YAALjIAADCyAAAgskAAInJAACMyQAArckAALDJAACzyQAA
t8kAAMzJAADRyQAA18kAANjJAADhyQAARMoAAGHKAABkygAAZ8oAAPDKAAAHABwABwAcAAcAHAAH
ABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcA
HAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAc
AAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcABAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwA
BwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAH
ABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcA
HAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAc
AAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwA
BwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAH
ABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcA
HAAHABwABwAcAAcAHAAHAAUABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAc
AAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHAAUABwAFAAcABQAHAAUABwAcAAcAHAAHAAUA
BwAFAAcAAAAAAJYAAADeAAAAxgEAAOcBAAA2AgAAQAIAAH8CAACHAgAAxQIAAM8CAAAKAwAAEQMA
AE8DAABUAwAAkgMAAJwDAADaAwAA3QMAAF8EAABkBAAA7QQAAPcEAACZBQAAoAUAAOEFAADqBQAA
aAYAAGsGAACwBgAAtAYAAPIGAAD6BgAAvwcAAMcHAAAxCQAAPAkAAHEJAAB6CQAAugkAALwJAAAC
CgAACQoAAEgKAABLCgAAiwoAAJQKAAAZCwAAKAsAAGoLAABtCwAAPQwAAEsMAACGDAAAkgwAANMM
AADZDAAAGA0AAB4NAABdDQAAYw0AAKMNAACpDQAA6w0AAPYNAAAxDgAANw4AAHIOAAB7DgAAuA4A
AMcOAAABDwAABw8AALcQAADDEAAAABEAAA4RAABEEQAATxEAANQSAADaEgAAFxMAADMTAABdEwAA
YxMAAOcTAADvEwAALxQAADgUAAB2FAAAfRQAALoUAAC/FAAAARUAAAQVAABIFQAAUBUAAOEVAADr
FQAAcBYAAHsWAAC3FgAAuhYAAPoWAAABFwAAJhcAACsXAAA+FwAARxcAAIYXAACPFwAA3RcAAOEX
AAAgGAAAJRgAAGEYAABrGAAAoBgAAKoYAADoGAAA7RgAAC0ZAAAyGQAAcRkAAHgZAAD/GQAABhoA
AH4aAACAGgAAvxoAAMkaAAAFGwAACxsAAEobAABOGwAAkhsAAJcbAADUGwAA3hsAABwcAAAfHAAA
2RwAAN0cAAAeHQAAJh0AAEUeAABJHgAAjh4AAJAeAADTHgAA2B4AAF0fAABmHwAAph8AAKofAADu
HwAA+R8AADAgAAA7IAAAdyAAAH8gAADAIAAAySAAAAkhAAALIQAAgiEAAK0hAAD5IQAA+yEAADki
AABDIgAAgiIAAIYiAADHIgAAzCIAAAojAAATIwAAUyMAAFQjAAAdJAAAQSQAAF8kAABpJAAAoiQA
AMgkAADNJAAA3CQAAHklAACBJQAAwSUAAMQlAAApJgAALyYAAG0mAAByJgAAsCYAALcmAAD1JgAA
+iYAAIcnAACLJwAAzCcAANAnAAAVKAAAGigAADopAAA/KQAAfykAAIMpAADFKQAAyikAAGQqAABq
KgAAqSoAALAqAADvKgAA9SoAADUrAAA5KwAAeCsAAIIrAADRKwAA2CsAABcsAAAdLAAAYCwAAGIs
AACnLAAAqywAAO4sAADxLAAAKy0AADYtAAB0LQAAdS0AAPotAAD/LQAAQy4AAEUuAACMLgAAjS4A
ANQuAADeLgAALi8AADEvAABzLwAAei8AALsvAAC/LwAA/y8AAAYwAACLMAAAkTAAANIwAADVMAAA
GjEAAB8xAABgMQAAZDEAAKQxAACpMQAA/TEAAAQyAABFMgAASjIAAI0yAACTMgAA1TIAANcyAAAX
MwAAHjMAAPUzAAD8MwAAPjQAAEU0AACENAAAiTQAAMg0AADQNAAACDUAABE1AABdNQAAZzUAAKQ1
AACmNQAA7TUAAPQ1AAAzNgAAODYAAHU2AAB8NgAAuzYAAL42AAABNwAACjcAAJA3AACUNwAA3jcA
AOc3AACiOAAAqTgAANQ4AADbOAAAGDkAAB05AABhOQAAYzkAAKg5AACsOQAA6zkAAPE5AABZOgAA
XjoAAOg6AADuOgAAMDsAADI7AAB2OwAAgDsAALc7AAC/OwAA/TsAAAI8AABUPAAAWDwAAJw8AACg
PAAA5TwAAOc8AAA1PgAAQD4AAHY+AACBPgAAuj4AAME+AAABPwAABD8AAEc/AABJPwAAvD8AAMI/
AAAEQAAAB0AAAE1AAABTQAAAtEAAALlAAAD6QAAAAUEAAD5BAABDQQAAg0EAAI1BAAATQgAAFUIA
AFxCAABgQgAAoUIAAKhCAADmQgAA7UIAAC5DAAA2QwAAwEMAAMJDAAAjRAAAJ0QAAGpEAAB0RAAA
r0QAALlEAAD4RAAA+0QAADxFAABDRQAAf0UAAItFAADGRQAAyEUAAAtGAAAQRgAAmUYAAJ1GAAAK
RwAADkcAAEVIAABPSAAAiUgAAI9IAADNSAAA00gAABNJAAAdSQAAWkkAAGFJAACbSQAApUkAAONJ
AADnSQAAXUoAAGlKAACfSgAApkoAACxLAAA1SwAAb0sAAHRLAADDSwAAyEsAAAxMAAAWTAAAVUwA
AF1MAACbTAAAn0wAAOJMAADpTAAAVk0AAFtNAACeTQAAoE0AAOJNAADsTQAAJU4AAC5OAAC8TgAA
x04AAARPAAAITwAARk8AAE5PAACJTwAAk08AABxQAAAlUAAAY1AAAGdQAACmUAAAsFAAAOxQAADv
UAAALFEAADVRAAB0UQAAeFEAAIlSAACTUgAA0VIAANxSAAATUwAAHFMAAE1TAABeUwAAlFMAAJtT
AADZUwAA3VMAAB9UAAAkVAAAYFQAAGpUAAClVAAAqFQAAOtUAADyVAAAjlUAAJJVAADUVQAA21UA
ABZWAAAiVgAAXFYAAGRWAACuVgAAu1YAAPJWAAD3VgAAMlcAADxXAAB6VwAAhlcAAMJXAADGVwAA
ClgAABlYAABQWAAAVlgAAJVYAACaWAAA3lgAAOVYAAAdWQAAJ1kAAKhZAACwWQAA61kAAPVZAAB9
WgAAfloAALxaAADIWgAAA1sAAApbAABMWwAAUVsAANRbAADYWwAAG1wAACNcAABhXAAAaFwAAOBc
AADkXAAAv10AAMVdAAAHXgAACV4AAEpeAABSXgAAiF4AAJNeAADPXgAA0V4AABBfAAAbXwAAWF8A
AF1fAACaXwAAoV8AABVgAAAXYAAAXWAAAGBgAADkYAAA72AAAC1hAAAuYQAAcmEAAHZhAACuYQAA
u2EAAPFhAAD6YQAAOmIAAD1iAAB7YgAAhGIAAMJiAADJYgAACWMAAA1jAABOYwAAUmMAAJRjAACZ
YwAA22MAAOFjAACEZAAAjmQAAM1kAADQZAAAEGUAABhlAACAZQAAhWUAAMRlAADNZQAAb2YAAHhm
AADEZgAAz2YAAK9nAAC6ZwAArmgAALhoAAAKaQAAD2kAAE5pAABVaQAAz2kAANhpAAAVagAAIWoA
AFtqAABeagAAmWsAAJtrAADaawAA42sAAB1sAAAibAAAY2wAAGdsAACpbAAArGwAAPFsAAD5bAAA
Om0AAD9tAAB/bQAAh20AAL9tAADNbQAAPm4AAEFuAADNbgAA1G4AABZvAAAabwAAOW8AAEVvAACq
bwAAr28AAPJvAAD0bwAAIHAAACtwAABdcAAAaHAAAIFwAACLcAAAvXAAAMVwAABDcQAASnEAAIpx
AACRcQAAynEAANNxAAAPcgAAFHIAACNzAAAtcwAAaXMAAGxzAAD4cwAA+nMAAEF0AABIdAAA8nQA
APV0AAA7dQAAQnUAAIJ1AACGdQAAynUAAM51AAATdgAAFnYAAFp2AABcdgAAo3YAAKV2AAAqdwAA
LncAAIN3AACKdwAA6ncAAPd3AAAseAAANngAAG94AACGeAAAtXgAALp4AAD8eAAAA3kAADx5AABE
eQAAZHkAAGp5AADKeQAAzHkAAAR6AAALegAATHoAAFV6AACRegAAmnoAANd6AADbegAAM3sAADp7
AAB8ewAAf3sAAMN7AADIewAACHwAABF8AACSfAAAnHwAANt8AADifAAA2H0AAN99AABjfgAAan4A
AN5+AADlfgAAHn8AACd/AABnfwAAcH8AAK9/AACyfwAA9H8AAPp/AABDgAAARIAAAH2AAACGgAAA
xoAAAMiAAAAigQAAJoEAAGWBAABrgQAA84EAAPmBAAA4ggAARYIAALyCAADHggAA9oIAAC2DAAC7
hgAAwYYAAP6GAAAJhwAAXYcAAGmHAACjhwAAp4cAAOiHAADthwAAb4gAAHWIAAC3iAAAvIgAAPqI
AAACiQAAcIkAAHaJAAC3iQAAuYkAAEOKAABFigAAiYoAAJCKAADQigAA0ooAAGOLAABoiwAAQ4wA
AEWMAACMjAAAkowAABWNAAAbjQAAXo0AAGKNAACPjQAAlY0AANWNAADdjQAANo4AAD2OAAB0jgAA
g44AALaOAAC9jgAA+Y4AAP+OAABqjwAAa48AALCPAAC5jwAA9I8AAP6PAAAzkAAAPpAAAHiQAAB8
kAAAAJEAAAqRAAA/kQAAS5EAAISRAACSkQAABZIAAAiSAACSkgAAmpIAANiSAADekgAA/ZIAAAyT
AAB+kwAAhpMAAASUAAALlAAARJQAAE6UAACDlAAAjZQAAMyUAADPlAAAE5UAABeVAABSlQAAXZUA
AJuVAACglQAA3JUAAOiVAAAllgAAJ5YAAG6WAABxlgAAhpcAAIiXAADOlwAA15cAAA6YAAAXmAAA
mJgAAJ2YAADemAAA4ZgAADaZAAA9mQAAfJkAAIKZAADEmQAAxpkAAAWaAAAWmgAASZoAAFGaAACS
mgAAl5oAANyaAADmmgAAIpsAACqbAABomwAAa5sAAK2bAACymwAA75sAAPabAABRnAAAVZwAAI6c
AACUnAAA/JwAAP+cAABFnQAASp0AAI2dAACPnQAAF54AAB2eAABdngAAYp4AAKKeAACnngAA6J4A
AO+eAABpnwAAcZ8AAK+fAACznwAA+J8AAPyfAAA/oAAAQaAAAISgAACIoAAAzKAAANKgAACsogAA
t6IAAOSiAADvogAAKaMAADGjAABwowAAe6MAAIGjAACLowAAzKMAANGjAADRpAAA1aQAALmlAAC8
pQAA/qUAAAKmAABGpgAASqYAAI+mAACZpgAA2KYAAN6mAAAdpwAAIacAAGWnAABrpwAArKcAALCn
AABDqQAAgakAAIupAACNqQAAz6kAANipAAAxqgAAM6oAALuqAADAqgAApqsAAKmrAAAqrAAANqwA
AKasAACprAAA460AAOWtAADCrwAAxa8AAAuxAABEsQAAU7EAAFaxAACasQAAnbEAAMixAADWsQAA
9bIAAACzAAD0swAA9rMAAFG0AABZtAAACrUAABa1AADTtQAA3rUAAAG2AAAGtgAA0bYAANW2AAAk
twAAKLcAAHO3AAB9twAA6LcAAOu3AABQuAAAXLgAAGDAAACYwAAAp8AAAKrAAABYwQAAmsEAAD3F
AAB3xQAAv8YAAPjGAAAixwAAKccAANnHAADmxwAABcgAABDIAACHyAAAksgAAB3JAAAlyQAAfMkA
AKzJAACwyQAAs8kAALfJAADMyQAAOsoAAFvKAADwygAABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
MwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAz
AAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMA
BwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAH
ADMABwAzAAcAMwAHADMABwAzAAcAMwAHAAUABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcA
BQAHAAUABwAFAAcABQAHAAAAAAC9ZQAA52UAAMajAADTowAA8MoAAAcABQAHAAUABwAAAAAA4mUA
AONlAAAZZwAAL2cAAIBqAABLawAArYIAAK2CAAAtgwAAy4MAAImcAACKnAAA96AAAPygAADwygAA
AwAEAAMABAADAAQAAwAEAAMABAADAAQAAwAEAAMAAQClRiw3tvwG4/8P/w//D/8P/w//D/8P/w//
DxAAAQAAABcQAAAAAAAAAAAAAGgBAAAAAAAAFRgAAA+E0AIRhJj+FcYFAAHQAgZehNACYISY/k9K
AQBRSgEAbygAh2gAAAAAiEgAAAEAt/ABAAAAF5AAAAAAAAAAAAAAaAEAAAAAAAAZGAAAD4SgBRGE
mP4VxgUAAaAFBl6EoAVghJj+T0oFAFFKBQBeSgUAbygAh2gAAAAAiEgAAAEAbwABAAAAF5AAAAAA
AAAAAAAAaAEAAAAAAAAVGAAAD4RwCBGEmP4VxgUAAXAIBl6EcAhghJj+T0oHAFFKBwBvKACHaAAA
AACISAAAAQCn8AEAAAAXkAAAAAAAAAAAAABoAQAAAAAAABUYAAAPhEALEYSY/hXGBQABQAsGXoRA
C2CEmP5PSgEAUUoBAG8oAIdoAAAAAIhIAAABALfwAQAAABeQAAAAAAAAAAAAAGgBAAAAAAAAGRgA
AA+EEA4RhJj+FcYFAAEQDgZehBAOYISY/k9KBQBRSgUAXkoFAG8oAIdoAAAAAIhIAAABAG8AAQAA
ABeQAAAAAAAAAAAAAGgBAAAAAAAAFRgAAA+E4BARhJj+FcYFAAHgEAZehOAQYISY/k9KBwBRSgcA
bygAh2gAAAAAiEgAAAEAp/ABAAAAF5AAAAAAAAAAAAAAaAEAAAAAAAAVGAAAD4SwExGEmP4VxgUA
AbATBl6EsBNghJj+T0oBAFFKAQBvKACHaAAAAACISAAAAQC38AEAAAAXkAAAAAAAAAAAAABoAQAA
AAAAABkYAAAPhIAWEYSY/hXGBQABgBYGXoSAFmCEmP5PSgUAUUoFAF5KBQBvKACHaAAAAACISAAA
AQBvAAEAAAAXkAAAAAAAAAAAAABoAQAAAAAAABUYAAAPhFAZEYSY/hXGBQABUBkGXoRQGWCEmP5P
SgcAUUoHAG8oAIdoAAAAAIhIAAABAKfwAQAAAKVGLDcAAAAAAAAAAAAAAAD///////8BAAAAAAD/
/wEAAAASAAEACQQDAAkEBQAJBAEACQQDAAkEBQAJBAEACQQDAAkEBQAJBC0AAAAEAAAACAAAAOUA
AAAAAAAALAAAAKUwCgAdfAsAy20MABg4HACJbCMANX8jAP5AJgBVHDAAth02AHhsNgBpCjgAUHE6
AOxsOwDbF1UAyWNZANpkWgCZSWcAj25oAApYdQC2QXYAQGN8AAIQfwC6WX8AsECAANchgQBvdYgA
sUWOAApUkQBzTJQAmGWeAKoZoQAjBKIAoFqmAM8zrQDKHLIAZB/GAFUAzAB6MNsAfWTkAC1S6wBF
L+8AAmr0AMNw9ADtFP0APzX/AP9AAYABANZqAADWagAA8BarD1AAUADWagAAAAAAANZqAAAAAAAA
AhAAAAAAAAAA7soAAGAAABAAQAAA//8CAAAABwBVAG4AawBuAG8AdwBuAAcAcwBoAGEAaABpAGQA
YQD//wIACAAAAAAAAAAAAAAAAAAAAAAAAAABAP//AgAAAAAAAAD//wAAAgD//wAAAAD//wAAAgD/
/wAAAAAIAAAARxaQAQAAAgIGAwUEBQIDBId6ACAAAACACAAAAAAAAAD/AQAAAAAAAFQAaQBtAGUA
cwAgAE4AZQB3ACAAUgBvAG0AYQBuAAAANRaQAQIABQUBAgEHBgIFBwAAAAAAAAAQAAAAAAAAAAAA
AACAAAAAAFMAeQBtAGIAbwBsAAAAMyaQAQAAAgsGBAICAgICBId6ACAAAACACAAAAAAAAAD/AQAA
AAAAAEEAcgBpAGEAbAAAAF0RkAGACgICBgkEAgUIAwQBAAAAAAAHCBAAAAAAAAAAAAACAAAAAABN
AFMAIABNAGkAbgBjAGgAbwAAAEEAcgBpAGEAbAAgAFUAbgBpAGMAbwBkAGUAIABNAFMAAABDJpAB
AAACCwUCAgIEAgMDhwAAAAAAAAAAAAAAAAAAABsAAAAAAAAARgB1AHQAdQByAGEAQQAgAEIAawAg
AEIAVAAAAD81kAEAAAIHAwkCAgUCBASHegAgAAAAgAgAAAAAAAAA/wEAAAAAAABDAG8AdQByAGkA
ZQByACAATgBlAHcAAAA1JpABAAACCwYEAwUEBAIEh3oAYQAAAIAIAAAAAAAAAP8BAQAAAAAAVABh
AGgAbwBtAGEAAAA7BpABAgAFAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAIAAAAAAVwBpAG4A
ZwBkAGkAbgBnAHMAAAAiAAQAcYiIGADw0AIAAGgBAAAAALEzG2e6OhuHAAAAAAYAzgAAAEkeAACl
rAAAAQBnAAAABAADEHABAABJHgAApawAAAEAZwAAAHABAAAAAAAAJAMA8BAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAgQSgBbQAtACBgTI0AAAQABkAZAAAABkAAACHygAAh8oAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAA
AAAAAAAFMoNRAPAQAN/fAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEhQAAAAAAnw/w8AAAE/
AAAAAAAA4mUAAP///3////9/////f////3////9/////f9sXVQAAAAAAMgAAAAAAAAAAAAAAAAAB
AAAA//8SAAAAAAAAADwATgBlAHQAdwBvAHIAawAgAFcAbwByAGsAaQBuAGcAIABHAHIAbwB1AHAA
IAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAg
ACAAIAAgACAAIAAgACAAIAAgAEYAAAAAAAAABwBzAGgAYQBoAGkAZABhAAcAcwBoAGEAaABpAGQA
YQAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAGAAAAAQAAAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAD+/wAABQECAAAAAAAAAAAAAAAAAAAAAAABAAAA4IWf8vlPaBCr
kQgAKyez2TAAAACoAQAAEQAAAAEAAACQAAAAAgAAAJgAAAADAAAA4AAAAAQAAADsAAAABQAAAPwA
AAAGAAAACAEAAAcAAAAUAQAACAAAACgBAAAJAAAAOAEAABIAAABEAQAACgAAAGQBAAAMAAAAcAEA
AA0AAAB8AQAADgAAAIgBAAAPAAAAkAEAABAAAACYAQAAEwAAAKABAAACAAAA5AQAAB4AAABAAAAA
TmV0d29yayBXb3JraW5nIEdyb3VwICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBGAAAAAB4AAAAEAAAAAAAAAB4AAAAIAAAAc2hhaGlkYQAeAAAABAAAAAAAAAAeAAAABAAAAAAA
AAAeAAAADAAAAG5vcm1hbC5kb3QAAB4AAAAIAAAAc2hhaGlkYQAeAAAABAAAADYAAAAeAAAAGAAA
AE1pY3Jvc29mdCBPZmZpY2UgV29yZAAAAEAAAAAAVCLHHAAAAEAAAAAARomeMdvOAUAAAAAA3L6D
2tvOAQMAAAABAAAAAwAAAEkeAAADAAAApawAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAA/v8AAAUBAgAAAAAAAAAAAAAAAAAAAAAAAQAAAALVzdWcLhsQk5cIACss+a4w
AAAAMAEAAAwAAAABAAAAaAAAAA8AAABwAAAABQAAAIgAAAAGAAAAkAAAABEAAACYAAAAFwAAAKAA
AAALAAAAqAAAABAAAACwAAAAEwAAALgAAAAWAAAAwAAAAA0AAADIAAAADAAAABEBAAACAAAA5AQA
AB4AAAAQAAAAQWxjYXRlbC1MdWNlbnQAAAMAAABwAQAAAwAAAGcAAAADAAAAh8oAAAMAAAAPJwsA
CwAAAAAAAAALAAAAAAAAAAsAAAAAAAAACwAAAAAAAAAeEAAAAQAAAD0AAABOZXR3b3JrIFdvcmtp
bmcgR3JvdXAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEYADBAAAAIAAAAe
AAAABgAAAFRpdGxlAAMAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAEAAAACAAAAAwAAAAQAAAAFAAAABgAAAAcAAAAIAAAACQAAAAoAAAALAAAADAAAAA0AAAAO
AAAADwAAABAAAAARAAAAEgAAABMAAAAUAAAAFQAAABYAAAAXAAAAGAAAABkAAAAaAAAAGwAAABwA
AAAdAAAAHgAAAB8AAAAgAAAAIQAAACIAAAAjAAAAJAAAACUAAAAmAAAAJwAAACgAAAApAAAAKgAA
ACsAAAAsAAAALQAAAC4AAAAvAAAAMAAAADEAAAAyAAAAMwAAADQAAAA1AAAANgAAADcAAAA4AAAA
OQAAADoAAAA7AAAAPAAAAD0AAAA+AAAAPwAAAEAAAABBAAAAQgAAAEMAAABEAAAARQAAAEYAAABH
AAAASAAAAEkAAABKAAAASwAAAEwAAABNAAAATgAAAE8AAABQAAAAUQAAAFIAAABTAAAAVAAAAFUA
AABWAAAAVwAAAFgAAABZAAAAWgAAAFsAAABcAAAAXQAAAF4AAABfAAAAYAAAAGEAAABiAAAAYwAA
AGQAAABlAAAAZgAAAGcAAABoAAAAaQAAAGoAAABrAAAAbAAAAG0AAABuAAAAbwAAAHAAAABxAAAA
cgAAAHMAAAB0AAAAdQAAAHYAAAB3AAAAeAAAAHkAAAB6AAAAewAAAHwAAAB9AAAAfgAAAH8AAACA
AAAAgQAAAIIAAACDAAAAhAAAAIUAAACGAAAAhwAAAIgAAACJAAAAigAAAIsAAACMAAAAjQAAAI4A
AACPAAAAkAAAAJEAAACSAAAAkwAAAJQAAACVAAAAlgAAAJcAAACYAAAAmQAAAJoAAACbAAAAnAAA
AJ0AAAD+////nwAAAKAAAAChAAAAogAAAKMAAACkAAAApQAAAP7///+nAAAAqAAAAKkAAACqAAAA
qwAAAKwAAACtAAAArgAAAK8AAACwAAAAsQAAALIAAACzAAAAtAAAALUAAAC2AAAAtwAAALgAAAC5
AAAAugAAALsAAAC8AAAAvQAAAL4AAAC/AAAAwAAAAMEAAADCAAAAwwAAAMQAAADFAAAAxgAAAMcA
AADIAAAAyQAAAMoAAADLAAAAzAAAAM0AAADOAAAAzwAAANAAAADRAAAA0gAAANMAAADUAAAA1QAA
ANYAAADXAAAA2AAAANkAAADaAAAA2wAAANwAAADdAAAA3gAAAN8AAADgAAAA4QAAAOIAAADjAAAA
5AAAAOUAAADmAAAA5wAAAOgAAADpAAAA6gAAAOsAAADsAAAA7QAAAO4AAADvAAAA8AAAAPEAAADy
AAAA8wAAAPQAAAD1AAAA9gAAAPcAAAD4AAAA+QAAAPoAAAD7AAAA/AAAAP0AAAD+AAAA/wAAAAAB
AAABAQAAAgEAAAMBAAAEAQAABQEAAAYBAAAHAQAACAEAAAkBAAAKAQAACwEAAAwBAAANAQAADgEA
AA8BAAAQAQAAEQEAABIBAAATAQAAFAEAABUBAAAWAQAAFwEAABgBAAAZAQAAGgEAABsBAAAcAQAA
HQEAAB4BAAAfAQAAIAEAACEBAAAiAQAAIwEAACQBAAAlAQAAJgEAACcBAAAoAQAAKQEAACoBAAAr
AQAALAEAAC0BAAAuAQAALwEAADABAAAxAQAAMgEAADMBAAA0AQAANQEAADYBAAA3AQAAOAEAADkB
AAA6AQAAOwEAADwBAAA9AQAAPgEAAD8BAABAAQAA/v///0IBAABDAQAARAEAAEUBAABGAQAARwEA
AEgBAAD+////SgEAAEsBAABMAQAATQEAAE4BAABPAQAAUAEAAP7////9/////f////3///9VAQAA
/v////7////+////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
/1IAbwBvAHQAIABFAG4AdAByAHkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAWAAUB//////////8DAAAABgkCAAAAAADAAAAAAAAARgAAAAAAAAAAAAAAAICpc6Ta
284BVwEAAIAAAAAAAAAARABhAHQAYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAoAAgH///////////////8AAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAACeAAAAABAAAAAAAAAxAFQAYQBiAGwAZQAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADgACAQEAAAAGAAAA/////wAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKYAAACgNQEAAAAAAFcAbwByAGQARABvAGMA
dQBtAGUAbgB0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAaAAIBAgAA
AAUAAAD/////AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQ6AQAAAAAA
BQBTAHUAbQBtAGEAcgB5AEkAbgBmAG8AcgBtAGEAdABpAG8AbgAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAACgAAgH///////////////8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AABBAQAAABAAAAAAAAAFAEQAbwBjAHUAbQBlAG4AdABTAHUAbQBtAGEAcgB5AEkAbgBmAG8AcgBt
AGEAdABpAG8AbgAAAAAAAAAAAAAAOAACAQQAAAD//////////wAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAEkBAAAAEAAAAAAAAAEAQwBvAG0AcABPAGIAagAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASAAIA////////////////AAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHEAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////
//////////8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB
AAAA/v//////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////wEA
/v8DCgAA/////wYJAgAAAAAAwAAAAAAAAEYfAAAATWljcm9zb2Z0IE9mZmljZSBXb3JkIERvY3Vt
ZW50AAoAAABNU1dvcmREb2MAEAAAAFdvcmQuRG9jdW1lbnQuOAD0ObJxAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

--_002_C0611F522A6FA9498A044C36073E49657E5FF524US70UWXCHMBA01z_--

From gorry@erg.abdn.ac.uk  Thu Nov  7 10:20:50 2013
Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8ED0C11E825F for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 10:20:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.363
X-Spam-Level: 
X-Spam-Status: No, score=-106.363 tagged_above=-999 required=5 tests=[AWL=0.009, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QBBknrxzT10P for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 10:20:45 -0800 (PST)
Received: from spey.erg.abdn.ac.uk (spey.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 861BB11E827E for <aqm@ietf.org>; Thu,  7 Nov 2013 10:19:18 -0800 (PST)
Received: from www.erg.abdn.ac.uk (blake.erg.abdn.ac.uk [139.133.210.30]) by spey.erg.abdn.ac.uk (Postfix) with ESMTPSA id 2ECD32B44A2; Thu,  7 Nov 2013 18:19:15 +0000 (GMT)
Received: from 139.133.204.42 (SquirrelMail authenticated user gorry) by www.erg.abdn.ac.uk with HTTP; Thu, 7 Nov 2013 18:19:15 -0000
Message-ID: <a3896f9a28ac0ff24eb88d6ac406db39.squirrel@www.erg.abdn.ac.uk>
In-Reply-To: <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com>
Date: Thu, 7 Nov 2013 18:19:15 -0000
From: gorry@erg.abdn.ac.uk
To: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
User-Agent: SquirrelMail/1.4.22
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
Cc: "Scheffenegger, Richard" <rs@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>, "Fred Baker \(fred\) \(fred@cisco.com\)" <fred@cisco.com>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 18:20:50 -0000

Thanks.

Please note the current version of the draft is:
draft-ietf-aqm-recommendation-00

Diff here:
http://www.ietf.org/rfcdiff?url1=draft-baker-aqm-recommendation-02&difftype=--html&submit=Go%21&url2=draft-ietf-aqm-recommendation-00

... and that when we did the last edits we tried to include text that
relates to some of these comments.  Fred & I WILL look at your comments
and how they relate to the new draft revision and get back on the specific
points you raise.

Gorry

> Hi All,
>
> Had some comments on Fred's document. I have added the comments as track
> changes in a word document to easily see them. I used the 02 version.
>
> Thanks.
>
> -----Original Message-----
> From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of
> Scheffenegger, Richard
> Sent: Tuesday, November 05, 2013 5:02 PM
> To: aqm@ietf.org
> Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi (naeemk@ifi.uio.no)
> Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
>
> Hi,
>
> this is just a simple remainder that the WG will convene also this Friday;
>
> As the mechanism update presentations took quite a bit longer than we had
> expected - and we appreciate the discussion these presentations have
> sparked - we agreed with Fred to have his presentation as the first agenda
> point on Friday; the second agenda point will be a presentation around AQM
> evaluation process and criteria.
>
>
> The Room will be Regency B (3. Floor), and we will start at 12:30.
>
>
> Best regards,
>
> Richard Scheffenegger
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>


From shahid.akhtar@alcatel-lucent.com  Thu Nov  7 10:21:49 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B20D11E8208 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 10:21:36 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.298
X-Spam-Level: 
X-Spam-Status: No, score=-10.298 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zWXB3oiTMGB9 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 10:21:19 -0800 (PST)
Received: from ihemail1.lucent.com (ihemail1.lucent.com [135.245.0.33]) by ietfa.amsl.com (Postfix) with ESMTP id 0CF5411E81C4 for <aqm@ietf.org>; Thu,  7 Nov 2013 10:21:08 -0800 (PST)
Received: from us70uusmtp4.zam.alcatel-lucent.com (h135-5-2-66.lucent.com [135.5.2.66]) by ihemail1.lucent.com (8.13.8/IER-o) with ESMTP id rA7IL6Gc029163 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Thu, 7 Nov 2013 12:21:07 -0600 (CST)
Received: from US70UWXCHHUB01.zam.alcatel-lucent.com (us70uwxchhub01.zam.alcatel-lucent.com [135.5.2.48]) by us70uusmtp4.zam.alcatel-lucent.com (GMO) with ESMTP id rA7IL4ba003423 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Nov 2013 13:21:06 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70UWXCHHUB01.zam.alcatel-lucent.com ([135.5.2.48]) with mapi id 14.02.0247.003; Thu, 7 Nov 2013 13:21:05 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: "Scheffenegger, Richard" <rs@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B - Comments on Fred's document
Thread-Index: AQHO2+YfcG8GmFJscU6ejZ0jcf22Yg==
Date: Thu, 7 Nov 2013 18:21:05 +0000
Message-ID: <C0611F522A6FA9498A044C36073E49657E5FF5C4@US70UWXCHMBA01.zam.alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>, <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <A500E5F0-C334-4C09-8E7A-329DF4A42FB1@netapp.com>
In-Reply-To: <A500E5F0-C334-4C09-8E7A-329DF4A42FB1@netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.5.27.18]
Content-Type: multipart/alternative; boundary="_000_C0611F522A6FA9498A044C36073E49657E5FF5C4US70UWXCHMBA01z_"
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.33
Cc: "Fred Baker \(fred\) \(fred@cisco.com\)" <fred@cisco.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B - Comments on Fred's document
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 18:21:49 -0000

--_000_C0611F522A6FA9498A044C36073E49657E5FF5C4US70UWXCHMBA01z_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi All,

I am resending the same comments I had sent in the earlier mail in e-mail f=
ormat only (for ease of reading) as some may not have access to MS word.

Section 4.3 - Manual tuning

The document states and implies that AQMs should not require manual tuning.=
 In section 4.3 it states "The algorithms that the IETF recommends SHOULD N=
OT require operational (especially manual) configuration or tuning". I reco=
mmend to change this sentence to: "The algorithms that the IETF recommends =
should minimize tuning or configurations changes for specific traffic or ne=
twork conditions"

I would further argue that all AQMs will likely require or implicitly assum=
e some type of configuration/tuning.

For example, if we take Codel:
*       For small thin links, such as 1-10Mbs DSL, the 5ms target would inc=
rease packet loss significantly and at 2Mbps, a single MTU time may even ex=
ceed the 5ms target.
*       If the average RTT of all flows going through a link is more than 5=
00ms, e.g. for satellite, then the 100ms interval would prematurely drop pa=
ckets before the sources have had a chance to reduce their sending rate. Or=
 if the average RTT is very low - e.g. 10ms - such as for flows between dat=
a-center elements, then 100ms interval may be slow to signal congestion bac=
k to the sources and significant packet loss may have occurred before such =
signaling.

One of the the objectives of newer AQMs being defined here should be to min=
imize tuning, but we should recognize that likely tuning or some configurat=
ion cannot be eliminated altogether.

Section 4.7 - Further Research

AQM impact on end-user QoE
Research should also focus on improving end user QoE from AQMs rather than =
network related metrics only. Often a significant change in a network metri=
c may only make a minimal change in end-user QoE and thus the value of such=
 change may be minimal.

Suggestion on best buffer sizes
Research should make suggestions on how to configure buffer sizes with each=
 type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer sizes improv=
e end-user QoE and network health.

Best way to leverage deployed AQMs
Research should be done on methods or configurations that leverage deployed=
 AQMs such as RED/WRED to reduce delays and lockout for typical traffic whi=
ch require minimal effort or tuning from the operator.

Best regards,

-Shahid.

-----Original Message-----
From: Scheffenegger, Richard [mailto:rs@netapp.com]
Sent: Thursday, November 07, 2013 11:44 AM
To: Akhtar, Shahid (Shahid)
Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi (naeemk@ifi.uio.no)
Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B

Hi shahid,

Thanks. Please note that not necessarily everyone has access to full featur=
ed ms word; a short summary of the nits sent to the list, or direct communi=
cation with the authors is the usual way.

It looks that many of you comments are not so much specifica of the draft b=
ut discussion points for the wg. Repeating them on list with some context w=
ould be good!

Best regards
   Richard

Von meinem iPhone gesendet

Am 07.11.2013 um 09:01 schrieb "Akhtar, Shahid (Shahid)" <shahid.akhtar@alc=
atel-lucent.com>:

> Hi All,
>
> Had some comments on Fred's document. I have added the comments as track =
changes in a word document to easily see them. I used the 02 version.
>
> Thanks.
>
> -----Original Message-----
> From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of
> Scheffenegger, Richard
> Sent: Tuesday, November 05, 2013 5:02 PM
> To: aqm@ietf.org
> Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi
> (naeemk@ifi.uio.no)
> Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
>
> Hi,
>
> this is just a simple remainder that the WG will convene also this
> Friday;
>
> As the mechanism update presentations took quite a bit longer than we had=
 expected - and we appreciate the discussion these presentations have spark=
ed - we agreed with Fred to have his presentation as the first agenda point=
 on Friday; the second agenda point will be a presentation around AQM evalu=
ation process and criteria.
>
>
> The Room will be Regency B (3. Floor), and we will start at 12:30.
>
>
> Best regards,
>
> Richard Scheffenegger
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
> <draft-baker-aqm-recommendation-02.doc>


--_000_C0611F522A6FA9498A044C36073E49657E5FF5C4US70UWXCHMBA01z_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Arial" size=3D"2"><span style=3D"font-size:10pt;">
<div>Hi All,</div>
<div>&nbsp;</div>
<div>I am resending the same comments I had sent in the earlier mail in e-m=
ail format only (for ease of reading) as some may not have access to MS wor=
d.</div>
<div>&nbsp;</div>
<div><b>Section 4.3 - Manual tuning</b></div>
<div>&nbsp;</div>
<div>The document states and implies that AQMs should not require manual tu=
ning. In section 4.3 it states &quot;The algorithms that the IETF recommend=
s SHOULD NOT require operational (especially manual) configuration or tunin=
g&quot;. I recommend to change this sentence
to: &#8220;The algorithms that the IETF recommends should minimize tuning o=
r configurations changes for specific traffic or network conditions&#8221;<=
/div>
<div>&nbsp;</div>
<div>I would further argue that all AQMs will likely require or implicitly =
assume some type of configuration/tuning. </div>
<div>&nbsp;</div>
<div>For example, if we take Codel:</div>
<div>&#8226;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; For small thin links, such=
 as 1-10Mbs DSL, the 5ms target would increase packet loss significantly an=
d at 2Mbps, a single MTU time may even exceed the 5ms target. </div>
<div>&#8226;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; If the average RTT of all =
flows going through a link is more than 500ms, e.g. for satellite, then the=
 100ms interval would prematurely drop packets before the sources have had =
a chance to reduce their sending rate. Or if the average RTT is very
low &#8211; e.g. 10ms &#8211; such as for flows between data-center element=
s, then 100ms interval may be slow to signal congestion back to the sources=
 and significant packet loss may have occurred before such signaling.</div>
<div>&nbsp;</div>
<div>One of the the objectives of newer AQMs being defined here should be t=
o minimize tuning, but we should recognize that likely tuning or some confi=
guration cannot be eliminated altogether.</div>
<div>&nbsp;</div>
<div><b>Section 4.7 - Further Research</b></div>
<div>&nbsp;</div>
<div><b>AQM impact on end-user QoE</b></div>
<div>Research should also focus on improving end user QoE from AQMs rather =
than network related metrics only. Often a significant change in a network =
metric may only make a minimal change in end-user QoE and thus the value of=
 such change may be minimal. </div>
<div>&nbsp;</div>
<div><b>Suggestion on best buffer sizes</b></div>
<div>Research should make suggestions on how to configure buffer sizes with=
 each type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer sizes i=
mprove end-user QoE and network health.</div>
<div>&nbsp;</div>
<div><b>Best way to leverage deployed AQMs</b></div>
<div>Research should be done on methods or configurations that leverage dep=
loyed AQMs such as RED/WRED to reduce delays and lockout for typical traffi=
c which require minimal effort or tuning from the operator. </div>
<div>&nbsp;</div>
<div>Best regards,</div>
<div>&nbsp;</div>
<div>-Shahid.</div>
<div>&nbsp;</div>
<div>-----Original Message-----</div>
<div>From: Scheffenegger, Richard [<a href=3D"mailto:rs@netapp.com"><font c=
olor=3D"blue"><u>mailto:rs@netapp.com</u></font></a>] </div>
<div>Sent: Thursday, November 07, 2013 11:44 AM</div>
<div>To: Akhtar, Shahid (Shahid)</div>
<div>Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi (naeemk@ifi.uio.=
no)</div>
<div>Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B</div>
<div>&nbsp;</div>
<div>Hi shahid,</div>
<div>&nbsp;</div>
<div>Thanks. Please note that not necessarily everyone has access to full f=
eatured ms word; a short summary of the nits sent to the list, or direct co=
mmunication with the authors is the usual way.</div>
<div>&nbsp;</div>
<div>It looks that many of you comments are not so much specifica of the dr=
aft but discussion points for the wg. Repeating them on list with some cont=
ext would be good!</div>
<div>&nbsp;</div>
<div>Best regards</div>
<div>&nbsp;&nbsp; Richard</div>
<div>&nbsp;</div>
<div>Von meinem iPhone gesendet</div>
<div>&nbsp;</div>
<div>Am 07.11.2013 um 09:01 schrieb &quot;Akhtar, Shahid (Shahid)&quot; &lt=
;shahid.akhtar@alcatel-lucent.com&gt;:</div>
<div>&nbsp;</div>
<div>&gt; Hi All,</div>
<div>&gt; </div>
<div>&gt; Had some comments on Fred's document. I have added the comments a=
s track changes in a word document to easily see them. I used the 02 versio=
n. </div>
<div>&gt; </div>
<div>&gt; Thanks. </div>
<div>&gt; </div>
<div>&gt; -----Original Message-----</div>
<div>&gt; From: aqm-bounces@ietf.org [<a href=3D"mailto:aqm-bounces@ietf.or=
g"><font color=3D"blue"><u>mailto:aqm-bounces@ietf.org</u></font></a>] On B=
ehalf Of </div>
<div>&gt; Scheffenegger, Richard</div>
<div>&gt; Sent: Tuesday, November 05, 2013 5:02 PM</div>
<div>&gt; To: aqm@ietf.org</div>
<div>&gt; Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi </div>
<div>&gt; (naeemk@ifi.uio.no)</div>
<div>&gt; Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B</div>
<div>&gt; </div>
<div>&gt; Hi,</div>
<div>&gt; </div>
<div>&gt; this is just a simple remainder that the WG will convene also thi=
s </div>
<div>&gt; Friday;</div>
<div>&gt; </div>
<div>&gt; As the mechanism update presentations took quite a bit longer tha=
n we had expected - and we appreciate the discussion these presentations ha=
ve sparked - we agreed with Fred to have his presentation as the first agen=
da point on Friday; the second agenda
point will be a presentation around AQM evaluation process and criteria.</d=
iv>
<div>&gt; </div>
<div>&gt; </div>
<div>&gt; The Room will be Regency B (3. Floor), and we will start at 12:30=
.</div>
<div>&gt; </div>
<div>&gt; </div>
<div>&gt; Best regards,</div>
<div>&gt; </div>
<div>&gt; Richard Scheffenegger</div>
<div>&gt; </div>
<div>&gt; _______________________________________________</div>
<div>&gt; aqm mailing list</div>
<div>&gt; aqm@ietf.org</div>
<div>&gt; <a href=3D"https://www.ietf.org/mailman/listinfo/aqm"><font color=
=3D"blue"><u>https://www.ietf.org/mailman/listinfo/aqm</u></font></a></div>
<div>&gt; &lt;draft-baker-aqm-recommendation-02.doc&gt;</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_C0611F522A6FA9498A044C36073E49657E5FF5C4US70UWXCHMBA01z_--

From bob.briscoe@bt.com  Thu Nov  7 10:39:57 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6154B21E814E; Thu,  7 Nov 2013 10:39:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.778
X-Spam-Level: 
X-Spam-Status: No, score=-2.778 tagged_above=-999 required=5 tests=[AWL=-0.406, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A6C9uWQVBfhs; Thu,  7 Nov 2013 10:39:51 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id B58A721E816A; Thu,  7 Nov 2013 10:38:59 -0800 (PST)
Received: from EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) by EVMHR66-UKRD.bt.com (10.187.101.21) with Microsoft SMTP Server (TLS) id 14.3.158.1; Thu, 7 Nov 2013 18:38:52 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) with Microsoft SMTP Server (TLS) id 8.3.297.1; Thu, 7 Nov 2013 18:38:48 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Thu, 7 Nov 2013 18:38:46 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.144.199])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA7Icg2o008256; Thu, 7 Nov 2013 18:38:43 GMT
Message-ID: <201311071838.rA7Icg2o008256@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Thu, 7 Nov 2013 18:38:42 +0000
To: Piers O'Hanlon <p.ohanlon@gmail.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <6DB095B2-E77B-457F-81D2-5E9CC940871B@gmail.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <8677FF97-849C-40A6-AE71-AE7ED18570E0@gmail.com> <201311051841.rA5IfUeP030224@bagheera.jungle.bt.co.uk> <6DB095B2-E77B-457F-81D2-5E9CC940871B@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 18:39:57 -0000

Piers,

OK. Text altered (even tho some of the gains are much less sensitive 
to marking scheme, the point is of course generally true).


Bob

At 13:26 07/11/2013, Piers O'Hanlon wrote:
>Bob,
>
>I was thinking that it would be good to add the following clarifications:
>
>Given a suitable marking scheme ECN provides for the removal of 
>nearly all congestion loss and it reduces delays for two main reasons:
>   i) It avoids the delay when recovering from congestion losses, 
> which particularly benefits small flows, making their
>       completion time predictably short [RFC2884]
>   ii) As ECN is used more widely by end-systems, it will gradually 
> remove the need to configure a degree of delay
>       into buffers before they start to notify congestion (the 
> cause of bufferbloat).
>The latter delay is because a drop involves a trade-off between 
>sending a timely signal and trying to avoid impairment, whereas ECN 
>is solely a signal not an impairment, so there is no harm triggering 
>it earlier.
>
>Cheers,
>
>Piers
>
>On 5 Nov 2013, at 18:41, Bob Briscoe wrote:
>
> > Piers,
> >
> > I tried to state exactly how ECN can benefit (2nd para of intro 
> below), rather than make overblown claims. You seem to be saying I 
> didn't succeed? I can't see how to do any better. Suggestions?
> >
> > 
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
> > ECN removes nearly all congestion loss and it cuts delays for two 
> main reasons: i) it avoids the delay when recovering from 
> congestion losses, which particularly benefits small flows, making 
> their completion time predictably short [RFC2884] and ii) as ECN is 
> used more widely by end-systems, it will gradually remove the need 
> to configure a degree of delay into buffers before they start to 
> notify congestion (the cause of bufferbloat). The latter delay is 
> because drop involves a trade-off between sending a timely signal 
> and trying to avoid impairment, whereas ECN is solely a signal not 
> an impairment, so there is no harm triggering it earlier.
> > 
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
> >
> > PS. Thx for supporting words.
> >
> > Bob
> >
> > At 09:42 05/11/2013, Piers O'Hanlon wrote:
> >> Hi Bob,
> >>
> >> I took a brief look at the draft and it's clearly useful work.
> >>
> >> One thing that could do with clarification in the Introduction 
> is that ECN - by itself - doesn't necessarily lead to low loss and 
> delay - it should be made clear that it reflects the marking 
> approach of the underlying scheme/AQM.
> >>
> >> Piers
> >>
> >> On 4 Nov 2013, at 22:03, Bob Briscoe wrote:
> >>
> >> > Folks,
> >> >
> >> > Pls respond if you support this being adopted as a work-group 
> item in the IETF transport services w-g (tsvwg). The WG chairs need 
> visibility of interest.
> >> > Even better, if you're willing to read / comment / review / implement
> >> >
> >> > Guidelines for Adding Congestion Notification to Protocols 
> that Encapsulate IP
> >> > <http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>
> >> >
> >> > Abstract
> >> >
> >> >   The purpose of this document is to guide the design of congestion
> >> >   notification in any lower layer or tunnelling protocol that
> >> >   encapsulates IP.  The aim is for explicit congestion signals to
> >> >   propagate consistently from lower layer protocols into IP.  Then the
> >> >   IP internetwork layer can act as a portability layer to carry
> >> >   congestion notification from non-IP-aware congested nodes up to the
> >> >   transport layer (L4).  Following these guidelines should assure
> >> >   interworking between new lower layer congestion notification
> >> >   mechanisms, whether specified by the IETF or other standards bodies.
> >> >
> >> >
> >> > [Cross-posting tsvwg & aqm, just in case]
> >> >
> >> >
> >> > Bob Briscoe,
> >> > also for co-authors Pat Thaler and John Kaippallimalil
> >> >
> >> >
> >> > ________________________________________________________________
> >> > Bob Briscoe,                                                  BT
> >>
> >> _______________________________________________
> >> aqm mailing list
> >> aqm@ietf.org
> >> https://www.ietf.org/mailman/listinfo/aqm
> >
> > ________________________________________________________________
> > Bob Briscoe,                                                  BT
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 


From fred@cisco.com  Thu Nov  7 10:45:47 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4ED9221E8147 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 10:45:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.415
X-Spam-Level: 
X-Spam-Status: No, score=-110.415 tagged_above=-999 required=5 tests=[AWL=0.184, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lCIdUBVQ4Yhs for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 10:45:42 -0800 (PST)
Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id 5911821E8163 for <aqm@ietf.org>; Thu,  7 Nov 2013 10:45:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=8753; q=dns/txt; s=iport; t=1383849906; x=1385059506; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=fYvtnldzL30C0t6yA+u6gjmmDaPUR8x7C3rZICWWHPY=; b=Jca/7Y7uBvf9EX1To+jlKzThDml51seraBPiv9TONLEnLLvOhrwX3pPZ dOW5jqDds6onwqk93EZp/eoD2h34t5EDnbrs2xJzUzGj405fxNPsZ47hn Qbwc6Z4fv2k6oqb6a+Cqr0vrwoqqjtc1HBGqHOmZjbPQkIFTGHibIrFmT I=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgYFAM/ee1KtJV2Z/2dsb2JhbABagwc4U78PgSUWdIIlAQEBAwEdSxEFCwIBCEYyJQIEDgUJBQaHZwYNvFiODwEQgTkHgyCBEAOQLoEwhi6SCoMmgXE5
X-IronPort-AV: E=Sophos;i="4.93,653,1378857600";  d="asc'?scan'208";a="282068932"
Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-7.cisco.com with ESMTP; 07 Nov 2013 18:45:02 +0000
Received: from xhc-aln-x05.cisco.com (xhc-aln-x05.cisco.com [173.36.12.79]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id rA7Ij1vK026194 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Nov 2013 18:45:01 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x05.cisco.com ([173.36.12.79]) with mapi id 14.03.0123.003; Thu, 7 Nov 2013 12:45:01 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: AQHO2+l2YVICOG/TQKCzUJnt0vYtGg==
Date: Thu, 7 Nov 2013 18:45:00 +0000
Message-ID: <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com>
In-Reply-To: <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.21.116.80]
Content-Type: multipart/signed; boundary="Apple-Mail=_3A73E134-1B2F-4AC9-BDA5-2F0CA4BFFF39"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 18:45:47 -0000

--Apple-Mail=_3A73E134-1B2F-4AC9-BDA5-2F0CA4BFFF39
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252


On Nov 7, 2013, at 8:59 AM, "Akhtar, Shahid (Shahid)" =
<shahid.akhtar@alcatel-lucent.com> wrote:

> Hi All,
>=20
> Had some comments on Fred's document. I have added the comments as =
track changes in a word document to easily see them. I used the 02 =
version.=20
>=20
> Thanks.=20


Permit me to put your comments in email, along with my own views. Also =
adding my co-author and the other working group chair on the CC line; if =
he is like me, he receives far too much email, and mail that is =
explicitly to or copies him bubbles higher in the column.

>> 4.  Conclusions and Recommendations
>>   [snip]=20
>>    3.  The algorithms that the IETF recommends SHOULD NOT require
>>        operational (especially manual) configuration or tuning.
>=20
> Some tuning may be required or implicitly assumed for virtually all =
AQMs =96 please see my comment later.

That's an opinion. One of the objectives of Van and Kathy's work, and =
separately of Rong Pan et al's work, is to design an algorithm that may =
have different initial conditions drawn from a table given the interface =
it finds itself on, but requires no manual tuning. The great failure of =
RED, recommended in RFC 2309, is not that it doesn't work when properly =
configured; it's that real humans don't have the time to properly tune =
it differently for each of the thousands of link endpoints in their =
networks. There is no point in changing away from RED if that is also =
true of the replacement.

>>    7.  Research, engineering, and measurement efforts are needed
>>        regarding the design of mechanisms to deal with flows that are
>>        unresponsive to congestion notification or are responsive, but
>>        are more aggressive than present TCP.
> =20
>       Do we want to make a suggestion on how to configure buffer sizes =
with each type of AQM here (e.g. 2xBDP etc) or simply state that =
research should be conducted on the best buffer sizes to use with AQM.
=20
I'm not sure that buffer sizes are specific to AQM algorithms; I'd =
entertain evidence otherwise. Buffer *thresholds* ("at what point do we =
start dropping/marking traffic?") may differ between algorithms. Buffer =
size ("how many bytes/packets do we allow into the queue in the worst =
case?") is a matter of the characteristics of burst behavior in a given =
network and the applications it supports. If I have, say, a Map/Reduce =
application that simultaneously asks thousands of systems a question, =
the queues in the intervening switches will need to be able to briefly =
absorb thousands of response packets. The key word here is "briefly". =
When Van or Kathy talk about "good queue" and "bad queue", they are =
saying that burst behavior may call for deep queues, but we really want =
the steady state to achieve 100% utilization with a statistically empty =
queue if we can possibly achieve that.

>> 4.3.  AQM algorithms deployed SHOULD NOT require operational tuning
>> =20
>>    A number of algorithms have been proposed.  Many require some form =
of
>>    tuning or initial condition.  This can make them difficult to use
>>    operationally.  Hence, self-tuning algorithms are to be preferred.
>>    The algorithms that the IETF recommends SHOULD NOT require
>>    operational (especially manual) configuration or tuning.
> =20
> May be better to state that tuning should be minimized. For the second =
sentence =93The algorithms that the IETF recommends should minimize =
tuning or configurations changes for specific traffic or network =
conditions=94
> =20
> I would argue that all AQMs will likely require or assume some type of =
configuration/tuning.
> =20
> For example, if we take Codel:
> =20
> =B7       For small thin links, such as 1-10Mbs DSL, the 5ms target =
would increase packet loss significantly and at 2Mbps, a single MTU time =
may even exceed the 5ms target.
> =20
> =B7       If the average RTT of all flows going through a link is more =
than 500ms, e.g. for satellite, then the 100ms interval would =
prematurely drop packets before the sources have had a chance to reduce =
their sending rate. Or if the average RTT is very low =96 e.g. 10ms =96 =
such as for flows between data-center elements, then 100ms interval may =
be slow to signal congestion back to the sources and significant packet =
loss may have occurred before such signaling.


What you describe is what I referred to as "initial conditions derived =
from the links the algorithm finds itself on". We may be in violent =
agreement there. If so, the wording I might suggest would be "SHOULD NOT =
require operational (especially manual) configuration or tuning apart =
from automated determination of initial conditions" or some such thing.

>> 4.7.  The need for further research
>> =20
>>    The second recommendation of [RFC2309] called for further research =
in
>>    the interaction between network queues and host applications, and =
the
>>    means of signaling between them.  This research has occurred, and =
we
>>    as a community have learned a lot.  However, we are not done.
>> =20
>>    We have learned that the problems of congestion, latency and =
buffer-
>>    sizing have not gone away, and are becoming more important to many
>>    users.  A number of self-tuning AQM algorithms have be found that
>>    offer significant advantages for deployed networks.  There is also
>>    renewed interest in deploying AQM and the potential of ECN.
>> =20
>>    An obvious example of further research in 2013 is the need to
>>    consider the use of Map/Reduce applications in data centers; do we
>>    need to extend our taxonomy of TCP/SCTP sessions to include not =
only
>>    "mice" and "elephants", but "lemmings"?  "Lemmings" are flash =
crowds
>>    of "mice" that the network inadvertently tries to signal to as if
>>    they were elephant flows, resulting in head of line blocking in =
data
>>    center applications.
> =20
> Such research should also focus on improving end user QoE from AQMs =
rather than network related metrics only. Often a significant change in =
a network metric may only make a minimal change in end-user QoE and thus =
the value of such change may be minimal.


There is also a question of what user is under discussion. If you take a =
look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx =
and specifically the third slide, you will see (and tomorrow afternoon I =
will discuss) a capture I took of pings from my home network to another =
site overnight. In the evening, we watched a movie (home network), and =
in the morning I had a video conference (office network). I'll tell you =
right now that both worked fine, and managing the delay to zero or =
allowing it to be one hundred ms would not have materially affected the =
QOE of either. However, my ping is a competing application, and it saw =
sustained increase in delay, variation in delay, and the possibility of =
loss as a result of the queuing. The value of AQM is in part to the =
application being throttled, but in large part to competing =
applications, and the QOE of both must be considered.

>>    Examples of other required research include:
>> =20
>>    o  Research into new AQM and scheduling algorithms.
>> =20
>>    o  Research into the use of and deployment of ECN alongside AQM.
>> =20
>>    o  Tools for enabling AQM (and ECN) deployment and measuring the
>>       performance.
>> =20
>>    o  Methods for mitigating the impact of non-conformant and =
malicious
>>       flows.
> =20
> Methods or configurations that leverage deployed AQMs such as RED/WRED =
to reduce delays and lockout for typical traffic which require minimal =
effort or tuning from the operator.
=20
Not a complete sentence, but I think I understand what you're getting =
at. You would like to have research determine how to easily configure =
existing systems using the tools at hand. I'm all for it in the near =
term.

--Apple-Mail=_3A73E134-1B2F-4AC9-BDA5-2F0CA4BFFF39
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSe9+qbjEdbHIsm0MRAjcoAKCUmHejyRaBOImPYycfXrYvA3hY2QCgkhia
Pxp4is0HA9OhKKuZ0TxXHgs=
=kzKk
-----END PGP SIGNATURE-----

--Apple-Mail=_3A73E134-1B2F-4AC9-BDA5-2F0CA4BFFF39--

From gorry@erg.abdn.ac.uk  Thu Nov  7 11:03:25 2013
Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D80611E81B3 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 11:03:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.476
X-Spam-Level: 
X-Spam-Status: No, score=-106.476 tagged_above=-999 required=5 tests=[AWL=0.123, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J1fF6HyMmecP for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 11:03:20 -0800 (PST)
Received: from spey.erg.abdn.ac.uk (spey.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 5136211E81BD for <aqm@ietf.org>; Thu,  7 Nov 2013 11:02:12 -0800 (PST)
Received: from www.erg.abdn.ac.uk (blake.erg.abdn.ac.uk [139.133.210.30]) by spey.erg.abdn.ac.uk (Postfix) with ESMTPSA id 3215A2B43AA; Thu,  7 Nov 2013 19:01:59 +0000 (GMT)
Received: from 139.133.204.42 (SquirrelMail authenticated user gorry) by www.erg.abdn.ac.uk with HTTP; Thu, 7 Nov 2013 19:01:59 -0000
Message-ID: <4b03d7b765d694cd3671ee2daeec33eb.squirrel@www.erg.abdn.ac.uk>
In-Reply-To: <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
Date: Thu, 7 Nov 2013 19:01:59 -0000
From: gorry@erg.abdn.ac.uk
To: "Fred Baker (fred)" <fred@cisco.com>
User-Agent: SquirrelMail/1.4.22
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "Akhtar, Shahid \(Shahid\)" <shahid.akhtar@alcatel-lucent.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 19:03:25 -0000

Gorry - I added a few words to add to those from Fred, see in-line as [gf].

>
> On Nov 7, 2013, at 8:59 AM, "Akhtar, Shahid (Shahid)"
> <shahid.akhtar@alcatel-lucent.com> wrote:
>
>> Hi All,
>>
>> Had some comments on Fred's document. I have added the comments as track
>> changes in a word document to easily see them. I used the 02 version.
>>
>> Thanks.
>
>
> Permit me to put your comments in email, along with my own views. Also
> adding my co-author and the other working group chair on the CC line; if
> he is like me, he receives far too much email, and mail that is explicitly
> to or copies him bubbles higher in the column.
>
>>> 4.  Conclusions and Recommendations
>>>   [snip]
>>>    3.  The algorithms that the IETF recommends SHOULD NOT require
>>>        operational (especially manual) configuration or tuning.
>>
>> Some tuning may be required or implicitly assumed for virtually all AQMs
>> – please see my comment later.
>
> That's an opinion. One of the objectives of Van and Kathy's work, and
> separately of Rong Pan et al's work, is to design an algorithm that may
> have different initial conditions drawn from a table given the interface
> it finds itself on, but requires no manual tuning. The great failure of
> RED, recommended in RFC 2309, is not that it doesn't work when properly
> configured; it's that real humans don't have the time to properly tune it
> differently for each of the thousands of link endpoints in their networks.
> There is no point in changing away from RED if that is also true of the
> replacement.
>
[gf] I note you commented on the previous individual submission. In the
latest version (WG -00 revision),  we tried to clarify the difference
between the overall goal and practicalities. Do you think the update
accommodate the need/ability for tuning to specific use-cases?

>>>    7.  Research, engineering, and measurement efforts are needed
>>>        regarding the design of mechanisms to deal with flows that are
>>>        unresponsive to congestion notification or are responsive, but
>>>        are more aggressive than present TCP.
>>
>>       Do we want to make a suggestion on how to configure buffer sizes
>> with each type of AQM here (e.g. 2xBDP etc) or simply state that
>> research should be conducted on the best buffer sizes to use with
>> AQM.
>
> I'm not sure that buffer sizes are specific to AQM algorithms; I'd
> entertain evidence otherwise. Buffer *thresholds* ("at what point do we
> start dropping/marking traffic?") may differ between algorithms. Buffer
> size ("how many bytes/packets do we allow into the queue in the worst
> case?") is a matter of the characteristics of burst behavior in a given
> network and the applications it supports. If I have, say, a Map/Reduce
> application that simultaneously asks thousands of systems a question, the
> queues in the intervening switches will need to be able to briefly absorb
> thousands of response packets. The key word here is "briefly". When Van or
> Kathy talk about "good queue" and "bad queue", they are saying that burst
> behavior may call for deep queues, but we really want the steady state to
> achieve 100% utilization with a statistically empty queue if we can
> possibly achieve that.
>
[gf] I would love to be able start making recommendations in this space,
but I think this is a lot more complicated.

But, "buffer size" in actual router/middlebox design isn't just a single
number, there are multiple buffers, thresholds and scheduling that
interplay and relate to specific methods.

I think this particular draft, as a BCP, should not delve into detail here.

>>> 4.3.  AQM algorithms deployed SHOULD NOT require operational tuning
>>>
>>>    A number of algorithms have been proposed.  Many require some form
>>> of
>>>    tuning or initial condition.  This can make them difficult to use
>>>    operationally.  Hence, self-tuning algorithms are to be preferred.
>>>    The algorithms that the IETF recommends SHOULD NOT require
>>>    operational (especially manual) configuration or tuning.
>>
>> May be better to state that tuning should be minimized. For the second
>> sentence “The algorithms that the IETF recommends should minimize tuning
>> or configurations changes for specific traffic or network conditions”
>>
>> I would argue that all AQMs will likely require or assume some type of
>> configuration/tuning.
>>
>> For example, if we take Codel:
>>
>> ·       For small thin links, such as 1-10Mbs DSL, the 5ms target would
>> increase packet loss significantly and at 2Mbps, a single MTU time may
>> even exceed the 5ms target.
>>
>> ·       If the average RTT of all flows going through a link is more
>> than 500ms, e.g. for satellite, then the 100ms interval would
>> prematurely drop packets before the sources have had a chance to reduce
>> their sending rate. Or if the average RTT is very low – e.g. 10ms – such
>> as for flows between data-center elements, then 100ms interval may be
>> slow to signal congestion back to the sources and significant packet
>> loss may have occurred before such signaling.
>
>
> What you describe is what I referred to as "initial conditions derived
> from the links the algorithm finds itself on". We may be in violent
> agreement there. If so, the wording I might suggest would be "SHOULD NOT
> require operational (especially manual) configuration or tuning apart from
> automated determination of initial conditions" or some such thing.
>
[gf] Again, is the WG -00 draft closer or not to what you are thinking?

>>> 4.7.  The need for further research
>>>
>>>    The second recommendation of [RFC2309] called for further research
>>> in
>>>    the interaction between network queues and host applications, and
>>> the
>>>    means of signaling between them.  This research has occurred, and we
>>>    as a community have learned a lot.  However, we are not done.
>>>
>>>    We have learned that the problems of congestion, latency and buffer-
>>>    sizing have not gone away, and are becoming more important to many
>>>    users.  A number of self-tuning AQM algorithms have be found that
>>>    offer significant advantages for deployed networks.  There is also
>>>    renewed interest in deploying AQM and the potential of ECN.
>>>
>>>    An obvious example of further research in 2013 is the need to
>>>    consider the use of Map/Reduce applications in data centers; do we
>>>    need to extend our taxonomy of TCP/SCTP sessions to include not only
>>>    "mice" and "elephants", but "lemmings"?  "Lemmings" are flash crowds
>>>    of "mice" that the network inadvertently tries to signal to as if
>>>    they were elephant flows, resulting in head of line blocking in data
>>>    center applications.
>>
>> Such research should also focus on improving end user QoE from AQMs
>> rather than network related metrics only. Often a significant change in
>> a network metric may only make a minimal change in end-user QoE and thus
>> the value of such change may be minimal.
>
>
> There is also a question of what user is under discussion. If you take a
> look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx and
> specifically the third slide, you will see (and tomorrow afternoon I will
> discuss) a capture I took of pings from my home network to another site
> overnight. In the evening, we watched a movie (home network), and in the
> morning I had a video conference (office network). I'll tell you right now
> that both worked fine, and managing the delay to zero or allowing it to be
> one hundred ms would not have materially affected the QOE of either.
> However, my ping is a competing application, and it saw sustained increase
> in delay, variation in delay, and the possibility of loss as a result of
> the queuing. The value of AQM is in part to the application being
> throttled, but in large part to competing applications, and the QOE of
> both must be considered.
>
>>>    Examples of other required research include:
>>>
>>>    o  Research into new AQM and scheduling algorithms.
>>>
>>>    o  Research into the use of and deployment of ECN alongside AQM.
>>>
>>>    o  Tools for enabling AQM (and ECN) deployment and measuring the
>>>       performance.
>>>
>>>    o  Methods for mitigating the impact of non-conformant and malicious
>>>       flows.
>>
>> Methods or configurations that leverage deployed AQMs such as RED/WRED
>> to reduce delays and lockout for typical traffic which require minimal
>> effort or tuning from the operator.
>
> Not a complete sentence, but I think I understand what you're getting at.
> You would like to have research determine how to easily configure existing
> systems using the tools at hand. I'm all for it in the near term.
>

[gf] Gorry


From davec-b@rogers.com  Thu Nov  7 11:19:43 2013
Return-Path: <davec-b@rogers.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B6D311E8290 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 11:19:43 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.371
X-Spam-Level: 
X-Spam-Status: No, score=-2.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TyMI3M05CsbI for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 11:19:38 -0800 (PST)
Received: from nm8-vm2.access.bullet.mail.bf1.yahoo.com (nm8-vm2.access.bullet.mail.bf1.yahoo.com [216.109.114.177]) by ietfa.amsl.com (Postfix) with ESMTP id 06D4F21F9649 for <aqm@ietf.org>; Thu,  7 Nov 2013 11:19:34 -0800 (PST)
Received: from [66.196.81.157] by nm8.access.bullet.mail.bf1.yahoo.com with NNFMP; 07 Nov 2013 19:19:34 -0000
Received: from [98.139.221.159] by tm3.access.bullet.mail.bf1.yahoo.com with NNFMP; 07 Nov 2013 19:19:34 -0000
Received: from [127.0.0.1] by smtp119.sbc.mail.bf1.yahoo.com with NNFMP; 07 Nov 2013 19:19:34 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1383851973; bh=zr7zzSXTs+dvKViRb376j6sK7A1C/fns2ZDJyu0uKNE=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:X-Enigmail-Version:Content-Type; b=tN6C/6EyN3vu6yj62wDz9NU9i6iDIqO45KNOPtUtbL5sixgObmK2qdsFuYURnBbOsuWYiumhf1eEdbz4XbrX4L22sI3RTzwsali/QIkkkOmfdyKzemqqttcQ5wac7rjLj+EiCDU+4fvzLS+cnQD8yVerg7azpJAkU7CXOkFZpfc=
X-Yahoo-Newman-Id: 993516.66012.bm@smtp119.sbc.mail.bf1.yahoo.com
X-Yahoo-Newman-Property: ymail-3
X-YMail-OSG: iY6azfAVM1kib0Jo3HTBp58rVQBE.tdvVbnmU27HeS0leb. MIjtWmlHuOGStOIQD6YHTO5gOAnE1LjHkBLyhbRW19t9M4Q0WMoGCI5pKW6L oq4OkRT2ZIA8LQcatsi1ZjqaEMEx72.cvHbBbKgijG59xtSxYZpH87fmiFcY ycpO4_ns9w.PRzwlEo9x62G7i3qjVo11inPMITXcKFR8m3GHHKVb4pOLZaTO PaMrlpp_nug9It_R.t1Sh5rneFz2noRruKUNrfI8K7A8c5t3f2RRPVNgBVfC zdmA3z5f7VDzshNt1W_Cd6652s3KPJtwSig6e4KA73IA7l8j_3Cd07Pw9WEX tpOqMDr9p22n5b3LHEh84sNEjv292Fy2m_5Td67agZb2AF_Sss58DkaXces8 0ejb8aBpnreE5hAwV4C.FZGRUPGxas04JYwG6sb.5eWX3TUFGXSFCV9xrBlh AacjKlJqo_pbX62skudRqLGhEzt5BZOaUlw5qdk_.S8Qpwm.8sh8c7RD_Xya ..Mf92QFh9y9kmEWqY5JE0CFy4a11ctYE.9Q9aZK1t8SqXSYMyoz1G7G8igX .7RNN
X-Yahoo-SMTP: sltvjZWswBCRD.ElTuB1l9j6s9wRYPpuyTNWOE5oEg--
X-Rocket-Received: from [10.111.100.153] (davec-b@74.213.188.46 with ) by smtp119.sbc.mail.bf1.yahoo.com with SMTP; 07 Nov 2013 19:19:33 +0000 UTC
Message-ID: <527BE7C6.2010205@rogers.com>
Date: Thu, 07 Nov 2013 14:19:34 -0500
From: David Collier-Brown <davec-b@rogers.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4
MIME-Version: 1.0
To: aqm@ietf.org
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com>, <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <A500E5F0-C334-4C09-8E7A-329DF4A42FB1@netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF5C4@US70UWXCHMBA01.zam.alcatel-lucent.com>
In-Reply-To: <C0611F522A6FA9498A044C36073E49657E5FF5C4@US70UWXCHMBA01.zam.alcatel-lucent.com>
X-Enigmail-Version: 1.6
Content-Type: multipart/alternative; boundary="------------030508050601030808090401"
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B - Comments on Fred's document
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
Reply-To: davecb@spamcop.net
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 19:19:43 -0000

This is a multi-part message in MIME format.
--------------030508050601030808090401
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

It's written in the light of recent work that suggest that tuning is
unnecessary, but is not a proof that it will never be necessary.  If it
were, the wording would arguably be "must not" as opposed to "should not".

I think the emphasis in "especially manual", however, is inappropriate. 
I would prefer to see language that says that they should adapt to
measurable conditions themselves, and not require outside intervention
to inject information they require.  I'd then weaken it for initial
settings that are never changed, like "this is strictly for
IP-over-carrier-pigeons".

--dave


On 11/07/2013 01:21 PM, Akhtar, Shahid (Shahid) wrote:
> Hi All,
>  
> I am resending the same comments I had sent in the earlier mail in
> e-mail format only (for ease of reading) as some may not have access
> to MS word.
>  
> *Section 4.3 - Manual tuning*
>  
> The document states and implies that AQMs should not require manual
> tuning. In section 4.3 it states "The algorithms that the IETF
> recommends SHOULD NOT require operational (especially manual)
> configuration or tuning". I recommend to change this sentence to: "The
> algorithms that the IETF recommends should minimize tuning or
> configurations changes for specific traffic or network conditions"
>  
> I would further argue that all AQMs will likely require or implicitly
> assume some type of configuration/tuning.
>  
> For example, if we take Codel:
> .       For small thin links, such as 1-10Mbs DSL, the 5ms target
> would increase packet loss significantly and at 2Mbps, a single MTU
> time may even exceed the 5ms target.
> .       If the average RTT of all flows going through a link is more
> than 500ms, e.g. for satellite, then the 100ms interval would
> prematurely drop packets before the sources have had a chance to
> reduce their sending rate. Or if the average RTT is very low -- e.g.
> 10ms -- such as for flows between data-center elements, then 100ms
> interval may be slow to signal congestion back to the sources and
> significant packet loss may have occurred before such signaling.
>  
> One of the the objectives of newer AQMs being defined here should be
> to minimize tuning, but we should recognize that likely tuning or some
> configuration cannot be eliminated altogether.
>  
> *Section 4.7 - Further Research*
>  
> *AQM impact on end-user QoE*
> Research should also focus on improving end user QoE from AQMs rather
> than network related metrics only. Often a significant change in a
> network metric may only make a minimal change in end-user QoE and thus
> the value of such change may be minimal.
>  
> *Suggestion on best buffer sizes*
> Research should make suggestions on how to configure buffer sizes with
> each type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer
> sizes improve end-user QoE and network health.
>  
> *Best way to leverage deployed AQMs*
> Research should be done on methods or configurations that leverage
> deployed AQMs such as RED/WRED to reduce delays and lockout for
> typical traffic which require minimal effort or tuning from the operator.
>  
> Best regards,
>  
> -Shahid.
>  
> -----Original Message-----
> From: Scheffenegger, Richard [_mailto:rs@netapp.com_]
> Sent: Thursday, November 07, 2013 11:44 AM
> To: Akhtar, Shahid (Shahid)
> Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi (naeemk@ifi.uio.no)
> Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B
>  
> Hi shahid,
>  
> Thanks. Please note that not necessarily everyone has access to full
> featured ms word; a short summary of the nits sent to the list, or
> direct communication with the authors is the usual way.
>  
> It looks that many of you comments are not so much specifica of the
> draft but discussion points for the wg. Repeating them on list with
> some context would be good!
>  
> Best regards
>    Richard
>  
> Von meinem iPhone gesendet
>  
> Am 07.11.2013 um 09:01 schrieb "Akhtar, Shahid (Shahid)"
> <shahid.akhtar@alcatel-lucent.com>:
>  
> > Hi All,
> > 
> > Had some comments on Fred's document. I have added the comments as track changes in a word document to easily see them.
> I used the 02 version.
> > 
> > Thanks. 
> > 
> > -----Original Message-----
> > From: aqm-bounces@ietf.org [_mailto:aqm-bounces@ietf.org_] On Behalf Of
> > Scheffenegger, Richard
> > Sent: Tuesday, November 05, 2013 5:02 PM
> > To: aqm@ietf.org
> > Cc: Fred Baker (fred) (fred@cisco.com); Naeem Khademi
> > (naeemk@ifi.uio.no)
> > Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
> > 
> > Hi,
> > 
> > this is just a simple remainder that the WG will convene also this
> > Friday;
> > 
> > As the mechanism update presentations took quite a bit longer than we had expected - and we appreciate the discussion
> these presentations have sparked - we agreed with Fred to have his
> presentation as the first agenda point on Friday; the second agenda
> point will be a presentation around AQM evaluation process and criteria.
> > 
> > 
> > The Room will be Regency B (3. Floor), and we will start at 12:30.
> > 
> > 
> > Best regards,
> > 
> > Richard Scheffenegger
> > 
> > _______________________________________________
> > aqm mailing list
> > aqm@ietf.org
> > _https://www.ietf.org/mailman/listinfo/aqm_
> > <draft-baker-aqm-recommendation-02.doc>
>  
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm


-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain
(416) 223-8968


--------------030508050601030808090401
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">It's written in the light of recent
      work that suggest that tuning is unnecessary, but is not a proof
      that it will never be necessary.&nbsp; If it were, the wording would
      arguably be "must not" as opposed to "should not".<br>
      <br>
      I think the emphasis in "especially manual", however, is
      inappropriate.&nbsp; I would prefer to see language that says that they
      should adapt to measurable conditions themselves, and not require
      outside intervention to inject information they require.&nbsp; I'd then
      weaken it for initial settings that are never changed, like "this
      is strictly for IP-over-carrier-pigeons".<br>
      <br>
      --dave<br>
      <br>
      <br>
      On 11/07/2013 01:21 PM, Akhtar, Shahid (Shahid) wrote:<br>
    </div>
    <blockquote
cite="mid:C0611F522A6FA9498A044C36073E49657E5FF5C4@US70UWXCHMBA01.zam.alcatel-lucent.com"
      type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=ISO-8859-1">
      <meta name="Generator" content="Microsoft Exchange Server">
      <!-- converted from rtf -->
      <style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
      <font face="Arial" size="2"><span style="font-size:10pt;">
          <div>Hi All,</div>
          <div>&nbsp;</div>
          <div>I am resending the same comments I had sent in the
            earlier mail in e-mail format only (for ease of reading) as
            some may not have access to MS word.</div>
          <div>&nbsp;</div>
          <div><b>Section 4.3 - Manual tuning</b></div>
          <div>&nbsp;</div>
          <div>The document states and implies that AQMs should not
            require manual tuning. In section 4.3 it states "The
            algorithms that the IETF recommends SHOULD NOT require
            operational (especially manual) configuration or tuning". I
            recommend to change this sentence
            to: &#8220;The algorithms that the IETF recommends should minimize
            tuning or configurations changes for specific traffic or
            network conditions&#8221;</div>
          <div>&nbsp;</div>
          <div>I would further argue that all AQMs will likely require
            or implicitly assume some type of configuration/tuning. </div>
          <div>&nbsp;</div>
          <div>For example, if we take Codel:</div>
          <div>&#8226;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; For small thin links, such as 1-10Mbs DSL, the
            5ms target would increase packet loss significantly and at
            2Mbps, a single MTU time may even exceed the 5ms target. </div>
          <div>&#8226;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; If the average RTT of all flows going through a
            link is more than 500ms, e.g. for satellite, then the 100ms
            interval would prematurely drop packets before the sources
            have had a chance to reduce their sending rate. Or if the
            average RTT is very
            low &#8211; e.g. 10ms &#8211; such as for flows between data-center
            elements, then 100ms interval may be slow to signal
            congestion back to the sources and significant packet loss
            may have occurred before such signaling.</div>
          <div>&nbsp;</div>
          <div>One of the the objectives of newer AQMs being defined
            here should be to minimize tuning, but we should recognize
            that likely tuning or some configuration cannot be
            eliminated altogether.</div>
          <div>&nbsp;</div>
          <div><b>Section 4.7 - Further Research</b></div>
          <div>&nbsp;</div>
          <div><b>AQM impact on end-user QoE</b></div>
          <div>Research should also focus on improving end user QoE from
            AQMs rather than network related metrics only. Often a
            significant change in a network metric may only make a
            minimal change in end-user QoE and thus the value of such
            change may be minimal. </div>
          <div>&nbsp;</div>
          <div><b>Suggestion on best buffer sizes</b></div>
          <div>Research should make suggestions on how to configure
            buffer sizes with each type of AQM (e.g. 2xBDP etc) -
            explaining why/how such buffer sizes improve end-user QoE
            and network health.</div>
          <div>&nbsp;</div>
          <div><b>Best way to leverage deployed AQMs</b></div>
          <div>Research should be done on methods or configurations that
            leverage deployed AQMs such as RED/WRED to reduce delays and
            lockout for typical traffic which require minimal effort or
            tuning from the operator. </div>
          <div>&nbsp;</div>
          <div>Best regards,</div>
          <div>&nbsp;</div>
          <div>-Shahid.</div>
          <div>&nbsp;</div>
          <div>-----Original Message-----</div>
          <div>From: Scheffenegger, Richard [<a moz-do-not-send="true"
              href="mailto:rs@netapp.com"><font color="blue"><u>mailto:rs@netapp.com</u></font></a>]
          </div>
          <div>Sent: Thursday, November 07, 2013 11:44 AM</div>
          <div>To: Akhtar, Shahid (Shahid)</div>
          <div>Cc: Fred Baker (fred) (<a class="moz-txt-link-abbreviated" href="mailto:fred@cisco.com">fred@cisco.com</a>); Naeem Khademi
            (<a class="moz-txt-link-abbreviated" href="mailto:naeemk@ifi.uio.no">naeemk@ifi.uio.no</a>)</div>
          <div>Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B</div>
          <div>&nbsp;</div>
          <div>Hi shahid,</div>
          <div>&nbsp;</div>
          <div>Thanks. Please note that not necessarily everyone has
            access to full featured ms word; a short summary of the nits
            sent to the list, or direct communication with the authors
            is the usual way.</div>
          <div>&nbsp;</div>
          <div>It looks that many of you comments are not so much
            specifica of the draft but discussion points for the wg.
            Repeating them on list with some context would be good!</div>
          <div>&nbsp;</div>
          <div>Best regards</div>
          <div>&nbsp;&nbsp; Richard</div>
          <div>&nbsp;</div>
          <div>Von meinem iPhone gesendet</div>
          <div>&nbsp;</div>
          <div>Am 07.11.2013 um 09:01 schrieb "Akhtar, Shahid (Shahid)"
            <a class="moz-txt-link-rfc2396E" href="mailto:shahid.akhtar@alcatel-lucent.com">&lt;shahid.akhtar@alcatel-lucent.com&gt;</a>:</div>
          <div>&nbsp;</div>
          <div>&gt; Hi All,</div>
          <div>&gt; </div>
          <div>&gt; Had some comments on Fred's document. I have added
            the comments as track changes in a word document to easily
            see them. I used the 02 version. </div>
          <div>&gt; </div>
          <div>&gt; Thanks. </div>
          <div>&gt; </div>
          <div>&gt; -----Original Message-----</div>
          <div>&gt; From: <a class="moz-txt-link-abbreviated" href="mailto:aqm-bounces@ietf.org">aqm-bounces@ietf.org</a> [<a
              moz-do-not-send="true" href="mailto:aqm-bounces@ietf.org"><font
                color="blue"><u>mailto:aqm-bounces@ietf.org</u></font></a>]
            On Behalf Of </div>
          <div>&gt; Scheffenegger, Richard</div>
          <div>&gt; Sent: Tuesday, November 05, 2013 5:02 PM</div>
          <div>&gt; To: <a class="moz-txt-link-abbreviated" href="mailto:aqm@ietf.org">aqm@ietf.org</a></div>
          <div>&gt; Cc: Fred Baker (fred) (<a class="moz-txt-link-abbreviated" href="mailto:fred@cisco.com">fred@cisco.com</a>); Naeem
            Khademi </div>
          <div>&gt; (<a class="moz-txt-link-abbreviated" href="mailto:naeemk@ifi.uio.no">naeemk@ifi.uio.no</a>)</div>
          <div>&gt; Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B</div>
          <div>&gt; </div>
          <div>&gt; Hi,</div>
          <div>&gt; </div>
          <div>&gt; this is just a simple remainder that the WG will
            convene also this </div>
          <div>&gt; Friday;</div>
          <div>&gt; </div>
          <div>&gt; As the mechanism update presentations took quite a
            bit longer than we had expected - and we appreciate the
            discussion these presentations have sparked - we agreed with
            Fred to have his presentation as the first agenda point on
            Friday; the second agenda
            point will be a presentation around AQM evaluation process
            and criteria.</div>
          <div>&gt; </div>
          <div>&gt; </div>
          <div>&gt; The Room will be Regency B (3. Floor), and we will
            start at 12:30.</div>
          <div>&gt; </div>
          <div>&gt; </div>
          <div>&gt; Best regards,</div>
          <div>&gt; </div>
          <div>&gt; Richard Scheffenegger</div>
          <div>&gt; </div>
          <div>&gt; _______________________________________________</div>
          <div>&gt; aqm mailing list</div>
          <div>&gt; <a class="moz-txt-link-abbreviated" href="mailto:aqm@ietf.org">aqm@ietf.org</a></div>
          <div>&gt; <a moz-do-not-send="true"
              href="https://www.ietf.org/mailman/listinfo/aqm"><font
                color="blue"><u>https://www.ietf.org/mailman/listinfo/aqm</u></font></a></div>
          <div>&gt; &lt;draft-baker-aqm-recommendation-02.doc&gt;</div>
          <div>&nbsp;</div>
        </span></font>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
aqm mailing list
<a class="moz-txt-link-abbreviated" href="mailto:aqm@ietf.org">aqm@ietf.org</a>
<a class="moz-txt-link-freetext" href="https://www.ietf.org/mailman/listinfo/aqm">https://www.ietf.org/mailman/listinfo/aqm</a>
</pre>
    </blockquote>
    <br>
    <br>
    <pre class="moz-signature" cols="72">-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
<a class="moz-txt-link-abbreviated" href="mailto:davecb@spamcop.net">davecb@spamcop.net</a>           |                      -- Mark Twain
(416) 223-8968
</pre>
  </body>
</html>

--------------030508050601030808090401--

From bob.briscoe@bt.com  Thu Nov  7 12:03:37 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BAA6111E82A3; Thu,  7 Nov 2013 12:03:37 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.766
X-Spam-Level: 
X-Spam-Status: No, score=-2.766 tagged_above=-999 required=5 tests=[AWL=-0.394, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UVJ-4Qk7eMgB; Thu,  7 Nov 2013 12:03:32 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id 65BC821F99F8; Thu,  7 Nov 2013 12:03:14 -0800 (PST)
Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR66-UKRD.bt.com (10.187.101.21) with Microsoft SMTP Server (TLS) id 14.3.158.1; Thu, 7 Nov 2013 20:03:12 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Thu, 7 Nov 2013 20:03:12 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Thu, 7 Nov 2013 20:03:12 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.109.144.199])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rA7K38dj008566; Thu, 7 Nov 2013 20:03:09 GMT
Message-ID: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Thu, 7 Nov 2013 20:03:08 +0000
To: tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
From: Bob Briscoe <bob.briscoe@bt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Subject: [aqm] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 20:03:37 -0000

Folks,

"Immediate ECN" slides:
<http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
<http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>

PS. This talk fell off the end of the TSVAREA agenda. It's mostly 
relevant to AQM, but I didn't originally bring it to AQM, because it 
affects 3 wgs: tsvwg, aqm & tcpm.

In the AQM wg, there was dismay about CableLabs not including 
anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and 
how ECN can take out the 100ms of delay that CoDel and PIE introduce 
- it's essentially about auto-tuning for RTT.

It gives an interim recommendation for hardware designers that there 
should be a second instance of the AQM algo for ECN packets so that 
it can be configured with different parameters (think of WRED instead of RED).

Specifically, for ECN packets:
interval = 0 (for CoDel)
max_burst = 0 (for PIE)


Bob

PS. We have a paper under submission, which we can supply on request.
We plan to document this in the IETF too.




________________________________________________________________
Bob Briscoe,                                                  BT 


From shahid.akhtar@alcatel-lucent.com  Thu Nov  7 12:44:57 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D677C21E81AF for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 12:44:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.374
X-Spam-Level: 
X-Spam-Status: No, score=-10.374 tagged_above=-999 required=5 tests=[AWL=0.225, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e2cP2iVUfxpZ for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 12:44:52 -0800 (PST)
Received: from ihemail1.lucent.com (ihemail1.lucent.com [135.245.0.33]) by ietfa.amsl.com (Postfix) with ESMTP id 85DF021E817D for <aqm@ietf.org>; Thu,  7 Nov 2013 12:44:52 -0800 (PST)
Received: from us70tusmtp1.zam.alcatel-lucent.com (h135-5-2-63.lucent.com [135.5.2.63]) by ihemail1.lucent.com (8.13.8/IER-o) with ESMTP id rA7KiaXd018472 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Thu, 7 Nov 2013 14:44:37 -0600 (CST)
Received: from US70UWXCHHUB02.zam.alcatel-lucent.com (us70uwxchhub02.zam.alcatel-lucent.com [135.5.2.49]) by us70tusmtp1.zam.alcatel-lucent.com (GMO) with ESMTP id rA7KiLtf008238 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Nov 2013 15:44:35 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70UWXCHHUB02.zam.alcatel-lucent.com ([135.5.2.49]) with mapi id 14.02.0247.003; Thu, 7 Nov 2013 15:44:29 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: Ac7aemDyYVICOG/TQKCzUJnt0vYtGgAxY8ewADTbyQD//82PKQ==
Date: Thu, 7 Nov 2013 20:44:28 +0000
Message-ID: <6CE3D838-64CB-42C2-B6CB-B07424C8EFB2@alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com>, <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
In-Reply-To: <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.33
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2013 20:44:58 -0000

Fred, All,

I am traveling today - so should respond by end of day.

- Shahid.

Sent from my iPhone

On Nov 7, 2013, at 10:45 AM, "Fred Baker (fred)" <fred@cisco.com> wrote:

>=20
> On Nov 7, 2013, at 8:59 AM, "Akhtar, Shahid (Shahid)" <shahid.akhtar@alca=
tel-lucent.com> wrote:
>=20
>> Hi All,
>>=20
>> Had some comments on Fred's document. I have added the comments as track=
 changes in a word document to easily see them. I used the 02 version.=20
>>=20
>> Thanks.
>=20
>=20
> Permit me to put your comments in email, along with my own views. Also ad=
ding my co-author and the other working group chair on the CC line; if he i=
s like me, he receives far too much email, and mail that is explicitly to o=
r copies him bubbles higher in the column.
>=20
>>> 4.  Conclusions and Recommendations
>>>  [snip]=20
>>>   3.  The algorithms that the IETF recommends SHOULD NOT require
>>>       operational (especially manual) configuration or tuning.
>>=20
>> Some tuning may be required or implicitly assumed for virtually all AQMs=
 =96 please see my comment later.
>=20
> That's an opinion. One of the objectives of Van and Kathy's work, and sep=
arately of Rong Pan et al's work, is to design an algorithm that may have d=
ifferent initial conditions drawn from a table given the interface it finds=
 itself on, but requires no manual tuning. The great failure of RED, recomm=
ended in RFC 2309, is not that it doesn't work when properly configured; it=
's that real humans don't have the time to properly tune it differently for=
 each of the thousands of link endpoints in their networks. There is no poi=
nt in changing away from RED if that is also true of the replacement.
>=20
>>>   7.  Research, engineering, and measurement efforts are needed
>>>       regarding the design of mechanisms to deal with flows that are
>>>       unresponsive to congestion notification or are responsive, but
>>>       are more aggressive than present TCP.
>>=20
>>      Do we want to make a suggestion on how to configure buffer sizes wi=
th each type of AQM here (e.g. 2xBDP etc) or simply state that research sho=
uld be conducted on the best buffer sizes to use with AQM.
>=20
> I'm not sure that buffer sizes are specific to AQM algorithms; I'd entert=
ain evidence otherwise. Buffer *thresholds* ("at what point do we start dro=
pping/marking traffic?") may differ between algorithms. Buffer size ("how m=
any bytes/packets do we allow into the queue in the worst case?") is a matt=
er of the characteristics of burst behavior in a given network and the appl=
ications it supports. If I have, say, a Map/Reduce application that simulta=
neously asks thousands of systems a question, the queues in the intervening=
 switches will need to be able to briefly absorb thousands of response pack=
ets. The key word here is "briefly". When Van or Kathy talk about "good que=
ue" and "bad queue", they are saying that burst behavior may call for deep =
queues, but we really want the steady state to achieve 100% utilization wit=
h a statistically empty queue if we can possibly achieve that.
>=20
>>> 4.3.  AQM algorithms deployed SHOULD NOT require operational tuning
>>>=20
>>>   A number of algorithms have been proposed.  Many require some form of
>>>   tuning or initial condition.  This can make them difficult to use
>>>   operationally.  Hence, self-tuning algorithms are to be preferred.
>>>   The algorithms that the IETF recommends SHOULD NOT require
>>>   operational (especially manual) configuration or tuning.
>>=20
>> May be better to state that tuning should be minimized. For the second s=
entence =93The algorithms that the IETF recommends should minimize tuning o=
r configurations changes for specific traffic or network conditions=94
>>=20
>> I would argue that all AQMs will likely require or assume some type of c=
onfiguration/tuning.
>>=20
>> For example, if we take Codel:
>>=20
>> =B7       For small thin links, such as 1-10Mbs DSL, the 5ms target woul=
d increase packet loss significantly and at 2Mbps, a single MTU time may ev=
en exceed the 5ms target.
>>=20
>> =B7       If the average RTT of all flows going through a link is more t=
han 500ms, e.g. for satellite, then the 100ms interval would prematurely dr=
op packets before the sources have had a chance to reduce their sending rat=
e. Or if the average RTT is very low =96 e.g. 10ms =96 such as for flows be=
tween data-center elements, then 100ms interval may be slow to signal conge=
stion back to the sources and significant packet loss may have occurred bef=
ore such signaling.
>=20
>=20
> What you describe is what I referred to as "initial conditions derived fr=
om the links the algorithm finds itself on". We may be in violent agreement=
 there. If so, the wording I might suggest would be "SHOULD NOT require ope=
rational (especially manual) configuration or tuning apart from automated d=
etermination of initial conditions" or some such thing.
>=20
>>> 4.7.  The need for further research
>>>=20
>>>   The second recommendation of [RFC2309] called for further research in
>>>   the interaction between network queues and host applications, and the
>>>   means of signaling between them.  This research has occurred, and we
>>>   as a community have learned a lot.  However, we are not done.
>>>=20
>>>   We have learned that the problems of congestion, latency and buffer-
>>>   sizing have not gone away, and are becoming more important to many
>>>   users.  A number of self-tuning AQM algorithms have be found that
>>>   offer significant advantages for deployed networks.  There is also
>>>   renewed interest in deploying AQM and the potential of ECN.
>>>=20
>>>   An obvious example of further research in 2013 is the need to
>>>   consider the use of Map/Reduce applications in data centers; do we
>>>   need to extend our taxonomy of TCP/SCTP sessions to include not only
>>>   "mice" and "elephants", but "lemmings"?  "Lemmings" are flash crowds
>>>   of "mice" that the network inadvertently tries to signal to as if
>>>   they were elephant flows, resulting in head of line blocking in data
>>>   center applications.
>>=20
>> Such research should also focus on improving end user QoE from AQMs rath=
er than network related metrics only. Often a significant change in a netwo=
rk metric may only make a minimal change in end-user QoE and thus the value=
 of such change may be minimal.
>=20
>=20
> There is also a question of what user is under discussion. If you take a =
look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx and =
specifically the third slide, you will see (and tomorrow afternoon I will d=
iscuss) a capture I took of pings from my home network to another site over=
night. In the evening, we watched a movie (home network), and in the mornin=
g I had a video conference (office network). I'll tell you right now that b=
oth worked fine, and managing the delay to zero or allowing it to be one hu=
ndred ms would not have materially affected the QOE of either. However, my =
ping is a competing application, and it saw sustained increase in delay, va=
riation in delay, and the possibility of loss as a result of the queuing. T=
he value of AQM is in part to the application being throttled, but in large=
 part to competing applications, and the QOE of both must be considered.
>=20
>>>   Examples of other required research include:
>>>=20
>>>   o  Research into new AQM and scheduling algorithms.
>>>=20
>>>   o  Research into the use of and deployment of ECN alongside AQM.
>>>=20
>>>   o  Tools for enabling AQM (and ECN) deployment and measuring the
>>>      performance.
>>>=20
>>>   o  Methods for mitigating the impact of non-conformant and malicious
>>>      flows.
>>=20
>> Methods or configurations that leverage deployed AQMs such as RED/WRED t=
o reduce delays and lockout for typical traffic which require minimal effor=
t or tuning from the operator.
>=20
> Not a complete sentence, but I think I understand what you're getting at.=
 You would like to have research determine how to easily configure existing=
 systems using the tools at hand. I'm all for it in the near term.

From john@jlc.net  Thu Nov  7 17:05:51 2013
Return-Path: <john@jlc.net>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5DC3121E814B for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 17:05:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.279
X-Spam-Level: 
X-Spam-Status: No, score=-106.279 tagged_above=-999 required=5 tests=[AWL=0.093, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bbZBSutHF2C4 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 17:05:47 -0800 (PST)
Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by ietfa.amsl.com (Postfix) with ESMTP id C712C21E8121 for <aqm@ietf.org>; Thu,  7 Nov 2013 17:05:46 -0800 (PST)
Received: by mailhost.jlc.net (Postfix, from userid 104) id 1785CC94C1; Thu,  7 Nov 2013 20:05:38 -0500 (EST)
Date: Thu, 7 Nov 2013 20:05:38 -0500
From: John Leslie <john@jlc.net>
To: Bob Briscoe <bob.briscoe@bt.com>
Message-ID: <20131108010538.GD70761@verdi>
References: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk>
User-Agent: Mutt/1.4.1i
Cc: AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 01:05:51 -0000

Bob Briscoe <bob.briscoe@bt.com> wrote:
> 
> "Immediate ECN" slides:
> 
> <http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
> 
> PS. This talk fell off the end of the TSVAREA agenda. It's mostly 
> relevant to AQM, but I didn't originally bring it to AQM, because it 
> affects 3 wgs: tsvwg, aqm & tcpm.

   (responding only on AQM...)

> In the AQM wg, there was dismay about CableLabs not including 
> anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and 
> how ECN can take out the 100ms of delay that CoDel and PIE introduce 
> - it's essentially about auto-tuning for RTT.

   I don't entirely follow what Bob is proposing; but in essence he
proposes that packets which would _not_ otherwise be dropped get an
ECN mark, and let the transport apply "smoothing" to correct for the
RTT of buffering necessary to fill the available bandwidth.

   There are various ways to accomplish this, and I won't consider
them here: the essential question is whether we consider latency
important enough to diddle with ECN signaling to allow short-RTT flows
to react to congestion sooner than long-RTT flows (rather than have
the forwarding node guess what RTT is proper).

   I strongly endorse work in that direction.

> It gives an interim recommendation for hardware designers that there 
> should be a second instance of the AQM algo for ECN packets so that 
> it can be configured with different parameters (think of WRED instead of 
> RED).
> 
> Specifically, for ECN packets:
> interval = 0 (for CoDel)
> max_burst = 0 (for PIE)

   As I understand it, Bob wishes to apply different parameters for
ECN-capable packets in order to remove the allowance for buffering
sufficient to cover the RTT. This is definitely not the only way to
think about the issue; and Bob could do a very good job of explaining
other ways (which I likely would prefer).

   But at this point, Bob is simply asking whether the whole idea
is worth talking about in the IETF...

====

   The AQM Friday agenda looks tight; so I'm not expecting Bob's
slides to be covered then. To be blunt, I will attend CLUE instead
unless ECN issues will be covered Friday...

   Regardless, I intend to follow the AQM list carefully.

--
John Leslie <john@jlc.net>

From preethi.cis@gmail.com  Thu Nov  7 18:47:09 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7C33221F9CA5 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 18:47:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dJcCX9c7qUE6 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 18:47:08 -0800 (PST)
Received: from mail-pd0-x22c.google.com (mail-pd0-x22c.google.com [IPv6:2607:f8b0:400e:c02::22c]) by ietfa.amsl.com (Postfix) with ESMTP id 76EAA21E80F4 for <aqm@ietf.org>; Thu,  7 Nov 2013 18:47:06 -0800 (PST)
Received: by mail-pd0-f172.google.com with SMTP id w10so1489580pde.31 for <aqm@ietf.org>; Thu, 07 Nov 2013 18:47:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:message-id:thread-topic:in-reply-to :mime-version:content-type; bh=WAejQoQ+Pnu7N0WV8qwNNQE39tHk6BHs1rgufb+7HWg=; b=DwbImV7Gj2Sq87rMwV5OmrXIPNiWKWB5M0yCBXKyWVEc8PTAdTgkfNnCHl+fSwgdQG h0oX4EmHC0BEbybQB0lJgL9gf4mRvdo4KD/qwvo7rKem7PRqIdIyt64DXoS9xcpwN9V6 Ep9FdSBChszemEYQHvlFX6KD4vp1GQXjimKgJx8+ikmYhCJnMF2her/7meTJwun079u2 J6qpFDGOrzr/CTHhz/M5gwGWugHvX39MmgspmWaJlRAcs72gpS6INYwom6EM1TjyKslE FPbnfih9pM7ROVzdt64TEbtK547qPggOssjnEpZNl4sODKfxQG2cvHK9EgBC2q9bpKkU Sxvg==
X-Received: by 10.68.179.4 with SMTP id dc4mr12263530pbc.45.1383878826037; Thu, 07 Nov 2013 18:47:06 -0800 (PST)
Received: from [192.168.1.105] (c-76-103-130-90.hsd1.ca.comcast.net. [76.103.130.90]) by mx.google.com with ESMTPSA id i10sm10149993pat.11.2013.11.07.18.46.56 for <aqm@ietf.org> (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 07 Nov 2013 18:47:05 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Thu, 07 Nov 2013 18:46:39 -0700
From: Preethi Natarajan <preethi.cis@gmail.com>
To: <aqm@ietf.org>
Message-ID: <CEA1905E.4AE3B%prenatar@cisco.com>
Thread-Topic: AQM schemes: Queue length vs. delay based
In-Reply-To: <CEA17DD6.4AE2D%preethi.cis@gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3466694820_9734681"
Subject: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 02:47:09 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3466694820_9734681
Content-type: text/plain;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

Hello AQMers:

Just wanted to bring up the following item for discussion either as part of
the recommendations draft or the evaluation criteria during Friday's
session/mailing list.

In access, edge and core routers the draining rate of a queue is affected b=
y
traffic on other queues and thus can vary a lot (depending on the deploymen=
t
and traffic conditions). A queue length based AQM scheme such as RED or
derivatives tries to maintain the average queue size around a predictable
value under these changing draining rates. However, this queue size
translates to high queuing delay under low draining rates and vice-versa.
The unpredictability in resulting queueing delay was one of the reasons why
we opted PIE to be a latency-based scheme.

A queue length based AQM scheme could be perfectly valid for certain
deployments. For deployments where predictable queuing delay is expected
under varying draining rates, a latency based AQM is critical. We believe
this should be brought about in discussions somewhere at AQM =8B perhaps in
the recommendations draft or w.r.t evaluation criteria.

Thanks,
Preethi (on behalf of PIE team)

=20



--B_3466694820_9734681
Content-type: text/html;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:=
 14px; font-family: Calibri, sans-serif; "><div><div>Hello AQMers:</div><div=
><br></div><div>Just wanted to bring up the following item for discussion ei=
ther as part of the recommendations draft or the evaluation criteria during =
Friday's session/mailing list.</div><div><br></div><div>In access, edge and =
core routers the draining rate of a queue is affected by traffic on other qu=
eues and thus can vary a lot (depending on the deployment and traffic condit=
ions). A queue length based AQM scheme such as RED or derivatives tries to m=
aintain the average queue size around a predictable value under these changi=
ng draining rates. However, this queue size translates to high queuing delay=
 under low draining rates and vice-versa. The unpredictability in resulting =
queueing delay was one of the reasons why we opted PIE to be a latency-based=
 scheme.&nbsp;</div><div><br></div><div>A queue length based AQM scheme coul=
d be perfectly valid for certain deployments. For deployments where predicta=
ble queuing delay is expected under varying draining rates, a latency based =
AQM is critical. We believe this should be brought about in discussions some=
where at AQM &#8212; perhaps in the recommendations draft or w.r.t evaluatio=
n criteria.&nbsp;</div><div><br></div><div>Thanks,</div><div>Preethi (on beh=
alf of PIE team)</div></div><span id=3D"OLK_SRC_BODY_SECTION"><div><div style=3D=
"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-=
white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, san=
s-serif; "><div><div><br></div></div><div>&nbsp;</div></div></div></span></b=
ody></html>

--B_3466694820_9734681--



From naeem.khademi@gmail.com  Thu Nov  7 21:50:30 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6113911E8158 for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 21:50:30 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.429
X-Spam-Level: 
X-Spam-Status: No, score=-2.429 tagged_above=-999 required=5 tests=[AWL=-0.057, BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xjtWoycv3uGu for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 21:50:29 -0800 (PST)
Received: from mail-ve0-x230.google.com (mail-ve0-x230.google.com [IPv6:2607:f8b0:400c:c01::230]) by ietfa.amsl.com (Postfix) with ESMTP id C11E411E817D for <aqm@ietf.org>; Thu,  7 Nov 2013 21:50:24 -0800 (PST)
Received: by mail-ve0-f176.google.com with SMTP id c14so1021194vea.7 for <aqm@ietf.org>; Thu, 07 Nov 2013 21:50:24 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=P5Jk9fP9+RZSQl6wta99pziZc0+h/kESp0lV4vIlYCA=; b=Yy8n/uxx1Gz/HAnZSuQXJ7uK8trrKGhvgz+YXXULOjYqkmFrR4t154mUucg1KZ6DbX 1yJoOLlTImQsuSXtja0Do/Ds5HfP7FTsoxR/0oDYCd/XU5WIeBU5n4SjAb/d0Z98ZSL4 EoALGVoQD6xru5qEAryKf+Hn/2RqeCoJPTpD9Yrjhjg5yR9lKa0thbPRAd/ulAYtopUd 4J1a0sOF6IRYKYULX1jbjixp7mKM/G0p5gfFT50mIlqFpBXOr++cjqcxeH/d0SbuNkUJ cjrpAbJPEXLiI4WItOqIfEqh6AcvVFgnVku5dn1ssyWdYz8jH85BrWc/+4GIKkTlb+7A JNrQ==
MIME-Version: 1.0
X-Received: by 10.52.97.138 with SMTP id ea10mr8524240vdb.31.1383889824156; Thu, 07 Nov 2013 21:50:24 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 7 Nov 2013 21:50:24 -0800 (PST)
In-Reply-To: <CEA1905E.4AE3B%prenatar@cisco.com>
References: <CEA17DD6.4AE2D%preethi.cis@gmail.com> <CEA1905E.4AE3B%prenatar@cisco.com>
Date: Thu, 7 Nov 2013 21:50:24 -0800
Message-ID: <CAEjQQ5UdW38nAm4WSO8CXL5B8Eo5weu65L17uYVzZJvzzZU3fw@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=20cf307f38f86093b904eaa3f570
Cc: "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 05:50:30 -0000

--20cf307f38f86093b904eaa3f570
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

I fully agree with the need to have AQMs that use "queuing delay" as metric
and not "queue length" and this is already a bonus for algorithms that use
this metric (e.g. PIE). However, I believe that one can possibly modify RED
or its derivatives (e.g. Adaptive RED) to set their thresholds based on
"queuing delay" as well (although such algorithms don't exists yet as of
now). Whether the outcome of these modifications to xRED algorithms would
make them better than PIE and/or CoDel is another subject and I'm not going
to speculate on that here.

One specific issue with PIE is that it tries to *estimate* the queuing
delay over a certain time interval (t_update) and still somehow uses the
"queue length" and departure rate (over the interval) to estimate the
queuing delay. So saying that PIE uses the queueing delay instead of queue
length may not be 100% accurate.

Moreover, in a varying bandwidth scenario, depending on how often the
bandwidth changes and at what granularity, the derived "estimated queuing
delay" by PIE might be an outdated information e.g. it shows how the
channel has been (in term of delay) in the last 30ms and now how it is now!
if I'm not mistaken that has been somehow resolved in DOCSIS 3.1 by
coupling PIE's departure rate to the actual link layer rate.

In overall, while there are of course significant improvements made in
PIE's (and (FQ)-CoDel's) design, there are probably still lessons to be
learned from xRED family (check http://urn.nb.no/URN:NBN:no-38868 to see
some examples). Throwing away all the knowledge and improvement gained from
RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs without
(experimentally) comparing them to the gallery of existing ones wouldn't be
great!

Cheers,
Naeem



On Thu, Nov 7, 2013 at 5:46 PM, Preethi Natarajan <preethi.cis@gmail.com>wr=
ote:

> Hello AQMers:
>
> Just wanted to bring up the following item for discussion either as part
> of the recommendations draft or the evaluation criteria during Friday's
> session/mailing list.
>
> In access, edge and core routers the draining rate of a queue is affected
> by traffic on other queues and thus can vary a lot (depending on the
> deployment and traffic conditions). A queue length based AQM scheme such =
as
> RED or derivatives tries to maintain the average queue size around a
> predictable value under these changing draining rates. However, this queu=
e
> size translates to high queuing delay under low draining rates and
> vice-versa. The unpredictability in resulting queueing delay was one of t=
he
> reasons why we opted PIE to be a latency-based scheme.
>
> A queue length based AQM scheme could be perfectly valid for certain
> deployments. For deployments where predictable queuing delay is expected
> under varying draining rates, a latency based AQM is critical. We believe
> this should be brought about in discussions somewhere at AQM =97 perhaps =
in
> the recommendations draft or w.r.t evaluation criteria.
>
> Thanks,
> Preethi (on behalf of PIE team)
>
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>

--20cf307f38f86093b904eaa3f570
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I fully agree with the need to have AQMs that use &quot;qu=
euing delay&quot; as metric and not &quot;queue length&quot; and this is al=
ready a bonus for algorithms that use this metric (e.g. PIE). However, I be=
lieve that one can possibly modify RED or its derivatives (e.g. Adaptive RE=
D) to set their thresholds based on &quot;queuing delay&quot; as well (alth=
ough such algorithms don&#39;t exists yet as of now). Whether the outcome o=
f these modifications to xRED algorithms would make them better than PIE an=
d/or CoDel is another subject and I&#39;m not going to speculate on that he=
re.=A0<div>
<br></div><div>One specific issue with PIE is that it tries to *estimate* t=
he queuing delay over a certain time interval (t_update) and still somehow =
uses the &quot;queue length&quot; and departure rate (over the interval) to=
 estimate the queuing delay. So saying that PIE uses the queueing delay ins=
tead of queue length may not be 100% accurate.=A0</div>
<div><br></div><div>Moreover, in a varying bandwidth scenario, depending on=
 how often the bandwidth changes and at what granularity, the derived &quot=
;estimated queuing delay&quot; by PIE might be an outdated information e.g.=
 it shows how the channel has been (in term of delay) in the last 30ms and =
now how it is now! if I&#39;m not mistaken that has been somehow resolved i=
n DOCSIS 3.1 by coupling PIE&#39;s departure rate to the actual link layer =
rate. =A0</div>
<div><br></div><div>In overall, while there are of course significant impro=
vements made in PIE&#39;s (and (FQ)-CoDel&#39;s) design, there are probably=
 still lessons to be learned from xRED family (check=A0<a href=3D"http://ur=
n.nb.no/URN:NBN:no-38868" target=3D"_blank" style=3D"font-size:13px;font-fa=
mily:arial,helvetica,sans-serif">http://urn.nb.no/URN:NBN:no-38868</a>=A0to=
 see some examples). Throwing away all the knowledge and improvement gained=
 from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs without (=
experimentally) comparing them to the gallery of existing ones wouldn&#39;t=
 be great!</div>
<div><br></div><div>Cheers,</div><div>Naeem<br><div class=3D"gmail_extra"><=
br></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On T=
hu, Nov 7, 2013 at 5:46 PM, Preethi Natarajan <span dir=3D"ltr">&lt;<a href=
=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</=
a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div style=3D"font-size:14px;font-family:Calibri,sans-seri=
f;word-wrap:break-word">
<div><div>Hello AQMers:</div><div><br></div><div>Just wanted to bring up th=
e following item for discussion either as part of the recommendations draft=
 or the evaluation criteria during Friday&#39;s session/mailing list.</div>
<div><br></div><div>In access, edge and core routers the draining rate of a=
 queue is affected by traffic on other queues and thus can vary a lot (depe=
nding on the deployment and traffic conditions). A queue length based AQM s=
cheme such as RED or derivatives tries to maintain the average queue size a=
round a predictable value under these changing draining rates. However, thi=
s queue size translates to high queuing delay under low draining rates and =
vice-versa. The unpredictability in resulting queueing delay was one of the=
 reasons why we opted PIE to be a latency-based scheme.=A0</div>
<div><br></div><div>A queue length based AQM scheme could be perfectly vali=
d for certain deployments. For deployments where predictable queuing delay =
is expected under varying draining rates, a latency based AQM is critical. =
We believe this should be brought about in discussions somewhere at AQM =97=
 perhaps in the recommendations draft or w.r.t evaluation criteria.=A0</div=
>
<div><br></div><div>Thanks,</div><div>Preethi (on behalf of PIE team)</div>=
</div><span><div><div style=3D"font-size:14px;font-family:Calibri,sans-seri=
f;word-wrap:break-word"><div><div><br></div></div><div>=A0</div></div></div=
>
</span></div>
<br>_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><br>
<br></blockquote></div><br></div></div></div>

--20cf307f38f86093b904eaa3f570--

From naeem.khademi@gmail.com  Thu Nov  7 22:38:29 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D745521E805D for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 22:38:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.41
X-Spam-Level: 
X-Spam-Status: No, score=-2.41 tagged_above=-999 required=5 tests=[AWL=-0.038,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ozFk4Jeov64S for <aqm@ietfa.amsl.com>; Thu,  7 Nov 2013 22:38:29 -0800 (PST)
Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) by ietfa.amsl.com (Postfix) with ESMTP id 5EFD221E81C9 for <aqm@ietf.org>; Thu,  7 Nov 2013 22:38:24 -0800 (PST)
Received: by mail-vc0-f181.google.com with SMTP id ij19so1110330vcb.12 for <aqm@ietf.org>; Thu, 07 Nov 2013 22:38:21 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ag3voyJY0WtD5Z7koh+Ambpw08qsL/ZAwGvJI04r5+g=; b=dKudc7T2kvaVngIvylSAthNZVJO+pLcoVzGzZkchOahpXOsoSs7rXEkkapVIWLiGoz jQp2MFuKo9INhAjMGsLWpTkeIjlNt8F+qq5p8DefO4lkQRZN3w9maLq/UTbxyo5sImsq vIaaJXgKx1sonnAHC/3NdzhJsu/JH3qhHMcG4SUbJpa15aBhtTLAWrjOu4ZLJu3VebCj Za9GIwxnChLzP04KKj2OyhWcVtb/EsA8Le80qUwVCfw7/Zjo3Zlo7GcLCQtZRs7TvIis V3kZDyBdSpLhRXTfPrzE3nk6pL70yWNaSoVUoPAekMYjYL5mtr6CiNS3/53Vg5YsjDqe O8hA==
MIME-Version: 1.0
X-Received: by 10.221.39.195 with SMTP id tn3mr10381211vcb.2.1383892701826; Thu, 07 Nov 2013 22:38:21 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 7 Nov 2013 22:38:21 -0800 (PST)
In-Reply-To: <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
Date: Thu, 7 Nov 2013 22:38:21 -0800
Message-ID: <CAEjQQ5U85aovjTM_L0+-2m4a72PYMusxKaVVxdCKmpgicr-atQ@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Content-Type: multipart/alternative; boundary=001a113375c2e6534e04eaa4a03d
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "Akhtar, Shahid \(Shahid\)" <shahid.akhtar@alcatel-lucent.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 06:38:30 -0000

--001a113375c2e6534e04eaa4a03d
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi

Snipping other stuff, I would like to add some specific points inline:


On Thu, Nov 7, 2013 at 10:45 AM, Fred Baker (fred) <fred@cisco.com> wrote:

>
> On Nov 7, 2013, at 8:59 AM, "Akhtar, Shahid (Shahid)" <
> shahid.akhtar@alcatel-lucent.com> wrote:
>
> >> 4.  Conclusions and Recommendations
> >>   [snip]
> >>    3.  The algorithms that the IETF recommends SHOULD NOT require
> >>        operational (especially manual) configuration or tuning.
> >
> > Some tuning may be required or implicitly assumed for virtually all AQM=
s
> =96 please see my comment later.
>
> That's an opinion. One of the objectives of Van and Kathy's work, and
> separately of Rong Pan et al's work, is to design an algorithm that may
> have different initial conditions drawn from a table given the interface =
it
> finds itself on, but requires no manual tuning. The great failure of RED,
> recommended in RFC 2309, is not that it doesn't work when properly
> configured; it's that real humans don't have the time to properly tune it
> differently for each of the thousands of link endpoints in their networks=
.


and "Adaptive RED" was an attempt to address exactly that issue! but it
seems that it has likely to have remained un-noticed since 2001. The draft
on Sally's homepage still says "August 1, 2001, under submission" while
it's been in Linux for a while :-).


> There is no point in changing away from RED if that is also true of the
> replacement.


> >>    7.  Research, engineering, and measurement efforts are needed
> >>        regarding the design of mechanisms to deal with flows that are
> >>        unresponsive to congestion notification or are responsive, but
> >>        are more aggressive than present TCP.
> >
> >       Do we want to make a suggestion on how to configure buffer sizes
> with each type of AQM here (e.g. 2xBDP etc) or simply state that research
> should be conducted on the best buffer sizes to use with AQM.
>
> I'm not sure that buffer sizes are specific to AQM algorithms; I'd
> entertain evidence otherwise. Buffer *thresholds* ("at what point do we
> start dropping/marking traffic?") may differ between algorithms. Buffer
> size ("how many bytes/packets do we allow into the queue in the worst
> case?") is a matter of the characteristics of burst behavior in a given
> network and the applications it supports. If I have, say, a Map/Reduce
> application that simultaneously asks thousands of systems a question, the
> queues in the intervening switches will need to be able to briefly absorb
> thousands of response packets. The key word here is "briefly". When Van o=
r
> Kathy talk about "good queue" and "bad queue", they are saying that burst
> behavior may call for deep queues, but we really want the steady state to
> achieve 100% utilization with a statistically empty queue if we can
> possibly achieve that.
>

The terms "good queue", "bad queue" or "briefly" are very subjective. How
"brief" is the key. We can possibly know that when we're talking about a
specific application in a specific environment (e.g. Map/Reduce in data
center). On the access link with a mixture of all sort of traffics, and
also RTTs , that's more controversial. A queue that gives a 100 ms burst a
pass, is a good queue from CoDel designer's point of view (!)

Cheers,
Naeem

--001a113375c2e6534e04eaa4a03d
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi=A0<div><br></div><div>Snipping other stuff, I would lik=
e to add some specific points inline:=A0<br><div class=3D"gmail_extra"><br>=
<br><div class=3D"gmail_quote">On Thu, Nov 7, 2013 at 10:45 AM, Fred Baker =
(fred) <span dir=3D"ltr">&lt;<a href=3D"mailto:fred@cisco.com" target=3D"_b=
lank">fred@cisco.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"im"><br>
On Nov 7, 2013, at 8:59 AM, &quot;Akhtar, Shahid (Shahid)&quot; &lt;<a href=
=3D"mailto:shahid.akhtar@alcatel-lucent.com">shahid.akhtar@alcatel-lucent.c=
om</a>&gt; wrote:<br><br></div>
&gt;&gt; 4. =A0Conclusions and Recommendations<br>
&gt;&gt; =A0 [snip]<br>
&gt;&gt; =A0 =A03. =A0The algorithms that the IETF recommends SHOULD NOT re=
quire<br>
&gt;&gt; =A0 =A0 =A0 =A0operational (especially manual) configuration or tu=
ning.<br>
&gt;<br>
&gt; Some tuning may be required or implicitly assumed for virtually all AQ=
Ms =96 please see my comment later.<br>
<br>
That&#39;s an opinion. One of the objectives of Van and Kathy&#39;s work, a=
nd separately of Rong Pan et al&#39;s work, is to design an algorithm that =
may have different initial conditions drawn from a table given the interfac=
e it finds itself on, but requires no manual tuning. The great failure of R=
ED, recommended in RFC 2309, is not that it doesn&#39;t work when properly =
configured; it&#39;s that real humans don&#39;t have the time to properly t=
une it differently for each of the thousands of link endpoints in their net=
works.</blockquote>
<div><br></div><div>and &quot;Adaptive RED&quot; was an attempt to address =
exactly that issue! but it seems that it has likely to have remained un-not=
iced since 2001. The draft on Sally&#39;s homepage still says &quot;August =
1, 2001, under submission&quot; while it&#39;s been in Linux for a while :-=
).=A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"> There is no point in changing=
 away from RED if that is also true of the replacement.=A0</blockquote><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex">

<br>
&gt;&gt; =A0 =A07. =A0Research, engineering, and measurement efforts are ne=
eded<br>
&gt;&gt; =A0 =A0 =A0 =A0regarding the design of mechanisms to deal with flo=
ws that are<br>
&gt;&gt; =A0 =A0 =A0 =A0unresponsive to congestion notification or are resp=
onsive, but<br>
&gt;&gt; =A0 =A0 =A0 =A0are more aggressive than present TCP.<br>
&gt;<br>
&gt; =A0 =A0 =A0 Do we want to make a suggestion on how to configure buffer=
 sizes with each type of AQM here (e.g. 2xBDP etc) or simply state that res=
earch should be conducted on the best buffer sizes to use with AQM.<br>
<br>
I&#39;m not sure that buffer sizes are specific to AQM algorithms; I&#39;d =
entertain evidence otherwise. Buffer *thresholds* (&quot;at what point do w=
e start dropping/marking traffic?&quot;) may differ between algorithms. Buf=
fer size (&quot;how many bytes/packets do we allow into the queue in the wo=
rst case?&quot;) is a matter of the characteristics of burst behavior in a =
given network and the applications it supports. If I have, say, a Map/Reduc=
e application that simultaneously asks thousands of systems a question, the=
 queues in the intervening switches will need to be able to briefly absorb =
thousands of response packets. The key word here is &quot;briefly&quot;. Wh=
en Van or Kathy talk about &quot;good queue&quot; and &quot;bad queue&quot;=
, they are saying that burst behavior may call for deep queues, but we real=
ly want the steady state to achieve 100% utilization with a statistically e=
mpty queue if we can possibly achieve that.<br>
</blockquote><div><br></div><div>The terms &quot;good queue&quot;, &quot;ba=
d queue&quot; or &quot;briefly&quot; are very subjective. How &quot;brief&q=
uot; is the key. We can possibly know that when we&#39;re talking about a s=
pecific application in a specific environment (e.g. Map/Reduce in data cent=
er). On the access link with a mixture of all sort of traffics, and also RT=
Ts , that&#39;s more controversial. A queue that gives a 100 ms burst a pas=
s, is a good queue from CoDel designer&#39;s point of view (!)=A0</div>
<div><br></div><div>Cheers,</div><div>Naeem</div></div></div></div></div>

--001a113375c2e6534e04eaa4a03d--

From shahid.akhtar@alcatel-lucent.com  Fri Nov  8 05:57:13 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B253321E8106 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 05:57:13 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.418
X-Spam-Level: 
X-Spam-Status: No, score=-10.418 tagged_above=-999 required=5 tests=[AWL=0.180, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DzhByJXfyWyP for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 05:56:57 -0800 (PST)
Received: from ihemail1.lucent.com (ihemail1.lucent.com [135.245.0.33]) by ietfa.amsl.com (Postfix) with ESMTP id 0C51121E8113 for <aqm@ietf.org>; Fri,  8 Nov 2013 05:56:56 -0800 (PST)
Received: from us70uusmtp3.zam.alcatel-lucent.com (h135-5-2-65.lucent.com [135.5.2.65]) by ihemail1.lucent.com (8.13.8/IER-o) with ESMTP id rA8Dulil013018 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 8 Nov 2013 07:56:48 -0600 (CST)
Received: from US70UWXCHHUB02.zam.alcatel-lucent.com (us70uwxchhub02.zam.alcatel-lucent.com [135.5.2.49]) by us70uusmtp3.zam.alcatel-lucent.com (GMO) with ESMTP id rA8DulQX006681 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Nov 2013 08:56:47 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70UWXCHHUB02.zam.alcatel-lucent.com ([135.5.2.49]) with mapi id 14.02.0247.003; Fri, 8 Nov 2013 08:56:46 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: Ac7aemDyYVICOG/TQKCzUJnt0vYtGgAxY8ewADTbyQAACoMYkA==
Date: Fri, 8 Nov 2013 13:56:46 +0000
Message-ID: <C0611F522A6FA9498A044C36073E49657E5FFA98@US70UWXCHMBA01.zam.alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
In-Reply-To: <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.5.27.16]
Content-Type: multipart/alternative; boundary="_000_C0611F522A6FA9498A044C36073E49657E5FFA98US70UWXCHMBA01z_"
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.33
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 13:57:14 -0000

--_000_C0611F522A6FA9498A044C36073E49657E5FFA98US70UWXCHMBA01z_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Fred,

I have taken the liberty to tie your comments to the e-mail list of comment=
s I had sent and put a "FB" next to your comment. I have added "SA" next to=
 my comments. This  is easier for people to read.

-Shahid.

____________________________________________________

Section 4.3 - Manual tuning

The document states and implies that AQMs should not require manual tuning.=
 In section 4.3 it states "The algorithms that the IETF recommends SHOULD N=
OT require operational (especially manual) configuration or tuning". I reco=
mmend to change this sentence to: "The algorithms that the IETF recommends =
should minimize tuning or configurations changes for specific traffic or ne=
twork conditions"

I would further argue that all AQMs will likely require or implicitly assum=
e some type of configuration/tuning.

For example, if we take Codel:
*       For small thin links, such as 1-10Mbs DSL, the 5ms target would inc=
rease packet loss significantly and at 2Mbps, a single MTU time may even ex=
ceed the 5ms target.
*       If the average RTT of all flows going through a link is more than 5=
00ms, e.g. for satellite, then the 100ms interval would prematurely drop pa=
ckets before the sources have had a chance to reduce their sending rate. Or=
 if the average RTT is very low - e.g. 10ms - such as for flows between dat=
a-center elements, then 100ms interval may be slow to signal congestion bac=
k to the sources and significant packet loss may have occurred before such =
signaling.

One of the the objectives of newer AQMs being defined here should be to min=
imize tuning, but we should recognize that likely tuning or some configurat=
ion cannot be eliminated altogether.

FB: That's an opinion. One of the objectives of Van and Kathy's work, and s=
eparately of Rong Pan et al's work, is to design an algorithm that may have=
 different initial conditions drawn from a table given the interface it fin=
ds itself on, but requires no manual tuning. The great failure of RED, reco=
mmended in RFC 2309, is not that it doesn't work when properly configured; =
it's that real humans don't have the time to properly tune it differently f=
or each of the thousands of link endpoints in their networks. There is no p=
oint in changing away from RED if that is also true of the replacement.

SA: You argue that "initial conditions" determine some of the parameters of=
 newer AQMs (like Codel and PIE), then those same initial conditions would =
also determine some of the key parameters for RED/WRED.

Can you explain further why "real humans don't have the time to properly tu=
ne it differently for each of the thousands of link endpoints in their netw=
orks" with a realistic example.

FB: What you describe is what I referred to as "initial conditions derived =
from the links the algorithm finds itself on". We may be in violent agreeme=
nt there. If so, the wording I might suggest would be "SHOULD NOT require o=
perational (especially manual) configuration or tuning apart from automated=
 determination of initial conditions" or some such thing.

SA: Looks like we may be agreeing on something here - which is great. It wo=
uld be valuable to describe the meaning of "initial conditions" somewhere.

Section 4.7 - Further Research

AQM impact on end-user QoE
Research should also focus on improving end user QoE from AQMs rather than =
network related metrics only. Often a significant change in a network metri=
c may only make a minimal change in end-user QoE and thus the value of such=
 change may be minimal.

FB: There is also a question of what user is under discussion. If you take =
a look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx an=
d specifically the third slide, you will see (and tomorrow afternoon I will=
 discuss) a capture I took of pings from my home network to another site ov=
ernight. In the evening, we watched a movie (home network), and in the morn=
ing I had a video conference (office network). I'll tell you right now that=
 both worked fine, and managing the delay to zero or allowing it to be one =
hundred ms would not have materially affected the QOE of either. However, m=
y ping is a competing application, and it saw sustained increase in delay, =
variation in delay, and the possibility of loss as a result of the queuing.=
 The value of AQM is in part to the application being throttled, but in lar=
ge part to competing applications, and the QOE of both must be considered.

SA: My point about bringing up QoE is to tie the impact of a new investment=
 (such as AQM algorithms) to an operator to revenue. By improving QoE of re=
venue making services, an operator may determine that it makes business sen=
se to invest. I do not understand how "ping" - is linked a revenue making s=
ervice? The QoE of Web browsing which is a key service to end-users is clos=
ely tied to latency as the PLT of a web page is directly linked to RTT. How=
ever the QoE of different services such as web browsing, HAS Video, progess=
ive download video and others should be weighed according to their importan=
ce to the end user.

Suggestion on best buffer sizes
Research should make suggestions on how to configure buffer sizes with each=
 type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer sizes improv=
e end-user QoE and network health.

FB: I'm not sure that buffer sizes are specific to AQM algorithms; I'd ente=
rtain evidence otherwise. Buffer *thresholds* ("at what point do we start d=
ropping/marking traffic?") may differ between algorithms. Buffer size ("how=
 many bytes/packets do we allow into the queue in the worst case?") is a ma=
tter of the characteristics of burst behavior in a given network and the ap=
plications it supports. If I have, say, a Map/Reduce application that simul=
taneously asks thousands of systems a question, the queues in the interveni=
ng switches will need to be able to briefly absorb thousands of response pa=
ckets. The key word here is "briefly". When Van or Kathy talk about "good q=
ueue" and "bad queue", they are saying that burst behavior may call for dee=
p queues, but we really want the steady state to achieve 100% utilization w=
ith a statistically empty queue if we can possibly achieve that.

SA: In access situations (with AQM) buffer sizes can determine the size of =
a burst that the buffer can withstand and our research shows that it can di=
rectly impact end user QoE.

Best way to leverage deployed AQMs
Research should be done on methods or configurations that leverage deployed=
 AQMs such as RED/WRED to reduce delays and lockout for typical traffic whi=
ch require minimal effort or tuning from the operator.

FB: Not a complete sentence, but I think I understand what you're getting a=
t. You would like to have research determine how to easily configure existi=
ng systems using the tools at hand. I'm all for it in the near term.

SA: Glad we agree on this - assume some update will be made around this top=
ic.


-----Original Message-----
From: Fred Baker (fred) [mailto:fred@cisco.com]
Sent: Thursday, November 07, 2013 12:45 PM
To: Akhtar, Shahid (Shahid)
Cc: Richard Scheffenegger; aqm@ietf.org; Naeem Khademi (naeemk@ifi.uio.no);=
 Gorry Fairhurst; Wesley Eddy
Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B


On Nov 7, 2013, at 8:59 AM, "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcate=
l-lucent.com> wrote:

> Hi All,
>
> Had some comments on Fred's document. I have added the comments as track =
changes in a word document to easily see them. I used the 02 version.
>
> Thanks.


Permit me to put your comments in email, along with my own views. Also addi=
ng my co-author and the other working group chair on the CC line; if he is =
like me, he receives far too much email, and mail that is explicitly to or =
copies him bubbles higher in the column.

>> 4.  Conclusions and Recommendations
>>   [snip]
>>    3.  The algorithms that the IETF recommends SHOULD NOT require
>>        operational (especially manual) configuration or tuning.
>
> Some tuning may be required or implicitly assumed for virtually all AQMs =
- please see my comment later.

FB: That's an opinion. One of the objectives of Van and Kathy's work, and s=
eparately of Rong Pan et al's work, is to design an algorithm that may have=
 different initial conditions drawn from a table given the interface it fin=
ds itself on, but requires no manual tuning. The great failure of RED, reco=
mmended in RFC 2309, is not that it doesn't work when properly configured; =
it's that real humans don't have the time to properly tune it differently f=
or each of the thousands of link endpoints in their networks. There is no p=
oint in changing away from RED if that is also true of the replacement.

SA: You argue that "initial conditions" determine some of the parameters of=
 newer AQMs (like Codel and PIE), then those same initial conditions would =
also determine the key parameters for RED/WRED.

Can you explain further why "real humans don't have the time to properly tu=
ne it differently for each of the thousands of link endpoints in their netw=
orks" with a realistic example.

>>    7.  Research, engineering, and measurement efforts are needed
>>        regarding the design of mechanisms to deal with flows that are
>>        unresponsive to congestion notification or are responsive, but
>>        are more aggressive than present TCP.
>
>       Do we want to make a suggestion on how to configure buffer sizes wi=
th each type of AQM here (e.g. 2xBDP etc) or simply state that research sho=
uld be conducted on the best buffer sizes to use with AQM.

FB: I'm not sure that buffer sizes are specific to AQM algorithms; I'd ente=
rtain evidence otherwise. Buffer *thresholds* ("at what point do we start d=
ropping/marking traffic?") may differ between algorithms. Buffer size ("how=
 many bytes/packets do we allow into the queue in the worst case?") is a ma=
tter of the characteristics of burst behavior in a given network and the ap=
plications it supports. If I have, say, a Map/Reduce application that simul=
taneously asks thousands of systems a question, the queues in the interveni=
ng switches will need to be able to briefly absorb thousands of response pa=
ckets. The key word here is "briefly". When Van or Kathy talk about "good q=
ueue" and "bad queue", they are saying that burst behavior may call for dee=
p queues, but we really want the steady state to achieve 100% utilization w=
ith a statistically empty queue if we can possibly achieve that.

SA: In access situations (with AQM) buffer sizes can determine the size of =
a burst that the buffer can withstand and our research shows that it can di=
rectly impact end user QoE.

>> 4.3.  AQM algorithms deployed SHOULD NOT require operational tuning
>>
>>    A number of algorithms have been proposed.  Many require some form of
>>    tuning or initial condition.  This can make them difficult to use
>>    operationally.  Hence, self-tuning algorithms are to be preferred.
>>    The algorithms that the IETF recommends SHOULD NOT require
>>    operational (especially manual) configuration or tuning.
>
> May be better to state that tuning should be minimized. For the second se=
ntence "The algorithms that the IETF recommends should minimize tuning or c=
onfigurations changes for specific traffic or network conditions"
>
> I would argue that all AQMs will likely require or assume some type of co=
nfiguration/tuning.
>
> For example, if we take Codel:
>
> *       For small thin links, such as 1-10Mbs DSL, the 5ms target would i=
ncrease packet loss significantly and at 2Mbps, a single MTU time may even =
exceed the 5ms target.
>
> *       If the average RTT of all flows going through a link is more than=
 500ms, e.g. for satellite, then the 100ms interval would prematurely drop =
packets before the sources have had a chance to reduce their sending rate. =
Or if the average RTT is very low - e.g. 10ms - such as for flows between d=
ata-center elements, then 100ms interval may be slow to signal congestion b=
ack to the sources and significant packet loss may have occurred before suc=
h signaling.


FB: What you describe is what I referred to as "initial conditions derived =
from the links the algorithm finds itself on". We may be in violent agreeme=
nt there. If so, the wording I might suggest would be "SHOULD NOT require o=
perational (especially manual) configuration or tuning apart from automated=
 determination of initial conditions" or some such thing.

SA: Looks like we may be agreeing on something here - which is great. It wo=
uld be valuable to describe the meaning of "initial conditions" somewhere.

>> 4.7.  The need for further research
>>
>>    The second recommendation of [RFC2309] called for further research in
>>    the interaction between network queues and host applications, and the
>>    means of signaling between them.  This research has occurred, and we
>>    as a community have learned a lot.  However, we are not done.
>>
>>    We have learned that the problems of congestion, latency and buffer-
>>    sizing have not gone away, and are becoming more important to many
>>    users.  A number of self-tuning AQM algorithms have be found that
>>    offer significant advantages for deployed networks.  There is also
>>    renewed interest in deploying AQM and the potential of ECN.
>>
>>    An obvious example of further research in 2013 is the need to
>>    consider the use of Map/Reduce applications in data centers; do we
>>    need to extend our taxonomy of TCP/SCTP sessions to include not only
>>    "mice" and "elephants", but "lemmings"?  "Lemmings" are flash crowds
>>    of "mice" that the network inadvertently tries to signal to as if
>>    they were elephant flows, resulting in head of line blocking in data
>>    center applications.
>
> Such research should also focus on improving end user QoE from AQMs rathe=
r than network related metrics only. Often a significant change in a networ=
k metric may only make a minimal change in end-user QoE and thus the value =
of such change may be minimal.


FB: There is also a question of what user is under discussion. If you take =
a look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx an=
d specifically the third slide, you will see (and tomorrow afternoon I will=
 discuss) a capture I took of pings from my home network to another site ov=
ernight. In the evening, we watched a movie (home network), and in the morn=
ing I had a video conference (office network). I'll tell you right now that=
 both worked fine, and managing the delay to zero or allowing it to be one =
hundred ms would not have materially affected the QOE of either. However, m=
y ping is a competing application, and it saw sustained increase in delay, =
variation in delay, and the possibility of loss as a result of the queuing.=
 The value of AQM is in part to the application being throttled, but in lar=
ge part to competing applications, and the QOE of both must be considered.

SA: My point about bringing up QoE is to tie the impact of a new investment=
 (such as AQM algorithms) to an operator to revenue. By improving QoE of re=
venue making services, an operator may determine that it makes business sen=
se to invest. I do not understand how "ping" - is linked a revenue making s=
ervice? The QoE of Web browsing which is a key service to end-users is clos=
ely tied to latency as the PLT of a web page is directly linked to RTT. How=
ever the QoE of different services such as web browsing, HAS Video, progess=
ive download video and others should be weighed according to their importan=
ce to the end user.

>>    Examples of other required research include:
>>
>>    o  Research into new AQM and scheduling algorithms.
>>
>>    o  Research into the use of and deployment of ECN alongside AQM.
>>
>>    o  Tools for enabling AQM (and ECN) deployment and measuring the
>>       performance.
>>
>>    o  Methods for mitigating the impact of non-conformant and malicious
>>       flows.
>
> Methods or configurations that leverage deployed AQMs such as RED/WRED to=
 reduce delays and lockout for typical traffic which require minimal effort=
 or tuning from the operator.

FB: Not a complete sentence, but I think I understand what you're getting a=
t. You would like to have research determine how to easily configure existi=
ng systems using the tools at hand. I'm all for it in the near term.

SA: Glad we agree on this - assume some update will be made around this top=
ic.


--_000_C0611F522A6FA9498A044C36073E49657E5FFA98US70UWXCHMBA01z_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Arial" size=3D"2"><span style=3D"font-size:10pt;">
<div>Fred, </div>
<div>&nbsp;</div>
<div>I have taken the liberty to tie your comments to the e-mail list of co=
mments I had sent and put a &quot;<b>FB</b>&quot; next to your comment. I h=
ave added &quot;<b>SA</b>&quot; next to my comments. This&nbsp; is easier f=
or people to read.</div>
<div>&nbsp;</div>
<div>-Shahid.</div>
<div>&nbsp;</div>
<div>____________________________________________________</div>
<div>&nbsp;</div>
<div><b>Section 4.3 - Manual tuning</b></div>
<div>&nbsp;</div>
<div>The document states and implies that AQMs should not require manual tu=
ning. In section 4.3 it states &quot;The algorithms that the IETF recommend=
s SHOULD NOT require operational (especially manual) configuration or tunin=
g&quot;. I recommend to change this sentence
to: &#8220;The algorithms that the IETF recommends should minimize tuning o=
r configurations changes for specific traffic or network conditions&#8221;<=
/div>
<div>&nbsp;</div>
<div>I would further argue that all AQMs will likely require or implicitly =
assume some type of configuration/tuning. </div>
<div>&nbsp;</div>
<div>For example, if we take Codel:</div>
<div>&#8226;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; For small thin links, such=
 as 1-10Mbs DSL, the 5ms target would increase packet loss significantly an=
d at 2Mbps, a single MTU time may even exceed the 5ms target. </div>
<div>&#8226;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; If the average RTT of all =
flows going through a link is more than 500ms, e.g. for satellite, then the=
 100ms interval would prematurely drop packets before the sources have had =
a chance to reduce their sending rate. Or if the average RTT is very
low &#8211; e.g. 10ms &#8211; such as for flows between data-center element=
s, then 100ms interval may be slow to signal congestion back to the sources=
 and significant packet loss may have occurred before such signaling.</div>
<div>&nbsp;</div>
<div>One of the the objectives of newer AQMs being defined here should be t=
o minimize tuning, but we should recognize that likely tuning or some confi=
guration cannot be eliminated altogether.</div>
<div>&nbsp;</div>
<div><b>FB</b>: That's an opinion. One of the objectives of Van and Kathy's=
 work, and separately of Rong Pan et al's work, is to design an algorithm t=
hat may have different initial conditions drawn from a table given the inte=
rface it finds itself on, but requires
no manual tuning. The great failure of RED, recommended in RFC 2309, is not=
 that it doesn't work when properly configured; it's that real humans don't=
 have the time to properly tune it differently for each of the thousands of=
 link endpoints in their networks.
There is no point in changing away from RED if that is also true of the rep=
lacement.</div>
<div>&nbsp;</div>
<div><b>SA</b>: You argue that &quot;initial conditions&quot; determine som=
e of the parameters of newer AQMs (like Codel and PIE), then those same ini=
tial conditions would also determine some of the key parameters for RED/WRE=
D. </div>
<div>&nbsp;</div>
<div>Can you explain further why &quot;real humans don't have the time to p=
roperly tune it differently for each of the thousands of link endpoints in =
their networks&quot; with a realistic example.</div>
<div>&nbsp;</div>
<div><b>FB</b>: What you describe is what I referred to as &quot;initial co=
nditions derived from the links the algorithm finds itself on&quot;. We may=
 be in violent agreement there. If so, the wording I might suggest would be=
 &quot;SHOULD NOT require operational (especially
manual) configuration or tuning apart from automated determination of initi=
al conditions&quot; or some such thing.</div>
<div>&nbsp;</div>
<div><b>SA</b>: Looks like we may be agreeing on something here - which is =
great. It would be valuable to describe the meaning of &quot;initial condit=
ions&quot; somewhere.</div>
<div>&nbsp;</div>
<div><b>Section 4.7 - Further Research</b></div>
<div>&nbsp;</div>
<div><b>AQM impact on end-user QoE</b></div>
<div>Research should also focus on improving end user QoE from AQMs rather =
than network related metrics only. Often a significant change in a network =
metric may only make a minimal change in end-user QoE and thus the value of=
 such change may be minimal. </div>
<div>&nbsp;</div>
<div><b>FB</b>: There is also a question of what user is under discussion. =
If you take a look at <a href=3D"http://www.ietf.org/proceedings/88/slides/=
slides-88-aqm-0.pptx"><font color=3D"blue"><u>http://www.ietf.org/proceedin=
gs/88/slides/slides-88-aqm-0.pptx</u></font></a>
and specifically the third slide, you will see (and tomorrow afternoon I wi=
ll discuss) a capture I took of pings from my home network to another site =
overnight. In the evening, we watched a movie (home network), and in the mo=
rning I had a video conference (office
network). I'll tell you right now that both worked fine, and managing the d=
elay to zero or allowing it to be one hundred ms would not have materially =
affected the QOE of either. However, my ping is a competing application, an=
d it saw sustained increase in delay,
variation in delay, and the possibility of loss as a result of the queuing.=
 The value of AQM is in part to the application being throttled, but in lar=
ge part to competing applications, and the QOE of both must be considered.<=
/div>
<div>&nbsp;</div>
<div><b>SA</b>: My point about bringing up QoE is to tie the impact of a ne=
w investment (such as AQM algorithms) to an operator to revenue. By improvi=
ng QoE of revenue making services, an operator may determine that it makes =
business sense to invest. I do not
understand how &quot;ping&quot; - is linked a revenue making service? The Q=
oE of Web browsing which is a key service to end-users is closely tied to l=
atency as the PLT of a web page is directly linked to RTT. However the QoE =
of different services such as web browsing,
HAS Video, progessive download video and others should be weighed according=
 to their importance to the end user.</div>
<div>&nbsp;</div>
<div><b>Suggestion on best buffer sizes</b></div>
<div>Research should make suggestions on how to configure buffer sizes with=
 each type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer sizes i=
mprove end-user QoE and network health.</div>
<div>&nbsp;</div>
<div><b>FB</b>: I'm not sure that buffer sizes are specific to AQM algorith=
ms; I'd entertain evidence otherwise. Buffer *thresholds* (&quot;at what po=
int do we start dropping/marking traffic?&quot;) may differ between algorit=
hms. Buffer size (&quot;how many bytes/packets
do we allow into the queue in the worst case?&quot;) is a matter of the cha=
racteristics of burst behavior in a given network and the applications it s=
upports. If I have, say, a Map/Reduce application that simultaneously asks =
thousands of systems a question, the
queues in the intervening switches will need to be able to briefly absorb t=
housands of response packets. The key word here is &quot;briefly&quot;. Whe=
n Van or Kathy talk about &quot;good queue&quot; and &quot;bad queue&quot;,=
 they are saying that burst behavior may call for deep queues,
but we really want the steady state to achieve 100% utilization with a stat=
istically empty queue if we can possibly achieve that.</div>
<div>&nbsp;</div>
<div><b>SA</b>: In access situations (with AQM) buffer sizes can determine =
the size of a burst that the buffer can withstand and our research shows th=
at it can directly impact end user QoE. </div>
<div>&nbsp;</div>
<div><b>Best way to leverage deployed AQMs</b></div>
<div>Research should be done on methods or configurations that leverage dep=
loyed AQMs such as RED/WRED to reduce delays and lockout for typical traffi=
c which require minimal effort or tuning from the operator. </div>
<div>&nbsp;</div>
<div><b>FB</b>: Not a complete sentence, but I think I understand what you'=
re getting at. You would like to have research determine how to easily conf=
igure existing systems using the tools at hand. I'm all for it in the near =
term.</div>
<div>&nbsp;</div>
<div><b>SA</b>: Glad we agree on this - assume some update will be made aro=
und this topic.</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div>-----Original Message-----</div>
<div>From: Fred Baker (fred) [<a href=3D"mailto:fred@cisco.com"><font color=
=3D"blue"><u>mailto:fred@cisco.com</u></font></a>] </div>
<div>Sent: Thursday, November 07, 2013 12:45 PM</div>
<div>To: Akhtar, Shahid (Shahid)</div>
<div>Cc: Richard Scheffenegger; aqm@ietf.org; Naeem Khademi (naeemk@ifi.uio=
.no); Gorry Fairhurst; Wesley Eddy</div>
<div>Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div>On Nov 7, 2013, at 8:59 AM, &quot;Akhtar, Shahid (Shahid)&quot; &lt;sh=
ahid.akhtar@alcatel-lucent.com&gt; wrote:</div>
<div>&nbsp;</div>
<div>&gt; Hi All,</div>
<div>&gt; </div>
<div>&gt; Had some comments on Fred's document. I have added the comments a=
s track changes in a word document to easily see them. I used the 02 versio=
n. </div>
<div>&gt; </div>
<div>&gt; Thanks. </div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div>Permit me to put your comments in email, along with my own views. Also=
 adding my co-author and the other working group chair on the CC line; if h=
e is like me, he receives far too much email, and mail that is explicitly t=
o or copies him bubbles higher in
the column.</div>
<div>&nbsp;</div>
<div>&gt;&gt; 4.&nbsp; Conclusions and Recommendations</div>
<div>&gt;&gt;&nbsp;&nbsp; [snip] </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; 3.&nbsp; The algorithms that the IETF recom=
mends SHOULD NOT require</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; operational (especi=
ally manual) configuration or tuning.</div>
<div>&gt; </div>
<div>&gt; Some tuning may be required or implicitly assumed for virtually a=
ll AQMs &#8211; please see my comment later.</div>
<div>&nbsp;</div>
<div>FB: That's an opinion. One of the objectives of Van and Kathy's work, =
and separately of Rong Pan et al's work, is to design an algorithm that may=
 have different initial conditions drawn from a table given the interface i=
t finds itself on, but requires
no manual tuning. The great failure of RED, recommended in RFC 2309, is not=
 that it doesn't work when properly configured; it's that real humans don't=
 have the time to properly tune it differently for each of the thousands of=
 link endpoints in their networks.
There is no point in changing away from RED if that is also true of the rep=
lacement.</div>
<div>&nbsp;</div>
<div>SA: You argue that &quot;initial conditions&quot; determine some of th=
e parameters of newer AQMs (like Codel and PIE), then those same initial co=
nditions would also determine the key parameters for RED/WRED. </div>
<div>&nbsp;</div>
<div>Can you explain further why &quot;real humans don't have the time to p=
roperly tune it differently for each of the thousands of link endpoints in =
their networks&quot; with a realistic example.</div>
<div>&nbsp;</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; 7.&nbsp; Research, engineering, and measure=
ment efforts are needed</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; regarding the desig=
n of mechanisms to deal with flows that are</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; unresponsive to con=
gestion notification or are responsive, but</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; are more aggressive=
 than present TCP.</div>
<div>&gt;&nbsp; </div>
<div>&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Do we want to make a suggesti=
on on how to configure buffer sizes with each type of AQM here (e.g. 2xBDP =
etc) or simply state that research should be conducted on the best buffer s=
izes to use with AQM.</div>
<div>&nbsp;</div>
<div>FB: I'm not sure that buffer sizes are specific to AQM algorithms; I'd=
 entertain evidence otherwise. Buffer *thresholds* (&quot;at what point do =
we start dropping/marking traffic?&quot;) may differ between algorithms. Bu=
ffer size (&quot;how many bytes/packets do we allow
into the queue in the worst case?&quot;) is a matter of the characteristics=
 of burst behavior in a given network and the applications it supports. If =
I have, say, a Map/Reduce application that simultaneously asks thousands of=
 systems a question, the queues in the
intervening switches will need to be able to briefly absorb thousands of re=
sponse packets. The key word here is &quot;briefly&quot;. When Van or Kathy=
 talk about &quot;good queue&quot; and &quot;bad queue&quot;, they are sayi=
ng that burst behavior may call for deep queues, but we really
want the steady state to achieve 100% utilization with a statistically empt=
y queue if we can possibly achieve that.</div>
<div>&nbsp;</div>
<div>SA: In access situations (with AQM) buffer sizes can determine the siz=
e of a burst that the buffer can withstand and our research shows that it c=
an directly impact end user QoE. </div>
<div>&nbsp;</div>
<div>&gt;&gt; 4.3.&nbsp; AQM algorithms deployed SHOULD NOT require operati=
onal tuning</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; A number of algorithms have been proposed.&=
nbsp; Many require some form of</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; tuning or initial condition.&nbsp; This can=
 make them difficult to use</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; operationally.&nbsp; Hence, self-tuning alg=
orithms are to be preferred.</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; The algorithms that the IETF recommends SHO=
ULD NOT require</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; operational (especially manual) configurati=
on or tuning.</div>
<div>&gt;&nbsp; </div>
<div>&gt; May be better to state that tuning should be minimized. For the s=
econd sentence &#8220;The algorithms that the IETF recommends should minimi=
ze tuning or configurations changes for specific traffic or network conditi=
ons&#8221;</div>
<div>&gt;&nbsp; </div>
<div>&gt; I would argue that all AQMs will likely require or assume some ty=
pe of configuration/tuning.</div>
<div>&gt;&nbsp; </div>
<div>&gt; For example, if we take Codel:</div>
<div>&gt;&nbsp; </div>
<div>&gt; &middot;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; For small thin links=
, such as 1-10Mbs DSL, the 5ms target would increase packet loss significan=
tly and at 2Mbps, a single MTU time may even exceed the 5ms target.</div>
<div>&gt;&nbsp; </div>
<div>&gt; &middot;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; If the average RTT o=
f all flows going through a link is more than 500ms, e.g. for satellite, th=
en the 100ms interval would prematurely drop packets before the sources hav=
e had a chance to reduce their sending rate. Or if the average RTT is very
low &#8211; e.g. 10ms &#8211; such as for flows between data-center element=
s, then 100ms interval may be slow to signal congestion back to the sources=
 and significant packet loss may have occurred before such signaling.</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div>FB: What you describe is what I referred to as &quot;initial condition=
s derived from the links the algorithm finds itself on&quot;. We may be in =
violent agreement there. If so, the wording I might suggest would be &quot;=
SHOULD NOT require operational (especially manual)
configuration or tuning apart from automated determination of initial condi=
tions&quot; or some such thing.</div>
<div>&nbsp;</div>
<div>SA: Looks like we may be agreeing on something here - which is great. =
It would be valuable to describe the meaning of &quot;initial conditions&qu=
ot; somewhere.</div>
<div>&nbsp;</div>
<div>&gt;&gt; 4.7.&nbsp; The need for further research</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; The second recommendation of [RFC2309] call=
ed for further research in</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; the interaction between network queues and =
host applications, and the</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; means of signaling between them.&nbsp; This=
 research has occurred, and we</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; as a community have learned a lot.&nbsp; Ho=
wever, we are not done.</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; We have learned that the problems of conges=
tion, latency and buffer-</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; sizing have not gone away, and are becoming=
 more important to many</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; users.&nbsp; A number of self-tuning AQM al=
gorithms have be found that</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; offer significant advantages for deployed n=
etworks.&nbsp; There is also</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; renewed interest in deploying AQM and the p=
otential of ECN.</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; An obvious example of further research in 2=
013 is the need to</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; consider the use of Map/Reduce applications=
 in data centers; do we</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; need to extend our taxonomy of TCP/SCTP ses=
sions to include not only</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; &quot;mice&quot; and &quot;elephants&quot;,=
 but &quot;lemmings&quot;?&nbsp; &quot;Lemmings&quot; are flash crowds</div=
>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; of &quot;mice&quot; that the network inadve=
rtently tries to signal to as if</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; they were elephant flows, resulting in head=
 of line blocking in data</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; center applications.</div>
<div>&gt;&nbsp; </div>
<div>&gt; Such research should also focus on improving end user QoE from AQ=
Ms rather than network related metrics only. Often a significant change in =
a network metric may only make a minimal change in end-user QoE and thus th=
e value of such change may be minimal.</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div>FB: There is also a question of what user is under discussion. If you =
take a look at <a href=3D"http://www.ietf.org/proceedings/88/slides/slides-=
88-aqm-0.pptx"><font color=3D"blue"><u>http://www.ietf.org/proceedings/88/s=
lides/slides-88-aqm-0.pptx</u></font></a>
and specifically the third slide, you will see (and tomorrow afternoon I wi=
ll discuss) a capture I took of pings from my home network to another site =
overnight. In the evening, we watched a movie (home network), and in the mo=
rning I had a video conference (office
network). I'll tell you right now that both worked fine, and managing the d=
elay to zero or allowing it to be one hundred ms would not have materially =
affected the QOE of either. However, my ping is a competing application, an=
d it saw sustained increase in delay,
variation in delay, and the possibility of loss as a result of the queuing.=
 The value of AQM is in part to the application being throttled, but in lar=
ge part to competing applications, and the QOE of both must be considered.<=
/div>
<div>&nbsp;</div>
<div>SA: My point about bringing up QoE is to tie the impact of a new inves=
tment (such as AQM algorithms) to an operator to revenue. By improving QoE =
of revenue making services, an operator may determine that it makes busines=
s sense to invest. I do not understand
how &quot;ping&quot; - is linked a revenue making service? The QoE of Web b=
rowsing which is a key service to end-users is closely tied to latency as t=
he PLT of a web page is directly linked to RTT. However the QoE of differen=
t services such as web browsing, HAS Video,
progessive download video and others should be weighed according to their i=
mportance to the end user.</div>
<div>&nbsp;</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; Examples of other required research include=
:</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; o&nbsp; Research into new AQM and schedulin=
g algorithms.</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; o&nbsp; Research into the use of and deploy=
ment of ECN alongside AQM.</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; o&nbsp; Tools for enabling AQM (and ECN) de=
ployment and measuring the</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; performance.</div>
<div>&gt;&gt;&nbsp; </div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp; o&nbsp; Methods for mitigating the impact o=
f non-conformant and malicious</div>
<div>&gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; flows.</div>
<div>&gt;&nbsp; </div>
<div>&gt; Methods or configurations that leverage deployed AQMs such as RED=
/WRED to reduce delays and lockout for typical traffic which require minima=
l effort or tuning from the operator.</div>
<div>&nbsp;</div>
<div>FB: Not a complete sentence, but I think I understand what you're gett=
ing at. You would like to have research determine how to easily configure e=
xisting systems using the tools at hand. I'm all for it in the near term.</=
div>
<div>&nbsp;</div>
<div>SA: Glad we agree on this - assume some update will be made around thi=
s topic.</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_C0611F522A6FA9498A044C36073E49657E5FFA98US70UWXCHMBA01z_--

From fred@cisco.com  Fri Nov  8 08:27:23 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2654311E8184 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 08:27:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.463
X-Spam-Level: 
X-Spam-Status: No, score=-110.463 tagged_above=-999 required=5 tests=[AWL=0.135, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ww+3adZO1DJ4 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 08:27:17 -0800 (PST)
Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) by ietfa.amsl.com (Postfix) with ESMTP id 17B7711E8189 for <aqm@ietf.org>; Fri,  8 Nov 2013 08:27:15 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6358; q=dns/txt; s=iport; t=1383928035; x=1385137635; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=WAsufx9J/Qgv24hvbzbmLQgY56oSKR1p/IHk58OCvUU=; b=SmntuCFfvG1b/i/KcKCpcy9Ee6VktG+62c/vXt4MyLAsIpO0znvjzMRU QdVwdqV8t31w1Fh87KkT4MAtAU0zv1b133NhOxpaqp4QvWoEFxNII/8HC s62Ja2kEmKw4W1JBLyH5IM7OUN9+zIYEMWlaRdBd748aMan4bLfvyPxYL c=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhYFAO4PfVKtJV2c/2dsb2JhbABagkNEgQu/FIEwFnSCJQEBAQMBeQULAgEIRjIlAgQOBQ6HbQa9CI4dgUoHgyCBEAOQMIEwhi+SC4Mmgio
X-IronPort-AV: E=Sophos;i="4.93,661,1378857600";  d="asc'?scan'208,217";a="282576028"
Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-4.cisco.com with ESMTP; 08 Nov 2013 16:27:14 +0000
Received: from xhc-rcd-x09.cisco.com (xhc-rcd-x09.cisco.com [173.37.183.83]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id rA8GREQW005623 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Nov 2013 16:27:14 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-rcd-x09.cisco.com ([173.37.183.83]) with mapi id 14.03.0123.003; Fri, 8 Nov 2013 10:27:13 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: AQHO3J9hYVICOG/TQKCzUJnt0vYtGg==
Date: Fri, 8 Nov 2013 16:27:13 +0000
Message-ID: <6A0A7988-7CEA-40E2-924A-80BCC141F1F1@cisco.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com> <C0611F522A6FA9498A044C36073E49657E5FFA98@US70UWXCHMBA01.zam.alcatel-lucent.com>
In-Reply-To: <C0611F522A6FA9498A044C36073E49657E5FFA98@US70UWXCHMBA01.zam.alcatel-lucent.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.21.148.94]
Content-Type: multipart/signed; boundary="Apple-Mail=_A1EDDF35-3469-4045-B5DB-DB1D81E71AD0"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 16:27:23 -0000

--Apple-Mail=_A1EDDF35-3469-4045-B5DB-DB1D81E71AD0
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_27176DB6-9B0F-462B-B449-BBC484CF30F4"


--Apple-Mail=_27176DB6-9B0F-462B-B449-BBC484CF30F4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Nov 8, 2013, at 5:56 AM, "Akhtar, Shahid (Shahid)" =
<shahid.akhtar@alcatel-lucent.com> wrote:

> One of the the objectives of newer AQMs being defined here should be =
to minimize tuning, but we should recognize that likely tuning or some =
configuration cannot be eliminated altogether.
> =20
> FB: That's an opinion. One of the objectives of Van and Kathy's work, =
and separately of Rong Pan et al's work, is to design an algorithm that =
may have different initial conditions drawn from a table given the =
interface it finds itself on, but requires no manual tuning. The great =
failure of RED, recommended in RFC 2309, is not that it doesn't work =
when properly configured; it's that real humans don't have the time to =
properly tune it differently for each of the thousands of link endpoints =
in their networks. There is no point in changing away from RED if that =
is also true of the replacement.
> =20
> SA: You argue that "initial conditions" determine some of the =
parameters of newer AQMs (like Codel and PIE), then those same initial =
conditions would also determine some of the key parameters for RED/WRED.

I'm simply going to point out that Van and Kathy spent quite a bit of =
time and effort trying to do exactly that, and it didn't pan out. Codel =
is their suggestion of a replacement that is largely auto-tuning within =
a specified range of situations.

On your other points, please suggest text, and the WG can discuss =
whether they buy it.

--Apple-Mail=_27176DB6-9B0F-462B-B449-BBC484CF30F4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On Nov 8, 2013, at 5:56 AM, "Akhtar, Shahid (Shahid)" =
&lt;<a =
href=3D"mailto:shahid.akhtar@alcatel-lucent.com">shahid.akhtar@alcatel-luc=
ent.com</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div =
style=3D"font-family: Arial; font-size: 13px; font-style: normal; =
font-variant: normal; font-weight: normal; letter-spacing: normal; =
line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: =
0px; text-transform: none; white-space: normal; widows: 2; word-spacing: =
0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
">One of the the objectives of newer AQMs being defined here should be =
to minimize tuning, but we should recognize that likely tuning or some =
configuration cannot be eliminated altogether.</div><div =
style=3D"font-family: Arial; font-size: 13px; font-style: normal; =
font-variant: normal; font-weight: normal; letter-spacing: normal; =
line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: =
0px; text-transform: none; white-space: normal; widows: 2; word-spacing: =
0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
">&nbsp;</div><div style=3D"font-family: Arial; font-size: 13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: 2; text-align: =
-webkit-auto; text-indent: 0px; text-transform: none; white-space: =
normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; =
-webkit-text-stroke-width: 0px; "><b>FB</b>: That's an opinion. One of =
the objectives of Van and Kathy's work, and separately of Rong Pan et =
al's work, is to design an algorithm that may have different initial =
conditions drawn from a table given the interface it finds itself on, =
but requires no manual tuning. The great failure of RED, recommended in =
RFC 2309, is not that it doesn't work when properly configured; it's =
that real humans don't have the time to properly tune it differently for =
each of the thousands of link endpoints in their networks. There is no =
point in changing away from RED if that is also true of the =
replacement.</div><div style=3D"font-family: Arial; font-size: 13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: 2; text-align: =
-webkit-auto; text-indent: 0px; text-transform: none; white-space: =
normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; =
-webkit-text-stroke-width: 0px; ">&nbsp;</div><div style=3D"font-family: =
Arial; font-size: 13px; font-style: normal; font-variant: normal; =
font-weight: normal; letter-spacing: normal; line-height: normal; =
orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: =
none; white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
"><b>SA</b>: You argue that "initial conditions" determine some of the =
parameters of newer AQMs (like Codel and PIE), then those same initial =
conditions would also determine some of the key parameters for =
RED/WRED.</div></blockquote></div><br>
I'm simply going to point out that Van and Kathy spent quite a bit of =
time and effort trying to do exactly that, and it didn't pan out. Codel =
is their suggestion of a replacement that is largely auto-tuning within =
a specified range of situations.<div><br></div><div>On your other =
points, please suggest text, and the WG can discuss whether they buy =
it.</div></body></html>=

--Apple-Mail=_27176DB6-9B0F-462B-B449-BBC484CF30F4--

--Apple-Mail=_A1EDDF35-3469-4045-B5DB-DB1D81E71AD0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSfRDfbjEdbHIsm0MRAhmXAKDX5ylDRlWd6ppxgRQQZd+yOXRrwgCg4RBL
58HLt1tOJZVwGp26A6geN/A=
=5sBY
-----END PGP SIGNATURE-----

--Apple-Mail=_A1EDDF35-3469-4045-B5DB-DB1D81E71AD0--

From shahid.akhtar@alcatel-lucent.com  Fri Nov  8 11:06:14 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0375921F9D46 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 11:06:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.448
X-Spam-Level: 
X-Spam-Status: No, score=-10.448 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gm68ER-u4PaE for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 11:06:02 -0800 (PST)
Received: from ihemail4.lucent.com (ihemail4.lucent.com [135.245.0.39]) by ietfa.amsl.com (Postfix) with ESMTP id B397C21E811B for <aqm@ietf.org>; Fri,  8 Nov 2013 11:04:28 -0800 (PST)
Received: from us70uusmtp4.zam.alcatel-lucent.com (h135-5-2-66.lucent.com [135.5.2.66]) by ihemail4.lucent.com (8.13.8/IER-o) with ESMTP id rA8Ixi80001697 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 8 Nov 2013 13:04:09 -0600 (CST)
Received: from US70TWXCHHUB04.zam.alcatel-lucent.com (us70twxchhub04.zam.alcatel-lucent.com [135.5.2.36]) by us70uusmtp4.zam.alcatel-lucent.com (GMO) with ESMTP id rA8Ic7jG020176 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Nov 2013 13:38:07 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70TWXCHHUB04.zam.alcatel-lucent.com ([135.5.2.36]) with mapi id 14.02.0247.003; Fri, 8 Nov 2013 13:38:07 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Thread-Topic: IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: AQHO3J9hYVICOG/TQKCzUJnt0vYtGpobobhg
Date: Fri, 8 Nov 2013 18:38:07 +0000
Message-ID: <C0611F522A6FA9498A044C36073E49657E5FFFB8@US70UWXCHMBA01.zam.alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com> <C0611F522A6FA9498A044C36073E49657E5FFA98@US70UWXCHMBA01.zam.alcatel-lucent.com> <6A0A7988-7CEA-40E2-924A-80BCC141F1F1@cisco.com>
In-Reply-To: <6A0A7988-7CEA-40E2-924A-80BCC141F1F1@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.5.27.16]
Content-Type: multipart/alternative; boundary="_000_C0611F522A6FA9498A044C36073E49657E5FFFB8US70UWXCHMBA01z_"
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.39
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 19:06:14 -0000

--_000_C0611F522A6FA9498A044C36073E49657E5FFFB8US70UWXCHMBA01z_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Fred, all,

Below is the recommneded text as requested.

-Shahid.

______________________________________________

Manual Tuning

Current text

section 4:

"The algorithms that the IETF recommends SHOULD NOT require operational (es=
pecially manual) configuration or tuning."

section 4.3:

"Algorithms recommended for general Internet deployment by the IETF need to=
 be designed so that they do not require operational (especially manual)con=
figuration or tuning."

Recommended text



section 4.3:

"Algorithms recommended for general Internet deployment by the IETF need to=
 be designed so that they minimize operational (especially manual) configur=
ation or tuning. An AQM considered by IETF SHOULD provide rules which deter=
mine its parameters based on a set of well-known initial conditions."



section 4:

"The algorithms that the IETF recommends SHOULD minimize operational (espec=
ially manual) configuration or tuning."



Recommend the following text in section 4.7

Research should focus on improving end user QoE from AQMs rather than netwo=
rk related metrics only. Often a significant change in a network metric may=
 only make a minimal change in end-user QoE and thus the value of such chan=
ge may be minimal.

Research should make suggestions on how to make good decisions on buffer si=
zes with each type of AQM (e.g. 2xBDP) - explaining why or how such buffer =
sizes improve end-user QoE and network health.

Research should be done on methods or good configurations that leverage dep=
loyed AQMs such as RED/WRED that reduce delays and prevent lockout with typ=
ical traffic and network conditions.



________________________________

From: Fred Baker (fred) [mailto:fred@cisco.com]
Sent: Friday, November 08, 2013 10:27 AM
To: Akhtar, Shahid (Shahid)
Cc: Richard Scheffenegger; aqm@ietf.org; Naeem Khademi (naeemk@ifi.uio.no);=
 Gorry Fairhurst; Wesley Eddy
Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B


On Nov 8, 2013, at 5:56 AM, "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcate=
l-lucent.com<mailto:shahid.akhtar@alcatel-lucent.com>> wrote:

One of the the objectives of newer AQMs being defined here should be to min=
imize tuning, but we should recognize that likely tuning or some configurat=
ion cannot be eliminated altogether.

FB: That's an opinion. One of the objectives of Van and Kathy's work, and s=
eparately of Rong Pan et al's work, is to design an algorithm that may have=
 different initial conditions drawn from a table given the interface it fin=
ds itself on, but requires no manual tuning. The great failure of RED, reco=
mmended in RFC 2309, is not that it doesn't work when properly configured; =
it's that real humans don't have the time to properly tune it differently f=
or each of the thousands of link endpoints in their networks. There is no p=
oint in changing away from RED if that is also true of the replacement.

SA: You argue that "initial conditions" determine some of the parameters of=
 newer AQMs (like Codel and PIE), then those same initial conditions would =
also determine some of the key parameters for RED/WRED.

I'm simply going to point out that Van and Kathy spent quite a bit of time =
and effort trying to do exactly that, and it didn't pan out. Codel is their=
 suggestion of a replacement that is largely auto-tuning within a specified=
 range of situations.

On your other points, please suggest text, and the WG can discuss whether t=
hey buy it.

--_000_C0611F522A6FA9498A044C36073E49657E5FFFB8US70UWXCHMBA01z_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta content=3D"MSHTML 6.00.2900.6452" name=3D"GENERATOR">
</head>
<body style=3D"WORD-WRAP: break-word; webkit-nbsp-mode: space; webkit-line-=
break: after-white-space">
<div dir=3D"ltr" align=3D"left">
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013">Fred, all,</span></pre>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013">Below is the recommneded text as requested. </span></pre>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013">-Shahid.</span></pre>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013">______________________________________________</span></pr=
e>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013"><strong>Manual Tuning</strong></span></pre>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013">Current text</span></pre>
<span class=3D"535100818-08112013">
<pre><span class=3D"535100818-08112013"><font size=3D"2">section 4:</font><=
/span></pre>
<pre><span class=3D"535100818-08112013"></span><span class=3D"535100818-081=
12013">&quot;The algorithms that the IETF recommends SHOULD NOT require ope=
rational (especially manual) configuration or tuning.&quot;</pre>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"></span><p><span c=
lass=3D"535100818-08112013"></span><span class=3D"535100818-08112013">secti=
on 4.3:</span></p>&quot;</span>Algorithms<span class=3D"535100818-08112013"=
> </span>recommended for general Internet deployment by the IETF need to be=
<span class=3D"535100818-08112013"> </span>designed so that they do not req=
uire operational (especially manual)configuration or tuning.<span class=3D"=
535100818-08112013">&quot;</span></pre>
<pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none;=
 COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: normal; FONT-STYLE: norm=
al; LETTER-SPACING: normal; FONT-VARIANT: normal; WORD-WRAP: break-word; or=
phans: auto; widows: auto; webkit-text-stroke-width: 0px"><span class=3D"53=
5100818-08112013"><span lang=3D"EN"><p><span class=3D"535100818-08112013">R=
ecommended text</span></p><p><span class=3D"535100818-08112013"></span>&nbs=
p;</p><p><span class=3D"535100818-08112013"></span><span class=3D"535100818=
-08112013">section 4.3:</span></p><p><span class=3D"535100818-08112013"></s=
pan><span class=3D"535100818-08112013">&quot;</span>Algorithms<span class=
=3D"535100818-08112013"> </span>recommended for general Internet deployment=
 by the IETF need to be<span class=3D"535100818-08112013"> </span>designed =
so that they <span class=3D"535100818-08112013">minimize </span>operational=
 (especially manual) configuration or tuning<span class=3D"535100818-081120=
13">. A</span><span class=3D"535100818-08112013">n</span><span class=3D"535=
100818-08112013"> AQM considered by IETF SHOULD provide rules which determi=
ne its parameters based on a set of well-known initial conditions.&quot;</s=
pan></p><p><span class=3D"535100818-08112013"></span>&nbsp;</p><p><span cla=
ss=3D"535100818-08112013"><font size=3D"2">section 4: </font></span><span c=
lass=3D"535100818-08112013"><font size=3D"2"></font></span></p><span class=
=3D"535100818-08112013"><pre style=3D"FONT-WEIGHT: normal; WORD-SPACING: 0p=
x; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; LINE-HEIGHT: =
normal; FONT-STYLE: normal; LETTER-SPACING: normal; FONT-VARIANT: normal; W=
ORD-WRAP: break-word; orphans: auto; widows: auto; webkit-text-stroke-width=
: 0px">&quot;The algorithms that the IETF recommends SHOULD minimize operat=
ional (especially manual) configuration or tuning.&quot;</pre></span><p><sp=
an class=3D"535100818-08112013"><b></b>&nbsp;</p><div><span class=3D"535100=
818-08112013">Recommend the following text in section 4.7</span></div><div>=
<span class=3D"535100818-08112013"><strong></strong></span>&nbsp;</div><div=
>Research should focus on improving end user QoE from AQMs rather than netw=
ork related metrics only. Often a significant change in a network metric ma=
y only make a minimal change in end-user QoE and thus the value of such cha=
nge may be minimal. </div><div>&nbsp;</div><div>Research should make sugges=
tions on how to <span class=3D"535100818-08112013">make good decisions on <=
/span>buffer sizes with each type of AQM (e.g. 2xBDP) - explaining why<span=
 class=3D"535100818-08112013"> or </span>how such buffer sizes improve end-=
user QoE and network health.</div><div>&nbsp;</div><div>Research should be =
done on methods or <span class=3D"535100818-08112013">good </span>configura=
tions that leverage deployed AQMs such as RED/WRED <span class=3D"535100818=
-08112013">that </span>reduce delays and <span class=3D"535100818-08112013"=
>prevent </span>lockout <span class=3D"535100818-08112013">with </span>typi=
cal traffic <span class=3D"535100818-08112013">and network conditions</span=
>. </div></span><p><span class=3D"535100818-08112013"></span>&nbsp;</p></sp=
an></span><p></p><hr tabindex=3D"-1"><p></p><p><font face=3D"Tahoma" size=
=3D"2"><b>From:</b> Fred Baker (fred) [mailto:fred@cisco.com] <br><b>Sent:<=
/b> Friday, November 08, 2013 10:27 AM<br><b>To:</b> Akhtar, Shahid (Shahid=
)<br><b>Cc:</b> Richard Scheffenegger; aqm@ietf.org; Naeem Khademi (naeemk@=
ifi.uio.no); Gorry Fairhurst; Wesley Eddy<br><b>Subject:</b> Re: IETF88 Fri=
 08Nov13 - 12:30 Regency B<br></font><br></p></pre>
</div>
<div></div>
<br>
<div>
<div>On Nov 8, 2013, at 5:56 AM, &quot;Akhtar, Shahid (Shahid)&quot; &lt;<a=
 href=3D"mailto:shahid.akhtar@alcatel-lucent.com">shahid.akhtar@alcatel-luc=
ent.com</a>&gt; wrote:</div>
<br class=3D"Apple-interchange-newline">
<blockquote type=3D"cite">
<div style=3D"WORD-SPACING: 0px; FONT: 13px Arial; TEXT-TRANSFORM: none; TE=
XT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; orphans: 2; wi=
dows: 2; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px">
One of the the objectives of newer AQMs being defined here should be to min=
imize tuning, but we should recognize that likely tuning or some configurat=
ion cannot be eliminated altogether.</div>
<div style=3D"WORD-SPACING: 0px; FONT: 13px Arial; TEXT-TRANSFORM: none; TE=
XT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; orphans: 2; wi=
dows: 2; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px">
&nbsp;</div>
<div style=3D"WORD-SPACING: 0px; FONT: 13px Arial; TEXT-TRANSFORM: none; TE=
XT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; orphans: 2; wi=
dows: 2; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px">
<b>FB</b>: That's an opinion. One of the objectives of Van and Kathy's work=
, and separately of Rong Pan et al's work, is to design an algorithm that m=
ay have different initial conditions drawn from a table given the interface=
 it finds itself on, but requires
 no manual tuning. The great failure of RED, recommended in RFC 2309, is no=
t that it doesn't work when properly configured; it's that real humans don'=
t have the time to properly tune it differently for each of the thousands o=
f link endpoints in their networks.
 There is no point in changing away from RED if that is also true of the re=
placement.</div>
<div style=3D"WORD-SPACING: 0px; FONT: 13px Arial; TEXT-TRANSFORM: none; TE=
XT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; orphans: 2; wi=
dows: 2; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px">
&nbsp;</div>
<div style=3D"WORD-SPACING: 0px; FONT: 13px Arial; TEXT-TRANSFORM: none; TE=
XT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; orphans: 2; wi=
dows: 2; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px">
<b>SA</b>: You argue that &quot;initial conditions&quot; determine some of =
the parameters of newer AQMs (like Codel and PIE), then those same initial =
conditions would also determine some of the key parameters for RED/WRED.</d=
iv>
</blockquote>
</div>
<br>
I'm simply going to point out that Van and Kathy spent quite a bit of time =
and effort trying to do exactly that, and it didn't pan out. Codel is their=
 suggestion of a replacement that is largely auto-tuning within a specified=
 range of situations.
<div><br>
</div>
<div>On your other points, please suggest text, and the WG can discuss whet=
her they buy it.</div>
</body>
</html>

--_000_C0611F522A6FA9498A044C36073E49657E5FFFB8US70UWXCHMBA01z_--

From David.Ros@telecom-bretagne.eu  Fri Nov  8 11:23:57 2013
Return-Path: <David.Ros@telecom-bretagne.eu>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF1C011E80FB for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 11:23:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.079
X-Spam-Level: 
X-Spam-Status: No, score=-2.079 tagged_above=-999 required=5 tests=[AWL=-0.057, BAYES_00=-2.599, HELO_EQ_FR=0.35, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UhSOaosSOARQ for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 11:23:53 -0800 (PST)
Received: from zproxy220.enst-bretagne.fr (zproxy220.enst-bretagne.fr [192.108.117.9]) by ietfa.amsl.com (Postfix) with ESMTP id 4693C11E8112 for <aqm@ietf.org>; Fri,  8 Nov 2013 11:23:53 -0800 (PST)
Received: from localhost (localhost [127.0.0.1]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTP id 62A5450013; Fri,  8 Nov 2013 20:23:52 +0100 (CET)
X-Virus-Scanned: amavisd-new at zproxy220.enst-bretagne.fr
Received: from zproxy220.enst-bretagne.fr ([127.0.0.1]) by localhost (zproxy220.enst-bretagne.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gvDG4xnLjEu3; Fri,  8 Nov 2013 20:23:51 +0100 (CET)
Received: from dhcp-9918.meeting.ietf.org (dhcp-9918.meeting.ietf.org [31.133.153.24]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTPSA id E163650012; Fri,  8 Nov 2013 20:23:48 +0100 (CET)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1816\))
From: David Ros <David.Ros@telecom-bretagne.eu>
In-Reply-To: <C0611F522A6FA9498A044C36073E49657E5FFFB8@US70UWXCHMBA01.zam.alcatel-lucent.com>
Date: Fri, 8 Nov 2013 11:23:45 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <B7249D76-0F19-4456-A01D-8C075E9946E9@telecom-bretagne.eu>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com> <C0611F522A6FA9498A044C36073E49657E5FFA98@US70UWXCHMBA01.zam.alcatel-lucent.com> <6A0A7988-7CEA-40E2-924A-80BCC141F1F1@cisco.com> <C0611F522A6FA9498A044C36073E49657E5FFFB8@US70UWXCHMBA01.zam.alcatel-lucent.com>
To: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
X-Mailer: Apple Mail (2.1816)
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "Fred Baker \(fred\)" <fred@cisco.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 19:23:58 -0000

On 8 nov. 2013, at 10:38, Akhtar, Shahid (Shahid) =
<shahid.akhtar@alcatel-lucent.com> wrote:

[snip]
> Recommend the following text in section 4.7
> =20
> Research should focus on improving end user QoE from AQMs rather than =
network related metrics only. Often a significant change in a network =
metric may only make a minimal change in end-user QoE and thus the value =
of such change may be minimal.=20

Can this be said to be true for the QoE of *any* application?

And, if we focus on QoE instead of network metrics, (a) what apps are we =
going to consider? (b) do we have easy-to-use, readily-available and =
accurate models for assessing QoE for all those apps?

(Note, I'm not trying to contradict what you say, I just want to better =
understand what concrete evaluation guidelines could be given on this)

Thanks,

David

> =20
> Research should make suggestions on how to make good decisions on =
buffer sizes with each type of AQM (e.g. 2xBDP) - explaining why or how =
such buffer sizes improve end-user QoE and network health.
> =20
> Research should be done on methods or good configurations that =
leverage deployed AQMs such as RED/WRED that reduce delays and prevent =
lockout with typical traffic and network conditions.=20
> =20
>=20
>=20
> From: Fred Baker (fred) [mailto:fred@cisco.com]=20
> Sent: Friday, November 08, 2013 10:27 AM
> To: Akhtar, Shahid (Shahid)
> Cc: Richard Scheffenegger; aqm@ietf.org; Naeem Khademi =
(naeemk@ifi.uio.no); Gorry Fairhurst; Wesley Eddy
> Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B
>=20
>=20
>=20
> On Nov 8, 2013, at 5:56 AM, "Akhtar, Shahid (Shahid)" =
<shahid.akhtar@alcatel-lucent.com> wrote:
>=20
>> One of the the objectives of newer AQMs being defined here should be =
to minimize tuning, but we should recognize that likely tuning or some =
configuration cannot be eliminated altogether.
>> =20
>> FB: That's an opinion. One of the objectives of Van and Kathy's work, =
and separately of Rong Pan et al's work, is to design an algorithm that =
may have different initial conditions drawn from a table given the =
interface it finds itself on, but requires no manual tuning. The great =
failure of RED, recommended in RFC 2309, is not that it doesn't work =
when properly configured; it's that real humans don't have the time to =
properly tune it differently for each of the thousands of link endpoints =
in their networks. There is no point in changing away from RED if that =
is also true of the replacement.
>> =20
>> SA: You argue that "initial conditions" determine some of the =
parameters of newer AQMs (like Codel and PIE), then those same initial =
conditions would also determine some of the key parameters for RED/WRED.
>=20
> I'm simply going to point out that Van and Kathy spent quite a bit of =
time and effort trying to do exactly that, and it didn't pan out. Codel =
is their suggestion of a replacement that is largely auto-tuning within =
a specified range of situations.
>=20
> On your other points, please suggest text, and the WG can discuss =
whether they buy it.
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
David ROS
http://www.rennes.enst-bretagne.fr/~dros/

Deadlines really start to press a week or two after they pass. -- John =
Perry


From fred@cisco.com  Fri Nov  8 11:49:56 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AD56A21E81FF; Fri,  8 Nov 2013 11:49:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.3
X-Spam-Level: 
X-Spam-Status: No, score=-110.3 tagged_above=-999 required=5 tests=[AWL=0.072,  BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id szmBjGR9CZBL; Fri,  8 Nov 2013 11:49:43 -0800 (PST)
Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) by ietfa.amsl.com (Postfix) with ESMTP id 3DEF721E8195; Fri,  8 Nov 2013 11:49:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2062; q=dns/txt; s=iport; t=1383940153; x=1385149753; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=KN9MDTSa2wA4qNUCkB6WLItKeImvaw2siNQUSG5qO8U=; b=OEpRXHjy5999X5hYYOcRBQcpx8AX6jsznLOMqQVfoEkKZifn8/B6/W6v 652w20ChC1xv/Ul+GOlfEkqkQXe1JlfCV/AJVM6T9Yp8aTZHxZ2lJZpac 7jOhoKZP9Df4jvAz/t6EMKyTd78ImOBVEdUXMfImmGc8Uou272g+al3s+ k=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AjQFAAk/fVKtJXHA/2dsb2JhbABZgweBC7wdgWiBEYExFnSCJQEBAQR5EAIBCBguMiUCBA4TDYdmvUWPZweDIIEQA5AwgTCGL5ILgyaCKg
X-IronPort-AV: E=Sophos;i="4.93,661,1378857600";  d="asc'?scan'208";a="282550953"
Received: from rcdn-core2-5.cisco.com ([173.37.113.192]) by rcdn-iport-2.cisco.com with ESMTP; 08 Nov 2013 19:49:06 +0000
Received: from xhc-aln-x06.cisco.com (xhc-aln-x06.cisco.com [173.36.12.80]) by rcdn-core2-5.cisco.com (8.14.5/8.14.5) with ESMTP id rA8Jn6Jq027658 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Nov 2013 19:49:06 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x06.cisco.com ([173.36.12.80]) with mapi id 14.03.0123.003; Fri, 8 Nov 2013 13:49:05 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Andrew McGregor <andrewmcgr@gmail.com>
Thread-Topic: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel	protocols?
Thread-Index: AQHO3LuUWaKyBh1riEa3sG4KFle+eg==
Date: Fri, 8 Nov 2013 19:49:05 +0000
Message-ID: <145AB1C0-B108-4196-AD5C-3667103FB4D8@cisco.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
In-Reply-To: <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.21.148.94]
Content-Type: multipart/signed; boundary="Apple-Mail=_B2184D50-10C6-4503-A028-FC8AAE7A6A38"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: AQM IETF list <aqm@ietf.org>, "<draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel	protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 19:49:56 -0000

--Apple-Mail=_B2184D50-10C6-4503-A028-FC8AAE7A6A38
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


At 23:04 04/11/2013, Andrew Mcgregor wrote:
> This seems like valuable work.  One question is, can we put in scope =
notification that losses are NOT due to congestion?

Speaking for myself, I'm not sure how I would do that.

A loss (or mark) due to congestion is pretty simple. The switch knows =
what it did.=20

To know that a packet was lost for reasons other than congestion, I need =
to somehow know what packets I should expect, and infer that something =
that I expected didn't happen. In TCP, we know about data segments =
because they are enumerated - I know what the next octet sequence number =
to expect, and it doesn't arrive. Control segments (SYN, ACK, FIN, RST, =
and so on) are not enumerated in that sense - if my peer sends ten =
identical acks and nine arrive, I as the receiver have no way to know =
that. At the link layer, most link protocols in use today (PPP, =
Ethernet, and so on) do not enumerate packets in flight - they are =
simply there. I *might* be able to see a burst of noise on the line, but =
only if it looks like it might be the start of a packet and then doesn't =
end with the right checksum. Even if I can see it, I have no way to know =
whether the noise garbled one packet or many.

If you want to do some research and come up with a solution, be my =
guest. But in a standard discussed in 2013... let's not.

--Apple-Mail=_B2184D50-10C6-4503-A028-FC8AAE7A6A38
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSfUAxbjEdbHIsm0MRApMhAJwOogQqdCvBmF4cfltYOtnMSOoIxQCg6YeQ
kgevVsY3Rao+Xmqzx1iDL8M=
=mNKB
-----END PGP SIGNATURE-----

--Apple-Mail=_B2184D50-10C6-4503-A028-FC8AAE7A6A38--

From preethi.cis@gmail.com  Fri Nov  8 12:00:54 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A01C021E81F3 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:00:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kTrDTI0nLbcb for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:00:31 -0800 (PST)
Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) by ietfa.amsl.com (Postfix) with ESMTP id BFB3721E8179 for <aqm@ietf.org>; Fri,  8 Nov 2013 12:00:31 -0800 (PST)
Received: by mail-pd0-f170.google.com with SMTP id v10so2601153pde.29 for <aqm@ietf.org>; Fri, 08 Nov 2013 12:00:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=jpI6ittpC462oD0RVZ29EbAiK9l2v0ti34zbDqbROQA=; b=hpSWvtcFccJ8zdZVh3o/kisPPuxxX1XrjQf/YVLhsuLI6o+xeMVeJXsP3gNynRK1UQ YUc28JRIyG7xetciyILE/cQn2Edy5wf5xThzXTv2xVCFXeMZYCBwVMQL9rNVl81W0WR6 Ammt1Lr+jXEk9qDQB6dILXt6fTQRUbnV3pZmaMM4bYAEJqfGUKqcCwFJI2hMNP/EFUkG VnwOlGcgVgGMUJZMukWFEpa/tmPothFgdxnYvqlELtF6h6bG+S+Eh2gOya0FdlaHC5EW yTUEDqWLMTS7ySzQ4DJtCxuuFF5uQopMawkJILUxHB3tzkUQKSTR2KcU/fOTtUPzQJG5 m0KQ==
X-Received: by 10.66.121.68 with SMTP id li4mr17294573pab.33.1383940814407; Fri, 08 Nov 2013 12:00:14 -0800 (PST)
Received: from [10.33.22.215] (128-107-239-235.cisco.com. [128.107.239.235]) by mx.google.com with ESMTPSA id lm2sm16397912pab.2.2013.11.08.12.00.13 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 08 Nov 2013 12:00:13 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Fri, 08 Nov 2013 12:00:09 -0700
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEA27FCC.4AF01%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CAEjQQ5UdW38nAm4WSO8CXL5B8Eo5weu65L17uYVzZJvzzZU3fw@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3466756813_10807102"
Cc: "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 20:00:55 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3466756813_10807102
Content-type: text/plain;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

Hi Naeem, all:

Please see detailed responses inline.

From:  Naeem Khademi <naeem.khademi@gmail.com>
Date:  Thursday, November 7, 2013 10:50 PM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  "aqm@ietf.org" <aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> I fully agree with the need to have AQMs that use "queuing delay" as metr=
ic
> and not "queue length" and this is already a bonus for algorithms that us=
e
> this metric (e.g. PIE). However, I believe that one can possibly modify R=
ED or
> its derivatives (e.g. Adaptive RED) to set their thresholds based on "que=
uing
> delay" as well (although such algorithms don't exists yet as of now). Whe=
ther
> the outcome of these modifications to xRED algorithms would make them bet=
ter
> than PIE and/or CoDel is another subject and I'm not going to speculate o=
n
> that here.=20

Of course. Our initial discussions considered a latency based RED
derivative. We later decided to focus on a proportional integral based
control law instead of RED (see below for more).

>=20
> One specific issue with PIE is that it tries to *estimate* the queuing de=
lay
> over a certain time interval (t_update) and still somehow uses the "queue
> length" and departure rate (over the interval) to estimate the queuing de=
lay.
> So saying that PIE uses the queueing delay instead of queue length may no=
t be
> 100% accurate.=20

Any latency based scheme requires the queueing delay to work with. The fact
of the matter is, the queueing delay is not directly available on platforms=
.
Some platforms can afford timestamping to figure the queuing delay but not
all platforms  can. In fact, over the past months we've come to realize tha=
t
a latency based scheme is actually two different items at work =8B (i) the
control law and (ii) the mechanism which provides the queueing delay to thi=
s
control law. This would be true for any latency based scheme, be it, PIE,
CoDEL, FQ_CoDEL or xRED. Based on this observation, we plan to separate out
the two components in PIE, possibly in the next version of the draft. The
objective being that (ii) would vary with the platform and it could be
direct input (if platform maintains it), timestamp-based (like in CoDEL
derivatives), dequeue-based, or even others that we are exploring at the
moment.=20

The current version of the PIE draft fully acknowledges the fact that what
PIE acquires with the dequee-based rate estimation is only an estimate of
the queuing delay and not the actual (Section 4.2 of v00). The larger point
for discussion here is a latency based AQM that uses queuing delay instead
of queue length.=20


>=20
> Moreover, in a varying bandwidth scenario, depending on how often the
> bandwidth changes and at what granularity, the derived "estimated queuing
> delay" by PIE might be an outdated information e.g. it shows how the chan=
nel
> has been (in term of delay) in the last 30ms and now how it is now! if I'=
m not
> mistaken that has been somehow resolved in DOCSIS 3.1 by coupling PIE's
> departure rate to the actual link layer rate.

We've experimented with timestamp-based delay inputs to PIE. We have not
seen any noticeable difference between the dequeue-based scheme and the
timestamp-based for all the scenarios Rong has presented at IETF thus far.
Even in extreme bandwidth changing conditions dequeue-based scheme can adap=
t
within few tupdate cycles. Again, PIE is flexible in its delay input
mechanism and will go with whatever is the best possible option available o=
n
the platform. I don't think any latency based scheme can get better than
that :).=20

>=20
> In overall, while there are of course significant improvements made in PI=
E's
> (and (FQ)-CoDel's) design, there are probably still lessons to be learned=
 from
> xRED family (check http://urn.nb.no/URN:NBN:no-38868 to see some examples=
).
> Throwing away all the knowledge and improvement gained from RED-like AQMs=
,
> CHoKe, etc and coming up with brand-new AQMs without (experimentally)
> comparing them to the gallery of existing ones wouldn't be great!

Our initial discussions considered more than just the RED varieties. As
we've mentioned before, PIE is based on the concept of proportional integra=
l
(PI). PI is not something we discovered. PI controller has been around for
long and you can refer to Hollot, Towsley et. al's paper from a decade ago
for more information on PI based AQM scheme. We believe PI's strong
theoretical stability is valuable for diverse deployment conditions and
incorporated PI into our PIE design. We did our own set of theoretical
analysis for PIE and we were proved right. You can find more details about
that in our HPSR paper (http://www.ieee-hpsr.org/2013/)

Thanks,
Preethi



--B_3466756813_10807102
Content-type: text/html;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:=
 14px; font-family: Calibri, sans-serif; "><div>Hi Naeem, all:</div><div><br=
></div><div>Please see detailed responses inline.</div><div><br></div><span =
id=3D"OLK_SRC_BODY_SECTION"><div style=3D"font-family:Calibri; font-size:11pt; t=
ext-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium=
 none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TO=
P: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span sty=
le=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D"mailto:naeem.=
khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br><span style=3D"font-weig=
ht:bold">Date: </span> Thursday, November 7, 2013 10:50 PM<br><span style=3D"f=
ont-weight:bold">To: </span> Preethi Natarajan &lt;<a href=3D"mailto:preethi.c=
is@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span style=3D"font-weight:bold=
">Cc: </span> "<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>" &lt;<a href=3D"=
mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><span style=3D"font-weight:bold">=
Subject: </span> Re: [aqm] AQM schemes: Queue length vs. delay based<br></di=
v><div><br></div><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"=
BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;"><div dir=3D"lt=
r">I fully agree with the need to have AQMs that use "queuing delay" as metr=
ic and not "queue length" and this is already a bonus for algorithms that us=
e this metric (e.g. PIE). However, I believe that one can possibly modify RE=
D or its derivatives (e.g. Adaptive RED) to set their thresholds based on "q=
ueuing delay" as well (although such algorithms don't exists yet as of now).=
 Whether the outcome of these modifications to xRED algorithms would make th=
em better than PIE and/or CoDel is another subject and I'm not going to spec=
ulate on that here.&nbsp;</div></blockquote></span><div><br></div><div><div>=
Of course. Our initial discussions considered a latency based RED derivative=
. We later decided to focus on a proportional integral based control law ins=
tead of RED (see below for more).&nbsp;</div></div><div><br></div><span id=3D"=
OLK_SRC_BODY_SECTION"><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" st=
yle=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;"><div di=
r=3D"ltr"><div><br></div><div>One specific issue with PIE is that it tries to =
*estimate* the queuing delay over a certain time interval (t_update) and sti=
ll somehow uses the "queue length" and departure rate (over the interval) to=
 estimate the queuing delay. So saying that PIE uses the queueing delay inst=
ead of queue length may not be 100% accurate.&nbsp;</div></div></blockquote>=
</span><div><br></div><div><div>Any latency based scheme requires the queuei=
ng delay to work with. The fact of the matter is, the queueing delay is not =
directly available on platforms. Some platforms can afford timestamping to f=
igure the queuing delay but not all platforms &nbsp;can. In fact, over the p=
ast months we've come to realize that a latency based scheme is actually two=
 different items at work &#8212; (i) the control law and (ii) the mechanism =
which provides the queueing delay to this control law. This would be true fo=
r&nbsp;<span style=3D"font-style: italic; ">any</span>&nbsp;latency based sche=
me, be it, PIE, CoDEL, FQ_CoDEL or xRED. Based on this observation, we plan =
to separate out the two components in PIE, possibly in the next version of t=
he draft. The objective being that (ii) would vary with the platform and it =
could be direct input (if platform maintains it), timestamp-based (like in C=
oDEL derivatives), dequeue-based, or even others that we are exploring at th=
e moment.&nbsp;</div><div><br></div><div>The current version of the PIE draf=
t fully acknowledges the fact that what PIE acquires with the dequee-based r=
ate estimation is only an&nbsp;<span style=3D"font-weight: bold; ">estimate</s=
pan>&nbsp;of the queuing delay and not the actual (Section 4.2 of v00). The =
larger point for discussion here is a latency based AQM that uses queuing de=
lay instead of queue length.&nbsp;</div></div><div><br></div><div><br></div>=
<span id=3D"OLK_SRC_BODY_SECTION"><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOC=
KQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5=
;"><div dir=3D"ltr"><div><br></div><div>Moreover, in a varying bandwidth scena=
rio, depending on how often the bandwidth changes and at what granularity, t=
he derived "estimated queuing delay" by PIE might be an outdated information=
 e.g. it shows how the channel has been (in term of delay) in the last 30ms =
and now how it is now! if I'm not mistaken that has been somehow resolved in=
 DOCSIS 3.1 by coupling PIE's departure rate to the actual link layer rate. =
&nbsp;</div></div></blockquote></span><div><br></div><div>We've experimented=
 with timestamp-based delay inputs to PIE. We have not seen any noticeable d=
ifference between the dequeue-based scheme and the timestamp-based for all t=
he scenarios Rong has presented at IETF thus far. Even in extreme bandwidth =
changing conditions dequeue-based scheme can adapt within few tupdate cycles=
. Again, PIE is flexible in its delay input mechanism and will go with whate=
ver is the best possible option available on the platform. I don't think any=
 latency based scheme can get better than that :).&nbsp;</div><div><br></div=
><span id=3D"OLK_SRC_BODY_SECTION"><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLO=
CKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 =
5;"><div dir=3D"ltr"><div><br></div><div>In overall, while there are of course=
 significant improvements made in PIE's (and (FQ)-CoDel's) design, there are=
 probably still lessons to be learned from xRED family (check&nbsp;<a href=3D"=
http://urn.nb.no/URN:NBN:no-38868" target=3D"_blank" style=3D"font-size:13px;fon=
t-family:arial,helvetica,sans-serif">http://urn.nb.no/URN:NBN:no-38868</a>&n=
bsp;to see some examples). Throwing away all the knowledge and improvement g=
ained from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs witho=
ut (experimentally) comparing them to the gallery of existing ones wouldn't =
be great!</div></div></blockquote></span><div><br></div><div>Our initial dis=
cussions considered more than just the RED varieties. As we've mentioned bef=
ore, PIE is based on the concept of proportional integral (PI). PI is not so=
mething we discovered. PI controller has been around for long and you can re=
fer to Hollot, Towsley et. al's paper from a decade ago for more information=
 on PI based AQM scheme. We believe PI's strong theoretical stability is val=
uable for diverse deployment conditions and incorporated PI into our PIE des=
ign. We did our own set of theoretical analysis for PIE and we were proved r=
ight. You can find more details about that in our HPSR paper (<a href=3D"http:=
//www.ieee-hpsr.org/2013">http://www.ieee-hpsr.org/2013</a>/)</div><div><br>=
</div><span id=3D"OLK_SRC_BODY_SECTION"><div dir=3D"ltr"><div><div class=3D"gmail_=
extra">Thanks,</div></div></div></span><div>Preethi</div></body></html>

--B_3466756813_10807102--



From shahid.akhtar@alcatel-lucent.com  Fri Nov  8 12:09:07 2013
Return-Path: <shahid.akhtar@alcatel-lucent.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 77DF321E8152 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:08:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.357
X-Spam-Level: 
X-Spam-Status: No, score=-10.357 tagged_above=-999 required=5 tests=[AWL=0.015, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id luEOA4HsaDfu for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:08:31 -0800 (PST)
Received: from ihemail2.lucent.com (ihemail2.lucent.com [135.245.0.35]) by ietfa.amsl.com (Postfix) with ESMTP id EA12921E8202 for <aqm@ietf.org>; Fri,  8 Nov 2013 12:08:25 -0800 (PST)
Received: from us70tusmtp2.zam.alcatel-lucent.com (h135-5-2-64.lucent.com [135.5.2.64]) by ihemail2.lucent.com (8.13.8/IER-o) with ESMTP id rA8K8Im3028944 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 8 Nov 2013 14:08:19 -0600 (CST)
Received: from US70TWXCHHUB04.zam.alcatel-lucent.com (us70twxchhub04.zam.alcatel-lucent.com [135.5.2.36]) by us70tusmtp2.zam.alcatel-lucent.com (GMO) with ESMTP id rA8K8HkV010052 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Nov 2013 15:08:17 -0500
Received: from US70UWXCHMBA01.zam.alcatel-lucent.com ([169.254.7.204]) by US70TWXCHHUB04.zam.alcatel-lucent.com ([135.5.2.36]) with mapi id 14.02.0247.003; Fri, 8 Nov 2013 15:08:17 -0500
From: "Akhtar, Shahid (Shahid)" <shahid.akhtar@alcatel-lucent.com>
To: David Ros <David.Ros@telecom-bretagne.eu>
Thread-Topic: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
Thread-Index: AQHO3LgYYRJ73SkRuUGeumV/fMXv9ZobwxY8
Date: Fri, 8 Nov 2013 20:08:17 +0000
Message-ID: <2763134A-F3B0-4BB2-BB76-B5AF52A1787E@alcatel-lucent.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A85C@SACEXCMBX02-PRD.hq.netapp.com> <C0611F522A6FA9498A044C36073E49657E5FF524@US70UWXCHMBA01.zam.alcatel-lucent.com> <5B72ED36-A189-4C01-80DB-F6D2F247CDDF@cisco.com> <C0611F522A6FA9498A044C36073E49657E5FFA98@US70UWXCHMBA01.zam.alcatel-lucent.com> <6A0A7988-7CEA-40E2-924A-80BCC141F1F1@cisco.com> <C0611F522A6FA9498A044C36073E49657E5FFFB8@US70UWXCHMBA01.zam.alcatel-lucent.com>, <B7249D76-0F19-4456-A01D-8C075E9946E9@telecom-bretagne.eu>
In-Reply-To: <B7249D76-0F19-4456-A01D-8C075E9946E9@telecom-bretagne.eu>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.57 on 135.245.2.35
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Richard Scheffenegger <rs@netapp.com>, Wesley Eddy <wes@mti-systems.com>, "Fred Baker \(fred\)" <fred@cisco.com>, "Naeem Khademi \(naeemk@ifi.uio.no\)" <naeemk@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 20:09:08 -0000

David,

Please see a quick answer to your questions below.

-Shahid.

Sent from my iPhone

On Nov 8, 2013, at 1:24 PM, "David Ros" <David.Ros@telecom-bretagne.eu> wro=
te:

>=20
> On 8 nov. 2013, at 10:38, Akhtar, Shahid (Shahid) <shahid.akhtar@alcatel-=
lucent.com> wrote:
>=20
> [snip]
>> Recommend the following text in section 4.7
>>=20
>> Research should focus on improving end user QoE from AQMs rather than ne=
twork related metrics only. Often a significant change in a network metric =
may only make a minimal change in end-user QoE and thus the value of such c=
hange may be minimal.
>=20
> Can this be said to be true for the QoE of *any* application?
>=20
> And, if we focus on QoE instead of network metrics, (a) what apps are we =
going to consider? (b) do we have easy-to-use, readily-available and accura=
te models for assessing QoE for all those apps?
>=20
> (Note, I'm not trying to contradict what you say, I just want to better u=
nderstand what concrete evaluation guidelines could be given on this)

My text says to focus on QoE as well - meaning both network metrics and ser=
vice QoE.

Key services on the Internet like web browsing have been around for like 20=
 years - so major services are slow to change.=20

Research does exist on translating application performance to QoE - key new=
 example may be HAS where I have mentioned a couple of papers in my talk at=
 ICCRG. Note that since this is a research topic, we do not need to have al=
l the answers now.

>=20
> Thanks,
>=20
> David
>=20
>>=20
>> Research should make suggestions on how to make good decisions on buffer=
 sizes with each type of AQM (e.g. 2xBDP) - explaining why or how such buff=
er sizes improve end-user QoE and network health.
>>=20
>> Research should be done on methods or good configurations that leverage =
deployed AQMs such as RED/WRED that reduce delays and prevent lockout with =
typical traffic and network conditions.=20
>>=20
>>=20
>>=20
>> From: Fred Baker (fred) [mailto:fred@cisco.com]=20
>> Sent: Friday, November 08, 2013 10:27 AM
>> To: Akhtar, Shahid (Shahid)
>> Cc: Richard Scheffenegger; aqm@ietf.org; Naeem Khademi (naeemk@ifi.uio.n=
o); Gorry Fairhurst; Wesley Eddy
>> Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B
>>=20
>>=20
>>=20
>> On Nov 8, 2013, at 5:56 AM, "Akhtar, Shahid (Shahid)" <shahid.akhtar@alc=
atel-lucent.com> wrote:
>>=20
>>> One of the the objectives of newer AQMs being defined here should be to=
 minimize tuning, but we should recognize that likely tuning or some config=
uration cannot be eliminated altogether.
>>>=20
>>> FB: That's an opinion. One of the objectives of Van and Kathy's work, a=
nd separately of Rong Pan et al's work, is to design an algorithm that may =
have different initial conditions drawn from a table given the interface it=
 finds itself on, but requires no manual tuning. The great failure of RED, =
recommended in RFC 2309, is not that it doesn't work when properly configur=
ed; it's that real humans don't have the time to properly tune it different=
ly for each of the thousands of link endpoints in their networks. There is =
no point in changing away from RED if that is also true of the replacement.
>>>=20
>>> SA: You argue that "initial conditions" determine some of the parameter=
s of newer AQMs (like Codel and PIE), then those same initial conditions wo=
uld also determine some of the key parameters for RED/WRED.
>>=20
>> I'm simply going to point out that Van and Kathy spent quite a bit of ti=
me and effort trying to do exactly that, and it didn't pan out. Codel is th=
eir suggestion of a replacement that is largely auto-tuning within a specif=
ied range of situations.
>>=20
>> On your other points, please suggest text, and the WG can discuss whethe=
r they buy it.
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> David ROS
> http://www.rennes.enst-bretagne.fr/~dros/
>=20
> Deadlines really start to press a week or two after they pass. -- John Pe=
rry
>=20
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

From lars@netapp.com  Fri Nov  8 12:20:52 2013
Return-Path: <lars@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C784D21E8139 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:20:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.882
X-Spam-Level: 
X-Spam-Status: No, score=-4.882 tagged_above=-999 required=5 tests=[AWL=-2.283, BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ywc3hgaVEKSO for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:20:46 -0800 (PST)
Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id B00A911E8115 for <aqm@ietf.org>; Fri,  8 Nov 2013 12:20:46 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,662,1378882800";  d="p7s'?scan'208";a="71803908"
Received: from vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) by mx11-out.netapp.com with ESMTP; 08 Nov 2013 12:20:46 -0800
Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.51]) by vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) with mapi id 14.03.0123.003; Fri, 8 Nov 2013 12:20:45 -0800
From: "Eggert, Lars" <lars@netapp.com>
To: "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: PIE & CableLabs
Thread-Index: AQHO3MAAFFH8PuCBIk6ZSa6cse1IYg==
Date: Fri, 8 Nov 2013 20:20:44 +0000
Message-ID: <7CF25FEB-7D1F-4AAA-92BC-C9DCE87EE029@netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.104.60.116]
Content-Type: multipart/signed; boundary="Apple-Mail=_6F6CE1BF-4219-4713-9481-74F2D7D060B1"; protocol="application/pkcs7-signature"; micalg=sha1
MIME-Version: 1.0
Cc: Ralph Droms <rdroms.ietf@gmail.com>
Subject: [aqm] PIE & CableLabs
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 20:20:52 -0000

--Apple-Mail=_6F6CE1BF-4219-4713-9481-74F2D7D060B1
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi,

just as a quick follow-up and maybe clarification to what I said at the =
meeting, could anyone find out:

(1) If the variant of PIE that CableLabs has settled on has a formal =
specification?

(2) If yes, if it is accessible to IETF participants?

(3) If yes, if their copyright, IPR and standardization rules let us =
work with it in the AQM WG like with any other contribution?

And I guess if any of these answers is "no", we'll need to figure out =
what that means for us in the AQM WG going forward.

Lars=

--Apple-Mail=_6F6CE1BF-4219-4713-9481-74F2D7D060B1
Content-Disposition: attachment; filename="smime.p7s"
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIINBzCCBkIw
ggUqoAMCAQICEDirAC//rpa3Vv85Wvtd5xswDQYJKoZIhvcNAQEFBQAwgcoxCzAJBgNVBAYTAlVT
MRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y
azE6MDgGA1UECxMxKGMpIDE5OTkgVmVyaVNpZ24sIEluYy4gLSBGb3IgYXV0aG9yaXplZCB1c2Ug
b25seTFFMEMGA1UEAxM8VmVyaVNpZ24gQ2xhc3MgMSBQdWJsaWMgUHJpbWFyeSBDZXJ0aWZpY2F0
aW9uIEF1dGhvcml0eSAtIEczMB4XDTExMDkwMTAwMDAwMFoXDTIxMDgzMTIzNTk1OVowgaYxCzAJ
BgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50
ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQD
Ey5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc0MIIBIjANBgkq
hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxuwn/R1j9DsdisHTHMjIgoa2uEqGkqqBXHLKMA0vnkEi
VzAhJZCao/SsKsaIF4ZhchN2LuwDyyebjyCAN+DkitpVplAP/LlcI2mJQqG6H6/vDvmkyQrx+Dey
xtmSSq5937hEH5u6P4wG/tgjT0hRI2pghKjuJy9g35byGiqMPI8AzE/L+iCOvDX24fCatgXz/B0/
xhR7DtryBeTTgwKmxWlwtKnkVunbHVz0pjbia7UeKi3cvrvuOgSwMAitX2hsxr0GloiE5+apZC28
ODC7iCbDZ2ZmtLR3+cChxw5y72bi5bnK4POFdzWY3tQcsP5mceI4y258T0BV65fZqBge7QIDAQAB
o4ICRDCCAkAwOAYIKwYBBQUHAQEELDAqMCgGCCsGAQUFBzABhhxodHRwOi8vcGtpLW9jc3AudmVy
aXNpZ24uY29tMBIGA1UdEwEB/wQIMAYBAf8CAQAwbAYDVR0gBGUwYzBhBgtghkgBhvhFAQcXATBS
MCYGCCsGAQUFBwIBFhpodHRwOi8vd3d3LnN5bWF1dGguY29tL2NwczAoBggrBgEFBQcCAjAcGhpo
dHRwOi8vd3d3LnN5bWF1dGguY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZl
cmlzaWduLmNvbS9wY2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAY
BgNVBAMTEVZlcmlTaWduTVBLSS0yLTk3MB0GA1UdDgQWBBSt+cOTci21uShh5KTXYNXECl4aATCB
8QYDVR0jBIHpMIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIElu
Yy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZl
cmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWdu
IENsYXNzIDEgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgLSBHM4IRAItb
dVaEVIULAM+vOEjOsaQwDQYJKoZIhvcNAQEFBQADggEBANaPwdqbiPKzbE0fWC+6AVFddMFG6MO4
e5/WQPHv/zK6iWvADjRDn6SZ5qTwXUgzYoWFYf4jiCKMYJsrnGVJlMSiOCRIpVylUEto6WIip5Po
mSJuPVu7EEIOH0x1RzRWCY/4vYw881y70pZwVHBiTe/REL6dSCxe7IZrB4LwPeElJygs4BZ2HrP9
5WKW0oo9Xyuu+1zCE7dlY8s0dkOf1oeZq26tlcEAP0Yngf813iMOQ9wUXzL5yinvwlIw9ZnduYH4
OiUgjYJo8rkhhXRmBOGGORYy8i3WKqjJ3tkAAk/jGCDFpYFWtpXe04Kt+HslvmR8LqC6cCz4+XXi
dE0HbYQwgga9MIIFpaADAgECAhBcHtb173S3Zimo+RJj8ZJuMA0GCSqGSIb3DQEBBQUAMIGmMQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5bWFu
dGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UE
AxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHNDAeFw0xMjEy
MTcwMDAwMDBaFw0xMzEyMTgyMzU5NTlaMIHBMS4wLAYDVQQDDCVQZXJzb25hIE5vdCBWYWxpZGF0
ZWQgLSAxMzU1NzU2NDEyNzI0MR4wHAYJKoZIhvcNAQkBFg9sYXJzQG5ldGFwcC5jb20xDzANBgNV
BAsMBlMvTUlNRTEeMBwGA1UECwwVUGVyc29uYSBOb3QgVmFsaWRhdGVkMR8wHQYDVQQLDBZTeW1h
bnRlYyBUcnVzdCBOZXR3b3JrMR0wGwYDVQQKDBRTeW1hbnRlYyBDb3Jwb3JhdGlvbjCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBANWkuu7uQOPw8ji7+u/ZDX+P8pwLX3a57b+qK9jkhUeW
9D5cKKI/7uU2u3HVqunJSvfz/cm6BCT9PF9QMitgKydjnzQvnhJmjo30uufsIIVbkrMgBamPmDck
tdW2wJA/IHvg6siTrRRJ+PIDFtNsu0IPQMSjYLBJN7OvYBl8WC291uEB3PW1XN77/RxhBdd00nP8
xeJjiSol5xUassDP5/gAkTGp6YCxulqtkMN4AqRNnnW1cS3GaSKGVnD+4/znuzNqrY5eEL87miPz
SKg3XvSaoTRNMb1SrE0kI8a35T7zeopaQ3cjlZD4cVlsQe/A4nTwZXnUYYGKPsxt0j8y5ykCAwEA
AaOCAsgwggLEMAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgWgMCAGA1UdJQEB/wQWMBQGCCsG
AQUFBwMEBggrBgEFBQcDAjAdBgNVHQ4EFgQUPsgtqNipyEoJQEaE5C/M7ejkMGUwGgYDVR0RBBMw
EYEPbGFyc0BuZXRhcHAuY29tMB8GA1UdIwQYMBaAFK35w5NyLbW5KGHkpNdg1cQKXhoBMIIBKwYI
KwYBBQUHAQEEggEdMIIBGTCCARUGCCsGAQUFBzAChoIBB2xkYXA6Ly9kaXJlY3RvcnkudmVyaXNp
Z24uY29tL0NOJTIwJTNEJTIwU3ltYW50ZWMlMjBDbGFzcyUyMDElMjBJbmRpdmlkdWFsJTIwU3Vi
c2NyaWJlciUyMENBJTIwLSUyMEc0JTJDJTIwT1UlMjAlM0QlMjBQZXJzb25hJTIwTm90JTIwVmFs
aWRhdGVkJTJDJTIwT1UlMjAlM0QlMjBTeW1hbnRlYyUyMFRydXN0JTIwTmV0d29yayUyQyUyME8l
MjAlM0QlMjBTeW1hbnRlYyUyMENvcnBvcmF0aW9uJTJDJTIwQyUyMCUzRCUyMFVTP2NBQ2VydGlm
aWNhdGU7YmluYXJ5MF0GA1UdHwRWMFQwUqBQoE6GTGh0dHA6Ly9wa2ktY3JsLnN5bWF1dGguY29t
L2NhXzU2MWMxMDM2OTBjOTdhNjkyNDdhMGVmMDcxYWM4MWFmL0xhdGVzdENSTC5jcmwwbAYDVR0g
BGUwYzBhBgtghkgBhvhFAQcXATBSMCYGCCsGAQUFBwIBFhpodHRwOi8vd3d3LnN5bWF1dGguY29t
L2NwczAoBggrBgEFBQcCAjAcGhpodHRwOi8vd3d3LnN5bWF1dGguY29tL3JwYTAqBgpghkgBhvhF
ARADBBwwGgYRYIZIAYb4RQEQAQICBAGGsxcWBTEwOTIyMA0GCSqGSIb3DQEBBQUAA4IBAQAzvR5s
H2ex5C1+qykxtOOdO+VkD1uygAoUxdh7K67SvXqbTF6c5oOVI80EwbjbZKFUk90Jf3D6HxlN/O69
smO+gCVAe3S+vXRB3XUaNPciBnqqUHJpuTlJhcqMPWpG30ZifNBQC8acFuNBHsdQyZNnbUeYHFYK
Di0CayGxT9bagFoUNBmDOJ7kvnBATaMQhX6y0aVBf3fUU5RXr0NEITVlEozE/KZWRN+vfuufICT0
keHCtoXkwI2kzH9VW7eroQk7t8/wowK8z2gW5Hm8ZWl+fbsfFEFVmJ633Ey7GiCmTzovQKgSuV4m
a7cYD68kzWVk+9SnBdZILEfefjjrRvmuMYID5DCCA+ACAQEwgbswgaYxCzAJBgNVBAYTAlVTMR0w
GwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0
d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBD
bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc0AhBcHtb173S3Zimo+RJj8ZJuMAkG
BSsOAwIaBQCgggH9MBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEz
MTEwODIwMjAzN1owIwYJKoZIhvcNAQkEMRYEFKVP7pEmk7YM/GD7gP/v9y/cKjUGMIHMBgkrBgEE
AYI3EAQxgb4wgbswgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlv
bjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3Qg
VmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJl
ciBDQSAtIEc0AhBcHtb173S3Zimo+RJj8ZJuMIHOBgsqhkiG9w0BCRACCzGBvqCBuzCBpjELMAkG
A1UEBhMCVVMxHTAbBgNVBAoTFFN5bWFudGVjIENvcnBvcmF0aW9uMR8wHQYDVQQLExZTeW1hbnRl
YyBUcnVzdCBOZXR3b3JrMR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMT
LlN5bWFudGVjIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzQCEFwe1vXvdLdm
Kaj5EmPxkm4wDQYJKoZIhvcNAQEBBQAEggEAPionxqhC9ejJgWms7vESpa5ucL7+XQWHsSiDDyRT
ih5zju6ACukiVNJJapZ8jA319Lr1nFxEfIOPjiZU2XyQr9s5TVF1ws/tEADOkvAWgiqa1DUf/sqy
lQZegrMMYFCC6iO2XJBksWllvcZ+CvOjL/LJvKhShQDZSU++S2p1CnJ2bt5OrnIAfa0/EjytU29J
vHXUHR8nLDC4dVUS2VwFdiAGs8LYjIORaNykz2S48I1Vj5WBIETwWvfA53Lwvl9SRA2bQvTopkRn
AF/sA7yYcW8XSUFABh6RUsFbdzyYSfLabC6hhnmr2joLZ85Fj5j084HRhsKeueEdVIpa1/7FvQAA
AAAAAA==

--Apple-Mail=_6F6CE1BF-4219-4713-9481-74F2D7D060B1--

From fred@cisco.com  Fri Nov  8 12:22:15 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E9F111E8241 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:22:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.354
X-Spam-Level: 
X-Spam-Status: No, score=-110.354 tagged_above=-999 required=5 tests=[AWL=0.018, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tM6oK4HxHFtn for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:22:10 -0800 (PST)
Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id 062AD11E8224 for <aqm@ietf.org>; Fri,  8 Nov 2013 12:22:03 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2275; q=dns/txt; s=iport; t=1383942124; x=1385151724; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=FgrkIP9LJ4J2AYsGVNDuJgrIetmQLH65qBS/5UOgSm0=; b=OfQyuEd4Ke68T0DUXiwTz4Qn+noxiNRutHoL7NsST8W1zNPeFaDy+cjv U5D1B/9pWM+CAuV4QIID1wYa+4rONxxGXaR7NOzl7cee+BgrZvnyMFryd RLc5C1BjIefxSm3yPJr0dqk4k89ypJ4OnHF3QcuJfLcCjGywezFdU949R 0=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhUFAAFHfVKtJXG9/2dsb2JhbABZgweBC78WgTEWdIIlAQEBAwF5BQsCAQhGMiUCBA4FDodtBr02j2cHgyCBEAOQMIEwhi+SC4Mmgio
X-IronPort-AV: E=Sophos;i="4.93,662,1378857600";  d="asc'?scan'208";a="282592947"
Received: from rcdn-core2-2.cisco.com ([173.37.113.189]) by rcdn-iport-7.cisco.com with ESMTP; 08 Nov 2013 20:22:03 +0000
Received: from xhc-rcd-x08.cisco.com (xhc-rcd-x08.cisco.com [173.37.183.82]) by rcdn-core2-2.cisco.com (8.14.5/8.14.5) with ESMTP id rA8KM3E9031862 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Nov 2013 20:22:03 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-rcd-x08.cisco.com ([173.37.183.82]) with mapi id 14.03.0123.003; Fri, 8 Nov 2013 14:22:03 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: "Scheffenegger, Richard" <rs@netapp.com>
Thread-Topic: [aqm] ECN use statement
Thread-Index: AQHO3MAvQS4FJP5dQ02Kex+90ftMtw==
Date: Fri, 8 Nov 2013 20:22:02 +0000
Message-ID: <ECC4EE5E-93F0-447F-94B5-EBFE2FFA2A7C@cisco.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25E6A876@SACEXCMBX02-PRD.hq.netapp.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E6A876@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.21.148.94]
Content-Type: multipart/signed; boundary="Apple-Mail=_8948142A-F9B5-406A-ACA2-C8A3C435DE25"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] ECN use statement
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 20:22:15 -0000

--Apple-Mail=_8948142A-F9B5-406A-ACA2-C8A3C435DE25
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252


On Nov 5, 2013, at 3:04 PM, "Scheffenegger, Richard" <rs@netapp.com> =
wrote:

> Hi,
> =20
> In today=92s discussion it became apparent, that we might need a =
stronger statement around the expected use of the IP ECN signaling, and =
encourage future implementers to support this, and also agree around the =
way how ECN should be used.
> =20
> I would like to encourage discussion around that interaction here on =
list=85

Part of the discussion, of course, should be exactly what words we want =
to say about it. Section 4.2.1 of draft-ietf-awm-recommendations is =
entirely on ECN, and in part says

   Network devices SHOULD use an AQM algorithm that marks ECN-capable
   traffic when making decisions about the response to congestion.
   Network devices need to implement this method by marking ECN-capable
   traffic or by dropping non-ECN-capable traffic.

Bob's comments this morning in tsvwg (which I wasn't present for but =
have been appraised of since) were in effect that ECN signals should be =
far more aggressive than loss signals. While loss is an effective =
signal, it also has QoE issues as identified in a separate thread, while =
ECN has no QoE impact. I have two views on that, diametrically opposed:

1) I wish everything would use ECN, so that we could achieve 100% =
utilization with statistically zero queuing delay and without loss.

2) Would that have the same effect as delay-based TCPs experience with =
respect to loss-based ones - they back out of the way too early?=20

It would be good to get some experience on that.


--Apple-Mail=_8948142A-F9B5-406A-ACA2-C8A3C435DE25
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSfUfobjEdbHIsm0MRAuGLAJ9kxXdFOU5WX0am92qhh6ZLbiW60gCghVwp
SdXZSXfUgBJPkyQ3EX05btA=
=6mu2
-----END PGP SIGNATURE-----

--Apple-Mail=_8948142A-F9B5-406A-ACA2-C8A3C435DE25--

From andrewmcgr@google.com  Fri Nov  8 12:39:00 2013
Return-Path: <andrewmcgr@google.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A531321E8099 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:39:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.51
X-Spam-Level: 
X-Spam-Status: No, score=-1.51 tagged_above=-999 required=5 tests=[AWL=0.240,  BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C+rid6IPoCh0 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:39:00 -0800 (PST)
Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) by ietfa.amsl.com (Postfix) with ESMTP id AF90021E8136 for <aqm@ietf.org>; Fri,  8 Nov 2013 12:38:26 -0800 (PST)
Received: by mail-qc0-f179.google.com with SMTP id k18so2180928qcv.38 for <aqm@ietf.org>; Fri, 08 Nov 2013 12:38:22 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lGq7IcjcYp/CfTe//QXENfxO0jdsm0PM94eEI+2CwYo=; b=f8yS/oWgZwUfLB4RwFgQRf8eA0RScrIndtL5+XKQsp1SwMht2d6i2aMbIp2m0I96Y3 6JREQcoOdYbX3C7gHyaUqUM14XRcU9LqrqCLfS3/pRHiaXiFYZq6VB54VTRY3AtS8B/L r4oR57Ip43oZhpcfjyfdhk42okyxo+nhUG5wQWGzSlUrjLDbC6kvmBPqkygaxxdGQDHM iFWC2frO+cHh48oq3GUWEK6yA3LLI/NgMbPyG6+3ZmHYJ4dsAUdPyK2cCVzIorjwBZKz FgugmQNXqG/LqHSE1sv3UA7LG5di7gzaoCBpWCZvZGpEr7NyOQ/kEEdV2dOnG1xxBXy9 quQg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=lGq7IcjcYp/CfTe//QXENfxO0jdsm0PM94eEI+2CwYo=; b=K6dMiwVks7CxztLEt+kRsqN9q3Bcg6tJOIrRul8uEWkxvhzuABbrVzLnkoax3p0Tor S9MP7kArYB3idl9BSoLDk92KPjx2pGGW1ZhK/CNtgi6z9zYJmN9BgLecHgPXU5EMRqmF e0vLqrbo8W7gqqZNqnp2aoTiuOfC7/l7s1UA41Z7GnE56Xc3DVTB+VCRGlyw1vKr7vHR ufelc9V/oJf6JqRxT2WHjk0tME53HYJA1JadWGSnMyeB/pH6lJK6/FwKyMhzUYooKi4v NwYtTfQuBnKwxup2yJjbPjaiFRCacjzhVQvgKbsFLYLALDTpmVDZ3DGvebsoyBpiOrGq IDiQ==
X-Gm-Message-State: ALoCoQljcj9DSW8UwhEzMli6UThwvy/uk5sqYbO6nZGxHSAAtngkSBiS2Uf/hbIBaM9c4pO4E6W1b2ZCGx6sLjM5MEB852VMVvtxeMeUlFqD54GdOS1GJtwnfKL6bTbaXJIPVSD+kKcMhaixqEu5m8L2T9YvUxgkyLC8pSj5tAHClmrdRUPeD7DuRDPcr+1u/xWD8xaDNIFn
MIME-Version: 1.0
X-Received: by 10.49.97.6 with SMTP id dw6mr25676406qeb.47.1383943101974; Fri, 08 Nov 2013 12:38:21 -0800 (PST)
Received: by 10.224.197.198 with HTTP; Fri, 8 Nov 2013 12:38:21 -0800 (PST)
In-Reply-To: <145AB1C0-B108-4196-AD5C-3667103FB4D8@cisco.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <145AB1C0-B108-4196-AD5C-3667103FB4D8@cisco.com>
Date: Fri, 8 Nov 2013 12:38:21 -0800
Message-ID: <CAPRuP3==GDrL2UqMkXbFgbWkS_s0c=TeUGXaemaO=Q9oMO9NcA@mail.gmail.com>
From: Andrew Mcgregor <andrewmcgr@google.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Content-Type: multipart/alternative; boundary=047d7bb0432afbaeb204eab05c51
Cc: Bob Briscoe <bob.briscoe@bt.com>, tsvwg IETF list <tsvwg@ietf.org>, "<draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, AQM IETF list <aqm@ietf.org>, Andrew McGregor <andrewmcgr@gmail.com>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 20:39:00 -0000

--047d7bb0432afbaeb204eab05c51
Content-Type: text/plain; charset=ISO-8859-1

I agree, non-congestive loss notification is a research problem.  So, out
of scope for the IETF right now but I hope someone thinks about it (I will,
as a background task).


On 8 November 2013 11:49, Fred Baker (fred) <fred@cisco.com> wrote:

>
> At 23:04 04/11/2013, Andrew Mcgregor wrote:
> > This seems like valuable work.  One question is, can we put in scope
> notification that losses are NOT due to congestion?
>
> Speaking for myself, I'm not sure how I would do that.
>
> A loss (or mark) due to congestion is pretty simple. The switch knows what
> it did.
>
> To know that a packet was lost for reasons other than congestion, I need
> to somehow know what packets I should expect, and infer that something that
> I expected didn't happen. In TCP, we know about data segments because they
> are enumerated - I know what the next octet sequence number to expect, and
> it doesn't arrive. Control segments (SYN, ACK, FIN, RST, and so on) are not
> enumerated in that sense - if my peer sends ten identical acks and nine
> arrive, I as the receiver have no way to know that. At the link layer, most
> link protocols in use today (PPP, Ethernet, and so on) do not enumerate
> packets in flight - they are simply there. I *might* be able to see a burst
> of noise on the line, but only if it looks like it might be the start of a
> packet and then doesn't end with the right checksum. Even if I can see it,
> I have no way to know whether the noise garbled one packet or many.
>
> If you want to do some research and come up with a solution, be my guest.
> But in a standard discussed in 2013... let's not.
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>


-- 
Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128

--047d7bb0432afbaeb204eab05c51
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I agree, non-congestive loss notification is a research pr=
oblem. =A0So, out of scope for the IETF right now but I hope someone thinks=
 about it (I will, as a background task).</div><div class=3D"gmail_extra"><=
br>
<br><div class=3D"gmail_quote">On 8 November 2013 11:49, Fred Baker (fred) =
<span dir=3D"ltr">&lt;<a href=3D"mailto:fred@cisco.com" target=3D"_blank">f=
red@cisco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class=3D"im"><br>
At 23:04 04/11/2013, Andrew Mcgregor wrote:<br>
&gt; This seems like valuable work. =A0One question is, can we put in scope=
 notification that losses are NOT due to congestion?<br>
<br>
</div>Speaking for myself, I&#39;m not sure how I would do that.<br>
<br>
A loss (or mark) due to congestion is pretty simple. The switch knows what =
it did.<br>
<br>
To know that a packet was lost for reasons other than congestion, I need to=
 somehow know what packets I should expect, and infer that something that I=
 expected didn&#39;t happen. In TCP, we know about data segments because th=
ey are enumerated - I know what the next octet sequence number to expect, a=
nd it doesn&#39;t arrive. Control segments (SYN, ACK, FIN, RST, and so on) =
are not enumerated in that sense - if my peer sends ten identical acks and =
nine arrive, I as the receiver have no way to know that. At the link layer,=
 most link protocols in use today (PPP, Ethernet, and so on) do not enumera=
te packets in flight - they are simply there. I *might* be able to see a bu=
rst of noise on the line, but only if it looks like it might be the start o=
f a packet and then doesn&#39;t end with the right checksum. Even if I can =
see it, I have no way to know whether the noise garbled one packet or many.=
<br>

<br>
If you want to do some research and come up with a solution, be my guest. B=
ut in a standard discussed in 2013... let&#39;s not.<br>
<br>_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><br>
<br></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div dir=
=3D"ltr"><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-siz=
e:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;borde=
r-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Andrew McGregor=A0|<=
/span><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:s=
mall;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-c=
olor:rgb(51,105,232);padding-top:2px;margin-top:2px">=A0SRE=A0|</span><span=
 style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-h=
eight:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,=
153,57);padding-top:2px;margin-top:2px">=A0<a href=3D"mailto:andrewmcgr@goo=
gle.com" target=3D"_blank">andrewmcgr@google.com</a>=A0|</span><span style=
=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:=
1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,=
17);padding-top:2px;margin-top:2px">=A0+61 4 8143 7128</span><br>
</div>
</div>

--047d7bb0432afbaeb204eab05c51--

From g.white@CableLabs.com  Fri Nov  8 12:52:38 2013
Return-Path: <g.white@CableLabs.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 29F8E21E8146 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:52:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.236
X-Spam-Level: 
X-Spam-Status: No, score=-0.236 tagged_above=-999 required=5 tests=[AWL=0.001,  BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oZtAk+zSJ2MS for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 12:52:34 -0800 (PST)
Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 1E3BF11E81F3 for <aqm@ietf.org>; Fri,  8 Nov 2013 12:52:13 -0800 (PST)
Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id rA8Kq9Br029737; Fri, 8 Nov 2013 13:52:09 -0700
Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 08 Nov 2013 13:52:08 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com)
Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Fri, 8 Nov 2013 13:52:08 -0700
From: Greg White <g.white@CableLabs.com>
To: "Eggert, Lars" <lars@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: [aqm] PIE & CableLabs
Thread-Index: AQHO3MAAFFH8PuCBIk6ZSa6cse1IYpobvoAA
Date: Fri, 8 Nov 2013 20:52:08 +0000
Message-ID: <CEA2994A.21C54%g.white@cablelabs.com>
In-Reply-To: <7CF25FEB-7D1F-4AAA-92BC-C9DCE87EE029@netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.5.0.27]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <460FF03C0929234D98F13866FCA93B95@cablelabs.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Approved: ondar
Cc: Ralph Droms <rdroms.ietf@gmail.com>
Subject: Re: [aqm] PIE & CableLabs
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 20:52:38 -0000

Lars,

1) Yes, CableLabs has settled on a formal specification.  It can be found
in Annex M of=20
http://www.cablelabs.com/specifications/CM-SP-MULPIv3.1-I01-131029.pdf

2) see above.

3) Probably not directly. We have requested that Cisco update the PIE I-D
to include the updates to the core PIE algorithm made as part of the
CableLabs evaluation. CableLabs will draft an I-D to cover the remaining
gaps between that and what can be found in 1.  I think this achieves what
you are after.  Let me know if not.

-Greg


On 11/8/13, 1:20 PM, "Eggert, Lars" <lars@netapp.com> wrote:

>Hi,
>
>just as a quick follow-up and maybe clarification to what I said at the
>meeting, could anyone find out:
>
>(1) If the variant of PIE that CableLabs has settled on has a formal
>specification?
>
>(2) If yes, if it is accessible to IETF participants?
>
>(3) If yes, if their copyright, IPR and standardization rules let us work
>with it in the AQM WG like with any other contribution?
>
>And I guess if any of these answers is "no", we'll need to figure out
>what that means for us in the AQM WG going forward.
>
>Lars


From michawe@ifi.uio.no  Fri Nov  8 13:11:38 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 609C321E819F for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:11:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.327
X-Spam-Level: 
X-Spam-Status: No, score=-102.327 tagged_above=-999 required=5 tests=[AWL=0.044, BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0HBw3Vg4L2dN for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:11:33 -0800 (PST)
Received: from mail-out4.uio.no (mail-out4.uio.no [IPv6:2001:700:100:10::15]) by ietfa.amsl.com (Postfix) with ESMTP id EA5C721E814E for <aqm@ietf.org>; Fri,  8 Nov 2013 13:11:15 -0800 (PST)
Received: from mail-mx4.uio.no ([129.240.10.45]) by mail-out4.uio.no with esmtp (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1VetKa-00089T-Vv; Fri, 08 Nov 2013 22:11:04 +0100
Received: from dhcp-98ed.meeting.ietf.org ([31.133.152.237]) by mail-mx4.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from <michawe@ifi.uio.no>) id 1VetKZ-0000OP-HK; Fri, 08 Nov 2013 22:11:04 +0100
Content-Type: multipart/alternative; boundary="Apple-Mail=_37DD1630-F207-4B53-8A19-0FFFA3B2827E"
Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\))
From: Michael Welzl <michawe@ifi.uio.no>
In-Reply-To: <CEA27FCC.4AF01%prenatar@cisco.com>
Date: Fri, 8 Nov 2013 13:11:00 -0800
Message-Id: <40D660CC-6A01-43DD-AF72-EE0A1C4503F6@ifi.uio.no>
References: <CEA27FCC.4AF01%prenatar@cisco.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
X-Mailer: Apple Mail (2.1510)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 3 msgs/h 1 sum rcpts/h 3 sum msgs/h 1 total rcpts 9620 max rcpts/h 44 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: DCE61B5D7A234AD153540D68066CD8EF66B44A6F
X-UiO-SPAM-Test: remote_host: 31.133.152.237 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 25 max/h 7 blacklist 0 greylist 0 ratelimit 0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 21:11:38 -0000

--Apple-Mail=_37DD1630-F207-4B53-8A19-0FFFA3B2827E
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252


On 8. nov. 2013, at 11:00, Preethi Natarajan <preethi.cis@gmail.com> =
wrote:

> Hi Naeem, all:
>=20
> Please see detailed responses inline.
>=20
> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Thursday, November 7, 2013 10:50 PM
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: "aqm@ietf.org" <aqm@ietf.org>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>=20
>> I fully agree with the need to have AQMs that use "queuing delay" as =
metric and not "queue length" and this is already a bonus for algorithms =
that use this metric (e.g. PIE). However, I believe that one can =
possibly modify RED or its derivatives (e.g. Adaptive RED) to set their =
thresholds based on "queuing delay" as well (although such algorithms =
don't exists yet as of now). Whether the outcome of these modifications =
to xRED algorithms would make them better than PIE and/or CoDel is =
another subject and I'm not going to speculate on that here.=20
>=20
>=20
> Of course. Our initial discussions considered a latency based RED =
derivative. We later decided to focus on a proportional integral based =
control law instead of RED (see below for more).=20
>=20
>>=20
>> One specific issue with PIE is that it tries to *estimate* the =
queuing delay over a certain time interval (t_update) and still somehow =
uses the "queue length" and departure rate (over the interval) to =
estimate the queuing delay. So saying that PIE uses the queueing delay =
instead of queue length may not be 100% accurate.=20
>=20
>=20
> Any latency based scheme requires the queueing delay to work with. The =
fact of the matter is, the queueing delay is not directly available on =
platforms. Some platforms can afford timestamping to figure the queuing =
delay but not all platforms  can. In fact, over the past months we've =
come to realize that a latency based scheme is actually two different =
items at work =97 (i) the control law and (ii) the mechanism which =
provides the queueing delay to this control law. This would be true for =
any latency based scheme, be it, PIE, CoDEL, FQ_CoDEL or xRED. Based on =
this observation, we plan to separate out the two components in PIE, =
possibly in the next version of the draft. The objective being that (ii) =
would vary with the platform and it could be direct input (if platform =
maintains it), timestamp-based (like in CoDEL derivatives), =
dequeue-based, or even others that we are exploring at the moment.=20
>=20
> The current version of the PIE draft fully acknowledges the fact that =
what PIE acquires with the dequee-based rate estimation is only an =
estimate of the queuing delay and not the actual (Section 4.2 of v00). =
The larger point for discussion here is a latency based AQM that uses =
queuing delay instead of queue length.=20
>=20
>=20
>>=20
>> Moreover, in a varying bandwidth scenario, depending on how often the =
bandwidth changes and at what granularity, the derived "estimated =
queuing delay" by PIE might be an outdated information e.g. it shows how =
the channel has been (in term of delay) in the last 30ms and now how it =
is now! if I'm not mistaken that has been somehow resolved in DOCSIS 3.1 =
by coupling PIE's departure rate to the actual link layer rate. =20
>=20
>=20
> We've experimented with timestamp-based delay inputs to PIE. We have =
not seen any noticeable difference between the dequeue-based scheme and =
the timestamp-based for all the scenarios Rong has presented at IETF =
thus far. Even in extreme bandwidth changing conditions dequeue-based =
scheme can adapt within few tupdate cycles. Again, PIE is flexible in =
its delay input mechanism and will go with whatever is the best possible =
option available on the platform. I don't think any latency based scheme =
can get better than that :).=20

An AQM scheme should also work when the bandwidth is *not* changing. The =
results that Naeem presented in ICCRG don't indicate that PIE is =
consistently better than CoDel (which is also delay-based). I'm not =
trying to say if this or that is better, just that statements like this =
one are rather handwavy.


>> In overall, while there are of course significant improvements made =
in PIE's (and (FQ)-CoDel's) design, there are probably still lessons to =
be learned from xRED family (check http://urn.nb.no/URN:NBN:no-38868 to =
see some examples). Throwing away all the knowledge and improvement =
gained from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs =
without (experimentally) comparing them to the gallery of existing ones =
wouldn't be great!
>=20
>=20
> Our initial discussions considered more than just the RED varieties. =
As we've mentioned before, PIE is based on the concept of proportional =
integral (PI). PI is not something we discovered. PI controller has been =
around for long and you can refer to Hollot, Towsley et. al's paper from =
a decade ago for more information on PI based AQM scheme. We believe =
PI's strong theoretical stability is valuable for diverse deployment =
conditions and incorporated PI into our PIE design. We did our own set =
of theoretical analysis for PIE and we were proved right. You can find =
more details about that in our HPSR paper =
(http://www.ieee-hpsr.org/2013/)

Another handwavy statement. Using a PI controller may be a good design =
idea, but that alone doesn't really tell us much about the overall =
behavior.

Cheers,
Michael


--Apple-Mail=_37DD1630-F207-4B53-8A19-0FFFA3B2827E
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On 8. nov. 2013, at 11:00, Preethi Natarajan &lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; font-size: 14px; =
font-family: Calibri, sans-serif; "><div>Hi Naeem, =
all:</div><div><br></div><div>Please see detailed responses =
inline.</div><div><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div =
style=3D"font-family: Calibri; font-size: 11pt; text-align: left; =
border-width: 1pt medium medium; border-style: solid none none; padding: =
3pt 0in 0in; border-top-color: rgb(181, 196, 223); "><span =
style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a =
href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br=
><span style=3D"font-weight:bold">Date: </span> Thursday, November 7, =
2013 10:50 PM<br><span style=3D"font-weight:bold">To: </span> Preethi =
Natarajan &lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><sp=
an style=3D"font-weight:bold">Cc: </span> "<a =
href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>" &lt;<a =
href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><span =
style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue =
length vs. delay based<br></div><div><br></div><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
dir=3D"ltr">I fully agree with the need to have AQMs that use "queuing =
delay" as metric and not "queue length" and this is already a bonus for =
algorithms that use this metric (e.g. PIE). However, I believe that one =
can possibly modify RED or its derivatives (e.g. Adaptive RED) to set =
their thresholds based on "queuing delay" as well (although such =
algorithms don't exists yet as of now). Whether the outcome of these =
modifications to xRED algorithms would make them better than PIE and/or =
CoDel is another subject and I'm not going to speculate on that =
here.&nbsp;</div></blockquote></span><div><br></div><div>Of course. Our =
initial discussions considered a latency based RED derivative. We later =
decided to focus on a proportional integral based control law instead of =
RED (see below for more).&nbsp;</div><div><br></div><span =
id=3D"OLK_SRC_BODY_SECTION"><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
dir=3D"ltr"><div><br></div><div>One specific issue with PIE is that it =
tries to *estimate* the queuing delay over a certain time interval =
(t_update) and still somehow uses the "queue length" and departure rate =
(over the interval) to estimate the queuing delay. So saying that PIE =
uses the queueing delay instead of queue length may not be 100% =
accurate.&nbsp;</div></div></blockquote></span><div><br></div><div><div>An=
y latency based scheme requires the queueing delay to work with. The =
fact of the matter is, the queueing delay is not directly available on =
platforms. Some platforms can afford timestamping to figure the queuing =
delay but not all platforms &nbsp;can. In fact, over the past months =
we've come to realize that a latency based scheme is actually two =
different items at work =97 (i) the control law and (ii) the mechanism =
which provides the queueing delay to this control law. This would be =
true for&nbsp;<span style=3D"font-style: italic; =
">any</span>&nbsp;latency based scheme, be it, PIE, CoDEL, FQ_CoDEL or =
xRED. Based on this observation, we plan to separate out the two =
components in PIE, possibly in the next version of the draft. The =
objective being that (ii) would vary with the platform and it could be =
direct input (if platform maintains it), timestamp-based (like in CoDEL =
derivatives), dequeue-based, or even others that we are exploring at the =
moment.&nbsp;</div><div><br></div><div>The current version of the PIE =
draft fully acknowledges the fact that what PIE acquires with the =
dequee-based rate estimation is only an&nbsp;<span style=3D"font-weight: =
bold; ">estimate</span>&nbsp;of the queuing delay and not the actual =
(Section 4.2 of v00). The larger point for discussion here is a latency =
based AQM that uses queuing delay instead of queue =
length.&nbsp;</div></div><div><br></div><div><br></div><span =
id=3D"OLK_SRC_BODY_SECTION"><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
dir=3D"ltr"><div><br></div><div>Moreover, in a varying bandwidth =
scenario, depending on how often the bandwidth changes and at what =
granularity, the derived "estimated queuing delay" by PIE might be an =
outdated information e.g. it shows how the channel has been (in term of =
delay) in the last 30ms and now how it is now! if I'm not mistaken that =
has been somehow resolved in DOCSIS 3.1 by coupling PIE's departure rate =
to the actual link layer rate. =
&nbsp;</div></div></blockquote></span><div><br></div><div>We've =
experimented with timestamp-based delay inputs to PIE. We have not seen =
any noticeable difference between the dequeue-based scheme and the =
timestamp-based for all the scenarios Rong has presented at IETF thus =
far. Even in extreme bandwidth changing conditions dequeue-based scheme =
can adapt within few tupdate cycles. Again, PIE is flexible in its delay =
input mechanism and will go with whatever is the best possible option =
available on the platform. I don't think any latency based scheme can =
get better than that =
:).&nbsp;</div></div></blockquote><div><br></div><div>An AQM scheme =
should also work when the bandwidth is *not* changing. The results that =
Naeem presented in ICCRG don't indicate that PIE is consistently better =
than CoDel (which is also delay-based). I'm not trying to say if this or =
that is better, just that statements like this one are rather =
handwavy.</div><div><br></div><br><blockquote type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 14px; font-family: =
Calibri, sans-serif; "><span id=3D"OLK_SRC_BODY_SECTION"><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
dir=3D"ltr"><div>In overall, while there are of course significant =
improvements made in PIE's (and (FQ)-CoDel's) design, there are probably =
still lessons to be learned from xRED family (check&nbsp;<a =
href=3D"http://urn.nb.no/URN:NBN:no-38868" target=3D"_blank" =
style=3D"font-size:13px;font-family:arial,helvetica,sans-serif">http://urn=
.nb.no/URN:NBN:no-38868</a>&nbsp;to see some examples). Throwing away =
all the knowledge and improvement gained from RED-like AQMs, CHoKe, etc =
and coming up with brand-new AQMs without (experimentally) comparing =
them to the gallery of existing ones wouldn't be =
great!</div></div></blockquote></span><div><br></div><div>Our initial =
discussions considered more than just the RED varieties. As we've =
mentioned before, PIE is based on the concept of proportional integral =
(PI). PI is not something we discovered. PI controller has been around =
for long and you can refer to Hollot, Towsley et. al's paper from a =
decade ago for more information on PI based AQM scheme. We believe PI's =
strong theoretical stability is valuable for diverse deployment =
conditions and incorporated PI into our PIE design. We did our own set =
of theoretical analysis for PIE and we were proved right. You can find =
more details about that in our HPSR paper (<a =
href=3D"http://www.ieee-hpsr.org/2013">http://www.ieee-hpsr.org/2013</a>/)=
</div></div></blockquote><div><br></div></div>Another handwavy =
statement. Using a PI controller may be a good design idea, but that =
alone doesn't really tell us much about the overall =
behavior.<div><br></div><div>Cheers,</div><div>Michael</div><div><br></div=
></body></html>=

--Apple-Mail=_37DD1630-F207-4B53-8A19-0FFFA3B2827E--

From dave.taht@gmail.com  Fri Nov  8 13:19:49 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 108DF21E814E for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:19:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.439
X-Spam-Level: 
X-Spam-Status: No, score=-2.439 tagged_above=-999 required=5 tests=[AWL=0.161,  BAYES_00=-2.599, NO_RELAYS=-0.001]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rrmXCc9t70N9 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:19:47 -0800 (PST)
Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) by ietfa.amsl.com (Postfix) with ESMTP id BE20E21E8147 for <aqm@ietf.org>; Fri,  8 Nov 2013 13:19:46 -0800 (PST)
Received: by mail-we0-f169.google.com with SMTP id q58so2560315wes.28 for <aqm@ietf.org>; Fri, 08 Nov 2013 13:19:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=aZCOXNcqqoOhh64V1SCOSFOwxXGzzHBMoGjwp6rXSMw=; b=TLZ2gAn8pg+k3nJ23QIV6Yd7sXx0Hg3RJURWgIN7lENymL+4TzeR6tx5qM/c0OLcmh gOVf2dQsMA9ase41tAHaTbykIeg1PjrDyKa+mrvgR562iip5SkvRqTXe3d37H4YUl7Gd 2UbLwZMb1L162DxJH8tIYb5I0jC6GtbFYQzyMTsdyYiE9HOxdiyAy+mBXT+HFcxoQpAw gM2Qq8mv9E06h5jtX49QVOqYiFQQxgaaCYdlOtGY7RyiChqkY8CXLebmEua4J2D+TSyX 72zKpeGmE3MH5Zvd/ZUN6ElwvapK7PFwmqDG/5n1Ua9+Z1D7upwfHmg/Vnq8k0sWNuyn h+VQ==
MIME-Version: 1.0
X-Received: by 10.180.19.133 with SMTP id f5mr3990102wie.60.1383945585435; Fri, 08 Nov 2013 13:19:45 -0800 (PST)
Received: by 10.217.51.5 with HTTP; Fri, 8 Nov 2013 13:19:45 -0800 (PST)
Date: Fri, 8 Nov 2013 13:19:45 -0800
Message-ID: <CAA93jw7mh3ceQ3KL_VSm+T3FJuQ3nG7G_NJxxRuUtWJjbVZhyQ@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: "aqm@ietf.org" <aqm@ietf.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Subject: [aqm] what do people regard as useful from the rrul tests?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 21:19:49 -0000

As most of the graphs that come from my side are based o this tool,
I'd like to refresh people's knowledge of the rrul specification, and
work on revising it and the code this winter.

https://github.com/dtaht/deBloat/blob/master/spec/rrule.doc?raw=3Dtrue


Already in the suite are tcp up/down,bidirectional, udp, vs udp flood,
mixed-rtt, ecn, and other tests.

https://github.com/tohojo/netperf-wrapper

upcoming additions are one way delay measurements in particular,
leveraging owamp.

Pulling useful metrics out of it rather than pretty graphs would be good.

--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From lars@netapp.com  Fri Nov  8 13:22:17 2013
Return-Path: <lars@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3060311E8112 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:22:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.75
X-Spam-Level: 
X-Spam-Status: No, score=-4.75 tagged_above=-999 required=5 tests=[AWL=-2.378,  BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GmDv-OI2HXlr for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:22:10 -0800 (PST)
Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id 5029311E80DE for <aqm@ietf.org>; Fri,  8 Nov 2013 13:22:10 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,662,1378882800";  d="p7s'?scan'208";a="71828317"
Received: from vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) by mx11-out.netapp.com with ESMTP; 08 Nov 2013 13:22:08 -0800
Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.51]) by vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) with mapi id 14.03.0158.001; Fri, 8 Nov 2013 13:22:08 -0800
From: "Eggert, Lars" <lars@netapp.com>
To: Greg White <g.white@cablelabs.com>
Thread-Topic: [aqm] PIE & CableLabs
Thread-Index: AQHO3MAAFFH8PuCBIk6ZSa6cse1IYpobvoAAgACfOoA=
Date: Fri, 8 Nov 2013 21:22:07 +0000
Message-ID: <17BA4215-B855-4424-9D8C-FCFE11A1849E@netapp.com>
References: <CEA2994A.21C54%g.white@cablelabs.com>
In-Reply-To: <CEA2994A.21C54%g.white@cablelabs.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.104.60.116]
Content-Type: multipart/signed; boundary="Apple-Mail=_5CAC4D28-EE66-4799-8110-B1D0EB749B27"; protocol="application/pkcs7-signature"; micalg=sha1
MIME-Version: 1.0
Cc: "aqm@ietf.org" <aqm@ietf.org>, Ralph Droms <rdroms.ietf@gmail.com>
Subject: Re: [aqm] PIE & CableLabs
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 21:22:17 -0000

--Apple-Mail=_5CAC4D28-EE66-4799-8110-B1D0EB749B27
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi,

On 08 Nov 2013, at 12:52, Greg White <g.white@cablelabs.com> wrote:
> 1) Yes, CableLabs has settled on a formal specification.  It can be =
found
> in Annex M of=20
> http://www.cablelabs.com/specifications/CM-SP-MULPIv3.1-I01-131029.pdf
>=20
> 2) see above.

great, thanks Greg!

> 3) Probably not directly. We have requested that Cisco update the PIE =
I-D
> to include the updates to the core PIE algorithm made as part of the
> CableLabs evaluation. CableLabs will draft an I-D to cover the =
remaining
> gaps between that and what can be found in 1.  I think this achieves =
what
> you are after.  Let me know if not.

That sounds like pretty good plan.

One remaining question is what will happen if the AQM WG - after =
adopting the PIE contribution - makes modifications. Is CableLabs =
prepared to update their specs based on whet we'll do, referring to an =
eventual IETF RFC rather than maintaining a separate spec? Or will =
CableLabs also continue work on the spec and we may end up with eventual =
divergence? (Or will the CableLabs spec of PIE remain frozen at its =
current version?)

Having asked that, I want to make clear that the working relationship =
with CableLabs has been good, and it's not that I'm expecting problems =
here. It's after all - at least at the moment - the same people working =
on PIE here and there. It's just that I've been involved in cases in the =
IETF before where things fell through the cracks when two SDOs =
simultaneously worked on something, and it's messy to create alignment =
after that has happened. So an explicit agreement about how this will go =
forward would be good to have.

Thanks,
Lars



--Apple-Mail=_5CAC4D28-EE66-4799-8110-B1D0EB749B27
Content-Disposition: attachment; filename="smime.p7s"
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIINBzCCBkIw
ggUqoAMCAQICEDirAC//rpa3Vv85Wvtd5xswDQYJKoZIhvcNAQEFBQAwgcoxCzAJBgNVBAYTAlVT
MRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y
azE6MDgGA1UECxMxKGMpIDE5OTkgVmVyaVNpZ24sIEluYy4gLSBGb3IgYXV0aG9yaXplZCB1c2Ug
b25seTFFMEMGA1UEAxM8VmVyaVNpZ24gQ2xhc3MgMSBQdWJsaWMgUHJpbWFyeSBDZXJ0aWZpY2F0
aW9uIEF1dGhvcml0eSAtIEczMB4XDTExMDkwMTAwMDAwMFoXDTIxMDgzMTIzNTk1OVowgaYxCzAJ
BgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50
ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQD
Ey5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc0MIIBIjANBgkq
hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxuwn/R1j9DsdisHTHMjIgoa2uEqGkqqBXHLKMA0vnkEi
VzAhJZCao/SsKsaIF4ZhchN2LuwDyyebjyCAN+DkitpVplAP/LlcI2mJQqG6H6/vDvmkyQrx+Dey
xtmSSq5937hEH5u6P4wG/tgjT0hRI2pghKjuJy9g35byGiqMPI8AzE/L+iCOvDX24fCatgXz/B0/
xhR7DtryBeTTgwKmxWlwtKnkVunbHVz0pjbia7UeKi3cvrvuOgSwMAitX2hsxr0GloiE5+apZC28
ODC7iCbDZ2ZmtLR3+cChxw5y72bi5bnK4POFdzWY3tQcsP5mceI4y258T0BV65fZqBge7QIDAQAB
o4ICRDCCAkAwOAYIKwYBBQUHAQEELDAqMCgGCCsGAQUFBzABhhxodHRwOi8vcGtpLW9jc3AudmVy
aXNpZ24uY29tMBIGA1UdEwEB/wQIMAYBAf8CAQAwbAYDVR0gBGUwYzBhBgtghkgBhvhFAQcXATBS
MCYGCCsGAQUFBwIBFhpodHRwOi8vd3d3LnN5bWF1dGguY29tL2NwczAoBggrBgEFBQcCAjAcGhpo
dHRwOi8vd3d3LnN5bWF1dGguY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZl
cmlzaWduLmNvbS9wY2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAY
BgNVBAMTEVZlcmlTaWduTVBLSS0yLTk3MB0GA1UdDgQWBBSt+cOTci21uShh5KTXYNXECl4aATCB
8QYDVR0jBIHpMIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIElu
Yy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZl
cmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWdu
IENsYXNzIDEgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgLSBHM4IRAItb
dVaEVIULAM+vOEjOsaQwDQYJKoZIhvcNAQEFBQADggEBANaPwdqbiPKzbE0fWC+6AVFddMFG6MO4
e5/WQPHv/zK6iWvADjRDn6SZ5qTwXUgzYoWFYf4jiCKMYJsrnGVJlMSiOCRIpVylUEto6WIip5Po
mSJuPVu7EEIOH0x1RzRWCY/4vYw881y70pZwVHBiTe/REL6dSCxe7IZrB4LwPeElJygs4BZ2HrP9
5WKW0oo9Xyuu+1zCE7dlY8s0dkOf1oeZq26tlcEAP0Yngf813iMOQ9wUXzL5yinvwlIw9ZnduYH4
OiUgjYJo8rkhhXRmBOGGORYy8i3WKqjJ3tkAAk/jGCDFpYFWtpXe04Kt+HslvmR8LqC6cCz4+XXi
dE0HbYQwgga9MIIFpaADAgECAhBcHtb173S3Zimo+RJj8ZJuMA0GCSqGSIb3DQEBBQUAMIGmMQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5bWFu
dGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UE
AxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHNDAeFw0xMjEy
MTcwMDAwMDBaFw0xMzEyMTgyMzU5NTlaMIHBMS4wLAYDVQQDDCVQZXJzb25hIE5vdCBWYWxpZGF0
ZWQgLSAxMzU1NzU2NDEyNzI0MR4wHAYJKoZIhvcNAQkBFg9sYXJzQG5ldGFwcC5jb20xDzANBgNV
BAsMBlMvTUlNRTEeMBwGA1UECwwVUGVyc29uYSBOb3QgVmFsaWRhdGVkMR8wHQYDVQQLDBZTeW1h
bnRlYyBUcnVzdCBOZXR3b3JrMR0wGwYDVQQKDBRTeW1hbnRlYyBDb3Jwb3JhdGlvbjCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBANWkuu7uQOPw8ji7+u/ZDX+P8pwLX3a57b+qK9jkhUeW
9D5cKKI/7uU2u3HVqunJSvfz/cm6BCT9PF9QMitgKydjnzQvnhJmjo30uufsIIVbkrMgBamPmDck
tdW2wJA/IHvg6siTrRRJ+PIDFtNsu0IPQMSjYLBJN7OvYBl8WC291uEB3PW1XN77/RxhBdd00nP8
xeJjiSol5xUassDP5/gAkTGp6YCxulqtkMN4AqRNnnW1cS3GaSKGVnD+4/znuzNqrY5eEL87miPz
SKg3XvSaoTRNMb1SrE0kI8a35T7zeopaQ3cjlZD4cVlsQe/A4nTwZXnUYYGKPsxt0j8y5ykCAwEA
AaOCAsgwggLEMAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgWgMCAGA1UdJQEB/wQWMBQGCCsG
AQUFBwMEBggrBgEFBQcDAjAdBgNVHQ4EFgQUPsgtqNipyEoJQEaE5C/M7ejkMGUwGgYDVR0RBBMw
EYEPbGFyc0BuZXRhcHAuY29tMB8GA1UdIwQYMBaAFK35w5NyLbW5KGHkpNdg1cQKXhoBMIIBKwYI
KwYBBQUHAQEEggEdMIIBGTCCARUGCCsGAQUFBzAChoIBB2xkYXA6Ly9kaXJlY3RvcnkudmVyaXNp
Z24uY29tL0NOJTIwJTNEJTIwU3ltYW50ZWMlMjBDbGFzcyUyMDElMjBJbmRpdmlkdWFsJTIwU3Vi
c2NyaWJlciUyMENBJTIwLSUyMEc0JTJDJTIwT1UlMjAlM0QlMjBQZXJzb25hJTIwTm90JTIwVmFs
aWRhdGVkJTJDJTIwT1UlMjAlM0QlMjBTeW1hbnRlYyUyMFRydXN0JTIwTmV0d29yayUyQyUyME8l
MjAlM0QlMjBTeW1hbnRlYyUyMENvcnBvcmF0aW9uJTJDJTIwQyUyMCUzRCUyMFVTP2NBQ2VydGlm
aWNhdGU7YmluYXJ5MF0GA1UdHwRWMFQwUqBQoE6GTGh0dHA6Ly9wa2ktY3JsLnN5bWF1dGguY29t
L2NhXzU2MWMxMDM2OTBjOTdhNjkyNDdhMGVmMDcxYWM4MWFmL0xhdGVzdENSTC5jcmwwbAYDVR0g
BGUwYzBhBgtghkgBhvhFAQcXATBSMCYGCCsGAQUFBwIBFhpodHRwOi8vd3d3LnN5bWF1dGguY29t
L2NwczAoBggrBgEFBQcCAjAcGhpodHRwOi8vd3d3LnN5bWF1dGguY29tL3JwYTAqBgpghkgBhvhF
ARADBBwwGgYRYIZIAYb4RQEQAQICBAGGsxcWBTEwOTIyMA0GCSqGSIb3DQEBBQUAA4IBAQAzvR5s
H2ex5C1+qykxtOOdO+VkD1uygAoUxdh7K67SvXqbTF6c5oOVI80EwbjbZKFUk90Jf3D6HxlN/O69
smO+gCVAe3S+vXRB3XUaNPciBnqqUHJpuTlJhcqMPWpG30ZifNBQC8acFuNBHsdQyZNnbUeYHFYK
Di0CayGxT9bagFoUNBmDOJ7kvnBATaMQhX6y0aVBf3fUU5RXr0NEITVlEozE/KZWRN+vfuufICT0
keHCtoXkwI2kzH9VW7eroQk7t8/wowK8z2gW5Hm8ZWl+fbsfFEFVmJ633Ey7GiCmTzovQKgSuV4m
a7cYD68kzWVk+9SnBdZILEfefjjrRvmuMYID5DCCA+ACAQEwgbswgaYxCzAJBgNVBAYTAlVTMR0w
GwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0
d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBD
bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc0AhBcHtb173S3Zimo+RJj8ZJuMAkG
BSsOAwIaBQCgggH9MBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEz
MTEwODIxMjE2MFowIwYJKoZIhvcNAQkEMRYEFGsjMWaZ/8YnB2LrhNT4soe+rpKSMIHMBgkrBgEE
AYI3EAQxgb4wgbswgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlv
bjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3Qg
VmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJl
ciBDQSAtIEc0AhBcHtb173S3Zimo+RJj8ZJuMIHOBgsqhkiG9w0BCRACCzGBvqCBuzCBpjELMAkG
A1UEBhMCVVMxHTAbBgNVBAoTFFN5bWFudGVjIENvcnBvcmF0aW9uMR8wHQYDVQQLExZTeW1hbnRl
YyBUcnVzdCBOZXR3b3JrMR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMT
LlN5bWFudGVjIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzQCEFwe1vXvdLdm
Kaj5EmPxkm4wDQYJKoZIhvcNAQEBBQAEggEAURa1GmP90Ns81CZCvYU9/jJFrqSyzvYGxC6u7MXt
MxWREXbsn3l0CkkcGbw4M+CQKqrTuoOBqVtYwSmdWa1QIeP7I7RhHPiQXpKLQp26rBIm3XQi7ogG
xiW+rMm+c9uGFh7pNC1HHqr4cCMAlOqiifdCLEhgzZfN/QWFftrqiLwUB+QPlwnMQte5m9HOtO9I
z/ksTgGc/zkOn3pj35zwMBqRDGdLymTNu2Dm7VDDCcXTVpCVgQZdGTe59tBElxaJZxbsfiKTMGFM
ivT/Jj3D6tPaHiTREX2891xusO2uDAfWNOvCqr1K2pBGLib/S7sUH74W7aE9dlxYoU4K0VLXoQAA
AAAAAA==

--Apple-Mail=_5CAC4D28-EE66-4799-8110-B1D0EB749B27--

From preethi.cis@gmail.com  Fri Nov  8 13:35:00 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E47711E80DE for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:35:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DBJtnAH7vaCP for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 13:34:57 -0800 (PST)
Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) by ietfa.amsl.com (Postfix) with ESMTP id BEE2F11E80F6 for <aqm@ietf.org>; Fri,  8 Nov 2013 13:34:57 -0800 (PST)
Received: by mail-pd0-f173.google.com with SMTP id r10so2661938pdi.18 for <aqm@ietf.org>; Fri, 08 Nov 2013 13:34:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=mgHlDd4fxK+WPSoOqnKOgqNsKiopnYkFlcqBGU6piKQ=; b=ou12DoixLuLlEjKrKegBhTr151aTxvhM0Ezw2lOOoprsyubgDP5sTgm2mCvs9aIybu pEJqqvxcJl+LimRxDrIIghN6J+Hit51LBr8kOLnfvc2gfseNcvvZ8ovRGgCtN4xPr2VD oP3UYU3xDNZF8cTgUfjwd1umghrRXEqilrNEutRgTFLknXlcPqrArja1V/v40OlDCNlJ z0sTX0XwOnBgeVRSqkJ2FMRQEisoSjab/xRvzrFNlLZUWXiz5OHEOhOFGGVHay0UZQaS b3S9hU/lLu2cuwWdcKiDsdIzGpG5HviV7PPoWkd1O62dl5Gt+2LLNJ7gXyEmsgw7vwa0 TscA==
X-Received: by 10.66.138.40 with SMTP id qn8mr8416343pab.154.1383946497508; Fri, 08 Nov 2013 13:34:57 -0800 (PST)
Received: from [10.33.22.215] (128-107-239-234.cisco.com. [128.107.239.234]) by mx.google.com with ESMTPSA id hz10sm14215996pbc.36.2013.11.08.13.34.55 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 08 Nov 2013 13:34:56 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Fri, 08 Nov 2013 13:34:53 -0700
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Michael Welzl <michawe@ifi.uio.no>
Message-ID: <CEA294E4.4AF38%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <40D660CC-6A01-43DD-AF72-EE0A1C4503F6@ifi.uio.no>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3466762496_11162096"
Cc: Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 21:35:00 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3466762496_11162096
Content-type: text/plain;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

Hello Michael, all:

Please see inline.

From:  Michael Welzl <michawe@ifi.uio.no>
Date:  Friday, November 8, 2013 2:11 PM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

>=20
> On 8. nov. 2013, at 11:00, Preethi Natarajan <preethi.cis@gmail.com> wrot=
e:
>=20
>> Hi Naeem, all:
>>=20
>> Please see detailed responses inline.
>>=20
>> From:  Naeem Khademi <naeem.khademi@gmail.com>
>> Date:  Thursday, November 7, 2013 10:50 PM
>> To:  Preethi Natarajan <preethi.cis@gmail.com>
>> Cc:  "aqm@ietf.org" <aqm@ietf.org>
>> Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based
>>=20
>>=20
>>=20
>>>=20
>>> Moreover, in a varying bandwidth scenario, depending on how often the
>>> bandwidth changes and at what granularity, the derived "estimated queui=
ng
>>> delay" by PIE might be an outdated information e.g. it shows how the ch=
annel
>>> has been (in term of delay) in the last 30ms and now how it is now! if =
I'm
>>> not mistaken that has been somehow resolved in DOCSIS 3.1 by coupling P=
IE's
>>> departure rate to the actual link layer rate.
>>=20
>> We've experimented with timestamp-based delay inputs to PIE. We have not=
 seen
>> any noticeable difference between the dequeue-based scheme and the
>> timestamp-based for all the scenarios Rong has presented at IETF thus fa=
r.
>> Even in extreme bandwidth changing conditions dequeue-based scheme can a=
dapt
>> within few tupdate cycles. Again, PIE is flexible in its delay input
>> mechanism and will go with whatever is the best possible option availabl=
e on
>> the platform. I don't think any latency based scheme can get better than=
 that
>> :).=20
>=20
> An AQM scheme should also work when the bandwidth is *not* changing. The
> results that Naeem presented in ICCRG don't indicate that PIE is consiste=
ntly
> better than CoDel (which is also delay-based). I'm not trying to say if t=
his
> or that is better, just that statements like this one are rather handwavy=
.

Just to clarify, my previous comment was w.r.t how queuing delay input to
PIE or any latency based AQM scheme can't get better than the fact that the
scheme should use the best possible queueing delay input available on the
platform =8B timestamp-based, dequeue-based or other flavors. There is nothin=
g
hand wavy in this statement.

Of course, an AQM scheme should also work when the bandwidth is *not*
changing. W.r.t the PIE results presented, there are few things we don't
understand. We plan to follow-up with you on that, maybe on a separate
thread.

=20
>=20
>=20
>>> In overall, while there are of course significant improvements made in =
PIE's
>>> (and (FQ)-CoDel's) design, there are probably still lessons to be learn=
ed
>>> from xRED family (check http://urn.nb.no/URN:NBN:no-38868 to see some
>>> examples). Throwing away all the knowledge and improvement gained from
>>> RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs without
>>> (experimentally) comparing them to the gallery of existing ones wouldn'=
t be
>>> great!
>>=20
>> Our initial discussions considered more than just the RED varieties. As =
we've
>> mentioned before, PIE is based on the concept of proportional integral (=
PI).
>> PI is not something we discovered. PI controller has been around for lon=
g and
>> you can refer to Hollot, Towsley et. al's paper from a decade ago for mo=
re
>> information on PI based AQM scheme. We believe PI's strong theoretical
>> stability is valuable for diverse deployment conditions and incorporated=
 PI
>> into our PIE design. We did our own set of theoretical analysis for PIE =
and
>> we were proved right. You can find more details about that in our HPSR p=
aper
>> (http://www.ieee-hpsr.org/2013/)
>=20
> Another handwavy statement. Using a PI controller may be a good design id=
ea,
> but that alone doesn't really tell us much about the overall behavior.

Again, I don't understand what is handwavy here. My comment was PI
controller has theoretical backing on why its good for various deployment
conditions. Here is the link to Hollot's paper I mentioned on PI design tha=
t
also talks in detail about the overall behavior --
ftp://ftp.cs.umass.edu/pub/net/pub/MisraInfocom01-AQM-Controller.pdf

Many thanks,
Preethi



--B_3466762496_11162096
Content-type: text/html;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:=
 14px; font-family: Calibri, sans-serif; "><div>Hello Michael, all:</div><di=
v><br></div><div>Please see inline.</div><div><br></div><span id=3D"OLK_SRC_BO=
DY_SECTION"><div style=3D"font-family:Calibri; font-size:11pt; text-align:left=
; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING=
-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt=
 solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style=3D"font-weigh=
t:bold">From: </span> Michael Welzl &lt;<a href=3D"mailto:michawe@ifi.uio.no">=
michawe@ifi.uio.no</a>&gt;<br><span style=3D"font-weight:bold">Date: </span> F=
riday, November 8, 2013 2:11 PM<br><span style=3D"font-weight:bold">To: </span=
> Preethi Natarajan &lt;<a href=3D"mailto:preethi.cis@gmail.com">preethi.cis@g=
mail.com</a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Naeem Khademi=
 &lt;<a href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt=
;, "<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>" &lt;<a href=3D"mailto:aqm@=
ietf.org">aqm@ietf.org</a>&gt;<br><span style=3D"font-weight:bold">Subject: </=
span> Re: [aqm] AQM schemes: Queue length vs. delay based<br></div><div><br>=
</div><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT=
: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;"><div><meta http-equiv=3D"=
Content-Type" content=3D"text/html charset=3Dwindows-1252"><div style=3D"word-wrap=
: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-spac=
e; "><br><div><div>On 8. nov. 2013, at 11:00, Preethi Natarajan &lt;<a href=3D=
"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt; wrote:</div><br=
 class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div style=3D"word-=
wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-=
space; font-size: 14px; font-family: Calibri, sans-serif; "><div>Hi Naeem, a=
ll:</div><div><br></div><div>Please see detailed responses inline.</div><div=
><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div style=3D"font-family: Calibri;=
 font-size: 11pt; text-align: left; border-width: 1pt medium medium; border-=
style: solid none none; padding: 3pt 0in 0in; border-top-color: rgb(181, 196=
, 223); "><span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a =
href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br><sp=
an style=3D"font-weight:bold">Date: </span> Thursday, November 7, 2013 10:50 P=
M<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a hre=
f=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span styl=
e=3D"font-weight:bold">Cc: </span> "<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org=
</a>" &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><span style=3D=
"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue length vs. =
delay based<br></div><div><br></div></span><div><br></div><div><br></div><sp=
an id=3D"OLK_SRC_BODY_SECTION"><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQU=
OTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" =
type=3D"cite"><div dir=3D"ltr"><div><br></div><div>Moreover, in a varying bandwi=
dth scenario, depending on how often the bandwidth changes and at what granu=
larity, the derived "estimated queuing delay" by PIE might be an outdated in=
formation e.g. it shows how the channel has been (in term of delay) in the l=
ast 30ms and now how it is now! if I'm not mistaken that has been somehow re=
solved in DOCSIS 3.1 by coupling PIE's departure rate to the actual link lay=
er rate. &nbsp;</div></div></blockquote></span><div><br></div><div>We've exp=
erimented with timestamp-based delay inputs to PIE. We have not seen any not=
iceable difference between the dequeue-based scheme and the timestamp-based =
for all the scenarios Rong has presented at IETF thus far. Even in extreme b=
andwidth changing conditions dequeue-based scheme can adapt within few tupda=
te cycles. Again, PIE is flexible in its delay input mechanism and will go w=
ith whatever is the best possible option available on the platform. I don't =
think any latency based scheme can get better than that :).&nbsp;</div></div=
></blockquote><div><br></div><div>An AQM scheme should also work when the ba=
ndwidth is *not* changing. The results that Naeem presented in ICCRG don't i=
ndicate that PIE is consistently better than CoDel (which is also delay-base=
d). I'm not trying to say if this or that is better, just that statements li=
ke this one are rather handwavy.</div></div></div></div></blockquote></span>=
<div><br></div><div>Just to clarify, my previous comment was w.r.t how queui=
ng delay input to PIE or any latency based AQM scheme can't get better than =
the fact that the scheme should use the best possible queueing delay input a=
vailable on the platform &#8212; timestamp-based, dequeue-based or other fla=
vors. There is nothing hand wavy in this statement.&nbsp;</div><div><br></di=
v><div>Of course, an AQM scheme should also work when the bandwidth is *not*=
 changing.&nbsp;W.r.t the PIE results presented, there are few things we don=
't understand. We plan to follow-up with you on that, maybe on a separate th=
read.</div><div><br></div><div>&nbsp;</div><span id=3D"OLK_SRC_BODY_SECTION"><=
blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4=
df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;"><div><div style=3D"word-wrap: br=
eak-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "=
><div><div><br></div><br><blockquote type=3D"cite"><div style=3D"word-wrap: brea=
k-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; fon=
t-size: 14px; font-family: Calibri, sans-serif; "><span id=3D"OLK_SRC_BODY_SEC=
TION"><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT=
: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div dir=3D"l=
tr"><div>In overall, while there are of course significant improvements made=
 in PIE's (and (FQ)-CoDel's) design, there are probably still lessons to be =
learned from xRED family (check&nbsp;<a href=3D"http://urn.nb.no/URN:NBN:no-38=
868" target=3D"_blank" style=3D"font-size:13px;font-family:arial,helvetica,sans-=
serif">http://urn.nb.no/URN:NBN:no-38868</a>&nbsp;to see some examples). Thr=
owing away all the knowledge and improvement gained from RED-like AQMs, CHoK=
e, etc and coming up with brand-new AQMs without (experimentally) comparing =
them to the gallery of existing ones wouldn't be great!</div></div></blockqu=
ote></span><div><br></div><div>Our initial discussions considered more than =
just the RED varieties. As we've mentioned before, PIE is based on the conce=
pt of proportional integral (PI). PI is not something we discovered. PI cont=
roller has been around for long and you can refer to Hollot, Towsley et. al'=
s paper from a decade ago for more information on PI based AQM scheme. We be=
lieve PI's strong theoretical stability is valuable for diverse deployment c=
onditions and incorporated PI into our PIE design. We did our own set of the=
oretical analysis for PIE and we were proved right. You can find more detail=
s about that in our HPSR paper (<a href=3D"http://www.ieee-hpsr.org/2013">http=
://www.ieee-hpsr.org/2013</a>/)</div></div></blockquote><div><br></div></div=
>Another handwavy statement. Using a PI controller may be a good design idea=
, but that alone doesn't really tell us much about the overall behavior.</di=
v></div></blockquote></span><div><br></div><div>Again, I don't understand wh=
at is handwavy here. My comment was PI controller has theoretical backing on=
 why its good for various deployment conditions. Here is the link to Hollot'=
s paper I mentioned on PI design that also talks in detail about the overall=
 behavior --&nbsp;<span class=3D"Apple-style-span" style=3D"font-family: Helveti=
ca; font-size: 16px; "><a href=3D"ftp://ftp.cs.umass.edu/pub/net/pub/MisraInfo=
com01-AQM-Controller.pdf" style=3D"word-wrap: break-word; font: inherit; ">ftp=
://ftp.cs.umass.edu/pub/net/pub/MisraInfocom01-AQM-Controller.pdf</a></span>=
</div><div><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div><div style=3D"word-w=
rap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-s=
pace; "><div>Many thanks,</div></div></div></span><div>Preethi</div></body><=
/html>

--B_3466762496_11162096--



From g.white@CableLabs.com  Fri Nov  8 14:43:01 2013
Return-Path: <g.white@CableLabs.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8B1911E80FB for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 14:43:01 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.236
X-Spam-Level: 
X-Spam-Status: No, score=-0.236 tagged_above=-999 required=5 tests=[AWL=0.000,  BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kvlWg1RwW0JK for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 14:42:56 -0800 (PST)
Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 6EDE121E809B for <aqm@ietf.org>; Fri,  8 Nov 2013 14:42:56 -0800 (PST)
Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id rA8MgqIO007760; Fri, 8 Nov 2013 15:42:52 -0700
Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 08 Nov 2013 15:42:51 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com)
Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Fri, 8 Nov 2013 15:42:51 -0700
From: Greg White <g.white@CableLabs.com>
To: "Eggert, Lars" <lars@netapp.com>
Thread-Topic: [aqm] PIE & CableLabs
Thread-Index: AQHO3MAAFFH8PuCBIk6ZSa6cse1IYpobvoAAgACfOoD//3+1gA==
Date: Fri, 8 Nov 2013 22:42:50 +0000
Message-ID: <CEA2A75B.21C82%g.white@cablelabs.com>
In-Reply-To: <17BA4215-B855-4424-9D8C-FCFE11A1849E@netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.5.0.27]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <79A40A3E3A414C488A28B595328657BE@cablelabs.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Approved: ondar
Cc: "aqm@ietf.org" <aqm@ietf.org>, Ralph Droms <rdroms.ietf@gmail.com>
Subject: Re: [aqm] PIE & CableLabs
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2013 22:43:02 -0000

On 11/8/13, 2:22 PM, "Eggert, Lars" <lars@netapp.com> wrote:

>One remaining question is what will happen if the AQM WG - after adopting
>the PIE contribution - makes modifications. Is CableLabs prepared to
>update their specs based on whet we'll do, referring to an eventual IETF
>RFC rather than maintaining a separate spec? Or will CableLabs also
>continue work on the spec and we may end up with eventual divergence? (Or
>will the CableLabs spec of PIE remain frozen at its current version?)

We would like convergence as well.  Our intent is that the algorithm can
be tweaked after deployment via software update, but there will of course
be limitations to how much can be changed.  Whether improvements come from
within IETF community or CableLabs community, we'll aim for maintaining
alignment.  That may mean that we'll push back if changes can't be
accommodated in  our networks, or if IPR issues arise.  Assuming that it
is feasible to maintain alignment with an eventual IETF RFC, we would
strongly consider updating our specification to point to it (I think that
would be a good outcome).


>Having asked that, I want to make clear that the working relationship
>with CableLabs has been good, and it's not that I'm expecting problems
>here. It's after all - at least at the moment - the same people working
>on PIE here and there. It's just that I've been involved in cases in the
>IETF before where things fell through the cracks when two SDOs
>simultaneously worked on something, and it's messy to create alignment
>after that has happened. So an explicit agreement about how this will go
>forward would be good to have.
>
>Thanks,
>Lars
>
>


From michawe@ifi.uio.no  Fri Nov  8 17:31:01 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 126FA11E80E3 for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 17:31:01 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.371
X-Spam-Level: 
X-Spam-Status: No, score=-102.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Aq0iBw-v1ovk for <aqm@ietfa.amsl.com>; Fri,  8 Nov 2013 17:30:56 -0800 (PST)
Received: from mail-out1.uio.no (mail-out1.uio.no [IPv6:2001:700:100:10::57]) by ietfa.amsl.com (Postfix) with ESMTP id 3534111E80E2 for <aqm@ietf.org>; Fri,  8 Nov 2013 17:30:56 -0800 (PST)
Received: from mail-mx2.uio.no ([129.240.10.30]) by mail-out1.uio.no with esmtp (Exim 4.75) (envelope-from <michawe@ifi.uio.no>) id 1VexO2-0001ky-5I; Sat, 09 Nov 2013 02:30:54 +0100
Received: from 209-82-80-116.dedicated.allstream.net ([209.82.80.116] helo=[10.199.1.120]) by mail-mx2.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from <michawe@ifi.uio.no>) id 1VexO0-0004XF-Gm; Sat, 09 Nov 2013 02:30:54 +0100
Content-Type: multipart/alternative; boundary="Apple-Mail=_7B9B7BF4-3841-48B3-AC86-EFA309F9E363"
Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\))
From: Michael Welzl <michawe@ifi.uio.no>
In-Reply-To: <CEA294E4.4AF38%prenatar@cisco.com>
Date: Fri, 8 Nov 2013 17:30:49 -0800
Message-Id: <E47DE53D-DEFE-4203-A239-C7615E384E65@ifi.uio.no>
References: <CEA294E4.4AF38%prenatar@cisco.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
X-Mailer: Apple Mail (2.1510)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 3 msgs/h 1 sum rcpts/h 4 sum msgs/h 1 total rcpts 9623 max rcpts/h 44 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: CFD6AD13F107409B21CB676B682C529B9D4BE5EF
X-UiO-SPAM-Test: remote_host: 209.82.80.116 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 1 max/h 1 blacklist 0 greylist 0 ratelimit 0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Nov 2013 01:31:01 -0000

--Apple-Mail=_7B9B7BF4-3841-48B3-AC86-EFA309F9E363
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

On 8. nov. 2013, at 12:34, Preethi Natarajan <preethi.cis@gmail.com> =
wrote:

> Hello Michael, all:
>=20
> Please see inline.
>=20
> From: Michael Welzl <michawe@ifi.uio.no>
> Date: Friday, November 8, 2013 2:11 PM
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" =
<aqm@ietf.org>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>=20
>>=20
>> On 8. nov. 2013, at 11:00, Preethi Natarajan <preethi.cis@gmail.com> =
wrote:
>>=20
>>> Hi Naeem, all:
>>>=20
>>> Please see detailed responses inline.
>>>=20
>>> From: Naeem Khademi <naeem.khademi@gmail.com>
>>> Date: Thursday, November 7, 2013 10:50 PM
>>> To: Preethi Natarajan <preethi.cis@gmail.com>
>>> Cc: "aqm@ietf.org" <aqm@ietf.org>
>>> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>>>=20
>>>=20
>>>=20
>>>>=20
>>>> Moreover, in a varying bandwidth scenario, depending on how often =
the bandwidth changes and at what granularity, the derived "estimated =
queuing delay" by PIE might be an outdated information e.g. it shows how =
the channel has been (in term of delay) in the last 30ms and now how it =
is now! if I'm not mistaken that has been somehow resolved in DOCSIS 3.1 =
by coupling PIE's departure rate to the actual link layer rate. =20
>>>=20
>>>=20
>>> We've experimented with timestamp-based delay inputs to PIE. We have =
not seen any noticeable difference between the dequeue-based scheme and =
the timestamp-based for all the scenarios Rong has presented at IETF =
thus far. Even in extreme bandwidth changing conditions dequeue-based =
scheme can adapt within few tupdate cycles. Again, PIE is flexible in =
its delay input mechanism and will go with whatever is the best possible =
option available on the platform. I don't think any latency based scheme =
can get better than that :).=20
>>=20
>> An AQM scheme should also work when the bandwidth is *not* changing. =
The results that Naeem presented in ICCRG don't indicate that PIE is =
consistently better than CoDel (which is also delay-based). I'm not =
trying to say if this or that is better, just that statements like this =
one are rather handwavy.
>=20
>=20
> Just to clarify, my previous comment was w.r.t how queuing delay input =
to PIE or any latency based AQM scheme can't get better than the fact =
that the scheme should use the best possible queueing delay input =
available on the platform =97 timestamp-based, dequeue-based or other =
flavors. There is nothing hand wavy in this statement.=20

ok, sorry -


> Of course, an AQM scheme should also work when the bandwidth is *not* =
changing. W.r.t the PIE results presented, there are few things we don't =
understand. We plan to follow-up with you on that, maybe on a separate =
thread.

>>>> In overall, while there are of course significant improvements made =
in PIE's (and (FQ)-CoDel's) design, there are probably still lessons to =
be learned from xRED family (check http://urn.nb.no/URN:NBN:no-38868 to =
see some examples). Throwing away all the knowledge and improvement =
gained from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs =
without (experimentally) comparing them to the gallery of existing ones =
wouldn't be great!
>>>=20
>>>=20
>>> Our initial discussions considered more than just the RED varieties. =
As we've mentioned before, PIE is based on the concept of proportional =
integral (PI). PI is not something we discovered. PI controller has been =
around for long and you can refer to Hollot, Towsley et. al's paper from =
a decade ago for more information on PI based AQM scheme. We believe =
PI's strong theoretical stability is valuable for diverse deployment =
conditions and incorporated PI into our PIE design. We did our own set =
of theoretical analysis for PIE and we were proved right. You can find =
more details about that in our HPSR paper =
(http://www.ieee-hpsr.org/2013/)
>>=20
>> Another handwavy statement. Using a PI controller may be a good =
design idea, but that alone doesn't really tell us much about the =
overall behavior.
>=20
>=20
> Again, I don't understand what is handwavy here. My comment was PI =
controller has theoretical backing on why its good for various =
deployment conditions. Here is the link to Hollot's paper I mentioned on =
PI design that also talks in detail about the overall behavior -- =
ftp://ftp.cs.umass.edu/pub/net/pub/MisraInfocom01-AQM-Controller.pdf

Thanks for the link. My point was just (along with Naeem) that it would =
be useful to compare PIE (or CoDel) against something else except CoDel =
and RED. When I just look at the graphs in the HPSR paper (sorry, didn't =
get to read it yet), again I see only PIE, CoDel, RED.

We tried ARED and were surprised to see how good it works - which is not =
to say that it is the ideal - but it was surprisingly often better than =
these newer schemes, despite its age of 10+ years... as we all know, =
there are many schemes out there, quite often designed with the goal of =
removing RED's dependence on parameters.

Cheers,
Michael


--Apple-Mail=_7B9B7BF4-3841-48B3-AC86-EFA309F9E363
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On 8. nov. 2013, at 12:34, Preethi Natarajan &lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; font-size: 14px; =
font-family: Calibri, sans-serif; "><div>Hello Michael, =
all:</div><div><br></div><div>Please see =
inline.</div><div><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div =
style=3D"font-family: Calibri; font-size: 11pt; text-align: left; =
border-width: 1pt medium medium; border-style: solid none none; padding: =
3pt 0in 0in; border-top-color: rgb(181, 196, 223); "><span =
style=3D"font-weight:bold">From: </span> Michael Welzl &lt;<a =
href=3D"mailto:michawe@ifi.uio.no">michawe@ifi.uio.no</a>&gt;<br><span =
style=3D"font-weight:bold">Date: </span> Friday, November 8, 2013 2:11 =
PM<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan =
&lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><sp=
an style=3D"font-weight:bold">Cc: </span> Naeem Khademi &lt;<a =
href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;, =
"<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>" &lt;<a =
href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><span =
style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue =
length vs. delay based<br></div><div><br></div><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><meta =
http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"><div style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On 8. nov. 2013, at 11:00, Preethi Natarajan &lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; font-size: 14px; =
font-family: Calibri, sans-serif; "><div>Hi Naeem, =
all:</div><div><br></div><div>Please see detailed responses =
inline.</div><div><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div =
style=3D"font-family: Calibri; font-size: 11pt; text-align: left; =
border-width: 1pt medium medium; border-style: solid none none; padding: =
3pt 0in 0in; border-top-color: rgb(181, 196, 223); "><span =
style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a =
href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br=
><span style=3D"font-weight:bold">Date: </span> Thursday, November 7, =
2013 10:50 PM<br><span style=3D"font-weight:bold">To: </span> Preethi =
Natarajan &lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><sp=
an style=3D"font-weight:bold">Cc: </span> "<a =
href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>" &lt;<a =
href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><span =
style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue =
length vs. delay =
based<br></div><div><br></div></span><div><br></div><div><br></div><span =
id=3D"OLK_SRC_BODY_SECTION"><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
dir=3D"ltr"><div><br></div><div>Moreover, in a varying bandwidth =
scenario, depending on how often the bandwidth changes and at what =
granularity, the derived "estimated queuing delay" by PIE might be an =
outdated information e.g. it shows how the channel has been (in term of =
delay) in the last 30ms and now how it is now! if I'm not mistaken that =
has been somehow resolved in DOCSIS 3.1 by coupling PIE's departure rate =
to the actual link layer rate. =
&nbsp;</div></div></blockquote></span><div><br></div><div>We've =
experimented with timestamp-based delay inputs to PIE. We have not seen =
any noticeable difference between the dequeue-based scheme and the =
timestamp-based for all the scenarios Rong has presented at IETF thus =
far. Even in extreme bandwidth changing conditions dequeue-based scheme =
can adapt within few tupdate cycles. Again, PIE is flexible in its delay =
input mechanism and will go with whatever is the best possible option =
available on the platform. I don't think any latency based scheme can =
get better than that =
:).&nbsp;</div></div></blockquote><div><br></div><div>An AQM scheme =
should also work when the bandwidth is *not* changing. The results that =
Naeem presented in ICCRG don't indicate that PIE is consistently better =
than CoDel (which is also delay-based). I'm not trying to say if this or =
that is better, just that statements like this one are rather =
handwavy.</div></div></div></blockquote></span><div><br></div><div>Just =
to clarify, my previous comment was w.r.t how queuing delay input to PIE =
or any latency based AQM scheme can't get better than the fact that the =
scheme should use the best possible queueing delay input available on =
the platform =97 timestamp-based, dequeue-based or other flavors. There =
is nothing hand wavy in this =
statement.&nbsp;</div></div></blockquote><div><br></div>ok, sorry =
-</div><div><br></div><div><br><blockquote type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 14px; font-family: =
Calibri, sans-serif; "><div>Of course, an AQM scheme should also work =
when the bandwidth is *not* changing.&nbsp;W.r.t the PIE results =
presented, there are few things we don't understand. We plan to =
follow-up with you on that, maybe on a separate =
thread.</div></div></blockquote><div><br></div><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; font-size: 14px; =
font-family: Calibri, sans-serif; "><span =
id=3D"OLK_SRC_BODY_SECTION"><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; "><div><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; font-size: 14px; =
font-family: Calibri, sans-serif; "><span =
id=3D"OLK_SRC_BODY_SECTION"><blockquote =
id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df =
5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" type=3D"cite"><div =
dir=3D"ltr">In overall, while there are of course significant =
improvements made in PIE's (and (FQ)-CoDel's) design, there are probably =
still lessons to be learned from xRED family (check&nbsp;<a =
href=3D"http://urn.nb.no/URN:NBN:no-38868" target=3D"_blank" =
style=3D"font-size:13px;font-family:arial,helvetica,sans-serif">http://urn=
.nb.no/URN:NBN:no-38868</a>&nbsp;to see some examples). Throwing away =
all the knowledge and improvement gained from RED-like AQMs, CHoKe, etc =
and coming up with brand-new AQMs without (experimentally) comparing =
them to the gallery of existing ones wouldn't be =
great!</div></blockquote></span><div><br></div><div>Our initial =
discussions considered more than just the RED varieties. As we've =
mentioned before, PIE is based on the concept of proportional integral =
(PI). PI is not something we discovered. PI controller has been around =
for long and you can refer to Hollot, Towsley et. al's paper from a =
decade ago for more information on PI based AQM scheme. We believe PI's =
strong theoretical stability is valuable for diverse deployment =
conditions and incorporated PI into our PIE design. We did our own set =
of theoretical analysis for PIE and we were proved right. You can find =
more details about that in our HPSR paper (<a =
href=3D"http://www.ieee-hpsr.org/2013">http://www.ieee-hpsr.org/2013</a>/)=
</div></div></blockquote><div><br></div></div>Another handwavy =
statement. Using a PI controller may be a good design idea, but that =
alone doesn't really tell us much about the overall =
behavior.</div></blockquote></span><div><br></div><div>Again, I don't =
understand what is handwavy here. My comment was PI controller has =
theoretical backing on why its good for various deployment conditions. =
Here is the link to Hollot's paper I mentioned on PI design that also =
talks in detail about the overall behavior --&nbsp;<span =
class=3D"Apple-style-span" style=3D"font-size: 16px; "><a =
href=3D"ftp://ftp.cs.umass.edu/pub/net/pub/MisraInfocom01-AQM-Controller.p=
df" style=3D"word-wrap: break-word; font: inherit; =
">ftp://ftp.cs.umass.edu/pub/net/pub/MisraInfocom01-AQM-Controller.pdf</a>=
</span></div></div></blockquote><div><br></div>Thanks for the link. My =
point was just (along with Naeem) that it would be useful to compare PIE =
(or CoDel) against something else except CoDel and RED. When I just look =
at the graphs in the HPSR paper (sorry, didn't get to read it yet), =
again I see only PIE, CoDel, RED.</div><div><br></div><div>We tried ARED =
and were surprised to see how good it works - which is not to say that =
it is the ideal - but it was surprisingly often better than these newer =
schemes, despite its age of 10+ years... as we all know, there are many =
schemes out there, quite often designed with the goal of removing RED's =
dependence on =
parameters.</div><div><br></div><div>Cheers,</div><div>Michael</div><div><=
br></div></body></html>=

--Apple-Mail=_7B9B7BF4-3841-48B3-AC86-EFA309F9E363--

From g.white@CableLabs.com  Fri Nov  8 22:54:28 2013
Return-Path: <g.white@CableLabs.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A28411E813A; Fri,  8 Nov 2013 22:54:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.236
X-Spam-Level: 
X-Spam-Status: No, score=-0.236 tagged_above=-999 required=5 tests=[AWL=0.000,  BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vCAe85yhVPLt; Fri,  8 Nov 2013 22:54:23 -0800 (PST)
Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id B0CB111E80E7; Fri,  8 Nov 2013 22:54:23 -0800 (PST)
Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id rA96sJdF008677; Fri, 8 Nov 2013 23:54:19 -0700
Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 08 Nov 2013 23:54:18 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com)
Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Fri, 8 Nov 2013 23:54:19 -0700
From: Greg White <g.white@CableLabs.com>
To: Bob Briscoe <bob.briscoe@bt.com>, tsv-area IETF list <tsv-area@ietf.org>,  AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Thread-Topic: [tsvwg] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO2/R7zNZxx+pCdEmOwIOO+63dlZocaFeA
Date: Sat, 9 Nov 2013 06:54:18 +0000
Message-ID: <CEA31AEB.21D9B%g.white@cablelabs.com>
In-Reply-To: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.5.0.27]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <4D0B19D98793114197CBFB5D06EF689F@cablelabs.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Approved: ondar
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Nov 2013 06:54:28 -0000

This is very interesting work.  There are a lot of unanswered questions
about ecn / no-ecn coexistence and differential treatment in an AQM, and
this could provide some answers.

To those who groaned that ECN was not included in DOCSIS 3.1, read these
slides (and Naeem Khademi's).

Bob, CoDel uses "interval" both as a hold-off for the first packet drop
and as the numerator in the invsqrt drop scheduling.  Setting interval =3D =
0
would result in ECN being signaled on *every* ECN capable packet when the
sojourn time is above threshold.    This jibes with some of your charts
for RED, but others show a ramp up in mark probability rather than a step
function. Could you clarify?

Setting max_burst =3D 0 in PIE would not result in the step function
behavior.

-Greg


On 11/7/13, 1:03 PM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:

>Folks,
>
>"Immediate ECN" slides:
><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
>
>PS. This talk fell off the end of the TSVAREA agenda. It's mostly
>relevant to AQM, but I didn't originally bring it to AQM, because it
>affects 3 wgs: tsvwg, aqm & tcpm.
>
>In the AQM wg, there was dismay about CableLabs not including
>anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and
>how ECN can take out the 100ms of delay that CoDel and PIE introduce
>- it's essentially about auto-tuning for RTT.
>
>It gives an interim recommendation for hardware designers that there
>should be a second instance of the AQM algo for ECN packets so that
>it can be configured with different parameters (think of WRED instead of
>RED).
>
>Specifically, for ECN packets:
>interval =3D 0 (for CoDel)
>max_burst =3D 0 (for PIE)
>
>
>Bob
>
>PS. We have a paper under submission, which we can supply on request.
>We plan to document this in the IETF too.
>
>
>
>
>________________________________________________________________
>Bob Briscoe,                                                  BT=20


From ropan@cisco.com  Fri Nov  8 23:27:01 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9B87A21E80F5; Fri,  8 Nov 2013 23:27:01 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.485
X-Spam-Level: 
X-Spam-Status: No, score=-10.485 tagged_above=-999 required=5 tests=[AWL=-0.113, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XQ+O8WZ1c+pT; Fri,  8 Nov 2013 23:26:57 -0800 (PST)
Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) by ietfa.amsl.com (Postfix) with ESMTP id 398F221E80F7; Fri,  8 Nov 2013 23:26:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2169; q=dns/txt; s=iport; t=1383982004; x=1385191604; h=from:to:subject:date:message-id:in-reply-to:content-id: content-transfer-encoding:mime-version; bh=/vpEnBrsNnzpJqprwmMdgr2Q7HT/AEkMVf9UhLZlggc=; b=CFfy2kNfHWh4G64KGlcSkZBHQyeBhNW5KnwmxTvYwt0wnHHQxGyrb5yn U94fzZDkRHlZJCkO0jaqrHJ2p9yNfWkpQhD1YolBCtV0ZplPhlHdhZ8C0 1WFfIesQvrlMDJ6pdN8fJInjPlLxd1mcge4bd70fHX1KGaM74NkdB+KRi g=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgkFABjjfVKtJXHB/2dsb2JhbABZgwc4U75QSoEqFnSCJwECAgEBATc0HQEINjcLJQIEARKIAQ29B49uhDADmA+BL5BbgyaCKg
X-IronPort-AV: E=Sophos;i="4.93,666,1378857600"; d="scan'208";a="282819352"
Received: from rcdn-core2-6.cisco.com ([173.37.113.193]) by rcdn-iport-4.cisco.com with ESMTP; 09 Nov 2013 07:26:40 +0000
Received: from xhc-aln-x07.cisco.com (xhc-aln-x07.cisco.com [173.36.12.81]) by rcdn-core2-6.cisco.com (8.14.5/8.14.5) with ESMTP id rA97QegQ027898 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sat, 9 Nov 2013 07:26:40 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.147]) by xhc-aln-x07.cisco.com ([173.36.12.81]) with mapi id 14.03.0123.003; Sat, 9 Nov 2013 01:26:40 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: Bob Briscoe <bob.briscoe@bt.com>, tsv-area IETF list <tsv-area@ietf.org>,  AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Thread-Topic: [aqm] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO2/SBttchdmU52kqbSkB491ElhpocT9oA
Date: Sat, 9 Nov 2013 07:26:39 +0000
Message-ID: <CEA32194.550FE%ropan@cisco.com>
In-Reply-To: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [10.21.114.54]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <33D401EC59EEEF4D98A6A1BA5C6A87CD@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [aqm] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Nov 2013 07:27:01 -0000

I just got a better understanding of Bob's proposal. Great direction
indeed! I fully agreed that doing the smoothing at the end hosts and
simple network congestion signals is the best way to achieve low averaging
latency.=20

We do need these three working groups working together to get this done.
But since folks usually attend all these three meetings at IETF, I am
hopeful that IETF can make it happen with all our efforts together here.

Bob,=20

a technical question. If we do no averaging for ECN, and hosts use the
tradition ECN-capable TCP, I am afraid its window might get cut too much
to achieve any reasonable throughput. Any thought on how to handle that?
Or maybe you already covered in your slides and I missed it?

Thanks,

Rong



On 11/7/13 12:03 PM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:

>Folks,
>
>"Immediate ECN" slides:
><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
>
>PS. This talk fell off the end of the TSVAREA agenda. It's mostly
>relevant to AQM, but I didn't originally bring it to AQM, because it
>affects 3 wgs: tsvwg, aqm & tcpm.
>
>In the AQM wg, there was dismay about CableLabs not including
>anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and
>how ECN can take out the 100ms of delay that CoDel and PIE introduce
>- it's essentially about auto-tuning for RTT.
>
>It gives an interim recommendation for hardware designers that there
>should be a second instance of the AQM algo for ECN packets so that
>it can be configured with different parameters (think of WRED instead of
>RED).
>
>Specifically, for ECN packets:
>interval =3D 0 (for CoDel)
>max_burst =3D 0 (for PIE)
>
>
>Bob
>
>PS. We have a paper under submission, which we can supply on request.
>We plan to document this in the IETF too.
>
>
>
>
>________________________________________________________________
>Bob Briscoe,                                                  BT
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm


From bob.briscoe@bt.com  Mon Nov 11 08:18:24 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 69DB521E8064; Mon, 11 Nov 2013 08:18:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.748
X-Spam-Level: 
X-Spam-Status: No, score=-2.748 tagged_above=-999 required=5 tests=[AWL=-0.376, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5mbiYKnv2su4; Mon, 11 Nov 2013 08:18:19 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id 2EC5A11E8122; Mon, 11 Nov 2013 08:18:18 -0800 (PST)
Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR65-UKRD.bt.com (10.187.101.20) with Microsoft SMTP Server (TLS) id 14.3.158.1; Mon, 11 Nov 2013 16:18:16 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 11 Nov 2013 16:18:16 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Mon, 11 Nov 2013 16:18:09 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.171.204])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rABGI8D4031128; Mon, 11 Nov 2013 16:18:09 GMT
Message-ID: <201311111618.rABGI8D4031128@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Mon, 11 Nov 2013 16:17:59 +0000
To: "Rong Pan (ropan)" <ropan@cisco.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CEA32194.550FE%ropan@cisco.com>
References: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk> <CEA32194.550FE%ropan@cisco.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2013 16:18:24 -0000

Rong,

At 07:26 09/11/2013, Rong Pan (ropan) wrote:
>I just got a better understanding of Bob's proposal. Great direction
>indeed! I fully agreed that doing the smoothing at the end hosts and
>simple network congestion signals is the best way to achieve low averaging
>latency.
>
>We do need these three working groups working together to get this done.
>But since folks usually attend all these three meetings at IETF, I am
>hopeful that IETF can make it happen with all our efforts together here.

It seems the IETF is willing to move quickly on this. We brought this 
work to the IETF earlier than I would normally have liked, given the 
urgency. We will try continue to move quickly, but it would be great 
if different groups could work in parallel. For instance, if your 
group and/or others could focus on testing this out with PIE.

more...


>Bob,
>
>a technical question. If we do no averaging for ECN, and hosts use the
>tradition ECN-capable TCP, I am afraid its window might get cut too much
>to achieve any reasonable throughput. Any thought on how to handle that?
>Or maybe you already covered in your slides and I missed it?

Bear in mind that altho traditional ECN hosts (we call them classic 
ECN) cut their rate by half in response to one CE mark they only do 
this once per RTT, then they ignore all the other marks in the same 
RTT. DCTCP only cuts its rate a little per mark, but it does so for 
every mark.

So it's not obvious what the outcome will be, but I'm hopeful that 
classic ECN hosts will achieve sufficient througput when competing 
with DCTCP, for some definition of 'sufficient'. Testing incremental 
deployment cases like this is next on our list of experiments to do.

I intended to say this on slide 13 (there's a brief footnote to this 
effect), but I had to cut short, because I had to squeeze a 20min 
presentation into the last 15 mins of tsvwg.

Regards


Bob


>Thanks,
>
>Rong
>
>
>
>On 11/7/13 12:03 PM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:
>
> >Folks,
> >
> >"Immediate ECN" slides:
> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
> >
> >PS. This talk fell off the end of the TSVAREA agenda. It's mostly
> >relevant to AQM, but I didn't originally bring it to AQM, because it
> >affects 3 wgs: tsvwg, aqm & tcpm.
> >
> >In the AQM wg, there was dismay about CableLabs not including
> >anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and
> >how ECN can take out the 100ms of delay that CoDel and PIE introduce
> >- it's essentially about auto-tuning for RTT.
> >
> >It gives an interim recommendation for hardware designers that there
> >should be a second instance of the AQM algo for ECN packets so that
> >it can be configured with different parameters (think of WRED instead of
> >RED).
> >
> >Specifically, for ECN packets:
> >interval = 0 (for CoDel)
> >max_burst = 0 (for PIE)
> >
> >
> >Bob
> >
> >PS. We have a paper under submission, which we can supply on request.
> >We plan to document this in the IETF too.
> >
> >
> >
> >
> >________________________________________________________________
> >Bob Briscoe,                                                  BT
> >
> >_______________________________________________
> >aqm mailing list
> >aqm@ietf.org
> >https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 


From bob.briscoe@bt.com  Mon Nov 11 08:18:34 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF68521E81E8; Mon, 11 Nov 2013 08:18:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.238
X-Spam-Level: 
X-Spam-Status: No, score=-3.238 tagged_above=-999 required=5 tests=[AWL=0.134,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rl-sd5K6thnZ; Mon, 11 Nov 2013 08:18:29 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 627CF21E81D6; Mon, 11 Nov 2013 08:18:28 -0800 (PST)
Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR68-UKRD.bt.com (10.187.101.23) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 11 Nov 2013 16:18:26 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 11 Nov 2013 16:18:25 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Mon, 11 Nov 2013 16:18:22 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.111.171.204])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rABGILbC031136; Mon, 11 Nov 2013 16:18:21 GMT
Message-ID: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Mon, 11 Nov 2013 16:02:36 +0000
To: Greg White <g.white@CableLabs.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CEA31AEB.21D9B%g.white@cablelabs.com>
References: <201311072003.rA7K38dj008566@bagheera.jungle.bt.co.uk> <CEA31AEB.21D9B%g.white@cablelabs.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2013 16:18:35 -0000

Greg,

At 06:54 09/11/2013, Greg White wrote:
>This is very interesting work.  There are a lot of unanswered questions
>about ecn / no-ecn coexistence and differential treatment in an AQM, and
>this could provide some answers.
>
>To those who groaned that ECN was not included in DOCSIS 3.1, read these
>slides (and Naeem Khademi's).

Indeed. However, I think it would be safe to recommend that ECN 
support should at least be implemented and separately configurable, 
then it can be turned on or off by operators later.

I'm aware that this doubles the amount of configuration, but we've 
had some success already (with RED) in relating all the ECN 
parameters to the drop parameters by a formula, so hopefully the 
vendor could configure the ECN parameters automatically based on the 
drop parameters.


>Bob, CoDel uses "interval" both as a hold-off for the first packet drop
>and as the numerator in the invsqrt drop scheduling.  Setting interval = 0
>would result in ECN being signaled on *every* ECN capable packet when the
>sojourn time is above threshold.    This jibes with some of your charts
>for RED, but others show a ramp up in mark probability rather than a step
>function. Could you clarify?

We've only looked at WRED in detail, because it was much more 
interesting (to us) to reconfigure existing implementations than have 
to wait for new code to be implemented and tested.

The suggestions for PIE and CoDel are just conceptual at this stage - 
we've done no implementation of this idea with either. (I said this 
verbally when presenting the slides, but I should have put it in 
writing too). Please read my suggestions for PIE and CoDel in this light.

I'm not surprised that CoDel derives other parameters from 'interval' 
that should have been declared and set separately. Andrew McGregor 
also pointed out to me that CoDel sets threshold = 0.2*interval, so 
threshold would have to be declared separately as well. This starts 
to reveal just how many magic numbers there really are in the CoDel algorithm.


>Setting max_burst = 0 in PIE would not result in the step function
>behavior.

It's not meant to result in a step-function.

In the WRED example, it solely avoids the delay of queue averaging, 
so that once the /instantaneous/ queue exceeds min_thresh it marks 
with increasing probability (not intended to be a step).

Similarly with PIE, the formula:
         p = p + alpha*(est_del-target_del) + beta*(est_del-est_del_old);
would still gradually increase the probability of drop (not a step 
function), but it would start to do so as soon as the queue exceeded 
target_del, rather than waiting for max_burst.

Is that what you meant?


Bob


>-Greg
>
>
>On 11/7/13, 1:03 PM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:
>
> >Folks,
> >
> >"Immediate ECN" slides:
> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
> >
> >PS. This talk fell off the end of the TSVAREA agenda. It's mostly
> >relevant to AQM, but I didn't originally bring it to AQM, because it
> >affects 3 wgs: tsvwg, aqm & tcpm.
> >
> >In the AQM wg, there was dismay about CableLabs not including
> >anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and
> >how ECN can take out the 100ms of delay that CoDel and PIE introduce
> >- it's essentially about auto-tuning for RTT.
> >
> >It gives an interim recommendation for hardware designers that there
> >should be a second instance of the AQM algo for ECN packets so that
> >it can be configured with different parameters (think of WRED instead of
> >RED).
> >
> >Specifically, for ECN packets:
> >interval = 0 (for CoDel)
> >max_burst = 0 (for PIE)
> >
> >
> >Bob
> >
> >PS. We have a paper under submission, which we can supply on request.
> >We plan to document this in the IETF too.
> >
> >
> >
> >
> >________________________________________________________________
> >Bob Briscoe,                                                  BT

________________________________________________________________
Bob Briscoe,                                                  BT 


From g.white@CableLabs.com  Mon Nov 11 10:57:31 2013
Return-Path: <g.white@CableLabs.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2D71C21E8224; Mon, 11 Nov 2013 10:57:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.236
X-Spam-Level: 
X-Spam-Status: No, score=-0.236 tagged_above=-999 required=5 tests=[AWL=0.000,  BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QozbaJe2ryM1; Mon, 11 Nov 2013 10:57:24 -0800 (PST)
Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 6E7F811E80E6; Mon, 11 Nov 2013 10:57:09 -0800 (PST)
Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id rABIv4to029629; Mon, 11 Nov 2013 11:57:04 -0700
Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Mon, 11 Nov 2013 11:57:04 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com)
Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Mon, 11 Nov 2013 11:57:04 -0700
From: Greg White <g.white@CableLabs.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [tsvwg] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO2/R7zNZxx+pCdEmOwIOO+63dlZocaFeAgARD+QD//7tjAA==
Date: Mon, 11 Nov 2013 18:57:03 +0000
Message-ID: <CEA65B19.21ED5%g.white@cablelabs.com>
In-Reply-To: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.5.0.27]
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <7BE71D21E1C7ED42B7253AF79810863D@cablelabs.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Approved: ondar
Cc: tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2013 18:57:31 -0000

On 11/11/13, 9:02 AM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:

>Greg,
>
>At 06:54 09/11/2013, Greg White wrote:
>>This is very interesting work.  There are a lot of unanswered questions
>>about ecn / no-ecn coexistence and differential treatment in an AQM, and
>>this could provide some answers.
>>
>>To those who groaned that ECN was not included in DOCSIS 3.1, read these
>>slides (and Naeem Khademi's).
>
>Indeed. However, I think it would be safe to recommend that ECN
>support should at least be implemented and separately configurable,
>then it can be turned on or off by operators later.
>
>I'm aware that this doubles the amount of configuration, but we've
>had some success already (with RED) in relating all the ECN
>parameters to the drop parameters by a formula, so hopefully the
>vendor could configure the ECN parameters automatically based on the
>drop parameters.

I don't have a lot of confidence that I can recommend something to be
burned into silicon at this point (and doing it in software is a
non-starter).  The "turned on or off by operators later" would be a given,
but if implemented based on what we know now, I don't have a lot of
confidence that the switch would be ever set to the "turned on" state.  I
can't recommend that vendors add gates for a feature that my intuition
tells me would never be used.  If separately configurable means something
that would increase the probability of the feature being used, then that's
great, but what specifically are we talking about?  It could be a lot of
gates.

Also, you've suggested *not* differentiating between "classic ECN" and
"immediate ECN" in the ECT flags, but this is just a suggestion at this
point.  Again, not something I feel confident about taking for granted.
We'll continue to track this discussion though.




>>Bob, CoDel uses "interval" both as a hold-off for the first packet drop
>>and as the numerator in the invsqrt drop scheduling.  Setting interval =
=3D
>>0
>>would result in ECN being signaled on *every* ECN capable packet when the
>>sojourn time is above threshold.    This jibes with some of your charts
>>for RED, but others show a ramp up in mark probability rather than a step
>>function. Could you clarify?
>
>We've only looked at WRED in detail, because it was much more
>interesting (to us) to reconfigure existing implementations than have
>to wait for new code to be implemented and tested.
>
>The suggestions for PIE and CoDel are just conceptual at this stage -
>we've done no implementation of this idea with either. (I said this
>verbally when presenting the slides, but I should have put it in
>writing too). Please read my suggestions for PIE and CoDel in this light.
>
>I'm not surprised that CoDel derives other parameters from 'interval'
>that should have been declared and set separately. Andrew McGregor
>also pointed out to me that CoDel sets threshold =3D 0.2*interval, so
>threshold would have to be declared separately as well. This starts
>to reveal just how many magic numbers there really are in the CoDel
>algorithm.


In the ns2 CoDel code that we've used (from Kathie), there are two
independent parameters, threshold and interval, and the defaults are 5ms
for threshold and 100ms for interval, so I don=B9t know where the "threshol=
d
=3D 0.2*interval" is coming from (maybe it is different in the current linu=
x
or openWRT distros).

To be more clear about my earlier statement, CoDel actually uses
"interval" to control a single aspect of the drop policy, however, the
code (and descriptions of it) imply that there are two different
functions, and my comments above were written in that vein. However,
here's how I would describe it more succinctly:  the "next drop time" is
always set according to interval/sqrt(count).  In the specific case of
sojourn time crossing above threshold from below, count happens to equal
1, so the first drop is set at one interval in the future.  This is what
is sometimes described as the "hold off" period.

That said, there are some "magic numbers" in the algorithm, specifically
the part that regulates very low drop probabilities (e.g. 8*interval,
count *=3D 0.9844), but that=B9s about it.

I=B9m still thinking about how to achieve differential treatment of marking
vs dropping in CoDel in a logical way.



>
>
>>Setting max_burst =3D 0 in PIE would not result in the step function
>>behavior.
>
>It's not meant to result in a step-function.
>
>In the WRED example, it solely avoids the delay of queue averaging,
>so that once the /instantaneous/ queue exceeds min_thresh it marks
>with increasing probability (not intended to be a step).
>
>Similarly with PIE, the formula:
>         p =3D p + alpha*(est_del-target_del) + beta*(est_del-est_del_old)=
;
>would still gradually increase the probability of drop (not a step
>function), but it would start to do so as soon as the queue exceeded
>target_del, rather than waiting for max_burst.
>
>Is that what you meant?


Yes, that is what I meant.

However, it seemed to me that what you were proposing was to move the
intelligence to the transport, and keep the queue simple. In that regard,
would a very simple threshold:  always mark when instantaneous queue
exceeds the threshold, never mark when it doesn=B9t, be an acceptable way t=
o
do things?

This would be easily implementable, and could be straightforwardly
combined (I think) with any AQM that is controlling drops (including
CoDel).=20

-Greg


>
>
>Bob
>
>
>>-Greg
>>
>>
>>On 11/7/13, 1:03 PM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:
>>
>> >Folks,
>> >
>> >"Immediate ECN" slides:
>> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
>> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
>> >
>> >PS. This talk fell off the end of the TSVAREA agenda. It's mostly
>> >relevant to AQM, but I didn't originally bring it to AQM, because it
>> >affects 3 wgs: tsvwg, aqm & tcpm.
>> >
>> >In the AQM wg, there was dismay about CableLabs not including
>> >anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and
>> >how ECN can take out the 100ms of delay that CoDel and PIE introduce
>> >- it's essentially about auto-tuning for RTT.
>> >
>> >It gives an interim recommendation for hardware designers that there
>> >should be a second instance of the AQM algo for ECN packets so that
>> >it can be configured with different parameters (think of WRED instead
>>of
>> >RED).
>> >
>> >Specifically, for ECN packets:
>> >interval =3D 0 (for CoDel)
>> >max_burst =3D 0 (for PIE)
>> >
>> >
>> >Bob
>> >
>> >PS. We have a paper under submission, which we can supply on request.
>> >We plan to document this in the IETF too.
>> >
>> >
>> >
>> >
>> >________________________________________________________________
>> >Bob Briscoe,                                                  BT
>
>________________________________________________________________
>Bob Briscoe,                                                  BT
>


From bob.briscoe@bt.com  Tue Nov 12 03:21:11 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 72FFD21E805D; Tue, 12 Nov 2013 03:21:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.241
X-Spam-Level: 
X-Spam-Status: No, score=-3.241 tagged_above=-999 required=5 tests=[AWL=0.130,  BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SbNd7ef0QCmD; Tue, 12 Nov 2013 03:21:06 -0800 (PST)
Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id 7BF4811E80E3; Tue, 12 Nov 2013 03:21:05 -0800 (PST)
Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 12 Nov 2013 11:21:00 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 12 Nov 2013 11:21:03 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Tue, 12 Nov 2013 11:20:59 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rACBKvrp001693; Tue, 12 Nov 2013 11:20:57 GMT
Message-ID: <201311121120.rACBKvrp001693@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 12 Nov 2013 11:20:57 +0000
To: "Fred Baker (fred)" <fred@cisco.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CAPRuP3==GDrL2UqMkXbFgbWkS_s0c=TeUGXaemaO=Q9oMO9NcA@mail.g mail.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <145AB1C0-B108-4196-AD5C-3667103FB4D8@cisco.com> <CAPRuP3==GDrL2UqMkXbFgbWkS_s0c=TeUGXaemaO=Q9oMO9NcA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=====================_129284674==.ALT"
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Andrew Mcgregor <andrewmcgr@google.com>, "<draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 11:21:11 -0000

--=====================_129284674==.ALT
Content-Type: text/plain; charset="us-ascii"; format=flowed

Fred,

For the record, Andrew and I spent an evening to see if we could come 
up with a solution to indicate non-congestive loss. We started with a 
list of 9 different causes of loss (from my PhD thesis). We worked 
through a number of ideas, but they all exhibit varying degrees of problems.

So we all agree indicating non-congestive loss is a research problem for now...


Bob

At 20:38 08/11/2013, Andrew Mcgregor wrote:
>I agree, non-congestive loss notification is a research 
>problem.  So, out of scope for the IETF right now but I hope someone 
>thinks about it (I will, as a background task).
>
>
>On 8 November 2013 11:49, Fred Baker (fred) 
><<mailto:fred@cisco.com>fred@cisco.com> wrote:
>
>At 23:04 04/11/2013, Andrew Mcgregor wrote:
> > This seems like valuable work.  One question is, can we put in 
> scope notification that losses are NOT due to congestion?
>
>Speaking for myself, I'm not sure how I would do that.
>
>A loss (or mark) due to congestion is pretty simple. The switch 
>knows what it did.
>
>To know that a packet was lost for reasons other than congestion, I 
>need to somehow know what packets I should expect, and infer that 
>something that I expected didn't happen. In TCP, we know about data 
>segments because they are enumerated - I know what the next octet 
>sequence number to expect, and it doesn't arrive. Control segments 
>(SYN, ACK, FIN, RST, and so on) are not enumerated in that sense - 
>if my peer sends ten identical acks and nine arrive, I as the 
>receiver have no way to know that. At the link layer, most link 
>protocols in use today (PPP, Ethernet, and so on) do not enumerate 
>packets in flight - they are simply there. I *might* be able to see 
>a burst of noise on the line, but only if it looks like it might be 
>the start of a packet and then doesn't end with the right checksum. 
>Even if I can see it, I have no way to know whether the noise 
>garbled one packet or many.
>
>If you want to do some research and come up with a solution, be my 
>guest. But in a standard discussed in 2013... let's not.
>
>_______________________________________________
>aqm mailing list
><mailto:aqm@ietf.org>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm
>
>
>
>
>--
>Andrew McGregor | SRE | 
><mailto:andrewmcgr@google.com>andrewmcgr@google.com | +61 4 8143 7128
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT 
--=====================_129284674==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
Fred,<br><br>
For the record, Andrew and I spent an evening to see if we could come up
with a solution to indicate non-congestive loss. We started with a list
of 9 different causes of loss (from my PhD thesis). We worked through a
number of ideas, but they all exhibit varying degrees of
problems.<br><br>
So we all agree indicating non-congestive loss is a research problem for
now...<br><br>
<br>
Bob<br><br>
At 20:38 08/11/2013, Andrew Mcgregor wrote:<br>
<blockquote type=cite class=cite cite="">I agree, non-congestive loss
notification is a research problem.&nbsp; So, out of scope for the IETF
right now but I hope someone thinks about it (I will, as a background
task).<br><br>
<br>
On 8 November 2013 11:49, Fred Baker (fred)
&lt;<a href="mailto:fred@cisco.com">fred@cisco.com</a>&gt; wrote:<br>

<dl><br>

<dd>At 23:04 04/11/2013, Andrew Mcgregor wrote:<br>

<dd>&gt; This seems like valuable work.&nbsp; One question is, can we put
in scope notification that losses are NOT due to congestion?<br><br>

<dd>Speaking for myself, I'm not sure how I would do that.<br><br>

<dd>A loss (or mark) due to congestion is pretty simple. The switch knows
what it did.<br><br>

<dd>To know that a packet was lost for reasons other than congestion, I
need to somehow know what packets I should expect, and infer that
something that I expected didn't happen. In TCP, we know about data
segments because they are enumerated - I know what the next octet
sequence number to expect, and it doesn't arrive. Control segments (SYN,
ACK, FIN, RST, and so on) are not enumerated in that sense - if my peer
sends ten identical acks and nine arrive, I as the receiver have no way
to know that. At the link layer, most link protocols in use today (PPP,
Ethernet, and so on) do not enumerate packets in flight - they are simply
there. I *might* be able to see a burst of noise on the line, but only if
it looks like it might be the start of a packet and then doesn't end with
the right checksum. Even if I can see it, I have no way to know whether
the noise garbled one packet or many.<br><br>

<dd>If you want to do some research and come up with a solution, be my
guest. But in a standard discussed in 2013... let's not.<br><br>

<dd>_______________________________________________<br>

<dd>aqm mailing list<br>

<dd><a href="mailto:aqm@ietf.org">aqm@ietf.org</a><br>

<dd><a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dl><br><br>
<br>
-- <br>
Andrew McGregor | SRE |
<a href="mailto:andrewmcgr@google.com">andrewmcgr@google.com</a> | +61 4
8143 7128<br>
_______________________________________________<br>
aqm mailing list<br>
aqm@ietf.org<br>
<a href="https://www.ietf.org/mailman/listinfo/aqm" eudora="autourl">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<x-sigsep><p></x-sigsep>
________________________________________________________________<br>
Bob
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
BT</body>
</html>

--=====================_129284674==.ALT--

From rs@netapp.com  Tue Nov 12 05:18:39 2013
Return-Path: <rs@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1204111E814F; Tue, 12 Nov 2013 05:18:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.585
X-Spam-Level: 
X-Spam-Status: No, score=-9.585 tagged_above=-999 required=5 tests=[AWL=0.787,  BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jv4KBCmxp3qI; Tue, 12 Nov 2013 05:18:33 -0800 (PST)
Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) by ietfa.amsl.com (Postfix) with ESMTP id E843B11E8158; Tue, 12 Nov 2013 05:18:33 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,684,1378882800"; d="scan'208";a="114455222"
Received: from vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) by mx12-out.netapp.com with ESMTP; 12 Nov 2013 05:18:32 -0800
Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.86]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.03.0123.003; Tue, 12 Nov 2013 05:18:31 -0800
From: "Scheffenegger, Richard" <rs@netapp.com>
To: Greg White <g.white@CableLabs.com>, Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [tsvwg] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO2/SJueB8pq8PMUCFvNdOsi72LJoc/zgAgAO92wCAADC+gIAAo9PQ
Date: Tue, 12 Nov 2013 13:18:28 +0000
Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25E8908F@SACEXCMBX02-PRD.hq.netapp.com>
References: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk> <CEA65B19.21ED5%g.white@cablelabs.com>
In-Reply-To: <CEA65B19.21ED5%g.white@cablelabs.com>
Accept-Language: de-AT, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.106.53.53]
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 13:18:39 -0000

Hi Greg et.al.,

(chair hat off)

> I don't have a lot of confidence that I can recommend something to be
> burned into silicon at this point (and doing it in software is a non-
> starter).  The "turned on or off by operators later" would be a given, bu=
t
> if implemented based on what we know now, I don't have a lot of confidenc=
e
> that the switch would be ever set to the "turned on" state.  I can't
> recommend that vendors add gates for a feature that my intuition tells me
> would never be used.  If separately configurable means something that
> would increase the probability of the feature being used, then that's
> great, but what specifically are we talking about?  It could be a lot of
> gates.

So, as I am not a HW guy, how many gates does one need for (W)RED, PIE, CoD=
el?

Apparently, as state needs to be kept separately for ECN and non-ECN flows,=
 additional memory plus some way of switching between which memory is used =
for what packet is necessary; however, the hardcoded algos should be the sa=
me (as far as some tuneables are stored in memory instead of hardwired gate=
s).

> Also, you've suggested *not* differentiating between "classic ECN" and
> "immediate ECN" in the ECT flags, but this is just a suggestion at this
> point.  Again, not something I feel confident about taking for granted.
> We'll continue to track this discussion though.

As Bob pointed out in another email, classic ECN will react to any number o=
f marks within one RTT (of that particular flow) with only a single halving=
 of the congestion window. Most other transports that have been modeled aft=
er TCP/ECN will behave similarly. Immediate ECN will react to each mark to =
a smaller extent, and only a full window (RTT) of marks will reduce the win=
dow by half. Thus immediate ECN would be much more aggressive when used wit=
h traditional RED, ie. push the AQM to higher marking probabilities.

One venue that presents itself here, of course, is to use 2nd order effect =
- i.e. the exact marking pattern - to achive reasonable fairness between tr=
ansports making use of immediate and classic reactions. However, I can not =
say offhand, if this would be a practical method (may require too much stat=
e in the scheduling/aqm, including fine grained flow separation).


> However, it seemed to me that what you were proposing was to move the
> intelligence to the transport, and keep the queue simple. In that regard,
> would a very simple threshold:  always mark when instantaneous queue
> exceeds the threshold, never mark when it doesn=B9t, be an acceptable way=
 to
> do things?
>=20
> This would be easily implementable, and could be straightforwardly
> combined (I think) with any AQM that is controlling drops (including
> CoDel).

That is what DCTCP currently does. However, a simple step function is incom=
patible with classic ECN.=20

While reading Bob's earlier comment on classic vs immediate ECN, I thought =
that probably an exponential increase in marking probability, above a speci=
fic threshold (lower than drop) could "autotune" to a reasonable marking ra=
te, without necessarily locking classic out when immediate is used. If the =
linear/step function employed with RED (and shown on Bobs slides) is a good=
 enough approximation needs to be verified. However, the details will proba=
bly also depend on the mixure of number of classic vs. immediate ECN flows =
at the bottleneck, and their respective RTTs...

Finally, as the gateways (switches) cannot rely on ECN to work (either on t=
he required timescales, as the RTT is too large for small buffers, or the t=
ransport/sender is not reacting), even ECN traffic has to be exposed to dro=
ps - and ideally here not to the tail-drop variant.


Richard Scheffenegger



From vbannai@paypal.com  Mon Nov 11 09:52:25 2013
Return-Path: <vbannai@paypal.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C337B11E81C1 for <aqm@ietfa.amsl.com>; Mon, 11 Nov 2013 09:52:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.466
X-Spam-Level: 
X-Spam-Status: No, score=-0.466 tagged_above=-999 required=5 tests=[AWL=-0.833, BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_FORGED_PAYPAL_C=1.3, SARE_HTML_USL_OBFU=1.666]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nvLXrvAZFH0B for <aqm@ietfa.amsl.com>; Mon, 11 Nov 2013 09:52:21 -0800 (PST)
Received: from phx-mipot-001.corp.ebay.com (phx-mipot-001.corp.ebay.com [216.113.172.64]) by ietfa.amsl.com (Postfix) with ESMTP id 9962411E81C0 for <aqm@ietf.org>; Mon, 11 Nov 2013 09:52:21 -0800 (PST)
DomainKey-Signature: s=paypalcorp; d=paypal.com; c=nofws; q=dns; h=X-EBay-Corp:X-IronPort-AV:Received:Received:From:To:CC: Subject:Thread-Topic:Thread-Index:Date:Message-ID: In-Reply-To:Accept-Language:Content-Language: X-MS-Has-Attach:X-MS-TNEF-Correlator:user-agent: x-originating-ip:Content-Type:MIME-Version:X-CFilter; b=SjBv4nz6SSnT6hxQ6lQfmRoPbNsCPzEM3ZNsWuvK2pBQ0QDrTKCFfluv mQgVWWTEtnJQuHJgYlK1ARS1mfOwbZSBPE8jBQjOp/zI5xRYSwNjrSse3 GIVJxXVGVq/pptyVEiL46b07inqagasxobSFtc+9jz2/DvpxPMc3ybBjy I=;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paypal.com; i=@paypal.com; q=dns/txt; s=paypalcorp; t=1384192342; x=1415728342; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=z6ZfWEAP8HHdihv2iTMDuS73rS/z47eVL5cHsvIIOhI=; b=KcQMO5K8DhqAkvAQJLNWaYKGWW9wOpr2Z4kaxorNiOP+vusVhxfLbZCL 51Nf9ETdA73qTBj69PAqfa0IDWB3PjhtLNyfBYJFujeMxNnViHQh0QT7d c/Qus9ttXybMY7LkXe08p77wvtc4Bfu10ZhA6PN6wE2U6zQE3UFhdYEBI w=;
X-EBay-Corp: Yes
X-IronPort-AV: E=Sophos;i="4.93,679,1378882800";  d="scan'208,217";a="154519313"
Received: from phx-vteml-003.corp.ebay.com (HELO PHX-EXMHT-003.corp.ebay.com) ([10.58.40.102]) by phx-mipot-001.corp.ebay.com with ESMTP; 11 Nov 2013 09:52:21 -0800
Received: from PHX-EXRDA-S12.corp.ebay.com ([169.254.13.58]) by PHX-EXMHT-003.corp.ebay.com ([10.58.12.74]) with mapi id 14.03.0158.001; Mon, 11 Nov 2013 10:52:20 -0700
From: "Bannai, Vinay" <vbannai@paypal.com>
To: "tvwg@ietf.org" <tvwg@ietf.org>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO3pDt+rFctRFdvkiXdz7/nqOocJogLu4A
Date: Mon, 11 Nov 2013 17:52:20 +0000
Message-ID: <CEA65841.11FC7%vbannai@paypal.com>
In-Reply-To: <CAO48XRH_yWjikyg26+8ZRD0cvsrTc8kXQ-dc7iV7hYH9aC=oiw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.58.61.242]
Content-Type: multipart/alternative; boundary="_000_CEA6584111FC7vbannaipaypalcom_"
MIME-Version: 1.0
X-CFilter: Scanned
X-Mailman-Approved-At: Tue, 12 Nov 2013 05:34:31 -0800
Cc: Pat Thaler <pthaler@broadcom.com>, John Kaippallimalil <John.Kaippallimalil@huawei.com>, "draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, Bob Briscoe <bob.briscoe@bt.com>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2013 17:52:25 -0000

--_000_CEA6584111FC7vbannaipaypalcom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I support this draft.

Vinay Bannai
Cloud Engineering
PayPal

Date: Mon, 04 Nov 2013 22:03:45 +0000
To: "tsvwg IETF list" <tsvwg@ietf.org<mailto:tsvwg@ietf.org>>, AQM IETF lis=
t <aqm@ietf.org<mailto:aqm@ietf.org>>
From: Bob Briscoe <bob.briscoe@bt.com<mailto:bob.briscoe@bt.com>>
Subject: Who supports tsvwg adoption of adding ECN to L2 or tunnel protocol=
s?
Cc: John Kaippallimalil <John.Kaippallimalil@huawei.com<mailto:John.Kaippal=
limalil@huawei.com>>, Pat Thaler <pthaler@broadcom.com<mailto:pthaler@broad=
com.com>>, draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org<mailto:d=
raft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>

Folks,

Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.
Even better, if you're willing to read / comment / review / implement

Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP
<http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines>

Abstract

   The purpose of this document is to guide the design of congestion
   notification in any lower layer or tunnelling protocol that
   encapsulates IP.  The aim is for explicit congestion signals to
   propagate consistently from lower layer protocols into IP.  Then the
   IP internetwork layer can act as a portability layer to carry
   congestion notification from non-IP-aware congested nodes up to the
   transport layer (L4).  Following these guidelines should assure
   interworking between new lower layer congestion notification
   mechanisms, whether specified by the IETF or other standards bodies.


[Cross-posting tsvwg & aqm, just in case]


Bob Briscoe,
also for co-authors Pat Thaler and John Kaippallimalil

________________________________________________________________
Bob Briscoe,                                                  BT

________________________________________________________________
Bob Briscoe,                                                  BT



--
Vinay Bannai
Email: vbannai@gmail.com<mailto:vbannai@gmail.com>
Google Voice: 415 938 7576

--_000_CEA6584111FC7vbannaipaypalcom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <B2DA573B1A52A04EABCABEDD8F3767D0@corp.ebay.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-fami=
ly: Calibri, sans-serif; ">
<div>
<div>
<div>I support this draft.&nbsp;</div>
</div>
</div>
<div><br>
</div>
<div>Vinay Bannai</div>
<div>Cloud Engineering</div>
<div>PayPal</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_quote"><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
Date: Mon, 04 Nov 2013 22:03:45 &#43;0000<br>
To: &quot;tsvwg IETF list&quot; &lt;<a href=3D"mailto:tsvwg@ietf.org" targe=
t=3D"_blank">tsvwg@ietf.org</a>&gt;, AQM IETF list &lt;<a href=3D"mailto:aq=
m@ietf.org" target=3D"_blank">aqm@ietf.org</a>&gt;<br>
From: Bob Briscoe &lt;<a href=3D"mailto:bob.briscoe@bt.com" target=3D"_blan=
k">bob.briscoe@bt.com</a>&gt;<br>
Subject: Who supports tsvwg adoption of adding ECN to L2 or tunnel protocol=
s?<br>
Cc: John Kaippallimalil &lt;<a href=3D"mailto:John.Kaippallimalil@huawei.co=
m" target=3D"_blank">John.Kaippallimalil@huawei.<u></u>com</a>&gt;, Pat Tha=
ler &lt;<a href=3D"mailto:pthaler@broadcom.com" target=3D"_blank">pthaler@b=
roadcom.com</a>&gt;,
<a href=3D"mailto:draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org" =
target=3D"_blank">
draft-briscoe-tsvwg-ecn-encap-<u></u>guidelines@tools.ietf.org</a><br>
<br>
Folks,<br>
<br>
Pls respond if you support this being adopted as a work-group item in the I=
ETF transport services w-g (tsvwg). The WG chairs need visibility of intere=
st.<br>
Even better, if you're willing to read / comment / review / implement<br>
<br>
Guidelines for Adding Congestion Notification to Protocols that Encapsulate=
 IP<br>
&lt;<a href=3D"http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-gui=
delines" target=3D"_blank">http://tools.ietf.org/html/<u></u>draft-briscoe-=
tsvwg-ecn-encap-<u></u>guidelines</a>&gt;<br>
<br>
Abstract<br>
<br>
&nbsp; &nbsp;The purpose of this document is to guide the design of congest=
ion<br>
&nbsp; &nbsp;notification in any lower layer or tunnelling protocol that<br=
>
&nbsp; &nbsp;encapsulates IP. &nbsp;The aim is for explicit congestion sign=
als to<br>
&nbsp; &nbsp;propagate consistently from lower layer protocols into IP. &nb=
sp;Then the<br>
&nbsp; &nbsp;IP internetwork layer can act as a portability layer to carry<=
br>
&nbsp; &nbsp;congestion notification from non-IP-aware congested nodes up t=
o the<br>
&nbsp; &nbsp;transport layer (L4). &nbsp;Following these guidelines should =
assure<br>
&nbsp; &nbsp;interworking between new lower layer congestion notification<b=
r>
&nbsp; &nbsp;mechanisms, whether specified by the IETF or other standards b=
odies.<br>
<br>
<br>
[Cross-posting tsvwg &amp; aqm, just in case]<br>
<br>
<br>
Bob Briscoe,<br>
also for co-authors Pat Thaler and John Kaippallimalil<br>
<br>
______________________________<u></u>______________________________<u></u>_=
___<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;=
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbs=
p; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT<br>
</blockquote>
<br>
______________________________<u></u>______________________________<u></u>_=
___<br>
Bob Briscoe, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;=
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbs=
p; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BT &nbsp;<br>
</div>
<br>
<br clear=3D"all">
<div><br>
</div>
-- <br>
Vinay Bannai<br>
Email: <a href=3D"mailto:vbannai@gmail.com">vbannai@gmail.com</a><br>
Google Voice: 415 938 7576<br>
</div>
</div>
</div>
</span>
</body>
</html>

--_000_CEA6584111FC7vbannaipaypalcom_--

From fred@cisco.com  Tue Nov 12 09:20:16 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E049711E8122; Tue, 12 Nov 2013 09:20:16 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.31
X-Spam-Level: 
X-Spam-Status: No, score=-110.31 tagged_above=-999 required=5 tests=[AWL=0.061, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1glmsO9LD9au; Tue, 12 Nov 2013 09:20:12 -0800 (PST)
Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) by ietfa.amsl.com (Postfix) with ESMTP id C420B11E8171; Tue, 12 Nov 2013 09:20:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=11813; q=dns/txt; s=iport; t=1384276812; x=1385486412; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=WD60/Y04B0Fqlxq0qyUZUILpWe12H4jxE1H2+Uh/4OQ=; b=mKFA/uONtWK9YOzW1oN9efcA1CV4OLb2caBV2y+qEwV2Ur/ayw2BewiE L6FgAhSNAll9av5W56Ll1Ttb1w3jfH7Weq5U56F9ouhhaH9UZkL7Bg07u gTceulmgGW9kqa49QhPrrcomyMaxmquaO+o8WVFzJLxaJAkosVlti48yk o=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhIFACViglKtJV2Y/2dsb2JhbABagkNEOFO/FoEqFnSCJQEBAQMBAQEBawsFCwIBCBguJwslAgQOBQ4Nh2AGDb8PBI9bBAeDIIERA5AwgTCGL5IKgyaCKg
X-IronPort-AV: E=Sophos;i="4.93,686,1378857600";  d="asc'?scan'208,217";a="283751498"
Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-2.cisco.com with ESMTP; 12 Nov 2013 17:20:11 +0000
Received: from xhc-aln-x02.cisco.com (xhc-aln-x02.cisco.com [173.36.12.76]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id rACHKBV3023566 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 12 Nov 2013 17:20:11 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x02.cisco.com ([173.36.12.76]) with mapi id 14.03.0123.003; Tue, 12 Nov 2013 11:20:10 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Bob Briscoe <bob.briscoe@bt.com>
Thread-Topic: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
Thread-Index: AQHO38twDsbxdPtw6UOkQQuUi2YoxQ==
Date: Tue, 12 Nov 2013 17:20:10 +0000
Message-ID: <9A8352AB-7EA4-42ED-AFD8-E127B51E5922@cisco.com>
References: <201311042203.rA4M3lo0026458@bagheera.jungle.bt.co.uk> <CAH56bmDfOxi2FBvg1P-UH-ds_WveZP4NvOyqopKdEcy5WX3XnQ@mail.gmail.com> <145AB1C0-B108-4196-AD5C-3667103FB4D8@cisco.com> <CAPRuP3==GDrL2UqMkXbFgbWkS_s0c=TeUGXaemaO=Q9oMO9NcA@mail.gmail.com> <201311121120.rACBKvrp001693@bagheera.jungle.bt.co.uk>
In-Reply-To: <201311121120.rACBKvrp001693@bagheera.jungle.bt.co.uk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.121]
Content-Type: multipart/signed; boundary="Apple-Mail=_FF882833-DA3B-4593-BE2D-0AF2C7A7F58A"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Andrew Mcgregor <andrewmcgr@google.com>, "<draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>" <draft-briscoe-tsvwg-ecn-encap-guidelines@tools.ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] Who supports tsvwg adoption of adding ECN to L2 or tunnel protocols?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 17:20:17 -0000

--Apple-Mail=_FF882833-DA3B-4593-BE2D-0AF2C7A7F58A
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_FAC78530-5001-407F-AC1D-579788339EED"


--Apple-Mail=_FAC78530-5001-407F-AC1D-579788339EED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Thanks.=20

Personally, I think it is two problems. One, which I imagine is the one =
you worked on, is the question of detection; we can detect loss at an =
endpoint (it doesn't see something it expects to see) but not reliably =
(it might not be expecting something, but an outside observer expects it =
to see something, and it doesn't see it). The other is the question of =
what one does about it. If it is an unreliable interface injecting =
packet errors, one can route around it. If it's a wireless interface =
(unreliable in the sense that the probability of error is =
non-negligible), one might route around it, or one might impose some =
variation on ARQ or FEC, as we do in WiFi and other technologies. If =
it's another source of non-congestive loss, I suspect that the solution =
is somehow tailored to the mode of operation.

I can tell you that we have a capability for video streaming =
applications in which we enumerate packets and literally send two =
streams on disjoint paths. At the receiving end, we discard the second =
instance of a packet that arrives. In the event of loss in the network, =
we expect that to reduce the probability of non-delivery to a negligible =
value. Solutions like that can be a cost-effective solution to =
non-congestive loss, I should think.

On Nov 12, 2013, at 3:20 AM, Bob Briscoe <bob.briscoe@bt.com> wrote:

> Fred,
>=20
> For the record, Andrew and I spent an evening to see if we could come =
up with a solution to indicate non-congestive loss. We started with a =
list of 9 different causes of loss (from my PhD thesis). We worked =
through a number of ideas, but they all exhibit varying degrees of =
problems.
>=20
> So we all agree indicating non-congestive loss is a research problem =
for now...
>=20
>=20
> Bob
>=20
> At 20:38 08/11/2013, Andrew Mcgregor wrote:
>> I agree, non-congestive loss notification is a research problem.  So, =
out of scope for the IETF right now but I hope someone thinks about it =
(I will, as a background task).
>>=20
>>=20
>> On 8 November 2013 11:49, Fred Baker (fred) <fred@cisco.com> wrote:
>>=20
>> At 23:04 04/11/2013, Andrew Mcgregor wrote:
>> > This seems like valuable work.  One question is, can we put in =
scope notification that losses are NOT due to congestion?
>>=20
>> Speaking for myself, I'm not sure how I would do that.
>>=20
>> A loss (or mark) due to congestion is pretty simple. The switch knows =
what it did.
>>=20
>> To know that a packet was lost for reasons other than congestion, I =
need to somehow know what packets I should expect, and infer that =
something that I expected didn't happen. In TCP, we know about data =
segments because they are enumerated - I know what the next octet =
sequence number to expect, and it doesn't arrive. Control segments (SYN, =
ACK, FIN, RST, and so on) are not enumerated in that sense - if my peer =
sends ten identical acks and nine arrive, I as the receiver have no way =
to know that. At the link layer, most link protocols in use today (PPP, =
Ethernet, and so on) do not enumerate packets in flight - they are =
simply there. I *might* be able to see a burst of noise on the line, but =
only if it looks like it might be the start of a packet and then doesn't =
end with the right checksum. Even if I can see it, I have no way to know =
whether the noise garbled one packet or many.
>>=20
>> If you want to do some research and come up with a solution, be my =
guest. But in a standard discussed in 2013... let's not.
>>=20
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>>=20
>>=20
>>=20
>>=20
>> --=20
>> Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
> ________________________________________________________________
> Bob Briscoe,                                                  BT
>=20

Make things as simple as possible, but not simpler.
Albert Einstein


--Apple-Mail=_FAC78530-5001-407F-AC1D-579788339EED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
">Thanks.&nbsp;<div><br></div><div>Personally, I think it is two =
problems. One, which I imagine is the one you worked on, is the question =
of detection; we can detect loss at an endpoint (it doesn't see =
something it expects to see) but not reliably (it might not be expecting =
something, but an outside observer expects it to see something, and it =
doesn't see it). The other is the question of what one does about it. If =
it is an unreliable interface injecting packet errors, one can route =
around it. If it's a wireless interface (unreliable in the sense that =
the probability of error is non-negligible), one might route around it, =
or one might impose some variation on ARQ or FEC, as we do in WiFi and =
other technologies. If it's another source of non-congestive loss, I =
suspect that the solution is somehow tailored to the mode of =
operation.</div><div><br></div><div>I can tell you that we have a =
capability for video streaming applications in which we enumerate =
packets and literally send two streams on disjoint paths. At the =
receiving end, we discard the second instance of a packet that arrives. =
In the event of loss in the network, we expect that to reduce the =
probability of non-delivery to a negligible value. Solutions like that =
can be a cost-effective solution to non-congestive loss, I should =
think.</div><div><br><div><div>On Nov 12, 2013, at 3:20 AM, Bob Briscoe =
&lt;<a href=3D"mailto:bob.briscoe@bt.com">bob.briscoe@bt.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite">

<meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dus-ascii">
<div>
Fred,<br><br>
For the record, Andrew and I spent an evening to see if we could come up
with a solution to indicate non-congestive loss. We started with a list
of 9 different causes of loss (from my PhD thesis). We worked through a
number of ideas, but they all exhibit varying degrees of
problems.<br><br>
So we all agree indicating non-congestive loss is a research problem for
now...<br><br>
<br>
Bob<br><br>
At 20:38 08/11/2013, Andrew Mcgregor wrote:<br>
<blockquote type=3D"cite" class=3D"cite" cite=3D"x-msg://1910/">I agree, =
non-congestive loss
notification is a research problem.&nbsp; So, out of scope for the IETF
right now but I hope someone thinks about it (I will, as a background
task).<br><br>
<br>
On 8 November 2013 11:49, Fred Baker (fred)
&lt;<a href=3D"mailto:fred@cisco.com">fred@cisco.com</a>&gt; wrote:<br>

<dl><br>

<dd>At 23:04 04/11/2013, Andrew Mcgregor wrote:<br>

</dd><dd>&gt; This seems like valuable work.&nbsp; One question is, can =
we put
in scope notification that losses are NOT due to congestion?<br><br>

</dd><dd>Speaking for myself, I'm not sure how I would do that.<br><br>

</dd><dd>A loss (or mark) due to congestion is pretty simple. The switch =
knows
what it did.<br><br>

</dd><dd>To know that a packet was lost for reasons other than =
congestion, I
need to somehow know what packets I should expect, and infer that
something that I expected didn't happen. In TCP, we know about data
segments because they are enumerated - I know what the next octet
sequence number to expect, and it doesn't arrive. Control segments (SYN,
ACK, FIN, RST, and so on) are not enumerated in that sense - if my peer
sends ten identical acks and nine arrive, I as the receiver have no way
to know that. At the link layer, most link protocols in use today (PPP,
Ethernet, and so on) do not enumerate packets in flight - they are =
simply
there. I *might* be able to see a burst of noise on the line, but only =
if
it looks like it might be the start of a packet and then doesn't end =
with
the right checksum. Even if I can see it, I have no way to know whether
the noise garbled one packet or many.<br><br>

</dd><dd>If you want to do some research and come up with a solution, be =
my
guest. But in a standard discussed in 2013... let's not.<br><br>

</dd><dd>_______________________________________________<br>

</dd><dd>aqm mailing list<br>

</dd><dd><a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>

</dd><dd><a href=3D"https://www.ietf.org/mailman/listinfo/aqm" =
eudora=3D"autourl">
https://www.ietf.org/mailman/listinfo/aqm</a><br><br>

</dd></dl><br><br>
<br>
-- <br>
Andrew McGregor | SRE |
<a href=3D"mailto:andrewmcgr@google.com">andrewmcgr@google.com</a> | +61 =
4
8143 7128<br>
_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" eudora=3D"autourl">
https://www.ietf.org/mailman/listinfo/aqm</a></blockquote>
<x-sigsep><p>
________________________________________________________________<br>
Bob
=
Briscoe,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;
BT</p></x-sigsep></div>

</blockquote></div><br><div apple-content-edited=3D"true">
<span class=3D"Apple-style-span" style=3D"border-collapse: separate; =
color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: 2; text-align: =
auto; text-indent: 0px; text-transform: none; white-space: normal; =
widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; =
-webkit-border-vertical-spacing: 0px; =
-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
auto; -webkit-text-stroke-width: 0px; "><span class=3D"Apple-style-span" =
style=3D"font-family: sans-serif; font-size: 12px; line-height: 18px; =
"><ul style=3D"line-height: 1.5em; list-style-type: square; margin-top: =
0.3em; margin-right: 0px; margin-bottom: 0px; margin-left: 1.5em; =
padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: =
0px; list-style-image: =
url(http://en.wikiquote.org/skins-1.5/monobook/bullet.gif); "><li =
style=3D"margin-bottom: 0.1em; ">Make things as simple as possible, but =
not simpler.</li><div>Albert Einstein</div></ul></span></span>
</div>
<br></div></body></html>=

--Apple-Mail=_FAC78530-5001-407F-AC1D-579788339EED--

--Apple-Mail=_FF882833-DA3B-4593-BE2D-0AF2C7A7F58A
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSgmNHbjEdbHIsm0MRApgFAJoDYJrdGlk8DEEhNTTpmQBfYjw0mgCfa8NU
JSxcxv+wFmayw2t6bPvn+bI=
=x9nx
-----END PGP SIGNATURE-----

--Apple-Mail=_FF882833-DA3B-4593-BE2D-0AF2C7A7F58A--

From bob.briscoe@bt.com  Tue Nov 12 14:20:53 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 114EC21E8090; Tue, 12 Nov 2013 14:20:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.745
X-Spam-Level: 
X-Spam-Status: No, score=-2.745 tagged_above=-999 required=5 tests=[AWL=-0.373, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ElDKZbE7AdQv; Tue, 12 Nov 2013 14:20:48 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id A953921E8095; Tue, 12 Nov 2013 14:20:47 -0800 (PST)
Received: from EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) by EVMHR66-UKRD.bt.com (10.187.101.21) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 12 Nov 2013 22:20:45 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 12 Nov 2013 22:20:45 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Tue, 12 Nov 2013 22:20:43 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rACMKfo9003510; Tue, 12 Nov 2013 22:20:41 GMT
Message-ID: <201311122220.rACMKfo9003510@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 12 Nov 2013 22:20:41 +0000
To: Greg White <g.white@CableLabs.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <CEA65B19.21ED5%g.white@cablelabs.com>
References: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk> <CEA65B19.21ED5%g.white@cablelabs.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 22:20:53 -0000

Greg, inline...

At 18:57 11/11/2013, Greg White wrote:


>On 11/11/13, 9:02 AM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:
>
> >Greg,
> >
> >At 06:54 09/11/2013, Greg White wrote:
> >>This is very interesting work.  There are a lot of unanswered questions
> >>about ecn / no-ecn coexistence and differential treatment in an AQM, and
> >>this could provide some answers.
> >>
> >>To those who groaned that ECN was not included in DOCSIS 3.1, read these
> >>slides (and Naeem Khademi's).
> >
> >Indeed. However, I think it would be safe to recommend that ECN
> >support should at least be implemented and separately configurable,
> >then it can be turned on or off by operators later.
> >
> >I'm aware that this doubles the amount of configuration, but we've
> >had some success already (with RED) in relating all the ECN
> >parameters to the drop parameters by a formula, so hopefully the
> >vendor could configure the ECN parameters automatically based on the
> >drop parameters.
>
>I don't have a lot of confidence that I can recommend something to be
>burned into silicon at this point (and doing it in software is a
>non-starter).  The "turned on or off by operators later" would be a given,
>but if implemented based on what we know now, I don't have a lot of
>confidence that the switch would be ever set to the "turned on" state.  I
>can't recommend that vendors add gates for a feature that my intuition
>tells me would never be used.  If separately configurable means something
>that would increase the probability of the feature being used, then that's
>great, but what specifically are we talking about?  It could be a lot of
>gates.
>
>Also, you've suggested *not* differentiating between "classic ECN" and
>"immediate ECN" in the ECT flags, but this is just a suggestion at this
>point.  Again, not something I feel confident about taking for granted.
>We'll continue to track this discussion though.

Understood.

Given the cable industry has decided to move on=20
this early schedule, we can now do the necessary=20
research so that competing access technologies can properly exploit ECN ;)


more...

> >>Bob, CoDel uses "interval" both as a hold-off for the first packet drop
> >>and as the numerator in the invsqrt drop scheduling.  Setting interval =
=3D
> >>0
> >>would result in ECN being signaled on *every* ECN capable packet when=
 the
> >>sojourn time is above threshold.    This jibes with some of your charts
> >>for RED, but others show a ramp up in mark probability rather than a=
 step
> >>function. Could you clarify?
> >
> >We've only looked at WRED in detail, because it was much more
> >interesting (to us) to reconfigure existing implementations than have
> >to wait for new code to be implemented and tested.
> >
> >The suggestions for PIE and CoDel are just conceptual at this stage -
> >we've done no implementation of this idea with either. (I said this
> >verbally when presenting the slides, but I should have put it in
> >writing too). Please read my suggestions for PIE and CoDel in this light.
> >
> >I'm not surprised that CoDel derives other parameters from 'interval'
> >that should have been declared and set separately. Andrew McGregor
> >also pointed out to me that CoDel sets threshold =3D 0.2*interval, so
> >threshold would have to be declared separately as well. This starts
> >to reveal just how many magic numbers there really are in the CoDel
> >algorithm.
>
>
>In the ns2 CoDel code that we've used (from Kathie), there are two
>independent parameters, threshold and interval, and the defaults are 5ms
>for threshold and 100ms for interval, so I don=B9t know where the=
 "threshold
>=3D 0.2*interval" is coming from (maybe it is different in the current=
 linux
>or openWRT distros).

Sorry, altho it turns out I was wrong anyway (see below), I meant to say:
         0.05 * interval
         (=3D interval / 20)
I was working from memory and I knew there was a 2 in there somewhere.

Andrew McGregor said at the mic in tsvwg last=20
Friday that threshold was defined relative to=20
interval, and I remembered it from when I first=20
looked at the CoDel code. However, I've just=20
checked and even Andrew's ns3 port of CoDel sets=20
threshold to 5ms, independent of the setting of=20
interval. So consider this a memory thow-back on everyone's part and ignore=
 it.


>To be more clear about my earlier statement, CoDel actually uses
>"interval" to control a single aspect of the drop policy, however, the
>code (and descriptions of it) imply that there are two different
>functions, and my comments above were written in that vein. However,
>here's how I would describe it more succinctly:  the "next drop time" is
>always set according to interval/sqrt(count).  In the specific case of
>sojourn time crossing above threshold from below, count happens to equal
>1, so the first drop is set at one interval in the future.  This is what
>is sometimes described as the "hold off" period.

OK. From an admittedly hasty scan through Andrew=20
McGregor's port of CoDel to ns3 (not the Linux=20
code), it seems to introduce this hold-off time=20
in the ControlLaw() function, but it only calls=20
ControlLaw() from the ShouldDrop() function if=20
the queue has remained above threshold for duration interval.

That implies that CoDel holds off signalling=20
anything for 2 * interval, ie 200ms for the=20
default setting of interval =3D 100ms.

>That said, there are some "magic numbers" in the algorithm, specifically
>the part that regulates very low drop probabilities (e.g. 8*interval,
>count *=3D 0.9844), but that=B9s about it.

And the following magic numbers, of course:
* threshold [5ms]
* the power used in the control law [0.5 in the sqrt function]


>I=B9m still thinking about how to achieve differential treatment of marking
>vs dropping in CoDel in a logical way.

That would only be possible if CoDel's control=20
law for drop was based on some understandable=20
logic in the first place. I've started another=20
thread on that, based on analysis I've just done of CoDel's control law.

A naive answer would be to create a second=20
variable for the interval used in ControlLaw(),=20
let's call it I.  Then solely set interval =3D 0.

But that would still leave a signalling delay of=20
I. As you say, CoDel's control law for increasing=20
the dropping frequency is also based on the assumption that I =3D RTT =3D=
 100ms.

By my analysis (in the other email), the=20
rationale for CoDel's current control law seems=20
incorrect, which is probably why we're finding=20
it's tough to base any new thinking on it.

> >
> >
> >>Setting max_burst =3D 0 in PIE would not result in the step function
> >>behavior.
> >
> >It's not meant to result in a step-function.
> >
> >In the WRED example, it solely avoids the delay of queue averaging,
> >so that once the /instantaneous/ queue exceeds min_thresh it marks
> >with increasing probability (not intended to be a step).
> >
> >Similarly with PIE, the formula:
> >         p =3D p + alpha*(est_del-target_del) +=
 beta*(est_del-est_del_old);
> >would still gradually increase the probability of drop (not a step
> >function), but it would start to do so as soon as the queue exceeded
> >target_del, rather than waiting for max_burst.
> >
> >Is that what you meant?
>
>
>Yes, that is what I meant.
>
>However, it seemed to me that what you were proposing was to move the
>intelligence to the transport, and keep the queue simple. In that regard,
>would a very simple threshold:  always mark when instantaneous queue
>exceeds the threshold, never mark when it doesn=B9t, be an acceptable way=
 to
>do things?
>
>This would be easily implementable, and could be straightforwardly
>combined (I think) with any AQM that is controlling drops (including
>CoDel).

A simple step threshold would be fine if all=20
traffic used ECN. DCTCP proves that such a really=20
dumb AQM works well if complemented by a smarter=20
TCP (which proved to me that our problem is with TCP).

But I don't think it's so easy to combine such a=20
simple AQM for ECN traffic, with a complex AQM like CoDel for drop.

The logic I started from was:
* If starvation is going to happen, it will happen over time
* So we should be able to design two AQMs (for=20
ECN and drop) that will not cause each other to=20
starve, by only considering where they eventually=20
converge to - solely in the presence of stable=20
long-running flows. We can elide dynamics, like=20
smoothing, that disappear over time.
* CoDel looks for the min queuing delay over=20
interval, so we can elide that part of its=20
behaviour. But the control law continues to=20
increase drop with time, irrespective of how the=20
queue is growing (as long as it is greater than=20
threshold). Then, when drop is high enough,=20
assuming responsive flows, the CoDel queue will=20
fall below threshold and switch back to=20
non-dropping mode. Then the cycle will repeat.=20
It's not easy to elide away behaviour that=20
stabilises by continually switching between two=20
discrete modes, rather than stabilising in one mode.

This is what the short-term memory within CoDel=20
does - if it returns to dropping mode soon enough=20
after leaving it, it starts dropping from where=20
it left off. So it's actually continually=20
switching in and out of dropping mode, which=20
makes it much harder to think about analytically.=20
And much harder to design a complementary ECN behaviour for it.


________
I prefer the principle: "Design for=20
Verifiability", which is why I always have an=20
allergic reaction to CoDel. See John Doyle's=20
paper below, which uses TCP and AQM design as a=20
case study for how to use and abuse this principle:

Doyle, J.C., Carlson, J., Low, S.H., Paganini,=20
F., Vinnicombe, G., Willinger, W., Hickey, J.,=20
Parrilo, P. & Vandenberghe, L., "Robustness and=20
the Internet: Theoretical Foundations," Caltech Draft Paper (March 2002)
<http://netlab.caltech.edu/pub/papers/RIPartII.pdf>

The IRTF network complexity research group is=20
trying to make this stuff understandable by us mere IETFers.



Bob


>-Greg
>
>
> >
> >
> >Bob
> >
> >
> >>-Greg
> >>
> >>
> >>On 11/7/13, 1:03 PM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:
> >>
> >> >Folks,
> >> >
> >> >"Immediate ECN" slides:
> >> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pptx>
> >> ><http://bobbriscoe.net/presents/1311ietf/1311tsvarea-iecn.pdf>
> >> >
> >> >PS. This talk fell off the end of the TSVAREA agenda. It's mostly
> >> >relevant to AQM, but I didn't originally bring it to AQM, because it
> >> >affects 3 wgs: tsvwg, aqm & tcpm.
> >> >
> >> >In the AQM wg, there was dismay about CableLabs not including
> >> >anything about ECN in DOCSIS3.1. This talk is about AQM dynamics; and
> >> >how ECN can take out the 100ms of delay that CoDel and PIE introduce
> >> >- it's essentially about auto-tuning for RTT.
> >> >
> >> >It gives an interim recommendation for hardware designers that there
> >> >should be a second instance of the AQM algo for ECN packets so that
> >> >it can be configured with different parameters (think of WRED instead
> >>of
> >> >RED).
> >> >
> >> >Specifically, for ECN packets:
> >> >interval =3D 0 (for CoDel)
> >> >max_burst =3D 0 (for PIE)
> >> >
> >> >
> >> >Bob
> >> >
> >> >PS. We have a paper under submission, which we can supply on request.
> >> >We plan to document this in the IETF too.
> >> >
> >> >
> >> >
> >> >
> >> >________________________________________________________________
> >> >Bob Briscoe,                                                  BT
> >
> >________________________________________________________________
> >Bob Briscoe,                                                  BT
> >
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT=20


From bob.briscoe@bt.com  Tue Nov 12 14:30:30 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CEA5F11E810B for <aqm@ietfa.amsl.com>; Tue, 12 Nov 2013 14:30:30 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.349
X-Spam-Level: 
X-Spam-Status: No, score=-3.349 tagged_above=-999 required=5 tests=[AWL=0.250,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WbE6pkQOWvGw for <aqm@ietfa.amsl.com>; Tue, 12 Nov 2013 14:30:25 -0800 (PST)
Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id 2286921E8095 for <aqm@ietf.org>; Tue, 12 Nov 2013 14:30:24 -0800 (PST)
Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 12 Nov 2013 22:30:18 +0000
Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.279.1; Tue, 12 Nov 2013 22:30:22 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Tue, 12 Nov 2013 22:30:12 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rACMUBmH003536; Tue, 12 Nov 2013 22:30:11 GMT
Message-ID: <201311122230.rACMUBmH003536@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 12 Nov 2013 22:30:10 +0000
To: <nichols@pollere.com>, <vanj@google.com>
From: Bob Briscoe <bob.briscoe@bt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: AQM IETF list <aqm@ietf.org>
Subject: [aqm] CoDel's control law that determines drop frequency
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 22:30:31 -0000

Kathie, Van,

In <http://tools.ietf.org/html/draft-nichols-tsvwg-codel-01> (and in 
the ACM queue paper), you say CoDel's control law was designed around 
TCP Reno's formula relating packet drop probability to throughput. Quoting:

"  The next
    drop time is decreased in inverse proportion to the square root of
    the number of drops since the dropping state was entered, using the
    well-known relationship of drop rate to throughput to get a linear
    change in throughput. [REDL1998, MACTCP1997]
"
1/ It's questionable whether it's desirable to design an AQM around 
any particular transport algorithm. However, let's leave this point 
aside for now...

2/ It's questionable whether it's reasonable to design an AQM around 
NewReno's square-root law that is known not to scale, which means 
NewReno is being superceded by algorithms like Cubic that move away 
from the square root. Specifically if the window W is proportional to 
1/p^b, then for:
         NewReno,        b = 0.5
         Cubic,          b = 0.75
         Compound,       b = 0.8
         HSTCP,          b = 0.835
         DCTCP,          b = 1
         Relentless,     b = 1
         Scalable TCP,   b = 1
However, let's leave this point aside too for now...

3/ Back in Apr'13, for myself, I plotted how the CoDel algorithm 
increases drop probability with time, and it looked pretty linear to 
me, not a square root law. I've just got round to doing the analysis, 
and indeed it tends very quickly to a linear law with time. So, 
Codel's control law will actually cause Reno's throughput to fall 
with the square root of time, not linearly.

4/ It seems to be a waste of cycles to implement CoDel's control law 
using a square-root. Given it results in a linear increase in drop 
frequency with time, it should be possible to design a much simpler algorithm.

5/ My analysis also shows that the rate of increase in drop 
probablity is inversely proportional to the link rate. I don't 
believe this is desirable, as it will require tuning for the expected 
number of flows on the link. However it is probably OK for variable rate links.

6/ The rate of increase of drop probability is also inversely 
proportional to the square of 'interval'. I suggest a new constant is 
defined for the control law, rather than relating the control law to 
interval in this way. Otherwise, when anyone tunes interval, they 
will get undesirable changes to the control law (e.g. reducing 
interval 10x [to 10ms] would make the control law 100x more sluggish).

7/ We'll need to debate how the control law should behave in the 
presence of unresponsive flows, so this analysis should at least 
support that discussion.


Here's my working (pls check it - I may have made mistakes)
_________________
For brevity, I'll define some briefer variable names:
         interval =      I [s]
         next_drop =     D [s]
         packet-rate =   R [pkt/s]
         count =         n [pkt]

 From the CoDel control law code:
         D(n) = I / sqrt(n)
And the instantaneous drop probability is:
         p(n) = 1/( R * D(n) )

Then the slope of the rise in drop probability with time is:
         Delta p / Delta t       = [p(n+1) - p(n)] / D(n)
                                 = [1/D(n+1) - 1/D(n)] / [ R * D(n) ]
                                 = sqrt(n) * [sqrt(n+1) - sqrt(n)] / [R*I*I]
                                 = [ sqrt(n(n+1)) - n ] / R*I^2

At count = 1, the numerator starts at sqrt(2)-1 = 0.414.
Amd as n increases, it rapidly tends to 1/2.

So CoDel's rate of increase of drop probability with time is nearly 
constant (it is always between 0.414 and 0.5) and it rapidly 
approaches 0.5 after a few drops, tending towards:
         dp/dt = 1/(2*R*I^2)

This constant increase clearly has very little to do with the 
square-root law of TCP Reno.

In the above formula, drop probability increases inversely 
proportional to the packet rate. For instance, with I = 100ms and 1500B packets
at 10Mb/s =>    R = 833 pkt/s =>        dp/dt = 6.0% /s
at 100Mb/s =>   R = 8333 pkt/s =>       dp/dt = 0.6% /s

If CoDel was designed for scalable transports like DCTCP, and it was 
designed for the same number of flows on any size link, this scaling 
with link rate would make sense. But there are two reasons for an AQM 
to have to scale with link rate:

a) Variable rate link
You would probably want to design for about the same number of flows 
no matter how the link rate varies, so the control law could work for 
variable rate links (assuming scalable transports, which we are 
gradually tending towards).

b) Deploying CoDel on links in different parts of a network
At any one time, we tend to design a network assuming there will be 
more flows on faster links. So, in order to remain as responsive on 
faster links, CoDel's control law will need to be tuned for the 
expected number of flows before it is deployed on different size links.


Cheers



Bob


________________________________________________________________
Bob Briscoe,                                                  BT 


From bob.briscoe@bt.com  Tue Nov 12 15:03:48 2013
Return-Path: <bob.briscoe@bt.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 652F211E8102; Tue, 12 Nov 2013 15:03:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.742
X-Spam-Level: 
X-Spam-Status: No, score=-2.742 tagged_above=-999 required=5 tests=[AWL=-0.370, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EquY7NDhynvd; Tue, 12 Nov 2013 15:03:43 -0800 (PST)
Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) by ietfa.amsl.com (Postfix) with ESMTP id 8146F21E80A5; Tue, 12 Nov 2013 15:03:32 -0800 (PST)
Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR66-UKRD.bt.com (10.187.101.21) with Microsoft SMTP Server (TLS) id 14.3.158.1; Tue, 12 Nov 2013 23:03:30 +0000
Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 12 Nov 2013 23:03:27 +0000
Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Tue, 12 Nov 2013 23:03:22 +0000
Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145])	by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id rACN3LUX003607; Tue, 12 Nov 2013 23:03:21 GMT
Message-ID: <201311122303.rACN3LUX003607@bagheera.jungle.bt.co.uk>
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 12 Nov 2013 23:03:20 +0000
To: "Scheffenegger, Richard" <rs@netapp.com>
From: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E8908F@SACEXCMBX02-PRD.h q.netapp.com>
References: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk> <CEA65B19.21ED5%g.white@cablelabs.com> <012C3117EDDB3C4781FD802A8C27DD4F25E8908F@SACEXCMBX02-PRD.hq.netapp.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158
Cc: Greg White <g.white@CableLabs.com>, tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 23:03:48 -0000

Richard,

I've replied separately to Greg - apologies for splitting the thread.=
 Inline...

At 13:18 12/11/2013, Scheffenegger, Richard wrote:
>Hi Greg et.al.,
>
>(chair hat off)
>
> > I don't have a lot of confidence that I can recommend something to be
> > burned into silicon at this point (and doing it in software is a non-
> > starter).  The "turned on or off by operators later" would be a given,=
 but
> > if implemented based on what we know now, I don't have a lot of=
 confidence
> > that the switch would be ever set to the "turned on" state.  I can't
> > recommend that vendors add gates for a feature that my intuition tells=
 me
> > would never be used.  If separately configurable means something that
> > would increase the probability of the feature being used, then that's
> > great, but what specifically are we talking about?  It could be a lot of
> > gates.
>
>So, as I am not a HW guy, how many gates does one need for (W)RED, PIE,=
 CoDel?
>
>Apparently, as state needs to be kept separately=20
>for ECN and non-ECN flows, additional memory=20
>plus some way of switching between which memory=20
>is used for what packet is necessary; however,=20
>the hardcoded algos should be the same (as far=20
>as some tuneables are stored in memory instead of hardwired gates).

I'll let the hw people answer that, but it seems a reasonable point.


> > Also, you've suggested *not* differentiating between "classic ECN" and
> > "immediate ECN" in the ECT flags, but this is just a suggestion at this
> > point.  Again, not something I feel confident about taking for granted.
> > We'll continue to track this discussion though.
>
>As Bob pointed out in another email, classic ECN=20
>will react to any number of marks within one RTT=20
>(of that particular flow) with only a single=20
>halving of the congestion window. Most other=20
>transports that have been modeled after TCP/ECN=20
>will behave similarly. Immediate ECN will react=20
>to each mark to a smaller extent, and only a=20
>full window (RTT) of marks will reduce the=20
>window by half. Thus immediate ECN would be much=20
>more aggressive when used with traditional RED,=20
>ie. push the AQM to higher marking probabilities.

The paper (available on request, because still=20
under submission) records experiments that show=20
we can get equal rates between otherwise=20
equivalent ECN and drop flows as long as the=20
parameters of the two ramps are connected by a=20
formula. This formula was found by fitting to the=20
very clear curve visible along the ridge of the=20
3-D plot on our slide 11=20
<http://www.ietf.org/proceedings/88/slides/slides-88-tsvwg-20.pdf>.

I'm sure this will not be the whole story, but=20
it's a good start. I believe that such parameter=20
settings only lead to equal rates between an ECN=20
and a drop flow for one of each type of flow. I=20
won't be particularly worried if the relative=20
rates stray from precise equality under different=20
conditions, as long as there is no starvation,=20
and they remain in the same ball-park.


>One venue that presents itself here, of course,=20
>is to use 2nd order effect - i.e. the exact=20
>marking pattern - to achive reasonable fairness=20
>between transports making use of immediate and=20
>classic reactions. However, I can not say=20
>offhand, if this would be a practical method=20
>(may require too much state in the=20
>scheduling/aqm, including fine grained flow separation).

Hopefully we won't need to go there.


> > However, it seemed to me that what you were proposing was to move the
> > intelligence to the transport, and keep the queue simple. In that=
 regard,
> > would a very simple threshold:  always mark when instantaneous queue
> > exceeds the threshold, never mark when it doesn=B9t, be an acceptable=
 way to
> > do things?
> >
> > This would be easily implementable, and could be straightforwardly
> > combined (I think) with any AQM that is controlling drops (including
> > CoDel).
>
>That is what DCTCP currently does.

I wasn't aware that DCTCP was already being used=20
alongside non-ECN traffic. Is this what you meant?

>However, a simple step function is incompatible with classic ECN.

I think you mean classic ECN TCP flows. If so, I=20
wouldn't state this quite so strongly (yet). I=20
suspect there will be conditions where a classic=20
ECN TCP will be somewhat disadvantaged. However, we need experiments to see:
* whether the disadvantage is slight,
* how many classic ECN hosts might be affected,
* and the likelihood that their owners would=20
rapidly upgrade them (AFAIK, their owners must=20
have already intervened manually for ECN hosts to=20
already be in the wild, given I believe ECN is=20
turned off by default in TCP clients under the major operating systems).


>While reading Bob's earlier comment on classic=20
>vs immediate ECN, I thought that probably an=20
>exponential increase in marking probability,=20
>above a specific threshold (lower than drop)=20
>could "autotune" to a reasonable marking rate,=20
>without necessarily locking classic out when=20
>immediate is used. If the linear/step function=20
>employed with RED (and shown on Bobs slides) is=20
>a good enough approximation needs to be=20
>verified. However, the details will probably=20
>also depend on the mixure of number of classic=20
>vs. immediate ECN flows at the bottleneck, and their respective RTTs...

Agreed.


>Finally, as the gateways (switches) cannot rely=20
>on ECN to work (either on the required=20
>timescales, as the RTT is too large for small=20
>buffers, or the transport/sender is not=20
>reacting), even ECN traffic has to be exposed to drops

Agreed

>  - and ideally here not to the tail-drop variant.

It may not be necessary to switch an ECN flow to=20
AQM-generated drops - it may be sufficient to=20
switch both ECN and drop traffic to tail drop at=20
the same time - when load exceeds the operating=20
region of the AQM as a whole. But, you may be right - needs more=
 experiments.


Bob



>Richard Scheffenegger
>
>
>_______________________________________________
>aqm mailing list
>aqm@ietf.org
>https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT=20


From ilpo.jarvinen@helsinki.fi  Tue Nov 12 15:20:41 2013
Return-Path: <ilpo.jarvinen@helsinki.fi>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 80D5921E8089; Tue, 12 Nov 2013 15:20:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.072
X-Spam-Level: 
X-Spam-Status: No, score=-6.072 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TA-rg+51T2Wc; Tue, 12 Nov 2013 15:20:34 -0800 (PST)
Received: from courier.cs.helsinki.fi (courier.cs.helsinki.fi [128.214.9.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8083911E814D; Tue, 12 Nov 2013 15:20:33 -0800 (PST)
Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by mail.cs.helsinki.fi with esmtp; Wed, 13 Nov 2013 01:20:16 +0200 id 00068157.5282B7B0.00003231
Date: Wed, 13 Nov 2013 01:20:16 +0200 (EET)
From: "=?ISO-8859-15?Q?Ilpo_J=E4rvinen?=" <ilpo.jarvinen@helsinki.fi>
X-X-Sender: ijjarvin@melkinpaasi.cs.helsinki.fi
To: Bob Briscoe <bob.briscoe@bt.com>
In-Reply-To: <201311122220.rACMKfo9003510@bagheera.jungle.bt.co.uk>
Message-ID: <alpine.DEB.2.02.1311130045490.18698@melkinpaasi.cs.helsinki.fi>
References: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk> <CEA65B19.21ED5%g.white@cablelabs.com> <201311122220.rACMKfo9003510@bagheera.jungle.bt.co.uk>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-CS-Test-DKIM: none
Cc: Greg White <g.white@CableLabs.com>, tsv-area IETF list <tsv-area@ietf.org>, AQM IETF list <aqm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 23:20:41 -0000

On Tue, 12 Nov 2013, Bob Briscoe wrote:

> Greg, inline...
> 
> At 18:57 11/11/2013, Greg White wrote:
> 
> 
> > On 11/11/13, 9:02 AM, "Bob Briscoe" <bob.briscoe@bt.com> wrote:
> > 
> > >Greg,
> > >
> > >At 06:54 09/11/2013, Greg White wrote:
> > >
> > >>Bob, CoDel uses "interval" both as a hold-off for the first packet drop
> > >>and as the numerator in the invsqrt drop scheduling.  Setting interval =
> > >>0
> > >>would result in ECN being signaled on *every* ECN capable packet when the
> > >>sojourn time is above threshold.    This jibes with some of your charts
> > >>for RED, but others show a ramp up in mark probability rather than a step
> > >>function. Could you clarify?
> > >
> > >We've only looked at WRED in detail, because it was much more
> > >interesting (to us) to reconfigure existing implementations than have
> > >to wait for new code to be implemented and tested.
> > >
> > >The suggestions for PIE and CoDel are just conceptual at this stage -
> > >we've done no implementation of this idea with either. (I said this
> > >verbally when presenting the slides, but I should have put it in
> > >writing too). Please read my suggestions for PIE and CoDel in this light.
> > >
> > >I'm not surprised that CoDel derives other parameters from 'interval'
> > >that should have been declared and set separately. Andrew McGregor
> > >also pointed out to me that CoDel sets threshold = 0.2*interval, so
> > >threshold would have to be declared separately as well. This starts
> > >to reveal just how many magic numbers there really are in the CoDel
> > >algorithm.

Btw, Van & Kathleen say about this in ID:

"A CoDel control law more independent of interval is future work."

I suppose it still is...

> > To be more clear about my earlier statement, CoDel actually uses
> > "interval" to control a single aspect of the drop policy, however, the
> > code (and descriptions of it) imply that there are two different
> > functions, and my comments above were written in that vein. However,
> > here's how I would describe it more succinctly:  the "next drop time" is
> > always set according to interval/sqrt(count).  In the specific case of
> > sojourn time crossing above threshold from below, count happens to equal
> > 1, so the first drop is set at one interval in the future.  This is what
> > is sometimes described as the "hold off" period.
> 
> OK. From an admittedly hasty scan through Andrew McGregor's port of CoDel to
> ns3 (not the Linux code), it seems to introduce this hold-off time in the
> ControlLaw() function, but it only calls ControlLaw() from the ShouldDrop()
> function if the queue has remained above threshold for duration interval.
> 
> That implies that CoDel holds off signalling anything for 2 * interval, ie
> 200ms for the default setting of interval = 100ms.

There are two calls to it at least in the ID. CoDel is also implemented 
like that in ns2, Linux, and (based on very brief check) also in ns3. The 
first call happens exactly at interval, not two away from crossing the 
target (/threshold) delay. Both calls are preceded by a "drop" (/signal).


-- 
 i.

From preethi.cis@gmail.com  Tue Nov 12 15:56:36 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5D60221E80C4 for <aqm@ietfa.amsl.com>; Tue, 12 Nov 2013 15:56:36 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v0zMdgzMrI4k for <aqm@ietfa.amsl.com>; Tue, 12 Nov 2013 15:56:35 -0800 (PST)
Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) by ietfa.amsl.com (Postfix) with ESMTP id B88C821E8089 for <aqm@ietf.org>; Tue, 12 Nov 2013 15:56:35 -0800 (PST)
Received: by mail-pa0-f46.google.com with SMTP id rd3so2552858pab.33 for <aqm@ietf.org>; Tue, 12 Nov 2013 15:56:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=7sKX40x+xDnV3jnl8ksy44E28CKcRFhFA59Q3hB5ldo=; b=LC0wQ2M4NYtYaEP//JxgPBlNXoVuZE9+3+4mNVL8yB3z0GC1EzIh55PQNEtMnaDNyx sUSRABZagYhJ+I2MQIoxYEIFq7YzVk/biObh7RSgodTnGDtVduGAk8OpDpfY+4VZ7nq6 iuCCZVR811VTFIXTaS36ISwiZF4K9ShQNH0ifDtBniqnJNQe36ReCu96PGbZp+tz876Z rB8N2VvFY7Xh4KkklzkgBwrHxHgn+UH17nVQVK2U16K6xKfOm/PsGa3mUtGcme8xSQEl 9tBQjl+cPlZn1yiNESGS8Voo+gcK10DUiIlt3HtPlJv8CxwXIbn50v2ejvfrQYozTiIs 9Luw==
X-Received: by 10.66.65.234 with SMTP id a10mr10384633pat.165.1384300595258; Tue, 12 Nov 2013 15:56:35 -0800 (PST)
Received: from [10.33.22.215] (128-107-239-234.cisco.com. [128.107.239.234]) by mx.google.com with ESMTPSA id wp8sm40116162pbc.26.2013.11.12.15.56.32 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 12 Nov 2013 15:56:34 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Tue, 12 Nov 2013 15:56:30 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Michael Welzl <michawe@ifi.uio.no>, Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEA7FE2F.4B2C5%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <E47DE53D-DEFE-4203-A239-C7615E384E65@ifi.uio.no>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467116593_4601438"
Cc: "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2013 23:56:36 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467116593_4601438
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit



From:  Michael Welzl <michawe@ifi.uio.no>
Date:  Friday, November 8, 2013 5:30 PM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> 
> Thanks for the link. My point was just (along with Naeem) that it would be
> useful to compare PIE (or CoDel) against something else except CoDel and RED.
> When I just look at the graphs in the HPSR paper (sorry, didn't get to read it
> yet), again I see only PIE, CoDel, RED.
> 
> We tried ARED and were surprised to see how good it works - which is not to
> say that it is the ideal - but it was surprisingly often better than these
> newer schemes, despite its age of 10+ years... as we all know, there are many
> schemes out there, quite often designed with the goal of removing RED's
> dependence on parameters.

Michael, Naeem:

This is just a follow-up to better understand the ARED results presented at
AQM WG (http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf).

Could you please clarify whether the ARED parameters for the tests were set
as specified in the 2001 paper? I.e., when average queueing delay target =
1ms, the min and max thresholds would translate to 0.5ms and 1.5ms, and for
5 ms target the min and max thresh will be 2.5ms and 7.5ms, respectively,
right? Can you confirm?

Assuming ARED logic is intact, ARED drops all packets when average queue
size goes above max_thresh  (max_thresh = 1.5ms or 7.5ms etc.); this would
be semantically similar to a drop tail queue with shallow buffer (can be
confirmed by looking at cumulative packet drops by ARED).

This is probably why the delay values are tight for ARED, and why ARED
translates to poor utilization, especially for the lower target delay
values, as seen in your slides (#8, #11, #13).

Also,  wondering if bursty traffic was considered for evaluations, since
semantics of drop tail with shallow buffering doesn't accommodate bursty
traffic. 

Thanks,
Preethi & Rong




--B_3467116593_4601438
Content-type: text/html;
	charset="US-ASCII"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-famil=
y: Calibri, sans-serif; "><div style=3D"font-size: 14px; "><br></div><div styl=
e=3D"font-size: 14px; "><br></div><span id=3D"OLK_SRC_BODY_SECTION" style=3D"font-=
size: 14px; "><div style=3D"font-family:Calibri; font-size:11pt; text-align:le=
ft; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDI=
NG-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1=
pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style=3D"font-wei=
ght:bold">From: </span> Michael Welzl &lt;<a href=3D"mailto:michawe@ifi.uio.no=
">michawe@ifi.uio.no</a>&gt;<br><span style=3D"font-weight:bold">Date: </span>=
 Friday, November 8, 2013 5:30 PM<br><span style=3D"font-weight:bold">To: </sp=
an> Preethi Natarajan &lt;<a href=3D"mailto:preethi.cis@gmail.com">preethi.cis=
@gmail.com</a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Naeem Khade=
mi &lt;<a href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&=
gt;, "<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>" &lt;<a href=3D"mailto:aq=
m@ietf.org">aqm@ietf.org</a>&gt;<br><span style=3D"font-weight:bold">Subject: =
</span> Re: [aqm] AQM schemes: Queue length vs. delay based<br></div><div><b=
r></div><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LE=
FT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;"><div><div style=3D"word=
-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white=
-space; "><div><blockquote type=3D"cite"><div style=3D"word-wrap: break-word; -w=
ebkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14=
px; font-family: Calibri, sans-serif; "></div></blockquote><div><br></div>Th=
anks for the link. My point was just (along with Naeem) that it would be use=
ful to compare PIE (or CoDel) against something else except CoDel and RED. W=
hen I just look at the graphs in the HPSR paper (sorry, didn't get to read i=
t yet), again I see only PIE, CoDel, RED.</div><div><br></div><div>We tried =
ARED and were surprised to see how good it works - which is not to say that =
it is the ideal - but it was surprisingly often better than these newer sche=
mes, despite its age of 10+ years... as we all know, there are many schemes =
out there, quite often designed with the goal of removing RED's dependence o=
n parameters.</div></div></div></blockquote></span><div style=3D"font-size: 14=
px; "><br></div><div><div style=3D"color: rgb(0, 0, 0); font-family: Calibri, =
sans-serif; ">Michael, Naeem:</div><div style=3D"color: rgb(0, 0, 0); font-fam=
ily: Calibri, sans-serif; "><br></div><div style=3D"color: rgb(0, 0, 0); font-=
family: Calibri, sans-serif; ">This is just a follow-up to better understand=
 the ARED results presented at AQM WG (http://www.ietf.org/proceedings/88/sl=
ides/slides-88-iccrg-4.pdf).</div><div style=3D"color: rgb(0, 0, 0); font-fami=
ly: Calibri, sans-serif; "><br></div><div><font class=3D"Apple-style-span" fac=
e=3D"Calibri,sans-serif">Could you please clarify whether the ARED parameters =
for the tests were set as specified in the 2001 paper? I.e., when average qu=
eueing delay target =3D 1ms, the&nbsp;min&nbsp;</font><span style=3D"color: rgb(=
0, 0, 0); font-family: Calibri, sans-serif; ">and</span><span style=3D"color: =
rgb(0, 0, 0); font-family: Calibri, sans-serif; ">&nbsp;</span><font class=3D"=
Apple-style-span" face=3D"Calibri,sans-serif">max thresholds would translate t=
o 0.5ms and 1.5ms, and for 5 ms target the min and max thresh will be 2.5ms =
and 7.5ms, respectively, right?&nbsp;</font>Can you confirm?</div><div><font=
 class=3D"Apple-style-span" face=3D"Calibri,sans-serif"><br></font></div><div><f=
ont class=3D"Apple-style-span" face=3D"Calibri,sans-serif">Assuming ARED logic i=
s intact, ARED drops all packets when average queue size goes above max_thre=
sh &nbsp;</font>(max_thresh =3D 1.5ms or 7.5ms etc.); this would be semantical=
ly similar to a drop tail queue with shallow buffer (can be confirmed by loo=
king at cumulative packet drops by ARED).</div><div><font class=3D"Apple-style=
-span" face=3D"Calibri,sans-serif"><br></font></div><div><font class=3D"Apple-st=
yle-span" face=3D"Calibri,sans-serif">This is probably why the delay values ar=
e tight for ARED, and why ARED translates to poor utilization, especially fo=
r the lower target delay values, as seen in your slides (#8, #11, #13).&nbsp=
;</font></div><div><br></div><div>Also, &nbsp;wondering if bursty traffic wa=
s considered for evaluations, since semantics of drop tail with shallow buff=
ering doesn't accommodate bursty traffic.&nbsp;</div><div><font class=3D"Apple=
-style-span" face=3D"Calibri,sans-serif"><i><br></i></font></div><div>Thanks,<=
/div><div>Preethi &amp; Rong</div></div><span id=3D"OLK_SRC_BODY_SECTION"><div=
><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-b=
reak: after-white-space; "><div><br></div></div></div></span></body></html>

--B_3467116593_4601438--



From naeem.khademi@gmail.com  Wed Nov 13 05:01:26 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 92C4311E8187 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 05:01:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.251
X-Spam-Level: 
X-Spam-Status: No, score=-2.251 tagged_above=-999 required=5 tests=[AWL=0.121,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lF5fu8nhqWEj for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 05:01:25 -0800 (PST)
Received: from mail-vb0-x22a.google.com (mail-vb0-x22a.google.com [IPv6:2607:f8b0:400c:c02::22a]) by ietfa.amsl.com (Postfix) with ESMTP id 10F0C11E8186 for <aqm@ietf.org>; Wed, 13 Nov 2013 05:01:24 -0800 (PST)
Received: by mail-vb0-f42.google.com with SMTP id p14so249157vbm.1 for <aqm@ietf.org>; Wed, 13 Nov 2013 05:01:24 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=5AAixqBBT7DY+nJlVlv5l/2fMxPDj4Yx8Epq8RDPhnY=; b=LXabmmZovTBbBIcKxAE3CID3lXop3dh3Jf9hwdWnYkFIf4NKk7Ct+AZKJVIj2zfOIK DMC918+E0+3Z+vi2voCNvkb9sxyVKOu2cxHVzz221GVGxbnZTuKCS7S/qG3wcri6oM2c I/etNb7zoe0kKO3B1tZIoJ+Vhcgd3XtVvWGWKgxIIUAlg13rrkXzWUXaz+c6W+C2bB6E a9JkV8SO2lMxGE95Vv2dXt6FXFdMqpiLs6zO+p6TLzklArGdLa6oHOgRzAEM5cOI594f qhvPGyvGk+EZ2NxPVKmOAJRSoiRwnz/gQ5CZEd97xMiY/rC4Hi3gsjvunkLeOVXR7pyh +Srg==
MIME-Version: 1.0
X-Received: by 10.52.119.198 with SMTP id kw6mr75754vdb.47.1384347684603; Wed, 13 Nov 2013 05:01:24 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Wed, 13 Nov 2013 05:01:24 -0800 (PST)
In-Reply-To: <CEA7FE2F.4B2C5%prenatar@cisco.com>
References: <E47DE53D-DEFE-4203-A239-C7615E384E65@ifi.uio.no> <CEA7FE2F.4B2C5%prenatar@cisco.com>
Date: Wed, 13 Nov 2013 14:01:24 +0100
Message-ID: <CAEjQQ5Vif73gWe-35nzbhmPH+Eh+gSZK+7xNm33+T-FVNsmPZw@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=089e013a1e7afc918e04eb0e8f0e
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 13:01:26 -0000

--089e013a1e7afc918e04eb0e8f0e
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On Wed, Nov 13, 2013 at 12:56 AM, Preethi Natarajan
<preethi.cis@gmail.com>wrote:

>
>
> From: Michael Welzl <michawe@ifi.uio.no>
> Date: Friday, November 8, 2013 5:30 PM
>
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org=
>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
> Thanks for the link. My point was just (along with Naeem) that it would b=
e
> useful to compare PIE (or CoDel) against something else except CoDel and
> RED. When I just look at the graphs in the HPSR paper (sorry, didn't get =
to
> read it yet), again I see only PIE, CoDel, RED.
>
> We tried ARED and were surprised to see how good it works - which is not
> to say that it is the ideal - but it was surprisingly often better than
> these newer schemes, despite its age of 10+ years... as we all know, ther=
e
> are many schemes out there, quite often designed with the goal of removin=
g
> RED's dependence on parameters.
>
>
>
Hi Preethi and Rong

replies follow inline


> Michael, Naeem:
>
> This is just a follow-up to better understand the ARED results presented
> at AQM WG (http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pd=
f
> ).
>
> Could you please clarify whether the ARED parameters for the tests were
> set as specified in the 2001 paper? I.e., when average queueing delay
> target =3D 1ms, the min and max thresholds would translate to 0.5ms and
> 1.5ms, and for 5 ms target the min and max thresh will be 2.5ms and 7.5ms=
,
> respectively, right? Can you confirm?
>

Copy/pasting from Section 3.1.3 of http://urn.nb.no/URN:NBN:no-38868
<http://urn.nb.no/URN:NBN:no-38868>

*"In case of ARED, the parameters are calculated based on the
recommendations in [19]*
*except when target delay=3D1 ms where th_min and th_max are set to 1 MTU*
*and 2 MTUs accordingly."*

*[19] S. Floyd, R. Gummadi, and S. Shenker. Adaptive RED: An Algorithm for*
*Increasing the Robustness of RED=92s Active Queue Management. Technical*
*report, 2001.*

This means "yes" for all values of target delay except for 1 ms cases where
the thresholds become smaller than a single MTU transmission time on the 10
Mbps bottleneck link used in this case.  The recommendations in Sally
Flyod' s ARED 2001  (th_max=3D3*th_min; target=3D(th_max+th_min)/2) are use=
d in
all other cases.


> Assuming ARED logic is intact, ARED drops all packets when average queue
> size goes above max_thresh  (max_thresh =3D 1.5ms or 7.5ms etc.); this
> would be semantically similar to a drop tail queue with shallow buffer (c=
an
> be confirmed by looking at cumulative packet drops by ARED).
>

Similar but not identical as ARED (and any other *RED) does averaging
instead of using instantaneous queue size (e.g. that being employed in
DCTCP AQM)


> This is probably why the delay values are tight for ARED, and why ARED
> translates to poor utilization, especially for the lower target delay
> values, as seen in your slides (#8, #11, #13).
>

True!


> Also,  wondering if bursty traffic was considered for evaluations, since
> semantics of drop tail with shallow buffering doesn't accommodate bursty
> traffic.
>

No, we did not consider bursty traffic (e.g. multimedia, etc) for this work
as this was our first (initial) fundamental step to understand in a
systematic way how AQMs work in real-life with long-lived TCP traffic
(which is the most common type of traffic that fills up any buffer and
consistently contributes to excessively long latencies experienced on the
Internet (a.k.a bufferbloat)).

The fact that AQMs *should* absorb sub-RTT bursts (due to rate mistmatch,
etc) does not diminish the basic requirement for AQMs that they should
control the latency induced by bulk (long-lived) TCP traffic in a well
manner. Any other type of traffic on the access links will most likely to
coexist with bulk TCP traffic when latency is observed to grow high. In
other words, if an AQM can't control the latency of (not so bursty as
others-) TCP traffic, it probably won't be able to do so for bursty traffic
as well.


> Thanks,
> Preethi & Rong
>
>
Cheers,
Naeem

--089e013a1e7afc918e04eb0e8f0e
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Wed, Nov 13, 2013 at 12:56 AM, Preethi Natarajan <span dir=3D"lt=
r">&lt;<a href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.c=
is@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div style=3D"font-family:Calibri,sans-serif;word-wrap:bre=
ak-word">
<div style=3D"font-size:14px"><br></div><div style=3D"font-size:14px"><br><=
/div><span style=3D"font-size:14px"><div style=3D"border-width:1pt medium m=
edium;border-style:solid none none;padding:3pt 0in 0in;text-align:left;font=
-size:11pt;font-family:Calibri;border-top-color:rgb(181,196,223)">
<span style=3D"font-weight:bold">From: </span> Michael Welzl &lt;<a href=3D=
"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no</a>&gt;<br=
><span style=3D"font-weight:bold">Date: </span> Friday, November 8, 2013 5:=
30 PM<div class=3D"im">
<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a h=
ref=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.co=
m</a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Naeem Khademi &lt=
;<a href=3D"mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi=
@gmail.com</a>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank"=
>aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org" target=3D"_blan=
k">aqm@ietf.org</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Qu=
eue length vs. delay based<br></div></div><div><br></div><blockquote style=
=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div><div s=
tyle=3D"word-wrap:break-word">
<div><blockquote type=3D"cite"><div style=3D"word-wrap:break-word;font-size=
:14px;font-family:Calibri,sans-serif"></div></blockquote><div><br></div><di=
v class=3D"im">Thanks for the link. My point was just (along with Naeem) th=
at it would be useful to compare PIE (or CoDel) against something else exce=
pt CoDel and RED. When I just look at the graphs in the HPSR paper (sorry, =
didn&#39;t get to read it yet), again I see only PIE, CoDel, RED.</div>
</div><div class=3D"im"><div><br></div><div>We tried ARED and were surprise=
d to see how good it works - which is not to say that it is the ideal - but=
 it was surprisingly often better than these newer schemes, despite its age=
 of 10+ years... as we all know, there are many schemes out there, quite of=
ten designed with the goal of removing RED&#39;s dependence on parameters.<=
/div>
</div></div></div></blockquote></span><div style=3D"font-size:14px"><br></d=
iv></div></blockquote><div><br></div><div style>Hi Preethi and Rong</div><d=
iv style><br></div><div style>replies follow inline</div><div>=A0</div><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left=
-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;paddi=
ng-left:1ex">
<div style=3D"font-family:Calibri,sans-serif;word-wrap:break-word"><div sty=
le=3D"font-size:14px"></div><div><div style=3D"font-family:Calibri,sans-ser=
if">Michael, Naeem:</div><div style=3D"font-family:Calibri,sans-serif"><br>=
</div>
<div style=3D"font-family:Calibri,sans-serif">This is just a follow-up to b=
etter understand the ARED results presented at AQM WG (<a href=3D"http://ww=
w.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf" target=3D"_blank">h=
ttp://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf</a>).</div>
<div style=3D"font-family:Calibri,sans-serif"><br></div><div><font face=3D"=
Calibri,sans-serif">Could you please clarify whether the ARED parameters fo=
r the tests were set as specified in the 2001 paper? I.e., when average que=
ueing delay target =3D 1ms, the=A0min=A0</font><span style=3D"font-family:C=
alibri,sans-serif">and</span><span style=3D"font-family:Calibri,sans-serif"=
>=A0</span><font face=3D"Calibri,sans-serif">max thresholds would translate=
 to 0.5ms and 1.5ms, and for 5 ms target the min and max thresh will be 2.5=
ms and 7.5ms, respectively, right?=A0</font>Can you confirm?</div>
</div></div></blockquote><div><br></div><div style>Copy/pasting from Sectio=
n 3.1.3 of=A0<a href=3D"http://urn.nb.no/URN:NBN:no-38868" target=3D"_blank=
" style=3D"font-size:13px;margin:0px;padding:0px;vertical-align:baseline;ba=
ckground-color:transparent;color:rgb(96,120,144);font-family:sans-serif;lin=
e-height:20px;text-align:justify">http://urn.nb.no/URN:NBN:no-38868=A0</a><=
/div>
<div style><br></div><div style><div><b>&quot;In case of ARED, the paramete=
rs are calculated based on the recommendations in [19]</b></div><div><b>exc=
ept when target delay=3D1 ms where th_min and th_max are set to 1 MTU</b></=
div>
<div><b>and 2 MTUs accordingly.&quot;</b></div><div><br></div><div><div><b>=
[19] S. Floyd, R. Gummadi, and S. Shenker. Adaptive RED: An Algorithm for</=
b></div><div><b>Increasing the Robustness of RED=92s Active Queue Managemen=
t. Technical</b></div>
<div><b>report, 2001.</b></div></div><div><br></div><div style>This means &=
quot;yes&quot; for all values of target delay except for 1 ms cases where t=
he thresholds become smaller than a single MTU transmission time on the 10 =
Mbps bottleneck link used in this case. =A0The recommendations in Sally Fly=
od&#39; s ARED 2001 =A0(th_max=3D3*th_min; target=3D(th_max+th_min)/2) are =
used in all other cases.=A0</div>
</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);borde=
r-left-style:solid;padding-left:1ex"><div style=3D"font-family:Calibri,sans=
-serif;word-wrap:break-word">
<div><div><font face=3D"Calibri,sans-serif">Assuming ARED logic is intact, =
ARED drops all packets when average queue size goes above max_thresh =A0</f=
ont>(max_thresh =3D 1.5ms or 7.5ms etc.); this would be semantically simila=
r to a drop tail queue with shallow buffer (can be confirmed by looking at =
cumulative packet drops by ARED).</div>
</div></div></blockquote><div><br></div><div style>Similar but not identica=
l as ARED (and any other *RED) does averaging instead of using=A0instantane=
ous=A0queue size (e.g. that being employed in DCTCP AQM)=A0</div><div>=A0</=
div>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div style=3D"font-family:Calibri,sans-serif;word-wrap:bre=
ak-word">
<div><div><font face=3D"Calibri,sans-serif">This is probably why the delay =
values are tight for ARED, and why ARED translates to poor utilization, esp=
ecially for the lower target delay values, as seen in your slides (#8, #11,=
 #13).=A0</font></div>
</div></div></blockquote><div><br></div><div style>True!=A0</div><div>=A0</=
div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:sol=
id;padding-left:1ex">
<div style=3D"font-family:Calibri,sans-serif;word-wrap:break-word"><div><di=
v>Also, =A0wondering if bursty traffic was considered for evaluations, sinc=
e semantics of drop tail with shallow buffering doesn&#39;t accommodate bur=
sty traffic.=A0</div>
</div></div></blockquote><div><br></div><div style>No, we did not consider =
bursty traffic (e.g. multimedia, etc) for this work as this was our first (=
initial) fundamental step to understand in a systematic way how AQMs work i=
n real-life with long-lived TCP traffic (which is the most common type of t=
raffic that fills up any buffer and consistently contributes to excessively=
 long latencies experienced on the Internet (a.k.a bufferbloat)).=A0</div>
<div style><br></div><div style>The fact that AQMs *should* absorb sub-RTT =
bursts (due to rate mistmatch, etc) does not diminish the basic requirement=
 for AQMs that they should control the latency induced by bulk (long-lived)=
 TCP traffic in a well manner. Any other type of traffic on the access link=
s will most likely to coexist with bulk TCP traffic when latency is observe=
d to grow high. In other words, if an AQM can&#39;t control the latency of =
(not so bursty as others-) TCP traffic, it probably won&#39;t be able to do=
 so for bursty traffic as well. =A0 =A0 =A0=A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px=
 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left=
-style:solid;padding-left:1ex"><div style=3D"font-family:Calibri,sans-serif=
;word-wrap:break-word">
<div><div>Thanks,</div><div>Preethi &amp; Rong</div></div><span><div><div s=
tyle=3D"word-wrap:break-word"><div><br></div></div></div></span></div></blo=
ckquote><div><br></div><div style>Cheers,</div><div style>Naeem</div><div>
=A0</div></div><br></div></div>

--089e013a1e7afc918e04eb0e8f0e--

From fred@cisco.com  Wed Nov 13 09:50:52 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B830121E8151 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 09:50:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.33
X-Spam-Level: 
X-Spam-Status: No, score=-110.33 tagged_above=-999 required=5 tests=[AWL=0.042, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qJdZmNRNnjx0 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 09:50:47 -0800 (PST)
Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76]) by ietfa.amsl.com (Postfix) with ESMTP id 03DF821E80BE for <aqm@ietf.org>; Wed, 13 Nov 2013 09:50:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3825; q=dns/txt; s=iport; t=1384365047; x=1385574647; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=FWmlX2R9LdctnpSRvZkHVR/C5pfkQX29F2X/YkpN6R0=; b=RsoXGsFR/sPeJBu0fJz2rIMchxMdbdp9bVhgIsWtdB+FJ+wiFFJyChAj CDucUN47GvXxBLiFS5KOEedcSDVVDK0C3GmUenMNY44r8O+5Sb5fImv1y lSCfqOWSSS1gIYSZTEFVOmXQtlCBoNZ/KKw3yd8kSLQlUOKIfCPAWG5hm c=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgoFAFm7g1KtJXG+/2dsb2JhbABbgweBC78rgSYWdIIlAQEBAwF5BQsCAQgOODIlAgQOBQ6HbQbAIY9fB4MggREDkDCBMIYwkguDKIIq
X-IronPort-AV: E=Sophos;i="4.93,693,1378857600";  d="asc'?scan'208";a="284350905"
Received: from rcdn-core2-3.cisco.com ([173.37.113.190]) by rcdn-iport-5.cisco.com with ESMTP; 13 Nov 2013 17:50:46 +0000
Received: from xhc-aln-x05.cisco.com (xhc-aln-x05.cisco.com [173.36.12.79]) by rcdn-core2-3.cisco.com (8.14.5/8.14.5) with ESMTP id rADHokm5014165 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 13 Nov 2013 17:50:46 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x05.cisco.com ([173.36.12.79]) with mapi id 14.03.0123.003; Wed, 13 Nov 2013 11:50:46 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Mikael Abrahamsson <swmike@swm.pp.se>
Thread-Topic: [Bloat] [aqm]  [iccrg]  AQM deployment status?
Thread-Index: AQHO4JjgklOmh8fMlEaXf65FqFuYEA==
Date: Wed, 13 Nov 2013 17:50:45 +0000
Message-ID: <BDD8F198-E471-4703-AE98-36970AF140F1@cisco.com>
References: <CAEjQQ5U2atKwXvucMKNHsRZ29RinpKMN4rZEvUGOAqNxxEvLBg@mail.gmail.com> <C0611F522A6FA9498A044C36073E49656D9185F2@US70UWXCHMBA01.zam.alcatel-lucent.com> <FB9F71A5-C19E-405D-A598-BC704C38B3AA@netapp.com> <20130925143510.GA6191@sesse.net> <C0611F522A6FA9498A044C36073E49656D91899C@US70UWXCHMBA01.zam.alcatel-lucent.com> <alpine.DEB.2.02.1309252118590.32315@uplift.swm.pp.se>
In-Reply-To: <alpine.DEB.2.02.1309252118590.32315@uplift.swm.pp.se>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.121]
Content-Type: multipart/signed; boundary="Apple-Mail=_194E5E8C-8EF8-48F9-B814-C7B192134F7F"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: "iccrg@irtf.org" <iccrg@irtf.org>, "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [aqm] [Bloat]   [iccrg]  AQM deployment status?
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 17:50:52 -0000

--Apple-Mail=_194E5E8C-8EF8-48F9-B814-C7B192134F7F
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


On Sep 25, 2013, at 12:24 PM, Mikael Abrahamsson <swmike@swm.pp.se> =
wrote:

> For higher end platforms, for instance all cisco CPU based routers =
(for some value of "all") can be configured with RED, fair-queue or =
similar, but they come with FIFO as default. This has been the same way =
since at least the mid 90ties as far as I know, long way back to cisco =
1600 device etc.
>=20
> Higher end Cisco equipment such as ASR9k, 12000, CRS etc, all support =
WRED, and here it makes sense since they all have ~50ms worth of =
buffering or more. They also come with FIFO as default setting.

Yes. There are two reasons that we don't deploy RED by default.=20

One is that we operate on a principle we call the "principle of least =
surprise"; we try to not change our customer's configurations by =
changing the defaults, and at the time I wrote the RED/WRED code the =
default was FIFO. Yes, that was quite some time ago, and one could =
imagine newer equipment coming out with different defaults on =
deployment, but that's not what the various business units do.=20

The other is this matter of configuration. In the late 1990's, Cisco =
employed Van and Kathy, and asked them to figure out autoconfiguration =
values. That didn't work out. I don't say that as a slam on V+K; it's =
simply a fact. They started by recommending an alternative algorithm =
called RED-Lite (don't bother googling that; you get discoth=E8ques), =
and are now recommending CoDel. Now, there is probably a simplistic =
value that one could throw in, such as setting max-threshold to the =
queue memory allocated to the queue and min-threshold to some function =
logarithmically related to the bit rate that serves as an estimator of =
the number of bit-times of delay to tune to. That would probably be =
better than nothing, but it would be nice to have some science behind =
it.

That brings us back to the auto-tuning discussion.

To my way of thinking, a simplistic algorithm such as that logarithmic =
approach or a lookup table that starts from (perhaps) bit rate and =
neighbor ping RTT can come up with an initial set of parameters that the =
algorithm's own processes can modify to ambient traffic behavior. =
Kathy's simulations in the study I mentioned suggested that a 1.5 MBPS =
link might do well with min-threshold at 30 ms (e.g., ~4 MTU-sized =
messages or a larger number of smaller messages), and rates higher at =
some single digit number of ms, decreasing as the speed increased.=20

CoDel suggests a flat 5 ms value (at 1.5 MBPS, less than a single MTU, =
and at one gigabit, 83 12K bit messages). I could imagine the =
initialization algorithm selecting 5 ms above a certain speed, and the =
equivalent of 4 MTU intervals at lower line speeds where 4*MTU exceeds 5 =
ms. I could imagine a related algorithm then adjusting that initial =
value to interact well with Google's IW=3D10 behavior or whatever else =
it finds on the link.=20

PIE would probably start with a similar set of values, and tune its =
mark/drop interval to reach a value that works well with ambient =
traffic.

--Apple-Mail=_194E5E8C-8EF8-48F9-B814-C7B192134F7F
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSg7vzbjEdbHIsm0MRAn0LAKCYyZyvzzM2JBDXaIqFzi2kqbUNwACdEeC8
vzgqNpjQSxhvsHrVxVlOG8U=
=JLqe
-----END PGP SIGNATURE-----

--Apple-Mail=_194E5E8C-8EF8-48F9-B814-C7B192134F7F--

From preethi.cis@gmail.com  Wed Nov 13 10:25:29 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7284D21F9E44 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 10:25:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PHgeZRixiCv4 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 10:25:25 -0800 (PST)
Received: from mail-pb0-x235.google.com (mail-pb0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) by ietfa.amsl.com (Postfix) with ESMTP id F35C821E8177 for <aqm@ietf.org>; Wed, 13 Nov 2013 10:25:24 -0800 (PST)
Received: by mail-pb0-f53.google.com with SMTP id ma3so803692pbc.12 for <aqm@ietf.org>; Wed, 13 Nov 2013 10:25:24 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=JSvRhcg/CCn/G4uGrzplxROuKIFlsL8QwQqkt6LGH50=; b=0ppniHVJnu1ejC32j8nBfmYavIxE8aDhZeDBDb8w0hk+eu2OCmYketN2TNlA/FIhAO kODEhckbgltE8+6pPipI/iQDDXm0IsmMhoQ1SFsRJGXVOBVG8vp9rVOeGLxQWdymuhdy BjsWNlwe+68Q4hd8zJ0aG2y2D3yY8PB94YAMP3UKvbROKhp6zHJk5G3S7LwYXYzp+RXQ i5LCrCu5Bgqw1OXzdoEtnZKhL8nKD3b8CrXOYtOo2JBZNwy0EJeVYKqwzeHPZKXf7AZN 6NwWNEaEUS7VQAifbhtMn+KkqVze6sX18vhjbu5NTeCb87QZk0D6a1LB7MjFvBs3bd50 ZFDQ==
X-Received: by 10.68.229.167 with SMTP id sr7mr15388249pbc.159.1384367124444;  Wed, 13 Nov 2013 10:25:24 -0800 (PST)
Received: from [10.21.76.42] (128-107-239-233.cisco.com. [128.107.239.233]) by mx.google.com with ESMTPSA id tu6sm45954948pbc.41.2013.11.13.10.25.21 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 13 Nov 2013 10:25:23 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Wed, 13 Nov 2013 10:25:19 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEA9034B.4B37C%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CAEjQQ5Vif73gWe-35nzbhmPH+Eh+gSZK+7xNm33+T-FVNsmPZw@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467183123_5208675"
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 18:25:29 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467183123_5208675
Content-type: text/plain;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

Please see inline=8A

From:  Naeem Khademi <naeem.khademi@gmail.com>
Date:  Wednesday, November 13, 2013 5:01 AM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> =20
>> Michael, Naeem:
>>=20
>> This is just a follow-up to better understand the ARED results presented=
 at
>> AQM WG (http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf)=
.
>>=20
>> Could you please clarify whether the ARED parameters for the tests were =
set
>> as specified in the 2001 paper? I.e., when average queueing delay target=
 =3D
>> 1ms, the min and max thresholds would translate to 0.5ms and 1.5ms, and =
for 5
>> ms target the min and max thresh will be 2.5ms and 7.5ms, respectively,
>> right? Can you confirm?
>=20
> Copy/pasting from Section 3.1.3 of http://urn.nb.no/URN:NBN:no-38868
> <http://urn.nb.no/URN:NBN:no-38868>
>=20
> "In case of ARED, the parameters are calculated based on the recommendati=
ons
> in [19]
> except when target delay=3D1 ms where th_min and th_max are set to 1 MTU
> and 2 MTUs accordingly."
>=20
> [19] S. Floyd, R. Gummadi, and S. Shenker. Adaptive RED: An Algorithm for
> Increasing the Robustness of RED=B9s Active Queue Management. Technical
> report, 2001.
>=20
> This means "yes" for all values of target delay except for 1 ms cases whe=
re
> the thresholds become smaller than a single MTU transmission time on the =
10
> Mbps bottleneck link used in this case.  The recommendations in Sally Fly=
od' s
> ARED 2001  (th_max=3D3*th_min; target=3D(th_max+th_min)/2) are used in all ot=
her
> cases.=20
> =20
>> Assuming ARED logic is intact, ARED drops all packets when average queue=
 size
>> goes above max_thresh  (max_thresh =3D 1.5ms or 7.5ms etc.); this would be
>> semantically similar to a drop tail queue with shallow buffer (can be
>> confirmed by looking at cumulative packet drops by ARED).
>=20
> Similar but not identical as ARED (and any other *RED) does averaging ins=
tead
> of using instantaneous queue size (e.g. that being employed in DCTCP AQM)
> =20
>> This is probably why the delay values are tight for ARED, and why ARED
>> translates to poor utilization, especially for the lower target delay va=
lues,
>> as seen in your slides (#8, #11, #13).
>=20
> True!=20
> =20

Thanks for clarifying. That definitely clears the air about why ARED's dela=
y
values were so tight; trying to maintain the average queue delay between
such a narrow band between min and max thresholds (0.5ms to 1.5ms or 2.5ms
to 7.5ms) would naturally result in tighter delay values at the cost of
utilization. Of course, ARED works on the average queue size, but when it
comes to such a narrow band between min and max thresholds, we suspect ther=
e
wouldn't be much difference between ARED and drop tail.


>> Also,  wondering if bursty traffic was considered for evaluations, since
>> semantics of drop tail with shallow buffering doesn't accommodate bursty
>> traffic.=20
>=20
> No, we did not consider bursty traffic (e.g. multimedia, etc) for this wo=
rk as
> this was our first (initial) fundamental step to understand in a systemat=
ic
> way how AQMs work in real-life with long-lived TCP traffic (which is the =
most
> common type of traffic that fills up any buffer and consistently contribu=
tes
> to excessively long latencies experienced on the Internet (a.k.a
> bufferbloat)).=20
>=20
> The fact that AQMs *should* absorb sub-RTT bursts (due to rate mistmatch,=
 etc)
> does not diminish the basic requirement for AQMs that they should control=
 the
> latency induced by bulk (long-lived) TCP traffic in a well manner. Any ot=
her
> type of traffic on the access links will most likely to coexist with bulk=
 TCP
> traffic when latency is observed to grow high. In other words, if an AQM =
can't
> control the latency of (not so bursty as others-) TCP traffic, it probabl=
y
> won't be able to do so for bursty traffic as well.


I would not limit an AQM design to absorb just sub-RTT bursts. A robust AQM
scheme should be able to handle any kind of bursts irrespective of RTT,
traffic patterns etc., since these bursts are quite common in deployments
today.

Thanks,
Preethi




--B_3467183123_5208675
Content-type: text/html;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:=
 14px; font-family: Calibri, sans-serif; "><div>Please see inline&#8230;</di=
v><div><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div style=3D"font-family:Cal=
ibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium no=
ne; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDIN=
G-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADD=
ING-TOP: 3pt"><span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt=
;<a href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br=
><span style=3D"font-weight:bold">Date: </span> Wednesday, November 13, 2013 5=
:01 AM<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<=
a href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span=
 style=3D"font-weight:bold">Cc: </span> Michael Welzl &lt;<a href=3D"mailto:mich=
awe@ifi.uio.no">michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailto:aqm@ietf.org">a=
qm@ietf.org</a>" &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><=
span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue =
length vs. delay based<br></div><div><br></div><blockquote id=3D"MAC_OUTLOOK_A=
TTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;=
 MARGIN:0 0 0 5;"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_=
quote"><div>&nbsp;</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-=
left-style:solid;padding-left:1ex"><div style=3D"font-family:Calibri,sans-seri=
f;word-wrap:break-word"><div style=3D"font-size:14px"></div><div><div style=3D"f=
ont-family:Calibri,sans-serif">Michael, Naeem:</div><div style=3D"font-family:=
Calibri,sans-serif"><br></div><div style=3D"font-family:Calibri,sans-serif">Th=
is is just a follow-up to better understand the ARED results presented at AQ=
M WG (<a href=3D"http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.p=
df" target=3D"_blank">http://www.ietf.org/proceedings/88/slides/slides-88-iccr=
g-4.pdf</a>).</div><div style=3D"font-family:Calibri,sans-serif"><br></div><di=
v><font face=3D"Calibri,sans-serif">Could you please clarify whether the ARED =
parameters for the tests were set as specified in the 2001 paper? I.e., when=
 average queueing delay target =3D 1ms, the&nbsp;min&nbsp;</font><span style=3D"=
font-family: Calibri, sans-serif; ">and</span><span style=3D"font-family: Cali=
bri, sans-serif; ">&nbsp;</span><font face=3D"Calibri,sans-serif">max threshol=
ds would translate to 0.5ms and 1.5ms, and for 5 ms target the min and max t=
hresh will be 2.5ms and 7.5ms, respectively, right?&nbsp;</font>Can you conf=
irm?</div></div></div></blockquote><div><br></div><div style=3D"">Copy/pasting=
 from Section 3.1.3 of&nbsp;<a href=3D"http://urn.nb.no/URN:NBN:no-38868" targ=
et=3D"_blank" style=3D"font-size:13px;margin:0px;padding:0px;vertical-align:base=
line;background-color:transparent;color:rgb(96,120,144);font-family:sans-ser=
if;line-height:20px;text-align:justify">http://urn.nb.no/URN:NBN:no-38868&nb=
sp;</a></div><div style=3D""><br></div><div style=3D""><div><b>"In case of ARED,=
 the parameters are calculated based on the recommendations in [19]</b></div=
><div><b>except when target delay=3D1 ms where th_min and th_max are set to 1 =
MTU</b></div><div><b>and 2 MTUs accordingly."</b></div><div><br></div><div><=
div><b>[19] S. Floyd, R. Gummadi, and S. Shenker. Adaptive RED: An Algorithm=
 for</b></div><div><b>Increasing the Robustness of RED&#8217;s Active Queue =
Management. Technical</b></div><div><b>report, 2001.</b></div></div><div><br=
></div><div style=3D"">This means "yes" for all values of target delay except =
for 1 ms cases where the thresholds become smaller than a single MTU transmi=
ssion time on the 10 Mbps bottleneck link used in this case. &nbsp;The recom=
mendations in Sally Flyod' s ARED 2001 &nbsp;(th_max=3D3*th_min; target=3D(th_ma=
x+th_min)/2) are used in all other cases.&nbsp;</div></div><div>&nbsp;</div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-=
width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding=
-left:1ex"><div style=3D"font-family:Calibri,sans-serif;word-wrap:break-word">=
<div><div><font face=3D"Calibri,sans-serif">Assuming ARED logic is intact, ARE=
D drops all packets when average queue size goes above max_thresh &nbsp;</fo=
nt>(max_thresh =3D 1.5ms or 7.5ms etc.); this would be semantically similar to=
 a drop tail queue with shallow buffer (can be confirmed by looking at cumul=
ative packet drops by ARED).</div></div></div></blockquote><div><br></div><d=
iv style=3D"">Similar but not identical as ARED (and any other *RED) does aver=
aging instead of using&nbsp;instantaneous&nbsp;queue size (e.g. that being e=
mployed in DCTCP AQM)&nbsp;</div><div>&nbsp;</div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-colo=
r:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style=3D"fon=
t-family:Calibri,sans-serif;word-wrap:break-word"><div><div><font face=3D"Cali=
bri,sans-serif">This is probably why the delay values are tight for ARED, an=
d why ARED translates to poor utilization, especially for the lower target d=
elay values, as seen in your slides (#8, #11, #13).&nbsp;</font></div></div>=
</div></blockquote><div><br></div><div style=3D"">True!&nbsp;</div><div>&nbsp;=
</div></div></div></div></blockquote></span><div><br></div><div>Thanks for c=
larifying. That definitely clears the air about why ARED's delay values were=
 so tight; trying to maintain the average queue delay between such a narrow =
band between min and max thresholds (0.5ms to 1.5ms or 2.5ms to 7.5ms) would=
 naturally result in tighter delay values at the cost of utilization. Of cou=
rse, ARED works on the average queue size, but when it comes to such a narro=
w band between min and max thresholds, we suspect there wouldn't be much dif=
ference between ARED and drop tail.</div><div><br></div><div><br></div><span=
 id=3D"OLK_SRC_BODY_SECTION"><blockquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOT=
E" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;"><d=
iv dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;bor=
der-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><d=
iv style=3D"font-family:Calibri,sans-serif;word-wrap:break-word"><div><div>Als=
o, &nbsp;wondering if bursty traffic was considered for evaluations, since s=
emantics of drop tail with shallow buffering doesn't accommodate bursty traf=
fic.&nbsp;</div></div></div></blockquote><div><br></div><div style=3D"">No, we=
 did not consider bursty traffic (e.g. multimedia, etc) for this work as thi=
s was our first (initial) fundamental step to understand in a systematic way=
 how AQMs work in real-life with long-lived TCP traffic (which is the most c=
ommon type of traffic that fills up any buffer and consistently contributes =
to excessively long latencies experienced on the Internet (a.k.a bufferbloat=
)).&nbsp;</div><div style=3D""><br></div><div style=3D"">The fact that AQMs *sho=
uld* absorb sub-RTT bursts (due to rate mistmatch, etc) does not diminish th=
e basic requirement for AQMs that they should control the latency induced by=
 bulk (long-lived) TCP traffic in a well manner. Any other type of traffic o=
n the access links will most likely to coexist with bulk TCP traffic when la=
tency is observed to grow high. In other words, if an AQM can't control the =
latency of (not so bursty as others-) TCP traffic, it probably won't be able=
 to do so for bursty traffic as well. &nbsp; &nbsp; &nbsp;</div></div></div>=
</div></blockquote></span><div><br></div><div><br></div><div>I would not lim=
it an AQM design to absorb just sub-RTT bursts. A robust AQM scheme should b=
e able to handle any kind of bursts irrespective of RTT, traffic patterns et=
c., since these bursts are quite common in deployments today.</div><div><br>=
</div><div>Thanks,</div><div>Preethi</div><div><br></div></body></html>

--B_3467183123_5208675--



From fred@cisco.com  Wed Nov 13 10:49:05 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 633C011E8120; Wed, 13 Nov 2013 10:49:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.033
X-Spam-Level: 
X-Spam-Status: No, score=-110.033 tagged_above=-999 required=5 tests=[AWL=-0.261, BAYES_00=-2.599, J_CHICKENPOX_16=0.6, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YesKSumrdCJu; Wed, 13 Nov 2013 10:48:59 -0800 (PST)
Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) by ietfa.amsl.com (Postfix) with ESMTP id 36F8221F9D7C; Wed, 13 Nov 2013 10:48:59 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4521; q=dns/txt; s=iport; t=1384368539; x=1385578139; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=FU2+omyjlRJJaO4P0SSQ56FmlEMzHm8t5qjGEDM+PVQ=; b=JgyKBja1fL/ftJWu0FJSM06olEX2gAYtnkg+wxMtql4Dk1kqRzjcNHL6 ILAd2ZogKa8Iy6/73kxrFmvTh1LxWNokHHmEow+osR310uUh/PNp6t+An POBfVIUioaPmBXpPnQUhGOq0AiHtHF490lteGM0mqVmckNrMKTtq9usNm 0=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgsFAD7Ig1KtJV2b/2dsb2JhbABbgwc4U78rgScWdIIlAQEBAwF5BQsCAQhGMiUCBA4FDguHYgYNwBePXweDIIERA5AwgTCGMJILgyiCKg
X-IronPort-AV: E=Sophos;i="4.93,693,1378857600";  d="asc'?scan'208";a="284191058"
Received: from rcdn-core-4.cisco.com ([173.37.93.155]) by rcdn-iport-1.cisco.com with ESMTP; 13 Nov 2013 18:48:57 +0000
Received: from xhc-aln-x07.cisco.com (xhc-aln-x07.cisco.com [173.36.12.81]) by rcdn-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id rADImuWx002728 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 13 Nov 2013 18:48:56 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x07.cisco.com ([173.36.12.81]) with mapi id 14.03.0123.003; Wed, 13 Nov 2013 12:48:56 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: "Scheffenegger, Richard" <rs@netapp.com>
Thread-Topic: [tsvwg] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO4KEBV7BAWUD5Q0OJOx469YFWCQ==
Date: Wed, 13 Nov 2013 18:48:56 +0000
Message-ID: <E7D77F29-36BD-45FD-AE22-D90F2A687964@cisco.com>
References: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk> <CEA65B19.21ED5%g.white@cablelabs.com> <012C3117EDDB3C4781FD802A8C27DD4F25E8908F@SACEXCMBX02-PRD.hq.netapp.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25E8908F@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.121]
Content-Type: multipart/signed; boundary="Apple-Mail=_3DBFD422-2C1D-4CE8-A19B-BE57478A0706"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Greg White <g.white@CableLabs.com>, tsvwg IETF list <tsvwg@ietf.org>, tsv-area IETF list <tsv-area@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 18:49:05 -0000

--Apple-Mail=_3DBFD422-2C1D-4CE8-A19B-BE57478A0706
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


On Nov 12, 2013, at 5:18 AM, "Scheffenegger, Richard" <rs@netapp.com> =
wrote:

> So, as I am not a HW guy, how many gates does one need for (W)RED, =
PIE, CoDel?

I'm not a hardware guy either, but I think a predictor would be the =
registers and calculations implied in the data path (a supporting =
software/firmware process can use RISC approaches to implement =
non-data-path components). The calculations of importance are =
multiplies, divides, and square roots, each of which can take an =
interesting amount of silicon if literally implemented in hardware.=20

For the hardware engineer, there are a number of analogs that can be =
used to simplify life. For example, where an algorithm calls for some =
number of milliseconds, s/he can divide down the clock rate of the chip =
and consider that to be a stated number of clock ticks rather than =
implementing a millisecond clock. Where an algorithm calls for a random =
number, there are ways the engineer can use noise to create it =
(http://en.wikipedia.org/wiki/Hardware_random_number_generator).

With RED, there are two basic calculations: maintaining the mean queue =
depth, which might mean an exponentially-weighted moving average with =
samples taken on events or at times, and determining the mark/drop =
probability as max(0, =
(mean_queue_depth-min_threshold)/(max_threshold-min_threshold)). An EWMA =
can be implemented using adds and shifts, so it's not a big deal. I'm =
not the guy that has done our chips, so I'm speaking out of turn, but I =
would expect that the divide in the mark/drop probability can be =
simplified to not require an actual division circuit. We need a random =
number; that comes down to a register.

With PIE, the data path calculation is "once per interval, pick up a =
random number, compare it to a threshold, and possibly mark/drop a =
packet". That comes down to a register for the random number generator. =
The math in the background process is "interesting", but - it's in the =
background process. Use RISC approaches.

With CoDel, the key issue is the sqrt. As the algorithm is described, it =
happens in the data path - upon interval expiration, a packet is =
dropped, and a new interval is calculated in inverse proportion to the =
square root of the number of drops since the dropping state was entered. =
I'll argue that the actual value could be calculated in the background =
and placed into a register to be copied to the interval register the =
next time a packet is actually dropped. The square root calculation =
itself can be pretty simple. Most chips these days have an instruction =
to tell you the bit number of the most significant "one" in a register, =
and have barrel shifters. OK, pick up the number, determine its most =
significant bit number, shift that right one bit (divide by two), shift =
the argument of the sqrt right the resulting number of bits, and then do =
one or two Newton-Raphston iterations. So a reasonable approximation to =
the sqrt can be implemented using RISC approaches.

My take is that one can use silicon real estate if there is another =
reason to have the circuits, but the algorithms on the table can be =
implemented in microcode given the ability to run a supporting =
background process on a RISC computer.



Appendix on EWMAs using adds and shifts:

An EWMA calculates
    new_mean =3D W*sample + (1-W)*old_mean, 0 <=3D W <=3D 1
in successive steps, calculating the mean of a set of samples.

Let's constrain W to 2^(-N) for some value of N.

Now, store the mean shifted left N bits.

The calculation becomes=20
    mean :=3D sample + mean - (mean shifted right N bits).

Any time you need to use the mean, either you shift it right N bits, or =
you shift the rest of the calculation left N bits.


--Apple-Mail=_3DBFD422-2C1D-4CE8-A19B-BE57478A0706
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSg8mVbjEdbHIsm0MRAlvtAJ9KD5J4+q/uk62dLjKXI6U+8qSxYQCgrh/+
pxOe71JFKQak4qF9nhbyga4=
=oXmU
-----END PGP SIGNATURE-----

--Apple-Mail=_3DBFD422-2C1D-4CE8-A19B-BE57478A0706--

From naeem.khademi@gmail.com  Wed Nov 13 12:14:40 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F7D021E80B2 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 12:14:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.271
X-Spam-Level: 
X-Spam-Status: No, score=-2.271 tagged_above=-999 required=5 tests=[AWL=0.101,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 80p1OXXeGTFP for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 12:14:38 -0800 (PST)
Received: from mail-vb0-x231.google.com (mail-vb0-x231.google.com [IPv6:2607:f8b0:400c:c02::231]) by ietfa.amsl.com (Postfix) with ESMTP id 8CA1C21E80CA for <aqm@ietf.org>; Wed, 13 Nov 2013 12:14:37 -0800 (PST)
Received: by mail-vb0-f49.google.com with SMTP id o19so665339vbm.8 for <aqm@ietf.org>; Wed, 13 Nov 2013 12:14:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wAD8sUZINVWVuy4wTbUjo3jG/PyxJCgwHApGO7UcAxs=; b=Apk9ajocpUwv+OVVZPEUGn6m9kzLRJllcr0bwWT0ZzhtaDBc1cX836xKZJolPDWMGH whr0yeVcsBdT/Cjn8yOoh1x9FNO+VataYWSTCwfdiTF8wnLuB3gFjecYkCTY9MxMOpUT 4BCMgnTZwiQC7CzmHOwAVSWekxh3ZDvjAq844G+bh41BIcqDzCEBsThZxB9DBcGi+wj7 gy3UM2BWlQngzK1iDjSr1nOM3Ofcujulp1J1Ub2e3d1Tb6B0d4cKcjh7vvvJYKfXaelZ I+3SxBPqGli8/cWQwZmZoUBEyeJKAlVISpnpJjRzWPJ2ZNFPIKXwn4AvMy4zJf0aM4QS mkiw==
MIME-Version: 1.0
X-Received: by 10.220.17.131 with SMTP id s3mr16893523vca.20.1384373676883; Wed, 13 Nov 2013 12:14:36 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Wed, 13 Nov 2013 12:14:36 -0800 (PST)
In-Reply-To: <CEA9034B.4B37C%prenatar@cisco.com>
References: <CAEjQQ5Vif73gWe-35nzbhmPH+Eh+gSZK+7xNm33+T-FVNsmPZw@mail.gmail.com> <CEA9034B.4B37C%prenatar@cisco.com>
Date: Wed, 13 Nov 2013 21:14:36 +0100
Message-ID: <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=001a11c3b1b03f475e04eb149d05
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 20:14:40 -0000

--001a11c3b1b03f475e04eb149d05
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On Wed, Nov 13, 2013 at 7:25 PM, Preethi Natarajan <preethi.cis@gmail.com>w=
rote:

> Please see inline=85
>
> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Wednesday, November 13, 2013 5:01 AM
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
>
>> Michael, Naeem:
>>
>> This is just a follow-up to better understand the ARED results presented
>> at AQM WG (
>> http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf).
>>
>> Could you please clarify whether the ARED parameters for the tests were
>> set as specified in the 2001 paper? I.e., when average queueing delay
>> target =3D 1ms, the min and max thresholds would translate to 0.5ms and
>> 1.5ms, and for 5 ms target the min and max thresh will be 2.5ms and 7.5m=
s,
>> respectively, right? Can you confirm?
>>
>
> Copy/pasting from Section 3.1.3 of http://urn.nb.no/URN:NBN:no-38868 <htt=
p://urn.nb.no/URN:NBN:no-38868>
>
> *"In case of ARED, the parameters are calculated based on the
> recommendations in [19]*
> *except when target delay=3D1 ms where th_min and th_max are set to 1 MTU=
*
> *and 2 MTUs accordingly."*
>
> *[19] S. Floyd, R. Gummadi, and S. Shenker. Adaptive RED: An Algorithm fo=
r*
> *Increasing the Robustness of RED=92s Active Queue Management. Technical*
> *report, 2001.*
>
> This means "yes" for all values of target delay except for 1 ms cases
> where the thresholds become smaller than a single MTU transmission time o=
n
> the 10 Mbps bottleneck link used in this case.  The recommendations in
> Sally Flyod' s ARED 2001  (th_max=3D3*th_min; target=3D(th_max+th_min)/2)=
 are
> used in all other cases.
>
>
>> Assuming ARED logic is intact, ARED drops all packets when average queue
>> size goes above max_thresh  (max_thresh =3D 1.5ms or 7.5ms etc.); this
>> would be semantically similar to a drop tail queue with shallow buffer (=
can
>> be confirmed by looking at cumulative packet drops by ARED).
>>
>
> Similar but not identical as ARED (and any other *RED) does averaging
> instead of using instantaneous queue size (e.g. that being employed in
> DCTCP AQM)
>
>
>> This is probably why the delay values are tight for ARED, and why ARED
>> translates to poor utilization, especially for the lower target delay
>> values, as seen in your slides (#8, #11, #13).
>>
>
> True!
>
>
>
> Thanks for clarifying. That definitely clears the air about why ARED's
> delay values were so tight; trying to maintain the average queue delay
> between such a narrow band between min and max thresholds (0.5ms to 1.5ms
> or 2.5ms to 7.5ms) would naturally result in tighter delay values at the
> cost of utilization. Of course, ARED works on the average queue size, but
> when it comes to such a narrow band between min and max thresholds, we
> suspect there wouldn't be much difference between ARED and drop tail.
>

Very true, though that's what ARED *is* with very low thresholds (e.g.
resonate between 1 or 2 packets on the average in the queue). Indeed there
can possibly be some future work on ARED's (or any other similar AQM) burst
allowance but I don't think e.g. letting a 100 ms burst of packets to pass
through can be a good way of fixing it as this can lead to massive variance
in overall RTT distribution.


>
>
> Also,  wondering if bursty traffic was considered for evaluations, since
>> semantics of drop tail with shallow buffering doesn't accommodate bursty
>> traffic.
>>
>
> No, we did not consider bursty traffic (e.g. multimedia, etc) for this
> work as this was our first (initial) fundamental step to understand in a
> systematic way how AQMs work in real-life with long-lived TCP traffic
> (which is the most common type of traffic that fills up any buffer and
> consistently contributes to excessively long latencies experienced on the
> Internet (a.k.a bufferbloat)).
>
> The fact that AQMs *should* absorb sub-RTT bursts (due to rate mistmatch,
> etc) does not diminish the basic requirement for AQMs that they should
> control the latency induced by bulk (long-lived) TCP traffic in a well
> manner. Any other type of traffic on the access links will most likely to
> coexist with bulk TCP traffic when latency is observed to grow high. In
> other words, if an AQM can't control the latency of (not so bursty as
> others-) TCP traffic, it probably won't be able to do so for bursty traff=
ic
> as well.
>
>
>
> I would not limit an AQM design to absorb just sub-RTT bursts.
>

Neither do I. What I said was that we don't necessarily need to have a very
long distribution tail for long-lived TCP traffic (as observed with PIE and
slightly with CoDel in the results presented at ICCRG, IETF88) to be able
to absorb the bursts.


> A robust AQM scheme should be able to handle any kind of bursts
> irrespective of RTT, traffic patterns etc., since these bursts are quite
> common in deployments today.
>

Agreed only in general terms -- but what would be considered as "packet
burst" and how would it be defined? This will probably have a subjective
answer e.g. one can argue that a size of TCP sawtooth of data is a burst
and therefore we need a BDP of buffering for that (that's what had actually
happened implicitly over the past decade). On the other hand, the
definition of the "burst" may likely to correspond to the application
generating it (e.g. video frames, IW10, etc) and therefore its size (and
even pattern) is application/transport dependent somehow. There are sub-RTT
and larger than one RTT bursts that the network (AQM node) has no clue
about their size (at least for the existing AQMs studied). In case of CoDel
for instance, the whole idea of "100 ms interval time" plus "dropping mode"
with packet drops with sqrt function of "TCP throughput vs. loss model" is
designed to absorb the sub-RTT TCP packet bursts (see CoDel paper @ ACM
Queue) and it assumes that a 100 ms interval works almost fine for most
ranges of RTTs. There's nothing explicit that I can find about bursts
bigger than an one RTT in CoDel's design. I'm curious to know how an AQM
can identify a burst and let it through without the implicit knowledge of
RTT or the transport/application behind it and I'm not sure if it's ever
supposed to? (note: I'm not saying AQM should know at all, just curious how
would it be possible to fulfill the goal you're stating above otherwise?)


Hopefully you or some people on the list can possibly help me with that
:-).

Thanks,
> Preethi
>

Cheers,
Naeem

--001a11c3b1b03f475e04eb149d05
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Wed, Nov 13, 2013 at 7:25 PM, Preethi Natarajan <span dir=3D"ltr=
">&lt;<a href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.ci=
s@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div style=3D"font-size:14px;font-family:Cal=
ibri,sans-serif;word-wrap:break-word"><div>Please see inline=85</div><div><=
br></div>
<span><div style=3D"border-right:medium none;padding-right:0in;padding-left=
:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium no=
ne;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;bord=
er-left:medium none">
<span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D=
"mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com<=
/a>&gt;<br><span style=3D"font-weight:bold">Date: </span> Wednesday, Novemb=
er 13, 2013 5:01 AM<br>
<span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a href=
=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</=
a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Michael Welzl &lt;<a=
 href=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no</a=
>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org=
</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.o=
rg</a>&gt;<div class=3D"im">
<br><span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes=
: Queue length vs. delay based<br></div></div><div><br></div><blockquote st=
yle=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div dir=
=3D"ltr">
<div class=3D"gmail_extra"><div class=3D"gmail_quote"><div>=A0</div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-wi=
dth:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-=
left:1ex">
<div style=3D"font-family:Calibri,sans-serif;word-wrap:break-word"><div sty=
le=3D"font-size:14px"></div><div><div style=3D"font-family:Calibri,sans-ser=
if">Michael, Naeem:</div><div class=3D"im"><div style=3D"font-family:Calibr=
i,sans-serif">
<br></div><div style=3D"font-family:Calibri,sans-serif">This is just a foll=
ow-up to better understand the ARED results presented at AQM WG (<a href=3D=
"http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf" target=3D=
"_blank">http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-4.pdf</a=
>).</div>
<div style=3D"font-family:Calibri,sans-serif"><br></div><div><font face=3D"=
Calibri,sans-serif">Could you please clarify whether the ARED parameters fo=
r the tests were set as specified in the 2001 paper? I.e., when average que=
ueing delay target =3D 1ms, the=A0min=A0</font><span style=3D"font-family:C=
alibri,sans-serif">and</span><span style=3D"font-family:Calibri,sans-serif"=
>=A0</span><font face=3D"Calibri,sans-serif">max thresholds would translate=
 to 0.5ms and 1.5ms, and for 5 ms target the min and max thresh will be 2.5=
ms and 7.5ms, respectively, right?=A0</font>Can you confirm?</div>
</div></div></div></blockquote><div class=3D"im"><div><br></div><div>Copy/p=
asting from Section 3.1.3 of=A0<a href=3D"http://urn.nb.no/URN:NBN:no-38868=
" style=3D"font-size:13px;margin:0px;padding:0px;vertical-align:baseline;ba=
ckground-color:transparent;color:rgb(96,120,144);font-family:sans-serif;lin=
e-height:20px;text-align:justify" target=3D"_blank">http://urn.nb.no/URN:NB=
N:no-38868=A0</a></div>
<div><br></div><div><div><b>&quot;In case of ARED, the parameters are calcu=
lated based on the recommendations in [19]</b></div><div><b>except when tar=
get delay=3D1 ms where th_min and th_max are set to 1 MTU</b></div><div><b>=
and 2 MTUs accordingly.&quot;</b></div>
<div><br></div><div><div><b>[19] S. Floyd, R. Gummadi, and S. Shenker. Adap=
tive RED: An Algorithm for</b></div><div><b>Increasing the Robustness of RE=
D=92s Active Queue Management. Technical</b></div><div><b>report, 2001.</b>=
</div>
</div><div><br></div><div>This means &quot;yes&quot; for all values of targ=
et delay except for 1 ms cases where the thresholds become smaller than a s=
ingle MTU transmission time on the 10 Mbps bottleneck link used in this cas=
e. =A0The recommendations in Sally Flyod&#39; s ARED 2001 =A0(th_max=3D3*th=
_min; target=3D(th_max+th_min)/2) are used in all other cases.=A0</div>
</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);borde=
r-left-style:solid;padding-left:1ex"><div style=3D"font-family:Calibri,sans=
-serif;word-wrap:break-word">
<div><div><font face=3D"Calibri,sans-serif">Assuming ARED logic is intact, =
ARED drops all packets when average queue size goes above max_thresh =A0</f=
ont>(max_thresh =3D 1.5ms or 7.5ms etc.); this would be semantically simila=
r to a drop tail queue with shallow buffer (can be confirmed by looking at =
cumulative packet drops by ARED).</div>
</div></div></blockquote><div><br></div><div>Similar but not identical as A=
RED (and any other *RED) does averaging instead of using=A0instantaneous=A0=
queue size (e.g. that being employed in DCTCP AQM)=A0</div><div>=A0</div><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;pad=
ding-left:1ex">
<div style=3D"font-family:Calibri,sans-serif;word-wrap:break-word"><div><di=
v><font face=3D"Calibri,sans-serif">This is probably why the delay values a=
re tight for ARED, and why ARED translates to poor utilization, especially =
for the lower target delay values, as seen in your slides (#8, #11, #13).=
=A0</font></div>
</div></div></blockquote><div><br></div><div>True!=A0</div><div>=A0</div></=
div></div></div></div></blockquote></span><div><br></div><div>Thanks for cl=
arifying. That definitely clears the air about why ARED&#39;s delay values =
were so tight; trying to maintain the average queue delay between such a na=
rrow band between min and max thresholds (0.5ms to 1.5ms or 2.5ms to 7.5ms)=
 would naturally result in tighter delay values at the cost of utilization.=
 Of course, ARED works on the average queue size, but when it comes to such=
 a narrow band between min and max thresholds, we suspect there wouldn&#39;=
t be much difference between ARED and drop tail.</div>
</div></blockquote><div><br></div><div style>Very true, though that&#39;s w=
hat ARED *is* with very low thresholds (e.g. resonate between 1 or 2 packet=
s on the average in the queue). Indeed there can possibly be some future wo=
rk on ARED&#39;s (or any other similar AQM) burst allowance but I don&#39;t=
 think e.g. letting a 100 ms burst of packets to pass through can be a good=
 way of fixing it as this can lead to massive variance in overall RTT distr=
ibution. =A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"font-size:14px;f=
ont-family:Calibri,sans-serif;word-wrap:break-word"><div class=3D"im"><div>=
<br></div>
<div><br></div><span><blockquote style=3D"BORDER-LEFT:#b5c4df 5 solid;PADDI=
NG:0 0 0 5;MARGIN:0 0 0 5"><div dir=3D"ltr"><div class=3D"gmail_extra"><div=
 class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0p=
x 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);bo=
rder-left-style:solid;padding-left:1ex">
<div style=3D"font-family:Calibri,sans-serif;word-wrap:break-word"><div><di=
v>Also, =A0wondering if bursty traffic was considered for evaluations, sinc=
e semantics of drop tail with shallow buffering doesn&#39;t accommodate bur=
sty traffic.=A0</div>
</div></div></blockquote><div><br></div><div>No, we did not consider bursty=
 traffic (e.g. multimedia, etc) for this work as this was our first (initia=
l) fundamental step to understand in a systematic way how AQMs work in real=
-life with long-lived TCP traffic (which is the most common type of traffic=
 that fills up any buffer and consistently contributes to excessively long =
latencies experienced on the Internet (a.k.a bufferbloat)).=A0</div>
<div><br></div><div>The fact that AQMs *should* absorb sub-RTT bursts (due =
to rate mistmatch, etc) does not diminish the basic requirement for AQMs th=
at they should control the latency induced by bulk (long-lived) TCP traffic=
 in a well manner. Any other type of traffic on the access links will most =
likely to coexist with bulk TCP traffic when latency is observed to grow hi=
gh. In other words, if an AQM can&#39;t control the latency of (not so burs=
ty as others-) TCP traffic, it probably won&#39;t be able to do so for burs=
ty traffic as well. =A0 =A0 =A0</div>
</div></div></div></blockquote></span><div><br></div><div><br></div></div><=
div>I would not limit an AQM design to absorb just sub-RTT bursts. </div></=
div></blockquote><div><br></div><div style>Neither do I. What I said was th=
at we don&#39;t necessarily need to have a very long distribution tail for =
long-lived TCP traffic (as observed with PIE and slightly with CoDel in the=
 results presented at ICCRG, IETF88) to be able to absorb the bursts.</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"font-size:14px;f=
ont-family:Calibri,sans-serif;word-wrap:break-word"><div>A robust AQM schem=
e should be able to handle any kind of bursts irrespective of RTT, traffic =
patterns etc., since these bursts are quite common in deployments today.=A0=
</div>
</div></blockquote><div><br></div><div style>Agreed only in general terms -=
- but what would be considered as &quot;packet burst&quot; and how would it=
 be defined? This will probably have a subjective answer e.g. one can argue=
 that a size of TCP=A0sawtooth of data is a burst and therefore we need a B=
DP of buffering for that (that&#39;s what had actually happened implicitly =
over the past decade). On the other hand, the definition of the &quot;burst=
&quot; may likely to correspond to the application generating it (e.g. vide=
o frames, IW10, etc) and therefore its size (and even pattern) is applicati=
on/transport dependent somehow. There are sub-RTT and larger than one RTT b=
ursts that the network (AQM node) has no clue about their size (at least fo=
r the existing AQMs studied). In case of CoDel for instance, the whole idea=
 of &quot;100 ms interval time&quot; plus &quot;dropping mode&quot; with pa=
cket drops with sqrt function of &quot;TCP throughput vs. loss model&quot; =
is designed to absorb the sub-RTT TCP packet bursts (see CoDel paper @ ACM =
Queue) and it assumes that a 100 ms interval works almost fine for most ran=
ges of RTTs. There&#39;s nothing explicit that I can find about bursts bigg=
er than an one RTT in CoDel&#39;s design. I&#39;m curious to know how an AQ=
M can identify a burst and let it through without the implicit knowledge of=
 RTT or the transport/application behind it and I&#39;m not sure if it&#39;=
s ever supposed to? (note: I&#39;m not saying AQM should know at all, just =
curious how would it be possible to fulfill the goal you&#39;re stating abo=
ve otherwise?) =A0 =A0 =A0 =A0 =A0=A0</div>
<div style><br></div><div style>Hopefully you or some people on the list ca=
n possibly help me with that :-).=A0</div><div style><br></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid=
;padding-left:1ex">
<div style=3D"font-size:14px;font-family:Calibri,sans-serif;word-wrap:break=
-word"><div>Thanks,</div><div>Preethi</div></div></blockquote><div><br></di=
v><div style>Cheers,</div><div style>Naeem=A0</div></div></div></div>

--001a11c3b1b03f475e04eb149d05--

From fred@cisco.com  Wed Nov 13 12:19:29 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 712B421E8162; Wed, 13 Nov 2013 12:19:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.325
X-Spam-Level: 
X-Spam-Status: No, score=-110.325 tagged_above=-999 required=5 tests=[AWL=0.047, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0uE4WvdMs6b9; Wed, 13 Nov 2013 12:19:23 -0800 (PST)
Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) by ietfa.amsl.com (Postfix) with ESMTP id 9371911E8136; Wed, 13 Nov 2013 12:19:01 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2228; q=dns/txt; s=iport; t=1384373941; x=1385583541; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=P5ckJkLXRaB9/x6j6JuIFz3h/RpyZVgt7Np9A/xoBWQ=; b=YZTFVm8DQaG812dzSwrzNM2MhPmsC9BPQHPK0A6NaQ1/QnKfQEnxv8tl KDKgL9AZBxbCUkpPziJp+zlsMINQGWPO4YN5lc4ITs1hcZDiPILtWEAlj 9aZBmjrdmxSJdmgKVr8pzPy8UYvM9tH9GND9lyMJVajrLPLi/Wk6RRy/w Y=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgIFAA3eg1KtJV2Y/2dsb2JhbABZgweBC78sgSgWdIIlAQEBAwF5BQsCAQgOODIlAgQOBQ6HbQbASo4hgT4HgyCBEQOQMIEwhjCSC4MogWlB
X-IronPort-AV: E=Sophos;i="4.93,693,1378857600";  d="asc'?scan'208";a="284455038"
Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-3.cisco.com with ESMTP; 13 Nov 2013 20:19:01 +0000
Received: from xhc-aln-x05.cisco.com (xhc-aln-x05.cisco.com [173.36.12.79]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id rADKJ0EN016315 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 13 Nov 2013 20:19:00 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x05.cisco.com ([173.36.12.79]) with mapi id 14.03.0123.003; Wed, 13 Nov 2013 14:19:00 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Richard Scheffenegger <rs@netapp.com>
Thread-Topic: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO4K2WdTQM8Xyau0qmjVP5kkPTjg==
Date: Wed, 13 Nov 2013 20:19:00 +0000
Message-ID: <2457257D-7DC7-4760-BAC2-EF46B8E04D45@cisco.com>
References: <201311111618.rABGILbC031136@bagheera.jungle.bt.co.uk> <CEA65B19.21ED5%g.white@cablelabs.com> <012C3117EDDB3C4781FD802A8C27DD4F25E8908F@SACEXCMBX02-PRD.hq.netapp.com> <E7D77F29-36BD-45FD-AE22-D90F2A687964@cisco.com>
In-Reply-To: <E7D77F29-36BD-45FD-AE22-D90F2A687964@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.121]
Content-Type: multipart/signed; boundary="Apple-Mail=_261738D0-DE20-459B-A0E9-2547D83D83F4"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Greg White <g.white@CableLabs.com>, tsv-area IETF list <tsv-area@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, AQM IETF list <aqm@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 20:19:29 -0000

--Apple-Mail=_261738D0-DE20-459B-A0E9-2547D83D83F4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


On Nov 13, 2013, at 10:48 AM, Fred Baker (fred) <fred@cisco.com> wrote:

> With CoDel, the key issue is the sqrt. As the algorithm is described, =
it happens in the data path - upon interval expiration, a packet is =
dropped, and a new interval is calculated in inverse proportion to the =
square root of the number of drops since the dropping state was entered. =
I'll argue that the actual value could be calculated in the background =
and placed into a register to be copied to the interval register the =
next time a packet is actually dropped. The square root calculation =
itself can be pretty simple. Most chips these days have an instruction =
to tell you the bit number of the most significant "one" in a register, =
and have barrel shifters. OK, pick up the number, determine its most =
significant bit number, shift that right one bit (divide by two), shift =
the argument of the sqrt right the resulting number of bits, and then do =
one or two Newton-Raphston iterations. So a reasonable approximation to =
the sqrt can be implemented using RISC approaches.

The sort could also be implemented as a table, given some assumptions. =
use the number of drops since the epoch as an index and read the sqrt =
(or actually, the calculation that results from it). At some point, we =
reach the minimum interval that is acceptable - we are perhaps dropping =
every packet, or dropping some limit percentage. At that point, there =
isn't a real value in tracking the actual value, as the interval ceases =
decreasing.

--Apple-Mail=_261738D0-DE20-459B-A0E9-2547D83D83F4
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSg96wbjEdbHIsm0MRAv9bAKDUreayW8Ky+S4HGT6u3KFHsRv04ACgtjih
2rxVVEI/guxHEUPqNFNitpk=
=YbJq
-----END PGP SIGNATURE-----

--Apple-Mail=_261738D0-DE20-459B-A0E9-2547D83D83F4--

From fred@cisco.com  Wed Nov 13 14:22:56 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E91221E8171; Wed, 13 Nov 2013 14:22:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.326
X-Spam-Level: 
X-Spam-Status: No, score=-110.326 tagged_above=-999 required=5 tests=[AWL=0.046, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id epK98Y5c1QAP; Wed, 13 Nov 2013 14:22:51 -0800 (PST)
Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) by ietfa.amsl.com (Postfix) with ESMTP id DDF0521E80CF; Wed, 13 Nov 2013 14:22:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1117; q=dns/txt; s=iport; t=1384381371; x=1385590971; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=iyTJ7IRFixcRu34+T9i+YOagO0XT9MV+mkWSgvg7r/M=; b=BHp+JJx34PLD+B8sFXSiLyebfEi6FBmik1KITyF6QQlgirx0pQl+YKf8 r4yYEV6mZauGYOyRWt8QLwpnsVUz2g+hXorWNxUx0R9zkbZJWU1KBYu3Y qMVtxw+u2PxDfIuTQmX0drYzQP47i32kRLfKNwSE+cAYBwyvj8D+pxCKD g=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgIFAGn6g1KtJXG8/2dsb2JhbABZgweBC78sgSgWdIIlAQEBAwEnUgULAgEIRjIlAgQOBQ6HbQbAMY9fB4MggREDkDCBMIYwkguDKIIq
X-IronPort-AV: E=Sophos;i="4.93,695,1378857600";  d="asc'?scan'208";a="284565200"
Received: from rcdn-core2-1.cisco.com ([173.37.113.188]) by rcdn-iport-4.cisco.com with ESMTP; 13 Nov 2013 22:22:50 +0000
Received: from xhc-aln-x03.cisco.com (xhc-aln-x03.cisco.com [173.36.12.77]) by rcdn-core2-1.cisco.com (8.14.5/8.14.5) with ESMTP id rADMMnMS004192 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 13 Nov 2013 22:22:50 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x03.cisco.com ([173.36.12.77]) with mapi id 14.03.0123.003; Wed, 13 Nov 2013 16:22:49 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: "<curtis@ipv6.occnc.com>" <curtis@ipv6.occnc.com>
Thread-Topic: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
Thread-Index: AQHO4L7ihqXe1x/pYE67TZ+HW0tSpQ==
Date: Wed, 13 Nov 2013 22:22:48 +0000
Message-ID: <B5E3D723-4301-4E8D-8733-2CBCE8D3FB53@cisco.com>
References: <201311132148.rADLmFMd043739@gateway1.ipv6.occnc.com>
In-Reply-To: <201311132148.rADLmFMd043739@gateway1.ipv6.occnc.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.121]
Content-Type: multipart/signed; boundary="Apple-Mail=_17D15C2E-54A9-463B-83C3-E50C6ABFFFA3"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: "Scheffenegger, Richard" <rs@netapp.com>, tsvwg IETF list <tsvwg@ietf.org>, Bob Briscoe <bob.briscoe@bt.com>, tsv-area IETF list <tsv-area@ietf.org>, Greg White <g.white@CableLabs.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] [tsvwg] Immediate ECN: Autotuning AQM for RTT
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 22:22:56 -0000

--Apple-Mail=_17D15C2E-54A9-463B-83C3-E50C6ABFFFA3
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Nov 13, 2013, at 1:48 PM, Curtis Villamizar <curtis@ipv6.occnc.com> =
wrote:

> Having briefly reviewed the execise with hardware guys quite a few
> times, the answer seems to come out to epsilon where epsilon -> 0
> compared to other things the forwarding engine already needs to do.

Pretty much. I was just trying to point out the issues that drive that =
cost, even if epsilon, and in so doing provide a useful response.

--Apple-Mail=_17D15C2E-54A9-463B-83C3-E50C6ABFFFA3
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFSg/u2bjEdbHIsm0MRAtrfAKDlfwt/fgQHIyAnUe8Xi9ZX3JjCJQCePnVv
NM9Msv+N8wGCNojzcMy+bh8=
=FQpA
-----END PGP SIGNATURE-----

--Apple-Mail=_17D15C2E-54A9-463B-83C3-E50C6ABFFFA3--

From ghanwani@gmail.com  Wed Nov 13 14:43:51 2013
Return-Path: <ghanwani@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E5AB21F918F for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 14:43:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.588
X-Spam-Level: 
X-Spam-Status: No, score=-1.588 tagged_above=-999 required=5 tests=[AWL=0.162,  BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZIDU8tklVEGz for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 14:43:50 -0800 (PST)
Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) by ietfa.amsl.com (Postfix) with ESMTP id 5576321F99FD for <aqm@ietf.org>; Wed, 13 Nov 2013 14:43:49 -0800 (PST)
Received: by mail-we0-f169.google.com with SMTP id q58so1114799wes.28 for <aqm@ietf.org>; Wed, 13 Nov 2013 14:43:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8twhVkVtHo14yHV5Qf5t9er3iqgriTCSD5dBm8OFyvk=; b=nPRHQKRYTfn/No9lXzSRkzYUfjl9JciwwVE6IH5wiw7YagM4nLLZLzLS7TKxMg7Wg7 srGRD1zSiqBV5OaX9SBMH/AHpjw0o4XI39CGBV5YYjd9NPi5TNdEbNlL+jEH870SNqd4 WI6oVbz9uylTJKVJpaIwSihRa+f2n8pkZttFqpxs7ImMXBtZW6Imjlp49yGO+j4vhTDG q/ZD2tQu2UG3oblNrYbQw4Hxx36yNvIsQ2cQGiVrUXprA7/IhJyP4tQx8oUaNPVAv0s1 8ShqMQqWNfR71zAtkyvZf+/UuqFMFcw+lv8FicM4nDrsyu6xNjfhsfxV8DwrRwr9NPp+ FTTg==
MIME-Version: 1.0
X-Received: by 10.180.72.238 with SMTP id g14mr22340020wiv.17.1384382628494; Wed, 13 Nov 2013 14:43:48 -0800 (PST)
Sender: ghanwani@gmail.com
Received: by 10.217.57.131 with HTTP; Wed, 13 Nov 2013 14:43:48 -0800 (PST)
In-Reply-To: <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com>
References: <CAEjQQ5Vif73gWe-35nzbhmPH+Eh+gSZK+7xNm33+T-FVNsmPZw@mail.gmail.com> <CEA9034B.4B37C%prenatar@cisco.com> <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com>
Date: Wed, 13 Nov 2013 14:43:48 -0800
X-Google-Sender-Auth: lypaUluzjse04EnogqoHHVukfu8
Message-ID: <CA+-tSzyKT9gS_12V=yyia8iPt9FiDg6=NRKrB3FLdCRDTi=6Pg@mail.gmail.com>
From: Anoop Ghanwani <anoop@alumni.duke.edu>
To: Naeem Khademi <naeem.khademi@gmail.com>
Content-Type: multipart/alternative; boundary=f46d043d6737ce066d04eb16b29d
Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>, Preethi Natarajan <preethi.cis@gmail.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 22:43:51 -0000

--f46d043d6737ce066d04eb16b29d
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Nov 13, 2013 at 12:14 PM, Naeem Khademi <naeem.khademi@gmail.com>wrote:

>
> Agreed only in general terms -- but what would be considered as "packet
> burst" and how would it be defined? This will probably have a subjective
> answer e.g. one can argue that a size of TCP sawtooth of data is a burst
> and therefore we need a BDP of buffering for that (that's what had actually
> happened implicitly over the past decade). On the other hand, the
> definition of the "burst" may likely to correspond to the application
> generating it (e.g. video frames, IW10, etc) and therefore its size (and
> even pattern) is application/transport dependent somehow.
>

There's one more case...the incast problem.  The sources themselves may not
be "bursty" but in a high port count switch, you could have 10's (or more)
ports sending traffic to a single output port at around the same time.

Perhaps it is useful to add a description of what we mean when we say
bursty.

Anoop

Anoop

--f46d043d6737ce066d04eb16b29d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">On Wed, Nov 13, 2013 at 12:14 PM, Naeem Khademi <span dir=
=3D"ltr">&lt;<a href=3D"mailto:naeem.khademi@gmail.com" target=3D"_blank">n=
aeem.khademi@gmail.com</a>&gt;</span> wrote:<br><div class=3D"gmail_extra">=
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><br><div class=3D"gmail_ext=
ra"><div class=3D"gmail_quote"><div>Agreed only in general terms -- but wha=
t would be considered as &quot;packet burst&quot; and how would it be defin=
ed? This will probably have a subjective answer e.g. one can argue that a s=
ize of TCP=A0sawtooth of data is a burst and therefore we need a BDP of buf=
fering for that (that&#39;s what had actually happened implicitly over the =
past decade). On the other hand, the definition of the &quot;burst&quot; ma=
y likely to correspond to the application generating it (e.g. video frames,=
 IW10, etc) and therefore its size (and even pattern) is application/transp=
ort dependent somehow.=A0</div>
</div></div></div></blockquote><div><br></div><div>There&#39;s one more cas=
e...the incast problem. =A0The sources themselves may not be &quot;bursty&q=
uot; but in a high port count switch, you could have 10&#39;s (or more) por=
ts sending traffic to a single output port at around the same time.</div>
<div><br></div><div>Perhaps it is useful to add a description of what we me=
an when we say bursty.</div><div><br></div><div>Anoop</div><div><br></div><=
div>Anoop=A0</div></div><br></div></div>

--f46d043d6737ce066d04eb16b29d--

From ghanwani@gmail.com  Wed Nov 13 15:33:33 2013
Return-Path: <ghanwani@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 42E8A21F9DF0 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 15:33:33 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.628
X-Spam-Level: 
X-Spam-Status: No, score=-1.628 tagged_above=-999 required=5 tests=[AWL=0.122,  BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AJv1ip4Var8I for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 15:33:32 -0800 (PST)
Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) by ietfa.amsl.com (Postfix) with ESMTP id BCC8311E810B for <aqm@ietf.org>; Wed, 13 Nov 2013 15:33:31 -0800 (PST)
Received: by mail-wg0-f50.google.com with SMTP id k14so1136597wgh.5 for <aqm@ietf.org>; Wed, 13 Nov 2013 15:33:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5azHD2dNMyoiUhs6rcgZYHl+SlVxWc+lFJK6/Q0hmsw=; b=v4Mm6QPwKHoIK+rOgFwEKvtYLICgmODqSzP3WlPgC4JwMjKnifnu4ZMHpBNMWBiQJm HlR+Z4oETcYyYA8ragCw+nBADv1mwd/VPYt9wC5BAxpAdWqchtpv7RE+CRWWBHLJxsDv 7+VhRvWBPngU2wNAi7kjSmxrmbm4iHUD+KRPF5qdY8RukAcgNWmiB3blzpNgHj8pYXkO 6l7bz55D9hqBSglG2WY2PblyQ6x/boyX50wzCbH8q1jV3SV3gSaYn0JgFJaQzs8l4o16 2Ht4rh8WTSIuaCFDy/rtCu9WOXR45BLxe/CDORMREUb8ZQGIxlyqzK1TBlachaChsobF /AWA==
MIME-Version: 1.0
X-Received: by 10.194.241.228 with SMTP id wl4mr33684295wjc.2.1384385610464; Wed, 13 Nov 2013 15:33:30 -0800 (PST)
Sender: ghanwani@gmail.com
Received: by 10.217.57.131 with HTTP; Wed, 13 Nov 2013 15:33:30 -0800 (PST)
In-Reply-To: <201311132323.rADNNIKM047235@gateway1.ipv6.occnc.com>
References: <CA+-tSzyKT9gS_12V=yyia8iPt9FiDg6=NRKrB3FLdCRDTi=6Pg@mail.gmail.com> <201311132323.rADNNIKM047235@gateway1.ipv6.occnc.com>
Date: Wed, 13 Nov 2013 15:33:30 -0800
X-Google-Sender-Auth: IVlmA96eUte-NGA8gjdop4W3Glk
Message-ID: <CA+-tSzyk=myS9B5tJ465C2JG9QvH8fPCG6Uu1iZ94TgPwV9s+w@mail.gmail.com>
From: Anoop Ghanwani <anoop@alumni.duke.edu>
To: curtis@ipv6.occnc.com
Content-Type: multipart/alternative; boundary=089e01493c328b463a04eb1764ea
Cc: Preethi Natarajan <preethi.cis@gmail.com>, Naeem Khademi <naeem.khademi@gmail.com>, Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2013 23:33:33 -0000

--089e01493c328b463a04eb1764ea
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Nov 13, 2013 at 3:23 PM, Curtis Villamizar <curtis@ipv6.occnc.com>wrote:

>
> Including unrealistic scenarios, like going from near zero traffic to 10
> interfaces feeding one at full speed until overflow occurs, is
> counterproductive.


It is actually a problem that keeps many people busy because a number of
data center switches have very high port count with very small buffers.
 Some people address these buy using switches with bigger buffers, but
that's not a luxury that everyone indulges in.

That is why I specifically asked the question at the AQM meeting about
applicability of AQM to all types of networks/switches.  I was told the
answer is "yes" and so I would like to see this scenario addressed as well.
 Perhaps AQM cannot help this, but hopefully it won't hurt.  Trying to do
fancy things with small buffers is challenging.

Anoop

--089e01493c328b463a04eb1764ea
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Wed, Nov 13, 2013 at 3:23 PM, Curtis Villamizar <span dir=3D"ltr=
">&lt;<a href=3D"mailto:curtis@ipv6.occnc.com" target=3D"_blank">curtis@ipv=
6.occnc.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><br>
Including unrealistic scenarios, like going from near zero traffic to 10<br=
>
interfaces feeding one at full speed until overflow occurs, is<br>
counterproductive.=A0</blockquote><div>=A0</div><div>It is actually a probl=
em that keeps many people busy because a number of data center switches hav=
e very high port count with very small buffers. =A0Some people address thes=
e buy using switches with bigger buffers, but that&#39;s not a luxury that =
everyone indulges in.</div>
<div><br></div><div>That is why I specifically asked the question at the AQ=
M meeting about applicability of AQM to all types of networks/switches. =A0=
I was told the answer is &quot;yes&quot; and so I would like to see this sc=
enario addressed as well. =A0Perhaps AQM cannot help this, but hopefully it=
 won&#39;t hurt. =A0Trying to do fancy things with small buffers is challen=
ging.</div>
<div><br></div><div>Anoop</div></div></div></div>

--089e01493c328b463a04eb1764ea--

From preethi.cis@gmail.com  Wed Nov 13 17:18:04 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E528521E80BF for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 17:18:03 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NBYnWMwbrr76 for <aqm@ietfa.amsl.com>; Wed, 13 Nov 2013 17:18:03 -0800 (PST)
Received: from mail-pb0-x22f.google.com (mail-pb0-x22f.google.com [IPv6:2607:f8b0:400e:c01::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 5BA7021E8098 for <aqm@ietf.org>; Wed, 13 Nov 2013 17:18:03 -0800 (PST)
Received: by mail-pb0-f47.google.com with SMTP id rq2so1242496pbb.20 for <aqm@ietf.org>; Wed, 13 Nov 2013 17:18:00 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=hPa6pKkLLGKLG4DZBnYH+X8dJiKLJUb9M65AWv4DQa0=; b=r0yiIrWOSNyeTbYrWAnt5maU5nrHuOuhtPV1IM1xbfAnVEL6fHNq5TwfGgzdbWCiGu bD7fTp8BJUaxZmhM3ObcKI5uw1R6n1UR9OBSNN9IUrGFVlPbu45o5sgfCwxCnztp4UqM MKGUdfUbgM8+0y8a/bXbXgeW+sy7jRbgeoP8IGN+tDoeMQpKsKAmLayte9whzrdQuK2+ tVnEuzgcWVNN5Hd9XXKHct7efKcn80a0v1MWKi9xxSHtyOdZ/D5IuRdIuHpgWD3ahYW8 VRnH13MrjzSboxIc3ezLOkanaImo8Ie3jkT7PGeluYHU+uKfgVy/6PnsnonisyJSDvpe 6j7A==
X-Received: by 10.66.118.129 with SMTP id km1mr17543914pab.127.1384391880796;  Wed, 13 Nov 2013 17:18:00 -0800 (PST)
Received: from [10.21.148.218] (128-107-239-233.cisco.com. [128.107.239.233]) by mx.google.com with ESMTPSA id ry4sm55038132pab.4.2013.11.13.17.17.58 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 13 Nov 2013 17:17:59 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Wed, 13 Nov 2013 17:17:56 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEA961B1.4B416%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467207879_6500777"
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 01:18:04 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467207879_6500777
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit



From:  Naeem Khademi <naeem.khademi@gmail.com>
Date:  Wednesday, November 13, 2013 12:14 PM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> 
> Very true, though that's what ARED *is* with very low thresholds (e.g.
> resonate between 1 or 2 packets on the average in the queue). Indeed there can
> possibly be some future work on ARED's (or any other similar AQM) burst
> allowance but I don't think e.g. letting a 100 ms burst of packets to pass
> through can be a good way of fixing it as this can lead to massive variance in
> overall RTT distribution.

If you are referring to the burst_allowance parameter in PIE, this parameter
allows burst to go through when a system goes from an idle state into some
congestion state. The one we are talking about is accommodating bursts when
the system is in equilibirum -- how does the AQM scheme accommodate bursts
when already operating around target delay or congestion? Delay-based ARED
behaves similar to tail drop at max threshold. This is unacceptable of an
AQM even for max thresholds around 50ms.
Clearly delay-based ARED needs significant and careful redesign. I suppose
one can argue this task is as laborious as designing a new AQM scheme as
opposed to previous notions that delay-based ARED would work right out of
the box. 
Preethi



--B_3467207879_6500777
Content-type: text/html;
	charset="US-ASCII"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; "><div style=3D"color: rgb(0, 0, =
0); font-family: Calibri, sans-serif; font-size: 14px; "><br></div><div styl=
e=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px; "=
><br></div><span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); font-=
size: 14px; font-family: Calibri, sans-serif; "><div style=3D"font-family:Cali=
bri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium non=
e; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING=
-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDI=
NG-TOP: 3pt"><span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;=
<a href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br>=
<span style=3D"font-weight:bold">Date: </span> Wednesday, November 13, 2013 12=
:14 PM<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<=
a href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span=
 style=3D"font-weight:bold">Cc: </span> Michael Welzl &lt;<a href=3D"mailto:mich=
awe@ifi.uio.no">michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailto:aqm@ietf.org">a=
qm@ietf.org</a>" &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><=
span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue =
length vs. delay based<br></div><div><br></div><blockquote id=3D"MAC_OUTLOOK_A=
TTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;=
 MARGIN:0 0 0 5;"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_=
quote"><div><br></div><div style=3D"">Very true, though that's what ARED *is* =
with very low thresholds (e.g. resonate between 1 or 2 packets on the averag=
e in the queue). Indeed there can possibly be some future work on ARED's (or=
 any other similar AQM) burst allowance but I don't think e.g. letting a 100=
 ms burst of packets to pass through can be a good way of fixing it as this =
can lead to massive variance in overall RTT distribution. &nbsp;</div></div>=
</div></div></blockquote></span><div style=3D"color: rgb(0, 0, 0); font-family=
: Calibri, sans-serif; font-size: 14px; "><br></div><div><div class=3D"text" s=
tyle=3D"color: rgb(0, 0, 0); word-wrap: break-word; padding-top: 12px; padding=
-right: 55px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margi=
n-right: 0px; margin-bottom: 0px; margin-left: 0px; overflow-x: auto; overfl=
ow-y: auto; font-family: Helvetica; font-size: medium; "><span style=3D"word-w=
rap: break-word; font: normal normal normal 14px/normal Calibri; ">If you ar=
e referring to the burst_allowance parameter in PIE, this parameter allows b=
urst to go through when a system goes from an idle state into some congestio=
n state. The one we are talking about is accommodating bursts when the syste=
m is in equilibirum -- how does the AQM scheme accommodate bursts when alrea=
dy operating around target delay or congestion? Delay-based ARED behaves sim=
ilar to tail drop at max threshold. This is unacceptable of an AQM even for =
max thresholds around 50ms.&nbsp;</span></div><div class=3D"text" style=3D"color=
: rgb(0, 0, 0); word-wrap: break-word; padding-top: 12px; padding-right: 55p=
x; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0p=
x; margin-bottom: 0px; margin-left: 0px; overflow-x: auto; overflow-y: auto;=
 font-family: Helvetica; font-size: medium; "><span style=3D"word-wrap: break-=
word; font: normal normal normal 14px/normal Calibri; ">Clearly delay-based =
ARED needs significant and careful redesign.&nbsp;</span><span class=3D"Apple-=
style-span" style=3D"font-size: 14px; font-family: Calibri; ">I suppose one ca=
n argue this task is as laborious as designing a new AQM scheme as opposed t=
o previous notions that delay-based ARED would work right out of the box.&nb=
sp;</span></div></div><div class=3D"text" style=3D"color: rgb(0, 0, 0); word-wra=
p: break-word; padding-top: 12px; padding-right: 55px; padding-bottom: 0px; =
padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; m=
argin-left: 0px; overflow-x: auto; overflow-y: auto; font-family: Helvetica;=
 font-size: medium; "><span class=3D"Apple-style-span" style=3D"font-family: Cal=
ibri; ">Preethi</span></div></body></html>

--B_3467207879_6500777--



From naeem.khademi@gmail.com  Thu Nov 14 05:32:59 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C1BE11E811A for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 05:32:59 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.285
X-Spam-Level: 
X-Spam-Status: No, score=-2.285 tagged_above=-999 required=5 tests=[AWL=0.087,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DdP0b5WJWVvL for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 05:32:58 -0800 (PST)
Received: from mail-vb0-x22f.google.com (mail-vb0-x22f.google.com [IPv6:2607:f8b0:400c:c02::22f]) by ietfa.amsl.com (Postfix) with ESMTP id C658F11E80E2 for <aqm@ietf.org>; Thu, 14 Nov 2013 05:32:57 -0800 (PST)
Received: by mail-vb0-f47.google.com with SMTP id g10so1683041vbg.6 for <aqm@ietf.org>; Thu, 14 Nov 2013 05:32:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nqIpuRqogvWlzFC53mhHx0vtY9fkpTDRz6KVGFOxux8=; b=fkAU+g04U9GZLGVB7XAxq06VH77+/Y9YeF8psG3sl5mTA/OGAMcS0o5wO8yRf934oh yVeEnzcPb538DmySUZNPTrIBzqOI9A/AQC2plpC0qttBFF1RtZ5fXyNEbbAomHiKMzIx XeFqMnczAZ312WK6hdtz4787sueeCOOaLFxNWe9EJqD/SyNEeen4+UfhVK0oqHu47hv/ GTLooASJ69HmIRfxo6/52ryR4UthWpl1FK9GAzdmLUtvhFuwulOS5KHjbxGrkxqyllrf EdUjeEWJpNfjc7+5Keotb2mYah+HjBeawDwYVUK8TQRo5Dblxu1bxQbZluSYTkrjGV+K OtCw==
MIME-Version: 1.0
X-Received: by 10.58.118.84 with SMTP id kk20mr787553veb.26.1384435976992; Thu, 14 Nov 2013 05:32:56 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 05:32:56 -0800 (PST)
In-Reply-To: <CEA961B1.4B416%prenatar@cisco.com>
References: <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com> <CEA961B1.4B416%prenatar@cisco.com>
Date: Thu, 14 Nov 2013 14:32:56 +0100
Message-ID: <CAEjQQ5VhMfgik_QS2GxWRbMDJQf7VXEC3-e_LP0tx5P16pC9PQ@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=089e0122a0869f860304eb231eb8
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 13:32:59 -0000

--089e0122a0869f860304eb231eb8
Content-Type: text/plain; charset=ISO-8859-1

Hi Preethi

Please see comments inline

On Thu, Nov 14, 2013 at 2:17 AM, Preethi Natarajan <preethi.cis@gmail.com>wrote:

>
>
> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Wednesday, November 13, 2013 12:14 PM
>
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
> Very true, though that's what ARED *is* with very low thresholds (e.g.
> resonate between 1 or 2 packets on the average in the queue). Indeed there
> can possibly be some future work on ARED's (or any other similar AQM) burst
> allowance but I don't think e.g. letting a 100 ms burst of packets to pass
> through can be a good way of fixing it as this can lead to massive variance
> in overall RTT distribution.
>
>
> If you are referring to the burst_allowance parameter in PIE, this
> parameter allows burst to go through when a system goes from an idle state
> into some congestion state. The one we are talking about is accommodating
> bursts when the system is in equilibirum
>

I wasn't referring to a PIE's parameter named "burst allowance" but rather
in a generic term.


> -- how does the AQM scheme accommodate bursts when already operating
> around target delay or congestion?
>

AFAIK all three studied AQMs (CoDel, PIE, ARED) have a parameter named
"target delay" or "target queuing". Judging based on the (param) name, they
supposedly try to maintain the queue delay/length around/at a certain value
although "target delay/queuing" has different semantics in each AQM and the
response to crossing that threshold/value is also different from one AQM to
another. Moreover, It is favorable for an AQM that tries to fix bufferbloat
problem to have a predictable delay distribution, most favorably around
(e.g. mean, median, max) a certain value. PIE uses it, CoDel has it and
ARED in fixed BW setups has it too. So, I'm not quite sure, why it
shouldn't be possible to allow a certain size of burst in such
threshold-based mechanisms (AFAIK (A)RED's Linux implementation already has
a "burst" param that can be tuned, although its semantics may not be
identical to the way CoDel or PIE handle bursts).


> Delay-based ARED behaves similar to tail drop at max threshold. This is
> unacceptable of an AQM even for max thresholds around 50ms.
>

I'm not sure if I understand this part correctly or even if I agree with
the conclusion it makes, assuming that I understood it correctly :-). If
I'm not mistaken, you're saying that "(delay-based) ARED  acts similar to
DropTail when the average queue size/delay grows beyond the th_max". If
that's the case, then I disagree since ARED uses a (delay-based) AIMD
function to tune the maximum drop probability and therefore it uses the AI
factor to increase its P_MAX when queue size/delay grows beyond the th_max
and on top of that, it does averaging, so nothing really similar to
DropTail.


> Clearly delay-based ARED needs significant and careful redesign. I
> suppose one can argue this task is as laborious as designing a new AQM
> scheme as opposed to previous notions that delay-based ARED would work
> right out of the box.
>

it's probably not accurate to say "redesign" as delay-based ARED doesn't
exist as of know. AFAIK given our latest results with ARED, we raised
the possibility of modifying ARED to use delay-based thresholds instead of
size-based thresholds during the ICCRG talk and in the TR, and declared it
as future work. I'm not aware of any other implementation/proposal of
delay-based ARED to exist as of now. We never claimed or speculated on how
this future delay-based AQM might perform and left it to when the future
work is done and the outcome is properly investigated and therefore we
never said that "it would work right out of the box" :-).


> Preethi
>

Cheers,
Naeem

--089e0122a0869f860304eb231eb8
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Preethi=A0<div><br></div><div>Please see comments inlin=
e<br><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Thu, Nov =
14, 2013 at 2:17 AM, Preethi Natarajan <span dir=3D"ltr">&lt;<a href=3D"mai=
lto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</a>&gt;<=
/span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-word"><div sty=
le=3D"font-size:14px;font-family:Calibri,sans-serif"><br></div><div style=
=3D"font-size:14px;font-family:Calibri,sans-serif">
<br></div><span style=3D"font-size:14px;font-family:Calibri,sans-serif"><di=
v style=3D"border-right:medium none;padding-right:0in;padding-left:0in;padd=
ing-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-f=
amily:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:m=
edium none">
<span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D=
"mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com<=
/a>&gt;<br><span style=3D"font-weight:bold">Date: </span> Wednesday, Novemb=
er 13, 2013 12:14 PM<div class=3D"im">
<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a h=
ref=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.co=
m</a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Michael Welzl &lt=
;<a href=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no=
</a>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.=
org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@iet=
f.org</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Qu=
eue length vs. delay based<br></div></div><div><br></div><blockquote style=
=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div dir=3D=
"ltr">
<div class=3D"gmail_extra"><div class=3D"gmail_quote"><div><br></div><div c=
lass=3D"im"><div>Very true, though that&#39;s what ARED *is* with very low =
thresholds (e.g. resonate between 1 or 2 packets on the average in the queu=
e). Indeed there can possibly be some future work on ARED&#39;s (or any oth=
er similar AQM) burst allowance but I don&#39;t think e.g. letting a 100 ms=
 burst of packets to pass through can be a good way of fixing it as this ca=
n lead to massive variance in overall RTT distribution. =A0</div>
</div></div></div></div></blockquote></span><div style=3D"font-size:14px;fo=
nt-family:Calibri,sans-serif"><br></div><div><div style=3D"padding-right:55=
px;padding-left:0px;padding-top:12px;margin-bottom:0px;padding-bottom:0px;m=
argin-right:0px;overflow-x:auto;overflow-y:auto;font-size:medium;margin-lef=
t:0px;font-family:Helvetica;word-wrap:break-word;margin-top:0px">
<span style=3D"word-wrap:break-word;font:normal normal normal 14px/normal C=
alibri">If you are referring to the burst_allowance parameter in PIE, this =
parameter allows burst to go through when a system goes from an idle state =
into some congestion state. The one we are talking about is accommodating b=
ursts when the system is in equilibirum </span></div>
</div></div></blockquote><div><br></div><div style>I wasn&#39;t referring t=
o a PIE&#39;s parameter named &quot;burst allowance&quot; but rather in a g=
eneric term. =A0</div><div>=A0</div><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><div><div style=3D"padding-right:55px;p=
adding-left:0px;padding-top:12px;margin-bottom:0px;padding-bottom:0px;margi=
n-right:0px;overflow-x:auto;overflow-y:auto;font-size:medium;margin-left:0p=
x;font-family:Helvetica;word-wrap:break-word;margin-top:0px">
<span style=3D"word-wrap:break-word;font:normal normal normal 14px/normal C=
alibri">-- how does the AQM scheme accommodate bursts when already operatin=
g around target delay or congestion? </span></div></div></div></blockquote>
<div><br></div><div style>AFAIK all three studied AQMs (CoDel, PIE, ARED) h=
ave a parameter named &quot;target delay&quot; or &quot;target queuing&quot=
;. Judging based on the (param) name, they supposedly try to maintain the q=
ueue delay/length around/at a certain value although &quot;target delay/que=
uing&quot; has different semantics in each AQM and the response to crossing=
 that threshold/value is also different from one AQM to another. Moreover, =
It is favorable for an AQM that tries to fix bufferbloat problem to have a=
=A0predictable delay distribution, most favorably around (e.g. mean, median=
, max) a certain value. PIE uses it, CoDel has it and ARED in fixed BW setu=
ps has it too. So, I&#39;m not quite sure, why it shouldn&#39;t be possible=
 to allow a certain size of burst in such threshold-based mechanisms (AFAIK=
 (A)RED&#39;s Linux implementation already has a &quot;burst&quot; param th=
at can be tuned, although its semantics may not be identical to the way CoD=
el or PIE handle bursts).=A0 =A0 =A0 =A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-=
word"><div><div style=3D"padding-right:55px;padding-left:0px;padding-top:12=
px;margin-bottom:0px;padding-bottom:0px;margin-right:0px;overflow-x:auto;ov=
erflow-y:auto;font-size:medium;margin-left:0px;font-family:Helvetica;word-w=
rap:break-word;margin-top:0px">
<span style=3D"word-wrap:break-word;font:normal normal normal 14px/normal C=
alibri">Delay-based ARED behaves similar to tail drop at max threshold. Thi=
s is unacceptable of an AQM even for max thresholds around 50ms.=A0</span><=
/div>
</div></div></blockquote><div><br></div><div style>I&#39;m not sure if I un=
derstand this part correctly or even if I agree with the conclusion it make=
s, assuming that I understood it correctly :-). If I&#39;m not mistaken, yo=
u&#39;re saying that &quot;(delay-based) ARED =A0acts similar to DropTail w=
hen the average queue size/delay grows beyond the th_max&quot;. If that&#39=
;s the case, then I disagree since ARED uses a (delay-based) AIMD function =
to tune the maximum drop probability and therefore it uses the AI factor to=
 increase its P_MAX when queue size/delay grows beyond the th_max and on to=
p of that, it does averaging, so nothing really similar to DropTail. =A0 =
=A0=A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-=
word"><div><div style=3D"padding-right:55px;padding-left:0px;padding-top:12=
px;margin-bottom:0px;padding-bottom:0px;margin-right:0px;overflow-x:auto;ov=
erflow-y:auto;font-size:medium;margin-left:0px;font-family:Helvetica;word-w=
rap:break-word;margin-top:0px">
<span style=3D"word-wrap:break-word;font:normal normal normal 14px/normal C=
alibri">Clearly delay-based ARED needs significant and careful redesign.=A0=
</span><span style=3D"font-size:14px;font-family:Calibri">I suppose one can=
 argue this task is as laborious as designing a new AQM scheme as opposed t=
o previous notions that delay-based ARED would work right out of the box.=
=A0</span></div>
</div></div></blockquote><div><br></div><div style>it&#39;s probably not ac=
curate to say &quot;redesign&quot; as delay-based ARED doesn&#39;t exist as=
 of know. AFAIK given our latest results with ARED, we raised the=A0possibi=
lity=A0of modifying ARED to use delay-based thresholds instead of size-base=
d thresholds during the ICCRG talk and in the TR, and declared it as future=
 work. I&#39;m not aware of any other implementation/proposal of delay-base=
d ARED to exist as of now. We never claimed or speculated on how this futur=
e delay-based AQM might perform and left it to when the future work is done=
 and the outcome is properly investigated and therefore we never said that =
&quot;it would work right out of the box&quot; :-). =A0 =A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-=
word"><span class=3D"HOEnZb"><font color=3D"#888888"><div style=3D"padding-=
right:55px;padding-left:0px;padding-top:12px;margin-bottom:0px;padding-bott=
om:0px;margin-right:0px;overflow-x:auto;overflow-y:auto;font-size:medium;ma=
rgin-left:0px;font-family:Helvetica;word-wrap:break-word;margin-top:0px">
<span style=3D"font-family:Calibri">Preethi</span></div></font></span></div=
>
</blockquote></div><br></div><div class=3D"gmail_extra" style>Cheers,</div>=
<div class=3D"gmail_extra" style>Naeem</div></div></div>

--089e0122a0869f860304eb231eb8--

From naeem.khademi@gmail.com  Thu Nov 14 07:43:52 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 66BEF21F979E for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 07:43:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.296
X-Spam-Level: 
X-Spam-Status: No, score=-2.296 tagged_above=-999 required=5 tests=[AWL=0.076,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ts3LQYpOSHhh for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 07:43:51 -0800 (PST)
Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) by ietfa.amsl.com (Postfix) with ESMTP id 5BA4811E8141 for <aqm@ietf.org>; Thu, 14 Nov 2013 07:43:41 -0800 (PST)
Received: by mail-vc0-f169.google.com with SMTP id hu8so838399vcb.0 for <aqm@ietf.org>; Thu, 14 Nov 2013 07:43:40 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CPA5fEZvhD2L2DIir1vM98MW/c5VPtPo3iWm8AifzX0=; b=BrllpFI1DocRWP39IAfORFx53N5XDOJNkw+0J0S19HiyKPJtxrZRm2tV93gKzgqT3d rIW2635vgSApEt+Va/UzNr7flnzBNphjzosMqKNc8HPCEetzcj+Vp49DBdtdQTY2uzx8 t2+cA62lq34o5iUyH4dXWpbwDCmvHkBqLi9i2GJr4T7bQvO0l02tAyiFmaBMKqxS8Z8k lb61gqMObzGCWHkbDLwFFApij/JiXCTr6rh/u1bpqXPOtwPuhbmuvlxHAA0VoNtVsPSj /rkcnYQoKcoPTtbTdBlBB6qQpgRzOkXw/ngQjtkuSXNjdW/GJkScdZ0Ue951uEhV0d5O +v/Q==
MIME-Version: 1.0
X-Received: by 10.52.122.15 with SMTP id lo15mr44658vdb.115.1384443820731; Thu, 14 Nov 2013 07:43:40 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 07:43:40 -0800 (PST)
In-Reply-To: <CAEjQQ5VhMfgik_QS2GxWRbMDJQf7VXEC3-e_LP0tx5P16pC9PQ@mail.gmail.com>
References: <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com> <CEA961B1.4B416%prenatar@cisco.com> <CAEjQQ5VhMfgik_QS2GxWRbMDJQf7VXEC3-e_LP0tx5P16pC9PQ@mail.gmail.com>
Date: Thu, 14 Nov 2013 16:43:40 +0100
Message-ID: <CAEjQQ5We5HHbePXTu4GfenM+UwcdpundVF30Zg9TCD+H7jXMCg@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=089e013a13302575e504eb24f233
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 15:43:52 -0000

--089e013a13302575e504eb24f233
Content-Type: text/plain; charset=ISO-8859-1

> Delay-based ARED behaves similar to tail drop at max threshold.
>>
>
I think I now understand what you mean by this sentence ;-) and therefore
please ignore the paragraph responding to this specific point in the
previous email (sorry about that, I got it mixed with max_target). Indeed
ARED drops every packet when the average queue length grows beyond th_max
as drop probability jumps to 1 (when it's not in gentle mode). For this to
happen, the "average queue length" has to grow beyond th_max and depending
on how the averaging is done, that can correspond to certain burst
allowance at AQM. Moreover, this is done over 500 ms interval which means
the average queue length has to stay above th_max for mostly few rounds of
RTTs (assuming the typical distribution of RTTs on the Internet to have a
mean/average somewhere below that value e.g. 100~200 ms). This allows burst
that don't take the average queue size to above th_max over 500 ms period
to pass and clear themselves up the queue.

 > This is unacceptable of an AQM even for max thresholds around 50ms.

 Whether this is a good thing or bad thing is a subjective matter in my
opinion, as most AQMs at some point will drop every packet whether it be
the maximum queue length or at some thresholds.

Cheers,
Naeem

--089e013a13302575e504eb24f233
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><span style=3D"color:rgb(80,0,80)">=A0</span><br><div clas=
s=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quo=
te" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-col=
or:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
 class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px=
 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left=
-style:solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><div><div style=3D"padding:12px 55px 0p=
x 0px;margin:0px;overflow:auto;font-size:medium;font-family:Helvetica;word-=
wrap:break-word">
<span style=3D"word-wrap:break-word;font-style:normal;font-variant:normal;f=
ont-weight:normal;font-size:14px;line-height:normal;font-family:Calibri">De=
lay-based ARED behaves similar to tail drop at max threshold.</span></div>
</div></div></blockquote></div></div></div></div></blockquote><div><br></di=
v><div style>I think I now understand what you mean by this sentence ;-) an=
d therefore please ignore the paragraph responding to this specific point i=
n the previous email (sorry about that, I got it mixed with max_target). In=
deed ARED drops every packet when the average queue length grows beyond th_=
max as drop probability jumps to 1 (when it&#39;s not in gentle mode). For =
this to happen, the &quot;average queue length&quot; has to grow beyond th_=
max and depending on how the averaging is done, that can correspond to cert=
ain burst allowance at AQM. Moreover, this is done over 500 ms interval whi=
ch means the average queue length has to stay above th_max for mostly few r=
ounds of RTTs (assuming the typical distribution of RTTs on the Internet to=
 have a mean/average somewhere below that value e.g. 100~200 ms). This allo=
ws burst that don&#39;t take the average queue size to above th_max over 50=
0 ms period to pass and clear themselves up the queue. =A0=A0=A0 =A0 =A0=A0=
</div>
<div style><br></div><div style><span style=3D"color:rgb(80,0,80);font-fami=
ly:Calibri;font-size:14px">=A0&gt; This is unacceptable of an AQM even for =
max thresholds around 50ms.=A0</span><br></div><div style><span style=3D"co=
lor:rgb(80,0,80);font-family:Calibri;font-size:14px"><br>
</span></div><div style>=A0Whether this is a good thing or bad thing is a s=
ubjective matter in my opinion, as most AQMs at some point will drop every =
packet whether it be the maximum queue length or at some thresholds.=A0<spa=
n style=3D"color:rgb(80,0,80);font-family:Calibri;font-size:14px"><br>
</span></div><div style><br></div><div style>Cheers,</div><div style>Naeem<=
/div></div></div></div>

--089e013a13302575e504eb24f233--

From naeem.khademi@gmail.com  Thu Nov 14 08:05:08 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F037F21E80B2 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 08:05:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.305
X-Spam-Level: 
X-Spam-Status: No, score=-2.305 tagged_above=-999 required=5 tests=[AWL=0.067,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gxo26L-mcWuG for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 08:05:07 -0800 (PST)
Received: from mail-vb0-x232.google.com (mail-vb0-x232.google.com [IPv6:2607:f8b0:400c:c02::232]) by ietfa.amsl.com (Postfix) with ESMTP id 2209D21E80D2 for <aqm@ietf.org>; Thu, 14 Nov 2013 08:05:07 -0800 (PST)
Received: by mail-vb0-f50.google.com with SMTP id x11so1866934vbb.37 for <aqm@ietf.org>; Thu, 14 Nov 2013 08:05:05 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hMR5IJ/Ux6Sbgfq/XXmavKItIDv0NORorPhFkC8NvJc=; b=rQGF4Xed5qEtB9T1syQF6TWh8LSihxnZcfL0glEYj+JT8hY3fqv+9ul+vpVvWPP57e Ds+AdrxwvMN42HOD6hGp1auIshyJL2H8eJUqrOGQ4Zh8sypAWuhKWJtq0uZXJRxjOdiy HT9oUOpgTxe88ofUP/44rj9FKC3hWpSSz3RaQRGCMMXK65vrhkXtvG3XzU7EZyMxr6lF Ls95X3ETcPhhmGCZVEkJcTd6Ltmlvla7pEK7uoZHIq4lTn5F6Cu1LKynj1puL78d1+vV RYc/Mt08fcOvZJ1E8buszuvtKlmJYsIA98YDh93LD0m3E6YTR1gCbOGEJL1uXiWGBMIK q+YA==
MIME-Version: 1.0
X-Received: by 10.58.151.3 with SMTP id um3mr111269veb.97.1384445104553; Thu, 14 Nov 2013 08:05:04 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 08:05:04 -0800 (PST)
In-Reply-To: <CA+-tSzyk=myS9B5tJ465C2JG9QvH8fPCG6Uu1iZ94TgPwV9s+w@mail.gmail.com>
References: <CA+-tSzyKT9gS_12V=yyia8iPt9FiDg6=NRKrB3FLdCRDTi=6Pg@mail.gmail.com> <201311132323.rADNNIKM047235@gateway1.ipv6.occnc.com> <CA+-tSzyk=myS9B5tJ465C2JG9QvH8fPCG6Uu1iZ94TgPwV9s+w@mail.gmail.com>
Date: Thu, 14 Nov 2013 17:05:04 +0100
Message-ID: <CAEjQQ5U+86aT=V9FxvSam9mQNzvUdtuf25HV55XXMQPE5095Og@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Anoop Ghanwani <anoop@alumni.duke.edu>
Content-Type: multipart/alternative; boundary=047d7b6d7b26ab050604eb253ebf
Cc: Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>, curtis@ipv6.occnc.com
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 16:05:08 -0000

--047d7b6d7b26ab050604eb253ebf
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Nov 14, 2013 at 12:33 AM, Anoop Ghanwani <anoop@alumni.duke.edu>wrote:

>
>
>
> On Wed, Nov 13, 2013 at 3:23 PM, Curtis Villamizar <curtis@ipv6.occnc.com>wrote:
>
>>
>> Including unrealistic scenarios, like going from near zero traffic to 10
>> interfaces feeding one at full speed until overflow occurs, is
>> counterproductive.
>
>
> It is actually a problem that keeps many people busy because a number of
> data center switches have very high port count with very small buffers.
>  Some people address these buy using switches with bigger buffers, but
> that's not a luxury that everyone indulges in.
>

Fair point about the incast problem and I personally think AQMs designed
for data centers should address that.


> That is why I specifically asked the question at the AQM meeting about
> applicability of AQM to all types of networks/switches.  I was told the
> answer is "yes" and so I would like to see this scenario addressed as well.
>
>

Below is my personal opinion, but hopefully Fred can clarify this better
based on the AQM recommendations draft:

"applicability of AQM to all types of networks/switches" => "yes"

"applicability of *any* AQM to all types of networks/switches" => "no"


> Perhaps AQM cannot help this, but hopefully it won't hurt.  Trying to do
> fancy things with small buffers is challenging.
>

AQM will most likely to help from data centers to the access links, and so
on. But we may possibly need different AQMs for different network
scenarios; The fact that an AQMs should be auto-tunable doesn't imply that
it can be applied everywhere and we may need different auto-tunable AQMs
specifically designed for different networks (ideally better if we could
use fewer of them and they could work everywhere). I hope I'm not wrong.


>
> Anoop
>

Naeem

--047d7b6d7b26ab050604eb253ebf
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Thu, Nov 14, 2013 at 12:33 AM, Anoop Ghanwani <span dir=3D"ltr">=
&lt;<a href=3D"mailto:anoop@alumni.duke.edu" target=3D"_blank">anoop@alumni=
.duke.edu</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><br><div class=3D"gmail_ext=
ra"><br><br><div class=3D"gmail_quote"><div class=3D"im">On Wed, Nov 13, 20=
13 at 3:23 PM, Curtis Villamizar <span dir=3D"ltr">&lt;<a href=3D"mailto:cu=
rtis@ipv6.occnc.com" target=3D"_blank">curtis@ipv6.occnc.com</a>&gt;</span>=
 wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><br>
Including unrealistic scenarios, like going from near zero traffic to 10<br=
>
interfaces feeding one at full speed until overflow occurs, is<br>
counterproductive.=A0</blockquote><div>=A0</div></div><div>It is actually a=
 problem that keeps many people busy because a number of data center switch=
es have very high port count with very small buffers. =A0Some people addres=
s these buy using switches with bigger buffers, but that&#39;s not a luxury=
 that everyone indulges in.</div>
</div></div></div></blockquote><div><br></div><div style>Fair point about t=
he incast problem and I personally think AQMs designed for data centers sho=
uld address that.=A0</div><div>=A0</div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
></div><div>That is why I specifically asked the question at the AQM meetin=
g about applicability of AQM to all types of networks/switches. =A0I was to=
ld the answer is &quot;yes&quot; and so I would like to see this scenario a=
ddressed as well. =A0</div>
</div></div></div></blockquote><div><br></div><div style>Below is my person=
al opinion, but hopefully Fred can clarify this better based on the AQM rec=
ommendations draft:</div><div style><br></div><div style>&quot;applicabilit=
y of AQM to all types of networks/switches&quot; =3D&gt; &quot;yes&quot;</d=
iv>
<div style><br></div><div style>&quot;applicability of *any* AQM to all typ=
es of networks/switches&quot; =3D&gt; &quot;no&quot; =A0</div><div>=A0</div=
><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1=
px #ccc solid;padding-left:1ex">
<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
>Perhaps AQM cannot help this, but hopefully it won&#39;t hurt. =A0Trying t=
o do fancy things with small buffers is challenging.</div></div></div></div=
></blockquote>
<div><br></div><div style>AQM will most likely to help from data centers to=
 the access links, and so on. But we may possibly need different AQMs for d=
ifferent network scenarios; The fact that an AQMs should be auto-tunable do=
esn&#39;t imply that it can be applied everywhere and we may need different=
 auto-tunable AQMs specifically designed for different networks (ideally be=
tter if we could use fewer of them and they could work everywhere). I hope =
I&#39;m not wrong. =A0=A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D=
"gmail_extra"><div class=3D"gmail_quote"><span class=3D"HOEnZb"><font color=
=3D"#888888">
<div><br></div><div>Anoop</div></font></span></div></div></div>
</blockquote></div><br></div><div class=3D"gmail_extra" style>Naeem</div></=
div>

--047d7b6d7b26ab050604eb253ebf--

From preethi.cis@gmail.com  Thu Nov 14 11:46:33 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BC1B21E80B6 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 11:46:33 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JFNLgpfWv4Vi for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 11:46:32 -0800 (PST)
Received: from mail-pb0-x22c.google.com (mail-pb0-x22c.google.com [IPv6:2607:f8b0:400e:c01::22c]) by ietfa.amsl.com (Postfix) with ESMTP id 904EA21E8098 for <aqm@ietf.org>; Thu, 14 Nov 2013 11:46:30 -0800 (PST)
Received: by mail-pb0-f44.google.com with SMTP id rp16so2488229pbb.31 for <aqm@ietf.org>; Thu, 14 Nov 2013 11:46:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=HhBSdPttBiMbnqUwbc2skpFZtpunyk8MGuickfyuWYE=; b=CC65h2RLXiU7GwuOeaEoiVglIJUIdgn9L5AZuGohKPCR87cJkRY+6niso0Cc5ffRQj c41VA8v4JJo5tJin81E4IEv4dtCD7Wutfma9o58PuyoLESUA0OrhRCBFQsk+7uaUAOcM qdHkPBEFJMdLaRZ1nr/pFoflZzNrjKQGwIJKiud9dv++hAyCkIDwV0CbsBopnflOMqyW vPCk2TFTQap+F7I43oVoYU91EWpvvZTCm34TF6By7ugh5CCLhVf5MDILEr2yXElKQoFV kKCuLhZjPW+A9tNWya9NeON5fgWL1J+PhyuqYQQOwp06kDP4peCLLoZhsY2VQdIhfD88 FhKw==
X-Received: by 10.68.99.99 with SMTP id ep3mr3002925pbb.107.1384458390354; Thu, 14 Nov 2013 11:46:30 -0800 (PST)
Received: from [192.168.1.105] (c-76-103-130-90.hsd1.ca.comcast.net. [76.103.130.90]) by mx.google.com with ESMTPSA id sg1sm53371252pbb.16.2013.11.14.11.46.26 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 14 Nov 2013 11:46:29 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Thu, 14 Nov 2013 11:46:21 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEAA66F5.4B52B%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CAEjQQ5VhMfgik_QS2GxWRbMDJQf7VXEC3-e_LP0tx5P16pC9PQ@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467274388_7553532"
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 19:46:33 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467274388_7553532
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit



From:  Naeem Khademi <naeem.khademi@gmail.com>
Date:  Thursday, November 14, 2013 5:32 AM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> 
> AFAIK all three studied AQMs (CoDel, PIE, ARED) have a parameter named "target
> delay" or "target queuing". Judging based on the (param) name, they supposedly
> try to maintain the queue delay/length around/at a certain value although
> "target delay/queuing" has different semantics in each AQM and the response to
> crossing that threshold/value is also different from one AQM to another.
> Moreover, It is favorable for an AQM that tries to fix bufferbloat problem to
> have a predictable delay distribution, most favorably around (e.g. mean,
> median, max) a certain value. PIE uses it, CoDel has it and ARED in fixed BW
> setups has it too. So, I'm not quite sure, why it shouldn't be possible to
> allow a certain size of burst in such threshold-based mechanisms (AFAIK
> (A)RED's Linux implementation already has a "burst" param that can be tuned,
> although its semantics may not be identical to the way CoDel or PIE handle
> bursts).    

Of course. The question is how effective is the mechanism. As you note,
whatever "burst" mechanism exists in Linux's ARED wasn't effective as
evident from the low throughput in the plots.
  
>   
>> Clearly delay-based ARED needs significant and careful redesign. I suppose
>> one can argue this task is as laborious as designing a new AQM scheme as
>> opposed to previous notions that delay-based ARED would work right out of the
>> box. 
> 
> it's probably not accurate to say "redesign" as delay-based ARED doesn't exist
> as of know. AFAIK given our latest results with ARED, we raised the
> possibility of modifying ARED to use delay-based thresholds instead of
> size-based thresholds during the ICCRG talk and in the TR, and declared it as
> future work. I'm not aware of any other implementation/proposal of delay-based
> ARED to exist as of now. We never claimed or speculated on how this future
> delay-based AQM might perform and left it to when the future work is done and
> the outcome is properly investigated and therefore we never said that "it
> would work right out of the box" :-).
>  

Great. Thanks for confirming this point that ARED as is does not work well
for the scenarios considered. What is required is new AQM design, not just
plugging delay into ARED.
Preethi



--B_3467274388_7553532
Content-type: text/html;
	charset="US-ASCII"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; "><div style=3D"color: rgb(0, 0, =
0); font-size: 14px; font-family: Calibri, sans-serif; "><br></div><div styl=
e=3D"color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "=
><br></div><span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); font-=
size: 14px; font-family: Calibri, sans-serif; "><div style=3D"font-family:Cali=
bri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium non=
e; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING=
-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDI=
NG-TOP: 3pt"><span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;=
<a href=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br>=
<span style=3D"font-weight:bold">Date: </span> Thursday, November 14, 2013 5:3=
2 AM<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a =
href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span s=
tyle=3D"font-weight:bold">Cc: </span> Michael Welzl &lt;<a href=3D"mailto:michaw=
e@ifi.uio.no">michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailto:aqm@ietf.org">aqm=
@ietf.org</a>" &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><sp=
an style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue le=
ngth vs. delay based<br></div><div><br></div><blockquote id=3D"MAC_OUTLOOK_ATT=
RIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; M=
ARGIN:0 0 0 5;"><div dir=3D"ltr"><div><div class=3D"gmail_extra"><div class=3D"gma=
il_quote"><div><br></div><div style=3D"">AFAIK all three studied AQMs (CoDel, =
PIE, ARED) have a parameter named "target delay" or "target queuing". Judgin=
g based on the (param) name, they supposedly try to maintain the queue delay=
/length around/at a certain value although "target delay/queuing" has differ=
ent semantics in each AQM and the response to crossing that threshold/value =
is also different from one AQM to another. Moreover, It is favorable for an =
AQM that tries to fix bufferbloat problem to have a&nbsp;predictable delay d=
istribution, most favorably around (e.g. mean, median, max) a certain value.=
 PIE uses it, CoDel has it and ARED in fixed BW setups has it too. So, I'm n=
ot quite sure, why it shouldn't be possible to allow a certain size of burst=
 in such threshold-based mechanisms (AFAIK (A)RED's Linux implementation alr=
eady has a "burst" param that can be tuned, although its semantics may not b=
e identical to the way CoDel or PIE handle bursts).&nbsp; &nbsp; </div></div=
></div></div></div></blockquote></span><div style=3D"color: rgb(0, 0, 0); font=
-size: 14px; font-family: Calibri, sans-serif; "><br></div><div style=3D"color=
: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">Of cour=
se. The question is how effective is the mechanism. As you note, whatever "b=
urst" mechanism exists in Linux's ARED wasn't effective as evident from the =
low throughput in the plots.&nbsp;</div><div style=3D"color: rgb(0, 0, 0); fon=
t-size: 14px; font-family: Calibri, sans-serif; ">&nbsp;&nbsp;</div><span id=
=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); font-size: 14px; "><bloc=
kquote id=3D"MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style=3D"border-left-color: rgb=
(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding-t=
op: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 5px; margin-=
top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 5px; "><div di=
r=3D"ltr"><div><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div style=3D"f=
ont-family: Calibri, sans-serif; ">&nbsp;&nbsp;</div><blockquote class=3D"gmai=
l_quote" style=3D"font-family: Calibri, sans-serif; margin-top: 0px; margin-ri=
ght: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; bo=
rder-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left:=
 1ex; "><div style=3D"word-wrap:break-word"><div><div style=3D"padding-right:55p=
x;padding-left:0px;padding-top:12px;margin-bottom:0px;padding-bottom:0px;mar=
gin-right:0px;overflow-x:auto;overflow-y:auto;font-size:medium;margin-left:0=
px;font-family:Helvetica;word-wrap:break-word;margin-top:0px"><span style=3D"w=
ord-wrap:break-word;font:normal normal normal 14px/normal Calibri">Clearly d=
elay-based ARED needs significant and careful redesign.&nbsp;</span><span st=
yle=3D"font-size:14px;font-family:Calibri">I suppose one can argue this task i=
s as laborious as designing a new AQM scheme as opposed to previous notions =
that delay-based ARED would work right out of the box.&nbsp;</span></div></d=
iv></div></blockquote><div style=3D"font-family: Calibri, sans-serif; "><br></=
div><div style=3D"font-family: Calibri, sans-serif; ">it's probably not accura=
te to say "redesign" as delay-based ARED doesn't exist as of know. AFAIK giv=
en our latest results with ARED, we raised the&nbsp;possibility&nbsp;of modi=
fying ARED to use delay-based thresholds instead of size-based thresholds du=
ring the ICCRG talk and in the TR, and declared it as future work. I'm not a=
ware of any other implementation/proposal of delay-based ARED to exist as of=
 now. We never claimed or speculated on how this future delay-based AQM migh=
t perform and left it to when the future work is done and the outcome is pro=
perly investigated and therefore we never said that "it would work right out=
 of the box" :-). &nbsp; &nbsp;</div><div style=3D"font-family: Calibri, sans-=
serif; ">&nbsp;</div></div></div></div></div></blockquote></span><div style=3D=
"color: rgb(0, 0, 0); font-size: 14px; "><br></div><div style=3D"color: rgb(0,=
 0, 0); font-size: 14px; ">Great. Thanks for confirming this point that&nbsp=
;ARED as is does not work well for the scenarios considered. What is require=
d is new AQM design, not just plugging delay into ARED.</div><span id=3D"OLK_S=
RC_BODY_SECTION"><div dir=3D"ltr"><div><div class=3D"gmail_extra"><div class=3D"gm=
ail_quote"><div style=3D"word-wrap: break-word; "><span class=3D"HOEnZb"><font><=
div style=3D"padding-right: 55px; padding-left: 0px; padding-top: 12px; margin=
-bottom: 0px; padding-bottom: 0px; margin-right: 0px; overflow-x: auto; over=
flow-y: auto; margin-left: 0px; word-wrap: break-word; margin-top: 0px; ">Pr=
eethi</div></font></span></div></div></div></div></div></span></body></html>

--B_3467274388_7553532--



From ropan@cisco.com  Thu Nov 14 11:59:39 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B7E0B21E80B6 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 11:59:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.371
X-Spam-Level: 
X-Spam-Status: No, score=-10.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KR1yNwL9Zqdg for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 11:59:33 -0800 (PST)
Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76]) by ietfa.amsl.com (Postfix) with ESMTP id 9465711E80FA for <aqm@ietf.org>; Thu, 14 Nov 2013 11:59:33 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=13414; q=dns/txt; s=iport; t=1384459173; x=1385668773; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=mm6TBN1clQfupcJzpsIfluR/TtfsRx0js87H2XJBwbY=; b=P83uJNO8KRBFV0C3HEKJrGD2SRRH3S4p3OzaPrkC7ITEjYlo4ZlRWe11 fKVB5dLRRSrl9PjTrP/2tWBqi+AfO05SyupcPbvPyx08Zg404alDYmIKm W/DY94EOchA89kDuxOXj/lUSIvC1UgIMfKfungXjFY07yE1+mo/4QbLwH Y=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgkFAJ4qhVKtJV2c/2dsb2JhbABbgkNEgQu/GYEhFnSCJQEBAQMBbAYHBQ0BCBEDAQIoKBEUCQgCBAENBYdvAwkGtxkNiT+MbYEpgTgRB4QxA5YlgWuMVIU4gyiBcTk
X-IronPort-AV: E=Sophos;i="4.93,701,1378857600";  d="scan'208,217";a="284937987"
Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-5.cisco.com with ESMTP; 14 Nov 2013 19:59:33 +0000
Received: from xhc-rcd-x03.cisco.com (xhc-rcd-x03.cisco.com [173.37.183.77]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id rAEJxWjR026087 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 14 Nov 2013 19:59:32 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.147]) by xhc-rcd-x03.cisco.com ([173.37.183.77]) with mapi id 14.03.0123.003; Thu, 14 Nov 2013 13:59:32 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
Thread-Index: AQHO3CzV4Y8AQmoIJE6ksgN55dTkopobOREAgADcp4CAACSPAP//9emAgABSr4CABi75AIAA200AgABagICAAB6IAIAAVMEAgADNWwCAACSHAP//sJoA
Date: Thu, 14 Nov 2013 19:59:31 +0000
Message-ID: <CEAA6932.55A55%ropan@cisco.com>
In-Reply-To: <CAEjQQ5We5HHbePXTu4GfenM+UwcdpundVF30Zg9TCD+H7jXMCg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [128.107.165.143]
Content-Type: multipart/alternative; boundary="_000_CEAA693255A55ropanciscocom_"
MIME-Version: 1.0
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 19:59:39 -0000

--_000_CEAA693255A55ropanciscocom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

Please see inline=85

Thanks,

Rong



From: Naeem Khademi <naeem.khademi@gmail.com<mailto:naeem.khademi@gmail.com=
>>
Date: Thursday, November 14, 2013 8:43 AM
To: Preethi Natarajan <preethi.cis@gmail.com<mailto:preethi.cis@gmail.com>>
Cc: "aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>=
, Michael Welzl <michawe@ifi.uio.no<mailto:michawe@ifi.uio.no>>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based


Delay-based ARED behaves similar to tail drop at max threshold.

I think I now understand what you mean by this sentence ;-) and therefore p=
lease ignore the paragraph responding to this specific point in the previou=
s email (sorry about that, I got it mixed with max_target). Indeed ARED dro=
ps every packet when the average queue length grows beyond th_max as drop p=
robability jumps to 1 (when it's not in gentle mode). For this to happen, t=
he "average queue length" has to grow beyond th_max and depending on how th=
e averaging is done, that can correspond to certain burst allowance at AQM.=
 Moreover, this is done over 500 ms interval which means the average queue =
length has to stay above th_max for mostly few rounds of RTTs (assuming the=
 typical distribution of RTTs on the Internet to have a mean/average somewh=
ere below that value e.g. 100~200 ms). This allows burst that don't take th=
e average queue size to above th_max over 500 ms period to pass and clear t=
hemselves up the queue.

 > This is unacceptable of an AQM even for max thresholds around 50ms.

-----------------------------
>>>>>>>>>>>>>RP: Two issues here, first, averaging has a parameter. How to =
tune it to avoid tail drop behavior? What would be its relationship to max_=
th? Do we understand ?  Second, one single tcp's behavior is very bursty, i=
f 500ms burst allowance is effective in ARED, how come is it not reflected =
in your throughput chart? The one single tcp throughput is very low for low=
 latency numbers, which is a clear indication of not enough buffering.
------------------------------------


Whether this is a good thing or bad thing is a subjective matter in my opin=
ion, as most AQMs at some point will drop every packet whether it be the ma=
ximum queue length or at some thresholds.

----------------------------
>>>>>>>>>>>>RP: I don=92t think so. Neither PIE nor Codel has drop-all thre=
shold as AQM part of it. Tail drop exists, but it is not part of AQM. Putti=
ng the latency-based  ARED aside, ARED in this current form has the issues =
that we mentioned in the previous emails. It needs significant and careful =
redesign.
---------------------------

There is also a comment about "Throwing away all the knowledge and improvem=
ent gained from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs=
 =85".

------------------------------------------
>>>>>>>>>>RP: After I worked on CHOKe (now in Linux Kernel)  in 2000 as par=
t of my Ph.D. thesis at Stanford, I have worked on AFD (implemented in many=
 Cisco flagship products), QCN (now IEEEE standard for data center ethernet=
, 802.1Qau), all of which are AQM schemes. I also helped writing guidelines=
 in tuning WRED at Cisco. New thinkings were brought into PIE after I learn=
ed precious lessons (both good and bad) from all of them. Discarding them a=
s "coming up with brand-new AQMs" is ignorant.

Regards,

Rong
--------------------------------


Cheers,
Naeem

>>>>>>>>>>>>RP:

--_000_CEAA693255A55ropanciscocom_
Content-Type: text/html; charset="Windows-1252"
Content-ID: <4D9CD117DE31EC458252E140312516C5@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1=
252">
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; ">
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
Please see inline=85</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
Thanks,</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
Rong</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); font-size: =
16px; font-family: Calibri, sans-serif; ">
<div style=3D"font-family:Calibri; font-size:14pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>Naeem Khademi &lt;<a href=3D"=
mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Thursday, November 14, 2013 8=
:43 AM<br>
<span style=3D"font-weight:bold">To: </span>Preethi Natarajan &lt;<a href=
=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br>
<span style=3D"font-weight:bold">Cc: </span>&quot;<a href=3D"mailto:aqm@iet=
f.org">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.=
org</a>&gt;, Michael Welzl &lt;<a href=3D"mailto:michawe@ifi.uio.no">michaw=
e@ifi.uio.no</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span>Re: [aqm] AQM schemes: Que=
ue length vs. delay based<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir=3D"ltr"><span style=3D"color:rgb(80,0,80)">&nbsp;</span><br>
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div class=3D"im">
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div style=3D"word-wrap:break-word">
<div>
<div style=3D"padding:12px 55px 0px 0px;margin:0px;overflow:auto;font-size:=
medium;font-family:Helvetica;word-wrap:break-word">
<span style=3D"word-wrap:break-word;font-style:normal;font-variant:normal;f=
ont-weight:normal;font-size:14px;line-height:normal;font-family:Calibri">De=
lay-based ARED behaves similar to tail drop at max threshold.</span></div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div style=3D"">I think I now understand what you mean by this sentence ;-)=
 and therefore please ignore the paragraph responding to this specific poin=
t in the previous email (sorry about that, I got it mixed with max_target).=
 Indeed ARED drops every packet when
 the average queue length grows beyond th_max as drop probability jumps to =
1 (when it's not in gentle mode). For this to happen, the &quot;average que=
ue length&quot; has to grow beyond th_max and depending on how the averagin=
g is done, that can correspond to certain
 burst allowance at AQM. Moreover, this is done over 500 ms interval which =
means the average queue length has to stay above th_max for mostly few roun=
ds of RTTs (assuming the typical distribution of RTTs on the Internet to ha=
ve a mean/average somewhere below
 that value e.g. 100~200 ms). This allows burst that don't take the average=
 queue size to above th_max over 500 ms period to pass and clear themselves=
 up the queue. &nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;</div>
<div style=3D""><br>
</div>
<div style=3D""><span style=3D"color:rgb(80,0,80);font-family:Calibri;font-=
size:14px">&nbsp;&gt; This is unacceptable of an AQM even for max threshold=
s around 50ms.&nbsp;</span></div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
-----------------------------</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: Two issues here, fi=
rst, averaging has a parameter. How to tune it to avoid tail drop behavior?=
 What would be its relationship to max_th? Do we understand ? &nbsp;Second,=
 one single tcp's behavior is very bursty, if 500ms burst allowance is effe=
ctive
 in ARED, how come is it not reflected in your throughput chart? The one si=
ngle tcp throughput is very low for low latency numbers, which is a clear i=
ndication of not enough buffering.&nbsp;</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<span class=3D"Apple-style-span" style=3D"color: rgb(80, 0, 80); font-size:=
 14px; font-family: Calibri; ">------------------------------------</span><=
/div>
</div>
</div>
</div>
</div>
</div>
</span>
<div><br>
</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); font-size: =
16px; font-family: Calibri, sans-serif; ">
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"">Whether this is a good thing or bad thing is a subjective m=
atter in my opinion, as most AQMs at some point will drop every packet whet=
her it be the maximum queue length or at some thresholds.&nbsp;</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
----------------------------</div>
<div style=3D"font-family: Calibri, sans-serif; color: rgb(0, 0, 0); ">
<div style=3D"font-size: 16px; ">&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&g=
t;&gt;RP: I don=92t think so. Neither PIE nor Codel has drop-all threshold =
as AQM part of it. Tail drop exists, but it is not part of AQM. Putting the=
 latency-based &nbsp;ARED aside, ARED in this current form has the issues t=
hat
 we mentioned in the previous emails. It needs significant and careful rede=
sign.&nbsp;</div>
<div style=3D"font-size: 16px; ">---------------------------</div>
<div style=3D"font-size: 16px; "><br>
</div>
<div style=3D"font-size: 16px; ">There is also a comment about &quot;<span =
class=3D"Apple-style-span" style=3D"font-size: 14px; ">Throwing away all th=
e knowledge and improvement gained from RED-like AQMs, CHoKe, etc and comin=
g up with brand-new AQMs =85&quot;.</span></div>
<div style=3D"font-size: 16px; "><span class=3D"Apple-style-span" style=3D"=
font-size: 14px; "><br>
</span></div>
<div style=3D"font-size: 16px; "><span class=3D"Apple-style-span" style=3D"=
font-size: 14px; ">------------------------------------------</span></div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: After I worked on CHOKe (n=
ow in Linux Kernel) &nbsp;in 2000 as part of my Ph.D. thesis at Stanford, I=
 have worked on AFD (implemented in many Cisco flagship products), QCN (now=
 IEEEE standard for data center ethernet, 802.1Qau), all of which are
 AQM schemes.&nbsp;I also helped writing guidelines in tuning WRED at Cisco=
.&nbsp;New thinkings were brought into PIE after I learned precious lessons=
 (both good and bad) from all of them. Discarding them as &quot;coming up w=
ith brand-new AQMs&quot; is ignorant.&nbsp;</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Rong</div>
</div>
<div><span class=3D"Apple-style-span" style=3D"color: rgb(80, 0, 80); ">---=
-----------------------------</span></div>
<div><span class=3D"Apple-style-span" style=3D"color: rgb(80, 0, 80); "><br=
>
</span></div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
Cheers,</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
Naeem</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
<br>
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-s=
ize: 16px; ">
&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP:</div>
</body>
</html>

--_000_CEAA693255A55ropanciscocom_--

From preethi.cis@gmail.com  Thu Nov 14 12:00:21 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CDA421E8137 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:00:21 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bLYYy3yuaGLC for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:00:20 -0800 (PST)
Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) by ietfa.amsl.com (Postfix) with ESMTP id 7503D21E8134 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:00:16 -0800 (PST)
Received: by mail-pa0-f45.google.com with SMTP id kx10so1788589pab.32 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:00:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=g6VkEVsQp5XOqAwQl3XQHaXtiTaIfL5CQdA4IhVAbWU=; b=axaEyLsORKhChhl+Ll122pyfrLawbb1LAqdg4gfvhXUk6uWF0fqFiFttRvHMRS0v9U uru5GK0Mm3k7XFsY4L8jH/ya77OGkxWS5FTvgMB6GZqywm4/a9BzBhG+FJB7rFui0INb KWbHKCf/l7JBPPwT9/IwmnJvVX5QydKFTxYFFdTbtBwHhQeJB0YBnRSGIXKkkmLGPo6y PpfoVE7yrxwDJSk9z/gLVu5d704xJA7sriDg9dG0DjYNlf3pax7Oo9OVuzr/nqiBfPcO lKIXUdJOZ9RmUZI3hUiRFXSXSMdPsgpupfNfKxfAvrYrssQQ2Iu9lElLqLDX4Q9BU1/R Q1bA==
X-Received: by 10.68.171.164 with SMTP id av4mr3214109pbc.94.1384459216151; Thu, 14 Nov 2013 12:00:16 -0800 (PST)
Received: from [192.168.1.105] (c-76-103-130-90.hsd1.ca.comcast.net. [76.103.130.90]) by mx.google.com with ESMTPSA id sg1sm53415406pbb.16.2013.11.14.12.00.14 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 14 Nov 2013 12:00:15 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Thu, 14 Nov 2013 12:00:05 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEAA68B4.4B53A%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CAEjQQ5U+86aT=V9FxvSam9mQNzvUdtuf25HV55XXMQPE5095Og@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467275215_7580249"
Cc: Anoop Ghanwani <anoop@alumni.duke.edu>, "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>, curtis@ipv6.occnc.com
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 20:00:21 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467275215_7580249
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit



From:  Naeem Khademi <naeem.khademi@gmail.com>
Date:  Thursday, November 14, 2013 8:05 AM
To:  Anoop Ghanwani <anoop@alumni.duke.edu>
Cc:  <curtis@ipv6.occnc.com>, Michael Welzl <michawe@ifi.uio.no>,
"aqm@ietf.org" <aqm@ietf.org>, Preethi Natarajan <preethi.cis@gmail.com>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> 
> 
> Below is my personal opinion, but hopefully Fred can clarify this better based
> on the AQM recommendations draft:
> 
> "applicability of AQM to all types of networks/switches" => "yes"
> 
> "applicability of *any* AQM to all types of networks/switches" => "no"
>  
>> Perhaps AQM cannot help this, but hopefully it won't hurt.  Trying to do
>> fancy things with small buffers is challenging.
> 
> AQM will most likely to help from data centers to the access links, and so on.
> But we may possibly need different AQMs for different network scenarios; The
> fact that an AQMs should be auto-tunable doesn't imply that it can be applied
> everywhere and we may need different auto-tunable AQMs specifically designed
> for different networks (ideally better if we could use fewer of them and they
> could work everywhere). I hope I'm not wrong.
>  

Again, please hold on. Even if its personal opinion, I would state facts
backed with evidence instead of "hopes", since the community's thought
process is at stake here.

>From the preliminary results we've seen, PIE has been able to address data
center issues quite well. There is no evidence thus far why PIE's control
law with auto-tuned parameters cannot adapt to data center or other network
environments. 

Given that, its too early to "hope" that different network scenarios need
different AQM schemes. Of course, different network environments may choose
to deploy different AQM schemes for various other reasons, which is not the
point of discussion here.

Preethi



--B_3467275215_7580249
Content-type: text/html;
	charset="US-ASCII"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; font-size: 14px; font-family: C=
alibri, sans-serif; color: rgb(0, 0, 0); "><div style=3D"color: rgb(0, 0, 0); =
"><br></div><div style=3D"color: rgb(0, 0, 0); "><br></div><span id=3D"OLK_SRC_B=
ODY_SECTION" style=3D"color: rgb(0, 0, 0); "><div style=3D"font-family:Calibri; =
font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BO=
RDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGH=
T: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TO=
P: 3pt"><span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a hr=
ef=3D"mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br><span=
 style=3D"font-weight:bold">Date: </span> Thursday, November 14, 2013 8:05 AM<=
br><span style=3D"font-weight:bold">To: </span> Anoop Ghanwani &lt;<a href=3D"ma=
ilto:anoop@alumni.duke.edu">anoop@alumni.duke.edu</a>&gt;<br><span style=3D"fo=
nt-weight:bold">Cc: </span> &lt;<a href=3D"mailto:curtis@ipv6.occnc.com">curti=
s@ipv6.occnc.com</a>&gt;, Michael Welzl &lt;<a href=3D"mailto:michawe@ifi.uio.=
no">michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org<=
/a>" &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;, Preethi Nataraj=
an &lt;<a href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<=
br><span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Qu=
eue length vs. delay based<br></div><div><br></div><blockquote id=3D"MAC_OUTLO=
OK_ATTRIBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 =
0 5; MARGIN:0 0 0 5;"><div dir=3D"ltr"><br><div class=3D"gmail_extra"><div class=
=3D"gmail_quote"><div><br></div><div style=3D"">Below is my personal opinion, bu=
t hopefully Fred can clarify this better based on the AQM recommendations dr=
aft:</div><div style=3D""><br></div><div style=3D"">"applicability of AQM to all=
 types of networks/switches" =3D&gt; "yes"</div><div style=3D""><br></div><div s=
tyle=3D"">"applicability of *any* AQM to all types of networks/switches" =3D&gt;=
 "no" &nbsp;</div><div>&nbsp;</div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">=
<div class=3D"gmail_extra"><div class=3D"gmail_quote"><div>Perhaps AQM cannot he=
lp this, but hopefully it won't hurt. &nbsp;Trying to do fancy things with s=
mall buffers is challenging.</div></div></div></div></blockquote><div><br></=
div><div style=3D"">AQM will most likely to help from data centers to the acce=
ss links, and so on. But we may possibly need different AQMs for different n=
etwork scenarios; The fact that an AQMs should be auto-tunable doesn't imply=
 that it can be applied everywhere and we may need different auto-tunable AQ=
Ms specifically designed for different networks (ideally better if we could =
use fewer of them and they could work everywhere). I hope I'm not wrong. &nb=
sp;&nbsp;</div><div>&nbsp;</div></div></div></div></blockquote></span><div s=
tyle=3D"color: rgb(0, 0, 0); "><br></div><span id=3D"OLK_SRC_BODY_SECTION"><div =
dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div dir=3D"ltr"><=
div class=3D"gmail_extra"><div class=3D"gmail_quote"><span class=3D"HOEnZb"><font>=
<div>Again, please hold on. Even if its personal opinion, I would state fact=
s backed with evidence instead of "hopes", since the community's thought pro=
cess is at stake here.</div></font></span></div></div></div></div></div></di=
v></span><span id=3D"OLK_SRC_BODY_SECTION"><div dir=3D"ltr"><div class=3D"gmail_ex=
tra"><div class=3D"gmail_quote"><div dir=3D"ltr"><div class=3D"gmail_extra"><div c=
lass=3D"gmail_quote"><font></font><font></font></div></div></div></div></div><=
/div></span><div><font></font></div><div><br></div><span id=3D"OLK_SRC_BODY_SE=
CTION"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
 dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div>From the p=
reliminary results we've seen, PIE has been able to address data center issu=
es quite well. There is no evidence thus far why PIE's control law with auto=
-tuned parameters cannot adapt to data center or other network environments.=
&nbsp;</div><div><br></div><div>Given that, its too early to "hope" that dif=
ferent network scenarios <span style=3D"font-style: italic; ">need</span> diff=
erent AQM schemes. Of course, different network environments may choose to d=
eploy different AQM schemes for various other reasons, which is not the poin=
t of discussion here.</div><div><br></div><div>Preethi</div></div></div></di=
v></div></div></div></span></body></html>

--B_3467275215_7580249--



From naeem.khademi@gmail.com  Thu Nov 14 12:20:01 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 39FE011E80FA for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:20:01 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.311
X-Spam-Level: 
X-Spam-Status: No, score=-2.311 tagged_above=-999 required=5 tests=[AWL=0.061,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j1YhzoCnfO36 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:20:00 -0800 (PST)
Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) by ietfa.amsl.com (Postfix) with ESMTP id 3B20521E80F7 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:20:00 -0800 (PST)
Received: by mail-vc0-f181.google.com with SMTP id lf12so1078044vcb.26 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:19:59 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=naSkwWpI6V2WKPVieIBFmf1ibdPbQhRc+0dRT4U7EE0=; b=Q3sk7xEtZnwDXcGyE+7ragjgWskWuGBlXXUn2J7vg61eCU9hdGyjJ6hSAPFJXQSgWe KKsb7WTNKDAmqH7tByeGthPKmA3LWt5y+dMnXj7WEU6HxaBhdgxFofevC2swCDDRRgaN BwKpZPv1K3wCpLe00PsF0Wy/43EvfagSHUTWy9AL5hpAChBL94qzYWzliidrj/YI5d8d qMSytMdzAL33pyz7rXXTGwCbB19aUJmcYRtu1rznGMh07Hj/yw9Xxr52ANKqziDWRmcX V8Vig5++Bk2V978i02vQkzbdyP1vlYxd8p0zmL95MychH30HDh/jDIdU0viX6ZuGFS5X W9eg==
MIME-Version: 1.0
X-Received: by 10.221.3.200 with SMTP id nz8mr678440vcb.67.1384460399686; Thu, 14 Nov 2013 12:19:59 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 12:19:59 -0800 (PST)
In-Reply-To: <CEAA66F5.4B52B%prenatar@cisco.com>
References: <CAEjQQ5VhMfgik_QS2GxWRbMDJQf7VXEC3-e_LP0tx5P16pC9PQ@mail.gmail.com> <CEAA66F5.4B52B%prenatar@cisco.com>
Date: Thu, 14 Nov 2013 21:19:59 +0100
Message-ID: <CAEjQQ5UbZPEDMXHiuPKV5UDVK5bsMvO3Zfg88Fwg_DmzbgR7Hw@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=089e01176885545b9804eb28cea8
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 20:20:01 -0000

--089e01176885545b9804eb28cea8
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Nov 14, 2013 at 8:46 PM, Preethi Natarajan <preethi.cis@gmail.com>wrote:

>
>
> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Thursday, November 14, 2013 5:32 AM
>
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
> AFAIK all three studied AQMs (CoDel, PIE, ARED) have a parameter named
> "target delay" or "target queuing". Judging based on the (param) name, they
> supposedly try to maintain the queue delay/length around/at a certain value
> although "target delay/queuing" has different semantics in each AQM and the
> response to crossing that threshold/value is also different from one AQM to
> another. Moreover, It is favorable for an AQM that tries to fix bufferbloat
> problem to have a predictable delay distribution, most favorably around
> (e.g. mean, median, max) a certain value. PIE uses it, CoDel has it and
> ARED in fixed BW setups has it too. So, I'm not quite sure, why it
> shouldn't be possible to allow a certain size of burst in such
> threshold-based mechanisms (AFAIK (A)RED's Linux implementation already has
> a "burst" param that can be tuned, although its semantics may not be
> identical to the way CoDel or PIE handle bursts).
>
>
> Of course. The question is how effective is the mechanism. As you note,
> whatever "burst" mechanism exists in Linux's ARED wasn't effective as
> evident from the low throughput in the plots.
>
>
>
>
>> Clearly delay-based ARED needs significant and careful redesign. I
>> suppose one can argue this task is as laborious as designing a new AQM
>> scheme as opposed to previous notions that delay-based ARED would work
>> right out of the box.
>>
>
> it's probably not accurate to say "redesign" as delay-based ARED doesn't
> exist as of know. AFAIK given our latest results with ARED, we raised
> the possibility of modifying ARED to use delay-based thresholds instead of
> size-based thresholds during the ICCRG talk and in the TR, and declared it
> as future work. I'm not aware of any other implementation/proposal of
> delay-based ARED to exist as of now. We never claimed or speculated on how
> this future delay-based AQM might perform and left it to when the future
> work is done and the outcome is properly investigated and therefore we
> never said that "it would work right out of the box" :-).
>
>
>
> Great. Thanks for confirming this point that ARED as is does not work well
> for the scenarios considered. What is required is new AQM design, not just
> plugging delay into ARED.
>

Agreed with all your points though to complete it, developing new AQMs
without looking back at and learning from what we have had available leads
to inventing things that improve utilization instead of delay for instance
:-). Again to emphasis we never recommended that people should turn on RED
or ARED (or any future variant of ARED) on their edge devices, but rather
we experimentally demonstrated that it worked better than CoDel and PIE is
most of the scenarios we studied except in very few. We thought it was
important to have more basic understandable experimental results (with
long-lived TCP flows) available before we indulge in looking at more
complex traffic scenarios.

What I would probably choose to investigate e.g. why PIE performs with a
very long upper distribution tail of RTTs with common long-lived TCP
traffic more specifically for lower target delay values even though we know
TCP isn't a very bursty type of traffic? Could that be related to the way
it handles bursts?  or any other reason?  The technical report provides a
very first insight into such issues that need to be explored like this one
but then the story doesn't stop here and ARED was only a benchmark that was
close enough to IETF-documented RED and could work with a certain target
delay (in our scenarios).


> Preethi
>

Naeem

--089e01176885545b9804eb28cea8
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Thu, Nov 14, 2013 at 8:46 PM, Preethi Natarajan <span dir=3D"ltr=
">&lt;<a href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.ci=
s@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-word"><div sty=
le=3D"font-size:14px;font-family:Calibri,sans-serif"><br></div><div style=
=3D"font-size:14px;font-family:Calibri,sans-serif">
<br></div><span style=3D"font-size:14px;font-family:Calibri,sans-serif"><di=
v style=3D"border-right:medium none;padding-right:0in;padding-left:0in;padd=
ing-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-f=
amily:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:m=
edium none">
<span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D=
"mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com<=
/a>&gt;<br><span style=3D"font-weight:bold">Date: </span> Thursday, Novembe=
r 14, 2013 5:32 AM<div class=3D"im">
<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a h=
ref=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.co=
m</a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Michael Welzl &lt=
;<a href=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no=
</a>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.=
org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@iet=
f.org</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Qu=
eue length vs. delay based<br></div></div><div><br></div><blockquote style=
=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div dir=3D=
"ltr">
<div><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div><br></div><=
div class=3D"im"><div>AFAIK all three studied AQMs (CoDel, PIE, ARED) have =
a parameter named &quot;target delay&quot; or &quot;target queuing&quot;. J=
udging based on the (param) name, they supposedly try to maintain the queue=
 delay/length around/at a certain value although &quot;target delay/queuing=
&quot; has different semantics in each AQM and the response to crossing tha=
t threshold/value is also different from one AQM to another. Moreover, It i=
s favorable for an AQM that tries to fix bufferbloat problem to have a=A0pr=
edictable delay distribution, most favorably around (e.g. mean, median, max=
) a certain value. PIE uses it, CoDel has it and ARED in fixed BW setups ha=
s it too. So, I&#39;m not quite sure, why it shouldn&#39;t be possible to a=
llow a certain size of burst in such threshold-based mechanisms (AFAIK (A)R=
ED&#39;s Linux implementation already has a &quot;burst&quot; param that ca=
n be tuned, although its semantics may not be identical to the way CoDel or=
 PIE handle bursts).=A0 =A0 </div>
</div></div></div></div></div></blockquote></span><div style=3D"font-size:1=
4px;font-family:Calibri,sans-serif"><br></div><div style=3D"font-size:14px;=
font-family:Calibri,sans-serif">Of course. The question is how effective is=
 the mechanism. As you note, whatever &quot;burst&quot; mechanism exists in=
 Linux&#39;s ARED wasn&#39;t effective as evident from the low throughput i=
n the plots.=A0</div>
<div class=3D"im"><div style=3D"font-size:14px;font-family:Calibri,sans-ser=
if">=A0=A0</div><span style=3D"font-size:14px"><blockquote style=3D"border-=
left-color:rgb(181,196,223);border-left-width:5px;border-left-style:solid;p=
adding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:5px;margin=
-top:0px;margin-right:0px;margin-bottom:0px;margin-left:5px">
<div dir=3D"ltr"><div><div class=3D"gmail_extra"><div class=3D"gmail_quote"=
><div style=3D"font-family:Calibri,sans-serif">=A0=A0</div><blockquote clas=
s=3D"gmail_quote" style=3D"font-family:Calibri,sans-serif;margin-top:0px;ma=
rgin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;bo=
rder-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><div><div style=3D"padding-right:55px;p=
adding-left:0px;padding-top:12px;margin-bottom:0px;padding-bottom:0px;margi=
n-right:0px;overflow-x:auto;overflow-y:auto;font-size:medium;margin-left:0p=
x;font-family:Helvetica;word-wrap:break-word;margin-top:0px">
<span style=3D"word-wrap:break-word;font:normal normal normal 14px/normal C=
alibri">Clearly delay-based ARED needs significant and careful redesign.=A0=
</span><span style=3D"font-size:14px;font-family:Calibri">I suppose one can=
 argue this task is as laborious as designing a new AQM scheme as opposed t=
o previous notions that delay-based ARED would work right out of the box.=
=A0</span></div>
</div></div></blockquote><div style=3D"font-family:Calibri,sans-serif"><br>=
</div><div style=3D"font-family:Calibri,sans-serif">it&#39;s probably not a=
ccurate to say &quot;redesign&quot; as delay-based ARED doesn&#39;t exist a=
s of know. AFAIK given our latest results with ARED, we raised the=A0possib=
ility=A0of modifying ARED to use delay-based thresholds instead of size-bas=
ed thresholds during the ICCRG talk and in the TR, and declared it as futur=
e work. I&#39;m not aware of any other implementation/proposal of delay-bas=
ed ARED to exist as of now. We never claimed or speculated on how this futu=
re delay-based AQM might perform and left it to when the future work is don=
e and the outcome is properly investigated and therefore we never said that=
 &quot;it would work right out of the box&quot; :-). =A0 =A0</div>
<div style=3D"font-family:Calibri,sans-serif">=A0</div></div></div></div></=
div></blockquote></span><div style=3D"font-size:14px"><br></div></div><div =
style=3D"font-size:14px">Great. Thanks for confirming this point that=A0ARE=
D as is does not work well for the scenarios considered. What is required i=
s new AQM design, not just plugging delay into ARED.</div>
</div></blockquote><div><br></div><div style>Agreed with all your points th=
ough to complete it, developing new AQMs without looking back at and learni=
ng from what we have had available leads to inventing things that improve u=
tilization instead of delay for instance :-). Again to emphasis we never re=
commended that people should turn on RED or ARED (or any future variant of =
ARED) on their edge devices, but rather we experimentally demonstrated that=
 it worked better than CoDel and PIE is most of the scenarios we studied ex=
cept in very few. We thought it was important to have more basic understand=
able experimental results (with long-lived TCP flows) available before we i=
ndulge in looking at more complex traffic scenarios. =A0 =A0=A0</div>
<div style><br></div><div style>What I would probably choose to investigate=
 e.g. why PIE performs with a very long upper distribution tail of RTTs wit=
h common long-lived TCP traffic more specifically for lower target delay va=
lues even though we know TCP isn&#39;t a very bursty type of traffic? Could=
 that be related to the way it handles bursts? =A0or any other reason? =A0T=
he technical report provides a very first insight into such issues that nee=
d to be explored like this one but then the story doesn&#39;t stop here and=
 ARED was only a benchmark that was close enough to IETF-documented RED and=
 could work with a certain target delay (in our scenarios). =A0 =A0=A0</div=
>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-=
word"><span class=3D"HOEnZb"><font color=3D"#888888"><span><div dir=3D"ltr"=
><div><div class=3D"gmail_extra">
<div class=3D"gmail_quote"><div style=3D"word-wrap:break-word"><span><font>=
<div style=3D"padding-right:55px;padding-left:0px;padding-top:12px;margin-b=
ottom:0px;padding-bottom:0px;margin-right:0px;overflow-x:auto;overflow-y:au=
to;margin-left:0px;word-wrap:break-word;margin-top:0px">
Preethi</div></font></span></div></div></div></div></div></span></font></sp=
an></div>
</blockquote></div><br></div><div class=3D"gmail_extra" style>Naeem</div></=
div>

--089e01176885545b9804eb28cea8--

From naeem.khademi@gmail.com  Thu Nov 14 12:25:42 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CADEF11E810D for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:25:42 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.317
X-Spam-Level: 
X-Spam-Status: No, score=-2.317 tagged_above=-999 required=5 tests=[AWL=0.055,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id netoCuAbxCfw for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:25:42 -0800 (PST)
Received: from mail-vb0-x22f.google.com (mail-vb0-x22f.google.com [IPv6:2607:f8b0:400c:c02::22f]) by ietfa.amsl.com (Postfix) with ESMTP id A865511E80FA for <aqm@ietf.org>; Thu, 14 Nov 2013 12:25:35 -0800 (PST)
Received: by mail-vb0-f47.google.com with SMTP id g10so2115590vbg.34 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:25:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WHFH51DJdeWY+oNnrvp8jVfYoKI2RN129fqiHrLGVBw=; b=jGvog39jbVpfPUQ9gl9dbJsjglKl97P477lnebXIbypsuzVtkt6HL/RiSpKjwIYc6X +WDRasKvavC1RVPpzBFwP/VxXUZzm8aTxggjX1Eesm8HDWWyPFYsLEQKj2qG6XIN89zN 5HjnYk3K2woyviCiX9xD6lsU/1Wtav0yHPPHRIgi2eBiLI2f1M00QfsfabI08Sdf390i jM03rZ1Lwgm+EHwqTliwFk/Of1f+1SNffY5FONjP37abv3qqioeGtaIt4jnWt4cdz2r+ IcDgmEiAAGkRo7vojI1CGlPc5snuIJwuInYQ+rraiba35XEJvbOyQTBd78YVX1G4o/vr RdGQ==
MIME-Version: 1.0
X-Received: by 10.58.133.77 with SMTP id pa13mr1855105veb.21.1384460735124; Thu, 14 Nov 2013 12:25:35 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 12:25:35 -0800 (PST)
In-Reply-To: <CEAA66F5.4B52B%prenatar@cisco.com>
References: <CAEjQQ5VhMfgik_QS2GxWRbMDJQf7VXEC3-e_LP0tx5P16pC9PQ@mail.gmail.com> <CEAA66F5.4B52B%prenatar@cisco.com>
Date: Thu, 14 Nov 2013 21:25:35 +0100
Message-ID: <CAEjQQ5V9FrouYRWfjKD-w=ztdEf=zS6ML2VXvK_ucUwR_-wr0w@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=047d7b67339852ce4e04eb28e2f5
Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 20:25:43 -0000

--047d7b67339852ce4e04eb28e2f5
Content-Type: text/plain; charset=ISO-8859-1

Very sorry for double-posting as for some reasons "what I was answering to"
was missed from the rest of my previous email. So here again:


On Thu, Nov 14, 2013 at 8:46 PM, Preethi Natarajan <preethi.cis@gmail.com>wrote:

>
>
> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Thursday, November 14, 2013 5:32 AM
>
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
> AFAIK all three studied AQMs (CoDel, PIE, ARED) have a parameter named
> "target delay" or "target queuing". Judging based on the (param) name, they
> supposedly try to maintain the queue delay/length around/at a certain value
> although "target delay/queuing" has different semantics in each AQM and the
> response to crossing that threshold/value is also different from one AQM to
> another. Moreover, It is favorable for an AQM that tries to fix bufferbloat
> problem to have a predictable delay distribution, most favorably around
> (e.g. mean, median, max) a certain value. PIE uses it, CoDel has it and
> ARED in fixed BW setups has it too. So, I'm not quite sure, why it
> shouldn't be possible to allow a certain size of burst in such
> threshold-based mechanisms (AFAIK (A)RED's Linux implementation already has
> a "burst" param that can be tuned, although its semantics may not be
> identical to the way CoDel or PIE handle bursts).
>
>
> Of course. The question is how effective is the mechanism. As you note,
> whatever "burst" mechanism exists in Linux's ARED wasn't effective as
> evident from the low throughput in the plots.
>
>
>
>
>> Clearly delay-based ARED needs significant and careful redesign. I
>> suppose one can argue this task is as laborious as designing a new AQM
>> scheme as opposed to previous notions that delay-based ARED would work
>> right out of the box.
>>
>
> it's probably not accurate to say "redesign" as delay-based ARED doesn't
> exist as of know. AFAIK given our latest results with ARED, we raised
> the possibility of modifying ARED to use delay-based thresholds instead of
> size-based thresholds during the ICCRG talk and in the TR, and declared it
> as future work. I'm not aware of any other implementation/proposal of
> delay-based ARED to exist as of now. We never claimed or speculated on how
> this future delay-based AQM might perform and left it to when the future
> work is done and the outcome is properly investigated and therefore we
> never said that "it would work right out of the box" :-).
>
>
>
> Great. Thanks for confirming this point that ARED as is does not work well
> for the scenarios considered. What is required is new AQM design, not just
> plugging delay into ARED.
> Preethi
>

Agreed with all your points though to complete it, developing new AQMs
without looking back at and learning from what we have had available leads
to inventing things that improve utilization instead of delay for instance
:-). Again to emphasis we never recommended that people should turn on RED
or ARED (or any future variant of ARED) on their edge devices, but rather
we experimentally demonstrated that it worked better than CoDel and PIE is
most of the scenarios we studied except in very few. We thought it was
important to have more basic understandable experimental results (with
long-lived TCP flows) available before we indulge in looking at more
complex traffic scenarios.

What I would probably choose to investigate e.g. why PIE performs with a
very long upper distribution tail of RTTs with common long-lived TCP
traffic more specifically for lower target delay values even though we know
TCP isn't a very bursty type of traffic? Could that be related to the way
it handles bursts?  or any other reason?  The technical report provides a
very first insight into such issues that need to be explored like this one
but then the story doesn't stop here and ARED was only a benchmark that was
close enough to IETF-documented RED and could work with a certain target
delay (in our scenarios).

--047d7b67339852ce4e04eb28e2f5
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Very sorry for double-posting as for some reasons &quot;wh=
at I was answering to&quot; was missed from the rest of my previous email. =
So here again: =A0<br><div class=3D"gmail_extra"><br><br><div class=3D"gmai=
l_quote">
On Thu, Nov 14, 2013 at 8:46 PM, Preethi Natarajan <span dir=3D"ltr">&lt;<a=
 href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.=
com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,2=
04);border-left-style:solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><div style=3D"font-size:14px;font-famil=
y:Calibri,sans-serif"><br></div><div style=3D"font-size:14px;font-family:Ca=
libri,sans-serif"><br></div><span style=3D"font-size:14px;font-family:Calib=
ri,sans-serif"><div style=3D"border-width:1pt medium medium;border-style:so=
lid none none;padding:3pt 0in 0in;text-align:left;font-size:11pt;font-famil=
y:Calibri;border-top-color:rgb(181,196,223)">
<span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D=
"mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com<=
/a>&gt;<br><span style=3D"font-weight:bold">Date: </span> Thursday, Novembe=
r 14, 2013 5:32 AM<div class=3D"im">
<br><span style=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a h=
ref=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.co=
m</a>&gt;<br><span style=3D"font-weight:bold">Cc: </span> Michael Welzl &lt=
;<a href=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no=
</a>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.=
org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@iet=
f.org</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Qu=
eue length vs. delay based<br></div></div><div><br></div><blockquote style=
=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div dir=3D=
"ltr">
<div><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div><br></div><=
div class=3D"im"><div>AFAIK all three studied AQMs (CoDel, PIE, ARED) have =
a parameter named &quot;target delay&quot; or &quot;target queuing&quot;. J=
udging based on the (param) name, they supposedly try to maintain the queue=
 delay/length around/at a certain value although &quot;target delay/queuing=
&quot; has different semantics in each AQM and the response to crossing tha=
t threshold/value is also different from one AQM to another. Moreover, It i=
s favorable for an AQM that tries to fix bufferbloat problem to have a=A0pr=
edictable delay distribution, most favorably around (e.g. mean, median, max=
) a certain value. PIE uses it, CoDel has it and ARED in fixed BW setups ha=
s it too. So, I&#39;m not quite sure, why it shouldn&#39;t be possible to a=
llow a certain size of burst in such threshold-based mechanisms (AFAIK (A)R=
ED&#39;s Linux implementation already has a &quot;burst&quot; param that ca=
n be tuned, although its semantics may not be identical to the way CoDel or=
 PIE handle bursts).=A0 =A0 </div>
</div></div></div></div></div></blockquote></span><div style=3D"font-size:1=
4px;font-family:Calibri,sans-serif"><br></div><div style=3D"font-size:14px;=
font-family:Calibri,sans-serif">Of course. The question is how effective is=
 the mechanism. As you note, whatever &quot;burst&quot; mechanism exists in=
 Linux&#39;s ARED wasn&#39;t effective as evident from the low throughput i=
n the plots.=A0</div>
<div class=3D"im"><div style=3D"font-size:14px;font-family:Calibri,sans-ser=
if">=A0=A0</div><span style=3D"font-size:14px"><blockquote style=3D"border-=
left-color:rgb(181,196,223);border-left-width:5px;border-left-style:solid;p=
adding:0px 0px 0px 5px;margin:0px 0px 0px 5px">
<div dir=3D"ltr"><div><div class=3D"gmail_extra"><div class=3D"gmail_quote"=
><div style=3D"font-family:Calibri,sans-serif">=A0=A0</div><blockquote clas=
s=3D"gmail_quote" style=3D"font-family:Calibri,sans-serif;margin:0px 0px 0p=
x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef=
t-style:solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><div><div style=3D"padding:12px 55px 0p=
x 0px;margin:0px;overflow:auto;font-size:medium;font-family:Helvetica;word-=
wrap:break-word"><span style=3D"word-wrap:break-word;font-style:normal;font=
-variant:normal;font-weight:normal;font-size:14px;line-height:normal;font-f=
amily:Calibri">Clearly delay-based ARED needs significant and careful redes=
ign.=A0</span><span style=3D"font-size:14px;font-family:Calibri">I suppose =
one can argue this task is as laborious as designing a new AQM scheme as op=
posed to previous notions that delay-based ARED would work right out of the=
 box.=A0</span></div>
</div></div></blockquote><div style=3D"font-family:Calibri,sans-serif"><br>=
</div><div style=3D"font-family:Calibri,sans-serif">it&#39;s probably not a=
ccurate to say &quot;redesign&quot; as delay-based ARED doesn&#39;t exist a=
s of know. AFAIK given our latest results with ARED, we raised the=A0possib=
ility=A0of modifying ARED to use delay-based thresholds instead of size-bas=
ed thresholds during the ICCRG talk and in the TR, and declared it as futur=
e work. I&#39;m not aware of any other implementation/proposal of delay-bas=
ed ARED to exist as of now. We never claimed or speculated on how this futu=
re delay-based AQM might perform and left it to when the future work is don=
e and the outcome is properly investigated and therefore we never said that=
 &quot;it would work right out of the box&quot; :-). =A0 =A0</div>
<div style=3D"font-family:Calibri,sans-serif">=A0</div></div></div></div></=
div></blockquote></span><div style=3D"font-size:14px"><br></div></div><div =
style=3D"font-size:14px">Great. Thanks for confirming this point that=A0ARE=
D as is does not work well for the scenarios considered. What is required i=
s new AQM design, not just plugging delay into ARED.</div>
<span class=3D""><font color=3D"#888888"><span><div dir=3D"ltr"><div><div c=
lass=3D"gmail_extra"><div class=3D"gmail_quote"><div style=3D"word-wrap:bre=
ak-word"><span><font><div style=3D"padding:12px 55px 0px 0px;margin:0px;ove=
rflow:auto;word-wrap:break-word">
Preethi</div></font></span></div></div></div></div></div></span></font></sp=
an></div></blockquote><div><br></div><div style=3D"font-family:arial,sans-s=
erif;font-size:13px">Agreed with all your points though to complete it, dev=
eloping new AQMs without looking back at and learning from what we have had=
 available leads to inventing things that improve utilization instead of de=
lay for instance :-). Again to emphasis we never recommended that people sh=
ould turn on RED or ARED (or any future variant of ARED) on their edge devi=
ces, but rather we experimentally demonstrated that it worked better than C=
oDel and PIE is most of the scenarios we studied except in very few. We tho=
ught it was important to have more basic understandable experimental result=
s (with long-lived TCP flows) available before we indulge in looking at mor=
e complex traffic scenarios. =A0 =A0=A0</div>
<div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div><=
span style=3D"font-family:arial,sans-serif;font-size:13px">What I would pro=
bably choose to investigate e.g. why PIE performs with a very long upper di=
stribution tail of RTTs with common long-lived TCP traffic more specificall=
y for lower target delay values even though we know TCP isn&#39;t a very bu=
rsty type of traffic? Could that be related to the way it handles bursts? =
=A0or any other reason? =A0The technical report provides a very first insig=
ht into such issues that need to be explored like this one but then the sto=
ry doesn&#39;t stop here and ARED was only a benchmark that was close enoug=
h to IETF-documented RED and could work with a certain target delay (in our=
 scenarios). =A0 =A0</span>=A0</div>
</div><br></div></div>

--047d7b67339852ce4e04eb28e2f5--

From ilpo.jarvinen@helsinki.fi  Thu Nov 14 12:33:17 2013
Return-Path: <ilpo.jarvinen@helsinki.fi>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C5C7211E80FA for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:33:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.148
X-Spam-Level: 
X-Spam-Status: No, score=-6.148 tagged_above=-999 required=5 tests=[AWL=-0.076, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uskLXaUODUFd for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:33:09 -0800 (PST)
Received: from courier.cs.helsinki.fi (courier.cs.helsinki.fi [128.214.9.1]) by ietfa.amsl.com (Postfix) with ESMTP id 76E9921E8120 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:33:07 -0800 (PST)
Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by mail.cs.helsinki.fi with esmtp; Thu, 14 Nov 2013 22:32:51 +0200 id 000680B6.52853373.000003F5
Date: Thu, 14 Nov 2013 22:32:51 +0200 (EET)
From: "=?ISO-8859-15?Q?Ilpo_J=E4rvinen?=" <ilpo.jarvinen@helsinki.fi>
X-X-Sender: ijjarvin@melkinpaasi.cs.helsinki.fi
To: Preethi Natarajan <preethi.cis@gmail.com>
In-Reply-To: <CEAA66F5.4B52B%prenatar@cisco.com>
Message-ID: <alpine.DEB.2.02.1311142151560.14052@melkinpaasi.cs.helsinki.fi>
References: <CEAA66F5.4B52B%prenatar@cisco.com>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; boundary="-696250871-1250435348-1384459485=:14052"
Content-ID: <alpine.DEB.2.02.1311142229080.14052@melkinpaasi.cs.helsinki.fi>
X-CS-Test-DKIM: none
Cc: Naeem Khademi <naeem.khademi@gmail.com>, Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 20:33:17 -0000

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

---696250871-1250435348-1384459485=:14052
Content-Type: TEXT/PLAIN; charset=ISO-8859-15
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.02.1311142205541.14052@melkinpaasi.cs.helsinki.fi>

On Thu, 14 Nov 2013, Preethi Natarajan wrote:

> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Thursday, November 14, 2013 5:32 AM
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
> 
> 
> AFAIK all three studied AQMs (CoDel, PIE, ARED) have a parameter named
> "target delay" or "target queuing". Judging based on the (param) name,
> they supposedly try to maintain the queue delay/length around/at a
> certain value although "target delay/queuing" has different semantics
> in each AQM and the response to crossing that threshold/value is also
> different from one AQM to another. Moreover, It is favorable for an
> AQM that tries to fix bufferbloat problem to have a predictable delay
> distribution, most favorably around (e.g. mean, median, max) a certain
> value. PIE uses it, CoDel has it and ARED in fixed BW setups has it
> too. So, I'm not quite sure, why it shouldn't be possible to allow a
> certain size of burst in such threshold-based mechanisms (AFAIK
> (A)RED's Linux implementation already has a "burst" param that can be
> tuned, although its semantics may not be identical to the way CoDel or
> PIE handle bursts). Of course. The question is how effective is the 
> mechanism. As you note, whatever "burst" mechanism exists in Linux's 
> ARED wasn't effective as evident from the low throughput in the plots. 

IIRC, the Linux burst parameter is taken into account in frontend tc-tool 
only to calculate parameters for the kernel. I think its meaning was quite 
close to what L is in formula 3 [1] but it's like ten years ago when I 
read that bit of code. As such, its value only applies to the case where 
the queue average is zero before the burst occurs.

However, the real problem with these "burst" parameters is that they 
do not specify a limit for the burst. Instead, they are only used to set a 
minimum burst they allow but by no means give a way to specify a hard 
deadline when the burst has to be stopped at latest. By no means 
drops/marks typically occur soon after the "burst" threshold is crossed 
because the drop probability will be extremely small.

I've shown this lack of control with a typical RED setup [2] (ARED is 
actually worse than RED because of auto-tuning makes it even slower than 
typical RED setup but I didn't go to prove that). It also talks about 
th_max cutter and tail-drop fallback, which both seemed to be already 
mentioned in this thread. Based on how timers work in CoDel and PIE it is 
pretty obvious that the lack of control problem is present in them too 
(I've a paper coming up very soon on these issues too).


-- 
 i.


[1] Random Early Detection gateways for Congestion Avoidance.
Floyd, S., and Jacobson, V.
IEEE/ACM Transactions on Networking, V.1 N.4, August 1993, p. 397-413.

[2] Ilpo Järvinen, Aaron Yi Ding, Aki Nyrhinen, Markku Kojo, "Harsh RED: 
Improving RED for Limited Aggregate Traffic", In Proc. IEEE International 
Conference on Advanced Information Networking and Applications (IEEE AINA 
2012), Fukuoka, Japan, March 2012.
[http://www.cs.helsinki.fi/group/wibra/aina12.pdf]
---696250871-1250435348-1384459485=:14052--

From naeem.khademi@gmail.com  Thu Nov 14 12:58:46 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 07C0821E811E for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:58:46 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.322
X-Spam-Level: 
X-Spam-Status: No, score=-2.322 tagged_above=-999 required=5 tests=[AWL=0.050,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QJvs7GafZsXy for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 12:58:45 -0800 (PST)
Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) by ietfa.amsl.com (Postfix) with ESMTP id D297321E8117 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:58:44 -0800 (PST)
Received: by mail-vc0-f176.google.com with SMTP id ia6so450469vcb.21 for <aqm@ietf.org>; Thu, 14 Nov 2013 12:58:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uv+Eb1riWlPFpnMSbM39m9LzL/OCMBsfNxlWG9iaSUM=; b=g4jJAdvY6ukss+QYJTxfz21+B/XrX759IvB6tjGH1pSFEF6d24WPBSezZM9xIfyKt/ 3uaq9zOUoYtP3WLLeJbVyfsMtuje1Yo6irvWzmw6MxPAmZGgaN8+tw2TjydGYMeqbELy SOQvVvAZ+S2BHZvic8TywN3lNJ1N3o4gd2mjr/Y7tMm/iXYvn5DFvbH5WsU4uysTp8eC v+Y8jL8ktfbjp2MUTwNSxnz0STFA6EMrpw0LadvYTrTpC8w433wjfTyb9CU3tH5d6JAz MAhQLkTpKE7M4jBhH2CD+MpPOsAEMkz9qrsV37DycskElRRnwrHsoVUIJdqTG0H2aHun f8qw==
MIME-Version: 1.0
X-Received: by 10.58.144.168 with SMTP id sn8mr1851366veb.33.1384462724213; Thu, 14 Nov 2013 12:58:44 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 12:58:44 -0800 (PST)
In-Reply-To: <CEAA6932.55A55%ropan@cisco.com>
References: <CAEjQQ5We5HHbePXTu4GfenM+UwcdpundVF30Zg9TCD+H7jXMCg@mail.gmail.com> <CEAA6932.55A55%ropan@cisco.com>
Date: Thu, 14 Nov 2013 21:58:44 +0100
Message-ID: <CAEjQQ5WW_VfXC_PQ=eaZnxg-9n3BrxLk3zKAX3heWedm_shTqw@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: "Rong Pan (ropan)" <ropan@cisco.com>
Content-Type: multipart/alternative; boundary=047d7b5da93be1b68d04eb295809
Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>, Preethi Natarajan <preethi.cis@gmail.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 20:58:46 -0000

--047d7b5da93be1b68d04eb295809
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi Rong

comments follow inline

Thanks
Naeem

On Thu, Nov 14, 2013 at 8:59 PM, Rong Pan (ropan) <ropan@cisco.com> wrote:

>  Please see inline=85
>
>  Thanks,
>
>  Rong
>
>
>
>   From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Thursday, November 14, 2013 8:43 AM
> To: Preethi Natarajan <preethi.cis@gmail.com>
> Cc: "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>
>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
>
>>     Delay-based ARED behaves similar to tail drop at max threshold.
>>>
>>
>  I think I now understand what you mean by this sentence ;-) and
> therefore please ignore the paragraph responding to this specific point i=
n
> the previous email (sorry about that, I got it mixed with max_target).
> Indeed ARED drops every packet when the average queue length grows beyond
> th_max as drop probability jumps to 1 (when it's not in gentle mode). For
> this to happen, the "average queue length" has to grow beyond th_max and
> depending on how the averaging is done, that can correspond to certain
> burst allowance at AQM. Moreover, this is done over 500 ms interval which
> means the average queue length has to stay above th_max for mostly few
> rounds of RTTs (assuming the typical distribution of RTTs on the Internet
> to have a mean/average somewhere below that value e.g. 100~200 ms). This
> allows burst that don't take the average queue size to above th_max over
> 500 ms period to pass and clear themselves up the queue.
>
>   > This is unacceptable of an AQM even for max thresholds around 50ms.
>
>  -----------------------------
>  >>>>>>>>>>>>>RP: Two issues here, first, averaging has a parameter. How
> to tune it to avoid tail drop behavior? What would be its relationship to
> max_th? Do we understand ?  Second, one single tcp's behavior is very
> bursty, if 500ms burst allowance is effective in ARED, how come is it not
> reflected in your throughput chart? The one single tcp throughput is very
> low for low latency numbers, which is a clear indication of not enough
> buffering.
>     ------------------------------------
>


NAEEMK: As stated before, we're not advocating using ARED whatsoever.
Therefore trying to fix/address ARED's inherent issues that have been
stated on this thread repeatedly would not be an option nor it enhances the
argument to support PIE or CoDel for deployment. So, I'm not going to
speculate on how one can possibly fix ARED thresholds and its burst
allowance. On the other hand, based on the presented experimental results,
we would alternatively like to see why CoDel and PIE performed worse that
ARED (designed in 2001) while they were expected to achieve latencies far
better than *RED.


>
>
>     Whether this is a good thing or bad thing is a subjective matter in
> my opinion, as most AQMs at some point will drop every packet whether it =
be
> the maximum queue length or at some thresholds.
>
>  ----------------------------
>  >>>>>>>>>>>>RP: I don=92t think so. Neither PIE nor Codel has drop-all
> threshold as AQM part of it. Tail drop exists, but it is not part of AQM.
> Putting the latency-based  ARED aside, ARED in this current form has the
> issues that we mentioned in the previous emails. It needs significant and
> careful redesign.
>

NAEEMK: The same response as above.


> ---------------------------
>
>  There is also a comment about "Throwing away all the knowledge and
> improvement gained from RED-like AQMs, CHoKe, etc and coming up with
> brand-new AQMs =85".
>
>  ------------------------------------------
> >>>>>>>>>>RP: After I worked on CHOKe (now in Linux Kernel)  in 2000 as
> part of my Ph.D. thesis at Stanford, I have worked on AFD (implemented in
> many Cisco flagship products), QCN (now IEEEE standard for data center
> ethernet, 802.1Qau), all of which are AQM schemes. I also helped writing
> guidelines in tuning WRED at Cisco. New thinkings were brought into PIE
> after I learned precious lessons (both good and bad) from all of them.
> Discarding them as "coming up with brand-new AQMs" is ignorant.
>

NAEEMK: agreed! and we are on same side :-) though it will be nice to see
the connection points between above AQMs and PIE and lessons learned being
somehow documented.


>
>  Regards,
>
>  Rong
>  --------------------------------
>
>
>  Cheers,
>  Naeem
>
>  >>>>>>>>>>>>RP:
>

--047d7b5da93be1b68d04eb295809
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Rong=A0<div><br></div><div>comments follow inline<br><d=
iv class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Thanks</div><=
div class=3D"gmail_extra">Naeem<br><br><div class=3D"gmail_quote">On Thu, N=
ov 14, 2013 at 8:59 PM, Rong Pan (ropan) <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:ropan@cisco.com" target=3D"_blank">ropan@cisco.com</a>&gt;</span> wro=
te:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">



<div style=3D"word-wrap:break-word">
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
Please see inline=85</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
Thanks,</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
Rong</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<span style=3D"font-size:16px;font-family:Calibri,sans-serif">
<div style=3D"border-right:medium none;padding-right:0in;padding-left:0in;p=
adding-top:3pt;text-align:left;font-size:14pt;border-bottom:medium none;fon=
t-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-lef=
t:medium none">

<span style=3D"font-weight:bold">From: </span>Naeem Khademi &lt;<a href=3D"=
mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com</=
a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Thursday, November 14, 2013 8=
:43 AM<br>
<span style=3D"font-weight:bold">To: </span>Preethi Natarajan &lt;<a href=
=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</=
a>&gt;<br>
<span style=3D"font-weight:bold">Cc: </span>&quot;<a href=3D"mailto:aqm@iet=
f.org" target=3D"_blank">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@i=
etf.org" target=3D"_blank">aqm@ietf.org</a>&gt;, Michael Welzl &lt;<a href=
=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no</a>&gt;=
<div class=3D"im">
<br>
<span style=3D"font-weight:bold">Subject: </span>Re: [aqm] AQM schemes: Que=
ue length vs. delay based<br>
</div></div>
<div><br>
</div>
<div>
<div>
<div dir=3D"ltr"><span style=3D"color:rgb(80,0,80)">=A0</span><br>
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div style=3D"word-wrap:break-word">
<div>
<div style=3D"padding:12px 55px 0px 0px;margin:0px;overflow:auto;font-size:=
medium;font-family:Helvetica;word-wrap:break-word">
<span style=3D"word-wrap:break-word;font-style:normal;font-variant:normal;f=
ont-weight:normal;font-size:14px;line-height:normal;font-family:Calibri">De=
lay-based ARED behaves similar to tail drop at max threshold.</span></div>

</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote><div class=3D"im">
<div><br>
</div>
<div>I think I now understand what you mean by this sentence ;-) and theref=
ore please ignore the paragraph responding to this specific point in the pr=
evious email (sorry about that, I got it mixed with max_target). Indeed ARE=
D drops every packet when
 the average queue length grows beyond th_max as drop probability jumps to =
1 (when it&#39;s not in gentle mode). For this to happen, the &quot;average=
 queue length&quot; has to grow beyond th_max and depending on how the aver=
aging is done, that can correspond to certain
 burst allowance at AQM. Moreover, this is done over 500 ms interval which =
means the average queue length has to stay above th_max for mostly few roun=
ds of RTTs (assuming the typical distribution of RTTs on the Internet to ha=
ve a mean/average somewhere below
 that value e.g. 100~200 ms). This allows burst that don&#39;t take the ave=
rage queue size to above th_max over 500 ms period to pass and clear themse=
lves up the queue. =A0=A0=A0 =A0 =A0=A0</div>
<div><br>
</div>
<div><span style=3D"color:rgb(80,0,80);font-family:Calibri;font-size:14px">=
=A0&gt; This is unacceptable of an AQM even for max thresholds around 50ms.=
=A0</span></div>
</div></div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
-----------------------------</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: Two issues here, fi=
rst, averaging has a parameter. How to tune it to avoid tail drop behavior?=
 What would be its relationship to max_th? Do we understand ? =A0Second, on=
e single tcp&#39;s behavior is very bursty, if 500ms burst allowance is eff=
ective
 in ARED, how come is it not reflected in your throughput chart? The one si=
ngle tcp throughput is very low for low latency numbers, which is a clear i=
ndication of not enough buffering.=A0</div>
<span>
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<span style=3D"color:rgb(80,0,80);font-size:14px;font-family:Calibri">-----=
-------------------------------</span></div></div></div></div></div></div><=
/span></div></blockquote><div><br></div><div><br></div><div style>NAEEMK: A=
s stated before, we&#39;re not advocating using ARED whatsoever. Therefore =
trying to fix/address ARED&#39;s inherent issues that have been stated on t=
his thread repeatedly would not be an option nor it enhances the argument t=
o support PIE or CoDel for deployment. So, I&#39;m not going to speculate o=
n how one can possibly fix ARED thresholds and its burst allowance. On the =
other hand, based on the presented experimental results, we would alternati=
vely like to see why CoDel and PIE performed worse that ARED (designed in 2=
001) while they were expected to achieve latencies far better than *RED. =
=A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-=
word"><span><div><div><div dir=3D"ltr"><div class=3D"gmail_extra"><div clas=
s=3D"gmail_quote">

</div>
</div>
</div>
</div>
</div>
</span><div class=3D"im">
<div><br>
</div>
<div><br>
</div>
<span style=3D"font-size:16px;font-family:Calibri,sans-serif">
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div>Whether this is a good thing or bad thing is a subjective matter in my=
 opinion, as most AQMs at some point will drop every packet whether it be t=
he maximum queue length or at some thresholds.=A0</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
</div><div style=3D"font-size:16px;font-family:Calibri,sans-serif">
----------------------------</div>
<div style=3D"font-family:Calibri,sans-serif">
<div style=3D"font-size:16px">&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&=
gt;RP: I don=92t think so. Neither PIE nor Codel has drop-all threshold as =
AQM part of it. Tail drop exists, but it is not part of AQM. Putting the la=
tency-based =A0ARED aside, ARED in this current form has the issues that
 we mentioned in the previous emails. It needs significant and careful rede=
sign.=A0</div></div></div></blockquote><div><br></div><div style>NAEEMK: Th=
e same response as above.=A0</div><div>=A0</div><blockquote class=3D"gmail_=
quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1=
ex">
<div style=3D"word-wrap:break-word"><div style=3D"font-family:Calibri,sans-=
serif">
<div style=3D"font-size:16px">---------------------------</div>
<div style=3D"font-size:16px"><br>
</div>
<div style=3D"font-size:16px">There is also a comment about &quot;<span sty=
le=3D"font-size:14px">Throwing away all the knowledge and improvement gaine=
d from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs =85&quot=
;.</span></div>

<div style=3D"font-size:16px"><span style=3D"font-size:14px"><br>
</span></div>
<div style=3D"font-size:16px"><span style=3D"font-size:14px">--------------=
----------------------------</span></div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: After I worked on CHOKe (n=
ow in Linux Kernel) =A0in 2000 as part of my Ph.D. thesis at Stanford, I ha=
ve worked on AFD (implemented in many Cisco flagship products), QCN (now IE=
EEE standard for data center ethernet, 802.1Qau), all of which are
 AQM schemes.=A0I also helped writing guidelines in tuning WRED at Cisco.=
=A0New thinkings were brought into PIE after I learned precious lessons (bo=
th good and bad) from all of them. Discarding them as &quot;coming up with =
brand-new AQMs&quot; is ignorant.=A0</div>
</div></div></blockquote><div><br></div><div style>NAEEMK: agreed! and we a=
re on same side :-) though it will be nice to=A0see the connection points b=
etween above AQMs and PIE and lessons learned being somehow documented.=A0<=
/div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-=
word"><div style=3D"font-family:Calibri,sans-serif">
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Rong</div>
</div>
<div><span style=3D"color:rgb(80,0,80)">--------------------------------</s=
pan></div>
<div><span style=3D"color:rgb(80,0,80)"><br>
</span></div>
<span>
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
Cheers,</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
Naeem</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
<br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">
&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP:</div></div></blockquote=
></div></div></div></div>

--047d7b5da93be1b68d04eb295809--

From naeem.khademi@gmail.com  Thu Nov 14 13:06:55 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 30C1311E810D for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 13:06:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.325
X-Spam-Level: 
X-Spam-Status: No, score=-2.325 tagged_above=-999 required=5 tests=[AWL=0.047,  BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YB4L-ZIZaHSA for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 13:06:54 -0800 (PST)
Received: from mail-vb0-x232.google.com (mail-vb0-x232.google.com [IPv6:2607:f8b0:400c:c02::232]) by ietfa.amsl.com (Postfix) with ESMTP id 5B09011E80FA for <aqm@ietf.org>; Thu, 14 Nov 2013 13:06:54 -0800 (PST)
Received: by mail-vb0-f50.google.com with SMTP id x11so2136392vbb.23 for <aqm@ietf.org>; Thu, 14 Nov 2013 13:06:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WpHPFIyajG7ntPhLlQMxzihZgoRqFdyywGQi0LRiHJk=; b=Ej2rZa/fDSmkNgmiMaJT85Za0+OmoS+HcPH6mSIX+Xb1EnL1rzAFn9GiUYFfKTNywT du09IZVhAOBbWy+djgyROjND0+v9pgQn8jeJBxun6OVfw/bOPz4gO5DXuSyc3Rs6bS9g N6xW7IM2KXLBeqYi6sufGU4jfRvzd05O2ZP8RYIZLRqt5Bkoxs/rkDoh8sRugTAtS6gc hix1dw9jPw9q8HmKbDCpLpMqGR2xDSu5sRwsDJJXm2uCTuhHrLOmp7dEhWoazjgg3HWB a3rxxt0Ef+wy+Pq82ktelVcmhi2SvqZrMAhlFMy6twVyW8DFJezAp6q+DOVSd0MpN6ks qCBQ==
MIME-Version: 1.0
X-Received: by 10.221.16.200 with SMTP id pz8mr756574vcb.53.1384463211012; Thu, 14 Nov 2013 13:06:51 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Thu, 14 Nov 2013 13:06:50 -0800 (PST)
In-Reply-To: <CEAA68B4.4B53A%prenatar@cisco.com>
References: <CAEjQQ5U+86aT=V9FxvSam9mQNzvUdtuf25HV55XXMQPE5095Og@mail.gmail.com> <CEAA68B4.4B53A%prenatar@cisco.com>
Date: Thu, 14 Nov 2013 22:06:50 +0100
Message-ID: <CAEjQQ5WpeB0L7pN3joh233QgvNV2VMkN2U9BPCwFmBYjoVCrAQ@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=001a11338bfee5a93c04eb297502
Cc: Anoop Ghanwani <anoop@alumni.duke.edu>, "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>, curtis <curtis@ipv6.occnc.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 21:06:55 -0000

--001a11338bfee5a93c04eb297502
Content-Type: text/plain; charset=ISO-8859-1

This should have probably been brought in different thread...

comments follow inline

On Thu, Nov 14, 2013 at 9:00 PM, Preethi Natarajan <preethi.cis@gmail.com>wrote:

>
>
> From: Naeem Khademi <naeem.khademi@gmail.com>
> Date: Thursday, November 14, 2013 8:05 AM
> To: Anoop Ghanwani <anoop@alumni.duke.edu>
> Cc: <curtis@ipv6.occnc.com>, Michael Welzl <michawe@ifi.uio.no>, "
> aqm@ietf.org" <aqm@ietf.org>, Preethi Natarajan <preethi.cis@gmail.com>
>
> Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
>
>
>
> Below is my personal opinion, but hopefully Fred can clarify this better
> based on the AQM recommendations draft:
>
> "applicability of AQM to all types of networks/switches" => "yes"
>
> "applicability of *any* AQM to all types of networks/switches" => "no"
>
>
>> Perhaps AQM cannot help this, but hopefully it won't hurt.  Trying to do
>> fancy things with small buffers is challenging.
>>
>
> AQM will most likely to help from data centers to the access links, and so
> on. But we may possibly need different AQMs for different network
> scenarios; The fact that an AQMs should be auto-tunable doesn't imply that
> it can be applied everywhere and we may need different auto-tunable AQMs
> specifically designed for different networks (ideally better if we could
> use fewer of them and they could work everywhere). I hope I'm not wrong.
>
>
>
> Again, please hold on. Even if its personal opinion, I would state facts
> backed with evidence instead of "hopes", since the community's thought
> process is at stake here.
>
> From the preliminary results we've seen, PIE has been able to address data
> center issues quite well. There is no evidence thus far why PIE's control
> law with auto-tuned parameters cannot adapt to data center or other network
> environments.
>

Taking what is mentioned in above sentence granted, there is no evidence if
it cannot or if it can and this point is also orthogonal to my response to
Anoop which was about his question at IETF about AQM deployment in general.



>
> Given that, its too early to "hope" that different network scenarios needdifferent AQM schemes. Of course, different network environments may choose
> to deploy different AQM schemes for various other reasons, which is not the
> point of discussion here.
>

It should have been on another thread. Sorry :-)


>
> Preethi
>

Naeem

--001a11338bfee5a93c04eb297502
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra">This should have probably b=
een brought in different thread...</div><div class=3D"gmail_extra"><br></di=
v><div class=3D"gmail_extra">comments follow inline<br><br><div class=3D"gm=
ail_quote">
On Thu, Nov 14, 2013 at 9:00 PM, Preethi Natarajan <span dir=3D"ltr">&lt;<a=
 href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.=
com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style=3D"font-size:14px;font-family:Calibri,sans-serif;word-wrap:break=
-word"><div style><br></div><div style><br></div><span style><div style=3D"=
border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt=
;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calib=
ri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none"=
>
<span style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D=
"mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com<=
/a>&gt;<br><span style=3D"font-weight:bold">Date: </span> Thursday, Novembe=
r 14, 2013 8:05 AM<br>
<span style=3D"font-weight:bold">To: </span> Anoop Ghanwani &lt;<a href=3D"=
mailto:anoop@alumni.duke.edu" target=3D"_blank">anoop@alumni.duke.edu</a>&g=
t;<br><span style=3D"font-weight:bold">Cc: </span> &lt;<a href=3D"mailto:cu=
rtis@ipv6.occnc.com" target=3D"_blank">curtis@ipv6.occnc.com</a>&gt;, Micha=
el Welzl &lt;<a href=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michaw=
e@ifi.uio.no</a>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org" target=3D"_blan=
k">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org" target=3D"_bl=
ank">aqm@ietf.org</a>&gt;, Preethi Natarajan &lt;<a href=3D"mailto:preethi.=
cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</a>&gt;<div class=3D=
"im">
<br><span style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes=
: Queue length vs. delay based<br></div></div><div><br></div><blockquote st=
yle=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div dir=
=3D"ltr">
<br><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div><br></div><d=
iv class=3D"im"><div>Below is my personal opinion, but hopefully Fred can c=
larify this better based on the AQM recommendations draft:</div><div><br></=
div>
<div>&quot;applicability of AQM to all types of networks/switches&quot; =3D=
&gt; &quot;yes&quot;</div><div><br></div><div>&quot;applicability of *any* =
AQM to all types of networks/switches&quot; =3D&gt; &quot;no&quot; =A0</div=
><div>
=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde=
r-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmai=
l_extra"><div class=3D"gmail_quote"><div>Perhaps AQM cannot help this, but =
hopefully it won&#39;t hurt. =A0Trying to do fancy things with small buffer=
s is challenging.</div>
</div></div></div></blockquote><div><br></div><div>AQM will most likely to =
help from data centers to the access links, and so on. But we may possibly =
need different AQMs for different network scenarios; The fact that an AQMs =
should be auto-tunable doesn&#39;t imply that it can be applied everywhere =
and we may need different auto-tunable AQMs specifically designed for diffe=
rent networks (ideally better if we could use fewer of them and they could =
work everywhere). I hope I&#39;m not wrong. =A0=A0</div>
<div>=A0</div></div></div></div></div></blockquote></span><div style><br></=
div><span><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_q=
uote"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote=
"><span><font><div>
Again, please hold on. Even if its personal opinion, I would state facts ba=
cked with evidence instead of &quot;hopes&quot;, since the community&#39;s =
thought process is at stake here.</div></font></span></div></div></div>
</div></div></div></span><span><div dir=3D"ltr"><div class=3D"gmail_extra">=
<div class=3D"gmail_quote"><div dir=3D"ltr"><div class=3D"gmail_extra"><div=
 class=3D"gmail_quote"><font></font><font></font></div></div></div></div></=
div></div>
</span><div><font></font></div><div><br></div><span><div dir=3D"ltr"><div c=
lass=3D"gmail_extra"><div class=3D"gmail_quote"><div dir=3D"ltr"><div class=
=3D"gmail_extra"><div class=3D"gmail_quote"><div>From the preliminary resul=
ts we&#39;ve seen, PIE has been able to address data center issues quite we=
ll. There is no evidence thus far why PIE&#39;s control law with auto-tuned=
 parameters cannot adapt to data center or other network environments.=A0</=
div>
</div></div></div></div></div></div></span></div></blockquote><div><br></di=
v><div style>Taking what is mentioned in above sentence granted, there is n=
o evidence if it cannot or if it can and this point is also orthogonal to m=
y response to Anoop which was about his question at IETF about AQM deployme=
nt in general. =A0=A0</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div style=3D"font-size:14px;f=
ont-family:Calibri,sans-serif;word-wrap:break-word"><span><div dir=3D"ltr">=
<div class=3D"gmail_extra">
<div class=3D"gmail_quote"><div dir=3D"ltr"><div class=3D"gmail_extra"><div=
 class=3D"gmail_quote"><div><br></div><div>Given that, its too early to &qu=
ot;hope&quot; that different network scenarios <span style=3D"font-style:it=
alic">need</span> different AQM schemes. Of course, different network envir=
onments may choose to deploy different AQM schemes for various other reason=
s, which is not the point of discussion here.</div>
</div></div></div></div></div></div></span></div></blockquote><div><br></di=
v><div style>It should have been on another thread. Sorry :-)</div><div>=A0=
</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l=
eft:1px #ccc solid;padding-left:1ex">
<div style=3D"font-size:14px;font-family:Calibri,sans-serif;word-wrap:break=
-word"><span><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmai=
l_quote"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_qu=
ote"><span class=3D"HOEnZb"><font color=3D"#888888"><div>
<br></div><div>Preethi</div></font></span></div></div></div></div></div></d=
iv></span></div></blockquote><div><br></div><div style>Naeem=A0</div></div>=
<br></div></div>

--001a11338bfee5a93c04eb297502--

From ropan@cisco.com  Thu Nov 14 14:32:23 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 903D511E80FE for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 14:32:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.371
X-Spam-Level: 
X-Spam-Status: No, score=-10.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2c-hyI2ooc5N for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 14:32:17 -0800 (PST)
Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id 5B87911E811B for <aqm@ietf.org>; Thu, 14 Nov 2013 14:32:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=18930; q=dns/txt; s=iport; t=1384468337; x=1385677937; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=Rz1neHRIWm8CK2Fc1aUcYJYr116VmIB4FsKAYpE5ICw=; b=aEZm2lpQINHJInsyVmmMFl2KA03sO0/GF0JOE3hke/Jccv3GTB4GK0oV RMKPMQoahN2nu5zdkRovPKPRhdJVHDuUDRy65qocuu1iV6JnoKBseilVt ufjQRAUBx/c40jE0gJu+6k/Xe8Xs71jcf4evQWgJNVxYWlug0zXgS/+Bk g=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgsFAH1OhVKtJV2c/2dsb2JhbABagkNEgQu/HoEhFnSCJQEBAQMBbAYHBQ0BCBEDAQIoKBEUCQgCBA4Fh28DCQa3CQ2JU4xtgSmBOBEHhDEDliWBa4xUhTiDKIFxOQ
X-IronPort-AV: E=Sophos;i="4.93,702,1378857600";  d="scan'208,217";a="284989134"
Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-7.cisco.com with ESMTP; 14 Nov 2013 22:32:16 +0000
Received: from xhc-aln-x03.cisco.com (xhc-aln-x03.cisco.com [173.36.12.77]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id rAEMWAcW009619 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 14 Nov 2013 22:32:16 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.147]) by xhc-aln-x03.cisco.com ([173.36.12.77]) with mapi id 14.03.0123.003; Thu, 14 Nov 2013 16:32:10 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
Thread-Index: AQHO3CzV4Y8AQmoIJE6ksgN55dTkopobOREAgADcp4CAACSPAP//9emAgABSr4CABi75AIAA200AgABagICAAB6IAIAAVMEAgADNWwCAACSHAP//sJoAgACnbQD//4M6gA==
Date: Thu, 14 Nov 2013 22:32:10 +0000
Message-ID: <CEAA8EDD.55A7D%ropan@cisco.com>
In-Reply-To: <CAEjQQ5WW_VfXC_PQ=eaZnxg-9n3BrxLk3zKAX3heWedm_shTqw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [128.107.165.143]
Content-Type: multipart/alternative; boundary="_000_CEAA8EDD55A7Dropanciscocom_"
MIME-Version: 1.0
Cc: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>, Preethi Natarajan <preethi.cis@gmail.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Nov 2013 22:32:23 -0000

--_000_CEAA8EDD55A7Dropanciscocom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable



From: Naeem Khademi <naeem.khademi@gmail.com<mailto:naeem.khademi@gmail.com=
>>
Date: Thursday, November 14, 2013 1:58 PM
To: Cisco Employee <ropan@cisco.com<mailto:ropan@cisco.com>>
Cc: Preethi Natarajan <preethi.cis@gmail.com<mailto:preethi.cis@gmail.com>>=
, "aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>, =
Michael Welzl <michawe@ifi.uio.no<mailto:michawe@ifi.uio.no>>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based

Hi Rong

comments follow inline

Thanks
Naeem

On Thu, Nov 14, 2013 at 8:59 PM, Rong Pan (ropan) <ropan@cisco.com<mailto:r=
opan@cisco.com>> wrote:
Please see inline=85

Thanks,

Rong



From: Naeem Khademi <naeem.khademi@gmail.com<mailto:naeem.khademi@gmail.com=
>>
Date: Thursday, November 14, 2013 8:43 AM
To: Preethi Natarajan <preethi.cis@gmail.com<mailto:preethi.cis@gmail.com>>
Cc: "aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>=
, Michael Welzl <michawe@ifi.uio.no<mailto:michawe@ifi.uio.no>>

Subject: Re: [aqm] AQM schemes: Queue length vs. delay based


Delay-based ARED behaves similar to tail drop at max threshold.

I think I now understand what you mean by this sentence ;-) and therefore p=
lease ignore the paragraph responding to this specific point in the previou=
s email (sorry about that, I got it mixed with max_target). Indeed ARED dro=
ps every packet when the average queue length grows beyond th_max as drop p=
robability jumps to 1 (when it's not in gentle mode). For this to happen, t=
he "average queue length" has to grow beyond th_max and depending on how th=
e averaging is done, that can correspond to certain burst allowance at AQM.=
 Moreover, this is done over 500 ms interval which means the average queue =
length has to stay above th_max for mostly few rounds of RTTs (assuming the=
 typical distribution of RTTs on the Internet to have a mean/average somewh=
ere below that value e.g. 100~200 ms). This allows burst that don't take th=
e average queue size to above th_max over 500 ms period to pass and clear t=
hemselves up the queue.

 > This is unacceptable of an AQM even for max thresholds around 50ms.

-----------------------------
>>>>>>>>>>>>>RP: Two issues here, first, averaging has a parameter. How to =
tune it to avoid tail drop behavior? What would be its relationship to max_=
th? Do we understand ?  Second, one single tcp's behavior is very bursty, i=
f 500ms burst allowance is effective in ARED, how come is it not reflected =
in your throughput chart? The one single tcp throughput is very low for low=
 latency numbers, which is a clear indication of not enough buffering.
------------------------------------


NAEEMK: As stated before, we're not advocating using ARED whatsoever. There=
fore trying to fix/address ARED's inherent issues that have been stated on =
this thread repeatedly would not be an option nor it enhances the argument =
to support PIE or CoDel for deployment. So, I'm not going to speculate on h=
ow one can possibly fix ARED thresholds and its burst allowance. On the oth=
er hand, based on the presented experimental results, we would alternativel=
y like to see why CoDel and PIE performed worse that ARED (designed in 2001=
) while they were expected to achieve latencies far better than *RED.




RP: If low latencies are achieved at the cost of losing quite a bit of thro=
ughput, it is a no-starter for AQM design.



Whether this is a good thing or bad thing is a subjective matter in my opin=
ion, as most AQMs at some point will drop every packet whether it be the ma=
ximum queue length or at some thresholds.

----------------------------
>>>>>>>>>>>>RP: I don=92t think so. Neither PIE nor Codel has drop-all thre=
shold as AQM part of it. Tail drop exists, but it is not part of AQM. Putti=
ng the latency-based  ARED aside, ARED in this current form has the issues =
that we mentioned in the previous emails. It needs significant and careful =
redesign.

NAEEMK: The same response as above.

---------------------------

There is also a comment about "Throwing away all the knowledge and improvem=
ent gained from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs=
 =85".

------------------------------------------
>>>>>>>>>>RP: After I worked on CHOKe (now in Linux Kernel)  in 2000 as par=
t of my Ph.D. thesis at Stanford, I have worked on AFD (implemented in many=
 Cisco flagship products), QCN (now IEEEE standard for data center ethernet=
, 802.1Qau), all of which are AQM schemes. I also helped writing guidelines=
 in tuning WRED at Cisco. New thinkings were brought into PIE after I learn=
ed precious lessons (both good and bad) from all of them. Discarding them a=
s "coming up with brand-new AQMs" is ignorant.

NAEEMK: agreed! and we are on same side :-) though it will be nice to see t=
he connection points between above AQMs and PIE and lessons learned being s=
omehow documented.

>>>>>>RP: starting reading some of the papers about those designs, especial=
ly PIE on HPSR2013, would be a good start=85.


Regards,

Rong



Regards,

Rong
--------------------------------


Cheers,
Naeem

>>>>>>>>>>>>RP:

--_000_CEAA8EDD55A7Dropanciscocom_
Content-Type: text/html; charset="Windows-1252"
Content-ID: <624F6AF457C65C45BEFEF1BE8DD3FEFB@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1=
252">
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-fami=
ly: Calibri, sans-serif; ">
<div><br>
</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div style=3D"font-family:Calibri; font-size:14pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>Naeem Khademi &lt;<a href=3D"=
mailto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Thursday, November 14, 2013 1=
:58 PM<br>
<span style=3D"font-weight:bold">To: </span>Cisco Employee &lt;<a href=3D"m=
ailto:ropan@cisco.com">ropan@cisco.com</a>&gt;<br>
<span style=3D"font-weight:bold">Cc: </span>Preethi Natarajan &lt;<a href=
=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a>&gt;, &quot;<a h=
ref=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aq=
m@ietf.org">aqm@ietf.org</a>&gt;, Michael Welzl &lt;<a href=3D"mailto:micha=
we@ifi.uio.no">michawe@ifi.uio.no</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span>Re: [aqm] AQM schemes: Que=
ue length vs. delay based<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir=3D"ltr">Hi Rong&nbsp;
<div><br>
</div>
<div>comments follow inline<br>
<div class=3D"gmail_extra"><br>
</div>
<div class=3D"gmail_extra">Thanks</div>
<div class=3D"gmail_extra">Naeem<br>
<br>
<div class=3D"gmail_quote">On Thu, Nov 14, 2013 at 8:59 PM, Rong Pan (ropan=
) <span dir=3D"ltr">
&lt;<a href=3D"mailto:ropan@cisco.com" target=3D"_blank">ropan@cisco.com</a=
>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word">
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">Please see inl=
ine=85</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">Thanks,</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">Rong</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<span style=3D"font-size: 16px; font-family: Calibri, sans-serif; ">
<div style=3D"border-right:medium none;padding-right:0in;padding-left:0in;p=
adding-top:3pt;text-align:left;font-size:14pt;border-bottom:medium none;fon=
t-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-lef=
t:medium none">
<span style=3D"font-weight:bold">From: </span>Naeem Khademi &lt;<a href=3D"=
mailto:naeem.khademi@gmail.com" target=3D"_blank">naeem.khademi@gmail.com</=
a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Thursday, November 14, 2013 8=
:43 AM<br>
<span style=3D"font-weight:bold">To: </span>Preethi Natarajan &lt;<a href=
=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</=
a>&gt;<br>
<span style=3D"font-weight:bold">Cc: </span>&quot;<a href=3D"mailto:aqm@iet=
f.org" target=3D"_blank">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@i=
etf.org" target=3D"_blank">aqm@ietf.org</a>&gt;, Michael Welzl &lt;<a href=
=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no</a>&gt;
<div class=3D"im"><br>
<span style=3D"font-weight:bold">Subject: </span>Re: [aqm] AQM schemes: Que=
ue length vs. delay based<br>
</div>
</div>
<div><br>
</div>
<div>
<div>
<div dir=3D"ltr"><span style=3D"color:rgb(80,0,80)">&nbsp;</span><br>
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex">
<div style=3D"word-wrap:break-word">
<div>
<div style=3D"padding:12px 55px 0px 0px;margin:0px;overflow:auto;font-size:=
medium;font-family:Helvetica;word-wrap:break-word">
<span style=3D"word-wrap:break-word;font-style:normal;font-variant:normal;f=
ont-weight:normal;font-size:14px;line-height:normal;font-family:Calibri">De=
lay-based ARED behaves similar to tail drop at max threshold.</span></div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<div class=3D"im">
<div><br>
</div>
<div>I think I now understand what you mean by this sentence ;-) and theref=
ore please ignore the paragraph responding to this specific point in the pr=
evious email (sorry about that, I got it mixed with max_target). Indeed ARE=
D drops every packet when the average
 queue length grows beyond th_max as drop probability jumps to 1 (when it's=
 not in gentle mode). For this to happen, the &quot;average queue length&qu=
ot; has to grow beyond th_max and depending on how the averaging is done, t=
hat can correspond to certain burst allowance
 at AQM. Moreover, this is done over 500 ms interval which means the averag=
e queue length has to stay above th_max for mostly few rounds of RTTs (assu=
ming the typical distribution of RTTs on the Internet to have a mean/averag=
e somewhere below that value e.g.
 100~200 ms). This allows burst that don't take the average queue size to a=
bove th_max over 500 ms period to pass and clear themselves up the queue. &=
nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;</div>
<div><br>
</div>
<div><span style=3D"color:rgb(80,0,80);font-family:Calibri;font-size:14px">=
&nbsp;&gt; This is unacceptable of an AQM even for max thresholds around 50=
ms.&nbsp;</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">--------------=
---------------</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">&gt;&gt;&gt;&g=
t;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: Two issues here, first, averaging=
 has a parameter. How to tune it to avoid tail drop behavior? What would be=
 its relationship to max_th? Do we understand ? &nbsp;Second, one single tc=
p's
 behavior is very bursty, if 500ms burst allowance is effective in ARED, ho=
w come is it not reflected in your throughput chart? The one single tcp thr=
oughput is very low for low latency numbers, which is a clear indication of=
 not enough buffering.&nbsp;</div>
<span>
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><span style=3D=
"color:rgb(80,0,80);font-size:14px;font-family:Calibri">-------------------=
-----------------</span></div>
</div>
</div>
</div>
</div>
</div>
</span></div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div style=3D"">NAEEMK: As stated before, we're not advocating using ARED w=
hatsoever. Therefore trying to fix/address ARED's inherent issues that have=
 been stated on this thread repeatedly would not be an option nor it enhanc=
es the argument to support PIE or
 CoDel for deployment. So, I'm not going to speculate on how one can possib=
ly fix ARED thresholds and its burst allowance. On the other hand, based on=
 the presented experimental results, we would alternatively like to see why=
 CoDel and PIE performed worse that
 ARED (designed in 2001) while they were expected to achieve latencies far =
better than *RED. &nbsp;</div>
<div>&nbsp;</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>RP: If low latencies are achieved at the cost of losing quite a bit of=
 throughput, it is a no-starter for AQM design.&nbsp;</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div>
<div dir=3D"ltr">
<div>
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><span>
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote"></div>
</div>
</div>
</div>
</div>
</span>
<div class=3D"im">
<div><br>
</div>
<div><br>
</div>
<span style=3D"font-size: 16px; font-family: Calibri, sans-serif; ">
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div>Whether this is a good thing or bad thing is a subjective matter in my=
 opinion, as most AQMs at some point will drop every packet whether it be t=
he maximum queue length or at some thresholds.&nbsp;</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">--------------=
--------------</div>
<div style=3D"font-family:Calibri,sans-serif">
<div style=3D"font-size:16px">&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&=
gt;RP: I don=92t think so. Neither PIE nor Codel has drop-all threshold as =
AQM part of it. Tail drop exists, but it is not part of AQM. Putting the la=
tency-based &nbsp;ARED aside, ARED in this current form has the issues that=
 we
 mentioned in the previous emails. It needs significant and careful redesig=
n.&nbsp;</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div style=3D"">NAEEMK: The same response as above.&nbsp;</div>
<div>&nbsp;</div>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word">
<div style=3D"font-family:Calibri,sans-serif">
<div style=3D"font-size:16px">---------------------------</div>
<div style=3D"font-size:16px"><br>
</div>
<div style=3D"font-size:16px">There is also a comment about &quot;<span sty=
le=3D"font-size:14px">Throwing away all the knowledge and improvement gaine=
d from RED-like AQMs, CHoKe, etc and coming up with brand-new AQMs =85&quot=
;.</span></div>
<div style=3D"font-size:16px"><span style=3D"font-size:14px"><br>
</span></div>
<div style=3D"font-size:16px"><span style=3D"font-size:14px">--------------=
----------------------------</span></div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: After I worked on CHOKe (n=
ow in Linux Kernel) &nbsp;in 2000 as part of my Ph.D. thesis at Stanford, I=
 have worked on AFD (implemented in many Cisco flagship products), QCN (now=
 IEEEE standard for data center ethernet, 802.1Qau), all of which are
 AQM schemes.&nbsp;I also helped writing guidelines in tuning WRED at Cisco=
.&nbsp;New thinkings were brought into PIE after I learned precious lessons=
 (both good and bad) from all of them. Discarding them as &quot;coming up w=
ith brand-new AQMs&quot; is ignorant.&nbsp;</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div style=3D"">NAEEMK: agreed! and we are on same side :-) though it will =
be nice to&nbsp;see the connection points between above AQMs and PIE and le=
ssons learned being somehow documented.&nbsp;</div>
<div>&nbsp;</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div>&gt;&gt;&gt;&gt;&gt;&gt;RP: starting reading some of the papers about =
those designs, especially PIE on HPSR2013, would be a good start=85.</div>
<div><br>
</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Rong</div>
<div><br>
</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div>
<div dir=3D"ltr">
<div>
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word">
<div style=3D"font-family:Calibri,sans-serif">
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Rong</div>
</div>
<div><span style=3D"color:rgb(80,0,80)">--------------------------------</s=
pan></div>
<div><span style=3D"color:rgb(80,0,80)"><br>
</span></div>
<span>
<div>
<div>
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">Cheers,</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">Naeem</div>
</div>
</div>
</div>
</div>
</div>
</span>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif"><br>
</div>
<div style=3D"font-size:16px;font-family:Calibri,sans-serif">&gt;&gt;&gt;&g=
t;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP:</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>

--_000_CEAA8EDD55A7Dropanciscocom_--

From preethi.cis@gmail.com  Thu Nov 14 17:33:41 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6778321E80F1 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 17:33:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.975
X-Spam-Level: 
X-Spam-Status: No, score=-0.975 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UGT7UWfldwI9 for <aqm@ietfa.amsl.com>; Thu, 14 Nov 2013 17:33:40 -0800 (PST)
Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) by ietfa.amsl.com (Postfix) with ESMTP id B0AE021E80EC for <aqm@ietf.org>; Thu, 14 Nov 2013 17:33:40 -0800 (PST)
Received: by mail-pd0-f170.google.com with SMTP id q10so2809735pdj.1 for <aqm@ietf.org>; Thu, 14 Nov 2013 17:33:40 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=ZBuDahNjzos8nlG92ibdYh8nTZjrp99jWgbZ56cWy3I=; b=nBniZ13rhR9A8L5l6mD98Gu3ax7B37fSthBcOLVPPUWEA2LuhIT1lB+X5TN8oyXNOD mwGHwHVCvObDIwLbOAMqmvd0VBhaNRGhFW5SykuEWi7H9bXqCVC/mt1AJZa10MrC1Kgk 8J9a/PxtzxA/gchGd+1cGQBvFV8vyRH8hWteiUPP4woYTVJjTI3zjRIfwPe5w2Jrv9n9 Hu8aUDQxhdS1kpdTda195EaKR20OwRhovZqOJUldqVilb3/ryNhXiN+3ujslBpmOINYV kNE7Ua5kyxvNQQ/B7YTbnIUtcfHpIeOGUEUG4iHb9IOs4l+n76BgwE7DurP/wb+5jUo6 AcMg==
X-Received: by 10.66.216.162 with SMTP id or2mr306172pac.179.1384479219381; Thu, 14 Nov 2013 17:33:39 -0800 (PST)
Received: from [10.21.113.167] (128-107-239-233.cisco.com. [128.107.239.233]) by mx.google.com with ESMTPSA id pl1sm619614pbb.20.2013.11.14.17.33.37 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 14 Nov 2013 17:33:38 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Thu, 14 Nov 2013 17:33:35 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Naeem Khademi <naeem.khademi@gmail.com>
Message-ID: <CEAAB727.4B5D1%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CAEjQQ5WpeB0L7pN3joh233QgvNV2VMkN2U9BPCwFmBYjoVCrAQ@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467295218_8405811"
Cc: Anoop Ghanwani <anoop@alumni.duke.edu>, "aqm@ietf.org" <aqm@ietf.org>, Michael Welzl <michawe@ifi.uio.no>, curtis <curtis@ipv6.occnc.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 01:33:41 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467295218_8405811
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit



From:  Naeem Khademi <naeem.khademi@gmail.com>
Date:  Thursday, November 14, 2013 1:06 PM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  curtis <curtis@ipv6.occnc.com>, Michael Welzl <michawe@ifi.uio.no>,
"aqm@ietf.org" <aqm@ietf.org>, Anoop Ghanwani <anoop@alumni.duke.edu>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> 
> This should have probably been brought in different thread...
> 
> comments follow inline
> 
> On Thu, Nov 14, 2013 at 9:00 PM, Preethi Natarajan <preethi.cis@gmail.com>
> wrote:
>> 
>> 
>> From:  Naeem Khademi <naeem.khademi@gmail.com>
>> Date:  Thursday, November 14, 2013 8:05 AM
>> To:  Anoop Ghanwani <anoop@alumni.duke.edu>
>> Cc:  <curtis@ipv6.occnc.com>, Michael Welzl <michawe@ifi.uio.no>,
>> "aqm@ietf.org" <aqm@ietf.org>, Preethi Natarajan <preethi.cis@gmail.com>
>> 
>> Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based
>> 
>>> 
>>> 
>>> Below is my personal opinion, but hopefully Fred can clarify this better
>>> based on the AQM recommendations draft:
>>> 
>>> "applicability of AQM to all types of networks/switches" => "yes"
>>> 
>>> "applicability of *any* AQM to all types of networks/switches" => "no"
>>>  
>>>> Perhaps AQM cannot help this, but hopefully it won't hurt.  Trying to do
>>>> fancy things with small buffers is challenging.
>>> 
>>> AQM will most likely to help from data centers to the access links, and so
>>> on. But we may possibly need different AQMs for different network scenarios;
>>> The fact that an AQMs should be auto-tunable doesn't imply that it can be
>>> applied everywhere and we may need different auto-tunable AQMs specifically
>>> designed for different networks (ideally better if we could use fewer of
>>> them and they could work everywhere). I hope I'm not wrong.
>>>  
>> 
>> Again, please hold on. Even if its personal opinion, I would state facts
>> backed with evidence instead of "hopes", since the community's thought
>> process is at stake here.
>> 
>> From the preliminary results we've seen, PIE has been able to address data
>> center issues quite well. There is no evidence thus far why PIE's control law
>> with auto-tuned parameters cannot adapt to data center or other network
>> environments. 
> 
> Taking what is mentioned in above sentence granted, there is no evidence if it
> cannot or if it can and this point is also orthogonal to my response to Anoop
> which was about his question at IETF about AQM deployment in general.


Just pointing out the evidence here --
http://www.ietf.org/proceedings/86/slides/slides-86-iccrg-5.pdf

The slides contain preliminary results from DC scenarios, showing PIE's
parameters and adaptability to DC scenarios. Thank god for the IETF archives
:)

Thanks,
Preethi



--B_3467295218_8405811
Content-type: text/html;
	charset="US-ASCII"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:=
 14px; font-family: Calibri, sans-serif; "><div><br></div><div><br></div><sp=
an id=3D"OLK_SRC_BODY_SECTION"><div style=3D"font-family:Calibri; font-size:11pt=
; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: med=
ium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER=
-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span =
style=3D"font-weight:bold">From: </span> Naeem Khademi &lt;<a href=3D"mailto:nae=
em.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;<br><span style=3D"font-w=
eight:bold">Date: </span> Thursday, November 14, 2013 1:06 PM<br><span style=
=3D"font-weight:bold">To: </span> Preethi Natarajan &lt;<a href=3D"mailto:preeth=
i.cis@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span style=3D"font-weight:b=
old">Cc: </span> curtis &lt;<a href=3D"mailto:curtis@ipv6.occnc.com">curtis@ip=
v6.occnc.com</a>&gt;, Michael Welzl &lt;<a href=3D"mailto:michawe@ifi.uio.no">=
michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>"=
 &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;, Anoop Ghanwani &lt;=
<a href=3D"mailto:anoop@alumni.duke.edu">anoop@alumni.duke.edu</a>&gt;<br><spa=
n style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue len=
gth vs. delay based<br></div><div><br></div><blockquote id=3D"MAC_OUTLOOK_ATTR=
IBUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MA=
RGIN:0 0 0 5;"><div dir=3D"ltr"><br><div class=3D"gmail_extra">This should have =
probably been brought in different thread...</div><div class=3D"gmail_extra"><=
br></div><div class=3D"gmail_extra">comments follow inline<br><br><div class=3D"=
gmail_quote">
On Thu, Nov 14, 2013 at 9:00 PM, Preethi Natarajan <span dir=3D"ltr">&lt;<a h=
ref=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</a>=
&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8=
ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"font-size:14px;f=
ont-family:Calibri,sans-serif;word-wrap:break-word"><div style=3D""><br></div>=
<div style=3D""><br></div><span style=3D""><div style=3D"border-right:medium none;=
padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size=
:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt s=
olid;padding-bottom:0in;border-left:medium none"><span style=3D"font-weight:bo=
ld">From: </span> Naeem Khademi &lt;<a href=3D"mailto:naeem.khademi@gmail.com"=
 target=3D"_blank">naeem.khademi@gmail.com</a>&gt;<br><span style=3D"font-weight=
:bold">Date: </span> Thursday, November 14, 2013 8:05 AM<br><span style=3D"fon=
t-weight:bold">To: </span> Anoop Ghanwani &lt;<a href=3D"mailto:anoop@alumni.d=
uke.edu" target=3D"_blank">anoop@alumni.duke.edu</a>&gt;<br><span style=3D"font-=
weight:bold">Cc: </span> &lt;<a href=3D"mailto:curtis@ipv6.occnc.com" target=3D"=
_blank">curtis@ipv6.occnc.com</a>&gt;, Michael Welzl &lt;<a href=3D"mailto:mic=
hawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailt=
o:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a>" &lt;<a href=3D"mailto:aqm@ie=
tf.org" target=3D"_blank">aqm@ietf.org</a>&gt;, Preethi Natarajan &lt;<a href=3D=
"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis@gmail.com</a>&gt;=
<div class=3D"im"><br><span style=3D"font-weight:bold">Subject: </span> Re: [aqm=
] AQM schemes: Queue length vs. delay based<br></div></div><div><br></div><b=
lockquote style=3D"BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"=
><div dir=3D"ltr"><br><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div><=
br></div><div class=3D"im"><div>Below is my personal opinion, but hopefully Fr=
ed can clarify this better based on the AQM recommendations draft:</div><div=
><br></div><div>"applicability of AQM to all types of networks/switches" =3D&g=
t; "yes"</div><div><br></div><div>"applicability of *any* AQM to all types o=
f networks/switches" =3D&gt; "no" &nbsp;</div><div>
&nbsp;</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border=
-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_extr=
a"><div class=3D"gmail_quote"><div>Perhaps AQM cannot help this, but hopefully=
 it won't hurt. &nbsp;Trying to do fancy things with small buffers is challe=
nging.</div></div></div></div></blockquote><div><br></div><div>AQM will most=
 likely to help from data centers to the access links, and so on. But we may=
 possibly need different AQMs for different network scenarios; The fact that=
 an AQMs should be auto-tunable doesn't imply that it can be applied everywh=
ere and we may need different auto-tunable AQMs specifically designed for di=
fferent networks (ideally better if we could use fewer of them and they coul=
d work everywhere). I hope I'm not wrong. &nbsp;&nbsp;</div><div>&nbsp;</div=
></div></div></div></div></blockquote></span><div style=3D""><br></div><span><=
div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div dir=3D"lt=
r"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><span><font><div>
Again, please hold on. Even if its personal opinion, I would state facts ba=
cked with evidence instead of "hopes", since the community's thought process=
 is at stake here.</div></font></span></div></div></div></div></div></div></=
span><span><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">=
<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><font></fon=
t><font></font></div></div></div></div></div></div></span><div><font></font>=
</div><div><br></div><span><div dir=3D"ltr"><div class=3D"gmail_extra"><div clas=
s=3D"gmail_quote"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_qu=
ote"><div>From the preliminary results we've seen, PIE has been able to addr=
ess data center issues quite well. There is no evidence thus far why PIE's c=
ontrol law with auto-tuned parameters cannot adapt to data center or other n=
etwork environments.&nbsp;</div></div></div></div></div></div></div></span><=
/div></blockquote><div><br></div><div style=3D"">Taking what is mentioned in a=
bove sentence granted, there is no evidence if it cannot or if it can and th=
is point is also orthogonal to my response to Anoop which was about his ques=
tion at IETF about AQM deployment in general. &nbsp;&nbsp;</div></div></div>=
</div></blockquote></span><div><br></div><div><br></div><div>Just pointing o=
ut the evidence here --&nbsp;<a href=3D"http://www.ietf.org/proceedings/86/sli=
des/slides-86-iccrg-5.pdf">http://www.ietf.org/proceedings/86/slides/slides-=
86-iccrg-5.pdf</a></div><div><br></div><div>The slides contain preliminary r=
esults from DC scenarios, showing PIE's parameters and adaptability to DC sc=
enarios. Thank god for the IETF archives :)</div><div><br></div><div>Thanks,=
</div><div>Preethi</div></body></html>

--B_3467295218_8405811--



From michawe@ifi.uio.no  Fri Nov 15 02:06:45 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CA05111E814D for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 02:06:45 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.371
X-Spam-Level: 
X-Spam-Status: No, score=-102.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mZDhKyyWDWvr for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 02:06:40 -0800 (PST)
Received: from mail-out5.uio.no (mail-out5.uio.no [IPv6:2001:700:100:10::17]) by ietfa.amsl.com (Postfix) with ESMTP id 9A53011E8151 for <aqm@ietf.org>; Fri, 15 Nov 2013 02:06:36 -0800 (PST)
Received: from mail-mx2.uio.no ([129.240.10.30]) by mail-out5.uio.no with esmtp (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1VhGIL-0002fl-7v; Fri, 15 Nov 2013 11:06:33 +0100
Received: from boomerang.ifi.uio.no ([129.240.68.135]) by mail-mx2.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from <michawe@ifi.uio.no>) id 1VhGIK-0000pL-Jd; Fri, 15 Nov 2013 11:06:33 +0100
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: multipart/alternative; boundary="Apple-Mail=_9FE79146-3F55-4054-BED8-34A4E55E8D41"
From: Michael Welzl <michawe@ifi.uio.no>
In-Reply-To: <CEAA8EDD.55A7D%ropan@cisco.com>
Date: Fri, 15 Nov 2013 11:06:32 +0100
Message-Id: <1B812DF5-1691-444A-AA6F-33C31732D16C@ifi.uio.no>
References: <CEAA8EDD.55A7D%ropan@cisco.com>
To: Rong Pan (ropan) <ropan@cisco.com>
X-Mailer: Apple Mail (2.1283)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 4 msgs/h 1 sum rcpts/h 5 sum msgs/h 1 total rcpts 9828 max rcpts/h 44 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-6.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.05, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: 309D5D99651F6588F00BD007060CD3CD65DDD832
X-UiO-SPAM-Test: remote_host: 129.240.68.135 spam_score: -59 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 3701 max/h 16 blacklist 0 greylist 0 ratelimit 0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 10:06:46 -0000

--Apple-Mail=_9FE79146-3F55-4054-BED8-34A4E55E8D41
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

Hi,

Interesting discussion!

I would like to say that in my opinion the real value of what Naeem has =
presented is not so much in pushing ARED, but in:

- reminding people that there was AQM stuff done between RED and CoDel, =
and some of this is worth looking at. Rong, we're aware of at least some =
of your previous work, and I'm personally a fan of CHOKe. However I do =
understand that CHOKe is probably not appropriate for the problem we're =
trying to solve here ... but you know, there were lots and lots of =
others made (e.g. AVQ, just to toss another name of an AQM that seemed =
to be well done to me when I read about it all these years ago)

- hopefully helping set the bar a bit higher regarding evaluations. I'm =
not saying we did enough (e.g. we didn't get to look at anything else =
than bulk TCP transfers yet), but I think we've done more than I have =
seen documented elsewhere so far (apologies if we missed a study; we =
tried to cover them in the table in our tech. rep.).


I just personally found it a bit pathetic to first see CoDel and PIE =
presented with comparisons against only themselves and RED, and then see =
that the more-than-a-decade-old-ARED very often performs better than =
both of them. It makes me wonder how many other 5-10 year old AQMs are =
at least as good as CoDel and PIE.


About this:


> RP: If low latencies are achieved at the cost of losing quite a bit of =
throughput, it is a no-starter for AQM design.=20

That's rather obvious  :-)    but it's about the relationship between =
the two. e.g., the evaluations in the original CoDel paper essentially =
showed that CoDel gave higher throughput than RED at the expense of more =
latency. That's not exactly convincing either, given that latency =
reduction is the main goal we're pursuing now. And for ARED, we're only =
talking about a subset of scenarios (and perhaps not even the most =
relevant ones -  e.g. do we really care that much about the throughput =
of one single TCP connection?). Let's not forget that in many scenarios =
that we looked at (in fact, I'd say this was the majority of cases) it =
had better throughput AND lower delay than CoDel and PIE. I would also =
venture to guess that the problem with low multiplexing cases could be =
addressed relatively easily.

Naeem and Preethi have agreed that some significant design changes would =
be necessary to make use of ARED as a new delay-based AQM. I agree, but =
mainly I'd expect these significant changes to deal with ECN (where we =
made the point that ALL AQMs should be changed in fact (*), but that's =
another story) and with WLAN scenarios, where ARED performed quite =
badly. We'll see - mainly we conclude that it's probably worth playing =
around with it.

Cheers,
Michael


--Apple-Mail=_9FE79146-3F55-4054-BED8-34A4E55E8D41
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
">Hi,<div><br></div><div>Interesting =
discussion!</div><div><br></div><div>I would like to say that in my =
opinion the real value of what Naeem has presented is not so much in =
pushing ARED, but in:</div><div><br></div><div>- reminding people that =
there was AQM stuff done between RED and CoDel, and some of this is =
worth looking at. Rong, we're aware of at least some of your previous =
work, and I'm personally a fan of CHOKe. However I do understand that =
CHOKe is probably not appropriate for the problem we're trying to solve =
here ... but you know, there were lots and lots of others made (e.g. =
AVQ, just to toss another name of an AQM that seemed to be well done to =
me when I read about it all these years ago)</div><div><br></div><div>- =
hopefully helping set the bar a bit higher regarding evaluations. I'm =
not saying we did enough (e.g. we didn't get to look at anything else =
than bulk TCP transfers yet), but I think we've done more than I have =
seen documented elsewhere so far (apologies if we missed a study; we =
tried to cover them in the table in our tech. =
rep.).</div><div><br></div><div><br></div><div>I just personally found =
it a bit pathetic to first see CoDel and PIE presented with comparisons =
against only themselves and RED, and then see that the =
more-than-a-decade-old-ARED very often performs better than both of =
them. It makes me wonder how many other 5-10 year old AQMs are at least =
as good as CoDel and PIE.</div><div><br></div><div><br></div><div>About =
this:</div><div><br></div><div><br></div><div><div><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); =
font-size: 16px; font-family: Calibri, sans-serif; ">
<div>RP: If low latencies are achieved at the cost of losing quite a bit =
of throughput, it is a no-starter for AQM =
design.&nbsp;</div></div></blockquote><div><br></div></div>That's rather =
obvious &nbsp;:-) &nbsp; &nbsp;but it's about the relationship between =
the two. e.g., the evaluations in the original CoDel paper essentially =
showed that CoDel gave higher throughput than RED at the expense of more =
latency. That's not exactly convincing either, given that latency =
reduction is the main goal we're pursuing now.&nbsp;And for ARED, we're =
only talking about a subset of scenarios (and perhaps not even the most =
relevant ones -&nbsp;&nbsp;e.g.&nbsp;do we really care that much about =
the throughput of one single TCP connection?). Let's not forget that in =
many scenarios that we looked at (in fact, I'd say this was the majority =
of cases) it had better throughput AND lower delay than CoDel and PIE. I =
would also venture to guess that the problem with low multiplexing cases =
could be addressed relatively easily.</div><div><br></div><div>Naeem and =
Preethi have agreed that some significant design changes would be =
necessary to make use of ARED as a new delay-based AQM. I agree, but =
mainly I'd expect these significant changes to deal with ECN (where we =
made the point that ALL AQMs should be changed in fact (*), but that's =
another story) and with WLAN scenarios, where ARED performed quite =
badly. We'll see - mainly we conclude that it's probably worth playing =
around with =
it.</div><div><br></div><div>Cheers,</div><div>Michael</div><div><br></div=
></body></html>=

--Apple-Mail=_9FE79146-3F55-4054-BED8-34A4E55E8D41--

From fred@cisco.com  Fri Nov 15 08:06:19 2013
Return-Path: <fred@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D619521F93FB for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 08:06:18 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.371
X-Spam-Level: 
X-Spam-Status: No, score=-110.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ShiWnH48eV5l for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 08:06:05 -0800 (PST)
Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) by ietfa.amsl.com (Postfix) with ESMTP id 5E6D611E8225 for <aqm@ietf.org>; Fri, 15 Nov 2013 08:03:53 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7150; q=dns/txt; s=iport; t=1384531433; x=1385741033; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=FNJxETaaxtrc7Mhzo2xacoRhf8rDv6oRB1rCehN6Ep8=; b=QgkmRs1Q4xu+5BacmDwcl+GACw0kAudN/oYrQlwJMMDy28YwYTzbEwQc dOVTk02vzaHp+t3cRwfzPKgOi+VYefJnJDg9zPv0HUZXa2q4vBhrKWEJg milzLISwcFtql9exsB2v70dAtt5v80xACaexQIawxx3UJ4358PeZLscZ5 I=;
X-Files: signature.asc : 195
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ah4FANtEhlKtJV2Y/2dsb2JhbABTBg6CeThTvyOBJxZ0giUBAQEDAQEBAWsLBQsCAQgYLiEGCyUCBA4FDodhAwkGDbc6DYlIjHOBSYEtB4MggREDkDCBMIRFgWuBL4smhTiCaT+BaAc7
X-IronPort-AV: E=Sophos;i="4.93,708,1378857600";  d="asc'?scan'208,217";a="285327548"
Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-3.cisco.com with ESMTP; 15 Nov 2013 16:03:52 +0000
Received: from xhc-aln-x03.cisco.com (xhc-aln-x03.cisco.com [173.36.12.77]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id rAFG3pTs005022 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 15 Nov 2013 16:03:51 GMT
Received: from xmb-rcd-x09.cisco.com ([169.254.9.122]) by xhc-aln-x03.cisco.com ([173.36.12.77]) with mapi id 14.03.0123.003; Fri, 15 Nov 2013 10:03:51 -0600
From: "Fred Baker (fred)" <fred@cisco.com>
To: Anoop Ghanwani <anoop@alumni.duke.edu>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
Thread-Index: AQHO4hxG1nbL2VWn50yTebvDj/kCrA==
Date: Fri, 15 Nov 2013 16:03:51 +0000
Message-ID: <AF47B543-0564-410D-9503-F73B04570C23@cisco.com>
References: <CAEjQQ5Vif73gWe-35nzbhmPH+Eh+gSZK+7xNm33+T-FVNsmPZw@mail.gmail.com> <CEA9034B.4B37C%prenatar@cisco.com> <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com> <CA+-tSzyKT9gS_12V=yyia8iPt9FiDg6=NRKrB3FLdCRDTi=6Pg@mail.gmail.com>
In-Reply-To: <CA+-tSzyKT9gS_12V=yyia8iPt9FiDg6=NRKrB3FLdCRDTi=6Pg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [10.19.64.121]
Content-Type: multipart/signed; boundary="Apple-Mail=_192F582C-A553-48D3-B336-1955CD5A8C84"; protocol="application/pgp-signature"; micalg=pgp-sha1
MIME-Version: 1.0
Cc: Preethi Natarajan <preethi.cis@gmail.com>, Naeem Khademi <naeem.khademi@gmail.com>, Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 16:06:19 -0000

--Apple-Mail=_192F582C-A553-48D3-B336-1955CD5A8C84
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_90D5279F-35B9-418A-BFFB-DB43E99EEA80"


--Apple-Mail=_90D5279F-35B9-418A-BFFB-DB43E99EEA80
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1


On Nov 13, 2013, at 2:43 PM, Anoop Ghanwani <anoop@alumni.duke.edu> =
wrote:

> On Wed, Nov 13, 2013 at 12:14 PM, Naeem Khademi =
<naeem.khademi@gmail.com> wrote:
>=20
> Agreed only in general terms -- but what would be considered as =
"packet burst" and how would it be defined? This will probably have a =
subjective answer e.g. one can argue that a size of TCP sawtooth of data =
is a burst and therefore we need a BDP of buffering for that (that's =
what had actually happened implicitly over the past decade). On the =
other hand, the definition of the "burst" may likely to correspond to =
the application generating it (e.g. video frames, IW10, etc) and =
therefore its size (and even pattern) is application/transport dependent =
somehow.=20
>=20
> There's one more case...the incast problem.  The sources themselves =
may not be "bursty" but in a high port count switch, you could have 10's =
(or more) ports sending traffic to a single output port at around the =
same time.
>=20
> Perhaps it is useful to add a description of what we mean when we say =
bursty.

Possibly, but what you just highlighted is that there are at least two =
definitions. At the Transport layer, it's not unusual for TCP to send a =
number of segments back to back - our specifications say 2-4, but Google =
seems to favor as many as ten, and I'm told that measurements suggest =
that some on the net are sending as many as 44. That would be in a =
single session. What you're discussing is what I call "lemmings"; a =
single thread in a map/reduce or multi-cache application might =
simultaneously open short sessions (either opening new TCP sessions or =
using standing TCP sessions) with hundreds or thousands of neighbors, =
each of which now sends a transport-layer burst with effects as you =
describe. That is an application-layer burst.

Bob's DCTCP suggestion could be useful in that.

I wonder if we need a separate term for the application layer behavior, =
however. Maybe "flash crowd"?

> Anoop
>=20
> Anoop=20
>=20
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

------------------------------------------------------
8 issues in virtual infrastructure
http://dcrocker.net/#fallacies


--Apple-Mail=_90D5279F-35B9-418A-BFFB-DB43E99EEA80
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=iso-8859-1

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Diso-8859-1"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On Nov 13, 2013, at 2:43 PM, Anoop Ghanwani &lt;<a =
href=3D"mailto:anoop@alumni.duke.edu">anoop@alumni.duke.edu</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Diso-8859-1"><div dir=3D"ltr">On Wed, Nov 13, 2013 at 12:14 PM, =
Naeem Khademi <span dir=3D"ltr">&lt;<a =
href=3D"mailto:naeem.khademi@gmail.com" =
target=3D"_blank">naeem.khademi@gmail.com</a>&gt;</span> wrote:<br><div =
class=3D"gmail_extra"><div class=3D"gmail_quote">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div =
dir=3D"ltr"><br><div class=3D"gmail_extra"><div =
class=3D"gmail_quote"><div>Agreed only in general terms -- but what =
would be considered as "packet burst" and how would it be defined? This =
will probably have a subjective answer e.g. one can argue that a size of =
TCP&nbsp;sawtooth of data is a burst and therefore we need a BDP of =
buffering for that (that's what had actually happened implicitly over =
the past decade). On the other hand, the definition of the "burst" may =
likely to correspond to the application generating it (e.g. video =
frames, IW10, etc) and therefore its size (and even pattern) is =
application/transport dependent somehow.&nbsp;</div>
</div></div></div></blockquote><div><br></div><div>There's one more =
case...the incast problem. &nbsp;The sources themselves may not be =
"bursty" but in a high port count switch, you could have 10's (or more) =
ports sending traffic to a single output port at around the same =
time.</div>
<div><br></div><div>Perhaps it is useful to add a description of what we =
mean when we say bursty.</div></div></div></div></blockquote><div =
dir=3D"ltr"><div class=3D"gmail_extra"><div =
class=3D"gmail_quote"><div><br></div><div>Possibly, but what you just =
highlighted is that there are at least two definitions. At the Transport =
layer, it's not unusual for TCP to send a number of segments back to =
back - our specifications say 2-4, but Google seems to favor as many as =
ten, and I'm told that measurements suggest that some on the net are =
sending as many as 44. That would be in a single session. What you're =
discussing is what I call "lemmings"; a single thread in a map/reduce or =
multi-cache application might simultaneously open short sessions (either =
opening new TCP sessions or using standing TCP sessions) with hundreds =
or thousands of neighbors, each of which now sends a transport-layer =
burst with effects as you describe. That is an application-layer =
burst.</div><div><br></div><div>Bob's DCTCP suggestion could be useful =
in that.</div><div><br></div><div>I wonder if we need a separate term =
for the application layer behavior, however. Maybe "flash =
crowd"?</div><div><br></div></div></div></div><blockquote =
type=3D"cite"><div dir=3D"ltr"><div class=3D"gmail_extra"><div =
class=3D"gmail_quote"><div>Anoop</div><div><br></div><div>Anoop&nbsp;</div=
></div><br></div></div>
_______________________________________________<br>aqm mailing =
list<br><a =
href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>https://www.ietf.org/mail=
man/listinfo/aqm<br></blockquote></div><br><div =
apple-content-edited=3D"true">
<div>------------------------------------------------------</div><div>8 =
issues in virtual infrastructure</div><div><a =
href=3D"http://dcrocker.net/#fallacies">http://dcrocker.net/#fallacies</a>=
</div>

</div>
<br></body></html>=

--Apple-Mail=_90D5279F-35B9-418A-BFFB-DB43E99EEA80--

--Apple-Mail=_192F582C-A553-48D3-B336-1955CD5A8C84
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iD8DBQFShkXmbjEdbHIsm0MRAkq3AKCMmx5X+rRPaw8Aivnb47iG8zw6cgCfZbdi
lSJMbqkxhwPgPo3tXj8bnRU=
=GAzI
-----END PGP SIGNATURE-----

--Apple-Mail=_192F582C-A553-48D3-B336-1955CD5A8C84--

From andrewmcgr@google.com  Fri Nov 15 10:05:05 2013
Return-Path: <andrewmcgr@google.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5337011E81DC for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 10:05:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.75
X-Spam-Level: 
X-Spam-Status: No, score=-1.75 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PY4qEckJ3jma for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 10:05:04 -0800 (PST)
Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) by ietfa.amsl.com (Postfix) with ESMTP id 3F95411E8164 for <aqm@ietf.org>; Fri, 15 Nov 2013 10:05:04 -0800 (PST)
Received: by mail-qa0-f52.google.com with SMTP id ii20so737578qab.18 for <aqm@ietf.org>; Fri, 15 Nov 2013 10:05:03 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dQMBFDphdI+XkkApLy4V/1Bi2QLOt5hWZET5KzTqgOA=; b=LPLZEJOHKZqoVh05JDPf+G+MRuSXRp4yRzPG3F11WpzuPwZ9n3ySambMNtJrlsTfRA /OnvXX99MKrBqc3xiAB3WPeeTgbbf5Yrf+q98PkMjUCbQvDsl7EwsmzKl5m8xrwmLbe3 pAxV2erB6eydJXQuvly1tNn6eK8O/9kymMg9CQv2KNQgYVYMM6vdo6iUkHMDqdoIqQ+/ pH8kOEpTgHN6TORgNkvYx6KoUslf7/9V9dEsYpIqHezFcJXXaIMQYr7hliPy2Ub1T+8H XQqQca3RvsZOAhCAJ9dUWNQ5iuuwn9/e9NZAW2TwtPQ7tVx4SR2LPrgBIt9EvThvbAYF yUSw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=dQMBFDphdI+XkkApLy4V/1Bi2QLOt5hWZET5KzTqgOA=; b=ObevuBRfbdjSpt0BcBFLFSQeVTLK5wEcFVTaR1BSeQTYdoyHIGksx+svci9a6+e0ci b3kHrkehj9uJ6GsPJzWV0yyz4/bMB6Gk/v5i4EMyz2h4nyAFpcfscMVRXrD7SM77RLwg qjVtAoVQtovpqAQOze3Iqxlzk28mEqQKoWikV6KF6VmzVXNJ5czR96B5RYytT2yriYhO AvlQSJ8Cko2Ly1UC6a0fNTGl0kwqrK4sPrS1OjF4sHANrDCv4yYIgM8rCSXO9lzgl8SM SejrdJZsOGEuvNRgFfyzpNZXsF1qfX/Ti3DjnSSU0WWXVOlw2tcK9r2ub8zFQgKdpUk5 SJ0A==
X-Gm-Message-State: ALoCoQk2TKgfWT3huJQNBTyhQEx0ynTlWz97JPod4sEbSrYTrIBan46H9Xc6OrqJKCOdeINDHjs/fYbdF9OFQrQVbtuX8x1ncVnhPhqlMC/QiZrPxogtENozPHOAP7ERxiB9n8kO6r9nte/HToJa5r5Nv6DLKFKghqWfNFVDsBgoV0Md3AZ5oFSBMA0MHa+VrutzoOsLg40+
MIME-Version: 1.0
X-Received: by 10.224.172.70 with SMTP id k6mr12756820qaz.100.1384538703655; Fri, 15 Nov 2013 10:05:03 -0800 (PST)
Received: by 10.224.197.198 with HTTP; Fri, 15 Nov 2013 10:05:03 -0800 (PST)
In-Reply-To: <AF47B543-0564-410D-9503-F73B04570C23@cisco.com>
References: <CAEjQQ5Vif73gWe-35nzbhmPH+Eh+gSZK+7xNm33+T-FVNsmPZw@mail.gmail.com> <CEA9034B.4B37C%prenatar@cisco.com> <CAEjQQ5XrRX=hTP9csoRJ04cK_6MAaMWA9o1Cwwqbm3RHRdBxpw@mail.gmail.com> <CA+-tSzyKT9gS_12V=yyia8iPt9FiDg6=NRKrB3FLdCRDTi=6Pg@mail.gmail.com> <AF47B543-0564-410D-9503-F73B04570C23@cisco.com>
Date: Fri, 15 Nov 2013 10:05:03 -0800
Message-ID: <CAPRuP3m1HH3B1y7V5Pbemo3Cd1AUV43M4iF=0_2ANaFzxaYeFA@mail.gmail.com>
From: Andrew Mcgregor <andrewmcgr@google.com>
To: "Fred Baker (fred)" <fred@cisco.com>
Content-Type: multipart/alternative; boundary=001a11c2462e9c099b04eb3b09e9
Cc: Michael Welzl <michawe@ifi.uio.no>, Anoop Ghanwani <anoop@alumni.duke.edu>, Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 18:05:05 -0000

--001a11c2462e9c099b04eb3b09e9
Content-Type: text/plain; charset=ISO-8859-1

TCP as deployed can be pretty bursty; TSO often results in 44 packet bursts
(later in connections, once the window has opened wide enough), which is
one reason IW10 caused very little harm.  I wouldn't say Google favours
that; as Yuchung's presentation in TCPM showed, the TCP team here is
working hard to reduce that burstiness.  However, that is deployed reality,
and will be for some years on the net at large, no matter how undesirable
it is in terms of excessive packet loss and buffer pressure.

I very much think we should be thinking about DCTCP-style ECN; that seems
more deployable and a significant benefit, unlike the ambiguous benefit of
classic ECN.


On 15 November 2013 08:03, Fred Baker (fred) <fred@cisco.com> wrote:

>
> On Nov 13, 2013, at 2:43 PM, Anoop Ghanwani <anoop@alumni.duke.edu> wrote:
>
> On Wed, Nov 13, 2013 at 12:14 PM, Naeem Khademi <naeem.khademi@gmail.com>wrote:
>
>>
>> Agreed only in general terms -- but what would be considered as "packet
>> burst" and how would it be defined? This will probably have a subjective
>> answer e.g. one can argue that a size of TCP sawtooth of data is a burst
>> and therefore we need a BDP of buffering for that (that's what had actually
>> happened implicitly over the past decade). On the other hand, the
>> definition of the "burst" may likely to correspond to the application
>> generating it (e.g. video frames, IW10, etc) and therefore its size (and
>> even pattern) is application/transport dependent somehow.
>>
>
> There's one more case...the incast problem.  The sources themselves may
> not be "bursty" but in a high port count switch, you could have 10's (or
> more) ports sending traffic to a single output port at around the same time.
>
> Perhaps it is useful to add a description of what we mean when we say
> bursty.
>
>
> Possibly, but what you just highlighted is that there are at least two
> definitions. At the Transport layer, it's not unusual for TCP to send a
> number of segments back to back - our specifications say 2-4, but Google
> seems to favor as many as ten, and I'm told that measurements suggest that
> some on the net are sending as many as 44. That would be in a single
> session. What you're discussing is what I call "lemmings"; a single thread
> in a map/reduce or multi-cache application might simultaneously open short
> sessions (either opening new TCP sessions or using standing TCP sessions)
> with hundreds or thousands of neighbors, each of which now sends a
> transport-layer burst with effects as you describe. That is an
> application-layer burst.
>
> Bob's DCTCP suggestion could be useful in that.
>
> I wonder if we need a separate term for the application layer behavior,
> however. Maybe "flash crowd"?
>
> Anoop
>
> Anoop
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>
> ------------------------------------------------------
> 8 issues in virtual infrastructure
> http://dcrocker.net/#fallacies
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>


-- 
Andrew McGregor | SRE | andrewmcgr@google.com | +61 4 8143 7128

--001a11c2462e9c099b04eb3b09e9
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">TCP as deployed can be pretty bursty; TSO often results in=
 44 packet bursts (later in connections, once the window has opened wide en=
ough), which is one reason IW10 caused very little harm. =A0I wouldn&#39;t =
say Google favours that; as Yuchung&#39;s presentation in TCPM showed, the =
TCP team here is working hard to reduce that burstiness. =A0However, that i=
s deployed reality, and will be for some years on the net at large, no matt=
er how undesirable it is in terms of excessive packet loss and buffer press=
ure.<div>
<br></div><div>I very much think we should be thinking about DCTCP-style EC=
N; that seems more deployable and a significant benefit, unlike the ambiguo=
us benefit of classic ECN.</div></div><div class=3D"gmail_extra"><br><br>
<div class=3D"gmail_quote">On 15 November 2013 08:03, Fred Baker (fred) <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:fred@cisco.com" target=3D"_blank">fred=
@cisco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word"><br><div><div><div class=3D"h5"><div>On=
 Nov 13, 2013, at 2:43 PM, Anoop Ghanwani &lt;<a href=3D"mailto:anoop@alumn=
i.duke.edu" target=3D"_blank">anoop@alumni.duke.edu</a>&gt; wrote:</div><br=
><blockquote type=3D"cite">
<div dir=3D"ltr">On Wed, Nov 13, 2013 at 12:14 PM, Naeem Khademi <span dir=
=3D"ltr">&lt;<a href=3D"mailto:naeem.khademi@gmail.com" target=3D"_blank">n=
aeem.khademi@gmail.com</a>&gt;</span> wrote:<br><div class=3D"gmail_extra">=
<div class=3D"gmail_quote">

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><br><div class=3D"gmail_ext=
ra"><div class=3D"gmail_quote"><div>Agreed only in general terms -- but wha=
t would be considered as &quot;packet burst&quot; and how would it be defin=
ed? This will probably have a subjective answer e.g. one can argue that a s=
ize of TCP=A0sawtooth of data is a burst and therefore we need a BDP of buf=
fering for that (that&#39;s what had actually happened implicitly over the =
past decade). On the other hand, the definition of the &quot;burst&quot; ma=
y likely to correspond to the application generating it (e.g. video frames,=
 IW10, etc) and therefore its size (and even pattern) is application/transp=
ort dependent somehow.=A0</div>

</div></div></div></blockquote><div><br></div><div>There&#39;s one more cas=
e...the incast problem. =A0The sources themselves may not be &quot;bursty&q=
uot; but in a high port count switch, you could have 10&#39;s (or more) por=
ts sending traffic to a single output port at around the same time.</div>

<div><br></div><div>Perhaps it is useful to add a description of what we me=
an when we say bursty.</div></div></div></div></blockquote></div></div><div=
 dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div><br=
>
</div><div>Possibly, but what you just highlighted is that there are at lea=
st two definitions. At the Transport layer, it&#39;s not unusual for TCP to=
 send a number of segments back to back - our specifications say 2-4, but G=
oogle seems to favor as many as ten, and I&#39;m told that measurements sug=
gest that some on the net are sending as many as 44. That would be in a sin=
gle session. What you&#39;re discussing is what I call &quot;lemmings&quot;=
; a single thread in a map/reduce or multi-cache application might simultan=
eously open short sessions (either opening new TCP sessions or using standi=
ng TCP sessions) with hundreds or thousands of neighbors, each of which now=
 sends a transport-layer burst with effects as you describe. That is an app=
lication-layer burst.</div>
<div><br></div><div>Bob&#39;s DCTCP suggestion could be useful in that.</di=
v><div><br></div><div>I wonder if we need a separate term for the applicati=
on layer behavior, however. Maybe &quot;flash crowd&quot;?</div><div><br>
</div></div></div></div><blockquote type=3D"cite"><div dir=3D"ltr"><div cla=
ss=3D"gmail_extra"><div class=3D"gmail_quote"><div>Anoop</div><div><br></di=
v><div>Anoop=A0</div></div><br></div></div><div class=3D"im">
_______________________________________________<br>aqm mailing list<br><a h=
ref=3D"mailto:aqm@ietf.org" target=3D"_blank">aqm@ietf.org</a><br><a href=
=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">https://ww=
w.ietf.org/mailman/listinfo/aqm</a><br>
</div></blockquote></div><br><div>
<div>------------------------------------------------------</div><div>8 iss=
ues in virtual infrastructure</div><div><a href=3D"http://dcrocker.net/#fal=
lacies" target=3D"_blank">http://dcrocker.net/#fallacies</a></div>

</div>
<br></div><br>_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><br>
<br></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div dir=
=3D"ltr"><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-siz=
e:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;borde=
r-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Andrew McGregor=A0|<=
/span><span style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:s=
mall;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-c=
olor:rgb(51,105,232);padding-top:2px;margin-top:2px">=A0SRE=A0|</span><span=
 style=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-h=
eight:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,=
153,57);padding-top:2px;margin-top:2px">=A0<a href=3D"mailto:andrewmcgr@goo=
gle.com" target=3D"_blank">andrewmcgr@google.com</a>=A0|</span><span style=
=3D"color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:=
1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,=
17);padding-top:2px;margin-top:2px">=A0+61 4 8143 7128</span><br>
</div>
</div>

--001a11c2462e9c099b04eb3b09e9--

From ropan@cisco.com  Fri Nov 15 10:20:04 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E92811E821A for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 10:20:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.371
X-Spam-Level: 
X-Spam-Status: No, score=-10.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Txwf8Xz5aKss for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 10:19:58 -0800 (PST)
Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) by ietfa.amsl.com (Postfix) with ESMTP id 0965E11E8210 for <aqm@ietf.org>; Fri, 15 Nov 2013 10:19:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=16347; q=dns/txt; s=iport; t=1384539589; x=1385749189; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=NENcwpB/Ubwyae3hF62or5KN6brwQ4u4psmZckEWlog=; b=DudaWRoygOPdFstUjD+ZPlAl9hngFn/o2nk+zW5XeChrw44mb8ejiOR2 szwY+fobp6jfyUyWIj7D8S9i5i11By05jfdcRCGx+UhGjvs1KC/t+UqSC 7h2TvegZ2YnVFLY7aWos8Pnyi7zhJqftX7ZD1NVGBdkZYrdJouPjGJ5DU A=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ah4FALBkhlKtJXHB/2dsb2JhbABZgkNEgQu/KoEqFnSCJQECBHIHEgEIDgMDAQIoKBEUCQgCBAoEBRSHWwMPt1ANiUiMc4JlEQeEMQOWJYFrjFWFOIMogio
X-IronPort-AV: E=Sophos;i="4.93,709,1378857600";  d="scan'208,217";a="285393964"
Received: from rcdn-core2-6.cisco.com ([173.37.113.193]) by rcdn-iport-3.cisco.com with ESMTP; 15 Nov 2013 18:19:48 +0000
Received: from xhc-rcd-x02.cisco.com (xhc-rcd-x02.cisco.com [173.37.183.76]) by rcdn-core2-6.cisco.com (8.14.5/8.14.5) with ESMTP id rAFIJmbZ009359 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 15 Nov 2013 18:19:48 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.147]) by xhc-rcd-x02.cisco.com ([173.37.183.76]) with mapi id 14.03.0123.003; Fri, 15 Nov 2013 12:19:47 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: Michael Welzl <michawe@ifi.uio.no>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
Thread-Index: AQHO3CzV4Y8AQmoIJE6ksgN55dTkopobOREAgADcp4CAACSPAP//9emAgABSr4CABi75AIAA200AgABagICAAB6IAIAAVMEAgADNWwCAACSHAP//sJoAgACnbQD//4M6gAArHEgA///qSYCAAAikgA==
Date: Fri, 15 Nov 2013 18:19:47 +0000
Message-ID: <CEABA46C.55B4B%ropan@cisco.com>
In-Reply-To: <CEAB91A5.55AD3%ropan@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [128.107.165.143]
Content-Type: multipart/alternative; boundary="_000_CEABA46C55B4Bropanciscocom_"
MIME-Version: 1.0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 18:20:04 -0000

--_000_CEABA46C55B4Bropanciscocom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

Please see inline=85

Thanks,

Rong


From: Michael Welzl <michawe@ifi.uio.no<mailto:michawe@ifi.uio.no>>
Date: Friday, November 15, 2013 3:06 AM
To: Cisco Employee <ropan@cisco.com<mailto:ropan@cisco.com>>
Cc: Naeem Khademi <naeem.khademi@gmail.com<mailto:naeem.khademi@gmail.com>>=
, Preethi Natarajan <preethi.cis@gmail.com<mailto:preethi.cis@gmail.com>>, =
"aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based

Hi,

Interesting discussion!

I would like to say that in my opinion the real value of what Naeem has pre=
sented is not so much in pushing ARED, but in:

- reminding people that there was AQM stuff done between RED and CoDel, and=
 some of this is worth looking at. Rong, we're aware of at least some of yo=
ur previous work, and I'm personally a fan of CHOKe. However I do understan=
d that CHOKe is probably not appropriate for the problem we're trying to so=
lve here ... but you know, there were lots and lots of others made (e.g. AV=
Q, just to toss another name of an AQM that seemed to be well done to me wh=
en I read about it all these years ago)

------------------------------------------
>>>>>>>RP: Thank you! The comment about throwing away all experiences sound=
ed ignorant.  I am aware of AVQ, and I am a fan of virtual queue concept (b=
rought out by Frank Kelly ) in general. PCN that Bob is pushing is along th=
ese lines of work. I wish the industry as a whole can embrace the concept.
------------------------------------------


- hopefully helping set the bar a bit higher regarding evaluations. I'm not=
 saying we did enough (e.g. we didn't get to look at anything else than bul=
k TCP transfers yet), but I think we've done more than I have seen document=
ed elsewhere so far (apologies if we missed a study; we tried to cover them=
 in the table in our tech. rep.).


I just personally found it a bit pathetic to first see CoDel and PIE presen=
ted with comparisons against only themselves and RED, and then see that the=
 more-than-a-decade-old-ARED very often performs better than both of them. =
It makes me wonder how many other 5-10 year old AQMs are at least as good a=
s CoDel and PIE.

------------------------------------------
>>>>>>>>RP:  RED is the only one out there that got wide adoptions. Compare=
d with the state of the art in products (not in papers) is the idea. But th=
at does not mean we don't know/learn from other AQM schemes. As matter of f=
act, PI controller and AVQ came out about the same time. PIE also adopts th=
e PI controller design. From my years of experience, I think PI controller =
is a better controller in this space as queue and rate lend themselves to b=
e the proportional and integral pair.

>>>>>>>>RP: I disagree with the notion that ARED often performs better. If =
tight delay control comes from narrow-band bang-bang control, the tail drop=
 with shallow buffer would behave similarly. For example, if target_delay =
=3D 1ms, min_th =3D 0.5ms and max_th =3D 1.5ms, the queue really allows one=
 packet in the queue. I don't see how this will be different from tail drop=
.  Your throughput plot showed 25% throughput loss. This is a case consider=
ed good?
-----------------------------------------







About this:


RP: If low latencies are achieved at the cost of losing quite a bit of thro=
ughput, it is a no-starter for AQM design.

That's rather obvious  :-)    but it's about the relationship between the t=
wo. e.g., the evaluations in the original CoDel paper essentially showed th=
at CoDel gave higher throughput than RED at the expense of more latency. Th=
at's not exactly convincing either, given that latency reduction is the mai=
n goal we're pursuing now. And for ARED, we're only talking about a subset =
of scenarios (and perhaps not even the most relevant ones -  e.g. do we rea=
lly care that much about the throughput of one single TCP connection?). Let=
's not forget that in many scenarios that we looked at (in fact, I'd say th=
is was the majority of cases) it had better throughput AND lower delay than=
 CoDel and PIE. I would also venture to guess that the problem with low mul=
tiplexing cases could be addressed relatively easily.

------------------------------------------
>>>>>>>>RP:  Again, the tight delay control comes from narrow-band bang-ban=
g control with hard drops. The scenarios that throughputs are good are in h=
igher congestion scenarios where there are enough packets showing up to fil=
l the pipe. That is not the effectiveness of AQM. Tail drop with shallow bu=
ffer will have high throughput in those cases too. Try a scenario when link=
 is congested and a burst comes, see how many packets of that burst can go =
through.
------------------------------------------




Naeem and Preethi have agreed that some significant design changes would be=
 necessary to make use of ARED as a new delay-based AQM. I agree, but mainl=
y I'd expect these significant changes to deal with ECN (where we made the =
point that ALL AQMs should be changed in fact (*), but that's another story=
) and with WLAN scenarios, where ARED performed quite badly. We'll see - ma=
inly we conclude that it's probably worth playing around with it.


---------------------------------------------
>>>>>>>>RP:  Sure. The significant design changes to ARED won't be just for=
 ECN, but to address the core issue of how effective ARED is as an AQM sche=
me. Don't get us wrong -- we would be happy to see these improvements on AR=
ED.  Just a side note and you seemed to know what happened in the early 200=
0 time frame.  Sally's ARED paper did not eventually publish because there =
was a consensus in the community that control analysis is essential in AQM =
(other papers like AVQ and PI do have control analysis). It would be great =
if your group can add in this space too, especially auto-tuning needs deepe=
r understanding of the knobs involved.

Best Regards,

Rong
-------------------------------------------


Cheers,
Michael


--_000_CEABA46C55B4Bropanciscocom_
Content-Type: text/html; charset="Windows-1252"
Content-ID: <3A064D9F4A0C9141B37DA6A96F04440F@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1=
252">
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-fami=
ly: Calibri, sans-serif; ">
<div>Please see inline=85</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Rong</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-famil=
y: Calibri, sans-serif; ">
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div style=3D"font-family:Calibri; font-size:14pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>Michael Welzl &lt;<a href=3D"=
mailto:michawe@ifi.uio.no">michawe@ifi.uio.no</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Friday, November 15, 2013 3:0=
6 AM<br>
<span style=3D"font-weight:bold">To: </span>Cisco Employee &lt;<a href=3D"m=
ailto:ropan@cisco.com">ropan@cisco.com</a>&gt;<br>
<span style=3D"font-weight:bold">Cc: </span>Naeem Khademi &lt;<a href=3D"ma=
ilto:naeem.khademi@gmail.com">naeem.khademi@gmail.com</a>&gt;, Preethi Nata=
rajan &lt;<a href=3D"mailto:preethi.cis@gmail.com">preethi.cis@gmail.com</a=
>&gt;, &quot;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&quot;
 &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span>Re: [aqm] AQM schemes: Que=
ue length vs. delay based<br>
</div>
<div><br>
</div>
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
Hi,
<div><br>
</div>
<div>Interesting discussion!</div>
<div><br>
</div>
<div>I would like to say that in my opinion the real value of what Naeem ha=
s presented is not so much in pushing ARED, but in:</div>
<div><br>
</div>
<div>- reminding people that there was AQM stuff done between RED and CoDel=
, and some of this is worth looking at. Rong, we're aware of at least some =
of your previous work, and I'm personally a fan of CHOKe. However I do unde=
rstand that CHOKe is probably not
 appropriate for the problem we're trying to solve here ... but you know, t=
here were lots and lots of others made (e.g. AVQ, just to toss another name=
 of an AQM that seemed to be well done to me when I read about it all these=
 years ago)</div>
</div>
</div>
</span>
<div><br>
</div>
<div>------------------------------------------</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: Thank you! The comment about throwing =
away all experiences sounded ignorant. &nbsp;I am aware of AVQ, and I am a =
fan of virtual queue concept (brought out by Frank Kelly ) in general. PCN =
that Bob is pushing is along these lines of work. I wish the
 industry as a whole can embrace the concept.</div>
<div>------------------------------------------</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<div><br>
</div>
<div>- hopefully helping set the bar a bit higher regarding evaluations. I'=
m not saying we did enough (e.g. we didn't get to look at anything else tha=
n bulk TCP transfers yet), but I think we've done more than I have seen doc=
umented elsewhere so far (apologies
 if we missed a study; we tried to cover them in the table in our tech. rep=
.).</div>
<div><br>
</div>
<div><br>
</div>
<div>I just personally found it a bit pathetic to first see CoDel and PIE p=
resented with comparisons against only themselves and RED, and then see tha=
t the more-than-a-decade-old-ARED very often performs better than both of t=
hem. It makes me wonder how many
 other 5-10 year old AQMs are at least as good as CoDel and PIE.</div>
</div>
</div>
</span>
<div><br>
</div>
<div>------------------------------------------</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: &nbsp;RED is the only one out ther=
e that got wide adoptions. Compared with the state of the art in products (=
not in papers) is the idea. But that does not mean we don't know/learn from=
 other AQM schemes. As matter of fact, PI controller and AVQ
 came out about the same time. PIE also adopts the PI controller design. Fr=
om my years of experience, I think PI controller is a better controller in =
this space as queue and rate lend themselves to be the proportional and int=
egral pair.</div>
<div><br>
</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: I disagree with the notion that AR=
ED often performs better. If tight delay control comes from narrow-band ban=
g-bang control, the tail drop with shallow buffer would behave similarly. F=
or example, if target_delay =3D 1ms, min_th =3D 0.5ms and max_th
 =3D 1.5ms, the queue really allows one packet in the queue. I don't see ho=
w this will be different from tail drop. &nbsp;Your throughput plot showed =
25% throughput loss. This is a case considered good?</div>
<div>-----------------------------------------</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<div><br>
</div>
<div><br>
</div>
<div>About this:</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>
<blockquote type=3D"cite">
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-famil=
y: Calibri, sans-serif; ">
<div>RP: If low latencies are achieved at the cost of losing quite a bit of=
 throughput, it is a no-starter for AQM design.&nbsp;</div>
</div>
</blockquote>
<div><br>
</div>
</div>
That's rather obvious &nbsp;:-) &nbsp; &nbsp;but it's about the relationshi=
p between the two. e.g., the evaluations in the original CoDel paper essent=
ially showed that CoDel gave higher throughput than RED at the expense of m=
ore latency. That's not exactly convincing either,
 given that latency reduction is the main goal we're pursuing now.&nbsp;And=
 for ARED, we're only talking about a subset of scenarios (and perhaps not =
even the most relevant ones -&nbsp;&nbsp;e.g.&nbsp;do we really care that m=
uch about the throughput of one single TCP connection?).
 Let's not forget that in many scenarios that we looked at (in fact, I'd sa=
y this was the majority of cases) it had better throughput AND lower delay =
than CoDel and PIE. I would also venture to guess that the problem with low=
 multiplexing cases could be addressed
 relatively easily.</div>
</div>
</div>
</span>
<div><br>
</div>
<div>
<div>------------------------------------------</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: &nbsp;Again, the tight delay contr=
ol comes from narrow-band bang-bang control with hard drops. The scenarios =
that throughputs are good are in higher congestion scenarios where there ar=
e enough packets showing up to fill the pipe. That is not the
 effectiveness of AQM. Tail drop with shallow buffer will have high through=
put in those cases too. Try a scenario when link is congested and a burst c=
omes, see how many packets of that burst can go through.</div>
</div>
<div>------------------------------------------</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<div><br>
</div>
<div>Naeem and Preethi have agreed that some significant design changes wou=
ld be necessary to make use of ARED as a new delay-based AQM. I agree, but =
mainly I'd expect these significant changes to deal with ECN (where we made=
 the point that ALL AQMs should
 be changed in fact (*), but that's another story) and with WLAN scenarios,=
 where ARED performed quite badly. We'll see - mainly we conclude that it's=
 probably worth playing around with it.</div>
</div>
</div>
</span>
<div><br>
</div>
<div><br>
</div>
<div>
<div>---------------------------------------------</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: &nbsp;Sure.&nbsp;<span class=3D"Ap=
ple-style-span" style=3D"font-family: Calibri; ">The significant design cha=
nges to ARED won't be just for ECN, but to address the core issue of how ef=
fective ARED is as an AQM scheme. Don't get us wrong -- we would be happy
 to see these improvements on ARED. &nbsp;Just a side note and you seemed t=
o know what happened in the early 2000 time frame.&nbsp;&nbsp;Sally's ARED =
paper did not eventually publish because there was a consensus in the commu=
nity that control analysis is essential in AQM
 (other papers like AVQ and PI do have control analysis). It would be great=
 if your group can add in this space too, especially auto-tuning needs deep=
er understanding of the knobs involved.&nbsp;</span></div>
</div>
<div><br>
</div>
<div>Best Regards,</div>
<div><br>
</div>
<div>Rong</div>
<div>-------------------------------------------</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<div><br>
</div>
<div>Cheers,</div>
<div>Michael</div>
<div><br>
</div>
</div>
</div>
</span></div>
</div>
</span>
</body>
</html>

--_000_CEABA46C55B4Bropanciscocom_--

From michawe@ifi.uio.no  Fri Nov 15 11:36:40 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0107E21F9B4B for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 11:36:13 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.371
X-Spam-Level: 
X-Spam-Status: No, score=-102.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xws7H4lccQg3 for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 11:36:07 -0800 (PST)
Received: from mail-out4.uio.no (mail-out4.uio.no [IPv6:2001:700:100:10::15]) by ietfa.amsl.com (Postfix) with ESMTP id 0267711E821B for <aqm@ietf.org>; Fri, 15 Nov 2013 11:35:58 -0800 (PST)
Received: from mail-mx2.uio.no ([129.240.10.30]) by mail-out4.uio.no with esmtp (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1VhPBH-0007iv-Cf; Fri, 15 Nov 2013 20:35:51 +0100
Received: from 59.115.34.95.customer.cdi.no ([95.34.115.59] helo=[192.168.0.114]) by mail-mx2.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from <michawe@ifi.uio.no>) id 1VhPBG-0004SH-LA; Fri, 15 Nov 2013 20:35:51 +0100
Content-Type: multipart/alternative; boundary="Apple-Mail=_830F8EC7-9A39-43D1-B3A6-1CC17291F9A5"
Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\))
From: Michael Welzl <michawe@ifi.uio.no>
In-Reply-To: <CEABA46C.55B4B%ropan@cisco.com>
Date: Fri, 15 Nov 2013 20:35:49 +0100
Message-Id: <19813CF2-4C78-43B8-9B75-5A4E605BE54A@ifi.uio.no>
References: <CEABA46C.55B4B%ropan@cisco.com>
To: "Rong Pan (ropan)" <ropan@cisco.com>
X-Mailer: Apple Mail (2.1510)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 4 msgs/h 1 sum rcpts/h 9 sum msgs/h 3 total rcpts 9863 max rcpts/h 44 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: A2BDF4D3525B69F999DE11E7AF2C7CCC1B1AEF6D
X-UiO-SPAM-Test: remote_host: 95.34.115.59 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 485 max/h 13 blacklist 0 greylist 0 ratelimit 0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 19:36:44 -0000

--Apple-Mail=_830F8EC7-9A39-43D1-B3A6-1CC17291F9A5
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

Hi,

In line, but snipping some things away:


> [About evaluating against other-than-RED AQMs]
> ------------------------------------------
> >>>>>>>>RP:  RED is the only one out there that got wide adoptions. =
Compared with the state of the art in products (not in papers) is the =
idea. But that does not mean we don't know/learn from other AQM schemes. =
As matter of fact, PI controller and AVQ came out about the same time. =
PIE also adopts the PI controller design. =46rom my years of experience, =
I think PI controller is a better controller in this space as queue and =
rate lend themselves to be the proportional and integral pair.

Ok, fair point about RED, and I agree that a PI controller can do well. =
The "SoA in products" bit makes me wonder - I never really knew, but =
since it's in principle easy to deploy an AQM scheme (you design it, you =
like it, you put it in your product?), I always assumed that RED =
wouldn't be the only one out there.

Now from this and other recent discussions I get the impression that RED =
*is* the only one that was ever turned on, and maybe the only one =
shipped? Do you know, is that the case? Just curious...  I guess that =
some products would sell with a strange secret AQM scheme that works =
very well, or not? That was just my imagination.


> >>>>>>>>RP: I disagree with the notion that ARED often performs =
better. If tight delay control comes from narrow-band bang-bang control, =
the tail drop with shallow buffer would behave similarly. For example, =
if target_delay =3D 1ms, min_th =3D 0.5ms and max_th =3D 1.5ms, the =
queue really allows one packet in the queue. I don't see how this will =
be different from tail drop.  Your throughput plot showed 25% throughput =
loss. This is a case considered good?

By "often" I mean pretty much all cases except the very lightly loaded =
or extremely low target delay cases that you keep referring to.


> ------------------------------------------
> >>>>>>>>RP:  Again, the tight delay control comes from narrow-band =
bang-bang control with hard drops. The scenarios that throughputs are =
good are in higher congestion scenarios where there are enough packets =
showing up to fill the pipe. That is not the effectiveness of AQM. Tail =
drop with shallow buffer will have high throughput in those cases too. =
Try a scenario when link is congested and a burst comes, see how many =
packets of that burst can go through.
> ------------------------------------------

Maybe we disagree about the goals... why would you want to allow a burst =
on an already congested link?


> Naeem and Preethi have agreed that some significant design changes =
would be necessary to make use of ARED as a new delay-based AQM. I =
agree, but mainly I'd expect these significant changes to deal with ECN =
(where we made the point that ALL AQMs should be changed in fact (*), =
but that's another story) and with WLAN scenarios, where ARED performed =
quite badly. We'll see - mainly we conclude that it's probably worth =
playing around with it.
>=20
>=20
> ---------------------------------------------
> >>>>>>>>RP:  Sure. The significant design changes to ARED won't be =
just for ECN, but to address the core issue of how effective ARED is as =
an AQM scheme. Don't get us wrong -- we would be happy to see these =
improvements on ARED.  Just a side note and you seemed to know what =
happened in the early 2000 time frame.  Sally's ARED paper did not =
eventually publish because there was a consensus in the community that =
control analysis is essential in AQM (other papers like AVQ and PI do =
have control analysis). It would be great if your group can add in this =
space too, especially auto-tuning needs deeper understanding of the =
knobs involved.=20

Well, sounds like your opinion about that consensus; no doubt that =
analysis makes a paper stronger, but also no point in speculating about =
the reasons why she never got it published.

Getting back to the matter at hand, call me naive, but it seems to me =
that ARED was doing just the right thing in mildly loaded to overloaded =
scenarios, i.e. "bang bang" control may be just the right thing to do =
then? On that basis, isn't it a matter of just detecting the lightly =
loaded scenarios and becoming less aggressive under these circumstances? =
I suppose that the control could be tuned to do that. But this is just =
speculative...

Cheers,
Michael


--Apple-Mail=_830F8EC7-9A39-43D1-B3A6-1CC17291F9A5
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
">Hi,<div><br></div><div>In line, but snipping some things =
away:</div><div><div><br></div><div><br><blockquote type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 16px; font-family: =
Calibri, sans-serif; "><span id=3D"OLK_SRC_BODY_SECTION"><div><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 16px; font-family: =
Calibri, sans-serif; "><div>[About evaluating against other-than-RED =
AQMs]
</div>
<div>------------------------------------------</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: &nbsp;RED is the only one out =
there that got wide adoptions. Compared with the state of the art in =
products (not in papers) is the idea. But that does not mean we don't =
know/learn from other AQM schemes. As matter of fact, PI controller and =
AVQ
 came out about the same time. PIE also adopts the PI controller design. =
=46rom my years of experience, I think PI controller is a better =
controller in this space as queue and rate lend themselves to be the =
proportional and integral =
pair.</div></div></div></span></div></blockquote><div><br></div>Ok, fair =
point about RED, and I agree that a PI controller can do well. The "SoA =
in products" bit makes me wonder - I never really knew, but since it's =
in principle easy to deploy an AQM scheme (you design it, you like it, =
you put it in your product?), I always assumed that RED wouldn't be the =
only one out there.</div><div><br></div><div>Now from this and other =
recent discussions I get the impression that RED *is* the only one that =
was ever turned on, and maybe the only one shipped? Do you know, is that =
the case? Just curious... &nbsp;I guess that some products would sell =
with a strange secret AQM scheme that works very well, or not? That was =
just my imagination.</div><div><br></div><div><br><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; font-size: 16px; =
font-family: Calibri, sans-serif; "><span =
id=3D"OLK_SRC_BODY_SECTION"><div><div style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
font-size: 16px; font-family: Calibri, sans-serif; ">

<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: I disagree with the notion that =
ARED often performs better. If tight delay control comes from =
narrow-band bang-bang control, the tail drop with shallow buffer would =
behave similarly. For example, if target_delay =3D 1ms, min_th =3D 0.5ms =
and max_th
 =3D 1.5ms, the queue really allows one packet in the queue. I don't see =
how this will be different from tail drop. &nbsp;Your throughput plot =
showed 25% throughput loss. This is a case considered =
good?</div></div></div></span></div></blockquote><div><br></div><div>By =
"often" I mean pretty much all cases except the very lightly loaded or =
extremely low target delay cases that you keep referring =
to.</div><div><br></div><div><br></div><blockquote type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 16px; font-family: =
Calibri, sans-serif; "><span id=3D"OLK_SRC_BODY_SECTION"><div><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 16px; font-family: =
Calibri, sans-serif; =
"><div>------------------------------------------</div><div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: &nbsp;Again, the tight delay =
control comes from narrow-band bang-bang control with hard drops. The =
scenarios that throughputs are good are in higher congestion scenarios =
where there are enough packets showing up to fill the pipe. That is not =
the
 effectiveness of AQM. Tail drop with shallow buffer will have high =
throughput in those cases too. Try a scenario when link is congested and =
a burst comes, see how many packets of that burst can go through.</div>
</div>
=
<div>------------------------------------------</div></div></div></span></=
div></blockquote><div><br></div>Maybe we disagree about the goals... why =
would you want to allow a burst on an already congested =
link?</div><div><br></div><div><br><blockquote type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 16px; font-family: =
Calibri, sans-serif; "><span id=3D"OLK_SRC_BODY_SECTION"><div><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; font-size: 16px; font-family: =
Calibri, sans-serif; ">
<span id=3D"OLK_SRC_BODY_SECTION"><div><div style=3D"word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space; ">
<div>Naeem and Preethi have agreed that some significant design changes =
would be necessary to make use of ARED as a new delay-based AQM. I =
agree, but mainly I'd expect these significant changes to deal with ECN =
(where we made the point that ALL AQMs should
 be changed in fact (*), but that's another story) and with WLAN =
scenarios, where ARED performed quite badly. We'll see - mainly we =
conclude that it's probably worth playing around with it.</div>
</div>
</div>
</span>
<div><br>
</div>
<div><br>
</div>
<div>
<div>---------------------------------------------</div>
<div>&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;RP: &nbsp;Sure.&nbsp;<span =
class=3D"Apple-style-span" style=3D"font-family: Calibri; ">The =
significant design changes to ARED won't be just for ECN, but to address =
the core issue of how effective ARED is as an AQM scheme. Don't get us =
wrong -- we would be happy
 to see these improvements on ARED. &nbsp;Just a side note and you =
seemed to know what happened in the early 2000 time =
frame.&nbsp;&nbsp;Sally's ARED paper did not eventually publish because =
there was a consensus in the community that control analysis is =
essential in AQM
 (other papers like AVQ and PI do have control analysis). It would be =
great if your group can add in this space too, especially auto-tuning =
needs deeper understanding of the knobs =
involved.&nbsp;</span></div></div></div></div></span></div></blockquote><d=
iv><br></div><div>Well, sounds like your opinion about that consensus; =
no doubt that analysis makes a paper stronger, but also no point in =
speculating about the reasons why she never got it =
published.</div><div><br></div><div>Getting back to the matter at hand, =
call me naive, but it seems to me that ARED was doing just the right =
thing in mildly loaded to overloaded scenarios, i.e. "bang bang" control =
may be just the right thing to do then? On that basis, isn't it a matter =
of just detecting the lightly loaded scenarios and becoming less =
aggressive under these circumstances? I suppose that the control could =
be tuned to do that. But this is just =
speculative...</div><div><br></div><div>Cheers,</div><div>Michael</div><di=
v><br></div></div></div></body></html>=

--Apple-Mail=_830F8EC7-9A39-43D1-B3A6-1CC17291F9A5--

From dave.taht@gmail.com  Fri Nov 15 12:00:52 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4D8BC11E8258 for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 12:00:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.373
X-Spam-Level: 
X-Spam-Status: No, score=-2.373 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R5-xibBwFkCw for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 12:00:51 -0800 (PST)
Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) by ietfa.amsl.com (Postfix) with ESMTP id 6372111E8226 for <aqm@ietf.org>; Fri, 15 Nov 2013 11:59:38 -0800 (PST)
Received: by mail-we0-f171.google.com with SMTP id t61so3972756wes.2 for <aqm@ietf.org>; Fri, 15 Nov 2013 11:59:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wzdeD/dyiaf2HSyhkX6EmVM/dGjEL2Kh+cjvUdVEYd4=; b=qDP7bYefpdp8N2YTlSKhPPrmJ5RqEmTD3ZUKYI8wyIiOl5/xPMe6X0qhknm3CPykrk U6t0cXVYqK2Zeq1bXarvpb6VQoQgJQd6QXdxUWwDxqRJR2TvR27Dk84O4CVdJsrFlp6M qVoi1nkmUg3RqVv6lNkNeiKcH0DVaKN7vyQjhdGUtrZJCV72lx1XVwQg9GbD+aY0ecLj n8u28KlrIuss8WCCvggYMO1NRUf21eoqcPamTyuqQq9C4w8qLVGPRC1sHrm3O7kaYyTX drXrxUi726JQKwJkntU2NV+MysNS9uVoSBknBLpchS+r9R8ZRVe5jcoFl35PfzHdMutF JQKQ==
MIME-Version: 1.0
X-Received: by 10.194.104.42 with SMTP id gb10mr8130054wjb.16.1384545577485; Fri, 15 Nov 2013 11:59:37 -0800 (PST)
Received: by 10.217.51.5 with HTTP; Fri, 15 Nov 2013 11:59:37 -0800 (PST)
In-Reply-To: <CEABA46C.55B4B%ropan@cisco.com>
References: <CEAB91A5.55AD3%ropan@cisco.com> <CEABA46C.55B4B%ropan@cisco.com>
Date: Fri, 15 Nov 2013 11:59:37 -0800
Message-ID: <CAA93jw6hDEwqsx8jKTAJOjbY8dV_nu6ntvmBX1jpTwyEN87G8w@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: "Rong Pan (ropan)" <ropan@cisco.com>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Preethi Natarajan <preethi.cis@gmail.com>, Naeem Khademi <naeem.khademi@gmail.com>, Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 20:00:52 -0000

I have been doing my best to ignore this thread.

Gripe #1:

What I saw of the ARED presentation seemed to show that if you
sacrificed quite a bit of throughput you'd get vastly better latency,
on a very simple set of benchmarks. I can go through each slide to see
where there were results that were in obvious.

The goal in this exercise is to aim for 100% utilization at 0 latency.

#2)

I also find box plots while often informative to be misleading when
thinking about network dynamics, where you care more about the
patterns of the outliers. At some point I suspect I'll go back to
using them on appropriate benchmarks but I generally prefer raw graphs
of the data and cdf plots of the summary data.

#3) The bufferbloat effort has been in full swing for 3+ years now.
Unlike previous efforts we didn't stop for publication, patents, etc,
but published results as fast as we could on the bloat list, and
cycled code into ns2 and the linux kernel as fast as possible. In
nearly cases there were negative results, which are hard to publish
anyway. I apologize for having not left a academic paper trail that
could be followed, but as there was no funding for the effort
whatsoever and very little benefit to me as to academic publishing, we
skipped it. Secondly, as fast as I found what seemed to be a problem,
or an interesting result worth writing about, Eric Dumazet and other
members of the netdev group of linux viewed it as a bug, and fixed it
- to the point where the linux stack looks nothing like it did when it
started. I would like it very much if others merely reported their
interesting results and treated them as bugs to be fixed rather than
papers to be written.

I hope to document some of this stuff in an upcoming fq_codel rfc. And
I'd certainly love to make CCR, ACM queue (again) and lwn, but we're
at the point where code is being seriously deployed and I'm mostly
engaged in fixing wifi right now.

Some history for the new:

I started with wonder shaper, and wondering why it didn't scale up,
and how to address it's flaws.

A big focus was how to fix variable bandwidth systems like cable,
wireless and wifi, where fixed rates were inadequate. Secondary was
finding a fixed rate system that would work against things like DSL.
We figured if we fixed the first, the second would be easy.

I'd tried RED back in the early 00s and wondershaper blew it away...
Moved from there to SFB, which was fixed and pushed into the linux
kernel first of all the ideas we had. It didn't work worth a damn.
Kathie and Van of course had RED at their disposal, and thinking about
various replacements like RED light and what became codel. In trying
to match up a multiplicity of RED related results, I noticed that RED
as implemented in linux behaved nothing like RED in the papers, and
eric dumazet found and fixed the two bugs in it that had existed for
several years (I think the broken period was from 2.6.36 through 3.4).

I went through I think nearly every paper published in the 90s and 00s
that was available online. There was a big survey of every RED variant
in there that as I recall had FRED in it and a few others that I
discarded as not useful.

Stephen hemminger did a couple qdiscs, like choke, which also didn't
work as well as advertised. I remember that kathie looked into AVQ
among others.

I spent a lot of time combining qdiscs, notably DRR or QFQ + a drop
system of some kind, like RED, choke, etc. This is partially
documented in the debloat script. Around august of the first year I'd
concluded the biggest win would be that combination but failed to
pursuade many.

Eric spent an enormous amount of time improving SFQ based on ideas
that Paul Mckenney had discarded as "too hard" back in the days of the
68020. The big version of that paper was pretty crucial to
understanding what we could achieve on modern hardware. Paul has been
a great help...

Along the way he got hot on ARED, for about two weeks, and implemented
that, from which I got generally lousy results in isolation. SFQRED
worked pretty good in a range of bandwidths between 4Mbit and 65Mbit
but proved hard to tune. Naheem rightly points out that SFQARED
variant was rather inadaquately tested (as about that time codel was
beginning to emerge) - but as I pointed out months ago, the code is
readily available in the linux kernel and GO FOR IT.

We had an idea ("hoq" or head of queue) in SFQRED that was very
SQF-like (as it turned out) but didn't work correctly, so we ended up
dropping it from linux 3.4

Andrew Mcgregor and I spent many a late night discussing among other
things the mixed-rtt problem and testing out all these algorithms
live, over video conferencing, and other systems. JG as well. There
were hundreds of people on the bloat list and dozens of cerowrt users
helping as well, I'm sorry for not listing names here and I'm sure
I've forgotten someone.

Kathie briefed us on codel one day in may, I got very excited, I wrote
the first  version for linux, eric dumazet then fixed it, sped it up
by 100x, an excited team assembled (see the early days of the codel
list) found a bug in the paper before it was even published, and fixed
that, one week later it was in the kernel and then one saturday
afternoon after we'd caught up on sleep, eric combined DRR and codel
to create fq_codel, and also got that slammed into the linux kernel in
under a week.

... I tried it on a wide range of benchmarks and said, at about 4AM
that day, that "wow. We just fixed the internet."

PIE took the fundamental insights of codel and created something
simpler from a straight aqm standpoint, and it is easier to implement
in *some* hardware... but doesn't interact with a packet scheduler
anywhere near as well as fq_codel.

I have been working ever since to help improve pie so that those forms
of hardware can have some form of AQM, and so that the linux
implementation is accurate and can be directly compared to all the
other packet schedulers and aqm systems also in there.

And there we stand today. If you want more papers published I have
hundreds of thousands of rrul and related plots, other harder to
express experiments, and will gladly co-author something or
participate in more experiments that seem sane in order to achieve
publication.

The ONLY paper I managed to get published was this one

http://www.enst.fr/ drossi/paper/rossi13tma-b.pdf

and all we were doing was pointing out that - if we won as big as we
expected with some form of aqm or packet scheduling - we'd at a single
stroke obsolete the field of low priority congestion control, with
some ideas towards making it work again...

Dario Rossi has since continued to analyze RED vs ledbat...

anyway...

I would not mind at all if others started with wondershaper and worked
up. I am VERY big on repeatable results, and repeatable experiments.
I'd love to see the presentation given at ietf against sfq, for
example. I'd like sources for the new DCTCP ECN results, too....

From ropan@cisco.com  Fri Nov 15 14:02:53 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 611F911E8105 for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 14:02:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.371
X-Spam-Level: 
X-Spam-Status: No, score=-10.371 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OwSQapnbi9Y1 for <aqm@ietfa.amsl.com>; Fri, 15 Nov 2013 14:02:48 -0800 (PST)
Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id 03E5F11E80DC for <aqm@ietf.org>; Fri, 15 Nov 2013 14:02:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2567; q=dns/txt; s=iport; t=1384552968; x=1385762568; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=SyIF17j1I+SNyzUzXB/zorGCznxxydWxHexvgDI4jaM=; b=iX7474uWj1GCSiXx7Cr/on/+JKgBTaHeKFZRPUCQVzids0tBJLu4tQJW LUoiPYvSJw78hvV+LE4Ack2f+MZ5L6NDHkoYE/g0j6hjS46JWcS8kSJP9 9rrfa8kRRjE3rOY9/zfa9enACvgio+H4edpRbqfvLAMLT/G7gktWBX6H8 k=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ah4FADaZhlKtJXG+/2dsb2JhbABZgkNEOFO/LIEqFnSCKgJ5EgEIBAEJaiUCBA6IBg3BGI9pB4QxA5gQkg2DKIIq
X-IronPort-AV: E=Sophos;i="4.93,710,1378857600";  d="scan'208,217";a="285453636"
Received: from rcdn-core2-3.cisco.com ([173.37.113.190]) by rcdn-iport-7.cisco.com with ESMTP; 15 Nov 2013 22:02:47 +0000
Received: from xhc-rcd-x05.cisco.com (xhc-rcd-x05.cisco.com [173.37.183.79]) by rcdn-core2-3.cisco.com (8.14.5/8.14.5) with ESMTP id rAFM2l0B022582 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 15 Nov 2013 22:02:47 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.147]) by xhc-rcd-x05.cisco.com ([173.37.183.79]) with mapi id 14.03.0123.003; Fri, 15 Nov 2013 16:02:47 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: Michael Welzl <michawe@ifi.uio.no>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
Thread-Index: AQHO3CzV4Y8AQmoIJE6ksgN55dTkopobOREAgADcp4CAACSPAP//9emAgABSr4CABi75AIAA200AgABagICAAB6IAIAAVMEAgADNWwCAACSHAP//sJoAgACnbQD//4M6gAArHEgA///qSYCAAAikgIAArCGA//+SLoA=
Date: Fri, 15 Nov 2013 22:02:46 +0000
Message-ID: <CEABD7B8.55BA9%ropan@cisco.com>
In-Reply-To: <19813CF2-4C78-43B8-9B75-5A4E605BE54A@ifi.uio.no>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [128.107.165.143]
Content-Type: multipart/alternative; boundary="_000_CEABD7B855BA9ropanciscocom_"
MIME-Version: 1.0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2013 22:02:53 -0000

--_000_CEABD7B855BA9ropanciscocom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable


>>>>Well, sounds like your opinion about that consensus; no doubt that anal=
ysis makes a paper stronger, but also no point in speculating about the rea=
sons why she never got it published.

You might speak with your opinion. I spoke with my scarred experience. Our =
original proposal of BCN/QCN got rejected in IEEE 802.1 because of lacking =
a control theoretic analysis. It was only after we presented control analys=
is in the fall of 2005 before PAR could be established. If you are interest=
ed in this, you can check out the history log here.

http://www.ieee802.org/1/files/public/minutes/2005-09-minutes.pdf

http://www.ieee802.org/1/pages/802.1au.html

Regards,

Rong


--_000_CEABD7B855BA9ropanciscocom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <44475465790A28408C5058031CF2EDC5@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-fami=
ly: Calibri, sans-serif; ">
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<div>
<div>
<div>&gt;&gt;&gt;&gt;Well, sounds like your opinion about that consensus; n=
o doubt that analysis makes a paper stronger, but also no point in speculat=
ing about the reasons why she never got it published.</div>
<div><br>
</div>
<div>You might speak with your opinion. I spoke with my scarred experience.=
 Our original proposal of BCN/QCN got rejected in IEEE 802.1 because of lac=
king a control theoretic analysis. It was only after we presented control a=
nalysis in the fall of 2005 before
 PAR could be established. If you are interested in this, you can check out=
 the history log here.</div>
</div>
</div>
</div>
</div>
</span>
<div><br>
</div>
<div><a href=3D"http://www.ieee802.org/1/files/public/minutes/2005-09-minut=
es.pdf">http://www.ieee802.org/1/files/public/minutes/2005-09-minutes.pdf</=
a></div>
<div><br>
</div>
<div>http://www.ieee802.org/1/pages/802.1au.html</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Rong</div>
<div><br>
</div>
</body>
</html>

--_000_CEABD7B855BA9ropanciscocom_--

From dave.taht@gmail.com  Sat Nov 16 12:02:15 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 09EDB11E8188 for <aqm@ietfa.amsl.com>; Sat, 16 Nov 2013 12:02:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.189
X-Spam-Level: 
X-Spam-Status: No, score=-0.189 tagged_above=-999 required=5 tests=[AWL=-0.003, BAYES_40=-0.185, NO_RELAYS=-0.001]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CI5K22Ae8e4Z for <aqm@ietfa.amsl.com>; Sat, 16 Nov 2013 12:02:14 -0800 (PST)
Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) by ietfa.amsl.com (Postfix) with ESMTP id 0579D11E814F for <aqm@ietf.org>; Sat, 16 Nov 2013 12:02:13 -0800 (PST)
Received: by mail-wi0-f174.google.com with SMTP id hi5so2347490wib.7 for <aqm@ietf.org>; Sat, 16 Nov 2013 12:02:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=7y7VqZM9u6mUMY/sLr6l78BcYhvMwwEtW9WsSj2gQPY=; b=hN2zz6w33aerE1zQXIGrwjG2mZtrXD+oXq02l46xX78llqfGX7HtQhW5sjk694O/ot oD+6ZFsII+53SgSDokljaVvPHtmEOHBCy/X6lvH9c+Thnc/A3kY+di/dU4j+osWmzbxu 2Sys9DJ6FFWUQur2Ecvhde+UvGaDQ94hNPlvTXOVBO5UXZriMXLz1McHHEfzTaCq/6M7 ogvQHUUE+qH0ms3lgU7YFwklKv/EI8uhAUdMFrr55USVw9qlzD1szHtO9aJdtPbVB4j4 GUqrQx55Ol05FqPa6FCHx9NwLKThT/QytON4Se73Oq7TJkxn1RRfJDIG+MZKSkP2KKZl vtow==
MIME-Version: 1.0
X-Received: by 10.194.119.106 with SMTP id kt10mr88508wjb.72.1384632133207; Sat, 16 Nov 2013 12:02:13 -0800 (PST)
Received: by 10.217.51.5 with HTTP; Sat, 16 Nov 2013 12:02:13 -0800 (PST)
Date: Sat, 16 Nov 2013 12:02:13 -0800
Message-ID: <CAA93jw6rwnBS6ghCEQCAG2e4Hp5oNJvP0guu8zm8jJUvk-jpjQ@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Subject: [aqm] Modeling the interdependency of low-priority congestion control and active queue management
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Nov 2013 20:02:15 -0000

This came out in september and I don't remember forwarding it along:

http://perso.telecom-paristech.fr/~drossi/paper/rossi13itc.pdf



--=20
Dave T=E4ht

From ghanwani@gmail.com  Sun Nov 17 10:54:00 2013
Return-Path: <ghanwani@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B0AFD11E90AA for <aqm@ietfa.amsl.com>; Sun, 17 Nov 2013 10:54:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.75
X-Spam-Level: 
X-Spam-Status: No, score=-1.75 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4lnZHfkbSrV9 for <aqm@ietfa.amsl.com>; Sun, 17 Nov 2013 10:54:00 -0800 (PST)
Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) by ietfa.amsl.com (Postfix) with ESMTP id 6094611E90A9 for <aqm@ietf.org>; Sun, 17 Nov 2013 10:53:58 -0800 (PST)
Received: by mail-wg0-f44.google.com with SMTP id k14so5387542wgh.35 for <aqm@ietf.org>; Sun, 17 Nov 2013 10:53:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/xUpevsSy/fLukR/0M1nrdEsETD7PF4EZcHDEqT2lOM=; b=kdZJ3v4vQS3Ps9gnen/OA6EXvpGCh5VMWu7DbZrAN9vRfDeI8sz0NgFjfQj+Ok3Tvl qd2NhmSEyr02t/BcIkNWQbAruK2hu4G/kXpoDgiJkI/QvBN0zVHz+9JiEsV9HUkdfnAp LV0KRVkxdL5LCWGd1H+d0ulF7jjYuUUOMKoC1LPKntAE+yhlbflhD10p/+WUgM79Xsn/ 4etPzK8L7OdQ53TFI+OE8b6srjYrikDFhC09+MmPJyeafhaq1THArfHhcBd0hE3ihIzu EnzeLLyr5NHeVuVe68N+vOTzshqI0RYyYpJUZ3hbLoLgCKM9sdKhSPxUK+teit7CQZ+U iHIg==
MIME-Version: 1.0
X-Received: by 10.195.13.45 with SMTP id ev13mr13766932wjd.20.1384714437502; Sun, 17 Nov 2013 10:53:57 -0800 (PST)
Sender: ghanwani@gmail.com
Received: by 10.217.57.131 with HTTP; Sun, 17 Nov 2013 10:53:57 -0800 (PST)
In-Reply-To: <CEAAB727.4B5D1%prenatar@cisco.com>
References: <CAEjQQ5WpeB0L7pN3joh233QgvNV2VMkN2U9BPCwFmBYjoVCrAQ@mail.gmail.com> <CEAAB727.4B5D1%prenatar@cisco.com>
Date: Sun, 17 Nov 2013 10:53:57 -0800
X-Google-Sender-Auth: GqyYZY2u1kRxXkSGrUDYeFr61HI
Message-ID: <CA+-tSzz+KR2OSLHF4ovAh2FycfQ-1duX=1LFPrB4y1A9gMPDsA@mail.gmail.com>
From: Anoop Ghanwani <anoop@alumni.duke.edu>
To: Preethi Natarajan <preethi.cis@gmail.com>
Content-Type: multipart/alternative; boundary=047d7bfced1e29b27004eb63f4db
Cc: "aqm@ietf.org" <aqm@ietf.org>, Naeem Khademi <naeem.khademi@gmail.com>, Michael Welzl <michawe@ifi.uio.no>, curtis <curtis@ipv6.occnc.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 17 Nov 2013 18:54:00 -0000

--047d7bfced1e29b27004eb63f4db
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Nov 14, 2013 at 5:33 PM, Preethi Natarajan <preethi.cis@gmail.com>wrote:
>
>
> Just pointing out the evidence here --
> http://www.ietf.org/proceedings/86/slides/slides-86-iccrg-5.pdf
>
> The slides contain preliminary results from DC scenarios, showing PIE's
> parameters and adaptability to DC scenarios. Thank god for the IETF
> archives :)
>
> Hi Preethi,

In these simulations what was the buffer size used?  From my read, it looks
like the setup had ~400 KB per port before any marking/discarding would
take place.  This is significantly more than what is available in
top-of-the-rack switches used in many data centers.

This should give you an idea of the kind of buffering that is actually
available.  As port-counts go up, the amount of buffering per port goes
down.
http://people.ucsc.edu/~warner/buffer.html

For the simulations to be relevant for such switches, they would have to be
done with significantly smaller buffer sizes.  The sizing gets even worse
when we consider there may be multiple priorities and some of those may be
lossless (requiring dedicated buffers be set aside for them).

Anoop

--047d7bfced1e29b27004eb63f4db
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Thu, Nov 14, 2013 at 5:33 PM, Preethi Natarajan <span dir=3D"ltr=
">&lt;<a href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.ci=
s@gmail.com</a>&gt;</span> wrote:<blockquote class=3D"gmail_quote" style=3D=
"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,2=
04,204);border-left-style:solid;padding-left:1ex">
<div style=3D"font-size:14px;font-family:Calibri,sans-serif;word-wrap:break=
-word"><div><br></div><div>Just pointing out the evidence here --=A0<a href=
=3D"http://www.ietf.org/proceedings/86/slides/slides-86-iccrg-5.pdf" target=
=3D"_blank">http://www.ietf.org/proceedings/86/slides/slides-86-iccrg-5.pdf=
</a></div>
<div><br></div><div>The slides contain preliminary results from DC scenario=
s, showing PIE&#39;s parameters and adaptability to DC scenarios. Thank god=
 for the IETF archives :)</div><div><br></div></div></blockquote><div>Hi Pr=
eethi,</div>
<div><br></div><div>In these simulations what was the buffer size used? =A0=
>From my read, it looks like the setup had ~400 KB per port before any marki=
ng/discarding would take place. =A0This is significantly more than what is =
available in top-of-the-rack switches used in many data centers.</div>
<div><br></div><div>This should give you an idea of the kind of buffering t=
hat is actually available. =A0As port-counts go up, the amount of buffering=
 per port goes down.</div><div><a href=3D"http://people.ucsc.edu/~warner/bu=
ffer.html">http://people.ucsc.edu/~warner/buffer.html</a><br>
</div><div><br></div><div>For the simulations to be relevant for such switc=
hes, they would have to be done with significantly smaller buffer sizes. =
=A0The sizing gets even worse when we consider there may be multiple priori=
ties and some of those may be lossless (requiring dedicated buffers be set =
aside for them).</div>
<div><br></div><div>Anoop</div></div></div></div>

--047d7bfced1e29b27004eb63f4db--

From swmike@swm.pp.se  Sun Nov 17 21:55:57 2013
Return-Path: <swmike@swm.pp.se>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3DB7711E839A for <aqm@ietfa.amsl.com>; Sun, 17 Nov 2013 21:55:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.486
X-Spam-Level: 
X-Spam-Status: No, score=-2.486 tagged_above=-999 required=5 tests=[AWL=-0.114, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WImF1e4MFqp1 for <aqm@ietfa.amsl.com>; Sun, 17 Nov 2013 21:55:54 -0800 (PST)
Received: from uplift.swm.pp.se (ipv6.swm.pp.se [IPv6:2a00:801::f]) by ietfa.amsl.com (Postfix) with ESMTP id E1C7E11E80EC for <aqm@ietf.org>; Sun, 17 Nov 2013 21:55:53 -0800 (PST)
Received: by uplift.swm.pp.se (Postfix, from userid 501) id 8197FA1; Mon, 18 Nov 2013 06:55:49 +0100 (CET)
Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 75EA59A; Mon, 18 Nov 2013 06:55:49 +0100 (CET)
Date: Mon, 18 Nov 2013 06:55:49 +0100 (CET)
From: Mikael Abrahamsson <swmike@swm.pp.se>
To: Anoop Ghanwani <anoop@alumni.duke.edu>
In-Reply-To: <CA+-tSzz+KR2OSLHF4ovAh2FycfQ-1duX=1LFPrB4y1A9gMPDsA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1311180653430.5805@uplift.swm.pp.se>
References: <CAEjQQ5WpeB0L7pN3joh233QgvNV2VMkN2U9BPCwFmBYjoVCrAQ@mail.gmail.com> <CEAAB727.4B5D1%prenatar@cisco.com> <CA+-tSzz+KR2OSLHF4ovAh2FycfQ-1duX=1LFPrB4y1A9gMPDsA@mail.gmail.com>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
Organization: People's Front Against WWW
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Cc: "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 05:55:58 -0000

On Sun, 17 Nov 2013, Anoop Ghanwani wrote:

> This should give you an idea of the kind of buffering that is actually 
> available.  As port-counts go up, the amount of buffering per port goes 
> down. http://people.ucsc.edu/~warner/buffer.html

Not only that, but some switches actually have shared buffer space for 
some or all ports, so if one port is congested, you can get immediate 
drops on other ports because of lack of buffer space.

> For the simulations to be relevant for such switches, they would have to 
> be done with significantly smaller buffer sizes.  The sizing gets even 
> worse when we consider there may be multiple priorities and some of 
> those may be lossless (requiring dedicated buffers be set aside for 
> them).

For instance, I have a switch at home with 24 gig ports that have 128 
kilobytes of memory total, shared for the entire switch for all ports. 
It's a really cheap one, but still meant for enterprise deployment.

How would we model something like that?

Guess it won't add much to bufferbloat so perhaps moot discussion :)

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

From rs@netapp.com  Mon Nov 18 04:44:49 2013
Return-Path: <rs@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 478C211E81CE for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 04:44:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.486
X-Spam-Level: 
X-Spam-Status: No, score=-2.486 tagged_above=-999 required=5 tests=[AWL=-0.114, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ROogful9VBcX for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 04:44:43 -0800 (PST)
Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) by ietfa.amsl.com (Postfix) with ESMTP id F0E6011E8116 for <aqm@ietf.org>; Mon, 18 Nov 2013 04:44:42 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,723,1378882800"; d="scan'208";a="76295627"
Received: from vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) by mx11-out.netapp.com with ESMTP; 18 Nov 2013 04:44:42 -0800
Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.147]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.03.0123.003; Mon, 18 Nov 2013 04:44:42 -0800
From: "Scheffenegger, Richard" <rs@netapp.com>
To: Michael Welzl <michawe@ifi.uio.no>, "Rong Pan (ropan)" <ropan@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
Thread-Index: AQHO3CzVh4eaSWzuP0KtNJxBK9aYhJobOREAgADcp4CAACSPAP//9emAgABSr4CABi75AIAA200AgABagICAAB6IAIAAVMEAgADNWwCAACSHAP//sJoAgACnbQD//4M6gAArHEgA///qSYCAAAikgIAAzaiA//xEA0A=
Date: Mon, 18 Nov 2013 12:44:41 +0000
Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25EB8601@SACEXCMBX02-PRD.hq.netapp.com>
References: <CEABA46C.55B4B%ropan@cisco.com> <19813CF2-4C78-43B8-9B75-5A4E605BE54A@ifi.uio.no>
In-Reply-To: <19813CF2-4C78-43B8-9B75-5A4E605BE54A@ifi.uio.no>
Accept-Language: de-AT, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.106.53.53]
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 12:44:49 -0000

[chair hat off]

Hi Michael,

>
>------------------------------------------
>>>>>>>>RP: =A0Again, the tight delay control comes from narrow-band bang-b=
ang control with hard drops. The scenarios >that throughputs are good are i=
n higher congestion scenarios where there are enough packets showing up to =
fill the >pipe. That is not the effectiveness of AQM. Tail drop with shallo=
w buffer will have high throughput in those cases >too. Try a scenario when=
 link is congested and a burst comes, see how many packets of that burst ca=
n go through.
>------------------------------------------
>
> Maybe we disagree about the goals... why would you want to allow a burst =
on an already congested link?

How do you define congested link?

You will want to be burst-tolerant on a link running at full capacity - but=
 is such a link already congested?

In any case, defining a scenario that would lead to excessive buffering del=
ay in the drop-tail case, and comparing this for various AQMs should be doc=
umented.



Richard Scheffenegger


From: aqm-bounces@ietf.org [mailto:aqm-bounces@ietf.org] On Behalf Of Micha=
el Welzl
Sent: Freitag, 15. November 2013 20:36
To: Rong Pan (ropan)
Cc: Naeem Khademi; Preethi Natarajan; aqm@ietf.org
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based

Hi,

In line, but snipping some things away:



[About evaluating against other-than-RED AQMs]=20
------------------------------------------
>>>>>>>>RP: =A0RED is the only one out there that got wide adoptions. Compa=
red with the state of the art in products (not in papers) is the idea. But =
that does not mean we don't know/learn from other AQM schemes. As matter of=
 fact, PI controller and AVQ came out about the same time. PIE also adopts =
the PI controller design. From my years of experience, I think PI controlle=
r is a better controller in this space as queue and rate lend themselves to=
 be the proportional and integral pair.

Ok, fair point about RED, and I agree that a PI controller can do well. The=
 "SoA in products" bit makes me wonder - I never really knew, but since it'=
s in principle easy to deploy an AQM scheme (you design it, you like it, yo=
u put it in your product?), I always assumed that RED wouldn't be the only =
one out there.

Now from this and other recent discussions I get the impression that RED *i=
s* the only one that was ever turned on, and maybe the only one shipped? Do=
 you know, is that the case? Just curious... =A0I guess that some products =
would sell with a strange secret AQM scheme that works very well, or not? T=
hat was just my imagination.



>>>>>>>>RP: I disagree with the notion that ARED often performs better. If =
tight delay control comes from narrow-band bang-bang control, the tail drop=
 with shallow buffer would behave similarly. For example, if target_delay =
=3D 1ms, min_th =3D 0.5ms and max_th =3D 1.5ms, the queue really allows one=
 packet in the queue. I don't see how this will be different from tail drop=
. =A0Your throughput plot showed 25% throughput loss. This is a case consid=
ered good?

By "often" I mean pretty much all cases except the very lightly loaded or e=
xtremely low target delay cases that you keep referring to.


------------------------------------------
>>>>>>>>RP: =A0Again, the tight delay control comes from narrow-band bang-b=
ang control with hard drops. The scenarios that throughputs are good are in=
 higher congestion scenarios where there are enough packets showing up to f=
ill the pipe. That is not the effectiveness of AQM. Tail drop with shallow =
buffer will have high throughput in those cases too. Try a scenario when li=
nk is congested and a burst comes, see how many packets of that burst can g=
o through.
------------------------------------------

Maybe we disagree about the goals... why would you want to allow a burst on=
 an already congested link?



Naeem and Preethi have agreed that some significant design changes would be=
 necessary to make use of ARED as a new delay-based AQM. I agree, but mainl=
y I'd expect these significant changes to deal with ECN (where we made the =
point that ALL AQMs should be changed in fact (*), but that's another story=
) and with WLAN scenarios, where ARED performed quite badly. We'll see - ma=
inly we conclude that it's probably worth playing around with it.


---------------------------------------------
>>>>>>>>RP: =A0Sure.=A0The significant design changes to ARED won't be just=
 for ECN, but to address the core issue of how effective ARED is as an AQM =
scheme. Don't get us wrong -- we would be happy to see these improvements o=
n ARED. =A0Just a side note and you seemed to know what happened in the ear=
ly 2000 time frame.=A0=A0Sally's ARED paper did not eventually publish beca=
use there was a consensus in the community that control analysis is essenti=
al in AQM (other papers like AVQ and PI do have control analysis). It would=
 be great if your group can add in this space too, especially auto-tuning n=
eeds deeper understanding of the knobs involved.=A0

Well, sounds like your opinion about that consensus; no doubt that analysis=
 makes a paper stronger, but also no point in speculating about the reasons=
 why she never got it published.

Getting back to the matter at hand, call me naive, but it seems to me that =
ARED was doing just the right thing in mildly loaded to overloaded scenario=
s, i.e. "bang bang" control may be just the right thing to do then? On that=
 basis, isn't it a matter of just detecting the lightly loaded scenarios an=
d becoming less aggressive under these circumstances? I suppose that the co=
ntrol could be tuned to do that. But this is just speculative...

Cheers,
Michael


From michawe@ifi.uio.no  Mon Nov 18 05:02:24 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD3AD11E810F for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 05:02:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.428
X-Spam-Level: 
X-Spam-Status: No, score=-102.428 tagged_above=-999 required=5 tests=[AWL=-0.056, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bDtBypO-lulW for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 05:02:19 -0800 (PST)
Received: from mail-out4.uio.no (mail-out4.uio.no [IPv6:2001:700:100:10::15]) by ietfa.amsl.com (Postfix) with ESMTP id DA38C11E839F for <aqm@ietf.org>; Mon, 18 Nov 2013 05:01:27 -0800 (PST)
Received: from mail-mx2.uio.no ([129.240.10.30]) by mail-out4.uio.no with esmtp (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1ViOS8-0002Ue-Ux; Mon, 18 Nov 2013 14:01:20 +0100
Received: from boomerang.ifi.uio.no ([129.240.68.135]) by mail-mx2.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80) (envelope-from <michawe@ifi.uio.no>) id 1ViOS8-0007In-GX; Mon, 18 Nov 2013 14:01:20 +0100
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Michael Welzl <michawe@ifi.uio.no>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25EB8601@SACEXCMBX02-PRD.hq.netapp.com>
Date: Mon, 18 Nov 2013 14:01:18 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <D55EB877-EF48-4C80-AC75-FE936B495C9E@ifi.uio.no>
References: <CEABA46C.55B4B%ropan@cisco.com> <19813CF2-4C78-43B8-9B75-5A4E605BE54A@ifi.uio.no> <012C3117EDDB3C4781FD802A8C27DD4F25EB8601@SACEXCMBX02-PRD.hq.netapp.com>
To: "Scheffenegger, Richard" <rs@netapp.com>
X-Mailer: Apple Mail (2.1283)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 5 msgs/h 1 sum rcpts/h 11 sum msgs/h 3 total rcpts 9932 max rcpts/h 44 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-6.0, required=5.0, autolearn=disabled, RP_MATCHES_RCVD=-1.05, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: 0D0999D3B51E29A7BC5407A836681C7D2540E317
X-UiO-SPAM-Test: remote_host: 129.240.68.135 spam_score: -59 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 3734 max/h 16 blacklist 0 greylist 0 ratelimit 0
Cc: "Rong Pan \(ropan\)" <ropan@cisco.com>, Naeem Khademi <naeem.khademi@gmail.com>, Preethi Natarajan <preethi.cis@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 13:02:25 -0000

On 18. nov. 2013, at 13:44, Scheffenegger, Richard wrote:

> [chair hat off]
>=20
> Hi Michael,
>=20
>>=20
>> ------------------------------------------
>>>>>>>>> RP:  Again, the tight delay control comes from narrow-band =
bang-bang control with hard drops. The scenarios >that throughputs are =
good are in higher congestion scenarios where there are enough packets =
showing up to fill the >pipe. That is not the effectiveness of AQM. Tail =
drop with shallow buffer will have high throughput in those cases >too. =
Try a scenario when link is congested and a burst comes, see how many =
packets of that burst can go through.
>> ------------------------------------------
>>=20
>> Maybe we disagree about the goals... why would you want to allow a =
burst on an already congested link?
>=20
> How do you define congested link?
>=20
> You will want to be burst-tolerant on a link running at full capacity =
- but is such a link already congested?

I'd say no, this is just perfectly saturated. I was thinking of a link =
running at full capacity + a non-empty queue.

BTW, there was talk about how to define congestion and how to define =
congestion control at the AQM meeting, during Fred's presentation. If we =
need text for this, it's worth looking at the ICCRG archives, I remember =
a long and boring discussion about these definitions early in the life =
of ICCRG.


> In any case, defining a scenario that would lead to excessive =
buffering delay in the drop-tail case, and comparing this for various =
AQMs should be documented.

Sure

Cheers,
Michael


From rs@netapp.com  Mon Nov 18 07:01:26 2013
Return-Path: <rs@netapp.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CA09311E81DF for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 07:01:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.542
X-Spam-Level: 
X-Spam-Status: No, score=-6.542 tagged_above=-999 required=5 tests=[AWL=4.056,  BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ew5S0bJZn7IO for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 07:01:22 -0800 (PST)
Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) by ietfa.amsl.com (Postfix) with ESMTP id 8877711E81C5 for <aqm@ietf.org>; Mon, 18 Nov 2013 06:54:20 -0800 (PST)
X-IronPort-AV: E=Sophos;i="4.93,724,1378882800";  d="scan'208,217";a="116960464"
Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx12-out.netapp.com with ESMTP; 18 Nov 2013 06:54:20 -0800
Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.147]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Mon, 18 Nov 2013 06:54:18 -0800
From: "Scheffenegger, Richard" <rs@netapp.com>
To: "aqm@ietf.org" <aqm@ietf.org>, Wesley Eddy <wes@mti-systems.com>
Thread-Topic: Draft minutes for the Vancouver meeting
Thread-Index: Ac7kbeBgf4Zjvn9dSSKEuxXMGiG90Q==
Date: Mon, 18 Nov 2013 14:54:16 +0000
Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25EBAEEA@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: de-AT, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.106.53.53]
Content-Type: multipart/alternative; boundary="_000_012C3117EDDB3C4781FD802A8C27DD4F25EBAEEASACEXCMBX02PRDh_"
MIME-Version: 1.0
Subject: [aqm] Draft minutes for the Vancouver meeting
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 15:01:26 -0000

--_000_012C3117EDDB3C4781FD802A8C27DD4F25EBAEEASACEXCMBX02PRDh_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

I've just uploaded the draft minutes for the meeting.
Please let us chairs know if there are any significant omissions or correct=
ions.

We would like to thank Pasi and Kevin for taking notes!


http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm

Richard Scheffenegger



--_000_012C3117EDDB3C4781FD802A8C27DD4F25EBAEEASACEXCMBX02PRDh_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;">
<div>Hi,</div>
<div>&nbsp;</div>
<div>I've just uploaded the draft minutes for the meeting.</div>
<div>Please let us chairs know if there are any significant omissions or co=
rrections.</div>
<div>&nbsp;</div>
<div>We would like to thank Pasi and Kevin for taking notes!</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div><a href=3D"http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm">=
<font color=3D"blue"><u>http://www.ietf.org/proceedings/88/minutes/minutes-=
88-aqm</u></font></a></div>
<div>&nbsp;</div>
<div>Richard Scheffenegger<br>

</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_012C3117EDDB3C4781FD802A8C27DD4F25EBAEEASACEXCMBX02PRDh_--

From ghanwani@gmail.com  Mon Nov 18 10:44:55 2013
Return-Path: <ghanwani@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A0A31A1F50 for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 10:44:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.623
X-Spam-Level: 
X-Spam-Status: No, score=0.623 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KvnrCzsDaE_l for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 10:44:53 -0800 (PST)
Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) by ietfa.amsl.com (Postfix) with ESMTP id 9AD301A1F4F for <aqm@ietf.org>; Mon, 18 Nov 2013 10:44:50 -0800 (PST)
Received: by mail-wg0-f54.google.com with SMTP id y10so6502867wgg.21 for <aqm@ietf.org>; Mon, 18 Nov 2013 10:44:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=t0KOnj6SsxOJE0REarNufaRzEg3j+Nab65U8yxh1cXY=; b=zfLOrs37clm/bzK31yHue3SQvIsUuYCVhHMjDh65wSjXugY5FyWAv2Lirw89gEZSF+ V06MMSrUN+LNX6uxwMTu1UvRZhPRc/jefh0CbiioWEZBpZ5fyvRQUnJpmtM16ShZKFkn DoHprlUwmIawalp06isVnOWV6XLGHmbVE/vYzRZXWb4AEsSBOBBRR04ehsM5Dr5jEsri nPhmnRlRH24NeIESkZob/6wOUZEyjkpUrrO1quBPHowykwlfA19/YXkrqCtKJ+rErLwe Xi95FuKRCqgRzAkVt9xB7eG2SrIN9g8Msd+ZxXZrXs9MTkg/r/l+Ewfzfhgk533XhhiD k0ow==
MIME-Version: 1.0
X-Received: by 10.194.236.169 with SMTP id uv9mr2924wjc.73.1384800284468; Mon, 18 Nov 2013 10:44:44 -0800 (PST)
Sender: ghanwani@gmail.com
Received: by 10.217.57.131 with HTTP; Mon, 18 Nov 2013 10:44:44 -0800 (PST)
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25EBAEEA@SACEXCMBX02-PRD.hq.netapp.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25EBAEEA@SACEXCMBX02-PRD.hq.netapp.com>
Date: Mon, 18 Nov 2013 10:44:44 -0800
X-Google-Sender-Auth: v5zytMbNj4OkZBEfbvXR8Nxfwjg
Message-ID: <CA+-tSzyeQPZjEY_H9Nto+6-P9bw5eFF-UWxpENKnu-hi1z5g3g@mail.gmail.com>
From: Anoop Ghanwani <anoop@alumni.duke.edu>
To: "Scheffenegger, Richard" <rs@netapp.com>
Content-Type: multipart/alternative; boundary=089e014940240a702304eb77f1e2
Cc: Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] Draft minutes for the Vancouver meeting
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 18:44:55 -0000

--089e014940240a702304eb77f1e2
Content-Type: text/plain; charset=ISO-8859-1

A couple of minor things:

>>>
Anoop Ghanwani:  Topology is very important; short RTT vs. long RTT, and
    mixed speeds, and varying numbers of hops.
>>>

This was specifically in the context of evaluating fairness of the
algorithms, and was about the fairness across sessions which are
experiencing different RTT, different speeds, different number of hops, etc.

I also don't see any mention of the question/discussion regarding whether
AQM should be used in all kinds of devices including those with very small
buffers.  Fred responded with an explanation of why it's important and I
was trying to find that but couldn't.

Anoop


On Mon, Nov 18, 2013 at 6:54 AM, Scheffenegger, Richard <rs@netapp.com>wrote:

>  Hi,
>
> I've just uploaded the draft minutes for the meeting.
> Please let us chairs know if there are any significant omissions or
> corrections.
>
> We would like to thank Pasi and Kevin for taking notes!
>
>
> *http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm*<http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm>
>
> Richard Scheffenegger
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>
>

--089e014940240a702304eb77f1e2
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">A couple of minor things:<div><br></div><div>&gt;&gt;&gt;<=
br><div><div>Anoop Ghanwani: =A0Topology is very important; short RTT vs. l=
ong RTT, and</div><div>=A0 =A0 mixed speeds, and varying numbers of hops.</=
div></div>
<div>&gt;&gt;&gt;</div><div><br></div><div>This was specifically in the con=
text of evaluating fairness of the algorithms, and was about the fairness a=
cross sessions which are experiencing different RTT, different speeds, diff=
erent number of hops, etc.</div>
<div><br></div><div>I also don&#39;t see any mention of the question/discus=
sion regarding whether AQM should be used in all kinds of devices including=
 those with very small buffers. =A0Fred responded with an explanation of wh=
y it&#39;s important and I was trying to find that but couldn&#39;t.</div>
<div><br></div></div><div>Anoop</div></div><div class=3D"gmail_extra"><br><=
br><div class=3D"gmail_quote">On Mon, Nov 18, 2013 at 6:54 AM, Scheffenegge=
r, Richard <span dir=3D"ltr">&lt;<a href=3D"mailto:rs@netapp.com" target=3D=
"_blank">rs@netapp.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">






<div>
<font face=3D"Calibri"><span style=3D"font-size:11pt">
<div>Hi,</div>
<div>=A0</div>
<div>I&#39;ve just uploaded the draft minutes for the meeting.</div>
<div>Please let us chairs know if there are any significant omissions or co=
rrections.</div>
<div>=A0</div>
<div>We would like to thank Pasi and Kevin for taking notes!</div>
<div>=A0</div>
<div>=A0</div>
<div><a href=3D"http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm" =
target=3D"_blank"><font color=3D"blue"><u>http://www.ietf.org/proceedings/8=
8/minutes/minutes-88-aqm</u></font></a></div><span class=3D"HOEnZb"><font c=
olor=3D"#888888">
<div>=A0</div>
<div>Richard Scheffenegger<br>

</div>
<div>=A0</div>
</font></span></span></font>
</div>

<br>_______________________________________________<br>
aqm mailing list<br>
<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/aqm" target=3D"_blank">htt=
ps://www.ietf.org/mailman/listinfo/aqm</a><br>
<br></blockquote></div><br></div>

--089e014940240a702304eb77f1e2--

From dave.taht@gmail.com  Mon Nov 18 11:34:47 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3083C1AE0F8 for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 11:34:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.1
X-Spam-Level: 
X-Spam-Status: No, score=-0.1 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WtOuV0d_VvrF for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 11:34:45 -0800 (PST)
Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) by ietfa.amsl.com (Postfix) with ESMTP id 4232E1ADFB6 for <aqm@ietf.org>; Mon, 18 Nov 2013 11:34:45 -0800 (PST)
Received: by mail-we0-f182.google.com with SMTP id q59so4752454wes.27 for <aqm@ietf.org>; Mon, 18 Nov 2013 11:34:39 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=RrcFtqf6w5ZJrHZrKl3NL2L9leK9MRGbP4CxhTi0c9U=; b=CYdtRafxm1Z4PVv9FK1qPb3nNt4qRoCOIAc3cBHatYHL19gpmlF+4Pxzt3RPlTGqSh 6hIIL+MWX7vyTPTC5WD6Es0VTLmn8lqYW/zL60jslgK76lflqLgbhk80WrMP5YVCZwYd tHa3xGXAC2UWybOhDyjKhaKb3tsWFlVwg/Z3ZV/igLcWX474jpYq1T0LQ+QYkJ/446oB 9VN4fTi8V7jmJb2jRaQabQZS1HvZYcEXdw2w1DagWR1cpJsTjqOwCEKScvsjnkjixsNZ j+R+eaHXLjFjEdyrkn++4W5fh6L73EqtfxKrFlxfFBcS6sbOPla1+A00cU0u6sAhSSUZ B+BA==
MIME-Version: 1.0
X-Received: by 10.180.184.14 with SMTP id eq14mr18200298wic.56.1384803279063;  Mon, 18 Nov 2013 11:34:39 -0800 (PST)
Received: by 10.217.51.5 with HTTP; Mon, 18 Nov 2013 11:34:38 -0800 (PST)
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25EBAEEA@SACEXCMBX02-PRD.hq.netapp.com>
References: <012C3117EDDB3C4781FD802A8C27DD4F25EBAEEA@SACEXCMBX02-PRD.hq.netapp.com>
Date: Mon, 18 Nov 2013 11:34:38 -0800
Message-ID: <CAA93jw41Lyw85w9t1tPCEMMo1nqjwd7-v6Dom9QexyoF6Nu5xw@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: "Scheffenegger, Richard" <rs@netapp.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: Wesley Eddy <wes@mti-systems.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] Draft minutes for the Vancouver meeting
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 19:34:47 -0000

... a decent aqm/packet scheduling testbed is nowhere near as
expensive as the large hadron collider.

Best joke I made all ietf, and it didn't make the minutes. Sigh.

On Mon, Nov 18, 2013 at 6:54 AM, Scheffenegger, Richard <rs@netapp.com> wro=
te:
> Hi,
>
> I've just uploaded the draft minutes for the meeting.
> Please let us chairs know if there are any significant omissions or
> corrections.
>
> We would like to thank Pasi and Kevin for taking notes!
>
>
> http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm
>
> Richard Scheffenegger
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From preethi.cis@gmail.com  Mon Nov 18 11:39:11 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 852481AE164 for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 11:39:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E4ZPg1WQKacS for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 11:39:10 -0800 (PST)
Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) by ietfa.amsl.com (Postfix) with ESMTP id 18EA31AE167 for <aqm@ietf.org>; Mon, 18 Nov 2013 11:39:10 -0800 (PST)
Received: by mail-pa0-f49.google.com with SMTP id kx10so797615pab.22 for <aqm@ietf.org>; Mon, 18 Nov 2013 11:39:04 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type:content-transfer-encoding; bh=8RRZ6oWuaemwaO8ezk0icwlt2htE5gjG//NyXPIytW4=; b=DynxflZgPFeLf6T/LcZL31Rq2vS0acllyJknD/iA1OdjWtPmbNWDoR017OrvTy2Euc H5nqfc9CYx/ZhfOw/3Tm6EnNF1spyorSkKTnUvG+1j+MLiQ0rdh1lWRQvwGArSJXkhcl c9lNeLNLlA87467R7aPUkBNkdlbNBVZou32sLKzZFQz+/pt89JEg0kYlT/ULf5BzBC0p uLGF2LriKKOXUhCpaS5c9xeJprd2z8mdy/qIjWEjPkEw4S4gmL1x8FQPJsK+YhTfPM9M iziM4wJLcU263RyZxE34bOmiBAy+JNGuGme8Tdw+NvEAFZFW3FsO+iWdV5yA+uhvV8Rx cxHg==
X-Received: by 10.66.180.200 with SMTP id dq8mr22835504pac.104.1384803544405;  Mon, 18 Nov 2013 11:39:04 -0800 (PST)
Received: from [10.33.22.215] (128-107-239-235.cisco.com. [128.107.239.235]) by mx.google.com with ESMTPSA id rz6sm16146781pab.22.2013.11.18.11.38.59 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 18 Nov 2013 11:39:03 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Mon, 18 Nov 2013 11:38:54 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: "Scheffenegger, Richard" <rs@netapp.com>, Michael Welzl <michawe@ifi.uio.no>, "Rong Pan (ropan)" <ropan@cisco.com>
Message-ID: <CEAFA67D.4B9D7%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25EB8601@SACEXCMBX02-PRD.hq.netapp.com>
Mime-version: 1.0
Content-type: text/plain; charset="US-ASCII"
Content-transfer-encoding: 7bit
Cc: Naeem Khademi <naeem.khademi@gmail.com>, "aqm@ietf.org" <aqm@ietf.org>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 19:39:11 -0000

On 11/18/13 4:44 AM, "Scheffenegger, Richard" <rs@netapp.com> wrote:

>[chair hat off]
>
>Hi Michael,
>
>>
>>------------------------------------------
>>>>>>>>>RP:  Again, the tight delay control comes from narrow-band
>>>>>>>>>bang-bang control with hard drops. The scenarios >that
>>>>>>>>>throughputs are good are in higher congestion scenarios where
>>>>>>>>>there are enough packets showing up to fill the >pipe. That is
>>>>>>>>>not the effectiveness of AQM. Tail drop with shallow buffer will
>>>>>>>>>have high throughput in those cases >too. Try a scenario when
>>>>>>>>>link is congested and a burst comes, see how many packets of that
>>>>>>>>>burst can go through.
>>------------------------------------------
>>
>> Maybe we disagree about the goals... why would you want to allow a
>>burst on an already congested link?
>
>How do you define congested link?
>
>You will want to be burst-tolerant on a link running at full capacity -
>but is such a link already congested?

Great questions Richard. Defining congestion is probably the main point of
contention here. Michael, thanks for mentioning about ICCRG discussions on
this topic, I think it'd be worthwhile to go through that as well. If you
can send everyone a pointer that'd be useful as well.

Here is my humble opinion -- congestion should be defined based on its
impact (to the applications). In that sense, there will be varying levels
of congestion -- mild to severe. Symptoms of mild congestion could be
slowly increasing (by some delta) buffering and queueing delay. Symptoms
of severe congestion could be packet drops at the tail, blocking other
flows etc. 

A full capacity + non-empty queue doesn't impact applications in any way,
so this doesn't signify congestion.

A latency-based AQM scheme should operate the queue at a specified
queueing delay, which means the outgoing link is under some level of
congestion already. Under these conditions, an AQM scheme should be able
to handle bursts in an effective way. A mechanism's effectiveness in
handling these bursts could be evaluated/determined by factors such as (i)
how long does it take to bring burst under control, (ii) packet drops
during this period, (iii) link utilization during this period, and may be
others that I am missing.



>
>In any case, defining a scenario that would lead to excessive buffering
>delay in the drop-tail case, and comparing this for various AQMs should
>be documented.


I agree. Thinking more, maybe we should evaluate an AQM for the varying
levels of congestion. Document scenarios that'd create the different
levels of congestion from mild to severe and compare AQMs for all these
levels, not just empirically but also highlight their behavioral
differences. 

Thanks,
Preethi



From preethi.cis@gmail.com  Mon Nov 18 12:03:19 2013
Return-Path: <preethi.cis@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C48E1AE26F for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 12:03:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level: 
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FqqQeDxCJ0Iu for <aqm@ietfa.amsl.com>; Mon, 18 Nov 2013 12:03:17 -0800 (PST)
Received: from mail-pd0-x232.google.com (mail-pd0-x232.google.com [IPv6:2607:f8b0:400e:c02::232]) by ietfa.amsl.com (Postfix) with ESMTP id A17031AE309 for <aqm@ietf.org>; Mon, 18 Nov 2013 12:03:17 -0800 (PST)
Received: by mail-pd0-f178.google.com with SMTP id y10so145799pdj.37 for <aqm@ietf.org>; Mon, 18 Nov 2013 12:03:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type; bh=Kh+75ngPoNUAKfXGNgDLmTVBubwNRc5TijNMcIKM/8Q=; b=NfOConpXnn/9J+uEt5NLaaUhnavgNwjHMgIqXKMa4hgrybLoKKTPEabmvFxMf/+B0P Go6ocw3dRo8SlT9pbRQDZ3y9Rfg0zmPvIIkaevXy2dv6EqDSacqHXmfRRqYCxAUylo4d /OxHxJ7OrV2qiGThburVvr6+IPkimjaDyPF5xIGVyP1civEX2/xgsdQeE5rjFntZaGk9 yy/5JgqHTXbO1mIly2ZjJ5YAq5kIJSnGo35GhYS9rblXvMbFWfKizAgRzCwAxNaAGDxc iqAoJwYzRaZWeYmo8GjiBLmhhf9WZ3zFQVx26Rdc6yn9ULJ+tTmcWADYngPgpsTwEvKT WxeQ==
X-Received: by 10.66.102.39 with SMTP id fl7mr23176028pab.43.1384804991960; Mon, 18 Nov 2013 12:03:11 -0800 (PST)
Received: from [10.33.22.215] (128-107-239-233.cisco.com. [128.107.239.233]) by mx.google.com with ESMTPSA id qv8sm25315072pbc.31.2013.11.18.12.03.09 for <multiple recipients> (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 18 Nov 2013 12:03:11 -0800 (PST)
User-Agent: Microsoft-MacOutlook/14.2.1.120420
Date: Mon, 18 Nov 2013 12:03:06 -0800
From: Preethi Natarajan <preethi.cis@gmail.com>
To: Anoop Ghanwani <anoop@alumni.duke.edu>
Message-ID: <CEAFAE42.4BA1F%prenatar@cisco.com>
Thread-Topic: [aqm] AQM schemes: Queue length vs. delay based
In-Reply-To: <CA+-tSzz+KR2OSLHF4ovAh2FycfQ-1duX=1LFPrB4y1A9gMPDsA@mail.gmail.com>
Mime-version: 1.0
Content-type: multipart/alternative; boundary="B_3467620990_11488709"
Cc: "aqm@ietf.org" <aqm@ietf.org>, Naeem Khademi <naeem.khademi@gmail.com>, Michael Welzl <michawe@ifi.uio.no>, curtis <curtis@ipv6.occnc.com>
Subject: Re: [aqm] AQM schemes: Queue length vs. delay based
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 20:03:19 -0000

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3467620990_11488709
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit



From:  Anoop Ghanwani <anoop@alumni.duke.edu>
Date:  Sunday, November 17, 2013 10:53 AM
To:  Preethi Natarajan <preethi.cis@gmail.com>
Cc:  Naeem Khademi <naeem.khademi@gmail.com>, curtis
<curtis@ipv6.occnc.com>, Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org"
<aqm@ietf.org>
Subject:  Re: [aqm] AQM schemes: Queue length vs. delay based

> 
> 
> 
> On Thu, Nov 14, 2013 at 5:33 PM, Preethi Natarajan <preethi.cis@gmail.com>
> wrote:
>> 
>> Just pointing out the evidence here --
>> http://www.ietf.org/proceedings/86/slides/slides-86-iccrg-5.pdf
>> 
>> The slides contain preliminary results from DC scenarios, showing PIE's
>> parameters and adaptability to DC scenarios. Thank god for the IETF archives
>> :)
>> 
> Hi Preethi,
> 
> In these simulations what was the buffer size used?  From my read, it looks
> like the setup had ~400 KB per port before any marking/discarding would take
> place.  This is significantly more than what is available in top-of-the-rack
> switches used in many data centers.
> 
> This should give you an idea of the kind of buffering that is actually
> available.  As port-counts go up, the amount of buffering per port goes down.
> http://people.ucsc.edu/~warner/buffer.html
> 
> For the simulations to be relevant for such switches, they would have to be
> done with significantly smaller buffer sizes.  The sizing gets even worse when
> we consider there may be multiple priorities and some of those may be lossless
> (requiring dedicated buffers be set aside for them).
> 

Hi Anoop,

Thanks for the pointer regarding small buffers.

We haven't run simulations with small buffers in DC environments. However,
from our experience, PIE's control law usually adapts well to small buffers
for a given network condition. Maybe you can help us refine the DC
simulation parameters and we can give this a try.

We'll follow-up offline if that makes sense.

Thanks,
Preethi 





--B_3467620990_11488709
Content-type: text/html;
	charset="US-ASCII"
Content-transfer-encoding: quoted-printable

<html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:=
 14px; font-family: Calibri, sans-serif; "><div><br></div><div><br></div><sp=
an id=3D"OLK_SRC_BODY_SECTION"><div style=3D"font-family:Calibri; font-size:11pt=
; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: med=
ium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER=
-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span =
style=3D"font-weight:bold">From: </span> Anoop Ghanwani &lt;<a href=3D"mailto:an=
oop@alumni.duke.edu">anoop@alumni.duke.edu</a>&gt;<br><span style=3D"font-weig=
ht:bold">Date: </span> Sunday, November 17, 2013 10:53 AM<br><span style=3D"fo=
nt-weight:bold">To: </span> Preethi Natarajan &lt;<a href=3D"mailto:preethi.ci=
s@gmail.com">preethi.cis@gmail.com</a>&gt;<br><span style=3D"font-weight:bold"=
>Cc: </span> Naeem Khademi &lt;<a href=3D"mailto:naeem.khademi@gmail.com">naee=
m.khademi@gmail.com</a>&gt;, curtis &lt;<a href=3D"mailto:curtis@ipv6.occnc.co=
m">curtis@ipv6.occnc.com</a>&gt;, Michael Welzl &lt;<a href=3D"mailto:michawe@=
ifi.uio.no">michawe@ifi.uio.no</a>&gt;, "<a href=3D"mailto:aqm@ietf.org">aqm@i=
etf.org</a>" &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.org</a>&gt;<br><span=
 style=3D"font-weight:bold">Subject: </span> Re: [aqm] AQM schemes: Queue leng=
th vs. delay based<br></div><div><br></div><blockquote id=3D"MAC_OUTLOOK_ATTRI=
BUTION_BLOCKQUOTE" style=3D"BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MAR=
GIN:0 0 0 5;"><div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=
=3D"gmail_quote">On Thu, Nov 14, 2013 at 5:33 PM, Preethi Natarajan <span dir=3D=
"ltr">&lt;<a href=3D"mailto:preethi.cis@gmail.com" target=3D"_blank">preethi.cis=
@gmail.com</a>&gt;</span> wrote:<blockquote class=3D"gmail_quote" style=3D"margi=
n:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204)=
;border-left-style:solid;padding-left:1ex"><div style=3D"font-size:14px;font-f=
amily:Calibri,sans-serif;word-wrap:break-word"><div><br></div><div>Just poin=
ting out the evidence here --&nbsp;<a href=3D"http://www.ietf.org/proceedings/=
86/slides/slides-86-iccrg-5.pdf" target=3D"_blank">http://www.ietf.org/proceed=
ings/86/slides/slides-86-iccrg-5.pdf</a></div><div><br></div><div>The slides=
 contain preliminary results from DC scenarios, showing PIE's parameters and=
 adaptability to DC scenarios. Thank god for the IETF archives :)</div><div>=
<br></div></div></blockquote><div>Hi Preethi,</div><div><br></div><div>In th=
ese simulations what was the buffer size used? &nbsp;From my read, it looks =
like the setup had ~400 KB per port before any marking/discarding would take=
 place. &nbsp;This is significantly more than what is available in top-of-th=
e-rack switches used in many data centers.</div><div><br></div><div>This sho=
uld give you an idea of the kind of buffering that is actually available. &n=
bsp;As port-counts go up, the amount of buffering per port goes down.</div><=
div><a href=3D"http://people.ucsc.edu/~warner/buffer.html">http://people.ucsc.=
edu/~warner/buffer.html</a><br></div><div><br></div><div>For the simulations=
 to be relevant for such switches, they would have to be done with significa=
ntly smaller buffer sizes. &nbsp;The sizing gets even worse when we consider=
 there may be multiple priorities and some of those may be lossless (requiri=
ng dedicated buffers be set aside for them).</div><div><br></div></div></div=
></div></blockquote></span><div><br></div><div>Hi Anoop,</div><div><br></div=
><div>Thanks for the pointer regarding small buffers.</div><div><br></div><d=
iv>We haven't run simulations with small buffers in DC environments. However=
, from our experience, PIE's control law usually adapts well to small buffer=
s for a given network condition. Maybe you can help us refine the DC simulat=
ion parameters and we can give this a try.&nbsp;</div><div><br></div><div>We=
'll follow-up offline if that makes sense.</div><div><br></div><div>Thanks,<=
/div><div>Preethi&nbsp;</div><div><br></div><div><br></div></body></html>

--B_3467620990_11488709--



From g.white@CableLabs.com  Wed Nov 20 08:54:38 2013
Return-Path: <g.white@CableLabs.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F6141AE04F for <aqm@ietfa.amsl.com>; Wed, 20 Nov 2013 08:54:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 1.611
X-Spam-Level: *
X-Spam-Status: No, score=1.611 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.525] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y2V1l-hBFT_n for <aqm@ietfa.amsl.com>; Wed, 20 Nov 2013 08:54:36 -0800 (PST)
Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 83E2E1AE037 for <aqm@ietf.org>; Wed, 20 Nov 2013 08:54:36 -0800 (PST)
Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id rAKGsTGc016155; Wed, 20 Nov 2013 09:54:29 -0700
Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Wed, 20 Nov 2013 09:54:28 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com)
Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Wed, 20 Nov 2013 09:54:28 -0700
From: Greg White <g.white@CableLabs.com>
To: "Scheffenegger, Richard" <rs@netapp.com>, "aqm@ietf.org" <aqm@ietf.org>, Wesley Eddy <wes@mti-systems.com>
Thread-Topic: [aqm] Draft minutes for the Vancouver meeting
Thread-Index: Ac7kbeBgf4Zjvn9dSSKEuxXMGiG90QBo0ssA
Date: Wed, 20 Nov 2013 16:54:27 +0000
Message-ID: <CEB22C36.22E85%g.white@cablelabs.com>
In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F25EBAEEA@SACEXCMBX02-PRD.hq.netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.3.8.130913
x-originating-ip: [10.4.10.154]
Content-Type: multipart/alternative; boundary="_000_CEB22C3622E85gwhitecablelabscom_"
MIME-Version: 1.0
Subject: Re: [aqm] Draft minutes for the Vancouver meeting
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Nov 2013 16:54:38 -0000

--_000_CEB22C3622E85gwhitecablelabscom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

A few suggested edits below.


-- Slide: CableLabs including AQM PIE in their products

should probably read

-- Slide: CableLabs requiring AQM PIE in DOCSIS 3.1 modems


=85 since CableLabs doesn't build products=85.


-------------------------------------------------------

suggested insertions from the Jabber log shown as >>>


Rong: possibly doing this on 3.0, but now doing this for 3.1

Thomas: What is the timeframe?

Rong: three years from now

>>>Greg White (via Jabber): early 2015 for initial products

Thomas: is it frozen now?

Rong: pretty much is, some modifications possible

Thomas: is it too late for this group to have influence?

Dave Taht: There are vendor specific solutions that can be applied...

Lars: Is there an open specification for PIE variant DOCSIS has chosen,
for us to look at?

>>>Dave Taht (via Jabber): http://www.cablelabs.com/specifications/CM-SP-MU=
LPIv3.1-101-131029.pdf

Jim: How is it different from Linux implementation?

Rong: Some parameters were optimized for the PHY/MAC layer of DOCSIS.



-------------------------------------------------------
just a nit.  8. Nov. was a Friday=85


-- End of Tuesday session --


Meeting:    IETF88, Tuesday 8. Nov. 2013, 12:30-13:30 PST
Location:   Vancouver, Canada (Regency B)
Note Taker: Kevin Lahey




-------------------------------------------------------



Jabber, Greg White:  How does ECN work with Codel?
Dave Taht:  We've had ECN in FQ-CoDel from the start.
>>>Jabber, GW:  question was regarding using different criteria for drop vs=
 mark in CoDel.




From: <Scheffenegger>, Richard <rs@netapp.com<mailto:rs@netapp.com>>
Date: Monday, November 18, 2013 at 7:54 AM
To: "aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>=
, Wesley Eddy <wes@mti-systems.com<mailto:wes@mti-systems.com>>
Subject: [aqm] Draft minutes for the Vancouver meeting

Hi,

I've just uploaded the draft minutes for the meeting.
Please let us chairs know if there are any significant omissions or correct=
ions.

We would like to thank Pasi and Kevin for taking notes!


http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm

Richard Scheffenegger


--_000_CEB22C3622E85gwhitecablelabscom_
Content-Type: text/html; charset="Windows-1252"
Content-ID: <935CCE2AB875054B81C9DCF97DF2CBAE@cablelabs.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1=
252">
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-fami=
ly: Calibri, sans-serif;">
<div>A few suggested edits below.</div>
<div>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">
-- Slide: CableLabs including AQM PIE in their products</pre>
</div>
<div>should probably read</div>
<div>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">-- Slide: Cabl=
eLabs requiring AQM PIE in DOCSIS 3.1 modems
</pre>
</div>
<div>=85 since CableLabs doesn't build products=85.</div>
<div><br>
</div>
<div><br>
</div>
<div>-------------------------------------------------------</div>
<div><br>
</div>
<div>suggested insertions from the Jabber log shown as &gt;&gt;&gt;</div>
<div><br>
</div>
<div>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">Rong: possibly=
 doing this on 3.0, but now doing this for 3.1

Thomas: What is the timeframe?

Rong: three years from now</pre>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">&gt;&gt;&gt;Gr=
eg White (via Jabber): early 2015 for initial products
=20
Thomas: is it frozen now?

Rong: pretty much is, some modifications possible</pre>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">Thomas: is it =
too late for this group to have influence?

Dave Taht: There are vendor specific solutions that can be applied...

Lars: Is there an open specification for PIE variant DOCSIS has chosen,
for us to look at?</pre>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">&gt;&gt;&gt;Da=
ve Taht (via Jabber): http://www.cablelabs.com/specifications/CM-SP-MULPIv3=
.1-101-131029.pdf

Jim: How is it different from Linux implementation?

Rong: Some parameters were optimized for the PHY/MAC layer of DOCSIS.

</pre>
</div>
<div>
<div>-------------------------------------------------------</div>
</div>
<div>just a nit. &nbsp;8. Nov. was a Friday=85</div>
<div>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">
-- End of Tuesday session --


Meeting:    IETF88, Tuesday 8. Nov. 2013, 12:30-13:30 PST
Location:   Vancouver, Canada (Regency B)
Note Taker: Kevin Lahey


</pre>
</div>
<div>
<div>
<div>-------------------------------------------------------</div>
<div></div>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<pre style=3D"word-wrap: break-word; white-space: pre-wrap;">Jabber, Greg W=
hite:  How does ECN work with Codel?
Dave Taht:  We've had ECN in FQ-CoDel from the start.
&gt;&gt;&gt;Jabber, GW:  question was regarding using different criteria fo=
r drop vs mark in CoDel.</pre>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div style=3D"font-family:Calibri; font-size:11pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>&lt;Scheffenegger&gt;, Richar=
d &lt;<a href=3D"mailto:rs@netapp.com">rs@netapp.com</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Monday, November 18, 2013 at =
7:54 AM<br>
<span style=3D"font-weight:bold">To: </span>&quot;<a href=3D"mailto:aqm@iet=
f.org">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.=
org</a>&gt;, Wesley Eddy &lt;<a href=3D"mailto:wes@mti-systems.com">wes@mti=
-systems.com</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span>[aqm] Draft minutes for th=
e Vancouver meeting<br>
</div>
<div><br>
</div>
<div>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf --><style><!-- .EmailQuote { margin-left: 1pt; padd=
ing-left: 4pt; border-left: #800000 2px solid; } --></style>
<div><font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;">
<div>Hi,</div>
<div>&nbsp;</div>
<div>I've just uploaded the draft minutes for the meeting.</div>
<div>Please let us chairs know if there are any significant omissions or co=
rrections.</div>
<div>&nbsp;</div>
<div>We would like to thank Pasi and Kevin for taking notes!</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div><a href=3D"http://www.ietf.org/proceedings/88/minutes/minutes-88-aqm">=
<font color=3D"blue"><u>http://www.ietf.org/proceedings/88/minutes/minutes-=
88-aqm</u></font></a></div>
<div>&nbsp;</div>
<div>Richard Scheffenegger<br>
</div>
<div>&nbsp;</div>
</span></font></div>
</div>
</span>
</body>
</html>

--_000_CEB22C3622E85gwhitecablelabscom_--

From dave.taht@gmail.com  Mon Nov 25 17:49:33 2013
Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F2D001AE104; Mon, 25 Nov 2013 17:49:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.101
X-Spam-Level: 
X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mq0PsumatAcV; Mon, 25 Nov 2013 17:49:30 -0800 (PST)
Received: from mail-we0-x233.google.com (mail-we0-x233.google.com [IPv6:2a00:1450:400c:c03::233]) by ietfa.amsl.com (Postfix) with ESMTP id CE14D1AE0F2; Mon, 25 Nov 2013 17:49:29 -0800 (PST)
Received: by mail-we0-f179.google.com with SMTP id q59so4616936wes.38 for <multiple recipients>; Mon, 25 Nov 2013 17:49:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=IFnV8dyjCOi8AaqYyFFRBPjJlVL+FXugSIq73UzhOLo=; b=bjcbDo1jbAgFiHLkB9QI420L1NoigUw6veoejj/jP3uDqAzRHtP6h0MmOja5wotNFR GgsZJxRK/ELvpixUkJw8FFkjPiDgZqvSvVhA9dtx2MMExsT3tvb+ir0Inc+AUj25pFWq sfGMwEK5qz6FIL9XBrJ+YtY670K6STCTRXmmsdpI4NtsFQAlx0qCi8xUcG0L7PO//hPo H5k/ycO2BFJfTTuGnOywYjAuHeE8IiZ1Wuc5ClHClVOxexra9R9QhNxEkTiQUg6XKtlz 1En+/L1I9ioPU0TWJfBHDJbruCtBI/LkV8oxVafA9DcHfpu9HUFR9Aa2WZHfz5iFR5N7 wJrw==
MIME-Version: 1.0
X-Received: by 10.194.186.167 with SMTP id fl7mr23786wjc.85.1385430569410; Mon, 25 Nov 2013 17:49:29 -0800 (PST)
Received: by 10.217.51.5 with HTTP; Mon, 25 Nov 2013 17:49:29 -0800 (PST)
Date: Mon, 25 Nov 2013 17:49:29 -0800
Message-ID: <CAA93jw4Y_6KbRwaorUoGX1Kj7hd9SP4J4wwx7JA2S6u2B4h4Pw@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: "aqm@ietf.org" <aqm@ietf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>, bloat <bloat@lists.bufferbloat.net>, "iccrg@irtf.org" <iccrg@irtf.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Subject: [aqm] Clearing up misunderstandings about Linux versioning
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Nov 2013 01:49:33 -0000

I am in the process of trying to reproduce the recent rite papers on
immediate congestion notification and the iccrg ARED work and need
to explain something to researchers so that I don't have to work so hard.

Linux kernel releases are numbered X.Y.Z-Q. All kernel versions
contain bugs. Linux 3.2.0 was, IMHO, the nadir of the network stack in
Linux.

The "X" is the major version number. It's only changed 3 times. "Y",
is the minor version number. These come out roughly quarterly and
consist of new development of "features". "Z" is critical patches
backported from newer releases. -Q is usually the vendor's kernel
build number which often contains more patches.

These numbers, clearly identified, in every academic paper on
networking, and every presentation, ever published,  would make me a
happier guy. A pointer to the git tree actually used would make me
even happier, with all the patches (like DCTCP in this case) applied,
would cause me to dance for joy, and sing hallelujah!

Anyway, on the "Z" part of X.Y.Z:

Periodically a "long term stable" release is picked and receives updates
for as long as someone is funded to do it.

*the only things that enter into a long term stable release* are fixes
for security bugs, crash bugs, and truly egregious bugs that can be
somewhat easily fixed.

But the rest of the development goes into X.Y+1.

Sane people never run a X.Y.0 release on hardware/data they care about.

So... anyway... when I was told that the recent paper on DCTCP had
been done against Linux 3.2.18, "which came out in july, 2013!" ...

I was partially happy - pretty stable release - but
my heart sank as I knew that very, very, very few of the relevant fixes
for bufferbloat and the tcp stack had landed in anything prior to
Linux 3.6. Those fixes had mostly qualified as "features". Several in
fact have been in such continuous development that I'd not want to
generalize from fq_codel in 3.5 vs what's in 3.8 now, as one example.

So it's my hope that folk will try to follow more closely the X.Y series
of kernels rather than the 3.2.Z series of kernels in the future. I'm
very happy with what happened in the 3.12 series in particular and
look forward to work against it in the near future. The 3.13 work is
just beginning, too.

In the hope that the showing the mechanics of researching what fixes
did land in an old stable release would help on future papers,
here's how to look:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable=
.git
git clone linux-stable linux-3.2.18
cd linux-3.2.18
git checkout v3.2.18
git checkout -b ritepaper
git log net include/net # just the networking bits, not the driver
bits for this example

BQL, enhanced SFQ, SFQRED, codel, fq_codel, tcp small queues,
retirement of some odd tcp logic, and hundreds of other changes were
made to the the stack since 3.2.0, and were NOT backported to 3.2.Z.

Some very relevant bugfixes did indeed land in 3.2.18 that were not in 3.2!

If you have an experiment that used TCP that is against an even older
release, perhaps some of these major bugs might explain your results.
Here's a sampling:

commit 4b9b05fd95c502521eaef111ba0f83c58b391587
Author: Eric Dumazet <edumazet@google.com>
Date:   Wed May 2 02:28:41 2012 +0000

    tcp: change tcp_adv_win_scale and tcp_rmem[2]

<snip snip>

    This also means tcp advertises a too optimistic window for a given
    allocated rcvspace : When receiving frames, sk_rmem_alloc can hit
    sk_rcvbuf limit and we call tcp_prune_queue()/tcp_collapse() too often,
    especially when application is slow to drain its receive queue or in
    case of losses (netperf is fast, scp is slow). This is a major latency
    source.

icommit b713f6c7d317c136f03c132203d0900f4a0de084
Author: Yuchung Cheng <ycheng@google.com>
Date:   Mon Apr 30 06:00:18 2012 +0000

    tcp: fix infinite cwnd in tcp_complete_cwr()

    [ Upstream commit 1cebce36d660c83bd1353e41f3e66abd4686f215 ]

    When the cwnd reduction is done, ssthresh may be infinite
    if TCP enters CWR via ECN or F-RTO. If cwnd is not undone, i.e.,
    undo_marker is set, tcp_complete_cwr() falsely set cwnd to the
    infinite ssthresh value. The correct operation is to keep cwnd
    intact because it has been updated in ECN or F-RTO.

commit 65355aea86b2a70cbc7cbe14466702bc5a4e2217
Author: Neal Cardwell <ncardwell@google.com>
Date:   Tue Apr 10 07:59:20 2012 +0000

    tcp: fix tcp_rcv_rtt_update() use of an unscaled RTT sample

    [ Upstream commit 18a223e0b9ec8979320ba364b47c9772391d6d05 ]

    Fix a code path in tcp_rcv_rtt_update() that was comparing scaled and
    unscaled RTT samples.

    The intent in the code was to only use the 'm' measurement if it was a
    new minimum.  However, since 'm' had not yet been shifted left 3 bits
    but 'new_sample' had, this comparison would nearly always succeed,
    leading us to erroneously set our receive-side RTT estimate to the 'm'
    sample when that sample could be nearly 8x too high to use.

    The overall effect is to often cause the receive-side RTT estimate to
    be significantly too large (up to 40% too large for brief periods in
    my tests).


commit 1ee5fa1e9970a16036e37c7b9d5ce81c778252fc

    [PATCH] sch_red: fix red_change()

    Now RED is classful, we must check q->qdisc->q.qlen, and if queue is em=
pt
    we start an idle period, not end it.



--=20
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.=
html

From michawe@ifi.uio.no  Tue Nov 26 01:56:00 2013
Return-Path: <michawe@ifi.uio.no>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A28A41AE029 for <aqm@ietfa.amsl.com>; Tue, 26 Nov 2013 01:56:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RShmrMNUbZpU for <aqm@ietfa.amsl.com>; Tue, 26 Nov 2013 01:55:58 -0800 (PST)
Received: from mail-out5.uio.no (mail-out5.uio.no [IPv6:2001:700:100:10::17]) by ietfa.amsl.com (Postfix) with ESMTP id 0191A1A1F55 for <aqm@ietf.org>; Tue, 26 Nov 2013 01:55:58 -0800 (PST)
Received: from mail-mx6.uio.no ([129.240.10.40]) by mail-out5.uio.no with esmtp (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1VlFN7-0006tJ-3G for aqm@ietf.org; Tue, 26 Nov 2013 10:55:57 +0100
Received: from boomerang.ifi.uio.no ([129.240.68.135]) by mail-mx6.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.80.1) (envelope-from <michawe@ifi.uio.no>) id 1VlFN6-0000vz-J2 for aqm@ietf.org; Tue, 26 Nov 2013 10:55:57 +0100
From: Michael Welzl <michawe@ifi.uio.no>
Content-Type: multipart/alternative; boundary="Apple-Mail=_8C5773B3-6582-43FB-88C5-09A2EA4B3351"
Date: Tue, 26 Nov 2013 10:55:56 +0100
Message-Id: <EA0A6403-5722-47F1-ACC2-1771457359BB@ifi.uio.no>
To: aqm@ietf.org
Mime-Version: 1.0 (Apple Message framework v1283)
X-Mailer: Apple Mail (2.1283)
X-UiO-SPF-Received: 
X-UiO-Ratelimit-Test: rcpts/h 1 msgs/h 1 sum rcpts/h 8 sum msgs/h 3 total rcpts 10277 max rcpts/h 44 ratelimit 0
X-UiO-Spam-info: not spam, SpamAssassin (score=-6.0, required=5.0, autolearn=disabled, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.05, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO)
X-UiO-Scanned: 6969B4DD64364E436347184E24AEF06B66130646
X-UiO-SPAM-Test: remote_host: 129.240.68.135 spam_score: -59 maxlevel 80 minaction 2 bait 0 mail/h: 1 total 3879 max/h 16 blacklist 0 greylist 0 ratelimit 0
Subject: [aqm] AQM survey
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Nov 2013 09:56:00 -0000

--Apple-Mail=_8C5773B3-6582-43FB-88C5-09A2EA4B3351
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi,

I just stumbled over this, and it looks really good to me - so I thought =
I'd share it:

"Active Queue Management: A Survey" by Richelle Adams, in IEEE Comm. =
Surveys & Tutorials,
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=3D6329367
(it seems you must have IEEE Xplore access to get it, but I thought I'll =
share the link anyway for those of us who do)

I think the author must be forgiven for not discussing CoDel and PIE, as =
the abstract says: "This survey attempts to travel the trajectory of AQM =
research from 1993 with the first algorithm, Random Early Detection =
(RED), to current work in 2011"

- even though it was published in the third quarter of 2013... that's =
just the time of academic publishing in journals / magazines.

Cheers,
Michael


--Apple-Mail=_8C5773B3-6582-43FB-88C5-09A2EA4B3351
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
">Hi,<div><br></div><div>I just stumbled over this, and it looks really =
good to me - so I thought I'd share it:</div><div><br></div><div>"Active =
Queue Management: A Survey" by Richelle Adams, in IEEE Comm. Surveys =
&amp; Tutorials,</div><div><a =
href=3D"http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=3D63293=
67">http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=3D6329367</=
a></div><div>(it seems you must have IEEE Xplore access to get it, but I =
thought I'll share the link anyway for those of us who =
do)</div><div><br></div><div>I think the author must be forgiven for not =
discussing CoDel and PIE, as the abstract says: "This survey attempts to =
travel the trajectory of AQM research from 1993 with the first =
algorithm, Random Early&nbsp;Detection (RED), to current work in =
2011"</div><div><br></div><div>- even though it was published in the =
third quarter of 2013... that's just the time of academic publishing in =
journals / =
magazines.</div><div><br></div><div>Cheers,</div><div>Michael</div><div><b=
r></div></body></html>=

--Apple-Mail=_8C5773B3-6582-43FB-88C5-09A2EA4B3351--

From naeem.khademi@gmail.com  Tue Nov 26 05:51:17 2013
Return-Path: <naeem.khademi@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE0351AE202; Tue, 26 Nov 2013 05:51:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=unavailable
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tu7BJVf8Bely; Tue, 26 Nov 2013 05:51:14 -0800 (PST)
Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 7AA8C1AE1F2; Tue, 26 Nov 2013 05:51:11 -0800 (PST)
Received: by mail-vc0-f175.google.com with SMTP id ld13so3761202vcb.20 for <multiple recipients>; Tue, 26 Nov 2013 05:51:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=w+OceW4KjkNTvwTXPSttwajC1yKbhiPvT5jIpoySj+M=; b=y4sdSSgVCmNb9TuLv5ZEgGYcLuXqHrMXlMZKlygewVkEsKiJj3IPNKYvdHtRATEUwX IfS1XJn3s16gSeQc2Oglg4d0+B1R1TG2GJfHLmzC9ugS+aAlgBCbiR1Dpt+z3kSD/uA8 ibGsWdu8bb5Rex5/svg+xee9c3nf+h2wPw1ln7oDi+s4CPz6A8WGpYzefHeevZhoeAj5 kJME76UCUdc1spMVYICf89hEFj4qhkQsR+mlmG0u876LduV0sDyqciYpUuBE/YclWml2 iitxn/tcSvmU49TsQjk1GRmOpEFXM3aGXImnA4dP0NClcIKzgWUWKHeuf7vLg7SkKGtB zveg==
MIME-Version: 1.0
X-Received: by 10.52.97.35 with SMTP id dx3mr26478159vdb.18.1385473870941; Tue, 26 Nov 2013 05:51:10 -0800 (PST)
Received: by 10.220.109.5 with HTTP; Tue, 26 Nov 2013 05:51:10 -0800 (PST)
In-Reply-To: <CAA93jw4Y_6KbRwaorUoGX1Kj7hd9SP4J4wwx7JA2S6u2B4h4Pw@mail.gmail.com>
References: <CAA93jw4Y_6KbRwaorUoGX1Kj7hd9SP4J4wwx7JA2S6u2B4h4Pw@mail.gmail.com>
Date: Tue, 26 Nov 2013 14:51:10 +0100
Message-ID: <CAEjQQ5W_2WYAiyGiHa=3YG3OejmGY=pkk_a1XyRr3mRQJAgZTQ@mail.gmail.com>
From: Naeem Khademi <naeem.khademi@gmail.com>
To: Dave Taht <dave.taht@gmail.com>
Content-Type: multipart/alternative; boundary=20cf307f31a8ec562c04ec14c5bf
Cc: bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>, "iccrg@irtf.org" <iccrg@irtf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>
Subject: Re: [aqm] [iccrg] Clearing up misunderstandings about Linux versioning
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Nov 2013 13:51:17 -0000

--20cf307f31a8ec562c04ec14c5bf
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Dave

Thanks for sharing this and great to know that you're working on
replicating the "AQM Kids..." work presented at ICCRG. jfyi we used Linux
3.10.4 on the AQM box for our real-life tests (also mentioned in Table 4,
page 8 of the TR) and it should contain most of the recent codes on the
latest AQMs judging from what you've written below.

Cheers,
Naeem

On Tue, Nov 26, 2013 at 2:49 AM, Dave Taht <dave.taht@gmail.com> wrote:

> I am in the process of trying to reproduce the recent rite papers on
> immediate congestion notification and the iccrg ARED work and need
> to explain something to researchers so that I don't have to work so hard.
>
> Linux kernel releases are numbered X.Y.Z-Q. All kernel versions
> contain bugs. Linux 3.2.0 was, IMHO, the nadir of the network stack in
> Linux.
>
> The "X" is the major version number. It's only changed 3 times. "Y",
> is the minor version number. These come out roughly quarterly and
> consist of new development of "features". "Z" is critical patches
> backported from newer releases. -Q is usually the vendor's kernel
> build number which often contains more patches.
>
> These numbers, clearly identified, in every academic paper on
> networking, and every presentation, ever published,  would make me a
> happier guy. A pointer to the git tree actually used would make me
> even happier, with all the patches (like DCTCP in this case) applied,
> would cause me to dance for joy, and sing hallelujah!
>
> Anyway, on the "Z" part of X.Y.Z:
>
> Periodically a "long term stable" release is picked and receives updates
> for as long as someone is funded to do it.
>
> *the only things that enter into a long term stable release* are fixes
> for security bugs, crash bugs, and truly egregious bugs that can be
> somewhat easily fixed.
>
> But the rest of the development goes into X.Y+1.
>
> Sane people never run a X.Y.0 release on hardware/data they care about.
>
> So... anyway... when I was told that the recent paper on DCTCP had
> been done against Linux 3.2.18, "which came out in july, 2013!" ...
>
> I was partially happy - pretty stable release - but
> my heart sank as I knew that very, very, very few of the relevant fixes
> for bufferbloat and the tcp stack had landed in anything prior to
> Linux 3.6. Those fixes had mostly qualified as "features". Several in
> fact have been in such continuous development that I'd not want to
> generalize from fq_codel in 3.5 vs what's in 3.8 now, as one example.
>
> So it's my hope that folk will try to follow more closely the X.Y series
> of kernels rather than the 3.2.Z series of kernels in the future. I'm
> very happy with what happened in the 3.12 series in particular and
> look forward to work against it in the near future. The 3.13 work is
> just beginning, too.
>
> In the hope that the showing the mechanics of researching what fixes
> did land in an old stable release would help on future papers,
> here's how to look:
>
> git clone git://
> git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> git clone linux-stable linux-3.2.18
> cd linux-3.2.18
> git checkout v3.2.18
> git checkout -b ritepaper
> git log net include/net # just the networking bits, not the driver
> bits for this example
>
> BQL, enhanced SFQ, SFQRED, codel, fq_codel, tcp small queues,
> retirement of some odd tcp logic, and hundreds of other changes were
> made to the the stack since 3.2.0, and were NOT backported to 3.2.Z.
>
> Some very relevant bugfixes did indeed land in 3.2.18 that were not in 3.=
2!
>
> If you have an experiment that used TCP that is against an even older
> release, perhaps some of these major bugs might explain your results.
> Here's a sampling:
>
> commit 4b9b05fd95c502521eaef111ba0f83c58b391587
> Author: Eric Dumazet <edumazet@google.com>
> Date:   Wed May 2 02:28:41 2012 +0000
>
>     tcp: change tcp_adv_win_scale and tcp_rmem[2]
>
> <snip snip>
>
>     This also means tcp advertises a too optimistic window for a given
>     allocated rcvspace : When receiving frames, sk_rmem_alloc can hit
>     sk_rcvbuf limit and we call tcp_prune_queue()/tcp_collapse() too ofte=
n,
>     especially when application is slow to drain its receive queue or in
>     case of losses (netperf is fast, scp is slow). This is a major latenc=
y
>     source.
>
> icommit b713f6c7d317c136f03c132203d0900f4a0de084
> Author: Yuchung Cheng <ycheng@google.com>
> Date:   Mon Apr 30 06:00:18 2012 +0000
>
>     tcp: fix infinite cwnd in tcp_complete_cwr()
>
>     [ Upstream commit 1cebce36d660c83bd1353e41f3e66abd4686f215 ]
>
>     When the cwnd reduction is done, ssthresh may be infinite
>     if TCP enters CWR via ECN or F-RTO. If cwnd is not undone, i.e.,
>     undo_marker is set, tcp_complete_cwr() falsely set cwnd to the
>     infinite ssthresh value. The correct operation is to keep cwnd
>     intact because it has been updated in ECN or F-RTO.
>
> commit 65355aea86b2a70cbc7cbe14466702bc5a4e2217
> Author: Neal Cardwell <ncardwell@google.com>
> Date:   Tue Apr 10 07:59:20 2012 +0000
>
>     tcp: fix tcp_rcv_rtt_update() use of an unscaled RTT sample
>
>     [ Upstream commit 18a223e0b9ec8979320ba364b47c9772391d6d05 ]
>
>     Fix a code path in tcp_rcv_rtt_update() that was comparing scaled and
>     unscaled RTT samples.
>
>     The intent in the code was to only use the 'm' measurement if it was =
a
>     new minimum.  However, since 'm' had not yet been shifted left 3 bits
>     but 'new_sample' had, this comparison would nearly always succeed,
>     leading us to erroneously set our receive-side RTT estimate to the 'm=
'
>     sample when that sample could be nearly 8x too high to use.
>
>     The overall effect is to often cause the receive-side RTT estimate to
>     be significantly too large (up to 40% too large for brief periods in
>     my tests).
>
>
> commit 1ee5fa1e9970a16036e37c7b9d5ce81c778252fc
>
>     [PATCH] sch_red: fix red_change()
>
>     Now RED is classful, we must check q->qdisc->q.qlen, and if queue is
> empt
>     we start an idle period, not end it.
>
>
>
> --
> Dave T=E4ht
>
> Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> iccrg mailing list
> iccrg@irtf.org
> https://www.irtf.org/mailman/listinfo/iccrg
>

--20cf307f31a8ec562c04ec14c5bf
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Dave<div><br></div><div>Thanks for sharing this and gre=
at to know that you&#39;re working on replicating the &quot;AQM Kids...&quo=
t; work presented at ICCRG. jfyi we used Linux 3.10.4 on the AQM box for ou=
r real-life tests (also mentioned in Table 4, page 8 of the TR) and it shou=
ld contain most of the recent codes on the latest AQMs judging from what yo=
u&#39;ve written below. =A0 =A0 =A0<br>
<div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Cheers,</di=
v><div class=3D"gmail_extra">Naeem<br><br><div class=3D"gmail_quote">On Tue=
, Nov 26, 2013 at 2:49 AM, Dave Taht <span dir=3D"ltr">&lt;<a href=3D"mailt=
o:dave.taht@gmail.com" target=3D"_blank">dave.taht@gmail.com</a>&gt;</span>=
 wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">I am in the process of trying to reproduce t=
he recent rite papers on<br>
immediate congestion notification and the iccrg ARED work and need<br>
to explain something to researchers so that I don&#39;t have to work so har=
d.<br>
<br>
Linux kernel releases are numbered X.Y.Z-Q. All kernel versions<br>
contain bugs. Linux 3.2.0 was, IMHO, the nadir of the network stack in<br>
Linux.<br>
<br>
The &quot;X&quot; is the major version number. It&#39;s only changed 3 time=
s. &quot;Y&quot;,<br>
is the minor version number. These come out roughly quarterly and<br>
consist of new development of &quot;features&quot;. &quot;Z&quot; is critic=
al patches<br>
backported from newer releases. -Q is usually the vendor&#39;s kernel<br>
build number which often contains more patches.<br>
<br>
These numbers, clearly identified, in every academic paper on<br>
networking, and every presentation, ever published, =A0would make me a<br>
happier guy. A pointer to the git tree actually used would make me<br>
even happier, with all the patches (like DCTCP in this case) applied,<br>
would cause me to dance for joy, and sing hallelujah!<br>
<br>
Anyway, on the &quot;Z&quot; part of X.Y.Z:<br>
<br>
Periodically a &quot;long term stable&quot; release is picked and receives =
updates<br>
for as long as someone is funded to do it.<br>
<br>
*the only things that enter into a long term stable release* are fixes<br>
for security bugs, crash bugs, and truly egregious bugs that can be<br>
somewhat easily fixed.<br>
<br>
But the rest of the development goes into X.Y+1.<br>
<br>
Sane people never run a X.Y.0 release on hardware/data they care about.<br>
<br>
So... anyway... when I was told that the recent paper on DCTCP had<br>
been done against Linux 3.2.18, &quot;which came out in july, 2013!&quot; .=
..<br>
<br>
I was partially happy - pretty stable release - but<br>
my heart sank as I knew that very, very, very few of the relevant fixes<br>
for bufferbloat and the tcp stack had landed in anything prior to<br>
Linux 3.6. Those fixes had mostly qualified as &quot;features&quot;. Severa=
l in<br>
fact have been in such continuous development that I&#39;d not want to<br>
generalize from fq_codel in 3.5 vs what&#39;s in 3.8 now, as one example.<b=
r>
<br>
So it&#39;s my hope that folk will try to follow more closely the X.Y serie=
s<br>
of kernels rather than the 3.2.Z series of kernels in the future. I&#39;m<b=
r>
very happy with what happened in the 3.12 series in particular and<br>
look forward to work against it in the near future. The 3.13 work is<br>
just beginning, too.<br>
<br>
In the hope that the showing the mechanics of researching what fixes<br>
did land in an old stable release would help on future papers,<br>
here&#39;s how to look:<br>
<br>
git clone git://<a href=3D"http://git.kernel.org/pub/scm/linux/kernel/git/s=
table/linux-stable.git" target=3D"_blank">git.kernel.org/pub/scm/linux/kern=
el/git/stable/linux-stable.git</a><br>
git clone linux-stable linux-3.2.18<br>
cd linux-3.2.18<br>
git checkout v3.2.18<br>
git checkout -b ritepaper<br>
git log net include/net # just the networking bits, not the driver<br>
bits for this example<br>
<br>
BQL, enhanced SFQ, SFQRED, codel, fq_codel, tcp small queues,<br>
retirement of some odd tcp logic, and hundreds of other changes were<br>
made to the the stack since 3.2.0, and were NOT backported to 3.2.Z.<br>
<br>
Some very relevant bugfixes did indeed land in 3.2.18 that were not in 3.2!=
<br>
<br>
If you have an experiment that used TCP that is against an even older<br>
release, perhaps some of these major bugs might explain your results.<br>
Here&#39;s a sampling:<br>
<br>
commit 4b9b05fd95c502521eaef111ba0f83c58b391587<br>
Author: Eric Dumazet &lt;<a href=3D"mailto:edumazet@google.com">edumazet@go=
ogle.com</a>&gt;<br>
Date: =A0 Wed May 2 02:28:41 2012 +0000<br>
<br>
=A0 =A0 tcp: change tcp_adv_win_scale and tcp_rmem[2]<br>
<br>
&lt;snip snip&gt;<br>
<br>
=A0 =A0 This also means tcp advertises a too optimistic window for a given<=
br>
=A0 =A0 allocated rcvspace : When receiving frames, sk_rmem_alloc can hit<b=
r>
=A0 =A0 sk_rcvbuf limit and we call tcp_prune_queue()/tcp_collapse() too of=
ten,<br>
=A0 =A0 especially when application is slow to drain its receive queue or i=
n<br>
=A0 =A0 case of losses (netperf is fast, scp is slow). This is a major late=
ncy<br>
=A0 =A0 source.<br>
<br>
icommit b713f6c7d317c136f03c132203d0900f4a0de084<br>
Author: Yuchung Cheng &lt;<a href=3D"mailto:ycheng@google.com">ycheng@googl=
e.com</a>&gt;<br>
Date: =A0 Mon Apr 30 06:00:18 2012 +0000<br>
<br>
=A0 =A0 tcp: fix infinite cwnd in tcp_complete_cwr()<br>
<br>
=A0 =A0 [ Upstream commit 1cebce36d660c83bd1353e41f3e66abd4686f215 ]<br>
<br>
=A0 =A0 When the cwnd reduction is done, ssthresh may be infinite<br>
=A0 =A0 if TCP enters CWR via ECN or F-RTO. If cwnd is not undone, i.e.,<br=
>
=A0 =A0 undo_marker is set, tcp_complete_cwr() falsely set cwnd to the<br>
=A0 =A0 infinite ssthresh value. The correct operation is to keep cwnd<br>
=A0 =A0 intact because it has been updated in ECN or F-RTO.<br>
<br>
commit 65355aea86b2a70cbc7cbe14466702bc5a4e2217<br>
Author: Neal Cardwell &lt;<a href=3D"mailto:ncardwell@google.com">ncardwell=
@google.com</a>&gt;<br>
Date: =A0 Tue Apr 10 07:59:20 2012 +0000<br>
<br>
=A0 =A0 tcp: fix tcp_rcv_rtt_update() use of an unscaled RTT sample<br>
<br>
=A0 =A0 [ Upstream commit 18a223e0b9ec8979320ba364b47c9772391d6d05 ]<br>
<br>
=A0 =A0 Fix a code path in tcp_rcv_rtt_update() that was comparing scaled a=
nd<br>
=A0 =A0 unscaled RTT samples.<br>
<br>
=A0 =A0 The intent in the code was to only use the &#39;m&#39; measurement =
if it was a<br>
=A0 =A0 new minimum. =A0However, since &#39;m&#39; had not yet been shifted=
 left 3 bits<br>
=A0 =A0 but &#39;new_sample&#39; had, this comparison would nearly always s=
ucceed,<br>
=A0 =A0 leading us to erroneously set our receive-side RTT estimate to the =
&#39;m&#39;<br>
=A0 =A0 sample when that sample could be nearly 8x too high to use.<br>
<br>
=A0 =A0 The overall effect is to often cause the receive-side RTT estimate =
to<br>
=A0 =A0 be significantly too large (up to 40% too large for brief periods i=
n<br>
=A0 =A0 my tests).<br>
<br>
<br>
commit 1ee5fa1e9970a16036e37c7b9d5ce81c778252fc<br>
<br>
=A0 =A0 [PATCH] sch_red: fix red_change()<br>
<br>
=A0 =A0 Now RED is classful, we must check q-&gt;qdisc-&gt;q.qlen, and if q=
ueue is empt<br>
=A0 =A0 we start an idle period, not end it.<br>
<span class=3D"HOEnZb"><font color=3D"#888888"><br>
<br>
<br>
--<br>
Dave T=E4ht<br>
<br>
Fixing bufferbloat with cerowrt: <a href=3D"http://www.teklibre.com/cerowrt=
/subscribe.html" target=3D"_blank">http://www.teklibre.com/cerowrt/subscrib=
e.html</a><br>
_______________________________________________<br>
iccrg mailing list<br>
<a href=3D"mailto:iccrg@irtf.org">iccrg@irtf.org</a><br>
<a href=3D"https://www.irtf.org/mailman/listinfo/iccrg" target=3D"_blank">h=
ttps://www.irtf.org/mailman/listinfo/iccrg</a><br>
</font></span></blockquote></div><br></div></div></div>

--20cf307f31a8ec562c04ec14c5bf--

From ropan@cisco.com  Tue Nov 26 10:32:26 2013
Return-Path: <ropan@cisco.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 290851ADF91 for <aqm@ietfa.amsl.com>; Tue, 26 Nov 2013 10:32:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.501
X-Spam-Level: 
X-Spam-Status: No, score=-14.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZsNnB2szZwCN for <aqm@ietfa.amsl.com>; Tue, 26 Nov 2013 10:32:24 -0800 (PST)
Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by ietfa.amsl.com (Postfix) with ESMTP id 40BD11AD72A for <aqm@ietf.org>; Tue, 26 Nov 2013 10:32:24 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3980; q=dns/txt; s=iport; t=1385490744; x=1386700344; h=from:to:subject:date:message-id:in-reply-to:mime-version; bh=WEb/APKMALDeTeU6vvvUDBvI9hRi9I3uHrjZ1wfvChs=; b=EcsHrlu6lwmMY+66FDuOVD8jE42y/X2A3IPC32UgbUhV0b6hPGFuIk4E Y9UwIW9gN0cb/Uwgs3pSYVCsVcTGvxmGiK6Rk8fJLCJJ8lgY+2eTVbZOZ RXUciTWMctTEBdNuJYw5CFn9UuWh2UQufAtfHUkGI2NiIttBmPuTDi8Of M=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AvoFAP/nlFKtJXG9/2dsb2JhbABZgkNEOFOoMolriE2BLRZ0giUBAgQYcwEIBAoDAwECKDkUCQgCBAESiAENv1uOaxiEMwOYFIEwkGKDKIIq
X-IronPort-AV: E=Sophos;i="4.93,776,1378857600";  d="scan'208,217";a="284668456"
Received: from rcdn-core2-2.cisco.com ([173.37.113.189]) by rcdn-iport-9.cisco.com with ESMTP; 26 Nov 2013 18:32:24 +0000
Received: from xhc-rcd-x10.cisco.com (xhc-rcd-x10.cisco.com [173.37.183.84]) by rcdn-core2-2.cisco.com (8.14.5/8.14.5) with ESMTP id rAQIWNoQ023914 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 26 Nov 2013 18:32:23 GMT
Received: from xmb-aln-x15.cisco.com ([169.254.9.147]) by xhc-rcd-x10.cisco.com ([173.37.183.84]) with mapi id 14.03.0123.003; Tue, 26 Nov 2013 12:32:23 -0600
From: "Rong Pan (ropan)" <ropan@cisco.com>
To: Michael Welzl <michawe@ifi.uio.no>, "aqm@ietf.org" <aqm@ietf.org>
Thread-Topic: [aqm] AQM survey
Thread-Index: AQHO6o25imgN/rp0gEOFzfDb+bGRpZo3tQ4A
Date: Tue, 26 Nov 2013 18:32:22 +0000
Message-ID: <CEBA2915.563D4%ropan@cisco.com>
In-Reply-To: <EA0A6403-5722-47F1-ACC2-1771457359BB@ifi.uio.no>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [10.21.70.101]
Content-Type: multipart/alternative; boundary="_000_CEBA2915563D4ropanciscocom_"
MIME-Version: 1.0
Subject: Re: [aqm] AQM survey
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Nov 2013 18:32:26 -0000

--_000_CEBA2915563D4ropanciscocom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Very good survey paper. Thanks for sharing.

Cheers,

Rong

From: Michael Welzl <michawe@ifi.uio.no<mailto:michawe@ifi.uio.no>>
Date: Tuesday, November 26, 2013 1:55 AM
To: "aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>
Subject: [aqm] AQM survey

Hi,

I just stumbled over this, and it looks really good to me - so I thought I'=
d share it:

"Active Queue Management: A Survey" by Richelle Adams, in IEEE Comm. Survey=
s & Tutorials,
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=3D6329367
(it seems you must have IEEE Xplore access to get it, but I thought I'll sh=
are the link anyway for those of us who do)

I think the author must be forgiven for not discussing CoDel and PIE, as th=
e abstract says: "This survey attempts to travel the trajectory of AQM rese=
arch from 1993 with the first algorithm, Random Early Detection (RED), to c=
urrent work in 2011"

- even though it was published in the third quarter of 2013... that's just =
the time of academic publishing in journals / magazines.

Cheers,
Michael


--_000_CEBA2915563D4ropanciscocom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <CEF6F9335222E045A8D3BD728E7E41E6@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 16px; font-fami=
ly: Calibri, sans-serif; ">
<div>Very good survey paper. Thanks for sharing.</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Rong</div>
<div><br>
</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div style=3D"font-family:Calibri; font-size:14pt; text-align:left; color:b=
lack; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM:=
 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;=
 BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style=3D"font-weight:bold">From: </span>Michael Welzl &lt;<a href=3D"=
mailto:michawe@ifi.uio.no">michawe@ifi.uio.no</a>&gt;<br>
<span style=3D"font-weight:bold">Date: </span>Tuesday, November 26, 2013 1:=
55 AM<br>
<span style=3D"font-weight:bold">To: </span>&quot;<a href=3D"mailto:aqm@iet=
f.org">aqm@ietf.org</a>&quot; &lt;<a href=3D"mailto:aqm@ietf.org">aqm@ietf.=
org</a>&gt;<br>
<span style=3D"font-weight:bold">Subject: </span>[aqm] AQM survey<br>
</div>
<div><br>
</div>
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
Hi,
<div><br>
</div>
<div>I just stumbled over this, and it looks really good to me - so I thoug=
ht I'd share it:</div>
<div><br>
</div>
<div>&quot;Active Queue Management: A Survey&quot; by Richelle Adams, in IE=
EE Comm. Surveys &amp; Tutorials,</div>
<div><a href=3D"http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=
=3D6329367">http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=3D63=
29367</a></div>
<div>(it seems you must have IEEE Xplore access to get it, but I thought I'=
ll share the link anyway for those of us who do)</div>
<div><br>
</div>
<div>I think the author must be forgiven for not discussing CoDel and PIE, =
as the abstract says: &quot;This survey attempts to travel the trajectory o=
f AQM research from 1993 with the first algorithm, Random Early&nbsp;Detect=
ion (RED), to current work in 2011&quot;</div>
<div><br>
</div>
<div>- even though it was published in the third quarter of 2013... that's =
just the time of academic publishing in journals / magazines.</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Michael</div>
<div><br>
</div>
</div>
</div>
</span>
</body>
</html>

--_000_CEBA2915563D4ropanciscocom_--
