
From nobody Sat Jun  3 15:34:48 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 396011200F3 for <bmwg@ietfa.amsl.com>; Sat,  3 Jun 2017 15:34:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id owXa-QkGaA7Q for <bmwg@ietfa.amsl.com>; Sat,  3 Jun 2017 15:34:46 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C7EA81200C5 for <bmwg@ietf.org>; Sat,  3 Jun 2017 15:34:45 -0700 (PDT)
Received: from pps.filterd (m0049459.ppops.net [127.0.0.1]) by m0049459.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v53MYgIS039263 for <bmwg@ietf.org>; Sat, 3 Jun 2017 18:34:42 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049459.ppops.net-00191d01. with ESMTP id 2aurxyck4w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Sat, 03 Jun 2017 18:34:42 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v53MYg8D081489 for <bmwg@ietf.org>; Sat, 3 Jun 2017 17:34:42 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v53MYaCt081437 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Sat, 3 Jun 2017 17:34:36 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Sat, 3 Jun 2017 22:34:19 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v53MYIhm022994 for <bmwg@ietf.org>; Sat, 3 Jun 2017 17:34:18 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v53MY9h2022755 for <bmwg@ietf.org>; Sat, 3 Jun 2017 17:34:12 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 2FEF2EF65B for <bmwg@ietf.org>; Sat,  3 Jun 2017 18:34:09 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Sat, 3 Jun 2017 18:34:08 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQ==
Date: Sat, 3 Jun 2017 22:34:08 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-03_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706030434
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/UjmgEMwfU881583qCxB_kMi4G_Y>
Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 03 Jun 2017 22:34:47 -0000

BMWG,

At our Chicago meeting, the BMWG reviewed a version of
the Internet Draft on

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-05

There was sufficient readership present at the meeting
and positive/constructive comments on the work, so the
chairs asked whether support existed to adopt the draft
as a WG item. There was support and no objections. As
always, such decisions are officially taken on the=20
mailing list.

This message*, therefore, starts a call for adoption on:

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-07

to run until Friday 23 June 2017.=20

Please reply to bmwg@ietf.org indicating:

(1) whether you support addition of the following milestone to=20
   the BMWG charter:

date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG

(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07=20
   as the basis document for this milestone

(3) whether you commit to reviewing the WG document if adopted

We are aware that BMWG has nearly completed all its existing
chartered items.  This usually begins discussion of=20
re-chartering, this time with Warren (our new AD Advisor). The final
decision/schedule for adoption will depend on these discussions.=20

thanks for your consideration of the draft, and regards,

Al
bmwg co-chair

*using Brian Trammell's recent message to IPPM as a template (thanks!)


From nobody Mon Jun  5 12:14:08 2017
Return-Path: <ietf-secretariat-reply@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 66E3F129C4C; Mon,  5 Jun 2017 12:14:07 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: IETF Secretariat <ietf-secretariat-reply@ietf.org>
To: <bmwg-chairs@ietf.org>, <bmwg@ietf.org>, <draft-kishjac-bmwg-evpntest@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.52.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149669004741.3140.12680273543528404528.idtracker@ietfa.amsl.com>
Date: Mon, 05 Jun 2017 12:14:07 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/ICG9STsxbpVon1YWr0BHcWIpqXw>
Subject: [bmwg] The BMWG WG has placed draft-kishjac-bmwg-evpntest in state "Call For Adoption By WG Issued"
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 05 Jun 2017 19:14:07 -0000

The BMWG WG has placed draft-kishjac-bmwg-evpntest in state 
Call For Adoption By WG Issued (entered by Al Morton)

The document is available at
https://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntest/


Comment:
runs until Friday 23 June 2017


From nobody Mon Jun  5 20:30:16 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E13C7127444; Mon,  5 Jun 2017 20:30:14 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Ben Campbell <ben@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, Al Morton <acmorton@att.com>, bmwg-chairs@ietf.org, acmorton@att.com, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149671981487.3963.7734816183624252257.idtracker@ietfa.amsl.com>
Date: Mon, 05 Jun 2017 20:30:14 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/iayYg37b5dPzE2esHz7wabxch0I>
Subject: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 03:30:15 -0000

Ben Campbell has entered the following ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

-7.2, Reporting Format: Is it conventional to use 2119 keywords to describe
report formatting? (Or is this paragraph really about content, rather than
format?)

(Comment repeats for other "format" related paragraphs.)



From nobody Mon Jun  5 20:39:26 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id F1A6912EB92; Mon,  5 Jun 2017 20:39:23 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Ben Campbell <ben@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149672036394.3976.8004729920890920652.idtracker@ietfa.amsl.com>
Date: Mon, 05 Jun 2017 20:39:23 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/fN61WLTMygPjNsL5MK52B27rWeA>
Subject: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 03:39:24 -0000

Ben Campbell has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

-General:

It seems a bit odd to me to use an IETF stream RFC to describe the status of an
open source project, even when that project is closely related to IETF work.
But I do not object strongly enough to get in the way of publication.

-3.4: "It is essential to increase the flow timeout   time on a vswitch before
conducting any performance tests that do not   intend to measure the flow setup
time."

Does this mean to make allowances for the startup characteristics of virtual
network elements, when physical elements might not have such limitations?  That
seems sort of like optimizing for the test.

-4, figures:

Figure numbers and cross references would be very helpful for this section.



From nobody Tue Jun  6 07:33:49 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E58A9129B69; Tue,  6 Jun 2017 07:33:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g5_xYVKG1DJu; Tue,  6 Jun 2017 07:33:38 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 00D1D129B8B; Tue,  6 Jun 2017 07:33:37 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v56EUZut047215; Tue, 6 Jun 2017 10:33:34 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by mx0a-00191d01.pphosted.com with ESMTP id 2awx7g84cn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 06 Jun 2017 10:33:34 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v56EXUUB075680; Tue, 6 Jun 2017 09:33:31 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v56EXOlS075643 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 6 Jun 2017 09:33:25 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor); Tue, 6 Jun 2017 14:33:07 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v56EX7fk018918; Tue, 6 Jun 2017 09:33:07 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v56EWxbv018362; Tue, 6 Jun 2017 09:32:59 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 62E8DE08D4; Tue,  6 Jun 2017 10:32:58 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 6 Jun 2017 10:32:58 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Ben Campbell <ben@nostrum.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
Thread-Index: AQHS3naBJlhcXyqY3kym69nuhrGy2aIX3CYw
Date: Tue, 6 Jun 2017 14:32:57 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD60E8@njmtexg5.research.att.com>
References: <149672036394.3976.8004729920890920652.idtracker@ietfa.amsl.com>
In-Reply-To: <149672036394.3976.8004729920890920652.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-06_11:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706060244
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/HMOMJHmi3mkpy_Is2lx4FQ6dgSA>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 14:33:41 -0000

SGkgQmVuLCB0aGFua3MgZm9yIHlvdXIgcmV2aWV3LCBwbGVhc2Ugc2VlIGJlbG93Li4uDQoNCj4g
LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogQmVuIENhbXBiZWxsIFttYWlsdG86
YmVuQG5vc3RydW0uY29tXQ0KPiBTZW50OiBNb25kYXksIEp1bmUgMDUsIDIwMTcgMTE6MzkgUE0N
Cj4gVG86IFRoZSBJRVNHDQo+IENjOiBkcmFmdC1pZXRmLWJtd2ctdnN3aXRjaC1vcG5mdkBpZXRm
Lm9yZzsgU2FyYWggQmFua3M7IGJtd2ctDQo+IGNoYWlyc0BpZXRmLm9yZzsgc2JhbmtzQGVuY3J5
cHRlZC5uZXQ7IGJtd2dAaWV0Zi5vcmcNCj4gU3ViamVjdDogQmVuIENhbXBiZWxsJ3MgTm8gT2Jq
ZWN0aW9uIG9uIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LQ0KPiAwMzogKHdpdGggQ09N
TUVOVCkNCj4gDQo+IEJlbiBDYW1wYmVsbCBoYXMgZW50ZXJlZCB0aGUgZm9sbG93aW5nIGJhbGxv
dCBwb3NpdGlvbiBmb3INCj4gZHJhZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDM6IE5vIE9i
amVjdGlvbg0KLi4uDQo+IA0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IENPTU1FTlQ6DQo+IC0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0NCj4gDQo+IC1HZW5lcmFsOg0KPiANCj4gSXQgc2VlbXMgYSBiaXQgb2RkIHRvIG1lIHRv
IHVzZSBhbiBJRVRGIHN0cmVhbSBSRkMgdG8gZGVzY3JpYmUgdGhlDQo+IHN0YXR1cyBvZiBhbg0K
PiBvcGVuIHNvdXJjZSBwcm9qZWN0LCBldmVuIHdoZW4gdGhhdCBwcm9qZWN0IGlzIGNsb3NlbHkg
cmVsYXRlZCB0byBJRVRGDQo+IHdvcmsuDQo+IEJ1dCBJIGRvIG5vdCBvYmplY3Qgc3Ryb25nbHkg
ZW5vdWdoIHRvIGdldCBpbiB0aGUgd2F5IG9mIHB1YmxpY2F0aW9uLg0KW0FDTV0gDQoNCkFsdGhv
dWdoIHRoaXMgd29yayBpcyBjbGVhcmx5IGluIEJNV0cncyBjaGFydGVyLCB0aGUgV0cgaXMgbm90
IHF1aXRlDQphdCB0aGUgcG9pbnQgd2hlcmUgd2UgYXJlIHJlYWR5IHRvIHdyaXRlIG5ldyBSRkNz
LiBIb3dldmVyLCB3ZSANCmRvIGhhdmUgYSBnb29kIGNvbGxhYm9yYXRpb24gd2l0aCB0aGUgc2V2
ZXJhbCBPUE5GViBwcm9qZWN0cw0KZG9pbmcgYmVuY2htYXJraW5nLCB0aGlzIG9uZSBvbiB2U3dp
dGNoZXMgaGFzIHJlY29tbWVuZGVkIHNvbWUNCnRlc3QgbWV0aG9kcyB3aGljaCB3ZSB3aWxsIGNl
cnRhaW5seSB0YWtlIGludG8gYWNjb3VudCBpbiBmdXR1cmUgDQpkZXZlbG9wbWVudC4gIA0KDQpU
aGlzIGRyYWZ0LCBhbmQgdGhlIFZTUEVSRiBwcm9qZWN0IHdvcmsgaW4gZ2VuZXJhbCwgYW5kIEJN
V0cncw0KdW5kZXJzdGFuZGluZyBvZiB0aGlzIHRvcGljICphbGwqIGJlbmVmaXRlZA0KZnJvbSBy
ZXZpZXcgYW5kIGNvbW1lbnRzIGluIEJNV0cuIFRoaXMgaXMgb25lIGZvcm0gb2YNCg0KIi4uLmlu
dGVyYWN0aW9uIGJldHdlZW4gc3RhbmRhcmRzIGRldmVsb3BtZW50IGluIHRoZSBJRVRGLCANCmRl
dmVsb3BtZW50IG9mIHJ1bm5pbmcgY29kZSwgYW5kIG9wZW4gc291cmNlIGVmZm9ydHMgaW4gdGhl
IGluZHVzdHJ5LiINCmZyb20gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvYmxvZy8yMDE3LzA1L2llc2ct
cmV0cmVhdC0zLw0KDQpUaGUgaW50ZXJhY3Rpb24gYmV0d2VlbiBvcGVuIHNvdXJjZSBlZmZvcnRz
IGFuZCBCTVdHIGlzDQphbHNvIHByb2R1Y3RpdmUgZm9yIG90aGVyIGRyYWZ0czsgdGhlIFdHDQpp
cyBhbG1vc3QgcmVhZHkgdG8gc2hpcCB0d28gbmV3IHNwZWNpZmljYXRpb25zDQpmb3IgU0ROIENv
bnRyb2xsZXIgQmVuY2htYXJraW5nICh3b3JrZWQgd2l0aCB0aGUgT1BORlYgQ3BlcmYgcHJvamVj
dCk6DQpodHRwczovL2RhdGF0cmFja2VyLmlldGYub3JnL2RvYy9kcmFmdC1pZXRmLWJtd2ctc2Ru
LWNvbnRyb2xsZXItYmVuY2htYXJrLW1ldGgvDQpodHRwczovL2RhdGF0cmFja2VyLmlldGYub3Jn
L2RvYy9kcmFmdC1pZXRmLWJtd2ctc2RuLWNvbnRyb2xsZXItYmVuY2htYXJrLXRlcm0vDQoNCg0K
PiANCj4gLTMuNDogIkl0IGlzIGVzc2VudGlhbCB0byBpbmNyZWFzZSB0aGUgZmxvdyB0aW1lb3V0
ICAgdGltZSBvbiBhIHZzd2l0Y2gNCj4gYmVmb3JlDQo+IGNvbmR1Y3RpbmcgYW55IHBlcmZvcm1h
bmNlIHRlc3RzIHRoYXQgZG8gbm90ICAgaW50ZW5kIHRvIG1lYXN1cmUgdGhlDQo+IGZsb3cgc2V0
dXANCj4gdGltZS4iDQo+IA0KPiBEb2VzIHRoaXMgbWVhbiB0byBtYWtlIGFsbG93YW5jZXMgZm9y
IHRoZSBzdGFydHVwIGNoYXJhY3RlcmlzdGljcyBvZiB2aXJ0dWFsDQo+IG5ldHdvcmsgZWxlbWVu
dHMsIHdoZW4gcGh5c2ljYWwgZWxlbWVudHMgbWlnaHQgbm90IGhhdmUgc3VjaCBsaW1pdGF0aW9u
cz8gIFRoYXQNCj4gc2VlbXMgc29ydCBvZiBsaWtlIG9wdGltaXppbmcgZm9yIHRoZSB0ZXN0Lg0K
W0FDTV0gDQpUaGlzIGlzIGFuIGFjY2VwdGVkIGFzcGVjdCBvZiBiZW5jaG1hcmtpbmcsIGFuZCBh
cHBsaWVzIHRvIGJvdGgNCnBoeXNpY2FsIGFuZCB2aXJ0dWFsIG5ldHdvcmsgZnVuY3Rpb25zLCBz
dWNoIGFzIHN3aXRjaGVzIGFuZCByb3V0ZXJzLg0KRm9yIGV4YW1wbGUsIFJGQyAyODg5IHNheXM6
DQogICAiIC4uLiBUaGUgRFVUL1NVVCBhZGRyZXNzIGFnaW5nIHRpbWUgU0hPVUxEIGJlIGNvbmZp
Z3VyZWQgdG8gYmUgZ3JlYXRlciB0aGFuDQogICB0aGUgcGVyaW9kIG9mIHRoZSBsZWFybmluZyBw
aGFzZSBvZiB0aGUgdGVzdCBwbHVzIHRoZSB0cmlhbCBkdXJhdGlvbg0KICAgcGx1cyBhbnkgY29u
ZmlndXJhdGlvbiB0aW1lIHJlcXVpcmVkIGJ5IHRoZSB0ZXN0aW5nIGRldmljZS4iDQpTZWU6DQpo
dHRwczovL3Rvb2xzLmlldGYub3JnL2h0bWwvcmZjMjg4OSNzZWN0aW9uLTMNClRoZSBsZWFybmlu
ZyBwaGFzZSBpcyBiZW5jaG1hcmtlZCBzZXBhcmF0ZWx5Lg0KDQo+IA0KPiAtNCwgZmlndXJlczoN
Cj4gDQo+IEZpZ3VyZSBudW1iZXJzIGFuZCBjcm9zcyByZWZlcmVuY2VzIHdvdWxkIGJlIHZlcnkg
aGVscGZ1bCBmb3IgdGhpcw0KPiBzZWN0aW9uLg0KPiANCltBQ01dIE9LLCB3ZSdsbCBhZGQgdGhl
bS4NCg0KdGhhbmtzIGFnYWluLA0KQWwNCg0K


From nobody Tue Jun  6 11:05:00 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4E1BA129411; Tue,  6 Jun 2017 11:04:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.721
X-Spam-Level: 
X-Spam-Status: No, score=-2.721 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cooperw.in header.b=l7KC1Vwi; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=jtqzZXTo
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nA5tchgF-9wy; Tue,  6 Jun 2017 11:04:56 -0700 (PDT)
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9FCAC1292F4; Tue,  6 Jun 2017 11:04:56 -0700 (PDT)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id EB2D22076A; Tue,  6 Jun 2017 14:04:55 -0400 (EDT)
Received: from frontend1 ([10.202.2.160]) by compute7.internal (MEProxy); Tue, 06 Jun 2017 14:04:55 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cooperw.in; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=frOc2CFXSiDjQL3LsN DZ5E4FTXTiCbnPnSh4eSqD0eg=; b=l7KC1VwiAnzGOxqPzWnB5gQ680RQNV60x5 dObXe0pkcs1XvReBm+EuC1bJoNcPkhYrWgWDuT03fcE19ze3VuWijMxl7ioH0n+x u9aRnLh2+uxXk7TQn4Bibr+j+KkG655Z96IDS0Kfv6r1AKWMDCc6Jd+3w0djVE35 dr7BvLXqECyOUanLaS3nSavdsDYgiB8aoTILTwibKzvdhKNzkj8AZwmulI7626Ax oKUR0nS22W8kkcah87yyWt05KhLIsjgMM9BHAYoIEDbou3ypcNU/6K+w1/K0LUji L3J43pcAaXGA6WUIqCTRoqdKOS7poxgrBlrsoniwqDCUPi5/l2aw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=frOc2CFXSiDjQL3LsNDZ5E4FTXTiCbnPnSh4eSqD0eg=; b=jtqzZXTo bj7prWGbmr49nceK5jF0lwi4bWPcIQJPKg/9/MXUNaZSqOOZWEjLtTdePTRgBafu atpn9IcFpYoERTVr2O8ksYn/owQztjzQ/amTo4VPYpeFHU6AnHJtZZu8dWPN00yU li4aFD9HWJWujnea+Iz/4CZojtWqE5E2CvSfE25z1QIdP76ojC408+WbCllBWWhR 6pPsMjbEDiZGdAhGtJ0qoRvy7ANkn51kCsC0zg00Dhj5hkKyrryv/nlUC0O31eC0 lhfG61xXPXy6iAJMlgySI3OeT1OChSmZtI84YOBRm8JPcclJ2t83850jw3IXtJTp rxmQoEYCg7yhIA==
X-ME-Sender: <xms:x-42WYuGBmImfehElORAcnYkfkWjRJA7H4rIJ2Zk5b6uAQjTfPmynQ>
X-Sasl-enc: PICX0PZRV/3R05hpJ+ZxfBePLSFYm8MGVNmKUz3vG+lf 1496772295
Received: from dhcp-10-150-9-191.cisco.com (unknown [173.38.117.92]) by mail.messagingengine.com (Postfix) with ESMTPA id 7D32F7E814; Tue,  6 Jun 2017 14:04:55 -0400 (EDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Alissa Cooper <alissa@cooperw.in>
In-Reply-To: <149615590409.14768.3879167079373046273@ietfa.amsl.com>
Date: Tue, 6 Jun 2017 14:04:54 -0400
Cc: "gen-art >> General area reviewing team" <gen-art@ietf.org>, draft-ietf-bmwg-ipv6-tran-tech-benchmarking.all@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <A6F08465-3199-406C-8A1A-2FAA03ADEFD9@cooperw.in>
References: <149615590409.14768.3879167079373046273@ietfa.amsl.com>
To: Robert Sparks <rjsparks@nostrum.com>
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/SnO7QT2s6RCDtRfMqqW_uzoJA0Q>
Subject: Re: [bmwg] [Gen-art] Genart telechat review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 18:04:58 -0000

Robert, thank you for your reviews. Authors, thanks for addressing =
Robert=E2=80=99s comments. I entered a No Objection ballot position.

Alissa


> On May 30, 2017, at 10:51 AM, Robert Sparks <rjsparks@nostrum.com> =
wrote:
>=20
> Reviewer: Robert Sparks
> Review result: Ready
>=20
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair. Please wait for direction from your
> document shepherd or AD before posting a new version of the draft.
>=20
> For more information, please see the FAQ at
>=20
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>=20
> Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08
> Reviewer: Robert Sparks
> Review Date: 2017-05-30
> IETF LC End Date: 2017-05-02
> IESG Telechat date: 2017-06-08
>=20
> Summary: Ready for publication as an Informational RFC
>=20
> Thanks for addressing my comments on -06
>=20
> _______________________________________________
> Gen-art mailing list
> Gen-art@ietf.org
> https://www.ietf.org/mailman/listinfo/gen-art


From nobody Tue Jun  6 11:12:58 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93A76127BA3; Tue,  6 Jun 2017 11:12:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.72
X-Spam-Level: 
X-Spam-Status: No, score=-2.72 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cooperw.in header.b=vplGmdHt; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Za9v/Egn
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dJDNoppfk77b; Tue,  6 Jun 2017 11:12:53 -0700 (PDT)
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D6D4F1242F5; Tue,  6 Jun 2017 11:12:52 -0700 (PDT)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id DD4F520821; Tue,  6 Jun 2017 14:12:51 -0400 (EDT)
Received: from frontend1 ([10.202.2.160]) by compute7.internal (MEProxy); Tue, 06 Jun 2017 14:12:51 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cooperw.in; h=cc :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=fm1; bh=gx8qZYoKR+PMwo8CqWAogQbUUT4n/maXg2H+ImIx9 VI=; b=vplGmdHt+abCapLr1QDlpmOXkFH961rBRpf1muIu/Bxq9bk+nZzO62YuX mjW7hlC2FnhogS3HkqEV/ylen+WqSPty1QUCxHeulGGyJ4D5lTtFZNeMjzUMLCcl 3uD7L67RTB4UFNQZXz6G9dL+wD/YLKUbBSWcFKUlEb941ja3wXMdMMlmFNVc2tn3 QGSVxoj/0ctHADbw8jLpgWGYqbsbw9iYGyy9UK0aoenqmSrRnXS+f0yp7IAk3XbL panCH8xvGDGL9ZBJYk9YYoYEG3rf4QXz3cQWAf+ETkDi1oQif7UR6JAqcjzP5Q6h 3ixzCUrY3MMMAnzwDlWGSRK8oqmhw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=gx8qZYoKR+PMwo8CqW AogQbUUT4n/maXg2H+ImIx9VI=; b=Za9v/EgndVMKDcTBo+kh4uLAOuULVhEbxu 6JMYsby6OleNXd7OnUGxtuyPMA4sd4GKUdpS3y6DAGFlq0FHsk9PBBJwJ65lTTXU 63H+TM9f2ZkljnqUD5NyCdhsofZaL4Ta9avNTTqsG0BZWtz8ynnpgnBhA2FcT7Z0 kd8ro8Plcf2SQjQbn7WAknABdXumbRHpYV3auz+sEHu5vHSUeQaiuJ5pCEjaeRht m/LIX4peD8Hcx5U7I7k9fjRTSniP2Rb6N8yVfqe6pGZTELf6gWIqA8WdMvqLVKfT 6Q1eSIv3Pue2GAEKoCOaPhp1KOpLAU+sA2RxP+dODqoXo5gXE7xA==
X-ME-Sender: <xms:o_A2WX0_ahMtPfMfozKAUm_DycEHqTaIgsapABnVd9NbDz6RiJEl6Q>
X-Sasl-enc: cQ9Uq0c5zZYD+e2cdxlNqNCpUa/+HORSx/4U+gVTixSH 1496772771
Received: from dhcp-10-150-9-191.cisco.com (unknown [173.38.117.92]) by mail.messagingengine.com (Postfix) with ESMTPA id BE1397E7B1; Tue,  6 Jun 2017 14:12:50 -0400 (EDT)
Content-Type: multipart/alternative; boundary="Apple-Mail=_534FDD10-75CE-4BB3-9794-DF6F12FD0D9D"
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Alissa Cooper <alissa@cooperw.in>
In-Reply-To: <CAFgnS4UAfMe5L7k=zwCqfUjgz14ArBpTE_bz675dCf0_Jk21fg@mail.gmail.com>
Date: Tue, 6 Jun 2017 14:12:50 -0400
Cc: "gen-art@ietf.org" <gen-art@ietf.org>, "ietf@ietf.org" <ietf@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>, Dan Romascanu <dromasca@gmail.com>
Message-Id: <21148194-B135-4FF6-9283-5459830671B0@cooperw.in>
References: <149450075072.16690.14546662616864459158@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25F9B072@njmtexg4.research.att.com> <CAFgnS4UAfMe5L7k=zwCqfUjgz14ArBpTE_bz675dCf0_Jk21fg@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" <draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/1yOs90Th4F8eop2LKr3uHcLRN58>
Subject: Re: [bmwg] [Gen-art] Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 18:12:56 -0000

--Apple-Mail=_534FDD10-75CE-4BB3-9794-DF6F12FD0D9D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Authors/shepherd/WG: any further responses to Dan?

Thanks,
Alissa


> On May 11, 2017, at 2:52 PM, Dan Romascanu <dromasca@gmail.com> wrote:
>=20
> Hi,=20
>=20
> Please see in-line.=20
>=20
> Regards,
>=20
> Dan
>=20
>=20
> On Thu, May 11, 2017 at 8:00 PM, MORTON, ALFRED C (AL) =
<acmorton@att.com <mailto:acmorton@att.com>> wrote:
> Hi Dan,
> please see replies, [ACM], below.
>=20
> > -----Original Message-----
> > From: Dan Romascanu [mailto:dromasca@gmail.com =
<mailto:dromasca@gmail.com>]
> > Sent: Thursday, May 11, 2017 7:06 AM
> > To: gen-art@ietf.org <mailto:gen-art@ietf.org>
> > Cc: draft-ietf-bmwg-vswitch-opnfv.all@ietf.org =
<mailto:draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>; ietf@ietf.org =
<mailto:ietf@ietf.org>;
> > bmwg@ietf.org <mailto:bmwg@ietf.org>; dromasca@gmail.com =
<mailto:dromasca@gmail.com>
> > Subject: Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
> >
> > Reviewer: Dan Romascanu
> > Review result: Almost Ready
> >
> > I am the assigned Gen-ART reviewer for this draft. The General Area
> > Review Team (Gen-ART) reviews all IETF documents being processed
> > by the IESG for the IETF Chair.  Please treat these comments just
> > like any other last call comments.
> >
> > For more information, please see the FAQ at
> >
> > <https://urldefense.proofpoint.com/v2/url?u=3Dhttps- =
<https://urldefense.proofpoint.com/v2/url?u=3Dhttps->
> > 3A__trac.ietf.org_trac_gen_wiki_GenArtfaq&d=3DDwICaQ&c=3DLFYZ-
> > o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DNTVlLBN-
> > L3u9zGPHm_CNVcXW7_OGX8_18CtaAalZin0&s=3D2Hr-
> > dhKaDHIguY7W97z33RlKjqPDtmoYmM2-jWrbS-o&e=3D >.
> >
> > Document: draft-ietf-bmwg-vswitch-opnfv-03
> > Reviewer: Dan Romascanu
> > Review Date: 2017-05-11
> > IETF LC End Date: 2017-05-15
> > IESG Telechat date: Not scheduled for a telechat
> >
> > Summary:
> >
> > Almost Ready.
> >
> > This document describes describes the progress of the Open Platform
> > for NFV (OPNFV) project on virtual switch performance "VSPERF". That
> > project reuses the BMWG framework and specifications to benchmark
> > virtual switches implemented in general-purpose hardware. Some
> > differences with the benchmarking of specialized HW platforms are
> > identified and they may become work items for BMWG in the future. =
It's
> > a well written and clear document, but I have reservations about it
> > being published as an RFC, and I cannot find coverage for it in the =
WG
> > charter. I also have concerns that parts of the methodology used by
> > OPNFV break the BMWG principles, especially repeatability and
> > 'black-box', and this is not clear enough articulated in the =
document.
> [ACM]
> Ok, let's address your specific issues, and come back to your =
reservations.
>=20
> >
> >
> > Major issues:
> >
> > 1. It is not clear to me why this document needs to be published as =
an
> > RFC. The introduction says: 'This memo describes the progress of the
> > Open Platform for NFV (OPNFV) project on virtual switch performance
> > "VSPERF".  This project intends to build on the current and =
completed
> > work of the Benchmarking Methodology Working Group in IETF, by
> > referencing existing literature.' Why should the WG and the IESG
> > invest resources in publishing this, why an I-D or an Independent
> > Stream RFC is not sufficient?
> [ACM]
> The WG considered and discussed this document over 3 revisions
> and a year of time before reaching consensus to develop it further
> as a chartered item, so this decision was not taken lightly.
> See more below.
>=20
> > The WG charter says something about:
> > 'VNF and Related Infrastructure Benchmarking: Benchmarking
> > Methodologies have reliably characterized many physical devices. =
This
> > work item extends and enhances the methods to virtual network
> > functions (VNF) and their unique supporting infrastructure. A first
> > deliverable from this activity will be a document that considers the
> > new benchmarking space to ensure that common issues are recognized
> > from the start, using background materials from industry and SDOs
> > (e.g., IETF, ETSI NFV).'. I do not believe that this document covers
> > the intent of the charter, as it focused on one organization only.
> [ACM]
> I'm sorry, but here you are mistaken. The document that satisfied
> the "first deliverable ... document that considers the new =
benchmarking space"
> is: https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05>
> titled: Considerations for Benchmarking Virtual Network Functions and =
Their Infrastructure
> which has been submitted to IESG and approved for publication.
> Further, the current draft (draft-ietf-bmwg-vswitch-opnfv-03)
> references the approved "Considerations" draft in Section 3
> (as does almost every related Industry spec I'm aware of).
>=20
> The BMWG Charter continues:
>   Benchmarks for platform capacity and performance characteristics of
>   virtual routers, switches, and related components will follow, =
including
>   comparisons between physical and virtual network functions. In many =
cases,
>   the traditional benchmarks should be applicable to VNFs, but the lab
>   set-ups, configurations, and measurement methods will likely need to
>   be revised or enhanced.
>=20
> This draft constitutes one of several follow-on efforts, approaching
> the problem exactly as we described in the last sentence above.
>=20
> How? What last sentence?=20
>=20
> Is it:=20
>=20
> 'In many cases,
>   the traditional benchmarks should be applicable to VNFs, but the lab
>   set-ups, configurations, and measurement methods will likely need to
>   be revised or enhanced.'
>=20
> How does this document approach this problem? If there is a need to =
revise or enhance existing BMWG work, what is needed is specific =
revisions of documents. This informational document only documents work =
in one external organization. I have reservations that this is a WG task =
to advance this document, and of the IESG to approve it. Why can't it =
stand as an I-D until the WG decides whatever work needs to be =
undertaken (if any) to meet the OPNFV needs? Or if they with to have an =
RFC, why can't it be Independent Stream?=20
>=20
> Will the WG write similar documents for all (or several) other =
organizations that implement VNFs one way or another? Should it?
>=20
>=20
> An aspect of Industry collaboration that we did not anticipate in the
> BMWG Charter is our current interactions with Open Source Communities.
> The current Charter was approved in June 2014, then OPNFV was founded
> on September 30, 2014 [0] and the VSPERF Project was created on
> Dec 16, 2014, so we did not anticipate extensive collaboration on
> this and other benchmarking topics.
>=20
>=20
>=20
> >
> > 2. In section 3 there 'repeatability' is mentioned, while
> > acknowledging that in a virtual environment there is no guarantee =
and
> > actually no way to know what other applications are being run.
> [ACM]
> See:
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.4 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.4>
>=20
> There are certainly ways to assess the current set of processes
> at a particular time. The Software configuration parameters in
> Section 3.3 are intended to capture this aspect as part of set-up.
> At the same time, there will be challenges to assess the DUT
> performance when resources are fully shared, and new testing
> strategies will be needed:
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3>
>=20
>=20
> > Measuring parameters as the ones listed in 3.3 provides just part of
> > the answer, and they are internal parameters to the SUT.
> [ACM]
> Yes, knowing the tested configuration is a critical pillar
> supporting repeatability (these items are not measured, but =
configured),
> and why we provided this section.
>=20
> > Also, the
> > different deployment scenarios in section 4 require different
> > configurations for the SUT, thus breaking the 'black-box' principle.
> [ACM]
> Specifying DUT configuration does not break any part of the
> black-box principle, which establishes that benchmark measurements
> will be based on externally observable phenomena. See:
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.2 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.2>
>=20
> Previous BMWG RFCs have identified the critical configuration
> parameters of the DUT, such as the number and type of
> network interfaces, the arrangement of DUTs in a SUT, etc.
>=20
> I may have not been clear enough. The document talks about =
repeatability and about comparing benchmarks with the ones of =
specialized HW implementations. Then it goes into a long but still =
partial list of factors that can influence the benchmarking, the =
majority of them depend on the HW and SW measurements and parameters of =
the internal systems. How can this be compared with a number of small =
and indeed externally observable configuration parameters like the =
number and type of network interfaces. This is several degrees of =
magnitude apart in complexity. =20
>=20
> > I believe that there is a need for a more clear explanation of why =
BMWG
> > specifications are appropriate and how comparison can be made while
> > repeatability cannot be ensured, and measurements are dependent upon
> > parameters internal to the SUT.
> [ACM]
> I believe that draft-ietf-bmwg-virtual-net-05 already
> indicates why the existing BMWG RFCs are a reasonable
> starting place for NFV benchmarks, in part because
> we want to measure the same benchmarks of physical
> network functions in many cases. See
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.1 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.1>
>=20
> Repeatability is a goal of all experiments, and we understand
> that there is more work to do in this regard, but what
> we know now (documented in this draft) should
> be a valuable contribution to the Industry.
>=20
>=20
> Measuring the same benchmarks is a good goal. I believe that the claim =
of repeatability needs to be better argued.=20
> =20
>=20
> >
> > Minor issues:
> >
> > 1. Some of the tests mentioned in Section 4 have no prior or in
> > progress work in the IETF: Control Path and Datapath Coupling Tests,
> > Noisy Neighbour Tests, characterization of acceleration =
technologies.
> [ACM]
> I'm sorry, but that's not an accurate portrayal of BMWG's literature.
>=20
> https://tools.ietf.org/html/rfc6413 =
<https://tools.ietf.org/html/rfc6413> examined Control Plane/Dataplane
> interactions, for example.
>=20
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3>
> item 2 specifically included Noisy Neighbour among the new
> testing strategies.
>=20
>=20
> Please provide references for each in text.=20
> =20
> Every network interface with an ASIC is an example of acceleration,
> one that we've characterized in physical network devices for years.
>=20
>=20
> Yes, but we do not deal here with externally observable interfaces =
only, and if the characterization of the acceleration technologies =
matters than you need a way to express it (where can we find it in =
existing BMWG work? new work?)
> =20
> > If new work is needed / proposed to be added for the BMWG scope and
> > framework it would be useful for BMWG to list these separately.
> >
> >
> > Nits/editorial comments:
> >
> > 1. What is called 'Deployment scenarios' from VS perspective in
> > Section 4 describe in fact different configurations of the SUT in =
BMWG
> > terms. It seems better to separate this second part of section 4 in =
a
> > separate section. If it belongs to an existing section it rather
> > belongs in 3 than in 4.
> >
> [ACM]
> Section 3 is more about extending the configuration guidance
> from =
https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.2 =
<https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.2>
>=20
> Section 4 summarizes the VSPERF Level Test Design document,
> of which these deployment scenarios are a key part.
>=20
> Yes, but this seams to belong to configurations of SUTs, even if they =
are called 'Deployment scenarios' in OPNFV-speak, and they impact =
repeatibility.=20
>  =20
>=20
> thanks for your comments; hopefully this detailed reply
> will reduce your reservations about publication.
>=20
> Al
> (for the co-authors)
>=20
> [0] =
https://www.opnfv.org/announcements/2014/09/30/telecom-industry-and-vendor=
s-unite-to-build-common-open-platform-to-accelerate-network-functions-virt=
ualization =
<https://www.opnfv.org/announcements/2014/09/30/telecom-industry-and-vendo=
rs-unite-to-build-common-open-platform-to-accelerate-network-functions-vir=
tualization>
>=20
>=20
> _______________________________________________
> Gen-art mailing list
> Gen-art@ietf.org
> https://www.ietf.org/mailman/listinfo/gen-art


--Apple-Mail=_534FDD10-75CE-4BB3-9794-DF6F12FD0D9D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">Authors/shepherd/WG: any further responses to =
Dan?</div><div class=3D""><br class=3D""></div><div =
class=3D"">Thanks,</div><div class=3D"">Alissa</div><div class=3D""><br =
class=3D""></div><br class=3D""><div><blockquote type=3D"cite" =
class=3D""><div class=3D"">On May 11, 2017, at 2:52 PM, Dan Romascanu =
&lt;<a href=3D"mailto:dromasca@gmail.com" =
class=3D"">dromasca@gmail.com</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"ltr" =
class=3D""><div class=3D""><div class=3D""><div class=3D"">Hi, <br =
class=3D""><br class=3D""></div>Please see in-line. <br class=3D""><br =
class=3D""></div>Regards,<br class=3D""><br class=3D""></div>Dan<br =
class=3D""><br class=3D""><div class=3D""><div class=3D""><div =
class=3D""><div class=3D"gmail_extra"><br class=3D""><div =
class=3D"gmail_quote">On Thu, May 11, 2017 at 8:00 PM, MORTON, ALFRED C =
(AL) <span dir=3D"ltr" class=3D"">&lt;<a href=3D"mailto:acmorton@att.com" =
target=3D"_blank" class=3D"">acmorton@att.com</a>&gt;</span> wrote:<br =
class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi =
Dan,<br class=3D"">
please see replies, [ACM], below.<br class=3D"">
<span class=3D"gmail-"><br class=3D"">
&gt; -----Original Message-----<br class=3D"">
&gt; From: Dan Romascanu [mailto:<a href=3D"mailto:dromasca@gmail.com" =
class=3D"">dromasca@gmail.com</a>]<br class=3D"">
&gt; Sent: Thursday, May 11, 2017 7:06 AM<br class=3D"">
&gt; To: <a href=3D"mailto:gen-art@ietf.org" =
class=3D"">gen-art@ietf.org</a><br class=3D"">
&gt; Cc: <a href=3D"mailto:draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" =
class=3D"">draft-ietf-bmwg-vswitch-opnfv.<wbr class=3D"">all@ietf.org</a>;=
 <a href=3D"mailto:ietf@ietf.org" class=3D"">ietf@ietf.org</a>;<br =
class=3D"">
&gt; <a href=3D"mailto:bmwg@ietf.org" class=3D"">bmwg@ietf.org</a>; <a =
href=3D"mailto:dromasca@gmail.com" class=3D"">dromasca@gmail.com</a><br =
class=3D"">
&gt; Subject: Genart last call review of =
draft-ietf-bmwg-vswitch-opnfv-<wbr class=3D"">03<br class=3D"">
&gt;<br class=3D"">
&gt; Reviewer: Dan Romascanu<br class=3D"">
&gt; Review result: Almost Ready<br class=3D"">
&gt;<br class=3D"">
&gt; I am the assigned Gen-ART reviewer for this draft. The General =
Area<br class=3D"">
&gt; Review Team (Gen-ART) reviews all IETF documents being processed<br =
class=3D"">
&gt; by the IESG for the IETF Chair.&nbsp; Please treat these comments =
just<br class=3D"">
&gt; like any other last call comments.<br class=3D"">
&gt;<br class=3D"">
&gt; For more information, please see the FAQ at<br class=3D"">
&gt;<br class=3D"">
</span>&gt; &lt;<a =
href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-" =
rel=3D"noreferrer" target=3D"_blank" class=3D"">https://urldefense.<wbr =
class=3D"">proofpoint.com/v2/url?u=3Dhttps-</a><br class=3D"">
&gt; 3A__trac.ietf.org_trac_gen_<wbr =
class=3D"">wiki_GenArtfaq&amp;d=3DDwICaQ&amp;c=3D<wbr class=3D"">LFYZ-<br =
class=3D"">
&gt; o9_HUMeMTSQicvjIg&amp;r=3D<wbr =
class=3D"">OfsSu8kTIltVyD1oL72cBw&amp;m=3D<wbr class=3D"">NTVlLBN-<br =
class=3D"">
&gt; L3u9zGPHm_CNVcXW7_OGX8_<wbr class=3D"">18CtaAalZin0&amp;s=3D2Hr-<br =
class=3D"">
&gt; dhKaDHIguY7W97z33RlKjqPDtmoYmM<wbr class=3D"">2-jWrbS-o&amp;e=3D =
&gt;.<br class=3D"">
&gt;<br class=3D"">
&gt; Document: draft-ietf-bmwg-vswitch-opnfv-<wbr class=3D"">03<br =
class=3D"">
<span class=3D"gmail-">&gt; Reviewer: Dan Romascanu<br class=3D"">
&gt; Review Date: 2017-05-11<br class=3D"">
&gt; IETF LC End Date: 2017-05-15<br class=3D"">
&gt; IESG Telechat date: Not scheduled for a telechat<br class=3D"">
&gt;<br class=3D"">
&gt; Summary:<br class=3D"">
&gt;<br class=3D"">
&gt; Almost Ready.<br class=3D"">
&gt;<br class=3D"">
&gt; This document describes describes the progress of the Open =
Platform<br class=3D"">
&gt; for NFV (OPNFV) project on virtual switch performance "VSPERF". =
That<br class=3D"">
&gt; project reuses the BMWG framework and specifications to =
benchmark<br class=3D"">
&gt; virtual switches implemented in general-purpose hardware. Some<br =
class=3D"">
&gt; differences with the benchmarking of specialized HW platforms =
are<br class=3D"">
&gt; identified and they may become work items for BMWG in the future. =
It's<br class=3D"">
&gt; a well written and clear document, but I have reservations about =
it<br class=3D"">
&gt; being published as an RFC, and I cannot find coverage for it in the =
WG<br class=3D"">
&gt; charter. I also have concerns that parts of the methodology used =
by<br class=3D"">
&gt; OPNFV break the BMWG principles, especially repeatability and<br =
class=3D"">
&gt; 'black-box', and this is not clear enough articulated in the =
document.<br class=3D"">
</span>[ACM]<br class=3D"">
Ok, let's address your specific issues, and come back to your =
reservations.<br class=3D"">
<span class=3D"gmail-"><br class=3D"">
&gt;<br class=3D"">
&gt;<br class=3D"">
&gt; Major issues:<br class=3D"">
&gt;<br class=3D"">
&gt; 1. It is not clear to me why this document needs to be published as =
an<br class=3D"">
&gt; RFC. The introduction says: 'This memo describes the progress of =
the<br class=3D"">
&gt; Open Platform for NFV (OPNFV) project on virtual switch =
performance<br class=3D"">
&gt; "VSPERF".&nbsp; This project intends to build on the current and =
completed<br class=3D"">
&gt; work of the Benchmarking Methodology Working Group in IETF, by<br =
class=3D"">
&gt; referencing existing literature.' Why should the WG and the IESG<br =
class=3D"">
&gt; invest resources in publishing this, why an I-D or an =
Independent<br class=3D"">
&gt; Stream RFC is not sufficient?<br class=3D"">
</span>[ACM]<br class=3D"">
The WG considered and discussed this document over 3 revisions<br =
class=3D"">
and a year of time before reaching consensus to develop it further<br =
class=3D"">
as a chartered item, so this decision was not taken lightly.<br =
class=3D"">
See more below.<br class=3D"">
<span class=3D"gmail-"><br class=3D"">
&gt; The WG charter says something about:<br class=3D"">
&gt; 'VNF and Related Infrastructure Benchmarking: Benchmarking<br =
class=3D"">
&gt; Methodologies have reliably characterized many physical devices. =
This<br class=3D"">
&gt; work item extends and enhances the methods to virtual network<br =
class=3D"">
&gt; functions (VNF) and their unique supporting infrastructure. A =
first<br class=3D"">
&gt; deliverable from this activity will be a document that considers =
the<br class=3D"">
&gt; new benchmarking space to ensure that common issues are =
recognized<br class=3D"">
&gt; from the start, using background materials from industry and =
SDOs<br class=3D"">
&gt; (e.g., IETF, ETSI NFV).'. I do not believe that this document =
covers<br class=3D"">
&gt; the intent of the charter, as it focused on one organization =
only.<br class=3D"">
</span>[ACM]<br class=3D"">
I'm sorry, but here you are mistaken. The document that satisfied<br =
class=3D"">
the "first deliverable ... document that considers the new benchmarking =
space"<br class=3D"">
is: <a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05"=
 rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-05</a><br class=3D"">
titled: Considerations for Benchmarking Virtual Network Functions and =
Their Infrastructure<br class=3D"">
which has been submitted to IESG and approved for publication.<br =
class=3D"">
Further, the current draft (draft-ietf-bmwg-vswitch-<wbr =
class=3D"">opnfv-03)<br class=3D"">
references the approved "Considerations" draft in Section 3<br class=3D"">=

(as does almost every related Industry spec I'm aware of).<br class=3D"">
<br class=3D"">
The BMWG Charter continues:<br class=3D"">
&nbsp; Benchmarks for platform capacity and performance characteristics =
of<br class=3D"">
&nbsp; virtual routers, switches, and related components will follow, =
including<br class=3D"">
&nbsp; comparisons between physical and virtual network functions. In =
many cases,<br class=3D"">
&nbsp; the traditional benchmarks should be applicable to VNFs, but the =
lab<br class=3D"">
&nbsp; set-ups, configurations, and measurement methods will likely need =
to<br class=3D"">
&nbsp; be revised or enhanced.<br class=3D"">
<br class=3D"">
This draft constitutes one of several follow-on efforts, approaching<br =
class=3D"">
the problem exactly as we described in the last sentence above.<br =
class=3D""></blockquote><div class=3D""><br class=3D""></div><div =
class=3D"">How? What last sentence? <br class=3D""><br =
class=3D""></div><div class=3D"">Is it: <br class=3D""></div><div =
class=3D""><br class=3D"">'In many cases,<br class=3D"">
&nbsp; the traditional benchmarks should be applicable to VNFs, but the =
lab<br class=3D"">
&nbsp; set-ups, configurations, and measurement methods will likely need =
to<br class=3D"">
&nbsp; be revised or enhanced.'<br class=3D""><br class=3D""></div><div =
class=3D"">How does this document approach this problem? If there is a =
need to revise or enhance existing BMWG work, what is needed is specific =
revisions of documents. This informational document only documents work =
in one external organization. I have reservations that this is a WG task =
to advance this document, and of the IESG to approve it. Why can't it =
stand as an I-D until the WG decides whatever work needs to be =
undertaken (if any) to meet the OPNFV needs? Or if they with to have an =
RFC, why can't it be Independent Stream? <br class=3D""><br =
class=3D""></div><div class=3D"">Will the WG write similar documents for =
all (or several) other organizations that implement VNFs one way or =
another? Should it?<br class=3D""></div><div class=3D""><br class=3D""> =
</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br class=3D"">
An aspect of Industry collaboration that we did not anticipate in the<br =
class=3D"">
BMWG Charter is our current interactions with Open Source =
Communities.<br class=3D"">
The current Charter was approved in June 2014, then OPNFV was founded<br =
class=3D"">
on September 30, 2014 [0] and the VSPERF Project was created on<br =
class=3D"">
Dec 16, 2014, so we did not anticipate extensive collaboration on<br =
class=3D"">
this and other benchmarking topics.<br class=3D"">
<span class=3D"gmail-"><br class=3D"">
<br class=3D"">
<br class=3D"">
&gt;<br class=3D"">
&gt; 2. In section 3 there 'repeatability' is mentioned, while<br =
class=3D"">
&gt; acknowledging that in a virtual environment there is no guarantee =
and<br class=3D"">
&gt; actually no way to know what other applications are being run.<br =
class=3D"">
</span>[ACM]<br class=3D"">
See:<br class=3D"">
<a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section=
-3.4" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-<wbr =
class=3D"">05#section-3.4</a><br class=3D"">
<br class=3D"">
There are certainly ways to assess the current set of processes<br =
class=3D"">
at a particular time. The Software configuration parameters in<br =
class=3D"">
Section 3.3 are intended to capture this aspect as part of set-up.<br =
class=3D"">
At the same time, there will be challenges to assess the DUT<br =
class=3D"">
performance when resources are fully shared, and new testing<br =
class=3D"">
strategies will be needed:<br class=3D"">
<a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section=
-3.3" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-<wbr =
class=3D"">05#section-3.3</a><br class=3D"">
<span class=3D"gmail-"><br class=3D"">
<br class=3D"">
&gt; Measuring parameters as the ones listed in 3.3 provides just part =
of<br class=3D"">
&gt; the answer, and they are internal parameters to the SUT.<br =
class=3D"">
</span>[ACM]<br class=3D"">
Yes, knowing the tested configuration is a critical pillar<br class=3D"">
supporting repeatability (these items are not measured, but =
configured),<br class=3D"">
and why we provided this section.<br class=3D"">
<span class=3D"gmail-"><br class=3D"">
&gt; Also, the<br class=3D"">
&gt; different deployment scenarios in section 4 require different<br =
class=3D"">
&gt; configurations for the SUT, thus breaking the 'black-box' =
principle.<br class=3D"">
</span>[ACM]<br class=3D"">
Specifying DUT configuration does not break any part of the<br class=3D"">=

black-box principle, which establishes that benchmark measurements<br =
class=3D"">
will be based on externally observable phenomena. See:<br class=3D"">
<a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section=
-4.2" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-<wbr =
class=3D"">05#section-4.2</a><br class=3D"">
<br class=3D"">
Previous BMWG RFCs have identified the critical configuration<br =
class=3D"">
parameters of the DUT, such as the number and type of<br class=3D"">
network interfaces, the arrangement of DUTs in a SUT, etc.<br =
class=3D""></blockquote><div class=3D""><br class=3D""></div><div =
class=3D"">I may have not been clear enough. The document talks about =
repeatability and about comparing benchmarks with the ones of =
specialized HW implementations. Then it goes into a long but still =
partial list of factors that can influence the benchmarking, the =
majority of them depend on the HW and SW measurements and parameters of =
the internal systems. How can this be compared with a number of small =
and indeed externally observable configuration parameters like the =
number and type of network interfaces. This is several degrees of =
magnitude apart in complexity.&nbsp; <br class=3D""></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex">
<span class=3D"gmail-"><br class=3D"">
&gt; I believe that there is a need for a more clear explanation of why =
BMWG<br class=3D"">
&gt; specifications are appropriate and how comparison can be made =
while<br class=3D"">
&gt; repeatability cannot be ensured, and measurements are dependent =
upon<br class=3D"">
&gt; parameters internal to the SUT.<br class=3D"">
</span>[ACM]<br class=3D"">
I believe that draft-ietf-bmwg-virtual-net-05 already<br class=3D"">
indicates why the existing BMWG RFCs are a reasonable<br class=3D"">
starting place for NFV benchmarks, in part because<br class=3D"">
we want to measure the same benchmarks of physical<br class=3D"">
network functions in many cases. See<br class=3D"">
<a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section=
-4.1" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-<wbr =
class=3D"">05#section-4.1</a><br class=3D"">
<br class=3D"">
Repeatability is a goal of all experiments, and we understand<br =
class=3D"">
that there is more work to do in this regard, but what<br class=3D"">
we know now (documented in this draft) should<br class=3D"">
be a valuable contribution to the Industry.<br class=3D"">
<span class=3D"gmail-"><br class=3D""></span></blockquote><div =
class=3D""><br class=3D""></div><div class=3D"">Measuring the same =
benchmarks is a good goal. I believe that the claim of repeatability =
needs to be better argued. <br class=3D"">&nbsp;<br =
class=3D""></div><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><span class=3D"gmail-">
<br class=3D"">
&gt;<br class=3D"">
&gt; Minor issues:<br class=3D"">
&gt;<br class=3D"">
&gt; 1. Some of the tests mentioned in Section 4 have no prior or in<br =
class=3D"">
&gt; progress work in the IETF: Control Path and Datapath Coupling =
Tests,<br class=3D"">
&gt; Noisy Neighbour Tests, characterization of acceleration =
technologies.<br class=3D"">
</span>[ACM]<br class=3D"">
I'm sorry, but that's not an accurate portrayal of BMWG's literature.<br =
class=3D"">
<br class=3D"">
<a href=3D"https://tools.ietf.org/html/rfc6413" rel=3D"noreferrer" =
target=3D"_blank" class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">rfc6413</a> examined Control Plane/Dataplane<br class=3D"">
interactions, for example.<br class=3D"">
<br class=3D"">
<a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section=
-3.3" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-<wbr =
class=3D"">05#section-3.3</a><br class=3D"">
item 2 specifically included Noisy Neighbour among the new<br class=3D"">
testing strategies.<br class=3D"">
<br class=3D""></blockquote><div class=3D""><br class=3D""></div><div =
class=3D"">Please provide references for each in text. <br =
class=3D"">&nbsp;<br class=3D""></div><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex">
Every network interface with an ASIC is an example of acceleration,<br =
class=3D"">
one that we've characterized in physical network devices for years.<br =
class=3D"">
<span class=3D"gmail-"><br class=3D""></span></blockquote><div =
class=3D""><br class=3D""></div><div class=3D"">Yes, but we do not deal =
here with externally observable interfaces only, and if the =
characterization of the acceleration technologies matters than you need =
a way to express it (where can we find it in existing BMWG work? new =
work?)<br class=3D"">&nbsp;<br class=3D""></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><span class=3D"gmail-">
&gt; If new work is needed / proposed to be added for the BMWG scope =
and<br class=3D"">
&gt; framework it would be useful for BMWG to list these separately.<br =
class=3D"">
&gt;<br class=3D"">
&gt;<br class=3D"">
&gt; Nits/editorial comments:<br class=3D"">
&gt;<br class=3D"">
&gt; 1. What is called 'Deployment scenarios' from VS perspective in<br =
class=3D"">
&gt; Section 4 describe in fact different configurations of the SUT in =
BMWG<br class=3D"">
&gt; terms. It seems better to separate this second part of section 4 in =
a<br class=3D"">
&gt; separate section. If it belongs to an existing section it rather<br =
class=3D"">
&gt; belongs in 3 than in 4.<br class=3D"">
&gt;<br class=3D"">
</span>[ACM]<br class=3D"">
Section 3 is more about extending the configuration guidance<br =
class=3D"">
from <a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section=
-3.2" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://tools.ietf.org/html/<wbr =
class=3D"">draft-ietf-bmwg-virtual-net-<wbr =
class=3D"">05#section-3.2</a><br class=3D"">
<br class=3D"">
Section 4 summarizes the VSPERF Level Test Design document,<br class=3D"">=

of which these deployment scenarios are a key part.<br =
class=3D""></blockquote><div class=3D""><br class=3D""></div><div =
class=3D"">Yes, but this seams to belong to configurations of SUTs, even =
if they are called 'Deployment scenarios' in OPNFV-speak, and they =
impact repeatibility. <br class=3D"">&nbsp; <br =
class=3D""></div><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br class=3D"">
thanks for your comments; hopefully this detailed reply<br class=3D"">
will reduce your reservations about publication.<br class=3D"">
<br class=3D"">
Al<br class=3D"">
(for the co-authors)<br class=3D"">
<br class=3D"">
[0] <a =
href=3D"https://www.opnfv.org/announcements/2014/09/30/telecom-industry-an=
d-vendors-unite-to-build-common-open-platform-to-accelerate-network-functi=
ons-virtualization" rel=3D"noreferrer" target=3D"_blank" =
class=3D"">https://www.opnfv.org/<wbr =
class=3D"">announcements/2014/09/30/<wbr =
class=3D"">telecom-industry-and-vendors-<wbr =
class=3D"">unite-to-build-common-open-<wbr =
class=3D"">platform-to-accelerate-<wbr class=3D"">network-functions-<wbr =
class=3D"">virtualization</a><br class=3D"">
<br class=3D"">
</blockquote></div><br class=3D""></div></div></div></div></div>
_______________________________________________<br class=3D"">Gen-art =
mailing list<br class=3D""><a href=3D"mailto:Gen-art@ietf.org" =
class=3D"">Gen-art@ietf.org</a><br =
class=3D"">https://www.ietf.org/mailman/listinfo/gen-art<br =
class=3D""></div></blockquote></div><br class=3D""></body></html>=

--Apple-Mail=_534FDD10-75CE-4BB3-9794-DF6F12FD0D9D--


From nobody Tue Jun  6 13:29:31 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8081129534 for <bmwg@ietfa.amsl.com>; Tue,  6 Jun 2017 13:29:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oRjnLT3y5Szz for <bmwg@ietfa.amsl.com>; Tue,  6 Jun 2017 13:29:26 -0700 (PDT)
Received: from mail-ua0-x234.google.com (mail-ua0-x234.google.com [IPv6:2607:f8b0:400c:c08::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 81EF11286B2 for <bmwg@ietf.org>; Tue,  6 Jun 2017 13:29:26 -0700 (PDT)
Received: by mail-ua0-x234.google.com with SMTP id x47so98461704uab.0 for <bmwg@ietf.org>; Tue, 06 Jun 2017 13:29:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ivK+tqRUc1yRwdFWJXrbWJKo4XldWDBk5aT2SmoLTgU=; b=MnB/DLARGBn4C50Ftj1NZ/U6CKoR+PzZijHSuj+Xi5lYArAiCojvdHYPHHjyeo4GTD hMTCCH+BS+IlBvhksPpaUqRU7lfmkLwgkZbvUFPK71voqht9QYzV31fvIo0/5fAiNRBZ qM4YVkVZj8nO0a1n2hOL8gTwjPrtwgY7y1CO0eDTNilSV2CF9+IbM28awnO1CbejgyKZ LSAUWul4Aa4/4VwPGhGiAEofft7xYIZeSU5ozsMvqJlZFx6A3S0YNAzxg6Yzb6Kty9eL GTzg8tx4t70IimLnNrH60Qr5A3yVp/OZMWm1JDIi8P0GunLFy/fR4mFF47QhUgZ7oToV bdlA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ivK+tqRUc1yRwdFWJXrbWJKo4XldWDBk5aT2SmoLTgU=; b=cKdfI8FSgef0fX/QMx32vP85+cH4C1ZX7xvCjfSEUxVgx/bAotKtak5PLazqVEp1r4 ktSRw6//dkHx7d7mX3W2OAGwVZBvJC9WujK0slcnrZVSRE+Wt7svfbyCIPQ0XGfcLpPd 2HMiCF8wtMCCbLeI1XLIO1nQcpD7OG5o9fA/i5rHACzx3eBp0/m9kigX0Omo1OcAAchc hbOJohAMj0r6wOVPDNLOlJVPnd6xdJulvSySDBf9ZGyzZiLOABAh5EURz0RFyQ0661re hUwEEs8fY1GqafeIWDo4qcTLA6HgscHGIiHcKdZwqOBgCa6xXxaenmBHXAAj7bvvad1y 02Og==
X-Gm-Message-State: AODbwcB7G4xHT8HpYpvIFfrG2rEmsZVXkx9D4gH+NXiJNFhxJJVqdNsh d7uSkO11tEeXWb3tlR5t/VsnI+JYasl3
X-Received: by 10.159.60.162 with SMTP id s34mr3715981uai.89.1496780965569; Tue, 06 Jun 2017 13:29:25 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.82.198 with HTTP; Tue, 6 Jun 2017 13:28:45 -0700 (PDT)
In-Reply-To: <CAFgnS4UAfMe5L7k=zwCqfUjgz14ArBpTE_bz675dCf0_Jk21fg@mail.gmail.com>
References: <149450075072.16690.14546662616864459158@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25F9B072@njmtexg4.research.att.com> <CAFgnS4UAfMe5L7k=zwCqfUjgz14ArBpTE_bz675dCf0_Jk21fg@mail.gmail.com>
From: Warren Kumari <warren@kumari.net>
Date: Tue, 6 Jun 2017 16:28:45 -0400
Message-ID: <CAHw9_iKZxmmYszcketrLwLQOq8wnVWZceLF=eXTudepODoBRHQ@mail.gmail.com>
To: Dan Romascanu <dromasca@gmail.com>
Cc: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "gen-art@ietf.org" <gen-art@ietf.org>,  "draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" <draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>,  "ietf@ietf.org" <ietf@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/m0VbWD8nKHudhxLLehTkUJKSP0U>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 20:29:30 -0000

On Thu, May 11, 2017 at 2:52 PM, Dan Romascanu <dromasca@gmail.com> wrote:
> Hi,
>
> Please see in-line.
>
> Regards,
>
> Dan
>
>
> On Thu, May 11, 2017 at 8:00 PM, MORTON, ALFRED C (AL) <acmorton@att.com>
> wrote:
>>
>> Hi Dan,
>> please see replies, [ACM], below.
>>
>> > -----Original Message-----
>> > From: Dan Romascanu [mailto:dromasca@gmail.com]
>> > Sent: Thursday, May 11, 2017 7:06 AM
>> > To: gen-art@ietf.org
>> > Cc: draft-ietf-bmwg-vswitch-opnfv.all@ietf.org; ietf@ietf.org;
>> > bmwg@ietf.org; dromasca@gmail.com
>> > Subject: Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
>> >
>> > Reviewer: Dan Romascanu
>> > Review result: Almost Ready
>> >
>> > I am the assigned Gen-ART reviewer for this draft. The General Area
>> > Review Team (Gen-ART) reviews all IETF documents being processed
>> > by the IESG for the IETF Chair.  Please treat these comments just
>> > like any other last call comments.
>> >
>> > For more information, please see the FAQ at
>> >
>> > <https://urldefense.proofpoint.com/v2/url?u=https-
>> > 3A__trac.ietf.org_trac_gen_wiki_GenArtfaq&d=DwICaQ&c=LFYZ-
>> > o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=NTVlLBN-
>> > L3u9zGPHm_CNVcXW7_OGX8_18CtaAalZin0&s=2Hr-
>> > dhKaDHIguY7W97z33RlKjqPDtmoYmM2-jWrbS-o&e= >.
>> >
>> > Document: draft-ietf-bmwg-vswitch-opnfv-03
>> > Reviewer: Dan Romascanu
>> > Review Date: 2017-05-11
>> > IETF LC End Date: 2017-05-15
>> > IESG Telechat date: Not scheduled for a telechat
>> >
>> > Summary:
>> >
>> > Almost Ready.
>> >
>> > This document describes describes the progress of the Open Platform
>> > for NFV (OPNFV) project on virtual switch performance "VSPERF". That
>> > project reuses the BMWG framework and specifications to benchmark
>> > virtual switches implemented in general-purpose hardware. Some
>> > differences with the benchmarking of specialized HW platforms are
>> > identified and they may become work items for BMWG in the future. It's
>> > a well written and clear document, but I have reservations about it
>> > being published as an RFC, and I cannot find coverage for it in the WG
>> > charter. I also have concerns that parts of the methodology used by
>> > OPNFV break the BMWG principles, especially repeatability and
>> > 'black-box', and this is not clear enough articulated in the document.
>> [ACM]
>> Ok, let's address your specific issues, and come back to your
>> reservations.
>>
>> >
>> >
>> > Major issues:
>> >
>> > 1. It is not clear to me why this document needs to be published as an
>> > RFC. The introduction says: 'This memo describes the progress of the
>> > Open Platform for NFV (OPNFV) project on virtual switch performance
>> > "VSPERF".  This project intends to build on the current and completed
>> > work of the Benchmarking Methodology Working Group in IETF, by
>> > referencing existing literature.' Why should the WG and the IESG
>> > invest resources in publishing this, why an I-D or an Independent
>> > Stream RFC is not sufficient?
>> [ACM]
>> The WG considered and discussed this document over 3 revisions
>> and a year of time before reaching consensus to develop it further
>> as a chartered item, so this decision was not taken lightly.
>> See more below.
>>
>> > The WG charter says something about:
>> > 'VNF and Related Infrastructure Benchmarking: Benchmarking
>> > Methodologies have reliably characterized many physical devices. This
>> > work item extends and enhances the methods to virtual network
>> > functions (VNF) and their unique supporting infrastructure. A first
>> > deliverable from this activity will be a document that considers the
>> > new benchmarking space to ensure that common issues are recognized
>> > from the start, using background materials from industry and SDOs
>> > (e.g., IETF, ETSI NFV).'. I do not believe that this document covers
>> > the intent of the charter, as it focused on one organization only.
>> [ACM]
>> I'm sorry, but here you are mistaken. The document that satisfied
>> the "first deliverable ... document that considers the new benchmarking
>> space"
>> is: https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05
>> titled: Considerations for Benchmarking Virtual Network Functions and
>> Their Infrastructure
>> which has been submitted to IESG and approved for publication.
>> Further, the current draft (draft-ietf-bmwg-vswitch-opnfv-03)
>> references the approved "Considerations" draft in Section 3
>> (as does almost every related Industry spec I'm aware of).
>>
>> The BMWG Charter continues:
>>   Benchmarks for platform capacity and performance characteristics of
>>   virtual routers, switches, and related components will follow, including
>>   comparisons between physical and virtual network functions. In many
>> cases,
>>   the traditional benchmarks should be applicable to VNFs, but the lab
>>   set-ups, configurations, and measurement methods will likely need to
>>   be revised or enhanced.
>>
>> This draft constitutes one of several follow-on efforts, approaching
>> the problem exactly as we described in the last sentence above.
>
>
> How? What last sentence?
>
> Is it:
>
> 'In many cases,
>   the traditional benchmarks should be applicable to VNFs, but the lab
>   set-ups, configurations, and measurement methods will likely need to
>   be revised or enhanced.'
>
> How does this document approach this problem? If there is a need to revise
> or enhance existing BMWG work, what is needed is specific revisions of
> documents. This informational document only documents work in one external
> organization. I have reservations that this is a WG task to advance this
> document, and of the IESG to approve it. Why can't it stand as an I-D until
> the WG decides whatever work needs to be undertaken (if any) to meet the
> OPNFV needs? Or if they with to have an RFC, why can't it be Independent
> Stream?
>
> Will the WG write similar documents for all (or several) other organizations
> that implement VNFs one way or another? Should it?
>

Apologies for the delay, which is my almost entirely fault.

>From the Shepherds write-up:
"There has been a fair amount of work done on this draft, and progress
made on revisions, feedback, and comments. Several presentations have
been made in the room during IETF meetings, and followup and
discussion taken to the BMWG list. This draft is particularly useful,
given the popularity of VNF's within the industry."

I believe that there is value in the IETF publishing this as an
Informational document - the document provides useful information for
the Internet community (especially those folk benchmarking VNFs :-)).
The OPNFV virtual switch performance characterization work is very
closely related and relevant to the BMWG work, and I think that having
a collaboration type document (written fom the IETF viewpoint, and
informing IETF participants) is useful.
This *could* have been an Independent Stream doc, but it was discussed
and worked on in the WG, and so having it be a WG document feels much
more correct to me.

W

>>
>> An aspect of Industry collaboration that we did not anticipate in the
>> BMWG Charter is our current interactions with Open Source Communities.
>> The current Charter was approved in June 2014, then OPNFV was founded
>> on September 30, 2014 [0] and the VSPERF Project was created on
>> Dec 16, 2014, so we did not anticipate extensive collaboration on
>> this and other benchmarking topics.
>>
>>
>>
>> >
>> > 2. In section 3 there 'repeatability' is mentioned, while
>> > acknowledging that in a virtual environment there is no guarantee and
>> > actually no way to know what other applications are being run.
>> [ACM]
>> See:
>> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.4
>>
>> There are certainly ways to assess the current set of processes
>> at a particular time. The Software configuration parameters in
>> Section 3.3 are intended to capture this aspect as part of set-up.
>> At the same time, there will be challenges to assess the DUT
>> performance when resources are fully shared, and new testing
>> strategies will be needed:
>> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3
>>
>>
>> > Measuring parameters as the ones listed in 3.3 provides just part of
>> > the answer, and they are internal parameters to the SUT.
>> [ACM]
>> Yes, knowing the tested configuration is a critical pillar
>> supporting repeatability (these items are not measured, but configured),
>> and why we provided this section.
>>
>> > Also, the
>> > different deployment scenarios in section 4 require different
>> > configurations for the SUT, thus breaking the 'black-box' principle.
>> [ACM]
>> Specifying DUT configuration does not break any part of the
>> black-box principle, which establishes that benchmark measurements
>> will be based on externally observable phenomena. See:
>> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.2
>>
>> Previous BMWG RFCs have identified the critical configuration
>> parameters of the DUT, such as the number and type of
>> network interfaces, the arrangement of DUTs in a SUT, etc.
>
>
> I may have not been clear enough. The document talks about repeatability and
> about comparing benchmarks with the ones of specialized HW implementations.
> Then it goes into a long but still partial list of factors that can
> influence the benchmarking, the majority of them depend on the HW and SW
> measurements and parameters of the internal systems. How can this be
> compared with a number of small and indeed externally observable
> configuration parameters like the number and type of network interfaces.
> This is several degrees of magnitude apart in complexity.
>>
>>
>> > I believe that there is a need for a more clear explanation of why BMWG
>> > specifications are appropriate and how comparison can be made while
>> > repeatability cannot be ensured, and measurements are dependent upon
>> > parameters internal to the SUT.
>> [ACM]
>> I believe that draft-ietf-bmwg-virtual-net-05 already
>> indicates why the existing BMWG RFCs are a reasonable
>> starting place for NFV benchmarks, in part because
>> we want to measure the same benchmarks of physical
>> network functions in many cases. See
>> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.1
>>
>> Repeatability is a goal of all experiments, and we understand
>> that there is more work to do in this regard, but what
>> we know now (documented in this draft) should
>> be a valuable contribution to the Industry.
>>
>
> Measuring the same benchmarks is a good goal. I believe that the claim of
> repeatability needs to be better argued.
>
>>
>>
>> >
>> > Minor issues:
>> >
>> > 1. Some of the tests mentioned in Section 4 have no prior or in
>> > progress work in the IETF: Control Path and Datapath Coupling Tests,
>> > Noisy Neighbour Tests, characterization of acceleration technologies.
>> [ACM]
>> I'm sorry, but that's not an accurate portrayal of BMWG's literature.
>>
>> https://tools.ietf.org/html/rfc6413 examined Control Plane/Dataplane
>> interactions, for example.
>>
>> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3
>> item 2 specifically included Noisy Neighbour among the new
>> testing strategies.
>>
>
> Please provide references for each in text.
>
>>
>> Every network interface with an ASIC is an example of acceleration,
>> one that we've characterized in physical network devices for years.
>>
>
> Yes, but we do not deal here with externally observable interfaces only, and
> if the characterization of the acceleration technologies matters than you
> need a way to express it (where can we find it in existing BMWG work? new
> work?)
>
>>
>> > If new work is needed / proposed to be added for the BMWG scope and
>> > framework it would be useful for BMWG to list these separately.
>> >
>> >
>> > Nits/editorial comments:
>> >
>> > 1. What is called 'Deployment scenarios' from VS perspective in
>> > Section 4 describe in fact different configurations of the SUT in BMWG
>> > terms. It seems better to separate this second part of section 4 in a
>> > separate section. If it belongs to an existing section it rather
>> > belongs in 3 than in 4.
>> >
>> [ACM]
>> Section 3 is more about extending the configuration guidance
>> from
>> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.2
>>
>> Section 4 summarizes the VSPERF Level Test Design document,
>> of which these deployment scenarios are a key part.
>
>
> Yes, but this seams to belong to configurations of SUTs, even if they are
> called 'Deployment scenarios' in OPNFV-speak, and they impact repeatibility.
>
>>
>>
>> thanks for your comments; hopefully this detailed reply
>> will reduce your reservations about publication.
>>
>> Al
>> (for the co-authors)
>>
>> [0]
>> https://www.opnfv.org/announcements/2014/09/30/telecom-industry-and-vendors-unite-to-build-common-open-platform-to-accelerate-network-functions-virtualization
>>
>



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Tue Jun  6 14:25:31 2017
Return-Path: <akatlas@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 74F5D1286AB; Tue,  6 Jun 2017 14:25:29 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alia Atlas <akatlas@gmail.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, Al Morton <acmorton@att.com>, bmwg-chairs@ietf.org, acmorton@att.com, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149678432947.3901.17980895572189082124.idtracker@ietfa.amsl.com>
Date: Tue, 06 Jun 2017 14:25:29 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Obxlt7FfBYZGbAi39DvWUsC_yHg>
Subject: [bmwg] Alia Atlas' No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 21:25:29 -0000

Alia Atlas has entered the following ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I note that this document discusses using jumbo Ethernet frames up.
The IEEE has a standard now for baby giant Ethernet frames - though
there is still concern about the EtherType value used.   An informative
reference might be helpful.



From nobody Tue Jun  6 14:36:50 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8954D128BB6; Tue,  6 Jun 2017 14:36:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xiowxCwAxQPv; Tue,  6 Jun 2017 14:36:42 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5C1761271FD; Tue,  6 Jun 2017 14:36:37 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net [127.0.0.1]) by m0048589.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v56LPAfo014462; Tue, 6 Jun 2017 17:36:34 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0048589.ppops.net-00191d01. with ESMTP id 2ax44y0g4g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 06 Jun 2017 17:36:34 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v56LaX39127853; Tue, 6 Jun 2017 16:36:33 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v56LaR6R127795 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 6 Jun 2017 16:36:29 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor); Tue, 6 Jun 2017 21:36:14 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v56LaEo1027081; Tue, 6 Jun 2017 16:36:14 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v56La8vV026676; Tue, 6 Jun 2017 16:36:08 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id BF9A0E10A5; Tue,  6 Jun 2017 17:36:02 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 6 Jun 2017 17:36:07 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Alia Atlas <akatlas@gmail.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org" <draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alia Atlas' No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
Thread-Index: AQHS3wtvTLFTluiu50OTEFS0QnH2RqIYWsbQ
Date: Tue, 6 Jun 2017 21:36:07 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD63EF@njmtexg5.research.att.com>
References: <149678432947.3901.17980895572189082124.idtracker@ietfa.amsl.com>
In-Reply-To: <149678432947.3901.17980895572189082124.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-06_14:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706060372
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/jpui3F4MIYhTejxvyXn2PinM3cQ>
Subject: Re: [bmwg] Alia Atlas' No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jun 2017 21:36:44 -0000

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBBbGlhIEF0bGFzIFttYWlsdG86
YWthdGxhc0BnbWFpbC5jb21dDQo+IFNlbnQ6IFR1ZXNkYXksIEp1bmUgMDYsIDIwMTcgNToyNSBQ
TQ0KPiBUbzogVGhlIElFU0cNCi4uLg0KPiANCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiBDT01NRU5UOg0K
PiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tDQo+IA0KPiBJIG5vdGUgdGhhdCB0aGlzIGRvY3VtZW50IGRpc2N1c3Nl
cyB1c2luZyBqdW1ibyBFdGhlcm5ldCBmcmFtZXMgdXAuDQo+IFRoZSBJRUVFIGhhcyBhIHN0YW5k
YXJkIG5vdyBmb3IgYmFieSBnaWFudCBFdGhlcm5ldCBmcmFtZXMgLSB0aG91Z2gNCj4gdGhlcmUg
aXMgc3RpbGwgY29uY2VybiBhYm91dCB0aGUgRXRoZXJUeXBlIHZhbHVlIHVzZWQuICAgQW4gaW5m
b3JtYXRpdmUNCj4gcmVmZXJlbmNlIG1pZ2h0IGJlIGhlbHBmdWwuDQo+IA0KW0FDTV0gDQpIaSBB
bGlhLCB0aGFua3MgZm9yIHlvdXIgcmV2aWV3Lg0KDQpXb3VsZCB5b3UgbGlrZSB0byBzdWdnZXN0
IHRoZSBtb3N0IGFwcHJvcHJpYXRlIHJlZmVyZW5jZT8NCkkgZG9uJ3QgZm9sbG93IElFRUUgODAy
IGF0IGFsbCBub3cuDQoNCnRoYW5rcywNCkFsDQoNCg0K


From nobody Tue Jun  6 18:48:40 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5DFAD129564; Tue,  6 Jun 2017 18:48:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TUnk6VoGK47r; Tue,  6 Jun 2017 18:48:16 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 39458129576; Tue,  6 Jun 2017 18:48:16 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v571j541028010; Tue, 6 Jun 2017 21:48:11 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by mx0a-00191d01.pphosted.com with ESMTP id 2ax37e6t9x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 06 Jun 2017 21:48:10 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v571m9H9103516; Tue, 6 Jun 2017 20:48:09 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v571m3Bx103503 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 6 Jun 2017 20:48:03 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor); Wed, 7 Jun 2017 01:47:47 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v571lliM017316; Tue, 6 Jun 2017 20:47:47 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v571lhOR017126; Tue, 6 Jun 2017 20:47:43 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id 62EDEE1097; Tue,  6 Jun 2017 21:47:37 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 6 Jun 2017 21:47:42 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Warren Kumari <warren@kumari.net>, "alissa@cooperw.in" <alissa@cooperw.in>
CC: "gen-art@ietf.org" <gen-art@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" <draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>, "ietf@ietf.org" <ietf@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>, "dromasca@gmail.com" <dromasca@gmail.com>
Thread-Topic: Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
Thread-Index: AQHSykaQpvZtcVQqDEiJ+tr7bcWy3KHvK1EQgACSnQCAKPeQgP//5Opg
Date: Wed, 7 Jun 2017 01:47:41 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD64E2@njmtexg5.research.att.com>
References: <149450075072.16690.14546662616864459158@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25F9B072@njmtexg4.research.att.com> <CAFgnS4UAfMe5L7k=zwCqfUjgz14ArBpTE_bz675dCf0_Jk21fg@mail.gmail.com> <CAHw9_iKZxmmYszcketrLwLQOq8wnVWZceLF=eXTudepODoBRHQ@mail.gmail.com>
In-Reply-To: <CAHw9_iKZxmmYszcketrLwLQOq8wnVWZceLF=eXTudepODoBRHQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-06_15:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706070030
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/fekSF_Fv9_lstBtVQ8XCpLeHOMQ>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 01:48:20 -0000

QWxpc3NhIGFuZCBXYXJyZW4sDQoNCkFmdGVyIFdhcnJlbidzIG1lc3NhZ2UsIGEgZmV3IG1vcmUg
cmVwbGllcyBpbi1saW5lLCBiZWxvdywNCg0KQWwgKGZvciB0aGUgY28tYXV0aG9ycykNCj4gLS0t
LS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogV2FycmVuIEt1bWFyaSBbbWFpbHRvOndh
cnJlbkBrdW1hcmkubmV0XQ0KPiBTZW50OiBUdWVzZGF5LCBKdW5lIDA2LCAyMDE3IDQ6MjkgUE0N
Cj4gVG86IERhbiBSb21hc2NhbnUNCj4gQ2M6IE1PUlRPTiwgQUxGUkVEIEMgKEFMKTsgZ2VuLWFy
dEBpZXRmLm9yZzsgZHJhZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtDQo+IG9wbmZ2LmFsbEBpZXRmLm9y
ZzsgaWV0ZkBpZXRmLm9yZzsgYm13Z0BpZXRmLm9yZw0KPiBTdWJqZWN0OiBSZTogR2VuYXJ0IGxh
c3QgY2FsbCByZXZpZXcgb2YgZHJhZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDMNCj4gDQo+
IE9uIFRodSwgTWF5IDExLCAyMDE3IGF0IDI6NTIgUE0sIERhbiBSb21hc2NhbnUgPGRyb21hc2Nh
QGdtYWlsLmNvbT4NCj4gd3JvdGU6DQo+ID4gSGksDQo+ID4NCj4gPiBQbGVhc2Ugc2VlIGluLWxp
bmUuDQo+ID4NCj4gPiBSZWdhcmRzLA0KPiA+DQo+ID4gRGFuDQo+ID4NCj4gPg0KPiA+IE9uIFRo
dSwgTWF5IDExLCAyMDE3IGF0IDg6MDAgUE0sIE1PUlRPTiwgQUxGUkVEIEMgKEFMKQ0KPiA8YWNt
b3J0b25AYXR0LmNvbT4NCj4gPiB3cm90ZToNCj4gPj4NCj4gPj4gSGkgRGFuLA0KPiA+PiBwbGVh
c2Ugc2VlIHJlcGxpZXMsIFtBQ01dLCBiZWxvdy4NCj4gPj4NCj4gPj4gPiAtLS0tLU9yaWdpbmFs
IE1lc3NhZ2UtLS0tLQ0KPiA+PiA+IEZyb206IERhbiBSb21hc2NhbnUgW21haWx0bzpkcm9tYXNj
YUBnbWFpbC5jb21dDQo+ID4+ID4gU2VudDogVGh1cnNkYXksIE1heSAxMSwgMjAxNyA3OjA2IEFN
DQo+ID4+ID4gVG86IGdlbi1hcnRAaWV0Zi5vcmcNCj4gPj4gPiBDYzogZHJhZnQtaWV0Zi1ibXdn
LXZzd2l0Y2gtb3BuZnYuYWxsQGlldGYub3JnOyBpZXRmQGlldGYub3JnOw0KPiA+PiA+IGJtd2dA
aWV0Zi5vcmc7IGRyb21hc2NhQGdtYWlsLmNvbQ0KPiA+PiA+IFN1YmplY3Q6IEdlbmFydCBsYXN0
IGNhbGwgcmV2aWV3IG9mIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LQ0KPiAwMw0KPiA+
PiA+DQo+ID4+ID4gUmV2aWV3ZXI6IERhbiBSb21hc2NhbnUNCj4gPj4gPiBSZXZpZXcgcmVzdWx0
OiBBbG1vc3QgUmVhZHkNCj4gPj4gPg0KPiA+PiA+IEkgYW0gdGhlIGFzc2lnbmVkIEdlbi1BUlQg
cmV2aWV3ZXIgZm9yIHRoaXMgZHJhZnQuIFRoZSBHZW5lcmFsIEFyZWENCj4gPj4gPiBSZXZpZXcg
VGVhbSAoR2VuLUFSVCkgcmV2aWV3cyBhbGwgSUVURiBkb2N1bWVudHMgYmVpbmcgcHJvY2Vzc2Vk
DQo+ID4+ID4gYnkgdGhlIElFU0cgZm9yIHRoZSBJRVRGIENoYWlyLiAgUGxlYXNlIHRyZWF0IHRo
ZXNlIGNvbW1lbnRzIGp1c3QNCj4gPj4gPiBsaWtlIGFueSBvdGhlciBsYXN0IGNhbGwgY29tbWVu
dHMuDQo+ID4+ID4NCj4gPj4gPiBGb3IgbW9yZSBpbmZvcm1hdGlvbiwgcGxlYXNlIHNlZSB0aGUg
RkFRIGF0DQo+ID4+ID4NCj4gPj4gPiA8aHR0cHM6Ly91cmxkZWZlbnNlLnByb29mcG9pbnQuY29t
L3YyL3VybD91PWh0dHBzLT4+ID4NCj4gM0FfX3RyYWMuaWV0Zi5vcmdfdHJhY19nZW5fd2lraV9H
ZW5BcnRmYXEmZD1Ed0lDYVEmYz1MRllaLQ0KPiA+PiA+IG85X0hVTWVNVFNRaWN2aklnJnI9T2Zz
U3U4a1RJbHRWeUQxb0w3MmNCdyZtPU5UVmxMQk4tDQo+ID4+ID4gTDN1OXpHUEhtX0NOVmNYVzdf
T0dYOF8xOEN0YUFhbFppbjAmcz0ySHItDQo+ID4+ID4gZGhLYURISWd1WTdXOTd6MzNSbEtqcVBE
dG1vWW1NMi1qV3JiUy1vJmU9ID4uDQo+ID4+ID4NCj4gPj4gPiBEb2N1bWVudDogZHJhZnQtaWV0
Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDMNCj4gPj4gPiBSZXZpZXdlcjogRGFuIFJvbWFzY2FudQ0K
PiA+PiA+IFJldmlldyBEYXRlOiAyMDE3LTA1LTExDQo+ID4+ID4gSUVURiBMQyBFbmQgRGF0ZTog
MjAxNy0wNS0xNQ0KPiA+PiA+IElFU0cgVGVsZWNoYXQgZGF0ZTogTm90IHNjaGVkdWxlZCBmb3Ig
YSB0ZWxlY2hhdA0KPiA+PiA+DQo+ID4+ID4gU3VtbWFyeToNCj4gPj4gPg0KPiA+PiA+IEFsbW9z
dCBSZWFkeS4NCj4gPj4gPg0KPiA+PiA+IFRoaXMgZG9jdW1lbnQgZGVzY3JpYmVzIGRlc2NyaWJl
cyB0aGUgcHJvZ3Jlc3Mgb2YgdGhlIE9wZW4gUGxhdGZvcm0NCj4gPj4gPiBmb3IgTkZWIChPUE5G
VikgcHJvamVjdCBvbiB2aXJ0dWFsIHN3aXRjaCBwZXJmb3JtYW5jZSAiVlNQRVJGIi4NCj4gVGhh
dA0KPiA+PiA+IHByb2plY3QgcmV1c2VzIHRoZSBCTVdHIGZyYW1ld29yayBhbmQgc3BlY2lmaWNh
dGlvbnMgdG8gYmVuY2htYXJrDQo+ID4+ID4gdmlydHVhbCBzd2l0Y2hlcyBpbXBsZW1lbnRlZCBp
biBnZW5lcmFsLXB1cnBvc2UgaGFyZHdhcmUuIFNvbWUNCj4gPj4gPiBkaWZmZXJlbmNlcyB3aXRo
IHRoZSBiZW5jaG1hcmtpbmcgb2Ygc3BlY2lhbGl6ZWQgSFcgcGxhdGZvcm1zIGFyZQ0KPiA+PiA+
IGlkZW50aWZpZWQgYW5kIHRoZXkgbWF5IGJlY29tZSB3b3JrIGl0ZW1zIGZvciBCTVdHIGluIHRo
ZSBmdXR1cmUuDQo+IEl0J3MNCj4gPj4gPiBhIHdlbGwgd3JpdHRlbiBhbmQgY2xlYXIgZG9jdW1l
bnQsIGJ1dCBJIGhhdmUgcmVzZXJ2YXRpb25zIGFib3V0IGl0DQo+ID4+ID4gYmVpbmcgcHVibGlz
aGVkIGFzIGFuIFJGQywgYW5kIEkgY2Fubm90IGZpbmQgY292ZXJhZ2UgZm9yIGl0IGluIHRoZQ0K
PiBXRw0KPiA+PiA+IGNoYXJ0ZXIuIEkgYWxzbyBoYXZlIGNvbmNlcm5zIHRoYXQgcGFydHMgb2Yg
dGhlIG1ldGhvZG9sb2d5IHVzZWQgYnkNCj4gPj4gPiBPUE5GViBicmVhayB0aGUgQk1XRyBwcmlu
Y2lwbGVzLCBlc3BlY2lhbGx5IHJlcGVhdGFiaWxpdHkgYW5kDQo+ID4+ID4gJ2JsYWNrLWJveCcs
IGFuZCB0aGlzIGlzIG5vdCBjbGVhciBlbm91Z2ggYXJ0aWN1bGF0ZWQgaW4gdGhlDQo+IGRvY3Vt
ZW50Lg0KPiA+PiBbQUNNXQ0KPiA+PiBPaywgbGV0J3MgYWRkcmVzcyB5b3VyIHNwZWNpZmljIGlz
c3VlcywgYW5kIGNvbWUgYmFjayB0byB5b3VyDQo+ID4+IHJlc2VydmF0aW9ucy4NCj4gPj4NCj4g
Pj4gPg0KPiA+PiA+DQo+ID4+ID4gTWFqb3IgaXNzdWVzOg0KPiA+PiA+DQo+ID4+ID4gMS4gSXQg
aXMgbm90IGNsZWFyIHRvIG1lIHdoeSB0aGlzIGRvY3VtZW50IG5lZWRzIHRvIGJlIHB1Ymxpc2hl
ZCBhcw0KPiBhbg0KPiA+PiA+IFJGQy4gVGhlIGludHJvZHVjdGlvbiBzYXlzOiAnVGhpcyBtZW1v
IGRlc2NyaWJlcyB0aGUgcHJvZ3Jlc3Mgb2YNCj4gdGhlDQo+ID4+ID4gT3BlbiBQbGF0Zm9ybSBm
b3IgTkZWIChPUE5GVikgcHJvamVjdCBvbiB2aXJ0dWFsIHN3aXRjaCBwZXJmb3JtYW5jZQ0KPiA+
PiA+ICJWU1BFUkYiLiAgVGhpcyBwcm9qZWN0IGludGVuZHMgdG8gYnVpbGQgb24gdGhlIGN1cnJl
bnQgYW5kDQo+IGNvbXBsZXRlZA0KPiA+PiA+IHdvcmsgb2YgdGhlIEJlbmNobWFya2luZyBNZXRo
b2RvbG9neSBXb3JraW5nIEdyb3VwIGluIElFVEYsIGJ5DQo+ID4+ID4gcmVmZXJlbmNpbmcgZXhp
c3RpbmcgbGl0ZXJhdHVyZS4nIFdoeSBzaG91bGQgdGhlIFdHIGFuZCB0aGUgSUVTRw0KPiA+PiA+
IGludmVzdCByZXNvdXJjZXMgaW4gcHVibGlzaGluZyB0aGlzLCB3aHkgYW4gSS1EIG9yIGFuIElu
ZGVwZW5kZW50DQo+ID4+ID4gU3RyZWFtIFJGQyBpcyBub3Qgc3VmZmljaWVudD8NCj4gPj4gW0FD
TV0NCj4gPj4gVGhlIFdHIGNvbnNpZGVyZWQgYW5kIGRpc2N1c3NlZCB0aGlzIGRvY3VtZW50IG92
ZXIgMyByZXZpc2lvbnMNCj4gPj4gYW5kIGEgeWVhciBvZiB0aW1lIGJlZm9yZSByZWFjaGluZyBj
b25zZW5zdXMgdG8gZGV2ZWxvcCBpdCBmdXJ0aGVyDQo+ID4+IGFzIGEgY2hhcnRlcmVkIGl0ZW0s
IHNvIHRoaXMgZGVjaXNpb24gd2FzIG5vdCB0YWtlbiBsaWdodGx5Lg0KPiA+PiBTZWUgbW9yZSBi
ZWxvdy4NCj4gPj4NCj4gPj4gPiBUaGUgV0cgY2hhcnRlciBzYXlzIHNvbWV0aGluZyBhYm91dDoN
Cj4gPj4gPiAnVk5GIGFuZCBSZWxhdGVkIEluZnJhc3RydWN0dXJlIEJlbmNobWFya2luZzogQmVu
Y2htYXJraW5nDQo+ID4+ID4gTWV0aG9kb2xvZ2llcyBoYXZlIHJlbGlhYmx5IGNoYXJhY3Rlcml6
ZWQgbWFueSBwaHlzaWNhbCBkZXZpY2VzLg0KPiBUaGlzDQo+ID4+ID4gd29yayBpdGVtIGV4dGVu
ZHMgYW5kIGVuaGFuY2VzIHRoZSBtZXRob2RzIHRvIHZpcnR1YWwgbmV0d29yaw0KPiA+PiA+IGZ1
bmN0aW9ucyAoVk5GKSBhbmQgdGhlaXIgdW5pcXVlIHN1cHBvcnRpbmcgaW5mcmFzdHJ1Y3R1cmUu
IEEgZmlyc3QNCj4gPj4gPiBkZWxpdmVyYWJsZSBmcm9tIHRoaXMgYWN0aXZpdHkgd2lsbCBiZSBh
IGRvY3VtZW50IHRoYXQgY29uc2lkZXJzDQo+IHRoZQ0KPiA+PiA+IG5ldyBiZW5jaG1hcmtpbmcg
c3BhY2UgdG8gZW5zdXJlIHRoYXQgY29tbW9uIGlzc3VlcyBhcmUgcmVjb2duaXplZA0KPiA+PiA+
IGZyb20gdGhlIHN0YXJ0LCB1c2luZyBiYWNrZ3JvdW5kIG1hdGVyaWFscyBmcm9tIGluZHVzdHJ5
IGFuZCBTRE9zDQo+ID4+ID4gKGUuZy4sIElFVEYsIEVUU0kgTkZWKS4nLiBJIGRvIG5vdCBiZWxp
ZXZlIHRoYXQgdGhpcyBkb2N1bWVudA0KPiBjb3ZlcnMNCj4gPj4gPiB0aGUgaW50ZW50IG9mIHRo
ZSBjaGFydGVyLCBhcyBpdCBmb2N1c2VkIG9uIG9uZSBvcmdhbml6YXRpb24gb25seS4NCj4gPj4g
W0FDTV0NCj4gPj4gSSdtIHNvcnJ5LCBidXQgaGVyZSB5b3UgYXJlIG1pc3Rha2VuLiBUaGUgZG9j
dW1lbnQgdGhhdCBzYXRpc2ZpZWQNCj4gPj4gdGhlICJmaXJzdCBkZWxpdmVyYWJsZSAuLi4gZG9j
dW1lbnQgdGhhdCBjb25zaWRlcnMgdGhlIG5ldw0KPiBiZW5jaG1hcmtpbmcNCj4gPj4gc3BhY2Ui
DQo+ID4+IGlzOiBodHRwczovL3VybGRlZmVuc2UucHJvb2Zwb2ludC5jb20vdjIvdXJsP3U9aHR0
cHMtDQo+IDNBX190b29scy5pZXRmLm9yZ19odG1sX2RyYWZ0LTJEaWV0Zi0yRGJtd2ctMkR2aXJ0
dWFsLTJEbmV0LQ0KPiAyRDA1JmQ9RHdJQmFRJmM9TEZZWi0NCj4gbzlfSFVNZU1UU1FpY3ZqSWcm
cj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3Jm09aG9IV1h2NThheTNXZ0xjMTE1WFlycHJWbUY5DQo+
IDE0enBpaUdDQTZNUWtuYnMmcz1fbWdKT3JETEszM056X3M5T0E0bVNHY2JrWkFrUVNMalBOSjVx
RElFQXU0JmU9DQo+ID4+IHRpdGxlZDogQ29uc2lkZXJhdGlvbnMgZm9yIEJlbmNobWFya2luZyBW
aXJ0dWFsIE5ldHdvcmsgRnVuY3Rpb25zIGFuZA0KPiA+PiBUaGVpciBJbmZyYXN0cnVjdHVyZQ0K
PiA+PiB3aGljaCBoYXMgYmVlbiBzdWJtaXR0ZWQgdG8gSUVTRyBhbmQgYXBwcm92ZWQgZm9yIHB1
YmxpY2F0aW9uLg0KPiA+PiBGdXJ0aGVyLCB0aGUgY3VycmVudCBkcmFmdCAoZHJhZnQtaWV0Zi1i
bXdnLXZzd2l0Y2gtb3BuZnYtMDMpDQo+ID4+IHJlZmVyZW5jZXMgdGhlIGFwcHJvdmVkICJDb25z
aWRlcmF0aW9ucyIgZHJhZnQgaW4gU2VjdGlvbiAzDQo+ID4+IChhcyBkb2VzIGFsbW9zdCBldmVy
eSByZWxhdGVkIEluZHVzdHJ5IHNwZWMgSSdtIGF3YXJlIG9mKS4NCj4gPj4NCj4gPj4gVGhlIEJN
V0cgQ2hhcnRlciBjb250aW51ZXM6DQo+ID4+ICAgQmVuY2htYXJrcyBmb3IgcGxhdGZvcm0gY2Fw
YWNpdHkgYW5kIHBlcmZvcm1hbmNlIGNoYXJhY3RlcmlzdGljcyBvZg0KPiA+PiAgIHZpcnR1YWwg
cm91dGVycywgc3dpdGNoZXMsIGFuZCByZWxhdGVkIGNvbXBvbmVudHMgd2lsbCBmb2xsb3csDQo+
IGluY2x1ZGluZw0KPiA+PiAgIGNvbXBhcmlzb25zIGJldHdlZW4gcGh5c2ljYWwgYW5kIHZpcnR1
YWwgbmV0d29yayBmdW5jdGlvbnMuIEluIG1hbnkNCj4gPj4gY2FzZXMsDQo+ID4+ICAgdGhlIHRy
YWRpdGlvbmFsIGJlbmNobWFya3Mgc2hvdWxkIGJlIGFwcGxpY2FibGUgdG8gVk5GcywgYnV0IHRo
ZQ0KPiBsYWINCj4gPj4gICBzZXQtdXBzLCBjb25maWd1cmF0aW9ucywgYW5kIG1lYXN1cmVtZW50
IG1ldGhvZHMgd2lsbCBsaWtlbHkgbmVlZA0KPiB0bw0KPiA+PiAgIGJlIHJldmlzZWQgb3IgZW5o
YW5jZWQuDQo+ID4+DQo+ID4+IFRoaXMgZHJhZnQgY29uc3RpdHV0ZXMgb25lIG9mIHNldmVyYWwg
Zm9sbG93LW9uIGVmZm9ydHMsIGFwcHJvYWNoaW5nDQo+ID4+IHRoZSBwcm9ibGVtIGV4YWN0bHkg
YXMgd2UgZGVzY3JpYmVkIGluIHRoZSBsYXN0IHNlbnRlbmNlIGFib3ZlLg0KPiA+DQo+ID5bRGFu
IHdyb3RlXQ0KPiA+IEhvdz8gV2hhdCBsYXN0IHNlbnRlbmNlPw0KPiA+DQo+ID4gSXMgaXQ6DQo+
ID4NCj4gPiAnSW4gbWFueSBjYXNlcywNCj4gPiAgIHRoZSB0cmFkaXRpb25hbCBiZW5jaG1hcmtz
IHNob3VsZCBiZSBhcHBsaWNhYmxlIHRvIFZORnMsIGJ1dCB0aGUgbGFiDQo+ID4gICBzZXQtdXBz
LCBjb25maWd1cmF0aW9ucywgYW5kIG1lYXN1cmVtZW50IG1ldGhvZHMgd2lsbCBsaWtlbHkgbmVl
ZCB0bw0KPiA+ICAgYmUgcmV2aXNlZCBvciBlbmhhbmNlZC4nDQo+ID4NCltBQ01dIFllcw0KDQo+
ID4gSG93IGRvZXMgdGhpcyBkb2N1bWVudCBhcHByb2FjaCB0aGlzIHByb2JsZW0/IA0KW0FDTV0g
DQpMYWIgc2V0dXBzIChtZW50aW9uZWQgaW4gdGhlIHNlbnRlbmNlKSBhcmUgaWxsdXN0cmF0ZWQg
aW4gU2VjdGlvbiA0Lg0KaHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtYm13
Zy12c3dpdGNoLW9wbmZ2LTAzI3BhZ2UtMTINCg0KQ29uZmlndXJhdGlvbiBwYXJhbWV0ZXJzICht
ZW50aW9uZWQgaW4gdGhlIHNlbnRlbmNlKSBhcmUgbGlzdGVkIGluIFNlY3Rpb24gMy4zDQpodHRw
czovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDMj
c2VjdGlvbi0zLjMNCg0KPiA+IElmIHRoZXJlIGlzIGEgbmVlZCB0byByZXZpc2UNCj4gPiBvciBl
bmhhbmNlIGV4aXN0aW5nIEJNV0cgd29yaywgd2hhdCBpcyBuZWVkZWQgaXMgc3BlY2lmaWMgcmV2
aXNpb25zIG9mDQo+ID4gZG9jdW1lbnRzLiBUaGlzIGluZm9ybWF0aW9uYWwgZG9jdW1lbnQgb25s
eSBkb2N1bWVudHMgd29yayBpbiBvbmUgZXh0ZXJuYWwNCj4gPiBvcmdhbml6YXRpb24uIEkgaGF2
ZSByZXNlcnZhdGlvbnMgdGhhdCB0aGlzIGlzIGEgV0cgdGFzayB0byBhZHZhbmNlIHRoaXMNCj4g
PiBkb2N1bWVudCwgYW5kIG9mIHRoZSBJRVNHIHRvIGFwcHJvdmUgaXQuIFdoeSBjYW4ndCBpdCBz
dGFuZCBhcyBhbiBJLUQgdW50aWwNCj4gPiB0aGUgV0cgZGVjaWRlcyB3aGF0ZXZlciB3b3JrIG5l
ZWRzIHRvIGJlIHVuZGVydGFrZW4gKGlmIGFueSkgdG8gbWVldCB0aGUNCj4gPiBPUE5GViBuZWVk
cz8gT3IgaWYgdGhleSB3aXRoIHRvIGhhdmUgYW4gUkZDLCB3aHkgY2FuJ3QgaXQgYmUgSW5kZXBl
bmRlbnQNCj4gPiBTdHJlYW0/DQo+ID4NCj4gPiBXaWxsIHRoZSBXRyB3cml0ZSBzaW1pbGFyIGRv
Y3VtZW50cyBmb3IgYWxsIChvciBzZXZlcmFsKSBvdGhlciBvcmdhbml6YXRpb25zDQo+ID4gdGhh
dCBpbXBsZW1lbnQgVk5GcyBvbmUgd2F5IG9yIGFub3RoZXI/IFNob3VsZCBpdD8NCj4gPg0KPiBb
V2FycmVuIHdyb3RlXQ0KPiBBcG9sb2dpZXMgZm9yIHRoZSBkZWxheSwgd2hpY2ggaXMgbXkgYWxt
b3N0IGVudGlyZWx5IGZhdWx0Lg0KPiANCj4gRnJvbSB0aGUgU2hlcGhlcmRzIHdyaXRlLXVwOg0K
PiAiVGhlcmUgaGFzIGJlZW4gYSBmYWlyIGFtb3VudCBvZiB3b3JrIGRvbmUgb24gdGhpcyBkcmFm
dCwgYW5kIHByb2dyZXNzDQo+IG1hZGUgb24gcmV2aXNpb25zLCBmZWVkYmFjaywgYW5kIGNvbW1l
bnRzLiBTZXZlcmFsIHByZXNlbnRhdGlvbnMgaGF2ZQ0KPiBiZWVuIG1hZGUgaW4gdGhlIHJvb20g
ZHVyaW5nIElFVEYgbWVldGluZ3MsIGFuZCBmb2xsb3d1cCBhbmQNCj4gZGlzY3Vzc2lvbiB0YWtl
biB0byB0aGUgQk1XRyBsaXN0LiBUaGlzIGRyYWZ0IGlzIHBhcnRpY3VsYXJseSB1c2VmdWwsDQo+
IGdpdmVuIHRoZSBwb3B1bGFyaXR5IG9mIFZORidzIHdpdGhpbiB0aGUgaW5kdXN0cnkuIg0KPiAN
Cj4gSSBiZWxpZXZlIHRoYXQgdGhlcmUgaXMgdmFsdWUgaW4gdGhlIElFVEYgcHVibGlzaGluZyB0
aGlzIGFzIGFuDQo+IEluZm9ybWF0aW9uYWwgZG9jdW1lbnQgLSB0aGUgZG9jdW1lbnQgcHJvdmlk
ZXMgdXNlZnVsIGluZm9ybWF0aW9uIGZvcg0KPiB0aGUgSW50ZXJuZXQgY29tbXVuaXR5IChlc3Bl
Y2lhbGx5IHRob3NlIGZvbGsgYmVuY2htYXJraW5nIFZORnMgOi0pKS4NCj4gVGhlIE9QTkZWIHZp
cnR1YWwgc3dpdGNoIHBlcmZvcm1hbmNlIGNoYXJhY3Rlcml6YXRpb24gd29yayBpcyB2ZXJ5DQo+
IGNsb3NlbHkgcmVsYXRlZCBhbmQgcmVsZXZhbnQgdG8gdGhlIEJNV0cgd29yaywgYW5kIEkgdGhp
bmsgdGhhdCBoYXZpbmcNCj4gYSBjb2xsYWJvcmF0aW9uIHR5cGUgZG9jdW1lbnQgKHdyaXR0ZW4g
Zm9tIHRoZSBJRVRGIHZpZXdwb2ludCwgYW5kDQo+IGluZm9ybWluZyBJRVRGIHBhcnRpY2lwYW50
cykgaXMgdXNlZnVsLg0KPiBUaGlzICpjb3VsZCogaGF2ZSBiZWVuIGFuIEluZGVwZW5kZW50IFN0
cmVhbSBkb2MsIGJ1dCBpdCB3YXMgZGlzY3Vzc2VkDQo+IGFuZCB3b3JrZWQgb24gaW4gdGhlIFdH
LCBhbmQgc28gaGF2aW5nIGl0IGJlIGEgV0cgZG9jdW1lbnQgZmVlbHMgbXVjaA0KPiBtb3JlIGNv
cnJlY3QgdG8gbWUuDQo+IA0KPiBXDQo+IA0KPiA+Pg0KPiA+PiBBbiBhc3BlY3Qgb2YgSW5kdXN0
cnkgY29sbGFib3JhdGlvbiB0aGF0IHdlIGRpZCBub3QgYW50aWNpcGF0ZSBpbiB0aGUNCj4gPj4g
Qk1XRyBDaGFydGVyIGlzIG91ciBjdXJyZW50IGludGVyYWN0aW9ucyB3aXRoIE9wZW4gU291cmNl
IENvbW11bml0aWVzLg0KPiA+PiBUaGUgY3VycmVudCBDaGFydGVyIHdhcyBhcHByb3ZlZCBpbiBK
dW5lIDIwMTQsIHRoZW4gT1BORlYgd2FzIGZvdW5kZWQNCj4gPj4gb24gU2VwdGVtYmVyIDMwLCAy
MDE0IFswXSBhbmQgdGhlIFZTUEVSRiBQcm9qZWN0IHdhcyBjcmVhdGVkIG9uDQo+ID4+IERlYyAx
NiwgMjAxNCwgc28gd2UgZGlkIG5vdCBhbnRpY2lwYXRlIGV4dGVuc2l2ZSBjb2xsYWJvcmF0aW9u
IG9uDQo+ID4+IHRoaXMgYW5kIG90aGVyIGJlbmNobWFya2luZyB0b3BpY3MuDQo+ID4+DQo+ID4+
DQo+ID4+DQo+ID4+ID5bRGFuIHdyb3RlXQ0KPiA+PiA+IDIuIEluIHNlY3Rpb24gMyB0aGVyZSAn
cmVwZWF0YWJpbGl0eScgaXMgbWVudGlvbmVkLCB3aGlsZQ0KPiA+PiA+IGFja25vd2xlZGdpbmcg
dGhhdCBpbiBhIHZpcnR1YWwgZW52aXJvbm1lbnQgdGhlcmUgaXMgbm8gZ3VhcmFudGVlIGFuZA0K
PiA+PiA+IGFjdHVhbGx5IG5vIHdheSB0byBrbm93IHdoYXQgb3RoZXIgYXBwbGljYXRpb25zIGFy
ZSBiZWluZyBydW4uDQo+ID4+IFtBQ01dDQo+ID4+IFNlZToNCj4gPj4gaHR0cHM6Ly91cmxkZWZl
bnNlLnByb29mcG9pbnQuY29tL3YyL3VybD91PWh0dHBzLQ0KPiAzQV9fdG9vbHMuaWV0Zi5vcmdf
aHRtbF9kcmFmdC0yRGlldGYtMkRibXdnLTJEdmlydHVhbC0yRG5ldC0yRDA1LQ0KPiAyM3NlY3Rp
b24tMkQzLjQmZD1Ed0lCYVEmYz1MRllaLQ0KPiBvOV9IVU1lTVRTUWljdmpJZyZyPU9mc1N1OGtU
SWx0VnlEMW9MNzJjQncmbT1ob0hXWHY1OGF5M1dnTGMxMTVYWXJwclZtRjkNCj4gMTR6cGlpR0NB
Nk1Ra25icyZzPXRLZ1lXcEpVMUpBMklnLVVyWEQ1Q2xVZFhYRDlXVVZzdnUzVWNXdjVDYXcmZT0N
Cj4gPj4NCj4gPj4gVGhlcmUgYXJlIGNlcnRhaW5seSB3YXlzIHRvIGFzc2VzcyB0aGUgY3VycmVu
dCBzZXQgb2YgcHJvY2Vzc2VzDQo+ID4+IGF0IGEgcGFydGljdWxhciB0aW1lLiBUaGUgU29mdHdh
cmUgY29uZmlndXJhdGlvbiBwYXJhbWV0ZXJzIGluDQo+ID4+IFNlY3Rpb24gMy4zIGFyZSBpbnRl
bmRlZCB0byBjYXB0dXJlIHRoaXMgYXNwZWN0IGFzIHBhcnQgb2Ygc2V0LXVwLg0KPiA+PiBBdCB0
aGUgc2FtZSB0aW1lLCB0aGVyZSB3aWxsIGJlIGNoYWxsZW5nZXMgdG8gYXNzZXNzIHRoZSBEVVQN
Cj4gPj4gcGVyZm9ybWFuY2Ugd2hlbiByZXNvdXJjZXMgYXJlIGZ1bGx5IHNoYXJlZCwgYW5kIG5l
dyB0ZXN0aW5nDQo+ID4+IHN0cmF0ZWdpZXMgd2lsbCBiZSBuZWVkZWQ6DQo+ID4+IGh0dHBzOi8v
dXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0FfX3Rvb2xzLmll
dGYub3JnX2h0bWxfZHJhZnQtMkRpZXRmLTJEYm13Zy0yRHZpcnR1YWwtMkRuZXQtMkQwNS0NCj4g
MjNzZWN0aW9uLTJEMy4zJmQ9RHdJQmFRJmM9TEZZWi0NCj4gbzlfSFVNZU1UU1FpY3ZqSWcmcj1P
ZnNTdThrVElsdFZ5RDFvTDcyY0J3Jm09aG9IV1h2NThheTNXZ0xjMTE1WFlycHJWbUY5DQo+IDE0
enBpaUdDQTZNUWtuYnMmcz1lb3BXM3hlaWEybE5FRkpTQ0NsMlBiLS01YXpXZlJlVzVJN2tjaGt0
ZzNZJmU9DQo+ID4+DQo+ID4+IFtEYW4gd3JvdGVdDQo+ID4+ID4gTWVhc3VyaW5nIHBhcmFtZXRl
cnMgYXMgdGhlIG9uZXMgbGlzdGVkIGluIDMuMyBwcm92aWRlcyBqdXN0IHBhcnQgb2YNCj4gPj4g
PiB0aGUgYW5zd2VyLCBhbmQgdGhleSBhcmUgaW50ZXJuYWwgcGFyYW1ldGVycyB0byB0aGUgU1VU
Lg0KPiA+PiBbQUNNXQ0KPiA+PiBZZXMsIGtub3dpbmcgdGhlIHRlc3RlZCBjb25maWd1cmF0aW9u
IGlzIGEgY3JpdGljYWwgcGlsbGFyDQo+ID4+IHN1cHBvcnRpbmcgcmVwZWF0YWJpbGl0eSAodGhl
c2UgaXRlbXMgYXJlIG5vdCBtZWFzdXJlZCwgYnV0IGNvbmZpZ3VyZWQpLA0KPiA+PiBhbmQgd2h5
IHdlIHByb3ZpZGVkIHRoaXMgc2VjdGlvbi4NCj4gPj4gW0RhbiB3cm90ZV0NCj4gPj4gPiBBbHNv
LCB0aGUNCj4gPj4gPiBkaWZmZXJlbnQgZGVwbG95bWVudCBzY2VuYXJpb3MgaW4gc2VjdGlvbiA0
IHJlcXVpcmUgZGlmZmVyZW50DQo+ID4+ID4gY29uZmlndXJhdGlvbnMgZm9yIHRoZSBTVVQsIHRo
dXMgYnJlYWtpbmcgdGhlICdibGFjay1ib3gnIHByaW5jaXBsZS4NCj4gPj4gW0FDTV0NCj4gPj4g
U3BlY2lmeWluZyBEVVQgY29uZmlndXJhdGlvbiBkb2VzIG5vdCBicmVhayBhbnkgcGFydCBvZiB0
aGUNCj4gPj4gYmxhY2stYm94IHByaW5jaXBsZSwgd2hpY2ggZXN0YWJsaXNoZXMgdGhhdCBiZW5j
aG1hcmsgbWVhc3VyZW1lbnRzDQo+ID4+IHdpbGwgYmUgYmFzZWQgb24gZXh0ZXJuYWxseSBvYnNl
cnZhYmxlIHBoZW5vbWVuYS4gU2VlOg0KPiA+PiBodHRwczovL3VybGRlZmVuc2UucHJvb2Zwb2lu
dC5jb20vdjIvdXJsP3U9aHR0cHMtM0FfX3Rvb2xzLmlldGYub3JnX2h0bWxfZHJhZnQtMkRpZXRm
LTJEYm13Zy0yRHZpcnR1YWwtMkRuZXQtMkQwNS0yM3NlY3Rpb24tMkQ0LjImZD1Ed0lCYVEmYz1M
RllaLW85X0hVTWVNVFNRaWN2aklnJnI9T2ZzU3U4a1RJbHRWeUQxb0w3MmNCdyZtPWhvSFdYdjU4
YXkzV2dMYzExNVhZcnByVm1GOTE0enBpaUdDQTZNUWtuYnMmcz1RRmFJb3piT2VsSm5PaDIwNEdK
Q0xseEEyTVJMbkhseFFKTk83QlprY3FVJmU9DQo+ID4+DQo+ID4+IFByZXZpb3VzIEJNV0cgUkZD
cyBoYXZlIGlkZW50aWZpZWQgdGhlIGNyaXRpY2FsIGNvbmZpZ3VyYXRpb24NCj4gPj4gcGFyYW1l
dGVycyBvZiB0aGUgRFVULCBzdWNoIGFzIHRoZSBudW1iZXIgYW5kIHR5cGUgb2YNCj4gPj4gbmV0
d29yayBpbnRlcmZhY2VzLCB0aGUgYXJyYW5nZW1lbnQgb2YgRFVUcyBpbiBhIFNVVCwgZXRjLg0K
PiA+DQo+ID5bRGFuIHdyb3RlXQ0KPiA+IEkgbWF5IGhhdmUgbm90IGJlZW4gY2xlYXIgZW5vdWdo
LiBUaGUgZG9jdW1lbnQgdGFsa3MgYWJvdXQgcmVwZWF0YWJpbGl0eSBhbmQNCj4gPiBhYm91dCBj
b21wYXJpbmcgYmVuY2htYXJrcyB3aXRoIHRoZSBvbmVzIG9mIHNwZWNpYWxpemVkIEhXIGltcGxl
bWVudGF0aW9ucy4NCj4gPiBUaGVuIGl0IGdvZXMgaW50byBhIGxvbmcgYnV0IHN0aWxsIHBhcnRp
YWwgbGlzdCBvZiBmYWN0b3JzIHRoYXQgY2FuDQo+ID4gaW5mbHVlbmNlIHRoZSBiZW5jaG1hcmtp
bmcsIHRoZSBtYWpvcml0eSBvZiB0aGVtIGRlcGVuZCBvbiB0aGUgSFcgYW5kIFNXDQo+ID4gbWVh
c3VyZW1lbnRzIGFuZCBwYXJhbWV0ZXJzIG9mIHRoZSBpbnRlcm5hbCBzeXN0ZW1zLiBIb3cgY2Fu
IHRoaXMgYmUNCj4gPiBjb21wYXJlZCB3aXRoIGEgbnVtYmVyIG9mIHNtYWxsIGFuZCBpbmRlZWQg
ZXh0ZXJuYWxseSBvYnNlcnZhYmxlDQo+ID4gY29uZmlndXJhdGlvbiBwYXJhbWV0ZXJzIGxpa2Ug
dGhlIG51bWJlciBhbmQgdHlwZSBvZiBuZXR3b3JrIGludGVyZmFjZXMuDQo+ID4gVGhpcyBpcyBz
ZXZlcmFsIGRlZ3JlZXMgb2YgbWFnbml0dWRlIGFwYXJ0IGluIGNvbXBsZXhpdHkuDQo+ID4+DQpb
QUNNXSBUaGVzZSBhcmUgcHJlZG9taW5hbnRseSBjb25maWd1cmVkIHBhcmFtZXRlcnMgaW4gU2Vj
dGlvbiAzLjMuDQpGcmFua2x5LCBJIGRvbid0IHNlZSBhbnl0aGluZyBsaXN0ZWQgdGhhdCBpcyBt
ZWFzdXJlZC4NCg0KQk1XRyBoYXMgb2Z0ZW4gc3BlY2lmaWVkIGNvbmZpZ3VyYXRpb24gcGFyYW1l
dGVycyB0aGF0DQphcmUgbm90IGRpcmVjdGx5IG9ic2VydmFibGUsIHN1Y2ggYXMgdGhlIHJvdXRp
bmcgcHJvdG9jb2wgDQpjb25maWd1cmF0aW9uIHBhcmFtZXRlcnMgaW4gcGFydHMgb2YgDQpodHRw
czovL3Rvb2xzLmlldGYub3JnL2h0bWwvcmZjNjQxMyNzZWN0aW9uLTUNCmZvciBleGFtcGxlLg0K
DQpFdmVyeW9uZSBhY3R1YWxseSB3b3JraW5nIHRoaXMgcHJvYmxlbSBhcHByZWNpYXRlcyB0aGUg
ZGlmZmljdWx0eQ0KYW5kIGluY3JlYXNlIG9mIGFib3V0IDMgb3JkZXJzIG9mIG1hZ25pdHVkZSBp
biBjb25maWd1cmF0aW9uIA0KY29tcGxleGl0eS4gIE5ldmVydGhlbGVzcywgdGhpcyBpcyB0aGUg
cmVhbGl0eSBvZiBuZXR3b3JraW5nDQp3aXRoIGdlbmVyYWwgcHVycG9zZSBjb21wdXRlcnMsIGFu
ZCB3ZSBoYXZlIHRvIHNvbHZlIHRoaXMgcHJvYmxlbQ0KdG8gYmVuY2htYXJrIHRoZSBwZXJmb3Jt
YW5jZSBpbiBhIHNjaWVudGlmaWMgd2F5Lg0KDQo+ID4+IHtEYW4gd3JvdGVdDQo+ID4+ID4gSSBi
ZWxpZXZlIHRoYXQgdGhlcmUgaXMgYSBuZWVkIGZvciBhIG1vcmUgY2xlYXIgZXhwbGFuYXRpb24g
b2Ygd2h5IEJNV0cNCj4gPj4gPiBzcGVjaWZpY2F0aW9ucyBhcmUgYXBwcm9wcmlhdGUgYW5kIGhv
dyBjb21wYXJpc29uIGNhbiBiZSBtYWRlIHdoaWxlDQo+ID4+ID4gcmVwZWF0YWJpbGl0eSBjYW5u
b3QgYmUgZW5zdXJlZCwgYW5kIG1lYXN1cmVtZW50cyBhcmUgZGVwZW5kZW50IHVwb24NCj4gPj4g
PiBwYXJhbWV0ZXJzIGludGVybmFsIHRvIHRoZSBTVVQuDQo+ID4+IFtBQ01dDQo+ID4+IEkgYmVs
aWV2ZSB0aGF0IGRyYWZ0LWlldGYtYm13Zy12aXJ0dWFsLW5ldC0wNSBhbHJlYWR5DQo+ID4+IGlu
ZGljYXRlcyB3aHkgdGhlIGV4aXN0aW5nIEJNV0cgUkZDcyBhcmUgYSByZWFzb25hYmxlDQo+ID4+
IHN0YXJ0aW5nIHBsYWNlIGZvciBORlYgYmVuY2htYXJrcywgaW4gcGFydCBiZWNhdXNlDQo+ID4+
IHdlIHdhbnQgdG8gbWVhc3VyZSB0aGUgc2FtZSBiZW5jaG1hcmtzIG9mIHBoeXNpY2FsDQo+ID4+
IG5ldHdvcmsgZnVuY3Rpb25zIGluIG1hbnkgY2FzZXMuIFNlZQ0KPiA+PiBodHRwczovL3VybGRl
ZmVuc2UucHJvb2Zwb2ludC5jb20vdjIvdXJsP3U9aHR0cHMtDQo+IDNBX190b29scy5pZXRmLm9y
Z19odG1sX2RyYWZ0LTJEaWV0Zi0yRGJtd2ctMkR2aXJ0dWFsLTJEbmV0LTJEMDUtDQo+IDIzc2Vj
dGlvbi0yRDQuMSZkPUR3SUJhUSZjPUxGWVotDQo+IG85X0hVTWVNVFNRaWN2aklnJnI9T2ZzU3U4
a1RJbHRWeUQxb0w3MmNCdyZtPWhvSFdYdjU4YXkzV2dMYzExNVhZcnByVm1GOQ0KPiAxNHpwaWlH
Q0E2TVFrbmJzJnM9dWlMSTBXaWxOSFJHeDRRbmlKX0E3SkZha2JGdXdhb2xXUVBMNl9jV09ZNCZl
PQ0KPiA+Pg0KPiA+PiBSZXBlYXRhYmlsaXR5IGlzIGEgZ29hbCBvZiBhbGwgZXhwZXJpbWVudHMs
IGFuZCB3ZSB1bmRlcnN0YW5kDQo+ID4+IHRoYXQgdGhlcmUgaXMgbW9yZSB3b3JrIHRvIGRvIGlu
IHRoaXMgcmVnYXJkLCBidXQgd2hhdA0KPiA+PiB3ZSBrbm93IG5vdyAoZG9jdW1lbnRlZCBpbiB0
aGlzIGRyYWZ0KSBzaG91bGQNCj4gPj4gYmUgYSB2YWx1YWJsZSBjb250cmlidXRpb24gdG8gdGhl
IEluZHVzdHJ5Lg0KPiA+Pg0KPiA+W0RhbiB3cm90ZV0NCj4gPiBNZWFzdXJpbmcgdGhlIHNhbWUg
YmVuY2htYXJrcyBpcyBhIGdvb2QgZ29hbC4gSSBiZWxpZXZlIHRoYXQgdGhlIGNsYWltIG9mDQo+
ID4gcmVwZWF0YWJpbGl0eSBuZWVkcyB0byBiZSBiZXR0ZXIgYXJndWVkLg0KPiA+DQpbQUNNXSAN
CkkgZGlzYWdyZWUsIHRoZSByZXBlYXRhYmlsaXR5IHByb2JsZW0gbmVlZHMgdG8gYmUgc29sdmVk
LCANCmFuZCBtYW55IG9mIHVzIGFyZSB3b3JraW5nIG9uIGl0Lg0KDQo+ID4+DQo+ID4+DQo+ID4+
ID4NCj4gPj4gPiBNaW5vciBpc3N1ZXM6DQo+ID4+ID4NCj4gPj4gPiAxLiBTb21lIG9mIHRoZSB0
ZXN0cyBtZW50aW9uZWQgaW4gU2VjdGlvbiA0IGhhdmUgbm8gcHJpb3Igb3IgaW4NCj4gPj4gPiBw
cm9ncmVzcyB3b3JrIGluIHRoZSBJRVRGOiBDb250cm9sIFBhdGggYW5kIERhdGFwYXRoIENvdXBs
aW5nDQo+IFRlc3RzLA0KPiA+PiA+IE5vaXN5IE5laWdoYm91ciBUZXN0cywgY2hhcmFjdGVyaXph
dGlvbiBvZiBhY2NlbGVyYXRpb24NCj4gdGVjaG5vbG9naWVzLg0KPiA+PiBbQUNNXQ0KPiA+PiBJ
J20gc29ycnksIGJ1dCB0aGF0J3Mgbm90IGFuIGFjY3VyYXRlIHBvcnRyYXlhbCBvZiBCTVdHJ3Mg
bGl0ZXJhdHVyZS4NCj4gPj4NCj4gPj4gaHR0cHM6Ly91cmxkZWZlbnNlLnByb29mcG9pbnQuY29t
L3YyL3VybD91PWh0dHBzLQ0KPiAzQV9fdG9vbHMuaWV0Zi5vcmdfaHRtbF9yZmM2NDEzJmQ9RHdJ
QmFRJmM9TEZZWi0NCj4gbzlfSFVNZU1UU1FpY3ZqSWcmcj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3
Jm09aG9IV1h2NThheTNXZ0xjMTE1WFlycHJWbUY5DQo+IDE0enBpaUdDQTZNUWtuYnMmcz1GZnd1
T3VZY0lGT3NKWFZySUgtcjRxMkp5T21MUDhXcXQ2X3NqR0RuUGU0JmU9DQo+IGV4YW1pbmVkIENv
bnRyb2wgUGxhbmUvRGF0YXBsYW5lDQo+ID4+IGludGVyYWN0aW9ucywgZm9yIGV4YW1wbGUuDQo+
ID4+DQo+ID4+IGh0dHBzOi8vdXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRw
cy0NCj4gM0FfX3Rvb2xzLmlldGYub3JnX2h0bWxfZHJhZnQtMkRpZXRmLTJEYm13Zy0yRHZpcnR1
YWwtMkRuZXQtMkQwNS0NCj4gMjNzZWN0aW9uLTJEMy4zJmQ9RHdJQmFRJmM9TEZZWi0NCj4gbzlf
SFVNZU1UU1FpY3ZqSWcmcj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3Jm09aG9IV1h2NThheTNXZ0xj
MTE1WFlycHJWbUY5DQo+IDE0enBpaUdDQTZNUWtuYnMmcz1lb3BXM3hlaWEybE5FRkpTQ0NsMlBi
LS01YXpXZlJlVzVJN2tjaGt0ZzNZJmU9DQo+ID4+IGl0ZW0gMiBzcGVjaWZpY2FsbHkgaW5jbHVk
ZWQgTm9pc3kgTmVpZ2hib3VyIGFtb25nIHRoZSBuZXcNCj4gPj4gdGVzdGluZyBzdHJhdGVnaWVz
Lg0KPiA+Pg0KPiA+W0RhbiB3cm90ZV0NCj4gPiBQbGVhc2UgcHJvdmlkZSByZWZlcmVuY2VzIGZv
ciBlYWNoIGluIHRleHQuDQo+ID4NCltBQ01dIA0KSSdtIHNvcnJ5LCBidXQgdGhlc2UgcmVmZXJl
bmNlcyB3ZXJlIGZvciB5b3UsIGFuZCANCm9ubHkgdG8gcmVtaW5kIHlvdSBvZiBCTVdHJ3Mgc2Nv
cGUgb2Ygd29yay4NClRoZSBJR1AgYmVuY2htYXJraW5nIHJlZmVyZW5jZSBpcyBub3QgYXBwcm9w
cmlhdGUgaGVyZSwgYW5kIA0KdGhlIG90aGVyIEJNV0cgZHJhZnQgaGFzIGJlZW4gc3VmZmljaWVu
dGx5IHJlZmVyZW5jZWQgZWFybGllci4NCg0KPiA+Pg0KPiA+PiBFdmVyeSBuZXR3b3JrIGludGVy
ZmFjZSB3aXRoIGFuIEFTSUMgaXMgYW4gZXhhbXBsZSBvZiBhY2NlbGVyYXRpb24sDQo+ID4+IG9u
ZSB0aGF0IHdlJ3ZlIGNoYXJhY3Rlcml6ZWQgaW4gcGh5c2ljYWwgbmV0d29yayBkZXZpY2VzIGZv
ciB5ZWFycy4NCj4gPj4NCj4gPg0KPiA+IFllcywgYnV0IHdlIGRvIG5vdCBkZWFsIGhlcmUgd2l0
aCBleHRlcm5hbGx5IG9ic2VydmFibGUgaW50ZXJmYWNlcyBvbmx5LCBhbmQNCj4gPiBpZiB0aGUg
Y2hhcmFjdGVyaXphdGlvbiBvZiB0aGUgYWNjZWxlcmF0aW9uIHRlY2hub2xvZ2llcyBtYXR0ZXJz
IHRoYW4geW91DQo+ID4gbmVlZCBhIHdheSB0byBleHByZXNzIGl0ICh3aGVyZSBjYW4gd2UgZmlu
ZCBpdCBpbiBleGlzdGluZyBCTVdHIHdvcms/IG5ldw0KPiA+IHdvcms/KQ0KW0FDTV0gDQpJIHRo
aW5rIHlvdSBtaXN1bmRlcnN0b29kIG15IHJlcGx5LiBUaGUgYWNjZWxlcmF0aW9uIHRlY2hub2xv
Z2llcw0Kd2UncmUgY2FsbGluZy1vdXQgcGVyZm9ybSBzcGVjaWZpYyBmdW5jdGlvbnMsIHN1Y2gg
YXMgZGUvZW5jYXBzdWxhdGlvbg0Kb3IgZW5jcnlwdGlvbi9kZWNyeXB0aW9uLiBBbiBleHRlcm5h
bC9ibGFjay1ib3ggdGVzdCBzaG91bGQgbWVhc3VyZQ0KcGVyZm9ybWFuY2UgaW1wcm92ZW1lbnRz
IGluIGEgd2F5IHRoYXQgY2FuIGJlIGNvbXBhcmVkIHRvIHN5c3RlbXMNCnVuZGVyIHRlc3QgdGhh
dCBkbyBub3QgaGF2ZSB0aGUgYmVuZWZpdCBvZiBhY2NlbGVyYXRpb24uDQpTbywgbGlrZSBJIHNh
aWQgYWJvdmUsIHdoZW4gcm91dGVyIHZlbmRvcnMgYWRkZWQgQVNJQ3Mgb24gaW50ZXJmYWNlcywN
Cml0IGRpZG7igJl0IGNoYW5nZSB0aGUgZnVuZGFtZW50YWwgYmVuY2htYXJrcywgYnV0IHRoZSBw
ZXJmb3JtYW5jZSANCmltcHJvdmVkLg0KDQo+ID4NCj4gPj4NCj4gPj4gPiBJZiBuZXcgd29yayBp
cyBuZWVkZWQgLyBwcm9wb3NlZCB0byBiZSBhZGRlZCBmb3IgdGhlIEJNV0cgc2NvcGUgYW5kDQo+
ID4+ID4gZnJhbWV3b3JrIGl0IHdvdWxkIGJlIHVzZWZ1bCBmb3IgQk1XRyB0byBsaXN0IHRoZXNl
IHNlcGFyYXRlbHkuDQo+ID4+ID4NCj4gPj4gPg0KPiA+PiA+IE5pdHMvZWRpdG9yaWFsIGNvbW1l
bnRzOg0KPiA+PiA+DQo+ID4+ID4gMS4gV2hhdCBpcyBjYWxsZWQgJ0RlcGxveW1lbnQgc2NlbmFy
aW9zJyBmcm9tIFZTIHBlcnNwZWN0aXZlIGluDQo+ID4+ID4gU2VjdGlvbiA0IGRlc2NyaWJlIGlu
IGZhY3QgZGlmZmVyZW50IGNvbmZpZ3VyYXRpb25zIG9mIHRoZSBTVVQgaW4NCj4gQk1XRw0KPiA+
PiA+IHRlcm1zLiBJdCBzZWVtcyBiZXR0ZXIgdG8gc2VwYXJhdGUgdGhpcyBzZWNvbmQgcGFydCBv
ZiBzZWN0aW9uIDQgaW4NCj4gYQ0KPiA+PiA+IHNlcGFyYXRlIHNlY3Rpb24uIElmIGl0IGJlbG9u
Z3MgdG8gYW4gZXhpc3Rpbmcgc2VjdGlvbiBpdCByYXRoZXINCj4gPj4gPiBiZWxvbmdzIGluIDMg
dGhhbiBpbiA0Lg0KPiA+PiA+DQo+ID4+IFtBQ01dDQo+ID4+IFNlY3Rpb24gMyBpcyBtb3JlIGFi
b3V0IGV4dGVuZGluZyB0aGUgY29uZmlndXJhdGlvbiBndWlkYW5jZQ0KPiA+PiBmcm9tDQo+ID4+
IGh0dHBzOi8vdXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0Ff
X3Rvb2xzLmlldGYub3JnX2h0bWxfZHJhZnQtMkRpZXRmLTJEYm13Zy0yRHZpcnR1YWwtMkRuZXQt
MkQwNS0NCj4gMjNzZWN0aW9uLTJEMy4yJmQ9RHdJQmFRJmM9TEZZWi0NCj4gbzlfSFVNZU1UU1Fp
Y3ZqSWcmcj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3Jm09aG9IV1h2NThheTNXZ0xjMTE1WFlycHJW
bUY5DQo+IDE0enBpaUdDQTZNUWtuYnMmcz1Md3Q5U2FwcGdLUUlsMnhCYlVYOXhFV3BVcnQwUVY0
MUI0M25TZWYwQnhNJmU9DQo+ID4+DQo+ID4+IFNlY3Rpb24gNCBzdW1tYXJpemVzIHRoZSBWU1BF
UkYgTGV2ZWwgVGVzdCBEZXNpZ24gZG9jdW1lbnQsDQo+ID4+IG9mIHdoaWNoIHRoZXNlIGRlcGxv
eW1lbnQgc2NlbmFyaW9zIGFyZSBhIGtleSBwYXJ0Lg0KPiA+DQo+ID5bRGFuIHdyb3RlXQ0KPiA+
IFllcywgYnV0IHRoaXMgc2VhbXMgdG8gYmVsb25nIHRvIGNvbmZpZ3VyYXRpb25zIG9mIFNVVHMs
IGV2ZW4gaWYgdGhleSBhcmUNCj4gPiBjYWxsZWQgJ0RlcGxveW1lbnQgc2NlbmFyaW9zJyBpbiBP
UE5GVi1zcGVhaywgYW5kIHRoZXkgaW1wYWN0IHJlcGVhdGliaWxpdHkuDQpbQUNNXSANCllvdSBj
YW4gb25seSBjb21wYXJlIHRoZSBwZXJmb3JtYW5jZSBvZiBkaWZmZXJlbnQgY29tcHV0ZXIgU1VU
cyB3aGVuIHRoZXkNCmltcGxlbWVudCB0aGUgc2FtZSB2U3dpdGNoIChhbmQgVk1zKSBhbmQgZGVw
bG95bWVudCBzY2VuYXJpby4NCkNvbnNpc3RlbnQgb25maWd1cmF0aW9uIHBhcmFtZXRlcnMgYW5k
IHRlc3Qgc2V0dXBzIGFyZSBhbGwgaW1wb3J0YW50IA0KY29tcG9uZW50cyBvZiBhY2hpZXZpbmcg
cmVwZWF0YWJsZSByZXN1bHRzLg0KDQo+ID4NCj4gPj4NCj4gPj4NCj4gPj4gdGhhbmtzIGZvciB5
b3VyIGNvbW1lbnRzOyBob3BlZnVsbHkgdGhpcyBkZXRhaWxlZCByZXBseQ0KPiA+PiB3aWxsIHJl
ZHVjZSB5b3VyIHJlc2VydmF0aW9ucyBhYm91dCBwdWJsaWNhdGlvbi4NCj4gPj4NCj4gPj4gQWwN
Cj4gPj4gKGZvciB0aGUgY28tYXV0aG9ycykNCj4gPj4NCj4gPj4gWzBdDQo+ID4+IGh0dHBzOi8v
dXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0FfX3d3dy5vcG5m
di5vcmdfYW5ub3VuY2VtZW50c18yMDE0XzA5XzMwX3RlbGVjb20tMkRpbmR1c3RyeS0yRGFuZC0N
Cj4gMkR2ZW5kb3JzLTJEdW5pdGUtMkR0by0yRGJ1aWxkLTJEY29tbW9uLTJEb3Blbi0yRHBsYXRm
b3JtLTJEdG8tDQo+IDJEYWNjZWxlcmF0ZS0yRG5ldHdvcmstMkRmdW5jdGlvbnMtMkR2aXJ0dWFs
aXphdGlvbiZkPUR3SUJhUSZjPUxGWVotDQo+IG85X0hVTWVNVFNRaWN2aklnJnI9T2ZzU3U4a1RJ
bHRWeUQxb0w3MmNCdyZtPWhvSFdYdjU4YXkzV2dMYzExNVhZcnByVm1GOQ0KPiAxNHpwaWlHQ0E2
TVFrbmJzJnM9dXZHb0kxMGpiVm90MS1FQWp4MFNDVDBmeTBSekV3VGxXeVc2RFdyZTRJVSZlPQ0K
PiA+Pg0KPiA+DQo+IA0KPiANCj4gDQo+IC0tDQo+IEkgZG9uJ3QgdGhpbmsgdGhlIGV4ZWN1dGlv
biBpcyByZWxldmFudCB3aGVuIGl0IHdhcyBvYnZpb3VzbHkgYSBiYWQNCj4gaWRlYSBpbiB0aGUg
Zmlyc3QgcGxhY2UuDQo+IFRoaXMgaXMgbGlrZSBwdXR0aW5nIHJhYmlkIHdlYXNlbHMgaW4geW91
ciBwYW50cywgYW5kIGxhdGVyIGV4cHJlc3NpbmcNCj4gcmVncmV0IGF0IGhhdmluZyBjaG9zZW4g
dGhvc2UgcGFydGljdWxhciByYWJpZCB3ZWFzZWxzIGFuZCB0aGF0IHBhaXINCj4gb2YgcGFudHMu
DQo+ICAgIC0tLW1hZg0K


From nobody Wed Jun  7 06:11:56 2017
Return-Path: <aretana@cisco.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6814D1270AC; Wed,  7 Jun 2017 06:11:46 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alvaro Retana <aretana@cisco.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, Al Morton <acmorton@att.com>, bmwg-chairs@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149684110633.2755.12077292205214337612.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 06:11:46 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/QajEWkKkrH_MIYq9Qj99O6r4Ux0>
Subject: [bmwg] Alvaro Retana's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 13:11:47 -0000

Alvaro Retana has entered the following ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Section 4. (Test Setup):

   In terms of route setup, the recommendations of [RFC2544] Section 13
   are valid for this document assuming that an IPv6 version of the
   routing packets shown in appendix C.2.6.2 is used.

However, rfc2544 says in several places that the packets in the appendix are
just examples.  The frame in C.2.6.2 is a RIP update -- but Section 11.3
references the rate at which "frames SHOULD be sent" (also in the appendix)
which include OSPF and IGRP, so I'm assuming that any routing protocol used
should work (if the recommendations are followed in terms of frequency, etc.). 
I note that rfc5180 doesn't really say anything about routing setup for IPv6
either. :-(

I know this is not the document to define a complete set of (or even update)
recommendations for routing setup, so my suggestion is to simply take off the
reference to the appendix:

   In terms of route setup, the recommendations of [RFC2544] Section 13
   are valid for this document assuming that IPv6 capable routing
   protocols are used.



From nobody Wed Jun  7 06:50:45 2017
Return-Path: <aretana@cisco.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id BBC1A127ABE; Wed,  7 Jun 2017 06:50:43 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alvaro Retana <aretana@cisco.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149684344367.2571.1473775185243026799.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 06:50:43 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/XgY5sDEOWCLYHsQCWToDvaNbHqY>
Subject: [bmwg] Alvaro Retana's Abstain on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 13:50:44 -0000

Alvaro Retana has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: Abstain

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I don't see the value of publishing this document as an RFC, specially because
it reflects the progress of the VSPERF project "through the Brahmaputra
(second) release" -- which would make this document obsolete instantly since
the project is already working on their Danube (fourth) release.  I appreciate
the fact that the WG has found the discussions around this document useful and
that the methodologies created are being referenced elsewhere.

Because the WG/Chairs/Responsible AD find value in publication I am not going
to stand in the way; I am then ABSTAINing.



From nobody Wed Jun  7 06:53:53 2017
Return-Path: <shabeer@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B983A127136 for <bmwg@ietfa.amsl.com>; Wed,  7 Jun 2017 06:53:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.021
X-Spam-Level: 
X-Spam-Status: No, score=-2.021 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gHGEdQEqc4Bp for <bmwg@ietfa.amsl.com>; Wed,  7 Jun 2017 06:53:50 -0700 (PDT)
Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0111.outbound.protection.outlook.com [104.47.33.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C51D3127B31 for <bmwg@ietf.org>; Wed,  7 Jun 2017 06:53:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=F2xQIevit5Zxk/qLMghhhekc97aLOcgAwP8lgfXlMM8=; b=ZhBtYnN0piwy+ygUgYpn/YRRSw2UO7Rvxmqeu8cG6iIghFYQ41VnvlMV9ZRKuhRQ7KyB82SE4tdwzVpn6IccgNBZ2Ufyq3Rc8EeohCThMgdVl/7sGBF+BauUUN0LymG5br9RvW27BGedhTgaxkssdlklbsqrFonFx5frLlPbO+A=
Received: from MWHPR05MB3264.namprd05.prod.outlook.com (10.173.230.15) by MWHPR05MB2925.namprd05.prod.outlook.com (10.168.245.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.9; Wed, 7 Jun 2017 13:53:47 +0000
Received: from MWHPR05MB3264.namprd05.prod.outlook.com ([10.173.230.15]) by MWHPR05MB3264.namprd05.prod.outlook.com ([10.173.230.15]) with mapi id 15.01.1157.012; Wed, 7 Jun 2017 13:53:48 +0000
From: Mohammed Shabeer A S <shabeer@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: REG: Draft draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLflXMIwMjauOejS9upwnnK0TOv1A==
Date: Wed, 7 Jun 2017 13:53:47 +0000
Message-ID: <MWHPR05MB3264F96D9FA408F52E923AC8BEC80@MWHPR05MB3264.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.10]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MWHPR05MB2925; 7:HLDj7z2JdolSZuCG9QF01K7mYgt4ZfXCnpYnpD4s8rSf5ldpNnOMfcfLE3rc45RYjdsFLs3dqOf66ctKNjavKdaZrNIsgCHXbNw5g0JU5NJEb629jAY69Lw1u1lZ19ZmzDCoMrfyslKb1CrDdp3mlJa0aSEthl7OgblcU+m6SLB9WG7TnMJLO3vN1cySDOv+s7IB4KgY6vt3YiA80WBYitA+Qy3XGxiQ89/W5suqAcLmKVPIuKrOXthENLu/qtYKA3Cl4gmQ8ZbkcJyt1qMaPD/lwJIaVNPezzkXL8qUhpv3EloT91C8RHA/K23qf3SWjeTkcScB3D/MR4a0MK093Q==
x-ms-traffictypediagnostic: MWHPR05MB2925:
x-ms-office365-filtering-correlation-id: fe5daefd-990b-48b2-a4c8-08d4adac9f11
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081);  SRVR:MWHPR05MB2925; 
x-microsoft-antispam-prvs: <MWHPR05MB2925D862D98B9E62756725A9BEC80@MWHPR05MB2925.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(100000703101)(100105400095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123562025)(20161123564025)(20161123560025)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR05MB2925; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR05MB2925; 
x-forefront-prvs: 03319F6FEF
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39850400002)(39400400002)(39410400002)(39860400002)(39450400003)(53936002)(110136004)(1730700003)(7736002)(3846002)(6116002)(102836003)(6306002)(305945005)(81166006)(25786009)(9686003)(74316002)(38730400002)(66066001)(122556002)(2501003)(54356999)(50986999)(230783001)(5660300001)(33656002)(6916009)(7696004)(5640700003)(189998001)(55016002)(2351001)(99286003)(8936002)(966005)(77096006)(478600001)(6436002)(2900100001)(2906002)(6506006)(3660700001)(3280700002)(86362001)(14454004); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR05MB2925; H:MWHPR05MB3264.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_MWHPR05MB3264F96D9FA408F52E923AC8BEC80MWHPR05MB3264namp_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jun 2017 13:53:47.9702 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB2925
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/b2AZ5o4inkZ5cx6q_h2KMBm6reE>
Subject: [bmwg] REG: Draft draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 13:53:52 -0000

--_000_MWHPR05MB3264F96D9FA408F52E923AC8BEC80MWHPR05MB3264namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

HI

I support this draft for workgroup adoption . This drafts exemplifies the k=
ey aspects of EVPN benchmarking.

Regards
Shabeer




--_000_MWHPR05MB3264F96D9FA408F52E923AC8BEC80MWHPR05MB3264namp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;">
<div>HI</div>
<div>&nbsp;</div>
<div>I support this draft for workgroup adoption . This drafts exemplifies =
the key aspects of EVPN benchmarking.</div>
<div>&nbsp;</div>
<div>Regards</div>
<div>Shabeer</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_MWHPR05MB3264F96D9FA408F52E923AC8BEC80MWHPR05MB3264namp_--


From nobody Wed Jun  7 07:38:09 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AD49F12EC8A; Wed,  7 Jun 2017 07:38:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WUMa7Z3Iy6UB; Wed,  7 Jun 2017 07:38:06 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1093E129485; Wed,  7 Jun 2017 07:38:03 -0700 (PDT)
Received: from pps.filterd (m0049458.ppops.net [127.0.0.1]) by m0049458.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v57EaIQS035585; Wed, 7 Jun 2017 10:37:59 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049458.ppops.net-00191d01. with ESMTP id 2axgbkx4sp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Jun 2017 10:37:59 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v57EbwL6076354; Wed, 7 Jun 2017 09:37:58 -0500
Received: from dalint02.pst.cso.att.com (dalint02.pst.cso.att.com [135.31.133.160]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v57Eblvf076232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 7 Jun 2017 09:37:51 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint02.pst.cso.att.com (RSA Interceptor); Wed, 7 Jun 2017 14:37:29 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v57EbTsP013155; Wed, 7 Jun 2017 09:37:29 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v57EbM9E012684; Wed, 7 Jun 2017 09:37:22 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id C0B42E08D5; Wed,  7 Jun 2017 10:37:21 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Wed, 7 Jun 2017 10:37:21 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Alvaro Retana <aretana@cisco.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alvaro Retana's Abstain on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
Thread-Index: AQHS35UTKk/JC7QJOEuWAPFd6/E17KIZa32Q
Date: Wed, 7 Jun 2017 14:37:21 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD6928@njmtexg5.research.att.com>
References: <149684344367.2571.1473775185243026799.idtracker@ietfa.amsl.com>
In-Reply-To: <149684344367.2571.1473775185243026799.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-07_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706070269
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/eq61IeZv3bOwnZQUZoOem6I8mbA>
Subject: Re: [bmwg] Alvaro Retana's Abstain on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 14:38:08 -0000

SGkgQWx2YXJvLA0KdGhhbmtzIGZvciB5b3VyIHJldmlldywgcGxlYXNlIHNlZSByZXBseSBiZWxv
dy4NCkFsIChmb3IgdGhlIGNvLWF1dGhvcnMpDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0t
LS0NCj4gRnJvbTogQWx2YXJvIFJldGFuYSBbbWFpbHRvOmFyZXRhbmFAY2lzY28uY29tXQ0KPiBT
ZW50OiBXZWRuZXNkYXksIEp1bmUgMDcsIDIwMTcgOTo1MSBBTQ0KPiBUbzogVGhlIElFU0cNCj4g
Q2M6IGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2QGlldGYub3JnOyBTYXJhaCBCYW5rczsg
Ym13Zy0NCj4gY2hhaXJzQGlldGYub3JnOyBibXdnQGlldGYub3JnDQo+IFN1YmplY3Q6IEFsdmFy
byBSZXRhbmEncyBBYnN0YWluIG9uIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LTAzOg0K
Li4uDQo+IA0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IENPTU1FTlQ6DQo+IC0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4g
DQo+IEkgZG9uJ3Qgc2VlIHRoZSB2YWx1ZSBvZiBwdWJsaXNoaW5nIHRoaXMgZG9jdW1lbnQgYXMg
YW4gUkZDLCBzcGVjaWFsbHkNCj4gYmVjYXVzZSBpdCByZWZsZWN0cyB0aGUgcHJvZ3Jlc3Mgb2Yg
dGhlIFZTUEVSRiBwcm9qZWN0ICJ0aHJvdWdoIHRoZSBCcmFobWFwdXRyYQ0KPiAoc2Vjb25kKSBy
ZWxlYXNlIiAtLSB3aGljaCB3b3VsZCBtYWtlIHRoaXMgZG9jdW1lbnQgb2Jzb2xldGUgaW5zdGFu
dGx5IHNpbmNlDQo+IHRoZSBwcm9qZWN0IGlzIGFscmVhZHkgd29ya2luZyBvbiB0aGVpciBEYW51
YmUgKGZvdXJ0aCkgcmVsZWFzZS4gIA0KW0FDTV0gDQpUaGUgVlNQRVJGIGRldmVsb3BtZW50IGVm
Zm9ydHMgZHVyaW5nIHRoZSBDb2xvcmFkbyBhbmQgRGFudWJlDQpyZWxlYXNlcyBhcmUgYWxtb3N0
IGVudGlyZWx5IGZvY3VzZWQgb24gdGhlIGZlYXR1cmVzIGZvciANCnRoZSBWU1BFUkYgdGVzdGlu
ZyB0b29sLiBNYXliZSBvbmUgb3IgdHdvIHRlc3Qgc3BlY2lmaWNhdGlvbnMgaGF2ZSANCmJlZW4g
YWRkZWQgdG8gdGhlIExURCwgYnV0IHRoZSB2YXN0IG1ham9yaXR5IGFyZSB0aGUgc2FtZSBpbg0K
RGFudWJlIDMuMCwgc28gdGhlIHRlc3RzIGRlc2NyaWJlZCB0aGlzIGRvY3VtZW50IGFyZSBzdGls
bCANCmNvbXBsZXRlbHkgcmVsZXZhbnQuIA0KDQpJbnN0ZWFkIG9mIGRldmVsb3BpbmcgbW9yZSB0
ZXN0IHNwZWNpZmljYXRpb25zLCB0aGUgY29udHJvbCBvZiANCnRocmVlIG1vcmUvbmV3IHRyYWZm
aWMgZ2VuZXJhdG9ycyBoYXMgYmVlbiBpbmNvcnBvcmF0ZWQgaW4gdGhlIA0KVlNQRVJGIHRvb2wu
IEFsc28sIHRoZSBmZC5pbyBWUFAgaGFzIGJlZW4gYWRkZWQgYXMgYSB0ZXN0aW5nDQpwb3NzaWJp
bGl0eSBpbiBEYW51YmUsIGluIGFkZGl0aW9uIHRvIHRoZSBsYXRlc3QgdmVyc2lvbnMgb2YgT1ZT
Lg0KQnV0IG5vbmUgb2YgdGhlc2UgZGV2ZWxvcG1lbnRzIGFmZmVjdCB3aGF0IHRoZSBXRyBhZ3Jl
ZWQgaW4gDQp0aGUgZHJhZnQuDQoNClRoZSBXRyBhZ3JlZWQgdG8gImZyZWV6ZSIgdGhlIHVwZGF0
ZXMgYXQgQnJhaG1hcHV0cmEgZm9yIHByYWN0aWNhbA0KcHVycG9zZXMuIFRoZSBhdXRob3JzIHdv
dWxkIGFncmVlIHRvIGFkZCB0aGUgZmV3IG5ldyB0ZXN0cyBpbiANCnNlY3Rpb24gNSB0aGF0IHdv
dWxkIHVwZGF0ZSB0aGlzIGRyYWZ0IHRvIERhbnViZSAzLjAsIGFuZCANCmluY3JlYXNlIGl0cyBy
ZWxldmFuY2UsIGlmIHRoYXQgc2VlbXMgcmVhc29uYWJsZSBhbmQgdmFsdWFibGUNCnRvIGFsbC4N
Cg0KDQoNCj4gSSBhcHByZWNpYXRlDQo+IHRoZSBmYWN0IHRoYXQgdGhlIFdHIGhhcyBmb3VuZCB0
aGUgZGlzY3Vzc2lvbnMgYXJvdW5kIHRoaXMgZG9jdW1lbnQgdXNlZnVsIGFuZA0KPiB0aGF0IHRo
ZSBtZXRob2RvbG9naWVzIGNyZWF0ZWQgYXJlIGJlaW5nIHJlZmVyZW5jZWQgZWxzZXdoZXJlLg0K
PiANCj4gQmVjYXVzZSB0aGUgV0cvQ2hhaXJzL1Jlc3BvbnNpYmxlIEFEIGZpbmQgdmFsdWUgaW4g
cHVibGljYXRpb24gSSBhbSBub3QgZ29pbmcNCj4gdG8gc3RhbmQgaW4gdGhlIHdheTsgSSBhbSB0
aGVuIEFCU1RBSU5pbmcuDQo+IA0KDQo=


From nobody Wed Jun  7 12:17:25 2017
Return-Path: <suresh.krishnan@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C8F85129407; Wed,  7 Jun 2017 12:17:23 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Suresh Krishnan <suresh.krishnan@gmail.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, Al Morton <acmorton@att.com>, bmwg-chairs@ietf.org, acmorton@att.com, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149686304373.2650.7741894590270773104.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 12:17:23 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/OXNhnj2KT-KBIyd5rYlE5y7ZHGk>
Subject: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 19:17:24 -0000

Suresh Krishnan has entered the following ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

* I am surprised that this document does not use (and does not even mention)
the Well Known Prefix (64:ff9b::/96) for the algorithmic mapping between IPv4
and IPv6 on translators as specified by RFC6052. Is there a reason why this is
omitted? * It is not clear from the document whether the time taken by the
DNS64 resolution procedure is included in the latency measurements. It might be
useful to note this.



From nobody Wed Jun  7 14:52:56 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 8D808131491; Wed,  7 Jun 2017 14:52:42 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alissa Cooper <alissa@cooperw.in>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 14:52:42 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/jgRwqu7_Rl10ZgipuFHSBZ4AGxU>
Subject: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 21:52:43 -0000

Alissa Cooper has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Picking up on the thread that started with the Gen-ART review, I'm not clear on
the position of this document when it comes to repeatability. If all of the
parameters listed in Section 3.3 (assuming they apply) are configured and
documented, is it assumed that the benchmarks will be repeatable and comparable
with hardware implementation benchmarks? Or is the position of the document
that the benchmarks are not likely to be repeatable/comparable in some (many?)
cases, given the increase in complexity? Or that more work needs to be done
(outside the specification process) to achieve repeatability? I think this
needs to be more clear in the document.


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I understand the debate about the publication of this document as an IETF
consensus RFC. To me it seems valuable to publish and that this item is within
the WG's charter. I think adding the updates necessary to make the document
up-to-date with the Danube 3.0 release as suggested in the thread with Alvaro
would be valuable.



From nobody Wed Jun  7 15:35:43 2017
Return-Path: <adam@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 8D9331270A7; Wed,  7 Jun 2017 15:35:35 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Adam Roach <adam@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149687493553.25630.10964914484670596603.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 15:35:35 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/-8AwShGUV_gY2LDxkkIRZwibh2s>
Subject: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 22:35:35 -0000

Adam Roach has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I think this is a variation on the issue that Alissa calls out, but I'm having
a hard time reconciling:

   It's unlikely that the virtual switch will be the only application
   running on the System Under Test (SUT), so CPU utilization, Cache
   utilization, and Memory footprint should also be recorded for the
   virtual implementations of internetworking functions.

...with...

   Further, benchmarking is performed on a "black-box" basis, relying
   solely on measurements observable external to the DUT/SUT.

Please add text that clarifies how the metrics that 3.1 says should be recorded
in section 3.1 relate to benchmarking.



From nobody Wed Jun  7 15:58:38 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1C7111270B4; Wed,  7 Jun 2017 15:58:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AcO90u9Egg6a; Wed,  7 Jun 2017 15:58:35 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2FF7E120454; Wed,  7 Jun 2017 15:58:35 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v57MwRSH015845; Wed, 7 Jun 2017 18:58:30 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049463.ppops.net-00191d01. with ESMTP id 2axptge2qj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Jun 2017 18:58:29 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v57MvZ5W033979; Wed, 7 Jun 2017 17:57:35 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v57MvTuS033964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 7 Jun 2017 17:57:29 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor); Wed, 7 Jun 2017 22:57:08 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v57Mv8dI017182; Wed, 7 Jun 2017 17:57:08 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v57Mv2u2016874; Wed, 7 Jun 2017 17:57:02 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 35AFEE03D5; Wed,  7 Jun 2017 18:57:02 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Wed, 7 Jun 2017 18:57:02 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Alissa Cooper <alissa@cooperw.in>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
Thread-Index: AQHS39hnThUImon/oEeFX43rE6nRmqIZ9L4w
Date: Wed, 7 Jun 2017 22:57:00 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com>
References: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com>
In-Reply-To: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-07_14:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706070409
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2dSgfLijZkkSWmbBDEKHRYnuKKM>
Subject: Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 22:58:37 -0000

SGkgQWxpc3NhLA0KVGhhbmtzIGZvciB5b3VyIHJldmlldywgcGxlYXNlIHNlZSByZXBseSBiZWxv
dy4NCkFsIChmb3IgdGhlIGNvLWF1dGhvcnMpDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0t
LS0NCj4gRnJvbTogQWxpc3NhIENvb3BlciBbbWFpbHRvOmFsaXNzYUBjb29wZXJ3LmluXQ0KPiBT
ZW50OiBXZWRuZXNkYXksIEp1bmUgMDcsIDIwMTcgNTo1MyBQTQ0KPiBUbzogVGhlIElFU0cNCj4g
Q2M6IGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2QGlldGYub3JnOyBTYXJhaCBCYW5rczsg
Ym13Zy0NCj4gY2hhaXJzQGlldGYub3JnOyBzYmFua3NAZW5jcnlwdGVkLm5ldDsgYm13Z0BpZXRm
Lm9yZw0KPiBTdWJqZWN0OiBBbGlzc2EgQ29vcGVyJ3MgRGlzY3VzcyBvbiBkcmFmdC1pZXRmLWJt
d2ctdnN3aXRjaC1vcG5mdi0wMzoNCj4gKHdpdGggRElTQ1VTUyBhbmQgQ09NTUVOVCkNCi4uLg0K
PiANCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiBESVNDVVNTOg0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IA0KPiBQ
aWNraW5nIHVwIG9uIHRoZSB0aHJlYWQgdGhhdCBzdGFydGVkIHdpdGggdGhlIEdlbi1BUlQgcmV2
aWV3LCBJJ20gbm90IGNsZWFyIG9uDQo+IHRoZSBwb3NpdGlvbiBvZiB0aGlzIGRvY3VtZW50IHdo
ZW4gaXQgY29tZXMgdG8gcmVwZWF0YWJpbGl0eS4gSWYgYWxsIG9mIHRoZQ0KPiBwYXJhbWV0ZXJz
IGxpc3RlZCBpbiBTZWN0aW9uIDMuMyAoYXNzdW1pbmcgdGhleSBhcHBseSkgYXJlIGNvbmZpZ3Vy
ZWQgYW5kDQo+IGRvY3VtZW50ZWQsIGlzIGl0IGFzc3VtZWQgdGhhdCB0aGUgYmVuY2htYXJrcyB3
aWxsIGJlIHJlcGVhdGFibGUgYW5kIGNvbXBhcmFibGUNCj4gd2l0aCBoYXJkd2FyZSBpbXBsZW1l
bnRhdGlvbiBiZW5jaG1hcmtzPw0KW0FDTV0gDQo+Pj4NCk9mIGNvdXJzZSwgaXQgaGFzIGJlZW4g
dGhlIGdvYWwgb2YgdGhlIHByb2plY3QgdG8gcHJvZHVjZSANCnJlcGVhdGFibGUgcmVzdWx0cywg
YW5kIGEgbGFyZ2Ugc2V0IG9mIHRoZSBwYXJhbWV0ZXJzDQpiZWxpZXZlZCB0byBiZSBjcml0aWNh
bCB3YXMgcHJvdmlkZWQgc28gdGhhdCB0aGUgYmVuY2htYXJraW5nDQpjb21tdW5pdHkgY291bGQg
YmV0dGVyIGFwcHJlY2lhdGUgdGhlIGluY3JlYXNlIGluIGNvbmZpZ3VyYXRpb24NCmNvbXBsZXhp
dHkgaW5oZXJlbnQgaW4gdGhpcyB3b3JrLiBTbywgeWVzLCB0aGlzIHNldCB3YXMgYXNzdW1lZCAN
CnN1ZmZpY2llbnQgZm9yIHRoZSBpbmZyYXN0cnVjdHVyZSBpbiB1c2UgYnkgdGhlIFZTUEVSRiBw
cm9qZWN0DQp0byBvYnRhaW4gcmVwZWF0YWJsZSByZXN1bHRzIGZyb20gdGVzdC10by10ZXN0LiAN
Cj4+PiBXZSBjb3VsZCBhZGQgc3VtbWFyeSBhYm92ZSBhcyBhIG5ldyBwYXJhZ3JhcGggaW4gc2Vj
dGlvbiAzLjMuDQoNCkJlbmNobWFyayBDb21wYXJhYmlsaXR5IGJldHdlZW4gdmlydHVhbCBhbmQg
cGh5c2ljYWwvaGFyZHdhcmUgDQppbXBsZW1lbnRhdGlvbnMgb2YgZXF1aXZhbGVudCBmdW5jdGlv
bnMgd2lsbCBsaWtlbHkgcGxhY2UgbW9yZQ0KZGV0YWlsZWQgYW5kIGV4YWN0IHJlcXVpcmVtZW50
cyBvbiB0aGUgKnRlc3Rpbmcgc3lzdGVtcyoNCihpbiB0ZXJtcyBvZiBzdHJlYW0gZ2VuZXJhdGlv
biwgYWxnb3JpdGhtcyB0byBzZWFyY2ggDQpmb3IgbWF4IHZhbHVlcywgYW5kIHRoZWlyIGNvbmZp
Z3VyYXRpb25zIG9mIGNvdXJzZSkuIA0KVGhpcyBpcyBhbm90aGVyIGFyZWEgZm9yIHN0YW5kYXJk
aXphdGlvbiB0byBhcHByZWNpYXRlLA0Kbm93IHRoYXQgd2UgaGF2ZSBhIGZldyB5ZWFycyB0ZXN0
aW5nIGV4cGVyaWVuY2UuDQpIb3dldmVyLCB0aGUgaXMgYSB0b3BpYyBmb3IgYSBmdXR1cmUgZHJh
ZnQuDQoNCj4gT3IgaXMgdGhlIHBvc2l0aW9uIG9mIHRoZSBkb2N1bWVudA0KPiB0aGF0IHRoZSBi
ZW5jaG1hcmtzIGFyZSBub3QgbGlrZWx5IHRvIGJlIHJlcGVhdGFibGUvY29tcGFyYWJsZSBpbiBz
b21lIChtYW55PykNCj4gY2FzZXMsIGdpdmVuIHRoZSBpbmNyZWFzZSBpbiBjb21wbGV4aXR5PyBP
ciB0aGF0IG1vcmUgd29yayBuZWVkcyB0byBiZSBkb25lDQo+IChvdXRzaWRlIHRoZSBzcGVjaWZp
Y2F0aW9uIHByb2Nlc3MpIHRvIGFjaGlldmUgcmVwZWF0YWJpbGl0eT8NCltBQ01dIA0KVGhhbmtz
IHRvIHNvbWUgc2VydmVyIHJlLWFycmFuZ2VtZW50cyBpbiB0aGUgbGFiIHRoYXQgaG9zdHMNCnRo
ZSBWU1BFUkYgcHJvamVjdCwgd2UgYXJlIGFibGUgdG8gZXZhbHVhdGUgb3VyIGxvbmctdGVybQ0K
cmVzdWx0IHJlcGVhdGFiaWxpdHkgYW5kIHJlcGVhdGFiaWxpdHkgYWNyb3NzIHR3byBzaW1pbGFy
DQpoYXJkd2FyZSBwbGF0Zm9ybXMgaW4gZGlmZmVyZW50IGJ1aWxkaW5ncy4gSSdtIHN0aWxsIHBy
b2Nlc3NpbmcNCnNvbWUgb2YgdGhlIHJlc3VsdHMgd2Ugd2lsbCBwcmVzZW50IG5leHQgd2VlayBh
dCB0aGUgT1BORlYgU3VtbWl0LA0KYnV0IHRoZSBjcm9zcyBoYXJkd2FyZSBwbGF0Zm9ybSByZXN1
bHRzIGxvb2sgcXVpdGUgY29uc2lzdGVudCwNCmV2ZW4gd2l0aCBhIGJlbmNobWFyayB3aGVyZSBT
Y290dCBCcmFkbmVyIGV4cGVyaWVuY2VkIHNvbWUNCnJlcGVhdGFiaWxpdHkgcHJvYmxlbXMgaW4g
aGlzIGxhYiBiYWNrIGluIHRoZSA5MOKAmXMuDQoNCj4gSSB0aGluayB0aGlzIG5lZWRzIHRvIGJl
IG1vcmUgY2xlYXIgaW4gdGhlIGRvY3VtZW50Lg0KW0FDTV0gDQpJZiB0aGUgPj4+cGFyYWdyYXBo
PDw8IGFjY29tcGxpc2hlcyB0aGF0IGZvciB5b3UsDQp3aXRoIHNvbWUgbWlub3IgdHdlYWtzIHRv
IHdvcmQgaXQgbGVzcyBsaWtlIGEgcmVwbHksIGxldCBtZSBrbm93Lg0KDQo+IA0KPiANCj4gLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLQ0KPiBDT01NRU5UOg0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IA0KPiBJIHVuZGVyc3Rh
bmQgdGhlIGRlYmF0ZSBhYm91dCB0aGUgcHVibGljYXRpb24gb2YgdGhpcyBkb2N1bWVudCBhcyBh
biBJRVRGDQo+IGNvbnNlbnN1cyBSRkMuIFRvIG1lIGl0IHNlZW1zIHZhbHVhYmxlIHRvIHB1Ymxp
c2ggYW5kIHRoYXQgdGhpcyBpdGVtIGlzIHdpdGhpbg0KPiB0aGUgV0cncyBjaGFydGVyLiBJIHRo
aW5rIGFkZGluZyB0aGUgdXBkYXRlcyBuZWNlc3NhcnkgdG8gbWFrZSB0aGUgZG9jdW1lbnQNCj4g
dXAtdG8tZGF0ZSB3aXRoIHRoZSBEYW51YmUgMy4wIHJlbGVhc2UgYXMgc3VnZ2VzdGVkIGluIHRo
ZSB0aHJlYWQgd2l0aCBBbHZhcm8NCj4gd291bGQgYmUgdmFsdWFibGUuDQo+IA0KW0FDTV0gDQpH
cmVhdCwgSSd2ZSBsb2NhdGVkIHRoZSBsaXN0IG9mIHRlc3RzIGFkZGVkIGluIENvbG9yYWRvIGFu
ZCBEYW51YmUsDQpzbyBJJ2xsIGFkZCB0aGVtIGluIHNlY3Rpb24gNSwgYW5kIHVwZGF0ZSB0aGUg
cmVsZWFzZSBpbmZvIGFjY29yZGluZ2x5Lg0KDQoNCg==


From nobody Wed Jun  7 16:02:36 2017
Return-Path: <adam@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F54F126C26; Wed,  7 Jun 2017 16:02:31 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Adam Roach <adam@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, Al Morton <acmorton@att.com>, bmwg-chairs@ietf.org, acmorton@att.com, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149687655097.25630.10122076655027013893.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 16:02:30 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/PWBDrdzqIoHdgGbrIHJ3UXXOxtk>
Subject: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 23:02:31 -0000

Adam Roach has entered the following ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I was surprised not to find any mention of RFC 4380 (Teredo) in this document.
If its omission was intentional, a statement to that effect (say, in the
introduction) is probably warranted.



From nobody Wed Jun  7 16:21:38 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EAF1A126D45; Wed,  7 Jun 2017 16:21:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VuOcGw1LsZYc; Wed,  7 Jun 2017 16:21:29 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2550C126CD6; Wed,  7 Jun 2017 16:21:29 -0700 (PDT)
Received: from pps.filterd (m0049287.ppops.net [127.0.0.1]) by m0049287.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v57NL7wB017445; Wed, 7 Jun 2017 19:21:25 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049287.ppops.net-00191d01. with ESMTP id 2axt6hh5qc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Jun 2017 19:21:25 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v57NLNZu047110; Wed, 7 Jun 2017 18:21:24 -0500
Received: from dalint02.pst.cso.att.com (dalint02.pst.cso.att.com [135.31.133.160]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v57NLFsI047030 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 7 Jun 2017 18:21:15 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint02.pst.cso.att.com (RSA Interceptor); Wed, 7 Jun 2017 23:21:02 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v57NL2hc028653; Wed, 7 Jun 2017 18:21:02 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v57NKtY0028251; Wed, 7 Jun 2017 18:20:55 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id D389BE10A8; Wed,  7 Jun 2017 19:20:48 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Wed, 7 Jun 2017 19:20:54 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Adam Roach <adam@nostrum.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Adam Roach's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
Thread-Index: AQHS395kGfAw0QclwU2J8cyDUH8NZaIaAxbA
Date: Wed, 7 Jun 2017 23:20:53 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD6E3C@njmtexg5.research.att.com>
References: <149687493553.25630.10964914484670596603.idtracker@ietfa.amsl.com>
In-Reply-To: <149687493553.25630.10964914484670596603.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-07_14:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706070415
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Ft_TwnaHJsk7e6jaSwfMBmwS7GI>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jun 2017 23:21:31 -0000

SGkgQWRhbSwNClRoYW5rcyBmb3IgeW91ciByZXZpZXcsIHBsZWFzZSBzZWUgcmVwbHkgYmVsb3cu
DQpBbCAoZm9yIHRoZSBjby1hdXRob3JzKQ0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0t
DQo+IEZyb206IEFkYW0gUm9hY2ggW21haWx0bzphZGFtQG5vc3RydW0uY29tXQ0KPiBTZW50OiBX
ZWRuZXNkYXksIEp1bmUgMDcsIDIwMTcgNjozNiBQTQ0KPiBUbzogVGhlIElFU0cNCj4gQ2M6IGRy
YWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2QGlldGYub3JnOyBTYXJhaCBCYW5rczsgYm13Zy0N
Cj4gY2hhaXJzQGlldGYub3JnOyBzYmFua3NAZW5jcnlwdGVkLm5ldDsgYm13Z0BpZXRmLm9yZw0K
PiBTdWJqZWN0OiBBZGFtIFJvYWNoJ3MgTm8gT2JqZWN0aW9uIG9uIGRyYWZ0LWlldGYtYm13Zy12
c3dpdGNoLW9wbmZ2LTAzOg0KPiAod2l0aCBDT01NRU5UKQ0KPiANCj4gQWRhbSBSb2FjaCBoYXMg
ZW50ZXJlZCB0aGUgZm9sbG93aW5nIGJhbGxvdCBwb3NpdGlvbiBmb3INCj4gZHJhZnQtaWV0Zi1i
bXdnLXZzd2l0Y2gtb3BuZnYtMDM6IE5vIE9iamVjdGlvbg0KPiANCi4uLg0KPiAgDQo+IC0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0NCj4gQ09NTUVOVDoNCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiANCj4gSSB0aGluayB0aGlz
IGlzIGEgdmFyaWF0aW9uIG9uIHRoZSBpc3N1ZSB0aGF0IEFsaXNzYSBjYWxscyBvdXQsIGJ1dCBJ
J20NCj4gaGF2aW5nIGEgaGFyZCB0aW1lIHJlY29uY2lsaW5nOg0KPiANCj4gICAgSXQncyB1bmxp
a2VseSB0aGF0IHRoZSB2aXJ0dWFsIHN3aXRjaCB3aWxsIGJlIHRoZSBvbmx5IGFwcGxpY2F0aW9u
DQo+ICAgIHJ1bm5pbmcgb24gdGhlIFN5c3RlbSBVbmRlciBUZXN0IChTVVQpLCBzbyBDUFUgdXRp
bGl6YXRpb24sIENhY2hlDQo+ICAgIHV0aWxpemF0aW9uLCBhbmQgTWVtb3J5IGZvb3RwcmludCBz
aG91bGQgYWxzbyBiZSByZWNvcmRlZCBmb3IgdGhlDQo+ICAgIHZpcnR1YWwgaW1wbGVtZW50YXRp
b25zIG9mIGludGVybmV0d29ya2luZyBmdW5jdGlvbnMuDQo+IA0KPiAuLi53aXRoLi4uDQo+IA0K
PiAgICBGdXJ0aGVyLCBiZW5jaG1hcmtpbmcgaXMgcGVyZm9ybWVkIG9uIGEgImJsYWNrLWJveCIg
YmFzaXMsIHJlbHlpbmcNCj4gICAgc29sZWx5IG9uIG1lYXN1cmVtZW50cyBvYnNlcnZhYmxlIGV4
dGVybmFsIHRvIHRoZSBEVVQvU1VULg0KPiANCj4gUGxlYXNlIGFkZCB0ZXh0IHRoYXQgY2xhcmlm
aWVzIGhvdyB0aGUgbWV0cmljcyB0aGF0IDMuMSBzYXlzIHNob3VsZCBiZSByZWNvcmRlZA0KPiBp
biBzZWN0aW9uIDMuMSByZWxhdGUgdG8gYmVuY2htYXJraW5nLg0KPiANCltBQ01dIA0KT2ssIHRo
aXMgaXMgYSBzb21ld2hhdCBkaWZmZXJlbnQgaXNzdWUuDQpUaGVyZSBhcmUgbWV0cmljcyBsaWtl
IENQVSwgQ2FjaGUsIGFuZCBNZW1vcnkgVXRpbGl6YXRpb24NCndoaWNoIGFyZSByZXBvcnRlZCBi
eSBtZWFzdXJlbWVudCBwcm9jZXNzZXMgcnVubmluZyBvbiB0aGUgDQpEZXZpY2UgVW5kZXIgVGVz
dC4gVGhlc2UgYXJlIHVzZWZ1bCBmb3IgdGhlIHVzZXIvb3BlcmF0aW9ucyBjb21tdW5pdHkNCnRv
IGtub3csIGFuZCBtYXkgYWxzbyBiZSB1c2VmdWwgaWYgdGhlcmUgaXMgYSBwcm9ibGVtIGVuY291
bnRlcmVkIA0KZHVyaW5nIHRlc3RpbmcuIEhvd2V2ZXIsIHRoZXNlIG1ldHJpY3MgY2Fubm90IGJl
IHByb21vdGVkIHRvDQoiQmVuY2htYXJrIiBzdGF0dXMsIGJlY2F1c2Ugd2UgY2Fubm90IG1lYXN1
cmUgYW55IG9mIHRoZW0gDQpleHRlcm5hbCB0byB0aGUgRFVUIG9yIFNVVC4gIEluIGZhY3QsIG9u
bHkgdGhlIG1vc3QgaW1wb3J0YW50IA0KZXh0ZXJuYWxseSBvYnNlcnZhYmxlIG1ldHJpY3MgYXJl
IGRlc2lnbmF0ZWQgYXMgQmVuY2htYXJrcw0KaW4gb3VyIGNvbnNlbnN1cyBwcm9jZXNzLCBhbmQg
Qk1XRyByZWZlcnMgdG8gaW50ZXJuYWxseSBtZWFzdXJlZA0KbWV0cmljcyBhcyAid2hpdGUtYm94
IG1ldHJpY3MiIHRvIGNsZWFybHkgZGlzdGluZ3Vpc2ggdGhlbQ0KZnJvbSB0aGUgb3RoZXJzLg0K
DQpTbywgSSBzdWdnZXN0IGFkZGluZyB0aGUgbmV3IHNlbnRlbmNlIGJlbG93Og0KPiAgICBJdCdz
IHVubGlrZWx5IHRoYXQgdGhlIHZpcnR1YWwgc3dpdGNoIHdpbGwgYmUgdGhlIG9ubHkgYXBwbGlj
YXRpb24NCj4gICAgcnVubmluZyBvbiB0aGUgU3lzdGVtIFVuZGVyIFRlc3QgKFNVVCksIHNvIENQ
VSB1dGlsaXphdGlvbiwgQ2FjaGUNCj4gICAgdXRpbGl6YXRpb24sIGFuZCBNZW1vcnkgZm9vdHBy
aW50IHNob3VsZCBhbHNvIGJlIHJlY29yZGVkIGZvciB0aGUNCj4gICAgdmlydHVhbCBpbXBsZW1l
bnRhdGlvbnMgb2YgaW50ZXJuZXR3b3JraW5nIGZ1bmN0aW9ucy4NCisgICAgSG93ZXZlciwgaW50
ZXJuYWxseS1tZWFzdXJlZCBtZXRyaWNzIHN1Y2ggYXMgdGhlc2UgYXJlIG5vdA0KKyAgICBiZW5j
aG1hcmtzOyB0aGV5IHNob3VsZCBiZSB1c2VmdWwgZm9yIHRoZSBhdWRpZW5jZSAob3BlcmF0aW9u
cykNCisgICAgdG8ga25vdywgYW5kIG1heSBhbHNvIGJlIHVzZWZ1bCBpZiB0aGVyZSBpcyBhIHBy
b2JsZW0gZW5jb3VudGVyZWQgDQorICAgIGR1cmluZyB0ZXN0aW5nLg0KDQoNCg==


From nobody Wed Jun  7 19:31:36 2017
Return-Path: <dafranke@akamai.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 10E51128B8D; Wed,  7 Jun 2017 19:31:34 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Daniel Franke <dafranke@akamai.com>
To: <secdir@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv.all@ietf.org, ietf@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149688909394.25708.7443355162118602638@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 19:31:33 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/SlWOofS-VGlbpaZy-f32-w6r6k0>
Subject: [bmwg] Secdir last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 02:31:34 -0000

Reviewer: Daniel Franke
Review result: Ready

I reviewed this document as part of the Security Directorate's ongoing effort
to review all IETF documents being processed by the IESG. These comments were
written primarily for the benefit of the Security Area Directors.  Document
authors, document editors, and WG chairs should treat these comments just like
any other IETF Last Call comments.

This Informational draft outlines methodology for benchmarking switches,
carried out on isolated test networks. It defines no new protocols or device
capabilities in support of the recommended methodology, or anything else which
would introduce any new security considerations.


From nobody Wed Jun  7 21:35:55 2017
Return-Path: <terry.manderson@icann.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 87102129549; Wed,  7 Jun 2017 21:35:53 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Terry Manderson <terry.manderson@icann.org>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149689655345.25660.6260961549529519952.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 21:35:53 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Kh71FX5OvcP1xplW7VakauNY8_k>
Subject: [bmwg] Terry Manderson's Abstain on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 04:35:54 -0000

Terry Manderson has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: Abstain

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I'm not comfortable using the RFC series to describe the _progress_ of the Open
Platform for NFV. This is orthogonal to the immutable nature of the RFC series
as in a very short period the progress on the Open Platform for NFV will be
different, and requiring a BIS or other revisions.  Thus to "inform the
industry of work-in-progress" (to quote the draft) is not something that I see
as a part of the RFC series.

I shall not stand in the way of publication and have balloted ABSTAIN. However,
my advise is to seek other avenues for publication more in line with the
white-paper that is draft-ietf-bmwg-vswitch-opnfv.



From nobody Wed Jun  7 22:05:56 2017
Return-Path: <terry.manderson@icann.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 04C0B127868; Wed,  7 Jun 2017 22:05:49 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Terry Manderson <terry.manderson@icann.org>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149689834892.25668.13617327112002292169.idtracker@ietfa.amsl.com>
Date: Wed, 07 Jun 2017 22:05:48 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Kxt0bvbnByDCTDq_LPGEtyI8sz8>
Subject: [bmwg] Terry Manderson's Abstain on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 05:05:49 -0000

Terry Manderson has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: Abstain

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I'm not comfortable using the RFC series to describe the _progress_ of the Open
Platform for NFV. This is orthogonal to the immutable nature of the RFC series
as in a very short period the progress on the Open Platform for NFV will be
different, and requiring a BIS or other revisions.  Thus to "inform the
industry of work-in-progress" (to quote the draft) is not something that I see
as a part of the RFC series.

I shall not stand in the way of publication and have balloted ABSTAIN. However,
my advice is to seek other avenues for publication more in line with the
white-paper that is draft-ietf-bmwg-vswitch-opnfv.

==-=-=-=-=- added comment
I should also add that I have seen the response in relation to Alvaro's
ABSTAIN, and that does not sway me. If you are interested in documenting the
the test specifications that ARE implemented and any associated metrics, then
that I think is a different document, especially when this document has such
words as "Future/planned test specs include"...



From nobody Wed Jun  7 23:23:45 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 37D4912954D; Wed,  7 Jun 2017 23:23:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.702
X-Spam-Level: 
X-Spam-Status: No, score=-2.702 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mNqxtQjrVi64; Wed,  7 Jun 2017 23:23:33 -0700 (PDT)
Received: from mailproxy5.rcs-rds.ro (mailproxy5.rcs-rds.ro [212.54.124.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8AA17126CD8; Wed,  7 Jun 2017 23:23:33 -0700 (PDT)
Received: from [10.172.5.198] (unknown [10.172.5.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy5.rcs-rds.ro (Postfix) with ESMTPSA id 5496128317A; Thu,  8 Jun 2017 09:23:31 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1496903011; bh=5crSm1gvPEnCjrmynGJ3ynnHiOYH/x5AZjHff5f+wcg=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=IxAAbMvgCn7owXimGtgklnNzK3Y2G4Oej+Efx/LcwtmV4poOiw9sktNvpcPtJeYxL pjwXEeeqgDaaFx7tDfr9DaAyFWfFzg4v0iEyd4dqvhjYcxjVaWN6Xvd2dN3Ei0UZdG M9nWcSM6I/nBYVVAVSxDtb6wK8NpwCpK7oqAoRKU=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
In-Reply-To: <149678432947.3901.17980895572189082124.idtracker@ietfa.amsl.com>
Date: Thu, 8 Jun 2017 09:23:28 +0300
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, Alfred C Morton <acmorton@att.com>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <93A5100B-87BA-4FA5-BAB2-64833B8B405C@rcs-rds.ro>
References: <149678432947.3901.17980895572189082124.idtracker@ietfa.amsl.com>
To: Alia Atlas <akatlas@gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/cXSH9pBLX1He1H6ZWSda32C1RP8>
Subject: Re: [bmwg] Alia Atlas' No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 06:23:36 -0000

Hello Alia,

Thank you for your review.
We will add an informative reference to IEEE802.1ac in the next version.

Best regards,
Marius
> On Jun 7, 2017, at 12:25 AM, Alia Atlas <akatlas@gmail.com> wrote:
>=20
> Alia Atlas has entered the following ballot position for
> draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarki=
ng/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> I note that this document discusses using jumbo Ethernet frames up.
> The IEEE has a standard now for baby giant Ethernet frames - though
> there is still concern about the EtherType value used.   An =
informative
> reference might be helpful.
>=20
>=20


From nobody Wed Jun  7 23:27:22 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 64BFE128C84; Wed,  7 Jun 2017 23:27:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.702
X-Spam-Level: 
X-Spam-Status: No, score=-2.702 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NG_NPlQpm3HS; Wed,  7 Jun 2017 23:27:18 -0700 (PDT)
Received: from mailproxy5.rcs-rds.ro (mailproxy5.rcs-rds.ro [212.54.124.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7C778129ACD; Wed,  7 Jun 2017 23:27:18 -0700 (PDT)
Received: from [10.172.5.198] (unknown [10.172.5.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy5.rcs-rds.ro (Postfix) with ESMTPSA id 247B1283170; Thu,  8 Jun 2017 09:27:17 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1496903237; bh=Tn8S4rlae+ba+fVMOs+TeJA+93f0d5ympCMhwjwe6TE=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=O6OBKoA0MgFwlcBhFjepsGQRoIfDB7abiZ3pH1PtXBTyN0OqBvFYSe8Gu+rsrW4Sc 2V4wnoaHqsIFt7/smHO2MWUn4d8NILdiIvFeU1Fe3CsA2om/RmsobGVDI+3AzZ87KE ciwGXMmc4NP1WUNCH8CFqhs1ANz7oKz2qALTf3jA=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
In-Reply-To: <149684110633.2755.12077292205214337612.idtracker@ietfa.amsl.com>
Date: Thu, 8 Jun 2017 09:27:14 +0300
Cc: The IESG <iesg@ietf.org>, bmwg-chairs@ietf.org, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org, Alfred C Morton <acmorton@att.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <8EAEB7D7-3D9E-47AB-B62F-59961B1D481E@rcs-rds.ro>
References: <149684110633.2755.12077292205214337612.idtracker@ietfa.amsl.com>
To: Alvaro Retana <aretana@cisco.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/4K5TvRFaSXL0GDS9DqfvDTR5CHM>
Subject: Re: [bmwg] Alvaro Retana's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 06:27:20 -0000

Hello Alvaro,

Thank you for your review.
I think that your suggestion is the best way to go and we will review =
the draft accordingly.

Best regards,
Marius
> On Jun 7, 2017, at 4:11 PM, Alvaro Retana <aretana@cisco.com> wrote:
>=20
> Alvaro Retana has entered the following ballot position for
> draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarki=
ng/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> Section 4. (Test Setup):
>=20
>   In terms of route setup, the recommendations of [RFC2544] Section 13
>   are valid for this document assuming that an IPv6 version of the
>   routing packets shown in appendix C.2.6.2 is used.
>=20
> However, rfc2544 says in several places that the packets in the =
appendix are
> just examples.  The frame in C.2.6.2 is a RIP update -- but Section =
11.3
> references the rate at which "frames SHOULD be sent" (also in the =
appendix)
> which include OSPF and IGRP, so I'm assuming that any routing protocol =
used
> should work (if the recommendations are followed in terms of =
frequency, etc.).=20
> I note that rfc5180 doesn't really say anything about routing setup =
for IPv6
> either. :-(
>=20
> I know this is not the document to define a complete set of (or even =
update)
> recommendations for routing setup, so my suggestion is to simply take =
off the
> reference to the appendix:
>=20
>   In terms of route setup, the recommendations of [RFC2544] Section 13
>   are valid for this document assuming that IPv6 capable routing
>   protocols are used.
>=20
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


From nobody Thu Jun  8 00:03:11 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EE6C41201F8; Thu,  8 Jun 2017 00:03:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.702
X-Spam-Level: 
X-Spam-Status: No, score=-2.702 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c2NsrSLt8A7R; Thu,  8 Jun 2017 00:03:01 -0700 (PDT)
Received: from mailproxy5.rcs-rds.ro (mailproxy5.rcs-rds.ro [212.54.124.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34B8A1292F5; Thu,  8 Jun 2017 00:03:01 -0700 (PDT)
Received: from [10.172.5.198] (unknown [10.172.5.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy5.rcs-rds.ro (Postfix) with ESMTPSA id E715C28315A; Thu,  8 Jun 2017 10:02:58 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1496905378; bh=2tozZIz603ib/M+rHz0SaSsFMY25vhERp4Tg30hUJqo=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=q33fx/ny+s0d8KvjsgttR4RFRrUX2AJv9W8+2qi+vXIDF7CHbCcyq/igAtB+Wli9p 8BqBX1QB4RN7v5BYwuMfiGm7q2fabGtSwzu5UN0SruxNi+m3Y1q5jBGsQ84+hznc6s KvdAFK6yuFtEqvklS+nS5YCbO6ATtSzSwCSezoTI=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
In-Reply-To: <149687655097.25630.10122076655027013893.idtracker@ietfa.amsl.com>
Date: Thu, 8 Jun 2017 10:02:55 +0300
Cc: The IESG <iesg@ietf.org>, bmwg-chairs@ietf.org, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org, Alfred C Morton <acmorton@att.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <E664951E-154E-4F6F-834A-BFDD0A980325@rcs-rds.ro>
References: <149687655097.25630.10122076655027013893.idtracker@ietfa.amsl.com>
To: Adam Roach <adam@nostrum.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/HfjPEuyOa_h9euT6J6p40FmjYh8>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 07:03:03 -0000

Hello Adam,

Thank you for your review.
Our intention was to include the most forward looking (popular) IPv6 =
transition technologies in the list.=20
Moreover, when starting the draft, teredo was towards the end of its =
"shelf life".
We can note in the Introduction that the list is not exhaustive.

Best regards,
Marius
=20
> On Jun 8, 2017, at 2:02 AM, Adam Roach <adam@nostrum.com> wrote:
>=20
> Adam Roach has entered the following ballot position for
> draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarki=
ng/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> I was surprised not to find any mention of RFC 4380 (Teredo) in this =
document.
> If its omission was intentional, a statement to that effect (say, in =
the
> introduction) is probably warranted.
>=20
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


From nobody Thu Jun  8 04:36:58 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED7BF12DFDB; Thu,  8 Jun 2017 04:36:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.721
X-Spam-Level: 
X-Spam-Status: No, score=-2.721 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cooperw.in header.b=VIcRgWkX; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Go61TxXV
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NE75EpkAgwEv; Thu,  8 Jun 2017 04:36:55 -0700 (PDT)
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DAF4912EAAA; Thu,  8 Jun 2017 04:36:54 -0700 (PDT)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 5296620AAC; Thu,  8 Jun 2017 07:36:54 -0400 (EDT)
Received: from frontend2 ([10.202.2.161]) by compute7.internal (MEProxy); Thu, 08 Jun 2017 07:36:54 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cooperw.in; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=bemEUIC52Fq76tEXmF 4c05OJTiF9A2+PpepSF35kY6Y=; b=VIcRgWkXBX9WdsvPIeuzTXann5W6iHL+VH 46A47wu5RBweYbik1Mj3n96uN9GcSWZH/bHSbGh9OlVPVDvYEExlT3qwRgr1QiOK rqMrRtpBNdWD9kO/PGj/t9lfmyrUGSTIL823GH1LbvDlrd6Uvzl7hasfAU9H1SsT TuG49jNWlnzfL0j8uQ8u65QUo2BBiS+IS5BRKdYNebYb75QPLNTnXHR2Xh4U768E 6NmUoKI+vYNPg0lIW1JBShal9NPgKEWoofnLZWx8ItrZp5NynIA7iBiBCswLMG+T JpOVO3hB4dJOph2o09hihpTXSqFb4A3Hk9S8vaunYXXTUu/YzwVA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=bemEUIC52Fq76tEXmF4c05OJTiF9A2+PpepSF35kY6Y=; b=Go61TxXV BU6cFoPKjh8yx1oDvhJJYAhJ40kvpdVTSMw8GhPuyHkC22v5cR7+yR769LE+p4uP 64/CIu3weO9vOoPIf5TTI0rLAgOoc38UK3IePqeKnDtchN/8rySvFL0yi05nyZCC 54VRzDRJewX1hLWer+difviR9FQYBZbGqQ+3OI495f+NRH32+cHroQGawhPpHXR+ ydAIALcza9hNBjLdKqxbWLHS9O7MlJZmzb2rxjbzQhLyFHjx3COfw8Luq3CEPTna RolyKQ+7kq4se8ucivjcb9MpP/SFNr7I0/nbEFOJZDbLdGY7AZuIE5ZMQHeivYaa AjGZboDgnlBjeg==
X-ME-Sender: <xms:1jY5WUMGVMUc4wA8C_QmxoqAMW3no2BXORuP8bsVIyrS9EyzwMSr0w>
X-Sasl-enc: LjSEAQwNklozHCNu6SX2PaPWBK48xrxQVQ74W1GtRN69 1496921813
Received: from [10.24.81.116] (unknown [128.107.241.183]) by mail.messagingengine.com (Postfix) with ESMTPA id 0328E248B4; Thu,  8 Jun 2017 07:36:52 -0400 (EDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Alissa Cooper <alissa@cooperw.in>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com>
Date: Thu, 8 Jun 2017 07:36:50 -0400
Cc: IESG <iesg@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>,  Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
References: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/RfkuYJzrTTN_PbSL4pLcRW_Sptk>
Subject: Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 11:36:57 -0000

Hi Al,

> On Jun 7, 2017, at 6:57 PM, MORTON, ALFRED C (AL) <acmorton@att.com> =
wrote:
>=20
> Hi Alissa,
> Thanks for your review, please see reply below.
> Al (for the co-authors)
>=20
>> -----Original Message-----
>> From: Alissa Cooper [mailto:alissa@cooperw.in]
>> Sent: Wednesday, June 07, 2017 5:53 PM
>> To: The IESG
>> Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org; Sarah Banks; bmwg-
>> chairs@ietf.org; sbanks@encrypted.net; bmwg@ietf.org
>> Subject: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03:
>> (with DISCUSS and COMMENT)
> ...
>>=20
>> =
----------------------------------------------------------------------
>> DISCUSS:
>> =
----------------------------------------------------------------------
>>=20
>> Picking up on the thread that started with the Gen-ART review, I'm =
not clear on
>> the position of this document when it comes to repeatability. If all =
of the
>> parameters listed in Section 3.3 (assuming they apply) are configured =
and
>> documented, is it assumed that the benchmarks will be repeatable and =
comparable
>> with hardware implementation benchmarks?
> [ACM]=20
>>>>=20
> Of course, it has been the goal of the project to produce=20
> repeatable results, and a large set of the parameters
> believed to be critical was provided so that the benchmarking
> community could better appreciate the increase in configuration
> complexity inherent in this work. So, yes, this set was assumed=20
> sufficient for the infrastructure in use by the VSPERF project
> to obtain repeatable results from test-to-test.=20
>>>> We could add summary above as a new paragraph in section 3.3.
>=20
> Benchmark Comparability between virtual and physical/hardware=20
> implementations of equivalent functions will likely place more
> detailed and exact requirements on the *testing systems*
> (in terms of stream generation, algorithms to search=20
> for max values, and their configurations of course).=20
> This is another area for standardization to appreciate,
> now that we have a few years testing experience.
> However, the is a topic for a future draft.
>=20
>> Or is the position of the document
>> that the benchmarks are not likely to be repeatable/comparable in =
some (many?)
>> cases, given the increase in complexity? Or that more work needs to =
be done
>> (outside the specification process) to achieve repeatability?
> [ACM]=20
> Thanks to some server re-arrangements in the lab that hosts
> the VSPERF project, we are able to evaluate our long-term
> result repeatability and repeatability across two similar
> hardware platforms in different buildings. I'm still processing
> some of the results we will present next week at the OPNFV Summit,
> but the cross hardware platform results look quite consistent,
> even with a benchmark where Scott Bradner experienced some
> repeatability problems in his lab back in the 90=E2=80=99s.
>=20
>> I think this needs to be more clear in the document.
> [ACM]=20
> If the >>>paragraph<<< accomplishes that for you,
> with some minor tweaks to word it less like a reply, let me know.

I think both of your first two paragraphs above make useful points for =
inclusion in the draft, so if you=E2=80=99re able to write up some text =
that incorporates them, that would resolve my issue.

Thanks,
Alissa


>=20
>>=20
>>=20
>> =
----------------------------------------------------------------------
>> COMMENT:
>> =
----------------------------------------------------------------------
>>=20
>> I understand the debate about the publication of this document as an =
IETF
>> consensus RFC. To me it seems valuable to publish and that this item =
is within
>> the WG's charter. I think adding the updates necessary to make the =
document
>> up-to-date with the Danube 3.0 release as suggested in the thread =
with Alvaro
>> would be valuable.
>>=20
> [ACM]=20
> Great, I've located the list of tests added in Colorado and Danube,
> so I'll add them in section 5, and update the release info =
accordingly.
>=20
>=20


From nobody Thu Jun  8 05:01:23 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6EF70127333; Thu,  8 Jun 2017 05:01:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.702
X-Spam-Level: 
X-Spam-Status: No, score=-2.702 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jFG_dGHPsUUC; Thu,  8 Jun 2017 05:01:18 -0700 (PDT)
Received: from mailproxy5.rcs-rds.ro (mailproxy5.rcs-rds.ro [212.54.124.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C0AFC127078; Thu,  8 Jun 2017 05:01:17 -0700 (PDT)
Received: from [10.172.5.198] (unknown [10.172.5.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy5.rcs-rds.ro (Postfix) with ESMTPSA id 50E252832D1; Thu,  8 Jun 2017 15:01:16 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1496923276; bh=/1JfWIowQG2H7xof+ir92czmkFswx+CafnVapUoPhcE=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=a9S3trR+PSpknUjjAbyS0FsYJZ+1ZspBeFTYovbt3/EFUQL6vnTaoR5YV5jl1R4TA /zw7eUVCY5LBR8ajFD8l16vAfPlHIToC+UUE+jKsTfb3xdKcO1lHkNRIMqKTSvnJ0J lvAIhpsEriELWS96aTMYtvvpfEjuZ68YWAZYIH2M=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
In-Reply-To: <149686304373.2650.7741894590270773104.idtracker@ietfa.amsl.com>
Date: Thu, 8 Jun 2017 15:01:13 +0300
Cc: The IESG <iesg@ietf.org>, bmwg-chairs@ietf.org, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org, Alfred C Morton <acmorton@att.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <26F883D1-E230-4EE6-863F-632981E008EA@rcs-rds.ro>
References: <149686304373.2650.7741894590270773104.idtracker@ietfa.amsl.com>
To: Suresh Krishnan <suresh.krishnan@gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/jq6oGqU-_sn39GxG86j5rCE4oZk>
Subject: Re: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:01:21 -0000

Hello Suresh,

Thank you for your review.
Please see my answers inline.
> On Jun 7, 2017, at 10:17 PM, Suresh Krishnan =
<suresh.krishnan@gmail.com> wrote:
>=20
> Suresh Krishnan has entered the following ballot position for
> draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarki=
ng/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> * I am surprised that this document does not use (and does not even =
mention)
> the Well Known Prefix (64:ff9b::/96) for the algorithmic mapping =
between IPv4
> and IPv6 on translators as specified by RFC6052. Is there a reason why =
this is
> omitted?

Section 3.1 of RFC6052 states the following.

3.1.  Restrictions on the Use of the Well-Known Prefix

   The Well-Known Prefix MUST NOT be used to represent non-global IPv4
   addresses, such as those defined in [RFC1918] or listed in Section 3
   of [RFC5735].  Address translators MUST NOT translate packets in
   which an address is composed of the Well-Known Prefix and a non-
   global IPv4 address; they MUST drop these packets.

Among the prefixes mentioned in Section 3 of RFC5735 is the IPv4 prefix =
reserved for benchmarking.
Keeping this is mind, I think using WKP for benchmarking  is not an =
option.=20
We could note that a /96 from the prefix reserved for benchmarking =
(2001:2::/48) can be used .=20

> * It is not clear from the document whether the time taken by the
> DNS64 resolution procedure is included in the latency measurements. It =
might be
> useful to note this.
>=20

We are not sure what you meant here.=20
It might not be connected with your comment, but DNS64 is benchmarked =
separately from NAT64.
For NAT64, the latency benchmarks detailed in Section 7.2 are indeed =
recommended, but for DNS64 we proposed only the DNS resolution =
performance benchmark, detailed in Section 9.

Best regards,
Marius


From nobody Thu Jun  8 05:09:18 2017
Return-Path: <bclaise@cisco.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id DF8A0127078; Thu,  8 Jun 2017 05:09:16 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Benoit Claise <bclaise@cisco.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org, dromasca@gmail.com <dromasca@gmail.com>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149692375682.25676.2403861848456561410.idtracker@ietfa.amsl.com>
Date: Thu, 08 Jun 2017 05:09:16 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dCw7ZIatNg1RqOO4IovzTeehr-U>
Subject: [bmwg] Benoit Claise's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:09:17 -0000

Benoit Claise has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

For sure, the topic of benchmarking of benchmarking virtual switches is
important in the industry today.

Technical Summary: This draft summarizes the progress of the OPNFV project on
virtual switch performance characterization, and notes that the OPNFV work
builds upon the IETF work, particularly the BMWG work within IETF, with an
emphasis on RFC2544.

I like the cross SDO aspects of this draft, as mentioned in the shepherd
writeup: The document itself was reviewed in ETSI NFV, and used as a reference
by at least one of the TST WG and IFA WF specifications (2 references in
total). The document has also been reviewed by several individuals from the
OPNFV community, and NFVRG (IRTF) is aware of the draft.

I would have preferred to see a document that specifies "benchmarking virtual
switches" instead of "benchmarking virtual switches in OPNFV". In other words,
just looking at the abstract, this is completely different to write:

   This memo describes the way to benchmark virtual switch performance.
   The specifications are based on the Open Platform for NFV (OPNFV) project on
   virtual switch performance "VSPERF".

As opposed to:

   This memo describes the progress of the Open Platform for NFV (OPNFV)
   project on virtual switch performance "VSPERF".

Regarding the "describes the progress of", sentences such as ...
    - This project intends to build ...
    - Therefore, this memo begins to describe the additional considerations ...
    - This memo summarizes the progress of the Open Platform for NFV (OPNFV)
    project on virtual switch performance characterization, "VSPERF", through
    the Brahmaputra (second) release [BrahRel].
... lead me to think that this is unfinished work, or that work will anyway
have to be updated.

Again, looking at the scope:

   The primary purpose and scope of the memo is to inform the industry
   of work-in-progress that builds on the body of extensive BMWG
   literature and experience, and describe the extensions needed for
   benchmarking virtual switches.

I was hoping that the primary purpose and scope of the memo would be the
specifications of the virtual switch benchmarking, not to inform about the
work-in-progress.

Reading further, I see text such as:

    Specifications to be included in future updates of the LTD include:

       o  [RFC3918] Methodology for IP Multicast Benchmarking

       o  [RFC4737] Packet Reordering Metrics

or

   When considering characteristics important to "telco" network
   functions, we must begin to consider additional performance metrics.

or

   Tests have been (or will be) designed to collect the metrics below:

On one side, I appreciate that you're publishing what you learned, which should
lead up to revisions in the future. On the other side, I've not been used to
that approach in BMWG. I'll trust the responsible AD regarding the publication
of this document as a kind of intermediary document.

I believe the document would be a better flow (and that some concerns would
disappear) if it would have the following structure
    - benchmarking virtual switches is key in the industry today
    - there are the virtual switches benchmarking specification today
    - we worked on this with OPNFV (VSPERF). Great collaboration.
    - this is a complex and evolving field:
      compared to RFC 2544, we learn that ...
      we realize that this is work in progress ...
    - future plan: we will need to integrate more "stuff" in the benchmark,
    such as RFC3918, RFC4737, additional perf metrics, you-name-it - we'll work
    with OPNFV (and other groups) on this future plan.

Editorial
- Section 3.4. Flow timeout.
You mean the active timeout, section 2.2.23 in RFC 6645?
Should this section refer to RFC6645, or RFC 5470 (section 5.1.1)?

-

   In addition to this, the LTD also re-uses the terminology defined by:

   o  [RFC2285] Benchmarking Terminology for LAN Switching Devices

   o  [RFC5481] Packet Delay Variation Applicability Statement

RFC5481 is already mentioned in the previous bullet list (The base performance
tests in the LTD are based on the pre-existing specifications developed by BMWG
to test the performance of physical switches. These specifications include). So
hopefully, it re-uses the terminology, right?



From nobody Thu Jun  8 05:19:54 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E5051201F2; Thu,  8 Jun 2017 05:19:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cOtoPsylavev; Thu,  8 Jun 2017 05:19:42 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2F8721293EE; Thu,  8 Jun 2017 05:19:42 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v58CEtgA019497; Thu, 8 Jun 2017 08:19:37 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049463.ppops.net-00191d01. with ESMTP id 2axv32prmw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2017 08:19:36 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJaQF126040; Thu, 8 Jun 2017 07:19:36 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJTtO126012 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 8 Jun 2017 07:19:29 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor); Thu, 8 Jun 2017 12:19:21 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJLOn028957; Thu, 8 Jun 2017 07:19:21 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJAp6028265; Thu, 8 Jun 2017 07:19:10 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 18A8AF0457; Thu,  8 Jun 2017 08:19:10 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 8 Jun 2017 08:19:09 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Alissa Cooper <alissa@cooperw.in>
CC: IESG <iesg@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
Thread-Index: AQHS39hnThUImon/oEeFX43rE6nRmqIZ9L4wgAElqAD//8eUcA==
Date: Thu, 8 Jun 2017 12:19:09 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD70B9@njmtexg5.research.att.com>
References: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com> <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
In-Reply-To: <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-08_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706080220
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2jF3U4gB6IDV6E0I1QpfCZ2ITB8>
Subject: Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:19:52 -0000

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBBbGlzc2EgQ29vcGVyIFttYWls
dG86YWxpc3NhQGNvb3BlcncuaW5dDQo+IFNlbnQ6IFRodXJzZGF5LCBKdW5lIDA4LCAyMDE3IDc6
MzcgQU0NCj4gVG86IE1PUlRPTiwgQUxGUkVEIEMgKEFMKQ0KPiANCj4gSGkgQWwsDQo+IA0KPiA+
IE9uIEp1biA3LCAyMDE3LCBhdCA2OjU3IFBNLCBNT1JUT04sIEFMRlJFRCBDIChBTCkgPGFjbW9y
dG9uQGF0dC5jb20+DQo+IHdyb3RlOg0KPiA+DQo+ID4gSGkgQWxpc3NhLA0KPiA+IFRoYW5rcyBm
b3IgeW91ciByZXZpZXcsIHBsZWFzZSBzZWUgcmVwbHkgYmVsb3cuDQo+ID4gQWwgKGZvciB0aGUg
Y28tYXV0aG9ycykNCj4gPg0KPiA+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+PiBG
cm9tOiBBbGlzc2EgQ29vcGVyIFttYWlsdG86YWxpc3NhQGNvb3BlcncuaW5dDQo+ID4+IFNlbnQ6
IFdlZG5lc2RheSwgSnVuZSAwNywgMjAxNyA1OjUzIFBNDQo+ID4+IFRvOiBUaGUgSUVTRw0KPiA+
PiBDYzogZHJhZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnZAaWV0Zi5vcmc7IFNhcmFoIEJhbmtz
OyBibXdnLQ0KPiA+PiBjaGFpcnNAaWV0Zi5vcmc7IHNiYW5rc0BlbmNyeXB0ZWQubmV0OyBibXdn
QGlldGYub3JnDQo+ID4+IFN1YmplY3Q6IEFsaXNzYSBDb29wZXIncyBEaXNjdXNzIG9uIGRyYWZ0
LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LTAzOg0KPiA+PiAod2l0aCBESVNDVVNTIGFuZCBDT01N
RU5UKQ0KPiA+IC4uLg0KPiA+Pg0KPiA+PiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gLQ0KPiA+PiBESVNDVVNT
Og0KPiA+PiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gLQ0KPiA+Pg0KPiA+PiBQaWNraW5nIHVwIG9uIHRoZSB0
aHJlYWQgdGhhdCBzdGFydGVkIHdpdGggdGhlIEdlbi1BUlQgcmV2aWV3LCBJJ20gbm90IGNsZWFy
IG9uDQo+ID4+IHRoZSBwb3NpdGlvbiBvZiB0aGlzIGRvY3VtZW50IHdoZW4gaXQgY29tZXMgdG8g
cmVwZWF0YWJpbGl0eS4gSWYgYWxsIG9mIHRoZQ0KPiA+PiBwYXJhbWV0ZXJzIGxpc3RlZCBpbiBT
ZWN0aW9uIDMuMyAoYXNzdW1pbmcgdGhleSBhcHBseSkgYXJlIGNvbmZpZ3VyZWQgYW5kDQo+ID4+
IGRvY3VtZW50ZWQsIGlzIGl0IGFzc3VtZWQgdGhhdCB0aGUgYmVuY2htYXJrcyB3aWxsIGJlIHJl
cGVhdGFibGUgYW5kIGNvbXBhcmFibGUNCj4gPj4gd2l0aCBoYXJkd2FyZSBpbXBsZW1lbnRhdGlv
biBiZW5jaG1hcmtzPw0KPiA+IFtBQ01dDQo+ID4+Pj4NCj4gPiBPZiBjb3Vyc2UsIGl0IGhhcyBi
ZWVuIHRoZSBnb2FsIG9mIHRoZSBwcm9qZWN0IHRvIHByb2R1Y2UNCj4gPiByZXBlYXRhYmxlIHJl
c3VsdHMsIGFuZCBhIGxhcmdlIHNldCBvZiB0aGUgcGFyYW1ldGVycw0KPiA+IGJlbGlldmVkIHRv
IGJlIGNyaXRpY2FsIHdhcyBwcm92aWRlZCBzbyB0aGF0IHRoZSBiZW5jaG1hcmtpbmcNCj4gPiBj
b21tdW5pdHkgY291bGQgYmV0dGVyIGFwcHJlY2lhdGUgdGhlIGluY3JlYXNlIGluIGNvbmZpZ3Vy
YXRpb24NCj4gPiBjb21wbGV4aXR5IGluaGVyZW50IGluIHRoaXMgd29yay4gU28sIHllcywgdGhp
cyBzZXQgd2FzIGFzc3VtZWQNCj4gPiBzdWZmaWNpZW50IGZvciB0aGUgaW5mcmFzdHJ1Y3R1cmUg
aW4gdXNlIGJ5IHRoZSBWU1BFUkYgcHJvamVjdA0KPiA+IHRvIG9idGFpbiByZXBlYXRhYmxlIHJl
c3VsdHMgZnJvbSB0ZXN0LXRvLXRlc3QuDQo+ID4+Pj4gV2UgY291bGQgYWRkIHN1bW1hcnkgYWJv
dmUgYXMgYSBuZXcgcGFyYWdyYXBoIGluIHNlY3Rpb24gMy4zLg0KPiA+DQo+ID4gQmVuY2htYXJr
IENvbXBhcmFiaWxpdHkgYmV0d2VlbiB2aXJ0dWFsIGFuZCBwaHlzaWNhbC9oYXJkd2FyZQ0KPiA+
IGltcGxlbWVudGF0aW9ucyBvZiBlcXVpdmFsZW50IGZ1bmN0aW9ucyB3aWxsIGxpa2VseSBwbGFj
ZSBtb3JlDQo+ID4gZGV0YWlsZWQgYW5kIGV4YWN0IHJlcXVpcmVtZW50cyBvbiB0aGUgKnRlc3Rp
bmcgc3lzdGVtcyoNCj4gPiAoaW4gdGVybXMgb2Ygc3RyZWFtIGdlbmVyYXRpb24sIGFsZ29yaXRo
bXMgdG8gc2VhcmNoDQo+ID4gZm9yIG1heCB2YWx1ZXMsIGFuZCB0aGVpciBjb25maWd1cmF0aW9u
cyBvZiBjb3Vyc2UpLg0KPiA+IFRoaXMgaXMgYW5vdGhlciBhcmVhIGZvciBzdGFuZGFyZGl6YXRp
b24gdG8gYXBwcmVjaWF0ZSwNCj4gPiBub3cgdGhhdCB3ZSBoYXZlIGEgZmV3IHllYXJzIHRlc3Rp
bmcgZXhwZXJpZW5jZS4NCj4gPiBIb3dldmVyLCB0aGUgaXMgYSB0b3BpYyBmb3IgYSBmdXR1cmUg
ZHJhZnQuDQo+ID4NCi4uLg0KPiA+PiBJIHRoaW5rIHRoaXMgbmVlZHMgdG8gYmUgbW9yZSBjbGVh
ciBpbiB0aGUgZG9jdW1lbnQuDQo+ID4gW0FDTV0NCj4gPiBJZiB0aGUgPj4+cGFyYWdyYXBoPDw8
IGFjY29tcGxpc2hlcyB0aGF0IGZvciB5b3UsDQo+ID4gd2l0aCBzb21lIG1pbm9yIHR3ZWFrcyB0
byB3b3JkIGl0IGxlc3MgbGlrZSBhIHJlcGx5LCBsZXQgbWUga25vdy4NCj4gDQo+IEkgdGhpbmsg
Ym90aCBvZiB5b3VyIGZpcnN0IHR3byBwYXJhZ3JhcGhzIGFib3ZlIG1ha2UgdXNlZnVsIHBvaW50
cyBmb3INCj4gaW5jbHVzaW9uIGluIHRoZSBkcmFmdCwgc28gaWYgeW914oCZcmUgYWJsZSB0byB3
cml0ZSB1cCBzb21lIHRleHQgdGhhdA0KPiBpbmNvcnBvcmF0ZXMgdGhlbSwgdGhhdCB3b3VsZCBy
ZXNvbHZlIG15IGlzc3VlLg0KW0FDTV0gDQpPSywgdGhlIHBhcmFncmFwaCBiZWdpbm5pbmcgDQoi
QmVuY2htYXJrIENvbXBhcmFiaWxpdHkgYmV0d2VlbiB2aXJ0dWFsIGFuZCBwaHlzaWNhbC9oYXJk
d2FyZS4uLiINCnNlZW1zIHRvIGZpdCBiZXN0IGluIHNlY3Rpb24gMy4xIChDb21wYXJpc29uIHdp
dGggUGh5c2ljYWwgTmV0d29yayBGdW5jdGlvbnMpLg0KDQpBbA0KDQo+IA0KPiBUaGFua3MsDQo+
IEFsaXNzYQ0KPiANCi4uLg0KDQo=


From nobody Thu Jun  8 05:29:47 2017
Return-Path: <dromasca@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 597E0129C6D; Thu,  8 Jun 2017 05:29:40 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Dan Romascanu <dromasca@gmail.com>
To: <gen-art@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology.all@ietf.org, ietf@ietf.org, bmwg@ietf.org, dromasca@gmail.com
X-Test-IDTracker: no
X-IETF-IDTracker: 6.53.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149692498024.25760.14056448148418579949@ietfa.amsl.com>
Date: Thu, 08 Jun 2017 05:29:40 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/iH7xNEqk2g6fAQtxcI0Es18HLmg>
Subject: [bmwg] Genart last call review of draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:29:40 -0000

Reviewer: Dan Romascanu
Review result: Ready with Issues

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-dcbench-methodology-??
Reviewer: Dan Romascanu
Review Date: 2017-06-08
IETF LC End Date: 2017-06-13
IESG Telechat date: Not scheduled for a telechat

Summary:

This Informational I-D describes test and evaluation methodology and
measurement techniques for physical network equipment in the data center. It's
an useful and clearly written document and it is ready for publication, after a
number of minor issues and nits will be addressed.

Major issues:

Minor issues:

1. There is one single mention of the scope being benchmarking of 'physical
network equipment in the data center'. I assume this is written as opposed to
virtual networking equipment. This is not included in the title or repeated any
place else. I suggest to make more clear the scope by mentioning 'physical
equipment' at least once more, for example in the Introduction section.

2. Section 1.2: I am a little concerned about adding additional interpretations
to the keywords in RFC2119.

Especially about:

'MAY: Comprehensive metric for the scenario described'

MAY has a different meaning in 2119, and the fact that a metric is
'comprehensive' (whatever this means) seems to me orthogonal to the 2119
semantics. The proposed interpretation makes little sense to me, if
'comprehensive' than why not required? or at least as a 'should'?

3. Section 6.2: 'The intensity of a microburst MAY be varied ...' It would be
useful to make clear what is meant by 'intensity'.

Nits/editorial comments:

1. The different formatting of the references [1], [2], etc. vs. [RFC2119],
etc. is slightly confusing. Unless there is some good reason I suggest to fix
this. Also use of references is inconsistent through the text, sometimes they
are mentioned, sometimes they are not.

2. Section 2.2: 'Alternatively when a traffic generator CAN NOT be connected '
- this capitalization is not conformant to RFC2119.

3. 'vlan ZZZ' is not consistent with 'vlan X' and 'vlan Y' previously used.

4. Missing reference - RFC 6985

5. Missing expansion and references for DSCP and COS

6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/

7. The methodology (e.g. in section 5) makes an assumption that the DUT has at
least nine ports. This may be trivial in a DC, but it's worth being mentioned



From nobody Thu Jun  8 05:32:14 2017
Return-Path: <marius.georgescul@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 545DB129B41; Thu,  8 Jun 2017 05:32:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NB9JXwJlgJjt; Thu,  8 Jun 2017 05:32:03 -0700 (PDT)
Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E02A91287A7; Thu,  8 Jun 2017 05:32:02 -0700 (PDT)
Received: by mail-wm0-x241.google.com with SMTP id 70so7078456wme.1; Thu, 08 Jun 2017 05:32:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=LlXMTW4ixyfW9ljkmZ1Ihn2SSS7dM+weYLufWU4JalQ=; b=D2fF32ajUvbBCIJ6ZRv2qE6OQp64+y7FeAIJP0Bi9MeIcS7Gsy1+vQ9W7lycfSi70b 49zYo8mo9sCs3nedpqytv+0x7NvpmrN+jzSmOB6l4tvvxKKnnEewDGgnmWAtId4wkXLV egEItfFwve3E8RQqlr0g5MrG1LOB9VGSGpl8wsse4fOZBzEttS4q32nrp6vujywwDt80 EzGDbeWRkD7N/5Vm/8EKGhYQBUcoa3wKK/85QgG8H8FiasrvwSzDpdo7c0XMR64Mf1se XyKIyn+UjPYmRRFuO0MQqczZkcXLUVqERV/lDHGpBTmNmOofcUtJQAUWxhgQvWcLlrPJ RsGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=LlXMTW4ixyfW9ljkmZ1Ihn2SSS7dM+weYLufWU4JalQ=; b=TqsbZK2P34ddO6QOhJr2eZ+/Qyl9km5QfS0+AagsfWWcXVST1GImQlsRokWUr//HkM CV2NONGPKtaw4Gmw0iuZva36U0EKdUjgG3fw8cAY84/UkwNEei5e6i7gJyWR1qVKsiOP 8c++3KMZHcfdol75JC264iBs3BqnaOyxDzWqqWOxlj7EDhS3gSaGZvxflFj2GHRNo9K7 Aaf+18oU5w2mciKerdyNo7SSf3wA4I2GKoDL/VnRM7oUtOxUDLAjIWbMsdL+cvv9RkEd kJ1KX9VfXd7gKPLWSqtdaDWwwmqtKEHgtSBxAvodmvMRa29z06emw+XdyQ1h7VCIJsZY 94MQ==
X-Gm-Message-State: AODbwcDhvvWynHCbd19Dh66mr/vVWNJZENNTVlzav95fN2J0FjFthEC/ 6CE6Utl92Wbdvw==
X-Received: by 10.28.66.2 with SMTP id p2mr3547250wma.20.1496925121399; Thu, 08 Jun 2017 05:32:01 -0700 (PDT)
Received: from [10.172.5.198] ([212.54.122.61]) by smtp.gmail.com with ESMTPSA id i3sm5808437wmb.13.2017.06.08.05.31.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jun 2017 05:32:00 -0700 (PDT)
From: Marius Georgescu <marius.georgescul@gmail.com>
Message-Id: <0E4C7D6C-0727-48F3-8C1B-F42D54C8A1C9@gmail.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_1B28803D-3CC4-4704-A105-5075CF090869"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Date: Thu, 8 Jun 2017 15:31:58 +0300
In-Reply-To: <F5108B2C-9699-45D5-B9D3-53B96778D687@rcs-rds.ro>
Cc: The IESG <iesg@ietf.org>, bmwg-chairs@ietf.org, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org, Alfred C Morton <acmorton@att.com>
To: ben@nostrum.com
References: <F5108B2C-9699-45D5-B9D3-53B96778D687@rcs-rds.ro>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/fGFCXTMi3ZZ5iK2T0kxcVb7Twvo>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:32:05 -0000

--Apple-Mail=_1B28803D-3CC4-4704-A105-5075CF090869
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hello Ben,

Thank you for your review.
Many of the BMWG RFCs I've read use 2119 keywords for the reporting =
format.
In our understanding, the reason is that the output report of a tester =
is supposed to follow the format recommendations.=20

Best regards,
Marius
> On Jun 8, 2017, at 3:14 PM, Marius Georgescu =
<marius.georgescu@rcs-rds.ro> wrote:
>=20
> Ben Campbell has entered the following ballot position for
> draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html =
<https://www.ietf.org/iesg/statement/discuss-criteria.html>
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarki=
ng/ =
<https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmark=
ing/>
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> -7.2, Reporting Format: Is it conventional to use 2119 keywords to =
describe
> report formatting? (Or is this paragraph really about content, rather =
than
> format?)
>=20
> (Comment repeats for other "format" related paragraphs.)
>=20


--Apple-Mail=_1B28803D-3CC4-4704-A105-5075CF090869
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">Hello Ben,<div class=3D""><br class=3D""></div><div =
class=3D"">Thank you for your review.</div><div class=3D"">Many of the =
BMWG RFCs I've read use 2119 keywords for the reporting =
format.</div><div class=3D"">In our understanding, the reason is that =
the output report of a tester is supposed to follow the format =
recommendations.&nbsp;</div><div class=3D""><br class=3D""></div><div =
class=3D"">Best regards,</div><div class=3D"">Marius<br =
class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On =
Jun 8, 2017, at 3:14 PM, Marius Georgescu &lt;<a =
href=3D"mailto:marius.georgescu@rcs-rds.ro" =
class=3D"">marius.georgescu@rcs-rds.ro</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><hr =
style=3D"font-family: Times; font-size: inherit; font-variant-ligatures: =
normal; orphans: 2; widows: 2;" class=3D""><pre =
style=3D"font-variant-ligatures: normal; orphans: 2; widows: 2;" =
class=3D"">Ben Campbell has entered the following ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to <a rel=3D"nofollow" =
href=3D"https://www.ietf.org/iesg/statement/discuss-criteria.html" =
class=3D"">https://www.ietf.org/iesg/statement/discuss-criteria.html</a>
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
<a rel=3D"nofollow" =
href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-be=
nchmarking/" =
class=3D"">https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech=
-benchmarking/</a>



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

-7.2, Reporting Format: Is it conventional to use 2119 keywords to =
describe
report formatting? (Or is this paragraph really about content, rather =
than
format?)

(Comment repeats for other "format" related paragraphs.)</pre><div =
class=3D""><br class=3D""></div></div></div></blockquote></div><br =
class=3D""></div></body></html>=

--Apple-Mail=_1B28803D-3CC4-4704-A105-5075CF090869--


From nobody Thu Jun  8 05:38:54 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A59BA127342; Thu,  8 Jun 2017 05:38:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.879
X-Spam-Level: 
X-Spam-Status: No, score=-1.879 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fLk19F2XMEtw; Thu,  8 Jun 2017 05:38:44 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B8AE61287A7; Thu,  8 Jun 2017 05:38:44 -0700 (PDT)
Received: from [10.0.2.172] (ip-13-230-239-173.toronto.ca.northamericancoax.com [173.239.230.13]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v58Ccetu092485 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 8 Jun 2017 07:38:41 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host ip-13-230-239-173.toronto.ca.northamericancoax.com [173.239.230.13] claimed to be [10.0.2.172]
Content-Type: multipart/alternative; boundary=Apple-Mail-37FDA06A-E216-45F6-BE5F-8FD61993492F
Mime-Version: 1.0 (1.0)
From: Ben Campbell <ben@nostrum.com>
X-Mailer: iPad Mail (14F89)
In-Reply-To: <0E4C7D6C-0727-48F3-8C1B-F42D54C8A1C9@gmail.com>
Date: Thu, 8 Jun 2017 08:38:39 -0400
Cc: Alfred C Morton <acmorton@att.com>, bmwg-chairs@ietf.org, The IESG <iesg@ietf.org>, bmwg@ietf.org, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org
Content-Transfer-Encoding: 7bit
Message-Id: <87BB90C5-4E50-4E11-8411-C8CAE4C97F37@nostrum.com>
References: <F5108B2C-9699-45D5-B9D3-53B96778D687@rcs-rds.ro> <0E4C7D6C-0727-48F3-8C1B-F42D54C8A1C9@gmail.com>
To: Marius Georgescu <marius.georgescul@gmail.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/BaI0s3cps7_CxOrt2JkbPY-f-Pk>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:38:47 -0000

--Apple-Mail-37FDA06A-E216-45F6-BE5F-8FD61993492F
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable



> On Jun 8, 2017, at 8:31 AM, Marius Georgescu <marius.georgescul@gmail.com>=
 wrote:
>=20
> Hello Ben,
>=20
> Thank you for your review.
> Many of the BMWG RFCs I've read use 2119 keywords for the reporting format=
.
> In our understanding, the reason is that the output report of a tester is s=
upposed to follow the format recommendations.=20

Okay. I just wanted to make sure they were used "on purpose".

Thanks!

Ben.

>=20
> Best regards,
> Marius
>> On Jun 8, 2017, at 3:14 PM, Marius Georgescu <marius.georgescu@rcs-rds.ro=
> wrote:
>>=20
>> Ben Campbell has entered the following ballot position for
>> draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection
>>=20
>> When responding, please keep the subject line intact and reply to all
>> email addresses included in the To and CC lines. (Feel free to cut this
>> introductory paragraph, however.)
>>=20
>>=20
>> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html=

>> for more information about IESG DISCUSS and COMMENT positions.
>>=20
>>=20
>> The document, along with other ballot positions, can be found here:
>> https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmark=
ing/
>>=20
>>=20
>>=20
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>=20
>> -7.2, Reporting Format: Is it conventional to use 2119 keywords to descri=
be
>> report formatting? (Or is this paragraph really about content, rather tha=
n
>> format?)
>>=20
>> (Comment repeats for other "format" related paragraphs.)
>>=20
>=20

--Apple-Mail-37FDA06A-E216-45F6-BE5F-8FD61993492F
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div></div><div><br></div><div><br>On Jun 8=
, 2017, at 8:31 AM, Marius Georgescu &lt;<a href=3D"mailto:marius.georgescul=
@gmail.com">marius.georgescul@gmail.com</a>&gt; wrote:<br><br></div><blockqu=
ote type=3D"cite"><div><meta http-equiv=3D"Content-Type" content=3D"text/htm=
l charset=3Dus-ascii">Hello Ben,<div class=3D""><br class=3D""></div><div cl=
ass=3D"">Thank you for your review.</div><div class=3D"">Many of the BMWG RFC=
s I've read use 2119 keywords for the reporting format.</div><div class=3D""=
>In our understanding, the reason is that the output report of a tester is s=
upposed to follow the format recommendations.&nbsp;</div></div></blockquote>=
<div><br></div><div>Okay. I just wanted to make sure they were used "on purp=
ose".</div><div><br></div><div>Thanks!</div><div><br></div><div>Ben.</div><b=
r><blockquote type=3D"cite"><div><div class=3D""><br class=3D""></div><div c=
lass=3D"">Best regards,</div><div class=3D"">Marius<br class=3D""><div><bloc=
kquote type=3D"cite" class=3D""><div class=3D"">On Jun 8, 2017, at 3:14 PM, M=
arius Georgescu &lt;<a href=3D"mailto:marius.georgescu@rcs-rds.ro" class=3D"=
">marius.georgescu@rcs-rds.ro</a>&gt; wrote:</div><br class=3D"Apple-interch=
ange-newline"><div class=3D""><div style=3D"word-wrap: break-word; -webkit-n=
bsp-mode: space; -webkit-line-break: after-white-space;" class=3D""><hr styl=
e=3D"font-family: Times; font-size: inherit; font-variant-ligatures: normal;=
 orphans: 2; widows: 2;" class=3D""><pre style=3D"font-variant-ligatures: no=
rmal; orphans: 2; widows: 2;" class=3D"">Ben Campbell has entered the follow=
ing ballot position for
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to <a rel=3D"nofollow" href=3D"https://www.ietf.org/iesg/statem=
ent/discuss-criteria.html" class=3D"">https://www.ietf.org/iesg/statement/di=
scuss-criteria.html</a>
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
<a rel=3D"nofollow" href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg=
-ipv6-tran-tech-benchmarking/" class=3D"">https://datatracker.ietf.org/doc/d=
raft-ietf-bmwg-ipv6-tran-tech-benchmarking/</a>



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

-7.2, Reporting Format: Is it conventional to use 2119 keywords to describe
report formatting? (Or is this paragraph really about content, rather than
format?)

(Comment repeats for other "format" related paragraphs.)</pre><div class=3D"=
"><br class=3D""></div></div></div></blockquote></div><br class=3D""></div><=
/div></blockquote></body></html>=

--Apple-Mail-37FDA06A-E216-45F6-BE5F-8FD61993492F--


From nobody Thu Jun  8 05:46:17 2017
Return-Path: <giuseppe.fioccola@telecomitalia.it>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4517812EAB5 for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 05:46:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BkQx96SLDkwR for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 05:46:13 -0700 (PDT)
Received: from mx03.telecomitalia.it (mx03.telecomitalia.it [217.169.121.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EC790129432 for <bmwg@ietf.org>; Thu,  8 Jun 2017 05:46:11 -0700 (PDT)
X-AuditID: d9a97917-637ff700000010e0-4b-593947119efb
Received: from TELMBXB02RM001.telecomitalia.local ( [10.14.252.27]) (using TLS with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client did not present a certificate) by mx03.telecomitalia.it () with SMTP id DC.2D.04320.11749395; Thu,  8 Jun 2017 14:46:09 +0200 (CEST)
From: Fioccola Giuseppe <giuseppe.fioccola@telecomitalia.it>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQDnIwGw
Date: Thu, 8 Jun 2017 12:46:08 +0000
Message-ID: <af1863055ab641f28170302a75b46dc6@TELMBXB02RM001.telecomitalia.local>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.14.252.230]
x-ti-disclaimer: Disclaimer1
content-type: text/plain; charset="ISO-8859-1"
content-transfer-encoding: quoted-printable
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrKKsWRmVeSWpSXmKPExsXCxfdHWlfQ3TLSoKOTw2LrsYmMFv1fb7I5 MHm87J/D6LFkyU+mAKaoBkabxLy8/JLEklSFlNTiZFsll8zi5JzEzNzUIoWQ1JzU5PxcJYXM FFslYyWFgpzE5NTc1LwSW6XEgoLUvBQlOy4FDGADVJaZp5Cal5yfkpmXbqvkGeyva2Fhaqlr qGQXWJpaXJKvkJtaXJyYnp6Zr5CasF4w486bH6wFy/grWmdvZWxgPMHTxcjJISFgIvF3bQ9T FyMXh5DAVCaJloPtrCAJNgEbiYOvTrCB2CICgRKT331mArGFBWwlnjc2MEHE7SV2XdkOZRtJ LJ32FKyeRUBF4tHKpSwgNi9Q79vPDcwgtpBAgMTzF5vBbE6g+LIrpxlBbEYBWYkJuxeB2cwC 4hIvpp9ghzhOQGLJnvPMELaoxMvH/1ghbAOJrUv3sUDYihJ7fxyAqpGRWHhkMivEHB2JBbs/ sUHY2hLLFr5mhrhHUOLkzCcsExhFZyFZNwtJyywkLbOQtCxgZFnFKJpbYWCsVwKJs8ySxJzM RL3Mkk2MwLRwc2Wl+A7G9pXOhxgFOBiVeHhL1S0jhVgTy4orcw8xSnAwK4nwHjUACvGmJFZW pRblxxeV5qQWH2L0AYbYRGYp0eR8YMrKK4k3NLGwNDS2sDAytDAzxSGsJM6ryAQ0SyAdmKiy U1MLUotgxjFxcEo1MIoIPmeXz3kq6J8R7ypcyjcrzOlu3MSzaS8NrlzKuu0kalP5ITXhQqnN VFfh2rqApWEi6oXPUp1mRHXkdrbba17YO+vEtV6zJ9ctmHf/8DSN2OdR0czJ933C+unaQstT tjumV6sqBywXD23TaJu/xkn4Z+KyW89762vaPOY3X37Q/nrT9cxSJZbijERDLeai4kQAaM4S FjgDAAA=
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/UfeFVU31aDaK8qH8nHCUCED_VsU>
Subject: [bmwg] R: Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:46:16 -0000

Support the adoption of this draft as the basis document for EVPN and PBB-EV=
PN Benchmarking Methodology.
Thanks,

Giuseppe

-----Messaggio originale-----
Da: bmwg [mailto:bmwg-bounces@ietf.org] Per conto di MORTON, ALFRED C (AL)
Inviato: domenica 4 giugno 2017 00:34
A: bmwg@ietf.org
Oggetto: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07

BMWG,

At our Chicago meeting, the BMWG reviewed a version of the Internet Draft on

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-05

There was sufficient readership present at the meeting and positive/construc=
tive comments on the work, so the chairs asked whether support existed to ad=
opt the draft as a WG item. There was support and no objections. As always,=
 such decisions are officially taken on the mailing list.

This message*, therefore, starts a call for adoption on:

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-07

to run until Friday 23 June 2017. 

Please reply to bmwg@ietf.org indicating:

(1) whether you support addition of the following milestone to 
   the BMWG charter:

date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG

(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07 
   as the basis document for this milestone

(3) whether you commit to reviewing the WG document if adopted

We are aware that BMWG has nearly completed all its existing chartered items=
.  This usually begins discussion of re-chartering, this time with Warren (o=
ur new AD Advisor). The final decision/schedule for adoption will depend on=
 these discussions. 

thanks for your consideration of the draft, and regards,

Al
bmwg co-chair

*using Brian Trammell's recent message to IPPM as a template (thanks!)

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle pers=
one indicate. La diffusione, copia o qualsiasi altra azione derivante dalla=
 conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia=
te ricevuto questo documento per errore siete cortesemente pregati di darne=
 immediata comunicazione al mittente e di provvedere alla sua distruzione, G=
razie. 

This e-mail and any attachments is confidential and may contain privileged i=
nformation intended for the addressee(s) only. Dissemination, copying, print=
ing or use by anybody else is unauthorised. If you are not the intended reci=
pient, please delete this message and any attachments and advise the sender=
 by return e-mail, Thanks. 

Rispetta l'ambiente. Non stampare questa mail se non =E8 necessario.


From nobody Thu Jun  8 05:47:39 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DB5E12EAB5; Thu,  8 Jun 2017 05:47:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.881
X-Spam-Level: 
X-Spam-Status: No, score=-1.881 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Aer_mNyJueo5; Thu,  8 Jun 2017 05:47:36 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 01229129432; Thu,  8 Jun 2017 05:47:35 -0700 (PDT)
Received: from [10.0.2.172] (ip-13-230-239-173.toronto.ca.northamericancoax.com [173.239.230.13]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v58ClTho093324 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 8 Jun 2017 07:47:31 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host ip-13-230-239-173.toronto.ca.northamericancoax.com [173.239.230.13] claimed to be [10.0.2.172]
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Ben Campbell <ben@nostrum.com>
X-Mailer: iPad Mail (14F89)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD60E8@njmtexg5.research.att.com>
Date: Thu, 8 Jun 2017 08:47:29 -0400
Cc: The IESG <iesg@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <36C02E8C-9F70-4C3C-9700-BC6897CA188F@nostrum.com>
References: <149672036394.3976.8004729920890920652.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD60E8@njmtexg5.research.att.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/q6Ul6nSYEzM7_Ac258zpcymgLY0>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:47:38 -0000

Hi, thanks for the response. Please see inline:

> On Jun 6, 2017, at 10:32 AM, MORTON, ALFRED C (AL) <acmorton@att.com> wrot=
e:
>=20
> Hi Ben, thanks for your review, please see below...
>=20
>> -----Original Message-----
>> From: Ben Campbell [mailto:ben@nostrum.com]
>> Sent: Monday, June 05, 2017 11:39 PM
>> To: The IESG
>> Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org; Sarah Banks; bmwg-
>> chairs@ietf.org; sbanks@encrypted.net; bmwg@ietf.org
>> Subject: Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-
>> 03: (with COMMENT)
>>=20
>> Ben Campbell has entered the following ballot position for
>> draft-ietf-bmwg-vswitch-opnfv-03: No Objection
> ...
>>=20
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>=20
>> -General:
>>=20
>> It seems a bit odd to me to use an IETF stream RFC to describe the
>> status of an
>> open source project, even when that project is closely related to IETF
>> work.
>> But I do not object strongly enough to get in the way of publication.
> [ACM]=20
>=20
> Although this work is clearly in BMWG's charter, the WG is not quite
> at the point where we are ready to write new RFCs. However, we=20
> do have a good collaboration with the several OPNFV projects
> doing benchmarking, this one on vSwitches has recommended some
> test methods which we will certainly take into account in future=20
> development. =20
>=20
> This draft, and the VSPERF project work in general, and BMWG's
> understanding of this topic *all* benefited
> from review and comments in BMWG. This is one form of
>=20
> "...interaction between standards development in the IETF,=20
> development of running code, and open source efforts in the industry."
> from https://www.ietf.org/blog/2017/05/iesg-retreat-3/
>=20
> The interaction between open source efforts and BMWG is
> also productive for other drafts; the WG
> is almost ready to ship two new specifications
> for SDN Controller Benchmarking (worked with the OPNFV Cperf project):
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-sdn-controller-benchmark-=
meth/
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-sdn-controller-benchmark-=
term/
>=20

It's great to see close collaboration between an open source project and a w=
orking group. But I don't see how it follows that it's a good idea to publis=
h project status as an RFC.  Especially a project that is moving rapidly. Bu=
t as I said, I'm not going to get in the way of publication. (I'm not going t=
o make a detailed argument, because others have already done so, and I conce=
de that this seems to fit the WG charter.)

>=20
>>=20
>> -3.4: "It is essential to increase the flow timeout   time on a vswitch
>> before
>> conducting any performance tests that do not   intend to measure the
>> flow setup
>> time."
>>=20
>> Does this mean to make allowances for the startup characteristics of virt=
ual
>> network elements, when physical elements might not have such limitations?=
  That
>> seems sort of like optimizing for the test.
> [ACM]=20
> This is an accepted aspect of benchmarking, and applies to both
> physical and virtual network functions, such as switches and routers.
> For example, RFC 2889 says:
>   " ... The DUT/SUT address aging time SHOULD be configured to be greater t=
han
>   the period of the learning phase of the test plus the trial duration
>   plus any configuration time required by the testing device."
> See:
> https://tools.ietf.org/html/rfc2889#section-3
> The learning phase is benchmarked separately.

Okay, I am willing to accept that this is common practice.

>=20
>>=20
>> -4, figures:
>>=20
>> Figure numbers and cross references would be very helpful for this
>> section.
>>=20
> [ACM] OK, we'll add them.

Thanks!

>=20
> thanks again,
> Al
>=20


From nobody Thu Jun  8 06:32:35 2017
Return-Path: <adam@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE5FA12EAF6; Thu,  8 Jun 2017 06:32:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.881
X-Spam-Level: 
X-Spam-Status: No, score=-1.881 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WR9bKRos7KOY; Thu,  8 Jun 2017 06:32:32 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2866212EAED; Thu,  8 Jun 2017 06:32:32 -0700 (PDT)
Received: from [172.18.0.15] (99-152-146-228.lightspeed.dllstx.sbcglobal.net [99.152.146.228]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v58DWPhm097923 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 8 Jun 2017 08:32:26 -0500 (CDT) (envelope-from adam@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host 99-152-146-228.lightspeed.dllstx.sbcglobal.net [99.152.146.228] claimed to be [172.18.0.15]
References: <149687493553.25630.10964914484670596603.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD6E3C@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD6E3C@njmtexg5.research.att.com>
Mime-Version: 1.0 (1.0)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=us-ascii
Message-Id: <6EDBF7CC-F7E5-4437-96A9-5CB91A80A24A@nostrum.com>
Cc: The IESG <iesg@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>,  Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
X-Mailer: iPhone Mail (14E304)
From: Adam Roach <adam@nostrum.com>
Date: Thu, 8 Jun 2017 08:32:19 -0500
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/zc_RV7UqBmACXYQ-Jy8ea8O5uY8>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 13:32:34 -0000

Thanks. Your proposed clarification looks good to me.=20

/a

> On Jun 7, 2017, at 18:20, MORTON, ALFRED C (AL) <acmorton@att.com> wrote:
>=20
> Hi Adam,
> Thanks for your review, please see reply below.
> Al (for the co-authors)
>=20
>> -----Original Message-----
>> From: Adam Roach [mailto:adam@nostrum.com]
>> Sent: Wednesday, June 07, 2017 6:36 PM
>> To: The IESG
>> Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org; Sarah Banks; bmwg-
>> chairs@ietf.org; sbanks@encrypted.net; bmwg@ietf.org
>> Subject: Adam Roach's No Objection on draft-ietf-bmwg-vswitch-opnfv-03:
>> (with COMMENT)
>>=20
>> Adam Roach has entered the following ballot position for
>> draft-ietf-bmwg-vswitch-opnfv-03: No Objection
>>=20
> ...
>>=20
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>=20
>> I think this is a variation on the issue that Alissa calls out, but I'm
>> having a hard time reconciling:
>>=20
>>   It's unlikely that the virtual switch will be the only application
>>   running on the System Under Test (SUT), so CPU utilization, Cache
>>   utilization, and Memory footprint should also be recorded for the
>>   virtual implementations of internetworking functions.
>>=20
>> ...with...
>>=20
>>   Further, benchmarking is performed on a "black-box" basis, relying
>>   solely on measurements observable external to the DUT/SUT.
>>=20
>> Please add text that clarifies how the metrics that 3.1 says should be re=
corded
>> in section 3.1 relate to benchmarking.
>>=20
> [ACM]=20
> Ok, this is a somewhat different issue.
> There are metrics like CPU, Cache, and Memory Utilization
> which are reported by measurement processes running on the=20
> Device Under Test. These are useful for the user/operations community
> to know, and may also be useful if there is a problem encountered=20
> during testing. However, these metrics cannot be promoted to
> "Benchmark" status, because we cannot measure any of them=20
> external to the DUT or SUT.  In fact, only the most important=20
> externally observable metrics are designated as Benchmarks
> in our consensus process, and BMWG refers to internally measured
> metrics as "white-box metrics" to clearly distinguish them
> from the others.
>=20
> So, I suggest adding the new sentence below:
>>   It's unlikely that the virtual switch will be the only application
>>   running on the System Under Test (SUT), so CPU utilization, Cache
>>   utilization, and Memory footprint should also be recorded for the
>>   virtual implementations of internetworking functions.
> +    However, internally-measured metrics such as these are not
> +    benchmarks; they should be useful for the audience (operations)
> +    to know, and may also be useful if there is a problem encountered=20
> +    during testing.
>=20
>=20


From nobody Thu Jun  8 06:46:11 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD73A129B77; Thu,  8 Jun 2017 06:46:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DhfkEzhygGpV; Thu,  8 Jun 2017 06:46:00 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CFA1B129413; Thu,  8 Jun 2017 06:45:59 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v58DjF7p034904; Thu, 8 Jun 2017 09:45:54 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049463.ppops.net-00191d01. with ESMTP id 2axv32uyar-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2017 09:45:53 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v58DjqQx020754; Thu, 8 Jun 2017 09:45:53 -0400
Received: from mlpi407.sfdc.sbc.com (mlpi407.sfdc.sbc.com [130.9.128.239]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v58Djgi4020508 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 8 Jun 2017 09:45:48 -0400
Received: from mlpi432.sfdc.sbc.com (mlpi432.sfdc.sbc.com [144.151.223.11]) by mlpi407.sfdc.sbc.com (RSA Interceptor); Thu, 8 Jun 2017 13:45:31 GMT
Received: from sfdc.sbc.com (localhost [127.0.0.1]) by mlpi432.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id v58DjVEl011262; Thu, 8 Jun 2017 09:45:31 -0400
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by mlpi432.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id v58DjTSo011192; Thu, 8 Jun 2017 09:45:29 -0400
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id 9CE6EE10B0; Thu,  8 Jun 2017 09:45:16 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 8 Jun 2017 09:45:22 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Benoit Claise <bclaise@cisco.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>, "dromasca@gmail.com" <dromasca@gmail.com>
Thread-Topic: Benoit Claise's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
Thread-Index: AQHS4FARB9YnambLi0K4quaSUp7RsaIa4NaA
Date: Thu, 8 Jun 2017 13:45:21 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD71A6@njmtexg5.research.att.com>
References: <149692375682.25676.2403861848456561410.idtracker@ietfa.amsl.com>
In-Reply-To: <149692375682.25676.2403861848456561410.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-08_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706080247
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Tr3Rsd_zoiQcFGExyjO7rS6sytE>
Subject: Re: [bmwg] Benoit Claise's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 13:46:03 -0000

SGkgQmVub2l0LA0KVGhhbmtzIGZvciB5b3VyIHJldmlldy4gcGxlYXNlIHNlZSByZXBsaWVzIGJl
bG93Lg0KSSByZWFsbHkgYXBwcmVjaWF0ZSB0aGUgc3BlY2lmaWMgcG9pbnRlcnMgeW91J3ZlDQpw
cm92aWRlZCBmb3IgcmV2aXNpb25zIHRvIG1ha2UgdGhpcyBhIG1vcmUgDQphcHByb3ByaWF0ZSBh
bmQgdmFsdWFibGUgUkZDLg0KDQpBbCAoZm9yIHRoZSBjby1hdXRob3JzKQ0KDQo+IC0tLS0tT3Jp
Z2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEJlbm9pdCBDbGFpc2UgW21haWx0bzpiY2xhaXNl
QGNpc2NvLmNvbV0NCj4gU2VudDogVGh1cnNkYXksIEp1bmUgMDgsIDIwMTcgODowOSBBTQ0KPiBU
bzogVGhlIElFU0cNCj4gQ2M6IGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2QGlldGYub3Jn
OyBTYXJhaCBCYW5rczsgYm13Zy0NCj4gY2hhaXJzQGlldGYub3JnOyBzYmFua3NAZW5jcnlwdGVk
Lm5ldDsgYm13Z0BpZXRmLm9yZzsgZHJvbWFzY2FAZ21haWwuY29tDQo+IFN1YmplY3Q6IEJlbm9p
dCBDbGFpc2UncyBObyBPYmplY3Rpb24gb24gZHJhZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnYt
DQo+IDAzOiAod2l0aCBDT01NRU5UKQ0KPiANCj4gQmVub2l0IENsYWlzZSBoYXMgZW50ZXJlZCB0
aGUgZm9sbG93aW5nIGJhbGxvdCBwb3NpdGlvbiBmb3INCj4gZHJhZnQtaWV0Zi1ibXdnLXZzd2l0
Y2gtb3BuZnYtMDM6IE5vIE9iamVjdGlvbg0KLi4uDQo+IA0KPiAtLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IENP
TU1FTlQ6DQo+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gDQo+IEZvciBzdXJlLCB0aGUgdG9waWMgb2YgYmVu
Y2htYXJraW5nIG9mIGJlbmNobWFya2luZyB2aXJ0dWFsIHN3aXRjaGVzIGlzDQo+IGltcG9ydGFu
dCBpbiB0aGUgaW5kdXN0cnkgdG9kYXkuDQo+IA0KPiBUZWNobmljYWwgU3VtbWFyeTogVGhpcyBk
cmFmdCBzdW1tYXJpemVzIHRoZSBwcm9ncmVzcyBvZiB0aGUgT1BORlYgcHJvamVjdCBvbg0KPiB2
aXJ0dWFsIHN3aXRjaCBwZXJmb3JtYW5jZSBjaGFyYWN0ZXJpemF0aW9uLCBhbmQgbm90ZXMgdGhh
dCB0aGUgT1BORlYgd29yaw0KPiBidWlsZHMgdXBvbiB0aGUgSUVURiB3b3JrLCBwYXJ0aWN1bGFy
bHkgdGhlIEJNV0cgd29yayB3aXRoaW4gSUVURiwgd2l0aCBhbg0KPiBlbXBoYXNpcyBvbiBSRkMy
NTQ0Lg0KPiANCj4gSSBsaWtlIHRoZSBjcm9zcyBTRE8gYXNwZWN0cyBvZiB0aGlzIGRyYWZ0LCBh
cyBtZW50aW9uZWQgaW4gdGhlIHNoZXBoZXJkDQo+IHdyaXRldXA6IFRoZSBkb2N1bWVudCBpdHNl
bGYgd2FzIHJldmlld2VkIGluIEVUU0kgTkZWLCBhbmQgdXNlZCBhcyBhIHJlZmVyZW5jZQ0KPiBi
eSBhdCBsZWFzdCBvbmUgb2YgdGhlIFRTVCBXRyBhbmQgSUZBIFdGIHNwZWNpZmljYXRpb25zICgy
IHJlZmVyZW5jZXMgaW4NCj4gdG90YWwpLiBUaGUgZG9jdW1lbnQgaGFzIGFsc28gYmVlbiByZXZp
ZXdlZCBieSBzZXZlcmFsIGluZGl2aWR1YWxzIGZyb20gdGhlDQo+IE9QTkZWIGNvbW11bml0eSwg
YW5kIE5GVlJHIChJUlRGKSBpcyBhd2FyZSBvZiB0aGUgZHJhZnQuDQo+IA0KPiBJIHdvdWxkIGhh
dmUgcHJlZmVycmVkIHRvIHNlZSBhIGRvY3VtZW50IHRoYXQgc3BlY2lmaWVzICJiZW5jaG1hcmtp
bmcgdmlydHVhbA0KPiBzd2l0Y2hlcyIgaW5zdGVhZCBvZiAiYmVuY2htYXJraW5nIHZpcnR1YWwg
c3dpdGNoZXMgaW4gT1BORlYiLiBJbiBvdGhlciB3b3JkcywNCj4ganVzdCBsb29raW5nIGF0IHRo
ZSBhYnN0cmFjdCwgdGhpcyBpcyBjb21wbGV0ZWx5IGRpZmZlcmVudCB0byB3cml0ZToNCj4gDQo+
ICAgIFRoaXMgbWVtbyBkZXNjcmliZXMgdGhlIHdheSB0byBiZW5jaG1hcmsgdmlydHVhbCBzd2l0
Y2ggcGVyZm9ybWFuY2UuDQo+ICAgIFRoZSBzcGVjaWZpY2F0aW9ucyBhcmUgYmFzZWQgb24gdGhl
IE9wZW4gUGxhdGZvcm0gZm9yIE5GViAoT1BORlYpIHByb2plY3Qgb24NCj4gICAgdmlydHVhbCBz
d2l0Y2ggcGVyZm9ybWFuY2UgIlZTUEVSRiIuDQo+IA0KPiBBcyBvcHBvc2VkIHRvOg0KPiANCj4g
ICAgVGhpcyBtZW1vIGRlc2NyaWJlcyB0aGUgcHJvZ3Jlc3Mgb2YgdGhlIE9wZW4gUGxhdGZvcm0g
Zm9yIE5GViAoT1BORlYpDQo+ICAgIHByb2plY3Qgb24gdmlydHVhbCBzd2l0Y2ggcGVyZm9ybWFu
Y2UgIlZTUEVSRiIuDQpbQUNNXSANClllcy4gIFdlIHNob3VsZCBoYXZlIG1hZGUgc29tZSB3b3Jk
aW5nIGNoYW5nZXMgd2hlbiB0aGlzIGRyYWZ0DQp0cmFuc2l0aW9uZWQgZnJvbSB0aGUgSW5kaXZp
ZHVhbCBkcmFmdCB0byBXRyBpdGVtLg0KSXQgd291bGQgYmUgZ3JlYXQgdG8gYmUgYWJsZSB0byB3
cml0ZSB5b3VyIHNlbnRlbmNlLCANCmJ1dCB0aGUgV0cgaXMgbm90IHJlYWR5IHRvIGRvIHRoYXQu
DQoNCkhvd2V2ZXIsIHdlIGNvdWxkIHNheSB0aGlzIGluc3RlYWQ6DQoNClRoaXMgbWVtbyBkZXNj
cmliZXMgdGhlIGNvbnRyaWJ1dGlvbnMgb2YgdGhlIE9wZW4gUGxhdGZvcm0gDQpmb3IgTkZWIChP
UE5GVikgcHJvamVjdCBvbiB2aXJ0dWFsIHN3aXRjaCBwZXJmb3JtYW5jZSAiVlNQRVJGIiwgDQpw
YXJ0aWN1bGFybHkgaW4gdGhlIGFyZWFzIG9mIHRlc3Qgc2V0LXVwcyBhbmQgY29uZmlndXJhdGlv
biANCnBhcmFtZXRlcnMgZm9yIHRoZSBzeXN0ZW0gdW5kZXIgdGVzdC4NCg0KYW5kIG1ha2Ugc29t
ZSBvdGhlciB1cGRhdGVzLCBzdWNoIGFzOg0KDQo+IA0KPiBSZWdhcmRpbmcgdGhlICJkZXNjcmli
ZXMgdGhlIHByb2dyZXNzIG9mIiwgc2VudGVuY2VzIHN1Y2ggYXMgLi4uDQo+ICAgICAtIFRoaXMg
cHJvamVjdCBpbnRlbmRzIHRvIGJ1aWxkIC4uLg0KW0FDTV0gDQpUaGlzIHByb2plY3QgaGFzIGV4
dGVuZGVkIC4uLg0KDQo+ICAgICAtIFRoZXJlZm9yZSwgdGhpcyBtZW1vIGJlZ2lucyB0byBkZXNj
cmliZSB0aGUgYWRkaXRpb25hbA0KPiBjb25zaWRlcmF0aW9ucyAuLi4NCltBQ01dIA0KVGhlcmVm
b3JlLCB0aGlzIG1lbW8gZGVzY3JpYmVzIHRoZSBhZGRpdGlvbmFsDQoNCj4gICAgIC0gVGhpcyBt
ZW1vIHN1bW1hcml6ZXMgdGhlIHByb2dyZXNzIG9mIHRoZSBPcGVuIFBsYXRmb3JtIGZvciBORlYg
KE9QTkZWKQ0KW0FDTV0gDQpUaGlzIG1lbW8gZGVzY3JpYmVzIHRoZSBjb250cmlidXRpb25zIG9m
IHRoZSBPcGVuIFBsYXRmb3JtLi4uDQoNCj4gICAgIHByb2plY3Qgb24gdmlydHVhbCBzd2l0Y2gg
cGVyZm9ybWFuY2UgY2hhcmFjdGVyaXphdGlvbiwgIlZTUEVSRiIsIHRocm91Z2gNCj4gICAgIHRo
ZSBCcmFobWFwdXRyYSAoc2Vjb25kKSByZWxlYXNlIFtCcmFoUmVsXS4NCltBQ01dIA0KLi4uIHRo
cm91Z2ggdGhlIERhbnViZSAzLjAgKGZvdXJ0aCkgcmVsZWFzZSB0byB0aGUgY2hhcnRlcmVkIHdv
cmsgDQpvZiB0aGUgQk1XRyAod2l0aCBzdGFibGUgcmVmZXJlbmNlcyB0byB0aGVpciB0ZXN0IGRl
c2NyaXB0aW9ucykuIA0KDQo+IC4uLiBsZWFkIG1lIHRvIHRoaW5rIHRoYXQgdGhpcyBpcyB1bmZp
bmlzaGVkIHdvcmssIG9yIHRoYXQgd29yayB3aWxsIGFueXdheQ0KPiBoYXZlIHRvIGJlIHVwZGF0
ZWQuDQpbQUNNXSANCkFzIEkndmUgbWVudGlvbmVkIGluIG90aGVyIGNvbW1lbnRzLCB0aGUgaW5p
dGlhbCBydXNoIG9mDQp0ZXN0IGRldmVsb3BtZW50IGhhcyB0cmFuc2l0aW9uZWQgdG8gaW5jbHVz
aW9uIG9mIG1vcmUNCnRlc3QgZGV2aWNlcywgaW5jbHVkaW5nIG9wZW4gc291cmNlIHRlc3Rpbmcg
c29mdHdhcmUuDQoNCkJ1dCBJIHRoaW5rIHdpdGggdGhlIGVtcGhhc2lzIG9uIHRoZSBzcGVjaWZp
YyBjb250cmlidXRpb25zDQphbmQgc3RhYmxlIHJlZmVyZW5jZXMsIHRoaXMgbWVtbyBpcyB0YWtp
bmcgYSBtb3JlIHNvbGlkIHN0YW5kDQooYXMgbXVjaCBhcyBhbnl0aGluZyBpcyBpbiBvdXIgd29y
bGQgd2hlcmUgd2hhdCB3ZSBrbm93IGNoYW5nZXMgZmFzdCkuDQogDQo+IA0KPiBBZ2FpbiwgbG9v
a2luZyBhdCB0aGUgc2NvcGU6DQo+IA0KPiAgICBUaGUgcHJpbWFyeSBwdXJwb3NlIGFuZCBzY29w
ZSBvZiB0aGUgbWVtbyBpcyB0byBpbmZvcm0gdGhlIGluZHVzdHJ5DQo+ICAgIG9mIHdvcmstaW4t
cHJvZ3Jlc3MgdGhhdCBidWlsZHMgb24gdGhlIGJvZHkgb2YgZXh0ZW5zaXZlIEJNV0cNCj4gICAg
bGl0ZXJhdHVyZSBhbmQgZXhwZXJpZW5jZSwgYW5kIGRlc2NyaWJlIHRoZSBleHRlbnNpb25zIG5l
ZWRlZCBmb3INCj4gICAgYmVuY2htYXJraW5nIHZpcnR1YWwgc3dpdGNoZXMuDQo+IA0KPiBJIHdh
cyBob3BpbmcgdGhhdCB0aGUgcHJpbWFyeSBwdXJwb3NlIGFuZCBzY29wZSBvZiB0aGUgbWVtbyB3
b3VsZCBiZSB0aGUNCj4gc3BlY2lmaWNhdGlvbnMgb2YgdGhlIHZpcnR1YWwgc3dpdGNoIGJlbmNo
bWFya2luZywgbm90IHRvIGluZm9ybSBhYm91dCB0aGUNCj4gd29yay1pbi1wcm9ncmVzcy4NCltB
Q01dIA0KWWVzLCBzbyBsZXQncyBzYXkgdGhpcyBpbnN0ZWFkOg0KDQpUaGUgcHJpbWFyeSBwdXJw
b3NlIGFuZCBzY29wZSBvZiB0aGUgbWVtbyBpcyB0byBkZXNjcmliZSBrZXkgDQphc3BlY3RzIG9m
IHZzd2l0Y2ggYmVuY2htYXJraW5nLCBwYXJ0aWN1bGFybHkgaW4gdGhlIGFyZWFzIG9mIHRlc3Qg
DQpzZXQtdXBzIGFuZCBjb25maWd1cmF0aW9uIHBhcmFtZXRlcnMgZm9yIHRoZSBzeXN0ZW0gdW5k
ZXIgdGVzdCwgDQphbmQgYnVpbGQgb24gdGhlIGJvZHkgb2YgZXh0ZW5zaXZlIEJNV0cgbGl0ZXJh
dHVyZSBhbmQgZXhwZXJpZW5jZS4NCg0KPiANCj4gUmVhZGluZyBmdXJ0aGVyLCBJIHNlZSB0ZXh0
IHN1Y2ggYXM6DQo+IA0KPiAgICAgU3BlY2lmaWNhdGlvbnMgdG8gYmUgaW5jbHVkZWQgaW4gZnV0
dXJlIHVwZGF0ZXMgb2YgdGhlIExURCBpbmNsdWRlOg0KPiANCj4gICAgICAgIG8gIFtSRkMzOTE4
XSBNZXRob2RvbG9neSBmb3IgSVAgTXVsdGljYXN0IEJlbmNobWFya2luZw0KPiANCj4gICAgICAg
IG8gIFtSRkM0NzM3XSBQYWNrZXQgUmVvcmRlcmluZyBNZXRyaWNzDQo+IA0KW0FDTV0gdHVybmlu
ZyB0aGlzIGludG8gYSByZWNvbW1lbmRhdGlvbjoNCkl0IGlzIHJlY29tbWVuZGVkIHRoYXQgdGhl
c2UgcmVmZXJlbmNlcyBhcmUgaW5jbHVkZWQgaW4gZnV0dXJlIGJlbmNobWFya2luZyBzcGVjaWZp
Y2F0aW9uczoNCg0KPiBvcg0KPiANCj4gICAgV2hlbiBjb25zaWRlcmluZyBjaGFyYWN0ZXJpc3Rp
Y3MgaW1wb3J0YW50IHRvICJ0ZWxjbyIgbmV0d29yaw0KPiAgICBmdW5jdGlvbnMsIHdlIG11c3Qg
YmVnaW4gdG8gY29uc2lkZXIgYWRkaXRpb25hbCBwZXJmb3JtYW5jZSBtZXRyaWNzLg0KW0FDTV0g
DQpXaGVuIGNvbnNpZGVyaW5nIGNoYXJhY3RlcmlzdGljcyBpbXBvcnRhbnQgdG8gInRlbGNvIiBu
ZXR3b3JrIA0KZnVuY3Rpb25zLCBhZGRpdGlvbmFsIHBlcmZvcm1hbmNlIG1ldHJpY3MgYXJlIG5l
ZWRlZC4NCj4gDQo+IG9yDQo+IA0KPiAgICBUZXN0cyBoYXZlIGJlZW4gKG9yIHdpbGwgYmUpIGRl
c2lnbmVkIHRvIGNvbGxlY3QgdGhlIG1ldHJpY3MgYmVsb3c6DQpbQUNNXSAodGhlc2UgYXJlIGFs
bCBpbmNsdWRlZCBub3cpOg0KVGVzdHMgaGF2ZSBiZWVuIGRlc2lnbmVkIHRvIGNvbGxlY3QgdGhl
IG1ldHJpY3MgYmVsb3c6DQoNCj4gDQo+IE9uIG9uZSBzaWRlLCBJIGFwcHJlY2lhdGUgdGhhdCB5
b3UncmUgcHVibGlzaGluZyB3aGF0IHlvdSBsZWFybmVkLCB3aGljaCBzaG91bGQNCj4gbGVhZCB1
cCB0byByZXZpc2lvbnMgaW4gdGhlIGZ1dHVyZS4gT24gdGhlIG90aGVyIHNpZGUsIEkndmUgbm90
IGJlZW4gdXNlZCB0bw0KPiB0aGF0IGFwcHJvYWNoIGluIEJNV0cuIEknbGwgdHJ1c3QgdGhlIHJl
c3BvbnNpYmxlIEFEIHJlZ2FyZGluZyB0aGUgcHVibGljYXRpb24NCj4gb2YgdGhpcyBkb2N1bWVu
dCBhcyBhIGtpbmQgb2YgaW50ZXJtZWRpYXJ5IGRvY3VtZW50Lg0KW0FDTV0gDQpUaGFuayB5b3Uu
IFdlIGtub3cgd2UgaGF2ZSBhIGxvdCBtb3JlIHRvIGxlYXJuIGJlZm9yZSB3ZSANCnVwZGF0ZSB0
cnVseSBmdW5kYW1lbnRhbCBSRkNzIGxpa2UgMjU0NCBhbmQgMjg4OS4NClRoYXQgd291bGQgYmUg
YSBiaWcgc3RlcCwgc28gd2UgbmVlZCB0byBiZSB2ZXJ5IHN1cmUuDQoNCj4gDQo+IEkgYmVsaWV2
ZSB0aGUgZG9jdW1lbnQgd291bGQgYmUgYSBiZXR0ZXIgZmxvdyAoYW5kIHRoYXQgc29tZSBjb25j
ZXJucyB3b3VsZA0KPiBkaXNhcHBlYXIpIGlmIGl0IHdvdWxkIGhhdmUgdGhlIGZvbGxvd2luZyBz
dHJ1Y3R1cmUNCj4gICAgIC0gYmVuY2htYXJraW5nIHZpcnR1YWwgc3dpdGNoZXMgaXMga2V5IGlu
IHRoZSBpbmR1c3RyeSB0b2RheQ0KPiAgICAgLSB0aGVyZSBhcmUgdGhlIHZpcnR1YWwgc3dpdGNo
ZXMgYmVuY2htYXJraW5nIHNwZWNpZmljYXRpb24gdG9kYXkNCj4gICAgIC0gd2Ugd29ya2VkIG9u
IHRoaXMgd2l0aCBPUE5GViAoVlNQRVJGKS4gR3JlYXQgY29sbGFib3JhdGlvbi4NCj4gICAgIC0g
dGhpcyBpcyBhIGNvbXBsZXggYW5kIGV2b2x2aW5nIGZpZWxkOg0KPiAgICAgICBjb21wYXJlZCB0
byBSRkMgMjU0NCwgd2UgbGVhcm4gdGhhdCAuLi4NCj4gICAgICAgd2UgcmVhbGl6ZSB0aGF0IHRo
aXMgaXMgd29yayBpbiBwcm9ncmVzcyAuLi4NCj4gICAgIC0gZnV0dXJlIHBsYW46IHdlIHdpbGwg
bmVlZCB0byBpbnRlZ3JhdGUgbW9yZSAic3R1ZmYiIGluIHRoZSBiZW5jaG1hcmssDQo+ICAgICBz
dWNoIGFzIFJGQzM5MTgsIFJGQzQ3MzcsIGFkZGl0aW9uYWwgcGVyZiBtZXRyaWNzLCB5b3UtbmFt
ZS1pdCAtIHdlJ2xsIHdvcmsNCj4gICAgIHdpdGggT1BORlYgKGFuZCBvdGhlciBncm91cHMpIG9u
IHRoaXMgZnV0dXJlIHBsYW4uDQpbQUNNXSANCkhvcGVmdWxseSwgdGhlIHN1Z2dlc3RlZCBjaGFu
Z2VzIGFib3ZlIHdpbGwgYWNjb21wbGlzaCB0aGF0LA0KYnV0IHNvbWUgb2YgeW91ciBpZGVhcyBj
YW4gYmUgZWFzaWx5IGFkZGVkIHRvIHRoZSBJbnRyb2R1Y3Rpb24uDQpQbGVhc2UgdHJ1c3QgbWUg
Zm9yIHRoYXQgdGV4dC4uLg0KPiANCj4gRWRpdG9yaWFsDQo+IC0gU2VjdGlvbiAzLjQuIEZsb3cg
dGltZW91dC4NCj4gWW91IG1lYW4gdGhlIGFjdGl2ZSB0aW1lb3V0LCBzZWN0aW9uIDIuMi4yMyBp
biBSRkMgNjY0NT8NCj4gU2hvdWxkIHRoaXMgc2VjdGlvbiByZWZlciB0byBSRkM2NjQ1LCBvciBS
RkMgNTQ3MCAoc2VjdGlvbiA1LjEuMSk/DQo+IA0KW0FDTV0gDQpCZW4gcmFpc2VkIGEgc2ltaWxh
ciBjb21tZW50LiAgDQpJJ2xsIGFkZCB0aGUgYmVzdCByZWZlcmVuY2UgZm9yIEJNV0cgZG9jcywg
d2hpY2ggaXMNCg0KRm9yIGV4YW1wbGUsIFJGQyAyODg5IHNheXM6DQogICAiIC4uLiBUaGUgRFVU
L1NVVCBhZGRyZXNzIGFnaW5nIHRpbWUgU0hPVUxEIGJlIGNvbmZpZ3VyZWQgdG8gYmUgZ3JlYXRl
ciB0aGFuDQogICB0aGUgcGVyaW9kIG9mIHRoZSBsZWFybmluZyBwaGFzZSBvZiB0aGUgdGVzdCBw
bHVzIHRoZSB0cmlhbCBkdXJhdGlvbg0KICAgcGx1cyBhbnkgY29uZmlndXJhdGlvbiB0aW1lIHJl
cXVpcmVkIGJ5IHRoZSB0ZXN0aW5nIGRldmljZS4iDQpTZWU6DQpodHRwczovL3Rvb2xzLmlldGYu
b3JnL2h0bWwvcmZjMjg4OSNzZWN0aW9uLTMNClRoZSBsZWFybmluZyBwaGFzZSBpcyBiZW5jaG1h
cmtlZCBzZXBhcmF0ZWx5Lg0KDQoNCj4gLQ0KPiANCj4gICAgSW4gYWRkaXRpb24gdG8gdGhpcywg
dGhlIExURCBhbHNvIHJlLXVzZXMgdGhlIHRlcm1pbm9sb2d5IGRlZmluZWQgYnk6DQo+IA0KPiAg
ICBvICBbUkZDMjI4NV0gQmVuY2htYXJraW5nIFRlcm1pbm9sb2d5IGZvciBMQU4gU3dpdGNoaW5n
IERldmljZXMNCj4gDQo+ICAgIG8gIFtSRkM1NDgxXSBQYWNrZXQgRGVsYXkgVmFyaWF0aW9uIEFw
cGxpY2FiaWxpdHkgU3RhdGVtZW50DQo+IA0KPiBSRkM1NDgxIGlzIGFscmVhZHkgbWVudGlvbmVk
IGluIHRoZSBwcmV2aW91cyBidWxsZXQgbGlzdCAoVGhlIGJhc2UgcGVyZm9ybWFuY2UNCj4gdGVz
dHMgaW4gdGhlIExURCBhcmUgYmFzZWQgb24gdGhlIHByZS1leGlzdGluZyBzcGVjaWZpY2F0aW9u
cyBkZXZlbG9wZWQgYnkgQk1XRw0KPiB0byB0ZXN0IHRoZSBwZXJmb3JtYW5jZSBvZiBwaHlzaWNh
bCBzd2l0Y2hlcy4gVGhlc2Ugc3BlY2lmaWNhdGlvbnMgaW5jbHVkZSkuIFNvDQo+IGhvcGVmdWxs
eSwgaXQgcmUtdXNlcyB0aGUgdGVybWlub2xvZ3ksIHJpZ2h0Pw0KW0FDTV0gDQpSaWdodCwgNTQ4
MSByZW1vdmVkLg0KDQpFeGNlcHQgZm9yIHRoZSBJbnRybyBwYXJhZ3JhcGggSSBtZW50aW9uZWQs
IA0KdGhlIHdvcmtpbmcgdGV4dCBpcyByZXZpc2VkIGFzIGRlc2NyaWJlZC4NCj4gDQoNCg==


From nobody Thu Jun  8 07:00:08 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79E7712943B; Thu,  8 Jun 2017 07:00:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wbkOnfq8WaNO; Thu,  8 Jun 2017 07:00:05 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DF44B126C3D; Thu,  8 Jun 2017 07:00:04 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v58DtI9X023777; Thu, 8 Jun 2017 10:00:00 -0400
Received: from flpd657.enaf.ffdc.sbc.com (sbcsmtp9.sbc.com [144.160.128.153]) by mx0a-00191d01.pphosted.com with ESMTP id 2axv8cvy4n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2017 10:00:00 -0400
Received: from enaf.ffdc.sbc.com (localhost [127.0.0.1]) by flpd657.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id v58DxxWb039888; Thu, 8 Jun 2017 07:00:00 -0700
Received: from flpi488.ffdc.sbc.com (flpi488.ffdc.sbc.com [130.4.162.182]) by flpd657.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id v58Dxr4s039745 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 8 Jun 2017 06:59:54 -0700
Received: from tlpd252.dadc.sbc.com (tlpd252.dadc.sbc.com [135.31.184.157]) by flpi488.ffdc.sbc.com (RSA Interceptor); Thu, 8 Jun 2017 13:59:34 GMT
Received: from dadc.sbc.com (localhost [127.0.0.1]) by tlpd252.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v58DxYpb069929; Thu, 8 Jun 2017 08:59:34 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by tlpd252.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v58DxQhP069818; Thu, 8 Jun 2017 08:59:26 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 31269F0467; Thu,  8 Jun 2017 09:59:26 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 8 Jun 2017 09:59:26 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Ben Campbell <ben@nostrum.com>
CC: The IESG <iesg@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
Thread-Index: AQHS3naBJlhcXyqY3kym69nuhrGy2aIX3CYwgANUwYD//87hEA==
Date: Thu, 8 Jun 2017 13:59:25 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD7237@njmtexg5.research.att.com>
References: <149672036394.3976.8004729920890920652.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD60E8@njmtexg5.research.att.com> <36C02E8C-9F70-4C3C-9700-BC6897CA188F@nostrum.com>
In-Reply-To: <36C02E8C-9F70-4C3C-9700-BC6897CA188F@nostrum.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-08_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706080250
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/GjBTOgJEBkJFj-Ki3R1Lx1NtwT8>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 14:00:06 -0000

> -----Original Message-----
> From: Ben Campbell [mailto:ben@nostrum.com]
> Sent: Thursday, June 08, 2017 8:47 AM
> To: MORTON, ALFRED C (AL)
> Cc: The IESG; sbanks@encrypted.net; bmwg-chairs@ietf.org; draft-ietf-
> bmwg-vswitch-opnfv@ietf.org; bmwg@ietf.org
> Subject: Re: Ben Campbell's No Objection on draft-ietf-bmwg-vswitch-
> opnfv-03: (with COMMENT)
>=20
> Hi, thanks for the response. Please see inline:
>=20
...
> >
>=20
> It's great to see close collaboration between an open source project and
> a working group. But I don't see how it follows that it's a good idea to
> publish project status as an RFC.  Especially a project that is moving
> rapidly. But as I said, I'm not going to get in the way of publication.
> (I'm not going to make a detailed argument, because others have already
> done so, and I concede that this seems to fit the WG charter.)
>=20
> >
...
[ACM]=20
Thanks.

Using Benoit's comments, I've suggested wording that makes this
draft more about *contributions* to work in BMWG, and less about
"progress" of the VSPERF project. The authors wanted BMWG to=20
take advantage of what we learned, and the best way to do that
was to clearly describe new test setups and new configuration
parameters in a less ephemeral form (wikis and OS docs disappear).
We also wanted to help BMWG folks appreciate the rest of the work
that was needed, and make some recommendations.=20

I'm sure the VSPERF project folks will continue to collaborate
as long it is useful.
Al


From nobody Thu Jun  8 11:34:21 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1BC9012EB45 for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 11:34:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QLRBBbDPyYBY for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 11:34:18 -0700 (PDT)
Received: from mail-ua0-x232.google.com (mail-ua0-x232.google.com [IPv6:2607:f8b0:400c:c08::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 94DCC12EB39 for <bmwg@ietf.org>; Thu,  8 Jun 2017 11:32:34 -0700 (PDT)
Received: by mail-ua0-x232.google.com with SMTP id h39so23688443uaa.3 for <bmwg@ietf.org>; Thu, 08 Jun 2017 11:32:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=m85O34Jh+nVCRPsbfeARcI/Oz+1Q7Ch2kTKXoWSd5Y0=; b=KAAzlnRhey7JsmrzsrlPETkaNevkPASnyCEaETdFstiimMnRIvE35PubrlSow2bbiJ nEjWkcObMRvfklE6SR6uUcmy8C5YcFxUyYVE95lE3y4cnJR+p6cunzfuQnO1dPzm+/SU gLMSN2XNNAkfN3fCrcUjgESFdcoKWbm+cFS0nNY+++ZNyd9rLOKHqhi8Rz3Q/+TioMUu HLWf9i5KBoOgQpGby039kVNv/kRGf/r/GFQ7Y+yiVsy0CjrLYViiNwJFXTOjbieFxmd4 5dyZYGPrntlqduNcpByRSeKN17seFF0iwHxcSey9cykRbHrQ1dJf+Gy8sMhVuafFOWZj 5nEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=m85O34Jh+nVCRPsbfeARcI/Oz+1Q7Ch2kTKXoWSd5Y0=; b=crOmIoA8upniw2SYTOlw+8O+jUsldgqKmJw7ManvmKQmsq0Eoxi8tGoK4em6oWRC+0 MJM0l6EPG2nc6y0PrvfRqqEBZxRPSCy3WZ5mQiUhgNzUWoMpy9CFzCQ0AVaRSKjwaXQ4 jhwftgDlMKJ389MbiB26Rx9eE/oLkrrtOox+nCi2Yahg8GjB7HekqaPzhyX1Wjrqye/v M71p0MJMvOJ4uc9dBLE4JOmry6eA8KgJ/N8diOnCM6ncDqv0OzQgmdvfxaD4V52oXoG+ HGusujy5kZFkD5SndnKboEaCA0Aw5RqQ94hJbwZxcAn7XBj6piJXJM/RF9BPrViLwYmQ 4hIw==
X-Gm-Message-State: AODbwcBh5aNNp4ZjNWsZoM1VoG3eOJhwxV87Rvb49ytNS0tzbJuPTkL6 mRRCeZPNzQXa2+bZQ2JBah6JgVTMvT16qnE=
X-Received: by 10.176.92.71 with SMTP id a7mr18696771uag.155.1496946753568; Thu, 08 Jun 2017 11:32:33 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.82.198 with HTTP; Thu, 8 Jun 2017 11:31:53 -0700 (PDT)
From: Warren Kumari <warren@kumari.net>
Date: Thu, 8 Jun 2017 14:31:53 -0400
Message-ID: <CAHw9_iJ8Q+6bmz7=ZXaFNH5ofOweM8uhCh-r=HffUjkGw1dMEg@mail.gmail.com>
To: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/HrXxYKnZgR-5EPDwVEVoctxaSnE>
Subject: [bmwg] Final edits on draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 18:34:20 -0000

Hi there,

Your document was on the IESG telechat today, and is now in Approved,
Point Raised raised state. It doesn't strictly **need** a revised ID,
but I do think that incorporating these helpful comments /
clarifications.

Alia thinks it could do with an informative reference for baby jumbo
frames, and says that she has provided a helpful to Al. I think that
it would be useful to include that, and, while you are editing it,
also address Alvaro and Adam's points (they are simply
clarifications).

Please explicitly let me know once done, and I can finish the
approval, and ship it to the RFC ed.

W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Thu Jun  8 11:38:24 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 041AE1298BA; Thu,  8 Jun 2017 11:38:22 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149694710197.15071.11570630378895730538@ietfa.amsl.com>
Date: Thu, 08 Jun 2017 11:38:22 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/j7D4sypdjz9r8gH8B8GmTBw-3Zo>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-vswitch-opnfv-04.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 18:38:22 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Virtual Switches in OPNFV
        Authors         : Maryam Tahhan
                          Billy O'Mahony
                          Al Morton
	Filename        : draft-ietf-bmwg-vswitch-opnfv-04.txt
	Pages           : 24
	Date            : 2017-06-08

Abstract:
   This memo describes the contributions of the Open Platform for NFV
   (OPNFV) project on virtual switch performance "VSPERF", particularly
   in the areas of test set-ups and configuration parameters for the
   system under test.  This project has extended the current and
   completed work of the Benchmarking Methodology Working Group in IETF,
   and references existing literature.  The Benchmarking Methodology
   Working Group has traditionally conducted laboratory characterization
   of dedicated physical implementations of internetworking functions.
   Therefore, this memo describes the additional considerations when
   virtual switches are implemented in general-purpose hardware.  The
   expanded tests and benchmarks are also influenced by the OPNFV
   mission to support virtualization of the "telco" infrastructure.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-vswitch-opnfv-04
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-vswitch-opnfv-04

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-vswitch-opnfv-04


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun  8 11:56:15 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0E89F128CF0 for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 11:56:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FYC5HdsvrrMe for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 11:56:11 -0700 (PDT)
Received: from mail-ua0-x231.google.com (mail-ua0-x231.google.com [IPv6:2607:f8b0:400c:c08::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C9841270FC for <bmwg@ietf.org>; Thu,  8 Jun 2017 11:56:09 -0700 (PDT)
Received: by mail-ua0-x231.google.com with SMTP id q15so23959049uaa.2 for <bmwg@ietf.org>; Thu, 08 Jun 2017 11:56:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Aiasbe7+Bj5A99JOXC+CjYPENb5hKoN6ny0rXqVPY3E=; b=rKKV9kMCeoSllaZT/qF4OE+LNANZEaGdJHC3BincE+1zbu9aSiiQ0iRk3w7WSyBaLE nEEl19rWl9acjd03r51FsiLo25d82Ajaixw5IBlD/ILNGBAwRJJPRtG7V/KDjuYoekdY HvMjCO1JobzitxOXwsJEHc5tpuxJqTEczOF68EhW2gM7uaA4N3/TCVKcasoFXtjSSxH6 NwPeuvzasNgQmYGpoBM6EFeFsvr+yELEeBHsCFmseIXE0FJ1Y+j879CEI+TE0uDn4L60 D5NUrzfIcv8MdGlI7elI9maKKO17qDYra8phb6wW1kDGUdz4yxjM1311JW/HVdud5zk2 AKkA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Aiasbe7+Bj5A99JOXC+CjYPENb5hKoN6ny0rXqVPY3E=; b=IVHHszhmXHYZnD6Bh5yD0ktNKOFtxn0PF4YSeMuh6ixN0cptI/TqkjFfKCwNkgYwlk Rl6ALhXroyyK47Bu3YLoeTXSrEjBbGfiEgSg89NipMtTgpQUyaIa3RerDWUXDurxxSWX BQLs6cCScX4DfcfwNolWmBq3DQ1aeXj1tK4Hl/ZuQ115owSVf0l3eMrDawXKa4AwQP5r jDYxVCfKJUgK1Hu7W0Mkq6tR1+feL3N+4XQKsyl38TJ0MHZxBjz7A+eHlyMIemzPk7iQ iM/lBTE2Y70tfIA+eY3ySzQ3tjTrUndlk2xlQOM0Jpt/7Tpwgq78N6POv5rwbya/M71B CLtg==
X-Gm-Message-State: AODbwcAhj1PeN7cJtMQfmSYuNI6jedotGOoTuEw7gC58dXickCguFAle PpMAUnlWQJ6IQAQRn+ZTFxKm3VE+p7IJ
X-Received: by 10.176.0.248 with SMTP id 111mr17268680uaj.133.1496948168181; Thu, 08 Jun 2017 11:56:08 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.82.198 with HTTP; Thu, 8 Jun 2017 11:55:27 -0700 (PDT)
In-Reply-To: <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
References: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com> <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
From: Warren Kumari <warren@kumari.net>
Date: Thu, 8 Jun 2017 14:55:27 -0400
Message-ID: <CAHw9_iK=KdgR4djxu1C4SQtmRVNxXPf6r2N-tWZJyi__=cN3Kw@mail.gmail.com>
To: Alissa Cooper <alissa@cooperw.in>
Cc: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, IESG <iesg@ietf.org>,  "bmwg@ietf.org" <bmwg@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/h4v0pOQht_AiqhN2Wg9lVajVJRY>
Subject: Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 18:56:14 -0000

Hi there,

The authors have just integrated these changes. I think that they
address your concern, and so I asked them to post a new version -
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/

Please let me / them know if they correctly address your DISCUSS.
W

On Thu, Jun 8, 2017 at 7:36 AM, Alissa Cooper <alissa@cooperw.in> wrote:
> Hi Al,
>
>> On Jun 7, 2017, at 6:57 PM, MORTON, ALFRED C (AL) <acmorton@att.com> wro=
te:
>>
>> Hi Alissa,
>> Thanks for your review, please see reply below.
>> Al (for the co-authors)
>>
>>> -----Original Message-----
>>> From: Alissa Cooper [mailto:alissa@cooperw.in]
>>> Sent: Wednesday, June 07, 2017 5:53 PM
>>> To: The IESG
>>> Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org; Sarah Banks; bmwg-
>>> chairs@ietf.org; sbanks@encrypted.net; bmwg@ietf.org
>>> Subject: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03:
>>> (with DISCUSS and COMMENT)
>> ...
>>>
>>> ----------------------------------------------------------------------
>>> DISCUSS:
>>> ----------------------------------------------------------------------
>>>
>>> Picking up on the thread that started with the Gen-ART review, I'm not =
clear on
>>> the position of this document when it comes to repeatability. If all of=
 the
>>> parameters listed in Section 3.3 (assuming they apply) are configured a=
nd
>>> documented, is it assumed that the benchmarks will be repeatable and co=
mparable
>>> with hardware implementation benchmarks?
>> [ACM]
>>>>>
>> Of course, it has been the goal of the project to produce
>> repeatable results, and a large set of the parameters
>> believed to be critical was provided so that the benchmarking
>> community could better appreciate the increase in configuration
>> complexity inherent in this work. So, yes, this set was assumed
>> sufficient for the infrastructure in use by the VSPERF project
>> to obtain repeatable results from test-to-test.
>>>>> We could add summary above as a new paragraph in section 3.3.
>>
>> Benchmark Comparability between virtual and physical/hardware
>> implementations of equivalent functions will likely place more
>> detailed and exact requirements on the *testing systems*
>> (in terms of stream generation, algorithms to search
>> for max values, and their configurations of course).
>> This is another area for standardization to appreciate,
>> now that we have a few years testing experience.
>> However, the is a topic for a future draft.
>>
>>> Or is the position of the document
>>> that the benchmarks are not likely to be repeatable/comparable in some =
(many?)
>>> cases, given the increase in complexity? Or that more work needs to be =
done
>>> (outside the specification process) to achieve repeatability?
>> [ACM]
>> Thanks to some server re-arrangements in the lab that hosts
>> the VSPERF project, we are able to evaluate our long-term
>> result repeatability and repeatability across two similar
>> hardware platforms in different buildings. I'm still processing
>> some of the results we will present next week at the OPNFV Summit,
>> but the cross hardware platform results look quite consistent,
>> even with a benchmark where Scott Bradner experienced some
>> repeatability problems in his lab back in the 90=E2=80=99s.
>>
>>> I think this needs to be more clear in the document.
>> [ACM]
>> If the >>>paragraph<<< accomplishes that for you,
>> with some minor tweaks to word it less like a reply, let me know.
>
> I think both of your first two paragraphs above make useful points for in=
clusion in the draft, so if you=E2=80=99re able to write up some text that =
incorporates them, that would resolve my issue.
>
> Thanks,
> Alissa
>
>
>>
>>>
>>>
>>> ----------------------------------------------------------------------
>>> COMMENT:
>>> ----------------------------------------------------------------------
>>>
>>> I understand the debate about the publication of this document as an IE=
TF
>>> consensus RFC. To me it seems valuable to publish and that this item is=
 within
>>> the WG's charter. I think adding the updates necessary to make the docu=
ment
>>> up-to-date with the Danube 3.0 release as suggested in the thread with =
Alvaro
>>> would be valuable.
>>>
>> [ACM]
>> Great, I've located the list of tests added in Colorado and Danube,
>> so I'll add them in section 5, and update the release info accordingly.
>>
>>
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg



--=20
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Thu Jun  8 21:14:01 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 8D091127275; Thu,  8 Jun 2017 21:13:54 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149698163453.2972.392355559518148560@ietfa.amsl.com>
Date: Thu, 08 Jun 2017 21:13:54 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Y8ho2Xp68ghXquEiCckDDDRJu0E>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-10.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 04:13:55 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-10.txt
	Pages           : 17
	Date            : 2017-06-08

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-10
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-10

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-10


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun  8 21:52:44 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A9BDC129420; Thu,  8 Jun 2017 21:52:37 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149698395767.2940.17349496159711838896@ietfa.amsl.com>
Date: Thu, 08 Jun 2017 21:52:37 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0xo7-PT3E-OvMUCs6ESBSKyicDY>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-08.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 04:52:38 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-08.txt
	Pages           : 18
	Date            : 2017-06-08

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-08
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-08


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun  8 22:00:05 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 76930129420; Thu,  8 Jun 2017 22:00:03 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149698440343.3104.4546535082991398026@ietfa.amsl.com>
Date: Thu, 08 Jun 2017 22:00:03 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/UAVMuRNMevA93FwKt_LLDvUKlm8>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-09.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 05:00:03 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-09.txt
	Pages           : 18
	Date            : 2017-06-08

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-09
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-09

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-09


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun  8 22:12:23 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 26365127275 for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 22:12:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pSIaqDxjsCkM for <bmwg@ietfa.amsl.com>; Thu,  8 Jun 2017 22:12:03 -0700 (PDT)
Received: from mail-pf0-x230.google.com (mail-pf0-x230.google.com [IPv6:2607:f8b0:400e:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D092E120721 for <bmwg@ietf.org>; Thu,  8 Jun 2017 22:12:00 -0700 (PDT)
Received: by mail-pf0-x230.google.com with SMTP id x63so24452949pff.3 for <bmwg@ietf.org>; Thu, 08 Jun 2017 22:12:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9SdCSEHUmSrPX3mnixqDt85cO9vDyNyOXZiZs1dyYq4=; b=gllOhCYTDj5kG89pXYKEEGwehJ5efAC2bhkKN2J++7KzFayZsmOaqFvsMqR6znvCMC c6EScU333T6Sujf94Xmi09QZO9nN7b72liMr0r9AtqAlrnTH8M0AMT1NNPfpZI18MFD6 EBZHxQCQPghvAxxGBVBDlBbgx+mz3aSkQBir7/yZVjZKrk4pL4n5HZzbwu1ElImQdxKh 5F7SpntTC6CRhR6xze2zvNngi3YPS9dSCGcBULomE6nnrO4eZgw3B2on3DXYZnPrAQT8 tZzc89U30LGTEgHwTI0/KgbcXD0zX5N8vEUOH85VMbi3D/DwlOCZXMD7DQk7DbvQg8gB qEag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9SdCSEHUmSrPX3mnixqDt85cO9vDyNyOXZiZs1dyYq4=; b=GU3pOxsohOaG2okM+hnIOACPM+Z+LlDpW7i96gl1MBs3fUJtFESuoouP8ZZzMAQNdn c9poIYXT6esYKVPe0dOS0mZnvHQlwS7lqERBpY5YbZmtbn0DmcGu1/hy1G3wlgsttXdv 5BK0j+Yeo8hTuaNHr945mJgwvHipxUcRTXauGLj85k2kLPxKRZnbYD2lme561isl0JNh MsBCy/zErryik63IV+lFkzYi0JxaqvXh4yS83Z1k38fmyomXKHXUUQMmMSBB+NS1M81i vza/GJNTi0NIKAyxPbyuiM1lIOkdBCOsQPkFQ4Ej8kmj/DpOPaoDxuzgXLUKykZmVl5c Nl4g==
X-Gm-Message-State: AODbwcDJ9T5eifc4BNuoQiTapO0D/aSxVX6nZJeo97NyEujMjZCyuHEo WupmxbQQTfo6UXF9Ujif3RvoZD26guBA
X-Received: by 10.98.19.16 with SMTP id b16mr16690518pfj.159.1496985120267; Thu, 08 Jun 2017 22:12:00 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.138.38 with HTTP; Thu, 8 Jun 2017 22:11:39 -0700 (PDT)
In-Reply-To: <149692498024.25760.14056448148418579949@ietfa.amsl.com>
References: <149692498024.25760.14056448148418579949@ietfa.amsl.com>
From: Lucien Avramov <lucienav@google.com>
Date: Thu, 8 Jun 2017 22:11:39 -0700
Message-ID: <CALTEt=DUhEVgSvYN+2XYUR8Ddq5n8HJtZW2RNj3DV3rSgH0YKQ@mail.gmail.com>
To: Dan Romascanu <dromasca@gmail.com>
Cc: gen-art@ietf.org, draft-ietf-bmwg-dcbench-methodology.all@ietf.org,  ietf@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a114760f854781a05518002c1"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/tv69_QplIaNjan9f5g6apCrcYjo>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 05:12:05 -0000

--001a114760f854781a05518002c1
Content-Type: text/plain; charset="UTF-8"

Hi Dan,

Thank you very much for the comments. I addressed them and published
version -09.

Inline are some the comments / answer to your feedback. The parts without
answers mean they have been addressed.

Again, thank you for spending the time to read this in this fine detail.

Lucien



On Thu, Jun 8, 2017 at 5:29 AM, Dan Romascanu <dromasca@gmail.com> wrote:

> Reviewer: Dan Romascanu
> Review result: Ready with Issues
>
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-bmwg-dcbench-methodology-??
> Reviewer: Dan Romascanu
> Review Date: 2017-06-08
> IETF LC End Date: 2017-06-13
> IESG Telechat date: Not scheduled for a telechat
>
> Summary:
>
> This Informational I-D describes test and evaluation methodology and
> measurement techniques for physical network equipment in the data center.
> It's
> an useful and clearly written document and it is ready for publication,
> after a
> number of minor issues and nits will be addressed.
>
> Major issues:
>
> Minor issues:
>
> 1. There is one single mention of the scope being benchmarking of 'physical
> network equipment in the data center'. I assume this is written as opposed
> to
> virtual networking equipment. This is not included in the title or
> repeated any
> place else. I suggest to make more clear the scope by mentioning 'physical
> equipment' at least once more, for example in the Introduction section.
>
> 2. Section 1.2: I am a little concerned about adding additional
> interpretations
> to the keywords in RFC2119.
>
> Especially about:
>
> 'MAY: Comprehensive metric for the scenario described'
>

*Thank you, MAY provides an optional metric, I have addressed your comment *

>
> MAY has a different meaning in 2119, and the fact that a metric is
> 'comprehensive' (whatever this means) seems to me orthogonal to the 2119
> semantics. The proposed interpretation makes little sense to me, if
> 'comprehensive' than why not required? or at least as a 'should'?
>
> 3. Section 6.2: 'The intensity of a microburst MAY be varied ...' It would
> be
> useful to make clear what is meant by 'intensity'.
>

The intensity of a microburst is defined in the definitions document [1] so
i referenced it that way


>
> Nits/editorial comments:
>
> 1. The different formatting of the references [1], [2], etc. vs. [RFC2119],
> etc. is slightly confusing. Unless there is some good reason I suggest to
> fix
> this. Also use of references is inconsistent through the text, sometimes
> they
> are mentioned, sometimes they are not.
>

Fixed it for [2] and [3], however kept [1] as [1], as this document is not
an RFC yet.


>
> 2. Section 2.2: 'Alternatively when a traffic generator CAN NOT be
> connected '
> - this capitalization is not conformant to RFC2119.
>
> 3. 'vlan ZZZ' is not consistent with 'vlan X' and 'vlan Y' previously used.
>
> 4. Missing reference - RFC 6985
>
> 5. Missing expansion and references for DSCP and COS
>
> 6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/
>
> 7. The methodology (e.g. in section 5) makes an assumption that the DUT
> has at
> least nine ports. This may be trivial in a DC, but it's worth being
> mentioned
>
> Thanks added this in section 2.3 as it was making more sense to be
clarified there

--001a114760f854781a05518002c1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Dan,=C2=A0<div><br></div><div>Thank you very much for t=
he comments. I addressed them and published version -09.=C2=A0</div><div><b=
r></div><div>Inline are some the comments / answer to your feedback. The pa=
rts without answers mean they have been addressed.=C2=A0</div><div><br></di=
v><div>Again, thank you for spending the time to read this in this fine det=
ail.=C2=A0</div><div><br></div><div>Lucien</div><div><br></div><div><br></d=
iv><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Thu, Jun 8,=
 2017 at 5:29 AM, Dan Romascanu <span dir=3D"ltr">&lt;<a href=3D"mailto:dro=
masca@gmail.com" target=3D"_blank">dromasca@gmail.com</a>&gt;</span> wrote:=
<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left:1px solid rgb(204,204,204);padding-left:1ex">Reviewer: Dan Romasca=
nu<br>
Review result: Ready with Issues<br>
<br>
I am the assigned Gen-ART reviewer for this draft. The General Area<br>
Review Team (Gen-ART) reviews all IETF documents being processed<br>
by the IESG for the IETF Chair.=C2=A0 Please treat these comments just<br>
like any other last call comments.<br>
<br>
For more information, please see the FAQ at<br>
<br>
&lt;<a href=3D"https://trac.ietf.org/trac/gen/wiki/GenArtfaq" rel=3D"norefe=
rrer" target=3D"_blank">https://trac.ietf.org/trac/ge<wbr>n/wiki/GenArtfaq<=
/a>&gt;.<br>
<br>
Document: draft-ietf-bmwg-dcbench-method<wbr>ology-??<br>
Reviewer: Dan Romascanu<br>
Review Date: 2017-06-08<br>
IETF LC End Date: 2017-06-13<br>
IESG Telechat date: Not scheduled for a telechat<br>
<br>
Summary:<br>
<br>
This Informational I-D describes test and evaluation methodology and<br>
measurement techniques for physical network equipment in the data center. I=
t&#39;s<br>
an useful and clearly written document and it is ready for publication, aft=
er a<br>
number of minor issues and nits will be addressed.<br>
<br>
Major issues:<br>
<br>
Minor issues:<br>
<br>
1. There is one single mention of the scope being benchmarking of &#39;phys=
ical<br>
network equipment in the data center&#39;. I assume this is written as oppo=
sed to<br>
virtual networking equipment. This is not included in the title or repeated=
 any<br>
place else. I suggest to make more clear the scope by mentioning &#39;physi=
cal<br>
equipment&#39; at least once more, for example in the Introduction section.=
<br>
<br>
2. Section 1.2: I am a little concerned about adding additional interpretat=
ions<br>
to the keywords in RFC2119.<br>
<br>
Especially about:<br>
<br>
&#39;MAY: Comprehensive metric for the scenario described&#39;<br></blockqu=
ote><div><br></div><div><b><span style=3D"font-size:12.8px">Thank you, MAY =
provides an optional metric, I have addressed your comment</span>=C2=A0</b>=
</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b=
order-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
MAY has a different meaning in 2119, and the fact that a metric is<br>
&#39;comprehensive&#39; (whatever this means) seems to me orthogonal to the=
 2119<br>
semantics. The proposed interpretation makes little sense to me, if<br>
&#39;comprehensive&#39; than why not required? or at least as a &#39;should=
&#39;?<br>
<br>
3. Section 6.2: &#39;The intensity of a microburst MAY be varied ...&#39; I=
t would be<br>
useful to make clear what is meant by &#39;intensity&#39;.<br></blockquote>=
<div><br></div><div><div style=3D"font-size:12.8px">The intensity of a micr=
oburst is defined in the definitions document [1] so i referenced it that w=
ay</div></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left=
:1ex">
<br>
Nits/editorial comments:<br>
<br>
1. The different formatting of the references [1], [2], etc. vs. [RFC2119],=
<br>
etc. is slightly confusing. Unless there is some good reason I suggest to f=
ix<br>
this. Also use of references is inconsistent through the text, sometimes th=
ey<br>
are mentioned, sometimes they are not.<br></blockquote><div><br></div><div>=
Fixed it for [2] and [3], however kept [1] as [1], as this document is not =
an RFC yet.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex">
<br>
2. Section 2.2: &#39;Alternatively when a traffic generator CAN NOT be conn=
ected &#39;<br>
- this capitalization is not conformant to RFC2119.<br>
<br>
3. &#39;vlan ZZZ&#39; is not consistent with &#39;vlan X&#39; and &#39;vlan=
 Y&#39; previously used.<br>
<br>
4. Missing reference - RFC 6985<br>
<br>
5. Missing expansion and references for DSCP and COS<br>
<br>
6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/<br>
<br>
7. The methodology (e.g. in section 5) makes an assumption that the DUT has=
 at<br>
least nine ports. This may be trivial in a DC, but it&#39;s worth being men=
tioned<br>
<br></blockquote><div><span style=3D"font-size:12.8px">Thanks added this in=
 section 2.3 as it was making more sense to be clarified there</span>=C2=A0=
</div></div><br></div></div>

--001a114760f854781a05518002c1--


From nobody Fri Jun  9 00:59:31 2017
Return-Path: <bclaise@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AA1D1129566; Fri,  9 Jun 2017 00:59:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.502
X-Spam-Level: 
X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5Fg50LlLqTUS; Fri,  9 Jun 2017 00:59:27 -0700 (PDT)
Received: from aer-iport-4.cisco.com (aer-iport-4.cisco.com [173.38.203.54]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 829AB129549; Fri,  9 Jun 2017 00:59:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4724; q=dns/txt; s=iport; t=1496995159; x=1498204759; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=Ec0OFppEDDOx8ozbwB21NkciljvgLUcYV3mRtXL32qQ=; b=lgtMJgZE5FtMzlROmG9xoG+nwaES1ee162vMa5v0h6FBvzyTJP6UMyMW 5MDrI2vuK40C35g1IXAluj9a4QI0Rpyd04aBSLcxARmZ/ecVz9uOLDcOK f88ZA85j7kbibk4xsRMbfcDh9nvZsmsHG2oc3ehiHxoGJgMQG3Tbh75JJ k=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0AMAQATUzpZ/xbLJq1cGQEBAQEBAQEBA?= =?us-ascii?q?QEBBwEBAQEBhDuBDYN0ihhzkFghlgOCESELhXgCgz0YAQIBAQEBAQEBayiFGAE?= =?us-ascii?q?BAQEDAQEhDwEFNgsMBAsRBAEBAQICIwMCAicfCQgGAQwGAgEBEIoXELBUgiaLa?= =?us-ascii?q?wEBAQEBAQEBAQEBAQEBAQEBAQEBARgFgQuFVoFgKwuBXoEMhFSDKIJhAQSJTY0?= =?us-ascii?q?7hzSHKIwZggaFQ4NLhnKJEYMciD0fOIEKMCEIGxVIhUGBTz42iT8BAQE?=
X-IronPort-AV: E=Sophos;i="5.39,317,1493683200"; d="scan'208";a="655294396"
Received: from aer-iport-nat.cisco.com (HELO aer-core-3.cisco.com) ([173.38.203.22]) by aer-iport-4.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2017 07:59:14 +0000
Received: from [10.55.221.37] (ams-bclaise-nitro4.cisco.com [10.55.221.37]) by aer-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id v597xD89008769; Fri, 9 Jun 2017 07:59:13 GMT
To: Warren Kumari <warren@kumari.net>, Alissa Cooper <alissa@cooperw.in>
Cc: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>, IESG <iesg@ietf.org>
References: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com> <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in> <CAHw9_iK=KdgR4djxu1C4SQtmRVNxXPf6r2N-tWZJyi__=cN3Kw@mail.gmail.com>
From: Benoit Claise <bclaise@cisco.com>
Message-ID: <a1df22d7-dd4a-0386-ecb6-40bdcf01b7a4@cisco.com>
Date: Fri, 9 Jun 2017 09:59:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1
MIME-Version: 1.0
In-Reply-To: <CAHw9_iK=KdgR4djxu1C4SQtmRVNxXPf6r2N-tWZJyi__=cN3Kw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0c_8R3rOuY4y15plKlj-sdwXI0c>
Subject: Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 07:59:30 -0000

Dear authors,

The new version reads way better.

Thank you.

Regards, B.
> Hi there,
>
> The authors have just integrated these changes. I think that they
> address your concern, and so I asked them to post a new version -
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/
>
> Please let me / them know if they correctly address your DISCUSS.
> W
>
> On Thu, Jun 8, 2017 at 7:36 AM, Alissa Cooper <alissa@cooperw.in> wrote:
>> Hi Al,
>>
>>> On Jun 7, 2017, at 6:57 PM, MORTON, ALFRED C (AL) <acmorton@att.com> wrote:
>>>
>>> Hi Alissa,
>>> Thanks for your review, please see reply below.
>>> Al (for the co-authors)
>>>
>>>> -----Original Message-----
>>>> From: Alissa Cooper [mailto:alissa@cooperw.in]
>>>> Sent: Wednesday, June 07, 2017 5:53 PM
>>>> To: The IESG
>>>> Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org; Sarah Banks; bmwg-
>>>> chairs@ietf.org; sbanks@encrypted.net; bmwg@ietf.org
>>>> Subject: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03:
>>>> (with DISCUSS and COMMENT)
>>> ...
>>>> ----------------------------------------------------------------------
>>>> DISCUSS:
>>>> ----------------------------------------------------------------------
>>>>
>>>> Picking up on the thread that started with the Gen-ART review, I'm not clear on
>>>> the position of this document when it comes to repeatability. If all of the
>>>> parameters listed in Section 3.3 (assuming they apply) are configured and
>>>> documented, is it assumed that the benchmarks will be repeatable and comparable
>>>> with hardware implementation benchmarks?
>>> [ACM]
>>> Of course, it has been the goal of the project to produce
>>> repeatable results, and a large set of the parameters
>>> believed to be critical was provided so that the benchmarking
>>> community could better appreciate the increase in configuration
>>> complexity inherent in this work. So, yes, this set was assumed
>>> sufficient for the infrastructure in use by the VSPERF project
>>> to obtain repeatable results from test-to-test.
>>>>>> We could add summary above as a new paragraph in section 3.3.
>>> Benchmark Comparability between virtual and physical/hardware
>>> implementations of equivalent functions will likely place more
>>> detailed and exact requirements on the *testing systems*
>>> (in terms of stream generation, algorithms to search
>>> for max values, and their configurations of course).
>>> This is another area for standardization to appreciate,
>>> now that we have a few years testing experience.
>>> However, the is a topic for a future draft.
>>>
>>>> Or is the position of the document
>>>> that the benchmarks are not likely to be repeatable/comparable in some (many?)
>>>> cases, given the increase in complexity? Or that more work needs to be done
>>>> (outside the specification process) to achieve repeatability?
>>> [ACM]
>>> Thanks to some server re-arrangements in the lab that hosts
>>> the VSPERF project, we are able to evaluate our long-term
>>> result repeatability and repeatability across two similar
>>> hardware platforms in different buildings. I'm still processing
>>> some of the results we will present next week at the OPNFV Summit,
>>> but the cross hardware platform results look quite consistent,
>>> even with a benchmark where Scott Bradner experienced some
>>> repeatability problems in his lab back in the 90’s.
>>>
>>>> I think this needs to be more clear in the document.
>>> [ACM]
>>> If the >>>paragraph<<< accomplishes that for you,
>>> with some minor tweaks to word it less like a reply, let me know.
>> I think both of your first two paragraphs above make useful points for inclusion in the draft, so if you’re able to write up some text that incorporates them, that would resolve my issue.
>>
>> Thanks,
>> Alissa
>>
>>
>>>>
>>>> ----------------------------------------------------------------------
>>>> COMMENT:
>>>> ----------------------------------------------------------------------
>>>>
>>>> I understand the debate about the publication of this document as an IETF
>>>> consensus RFC. To me it seems valuable to publish and that this item is within
>>>> the WG's charter. I think adding the updates necessary to make the document
>>>> up-to-date with the Danube 3.0 release as suggested in the thread with Alvaro
>>>> would be valuable.
>>>>
>>> [ACM]
>>> Great, I've located the list of tests added in Colorado and Danube,
>>> so I'll add them in section 5, and update the release info accordingly.
>>>
>>>
>> _______________________________________________
>> bmwg mailing list
>> bmwg@ietf.org
>> https://www.ietf.org/mailman/listinfo/bmwg
>
>


From nobody Fri Jun  9 03:24:24 2017
Return-Path: <panantha.ietf@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 545491294B2 for <bmwg@ietfa.amsl.com>; Fri,  9 Jun 2017 03:24:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level: 
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 98JlO-VQku-E for <bmwg@ietfa.amsl.com>; Fri,  9 Jun 2017 03:24:20 -0700 (PDT)
Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DA7611201F8 for <bmwg@ietf.org>; Fri,  9 Jun 2017 03:24:19 -0700 (PDT)
Received: by mail-lf0-x22d.google.com with SMTP id v20so28049052lfa.1 for <bmwg@ietf.org>; Fri, 09 Jun 2017 03:24:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:references:in-reply-to:from:date:message-id:subject:to;  bh=AJ7e2fk8i26+ujZJqMInnJZWi3oFFJQYZsufS35vTaE=; b=e2nrAP7+8YsCOL7WWI3T4XqPEuTMD/YVqtN/17hqGsGrbMbALQGoPdWjUw4LSCoaFv fDsirxc417NOM8jKsbkQpGm0ODRjwjfPQ/w5Jtt/oh5bRTAwxQS1ffauVeFwfjdjmZRu VSpi7IVdLl4/p8xfX5WnxH0Hv16VsB5RanTtZ0ZgIBB9fHMcUUHmpTcVx4gVlNV2X4f5 XwuomEzuUXykXEG/s1SdT82dv3SwXUTftONtKNXg93qUXPUkJ/wXC1mDZmeKAKZfAaDl 9k7lTW1CZaeA06aIEcMZOatA+vlIcgZ7dVjhAmCz43NwqACLIMo8aPT968+QP+sC9xIN yvbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=AJ7e2fk8i26+ujZJqMInnJZWi3oFFJQYZsufS35vTaE=; b=KWfPnVYdK1v1nyMEdkeQ4GueN/J7fkjTn3/i4o57yyNwacEPXcb+oSItMVkeabiRIa zzz2SX2Y8DOTdQq+1EzF9YnjCZOAhcM0hIuwr+UwVyfMw6iVqT8EInjnK+argdjhPQIG w9Y/OFCLxoH93H4tKauvf3RBZIV1u4TOJpNTyBwOU+r75SC8Ibk+AWrUrQl2Hsy3McSo PDXgOzYp75lPTXm2Ak5EjnXT4gmdPahU9befb+QEAGenVZPEqYnk0CKDBaPIpzeAmRXx hNn8N3WD4NpwLw1eDFxC0lf3J52I/84pUChmgqyFj4xJZ1zZaF0E4jcGAbGbMbAZdQgW NhQQ==
X-Gm-Message-State: AODbwcCsJY3W84H+p3uIzC7uOxDmn0ZhxaWrTVuDLnnYx70N9YFq060F 0xHqfMVImsGn2Mbzn1d8SDGKJmGlOQ==
X-Received: by 10.46.80.73 with SMTP id v9mr12662421ljd.135.1497003858215; Fri, 09 Jun 2017 03:24:18 -0700 (PDT)
MIME-Version: 1.0
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
From: Praveen Ananthasankaran <panantha.ietf@gmail.com>
Date: Fri, 09 Jun 2017 10:24:07 +0000
Message-ID: <CAArMwM9Wt9iEe42E_PaOmOOn70uYd9RQ=VoV05+KxwKNCke7hg@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="f403045fc0ba32498f0551845fd8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0YCewQbPBYVnacMFrfRK9rhrCaA>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 10:24:22 -0000

--f403045fc0ba32498f0551845fd8
Content-Type: text/plain; charset="UTF-8"

Hi,

I support the adoption of draft-kishjac-bmwg-evpntest-07 as the basis
document for EVPN & PBB-EVPN Benchmarking Methodology.

Regards,
Praveen


On Sun, Jun 4, 2017 at 4:04 AM MORTON, ALFRED C (AL) <acmorton@att.com>
wrote:

> BMWG,
>
> At our Chicago meeting, the BMWG reviewed a version of
> the Internet Draft on
>
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
>
> There was sufficient readership present at the meeting
> and positive/constructive comments on the work, so the
> chairs asked whether support existed to adopt the draft
> as a WG item. There was support and no objections. As
> always, such decisions are officially taken on the
> mailing list.
>
> This message*, therefore, starts a call for adoption on:
>
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
>
> to run until Friday 23 June 2017.
>
> Please reply to bmwg@ietf.org indicating:
>
> (1) whether you support addition of the following milestone to
>    the BMWG charter:
>
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
>
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>    as the basis document for this milestone
>
> (3) whether you commit to reviewing the WG document if adopted
>
> We are aware that BMWG has nearly completed all its existing
> chartered items.  This usually begins discussion of
> re-chartering, this time with Warren (our new AD Advisor). The final
> decision/schedule for adoption will depend on these discussions.
>
> thanks for your consideration of the draft, and regards,
>
> Al
> bmwg co-chair
>
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>

--f403045fc0ba32498f0551845fd8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>Hi,<br><br>I support the adoption of draft-kishj=
ac-bmwg-evpntest-07 as the basis document for EVPN &amp; PBB-EVPN Benchmark=
ing Methodology.<br><br></div>Regards,<br></div>Praveen<br><div><div><br><d=
iv><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Sun, Jun 4, 2017 at 4=
:04 AM MORTON, ALFRED C (AL) &lt;<a href=3D"mailto:acmorton@att.com">acmort=
on@att.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">BMWG,<br=
>
<br>
At our Chicago meeting, the BMWG reviewed a version of<br>
the Internet Draft on<br>
<br>
Benchmarking Methodology for EVPN and PBB-EVPN<br>
draft-kishjac-bmwg-evpntest-05<br>
<br>
There was sufficient readership present at the meeting<br>
and positive/constructive comments on the work, so the<br>
chairs asked whether support existed to adopt the draft<br>
as a WG item. There was support and no objections. As<br>
always, such decisions are officially taken on the<br>
mailing list.<br>
<br>
This message*, therefore, starts a call for adoption on:<br>
<br>
Benchmarking Methodology for EVPN and PBB-EVPN<br>
draft-kishjac-bmwg-evpntest-07<br>
<br>
to run until Friday 23 June 2017.<br>
<br>
Please reply to <a href=3D"mailto:bmwg@ietf.org" target=3D"_blank">bmwg@iet=
f.org</a> indicating:<br>
<br>
(1) whether you support addition of the following milestone to<br>
=C2=A0 =C2=A0the BMWG charter:<br>
<br>
date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG<br>
<br>
(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07<br>
=C2=A0 =C2=A0as the basis document for this milestone<br>
<br>
(3) whether you commit to reviewing the WG document if adopted<br>
<br>
We are aware that BMWG has nearly completed all its existing<br>
chartered items.=C2=A0 This usually begins discussion of<br>
re-chartering, this time with Warren (our new AD Advisor). The final<br>
decision/schedule for adoption will depend on these discussions.<br>
<br>
thanks for your consideration of the draft, and regards,<br>
<br>
Al<br>
bmwg co-chair<br>
<br>
*using Brian Trammell&#39;s recent message to IPPM as a template (thanks!)<=
br>
<br>
_______________________________________________<br>
bmwg mailing list<br>
<a href=3D"mailto:bmwg@ietf.org" target=3D"_blank">bmwg@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/bmwg" rel=3D"noreferrer" t=
arget=3D"_blank">https://www.ietf.org/mailman/listinfo/bmwg</a><br>
</blockquote></div></div></div></div></div>

--f403045fc0ba32498f0551845fd8--


From nobody Fri Jun  9 05:42:17 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 90A66126BF3; Fri,  9 Jun 2017 05:42:10 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alissa Cooper <alissa@cooperw.in>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149701213054.30067.9344581960553002293.idtracker@ietfa.amsl.com>
Date: Fri, 09 Jun 2017 05:42:10 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/wzBP9WKfyvbfXP3wSzPiiPJQQhM>
Subject: [bmwg] Alissa Cooper's No Objection on draft-ietf-bmwg-vswitch-opnfv-04: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 12:42:11 -0000

Alissa Cooper has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-04: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks for addressing my DISCUSS and comments. One nit:

s/for a future draft/for a future document/



From nobody Fri Jun  9 07:05:24 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ACE99129483 for <bmwg@ietfa.amsl.com>; Fri,  9 Jun 2017 07:05:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kEOylSOHujhC for <bmwg@ietfa.amsl.com>; Fri,  9 Jun 2017 07:05:20 -0700 (PDT)
Received: from mail-ua0-x236.google.com (mail-ua0-x236.google.com [IPv6:2607:f8b0:400c:c08::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A64D9129432 for <bmwg@ietf.org>; Fri,  9 Jun 2017 07:05:20 -0700 (PDT)
Received: by mail-ua0-x236.google.com with SMTP id 68so16265237uas.0 for <bmwg@ietf.org>; Fri, 09 Jun 2017 07:05:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=wA7cegSh3QWmIjAgSmbUirw9WsSTh+ABqrwLMx3i2nU=; b=Pyt8CPX1ylt/Sd8cUhkDIJCph+0ZuovNWM0/xoYrGruP+SSoPMnqk0eOKTTLSP7JCJ cxmgAYBtz4T4kkA/mZXniWEDFbZ+onl/tBrk1eKPBuMooV2FuL3Es70LLJau7TSP7cQA lMvPdQpY4UxZRPcaWdfElgHrnPuR+WmvVM2dALBNgYH7VAvXA0Ax8s5jQdf8keeRB6ai J6ffmLHXxvq2r11gutxscm4olf26QXZ7nHywvFSDHFyjNsFLtS5p0RxOlbCJl/OXUC+X BQjQkENUMAotzvV/cBhfPc2rcWlYVVMCLlHSPpJ92bIbaZwgQa7RI9yrY9LemKX/5NZY upJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=wA7cegSh3QWmIjAgSmbUirw9WsSTh+ABqrwLMx3i2nU=; b=IkdUPmQ30caL+vKOrd4BFqDuEUJ6kcDzoa4LQ7K2lnAyJVuODW6FAAYxmjnjxNaU/t OEj9uBtmvWdnYEdU03VCBneJ3r2g4LuWcYQARwJI45wCHQ/5Cfqwsye4u3/GnsMVnK5g 4MbVKMZxtcDM/s5FxAJe7fsszqfTzMsGqGtnf/Zum0FqiSUUW+vzM3AFS6WqNN4DY0c4 MNBt1kUGf9/K/PzcS96ziDNNmC2vVXxrZSojd7ENrn+nOONcQyvxLufhLQ5B5Kba2dxr QjAuRFkoqIoVvAKIvK/tfKT3IOia7Ho5PGytjWjRF7ag9auCil8ExOoLrbZ7cRno6gYQ VK8Q==
X-Gm-Message-State: AODbwcAniD62lBAnLV2zanN5flfiodikWyM2NsL6XQtgdlbndEA6M22q Ymb4dKrrly9eD1FJrWmfaNOKWFiBQ732
X-Received: by 10.176.77.230 with SMTP id b38mr25356301uah.76.1497017119621; Fri, 09 Jun 2017 07:05:19 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.82.198 with HTTP; Fri, 9 Jun 2017 07:04:39 -0700 (PDT)
In-Reply-To: <149701213054.30067.9344581960553002293.idtracker@ietfa.amsl.com>
References: <149701213054.30067.9344581960553002293.idtracker@ietfa.amsl.com>
From: Warren Kumari <warren@kumari.net>
Date: Fri, 9 Jun 2017 10:04:39 -0400
Message-ID: <CAHw9_iLOHfNwwcC2ET27vxJTqrz8XCgTheLRKr7N5kjf6z1n3g@mail.gmail.com>
To: Alissa Cooper <alissa@cooperw.in>
Cc: The IESG <iesg@ietf.org>, sbanks@encrypted.net, bmwg-chairs@ietf.org,  draft-ietf-bmwg-vswitch-opnfv@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/vdRq--5VBZS-8yp04sIQvMCsujA>
Subject: Re: [bmwg] Alissa Cooper's No Objection on draft-ietf-bmwg-vswitch-opnfv-04: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 14:05:23 -0000

On Fri, Jun 9, 2017 at 8:42 AM, Alissa Cooper <alissa@cooperw.in> wrote:
> Alissa Cooper has entered the following ballot position for
> draft-ietf-bmwg-vswitch-opnfv-04: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> Thanks for addressing my DISCUSS and comments. One nit:
>
> s/for a future draft/for a future document/
>

Good catch! Thank you for helping improve the document - I think that
the changes make it much better.

Dear authors, please note this to flag for the RFC Editor ( will as
well, but just in case).

W
-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Fri Jun  9 07:10:25 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 406D31294A1; Fri,  9 Jun 2017 07:10:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dQdcbgzps52f; Fri,  9 Jun 2017 07:10:11 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20A6912943B; Fri,  9 Jun 2017 07:10:11 -0700 (PDT)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v59E6DG8039216; Fri, 9 Jun 2017 10:10:06 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0083689.ppops.net-00191d01. with ESMTP id 2aytjkm0vm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 09 Jun 2017 10:10:06 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v59EA4Mh081512; Fri, 9 Jun 2017 09:10:05 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v59EA022081475 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 9 Jun 2017 09:10:01 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor); Fri, 9 Jun 2017 14:09:44 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v59E9hiN001916; Fri, 9 Jun 2017 09:09:44 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v59E9Zfl001481; Fri, 9 Jun 2017 09:09:35 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id EF1D4E0521; Fri,  9 Jun 2017 10:09:34 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Fri, 9 Jun 2017 10:09:34 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Warren Kumari <warren@kumari.net>, Alissa Cooper <alissa@cooperw.in>
CC: The IESG <iesg@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alissa Cooper's No Objection on draft-ietf-bmwg-vswitch-opnfv-04: (with COMMENT)
Thread-Index: AQHS4R3SVrz8MKFmnUC75cmrGvf8q6Ic032A//+9VBA=
Date: Fri, 9 Jun 2017 14:09:33 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD7B3B@njmtexg5.research.att.com>
References: <149701213054.30067.9344581960553002293.idtracker@ietfa.amsl.com> <CAHw9_iLOHfNwwcC2ET27vxJTqrz8XCgTheLRKr7N5kjf6z1n3g@mail.gmail.com>
In-Reply-To: <CAHw9_iLOHfNwwcC2ET27vxJTqrz8XCgTheLRKr7N5kjf6z1n3g@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.211.228]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-09_07:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706090247
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/aeDK112skfDepl5nB9aTe3NxMhc>
Subject: Re: [bmwg] Alissa Cooper's No Objection on draft-ietf-bmwg-vswitch-opnfv-04: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 14:10:13 -0000

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBXYXJyZW4gS3VtYXJpIFttYWls
dG86d2FycmVuQGt1bWFyaS5uZXRdDQo+IFNlbnQ6IEZyaWRheSwgSnVuZSAwOSwgMjAxNyAxMDow
NSBBTQ0KPiBUbzogQWxpc3NhIENvb3Blcg0KPiBDYzogVGhlIElFU0c7IHNiYW5rc0BlbmNyeXB0
ZWQubmV0OyBibXdnLWNoYWlyc0BpZXRmLm9yZzsgZHJhZnQtaWV0Zi0NCj4gYm13Zy12c3dpdGNo
LW9wbmZ2QGlldGYub3JnOyBibXdnQGlldGYub3JnDQo+IFN1YmplY3Q6IFJlOiBBbGlzc2EgQ29v
cGVyJ3MgTm8gT2JqZWN0aW9uIG9uIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLQ0KPiBvcG5mdi0w
NDogKHdpdGggQ09NTUVOVCkNCj4gDQo+IE9uIEZyaSwgSnVuIDksIDIwMTcgYXQgODo0MiBBTSwg
QWxpc3NhIENvb3BlciA8YWxpc3NhQGNvb3BlcncuaW4+IHdyb3RlOg0KPiA+IEFsaXNzYSBDb29w
ZXIgaGFzIGVudGVyZWQgdGhlIGZvbGxvd2luZyBiYWxsb3QgcG9zaXRpb24gZm9yDQo+ID4gZHJh
ZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDQ6IE5vIE9iamVjdGlvbg0KPiA+DQouLi4NCj4g
Pg0KPiA+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gPiBDT01NRU5UOg0KPiA+IC0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4g
Pg0KPiA+IFRoYW5rcyBmb3IgYWRkcmVzc2luZyBteSBESVNDVVNTIGFuZCBjb21tZW50cy4gT25l
IG5pdDoNCj4gPg0KPiA+IHMvZm9yIGEgZnV0dXJlIGRyYWZ0L2ZvciBhIGZ1dHVyZSBkb2N1bWVu
dC8NCj4gPg0KPiANCj4gR29vZCBjYXRjaCEgVGhhbmsgeW91IGZvciBoZWxwaW5nIGltcHJvdmUg
dGhlIGRvY3VtZW50IC0gSSB0aGluayB0aGF0DQo+IHRoZSBjaGFuZ2VzIG1ha2UgaXQgbXVjaCBi
ZXR0ZXIuDQpbQUNNXSANCisxLCBhbmQgZm9yIHNlZWluZyB0aGUgdmFsdWUgaW4gdGhpcyBkcmFm
dCBhcyBhIGNvbnRyaWJ1dGlvbi9SRkMuDQpCZW5vaXQncyBjb21tZW50cyBoZWxwZWQgY2FwdHVy
ZSB0aGF0IHZhbHVlIG1vcmUgYWNjdXJhdGVseSwNCnRvby4NCg0KPiANCj4gRGVhciBhdXRob3Jz
LCBwbGVhc2Ugbm90ZSB0aGlzIHRvIGZsYWcgZm9yIHRoZSBSRkMgRWRpdG9yICggd2lsbCBhcw0K
PiB3ZWxsLCBidXQganVzdCBpbiBjYXNlKS4NCltBQ01dIA0KV2lsbCBkbywgYXQgQVVUSDQ4IG9y
IHNvb25lci4NCg0KQWwNCg0KPiANCj4gVw0KPiAtLQ0KPiBJIGRvbid0IHRoaW5rIHRoZSBleGVj
dXRpb24gaXMgcmVsZXZhbnQgd2hlbiBpdCB3YXMgb2J2aW91c2x5IGEgYmFkDQo+IGlkZWEgaW4g
dGhlIGZpcnN0IHBsYWNlLg0KPiBUaGlzIGlzIGxpa2UgcHV0dGluZyByYWJpZCB3ZWFzZWxzIGlu
IHlvdXIgcGFudHMsIGFuZCBsYXRlciBleHByZXNzaW5nDQo+IHJlZ3JldCBhdCBoYXZpbmcgY2hv
c2VuIHRob3NlIHBhcnRpY3VsYXIgcmFiaWQgd2Vhc2VscyBhbmQgdGhhdCBwYWlyDQo+IG9mIHBh
bnRzLg0KPiAgICAtLS1tYWYNCg==


From nobody Fri Jun  9 09:03:13 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 839D3129B74; Fri,  9 Jun 2017 09:02:59 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Cc: Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, The IESG <iesg@ietf.org>, draft-ietf-bmwg-vswitch-opnfv@ietf.org, bmwg@ietf.org, sbanks@encrypted.net, warren@kumari.net, rfc-editor@rfc-editor.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149702417949.29940.5876180255691330256.idtracker@ietfa.amsl.com>
Date: Fri, 09 Jun 2017 09:02:59 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/nxkIqtHN-ha9EaTpoODhlXeZkHQ>
Subject: [bmwg] Document Action: 'Benchmarking Virtual Switches in OPNFV' to Informational RFC (draft-ietf-bmwg-vswitch-opnfv-04.txt)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Jun 2017 16:03:00 -0000

The IESG has approved the following document:
- 'Benchmarking Virtual Switches in OPNFV'
  (draft-ietf-bmwg-vswitch-opnfv-04.txt) as Informational RFC

This document is the product of the Benchmarking Methodology Working Group.

The IESG contact persons are Warren Kumari and Benoit Claise.

A URL of this Internet Draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/





Technical Summary

   This draft summarizes the progress of the OPNFV project on virtual switch performance characterization, and notes that the OPNFV work builds upon the IETF work, particularly the BMWG work within IETF, with an emphasis on RFC2544.


Working Group Summary

   There has been a fair amount of work done on this draft, and progress made on revisions, feedback, and comments. Several presentations have been made in the room during IETF meetings, and followup and discussion taken to the BMWG list. This draft is particularly useful, given the popularity of VNF's within the industry.


Document Quality

  There has been a fair amount of work done on this draft, and progress made on revisions, feedback, and comments. Several presentations have been made in the room during IETF meetings, and followup and discussion taken to the BMWG list. This draft is particularly useful, given the popularity of VNF's within the industry. 
 The document itself was reviewed in ETSI NFV, and used as a reference by at least one of the TST WG and IFA WF specifications (2 references in total). The document has also been reviewed by several individuals from the OPNFV community, and NFVRG (IRTF) is aware of the draft.

Personnel

   Sarah Banks is the Shepherd.
   Warren Kumari is RAD.



From nobody Sat Jun 10 08:28:04 2017
Return-Path: <dromasca@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F405129437; Sat, 10 Jun 2017 08:28:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1RmU7x4qxKK9; Sat, 10 Jun 2017 08:28:00 -0700 (PDT)
Received: from mail-qk0-x22c.google.com (mail-qk0-x22c.google.com [IPv6:2607:f8b0:400d:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B6A1A1200E5; Sat, 10 Jun 2017 08:28:00 -0700 (PDT)
Received: by mail-qk0-x22c.google.com with SMTP id d14so14867990qkb.1; Sat, 10 Jun 2017 08:28:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uA4EcL4Plhm1/Af6UUXZTzs4HmJ0cTgYxzmPEml63s4=; b=O1pI0YeVMjxy86XzjJicapWld37Gs9pif0aQtBP5lROdUYxOyL0B62VLKfe1l2v0sm Oa25sucd9IJRFHcsKqn3YOdlAcYfqXFv9AYnnQMI1jmPwzG8z5vdFk1/LYYE2N9gfW4j 5Km63mYIYUA7m0Tlv+TZ4r180CKMKWa2sayQ4bzHBUlsMxe2yWjRE10f6yKm6tCVqeN3 TlduPiSONBUvjTJyDbMVlnm8Fj2M3jE4+sYLe81Gmc/Fd5idLXoGEw5Q322TJb/iQ2a9 x/rTz9Dqu5dcr9Ep0k4eI1oAm30pLWfrme602Hi4fYfSsau811wkeQxLAOMHL/ke1Z8p Sd3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uA4EcL4Plhm1/Af6UUXZTzs4HmJ0cTgYxzmPEml63s4=; b=Sofixyc4C1ZCBDiF+/kP50zu5SAtTHZkVOw1HVvCMwUmo5bKpmCoMf/ghb3ckYOQWH Q+ajF2ePZydTrVBQCXYCv/pQYP3U/a/vRPppbNGlIKPlm9k6k6IHR+vi0JA6yzi4g/mT ReNNtIOhJrOXT10gorAaarCVZo8SoREOie9kCOefjaUrAc6ukn8mkdoK8rb1o3GUpgZq 9dAl2lEX2IUZoApEYEWmuGsGLR7RQd2SYb7WctiEqJyZAWYLi3g+s6fPwY63KSErfEzI wfGVlV3Yjc8XwUtXwNh4WwDxk0IguiujpBmraRYu9DgmOktBoUHSNFKR5ohDdGvPEDg0 4mfA==
X-Gm-Message-State: AKS2vOwjHcRYqoIzUpc+d8hrBekPhGrIW3c1oJTMeuT5ZNkkL/aI67Pc 4qWoHzOPU7Qpr6Sq/u3K4i3l9emNyg==
X-Received: by 10.55.159.13 with SMTP id i13mr8952731qke.150.1497108479870; Sat, 10 Jun 2017 08:27:59 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.140.31.101 with HTTP; Sat, 10 Jun 2017 08:27:59 -0700 (PDT)
In-Reply-To: <CALTEt=DUhEVgSvYN+2XYUR8Ddq5n8HJtZW2RNj3DV3rSgH0YKQ@mail.gmail.com>
References: <149692498024.25760.14056448148418579949@ietfa.amsl.com> <CALTEt=DUhEVgSvYN+2XYUR8Ddq5n8HJtZW2RNj3DV3rSgH0YKQ@mail.gmail.com>
From: Dan Romascanu <dromasca@gmail.com>
Date: Sat, 10 Jun 2017 18:27:59 +0300
Message-ID: <CAFgnS4Vstv9zyg9rE4-K-q-6GH-P6noXCmoOLvdjTY+dU_uRHw@mail.gmail.com>
To: Lucien Avramov <lucienav@google.com>
Cc: gen-art <gen-art@ietf.org>, draft-ietf-bmwg-dcbench-methodology.all@ietf.org,  ietf <ietf@ietf.org>, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a114d8aa0220ec305519cbbdd"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/581MPZ1CR8uj2GbvUcdsdUl3lUI>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Jun 2017 15:28:03 -0000

--001a114d8aa0220ec305519cbbdd
Content-Type: text/plain; charset="UTF-8"

Hi,

All answers are acceptable. Thanks for addressing my comments.

Regards,

Dan


On Fri, Jun 9, 2017 at 8:11 AM, Lucien Avramov <lucienav@google.com> wrote:

> Hi Dan,
>
> Thank you very much for the comments. I addressed them and published
> version -09.
>
> Inline are some the comments / answer to your feedback. The parts without
> answers mean they have been addressed.
>
> Again, thank you for spending the time to read this in this fine detail.
>
> Lucien
>
>
>
> On Thu, Jun 8, 2017 at 5:29 AM, Dan Romascanu <dromasca@gmail.com> wrote:
>
>> Reviewer: Dan Romascanu
>> Review result: Ready with Issues
>>
>> I am the assigned Gen-ART reviewer for this draft. The General Area
>> Review Team (Gen-ART) reviews all IETF documents being processed
>> by the IESG for the IETF Chair.  Please treat these comments just
>> like any other last call comments.
>>
>> For more information, please see the FAQ at
>>
>> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>>
>> Document: draft-ietf-bmwg-dcbench-methodology-??
>> Reviewer: Dan Romascanu
>> Review Date: 2017-06-08
>> IETF LC End Date: 2017-06-13
>> IESG Telechat date: Not scheduled for a telechat
>>
>> Summary:
>>
>> This Informational I-D describes test and evaluation methodology and
>> measurement techniques for physical network equipment in the data center.
>> It's
>> an useful and clearly written document and it is ready for publication,
>> after a
>> number of minor issues and nits will be addressed.
>>
>> Major issues:
>>
>> Minor issues:
>>
>> 1. There is one single mention of the scope being benchmarking of
>> 'physical
>> network equipment in the data center'. I assume this is written as
>> opposed to
>> virtual networking equipment. This is not included in the title or
>> repeated any
>> place else. I suggest to make more clear the scope by mentioning 'physical
>> equipment' at least once more, for example in the Introduction section.
>>
>> 2. Section 1.2: I am a little concerned about adding additional
>> interpretations
>> to the keywords in RFC2119.
>>
>> Especially about:
>>
>> 'MAY: Comprehensive metric for the scenario described'
>>
>
> *Thank you, MAY provides an optional metric, I have addressed your
> comment *
>
>>
>> MAY has a different meaning in 2119, and the fact that a metric is
>> 'comprehensive' (whatever this means) seems to me orthogonal to the 2119
>> semantics. The proposed interpretation makes little sense to me, if
>> 'comprehensive' than why not required? or at least as a 'should'?
>>
>> 3. Section 6.2: 'The intensity of a microburst MAY be varied ...' It
>> would be
>> useful to make clear what is meant by 'intensity'.
>>
>
> The intensity of a microburst is defined in the definitions document [1]
> so i referenced it that way
>
>
>>
>> Nits/editorial comments:
>>
>> 1. The different formatting of the references [1], [2], etc. vs.
>> [RFC2119],
>> etc. is slightly confusing. Unless there is some good reason I suggest to
>> fix
>> this. Also use of references is inconsistent through the text, sometimes
>> they
>> are mentioned, sometimes they are not.
>>
>
> Fixed it for [2] and [3], however kept [1] as [1], as this document is not
> an RFC yet.
>
>
>>
>> 2. Section 2.2: 'Alternatively when a traffic generator CAN NOT be
>> connected '
>> - this capitalization is not conformant to RFC2119.
>>
>> 3. 'vlan ZZZ' is not consistent with 'vlan X' and 'vlan Y' previously
>> used.
>>
>> 4. Missing reference - RFC 6985
>>
>> 5. Missing expansion and references for DSCP and COS
>>
>> 6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/
>>
>> 7. The methodology (e.g. in section 5) makes an assumption that the DUT
>> has at
>> least nine ports. This may be trivial in a DC, but it's worth being
>> mentioned
>>
>> Thanks added this in section 2.3 as it was making more sense to be
> clarified there
>
>

--001a114d8aa0220ec305519cbbdd
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div>Hi,<br><br></div>All answers are acceptable=
. Thanks for addressing my comments. <br><br></div>Regards,<br><br></div>Da=
n<br><br></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On=
 Fri, Jun 9, 2017 at 8:11 AM, Lucien Avramov <span dir=3D"ltr">&lt;<a href=
=3D"mailto:lucienav@google.com" target=3D"_blank">lucienav@google.com</a>&g=
t;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi Dan,=
=C2=A0<div><br></div><div>Thank you very much for the comments. I addressed=
 them and published version -09.=C2=A0</div><div><br></div><div>Inline are =
some the comments / answer to your feedback. The parts without answers mean=
 they have been addressed.=C2=A0</div><div><br></div><div>Again, thank you =
for spending the time to read this in this fine detail.=C2=A0</div><div><br=
></div><div>Lucien</div><div><br></div><div><br></div><div class=3D"gmail_e=
xtra"><br><div class=3D"gmail_quote"><div><div class=3D"h5">On Thu, Jun 8, =
2017 at 5:29 AM, Dan Romascanu <span dir=3D"ltr">&lt;<a href=3D"mailto:drom=
asca@gmail.com" target=3D"_blank">dromasca@gmail.com</a>&gt;</span> wrote:<=
br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord=
er-left:1px solid rgb(204,204,204);padding-left:1ex">Reviewer: Dan Romascan=
u<br>
Review result: Ready with Issues<br>
<br>
I am the assigned Gen-ART reviewer for this draft. The General Area<br>
Review Team (Gen-ART) reviews all IETF documents being processed<br>
by the IESG for the IETF Chair.=C2=A0 Please treat these comments just<br>
like any other last call comments.<br>
<br>
For more information, please see the FAQ at<br>
<br>
&lt;<a href=3D"https://trac.ietf.org/trac/gen/wiki/GenArtfaq" rel=3D"norefe=
rrer" target=3D"_blank">https://trac.ietf.org/trac/ge<wbr>n/wiki/GenArtfaq<=
/a>&gt;.<br>
<br>
Document: draft-ietf-bmwg-dcbench-method<wbr>ology-??<br>
Reviewer: Dan Romascanu<br>
Review Date: 2017-06-08<br>
IETF LC End Date: 2017-06-13<br>
IESG Telechat date: Not scheduled for a telechat<br>
<br>
Summary:<br>
<br>
This Informational I-D describes test and evaluation methodology and<br>
measurement techniques for physical network equipment in the data center. I=
t&#39;s<br>
an useful and clearly written document and it is ready for publication, aft=
er a<br>
number of minor issues and nits will be addressed.<br>
<br>
Major issues:<br>
<br>
Minor issues:<br>
<br>
1. There is one single mention of the scope being benchmarking of &#39;phys=
ical<br>
network equipment in the data center&#39;. I assume this is written as oppo=
sed to<br>
virtual networking equipment. This is not included in the title or repeated=
 any<br>
place else. I suggest to make more clear the scope by mentioning &#39;physi=
cal<br>
equipment&#39; at least once more, for example in the Introduction section.=
<br>
<br>
2. Section 1.2: I am a little concerned about adding additional interpretat=
ions<br>
to the keywords in RFC2119.<br>
<br>
Especially about:<br>
<br>
&#39;MAY: Comprehensive metric for the scenario described&#39;<br></blockqu=
ote><div><br></div></div></div><div><b><span style=3D"font-size:12.8px">Tha=
nk you, MAY provides an optional metric, I have addressed your comment</spa=
n>=C2=A0</b></div><span class=3D""><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex">
<br>
MAY has a different meaning in 2119, and the fact that a metric is<br>
&#39;comprehensive&#39; (whatever this means) seems to me orthogonal to the=
 2119<br>
semantics. The proposed interpretation makes little sense to me, if<br>
&#39;comprehensive&#39; than why not required? or at least as a &#39;should=
&#39;?<br>
<br>
3. Section 6.2: &#39;The intensity of a microburst MAY be varied ...&#39; I=
t would be<br>
useful to make clear what is meant by &#39;intensity&#39;.<br></blockquote>=
<div><br></div></span><div><div style=3D"font-size:12.8px">The intensity of=
 a microburst is defined in the definitions document [1] so i referenced it=
 that way</div></div><span class=3D""><div>=C2=A0</div><blockquote class=3D=
"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(2=
04,204,204);padding-left:1ex">
<br>
Nits/editorial comments:<br>
<br>
1. The different formatting of the references [1], [2], etc. vs. [RFC2119],=
<br>
etc. is slightly confusing. Unless there is some good reason I suggest to f=
ix<br>
this. Also use of references is inconsistent through the text, sometimes th=
ey<br>
are mentioned, sometimes they are not.<br></blockquote><div><br></div></spa=
n><div>Fixed it for [2] and [3], however kept [1] as [1], as this document =
is not an RFC yet.</div><span class=3D""><div>=C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex">
<br>
2. Section 2.2: &#39;Alternatively when a traffic generator CAN NOT be conn=
ected &#39;<br>
- this capitalization is not conformant to RFC2119.<br>
<br>
3. &#39;vlan ZZZ&#39; is not consistent with &#39;vlan X&#39; and &#39;vlan=
 Y&#39; previously used.<br>
<br>
4. Missing reference - RFC 6985<br>
<br>
5. Missing expansion and references for DSCP and COS<br>
<br>
6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/<br>
<br>
7. The methodology (e.g. in section 5) makes an assumption that the DUT has=
 at<br>
least nine ports. This may be trivial in a DC, but it&#39;s worth being men=
tioned<br>
<br></blockquote></span><div><span style=3D"font-size:12.8px">Thanks added =
this in section 2.3 as it was making more sense to be clarified there</span=
>=C2=A0</div></div><br></div></div>
</blockquote></div><br></div>

--001a114d8aa0220ec305519cbbdd--


From nobody Sat Jun 10 10:26:17 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A429C128768 for <bmwg@ietfa.amsl.com>; Sat, 10 Jun 2017 10:26:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yRRU8_Mnd5qz for <bmwg@ietfa.amsl.com>; Sat, 10 Jun 2017 10:26:09 -0700 (PDT)
Received: from mail-pg0-x229.google.com (mail-pg0-x229.google.com [IPv6:2607:f8b0:400e:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 25E7C1279EB for <bmwg@ietf.org>; Sat, 10 Jun 2017 10:26:07 -0700 (PDT)
Received: by mail-pg0-x229.google.com with SMTP id a70so34560946pge.3 for <bmwg@ietf.org>; Sat, 10 Jun 2017 10:26:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ZtGTPVNvFOUzYfWK8go3UQ72lUnqON4f8m60bYv6k4I=; b=BJ11Os8/GrHAUniJQNH47dCojHjE5bSsQEXCXNMhHIvuuHmbtmyuDMPdAW6SbYEXE4 VQveikwBtiMZRA3t459N5STf8sdLALORzhQvwqso5l4pTQgvduGCjiX/hdDpR+TIYYyy m9qWy/zxDcdadmcqxVe+0p1QeMqCe4B6wO5N3HfR0ZkGXuowxeKk9bvay1LswKO+H/hg eFa1chKPxNI3NtpHlvCaikWREe22eGmRCYQxrdkZfVH/PF4+xzAaKLymbFJZryZNtNho hPd4cKxX1cymUqV4as4cvyS0a5+GdQwMZjuSwzBl8obpn3eJPi+JtelcotByaaNvYjAw XfYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ZtGTPVNvFOUzYfWK8go3UQ72lUnqON4f8m60bYv6k4I=; b=BQ0u9S2X8BCeZtUa7o4CSU4XCOutNYxFyqUwbh2jfje+tI/t7MCQQRHYB76zrOH60J 2APJ/Q+nZrKKvk2kMu0HK8LmRUqaWKxiZgAK88B/dYPOY2VKfYK4mOqOgzM7yS6W6Q0S OBOup42YLm+b6W9+rBobf8NeajuqO8WDD0wg4FoInGiorpkjiWTsCpiCedOrQFNuN1SE DF6E4HQ4A3gqDE5OVGQq6Z0oPQpV6d80Jz7P8II9rW454IGUNqWoaOX/7yAT0A7lNIYF oJOTXlH/rMTiuYryBeGRT76fHqEuyrIT8kL5+O0kMtjgmv4thJ8bpKLrd2nse6npI+oO CWqA==
X-Gm-Message-State: AODbwcCXmtueZT/idqOHZj08SqQcQW4aHN48hBiYicrs/+r2U9kpqGeX uhWXbuAfZQE5Xaby
X-Received: by 10.98.80.140 with SMTP id g12mr17181420pfj.146.1497115566443; Sat, 10 Jun 2017 10:26:06 -0700 (PDT)
Received: from [10.0.1.76] (c-98-248-223-72.hsd1.ca.comcast.net. [98.248.223.72]) by smtp.gmail.com with ESMTPSA id k15sm4542588pfb.1.2017.06.10.10.26.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 10 Jun 2017 10:26:05 -0700 (PDT)
Content-Type: multipart/alternative; boundary=Apple-Mail-07F97384-E491-4FE1-AD34-2328613D6C54
Mime-Version: 1.0 (1.0)
From: Lucien Avramov <lucienav@google.com>
X-Mailer: iPhone Mail (14E304)
In-Reply-To: <CAFgnS4Vstv9zyg9rE4-K-q-6GH-P6noXCmoOLvdjTY+dU_uRHw@mail.gmail.com>
Date: Sat, 10 Jun 2017 10:26:04 -0700
Cc: gen-art <gen-art@ietf.org>, draft-ietf-bmwg-dcbench-methodology.all@ietf.org, ietf <ietf@ietf.org>, bmwg@ietf.org
Content-Transfer-Encoding: 7bit
Message-Id: <AC86F980-D9C1-4DE1-8296-70DB2A6482F9@google.com>
References: <149692498024.25760.14056448148418579949@ietfa.amsl.com> <CALTEt=DUhEVgSvYN+2XYUR8Ddq5n8HJtZW2RNj3DV3rSgH0YKQ@mail.gmail.com> <CAFgnS4Vstv9zyg9rE4-K-q-6GH-P6noXCmoOLvdjTY+dU_uRHw@mail.gmail.com>
To: Dan Romascanu <dromasca@gmail.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/YF0d73Sh37tVnoeVQLNrDd-RfE0>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Jun 2017 17:26:11 -0000

--Apple-Mail-07F97384-E491-4FE1-AD34-2328613D6C54
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

Thank you Dan, much appreciative of your feedback.

Lucien

> On Jun 10, 2017, at 08:27, Dan Romascanu <dromasca@gmail.com> wrote:
>=20
> Hi,
>=20
> All answers are acceptable. Thanks for addressing my comments.=20
>=20
> Regards,
>=20
> Dan
>=20
>=20
>> On Fri, Jun 9, 2017 at 8:11 AM, Lucien Avramov <lucienav@google.com> wrot=
e:
>> Hi Dan,=20
>>=20
>> Thank you very much for the comments. I addressed them and published vers=
ion -09.=20
>>=20
>> Inline are some the comments / answer to your feedback. The parts without=
 answers mean they have been addressed.=20
>>=20
>> Again, thank you for spending the time to read this in this fine detail.=20=

>>=20
>> Lucien
>>=20
>>=20
>>=20
>>> On Thu, Jun 8, 2017 at 5:29 AM, Dan Romascanu <dromasca@gmail.com> wrote=
:
>>> Reviewer: Dan Romascanu
>>> Review result: Ready with Issues
>>>=20
>>> I am the assigned Gen-ART reviewer for this draft. The General Area
>>> Review Team (Gen-ART) reviews all IETF documents being processed
>>> by the IESG for the IETF Chair.  Please treat these comments just
>>> like any other last call comments.
>>>=20
>>> For more information, please see the FAQ at
>>>=20
>>> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>>>=20
>>> Document: draft-ietf-bmwg-dcbench-methodology-??
>>> Reviewer: Dan Romascanu
>>> Review Date: 2017-06-08
>>> IETF LC End Date: 2017-06-13
>>> IESG Telechat date: Not scheduled for a telechat
>>>=20
>>> Summary:
>>>=20
>>> This Informational I-D describes test and evaluation methodology and
>>> measurement techniques for physical network equipment in the data center=
. It's
>>> an useful and clearly written document and it is ready for publication, a=
fter a
>>> number of minor issues and nits will be addressed.
>>>=20
>>> Major issues:
>>>=20
>>> Minor issues:
>>>=20
>>> 1. There is one single mention of the scope being benchmarking of 'physi=
cal
>>> network equipment in the data center'. I assume this is written as oppos=
ed to
>>> virtual networking equipment. This is not included in the title or repea=
ted any
>>> place else. I suggest to make more clear the scope by mentioning 'physic=
al
>>> equipment' at least once more, for example in the Introduction section.
>>>=20
>>> 2. Section 1.2: I am a little concerned about adding additional interpre=
tations
>>> to the keywords in RFC2119.
>>>=20
>>> Especially about:
>>>=20
>>> 'MAY: Comprehensive metric for the scenario described'
>>=20
>> Thank you, MAY provides an optional metric, I have addressed your comment=
=20
>>>=20
>>> MAY has a different meaning in 2119, and the fact that a metric is
>>> 'comprehensive' (whatever this means) seems to me orthogonal to the 2119=

>>> semantics. The proposed interpretation makes little sense to me, if
>>> 'comprehensive' than why not required? or at least as a 'should'?
>>>=20
>>> 3. Section 6.2: 'The intensity of a microburst MAY be varied ...' It wou=
ld be
>>> useful to make clear what is meant by 'intensity'.
>>=20
>> The intensity of a microburst is defined in the definitions document [1] s=
o i referenced it that way
>> =20
>>>=20
>>> Nits/editorial comments:
>>>=20
>>> 1. The different formatting of the references [1], [2], etc. vs. [RFC211=
9],
>>> etc. is slightly confusing. Unless there is some good reason I suggest t=
o fix
>>> this. Also use of references is inconsistent through the text, sometimes=
 they
>>> are mentioned, sometimes they are not.
>>=20
>> Fixed it for [2] and [3], however kept [1] as [1], as this document is no=
t an RFC yet.
>> =20
>>>=20
>>> 2. Section 2.2: 'Alternatively when a traffic generator CAN NOT be conne=
cted '
>>> - this capitalization is not conformant to RFC2119.
>>>=20
>>> 3. 'vlan ZZZ' is not consistent with 'vlan X' and 'vlan Y' previously us=
ed.
>>>=20
>>> 4. Missing reference - RFC 6985
>>>=20
>>> 5. Missing expansion and references for DSCP and COS
>>>=20
>>> 6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/
>>>=20
>>> 7. The methodology (e.g. in section 5) makes an assumption that the DUT h=
as at
>>> least nine ports. This may be trivial in a DC, but it's worth being ment=
ioned
>>>=20
>>=20
>> Thanks added this in section 2.3 as it was making more sense to be clarif=
ied there=20
>>=20
>=20

--Apple-Mail-07F97384-E491-4FE1-AD34-2328613D6C54
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div>Thank you Dan, much appreciative of yo=
ur feedback.</div><div id=3D"AppleMailSignature"><br>Lucien</div><div><br>On=
 Jun 10, 2017, at 08:27, Dan Romascanu &lt;<a href=3D"mailto:dromasca@gmail.=
com">dromasca@gmail.com</a>&gt; wrote:<br><br></div><blockquote type=3D"cite=
"><div><div dir=3D"ltr"><div><div><div>Hi,<br><br></div>All answers are acce=
ptable. Thanks for addressing my comments. <br><br></div>Regards,<br><br></d=
iv>Dan<br><br></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote=
">On Fri, Jun 9, 2017 at 8:11 AM, Lucien Avramov <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:lucienav@google.com" target=3D"_blank">lucienav@google.com</a>&=
gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi Dan,&=
nbsp;<div><br></div><div>Thank you very much for the comments. I addressed t=
hem and published version -09.&nbsp;</div><div><br></div><div>Inline are som=
e the comments / answer to your feedback. The parts without answers mean the=
y have been addressed.&nbsp;</div><div><br></div><div>Again, thank you for s=
pending the time to read this in this fine detail.&nbsp;</div><div><br></div=
><div>Lucien</div><div><br></div><div><br></div><div class=3D"gmail_extra"><=
br><div class=3D"gmail_quote"><div><div class=3D"h5">On Thu, Jun 8, 2017 at 5=
:29 AM, Dan Romascanu <span dir=3D"ltr">&lt;<a href=3D"mailto:dromasca@gmail=
.com" target=3D"_blank">dromasca@gmail.com</a>&gt;</span> wrote:<br><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px s=
olid rgb(204,204,204);padding-left:1ex">Reviewer: Dan Romascanu<br>
Review result: Ready with Issues<br>
<br>
I am the assigned Gen-ART reviewer for this draft. The General Area<br>
Review Team (Gen-ART) reviews all IETF documents being processed<br>
by the IESG for the IETF Chair.&nbsp; Please treat these comments just<br>
like any other last call comments.<br>
<br>
For more information, please see the FAQ at<br>
<br>
&lt;<a href=3D"https://trac.ietf.org/trac/gen/wiki/GenArtfaq" rel=3D"norefer=
rer" target=3D"_blank">https://trac.ietf.org/trac/ge<wbr>n/wiki/GenArtfaq</a=
>&gt;.<br>
<br>
Document: draft-ietf-bmwg-dcbench-method<wbr>ology-??<br>
Reviewer: Dan Romascanu<br>
Review Date: 2017-06-08<br>
IETF LC End Date: 2017-06-13<br>
IESG Telechat date: Not scheduled for a telechat<br>
<br>
Summary:<br>
<br>
This Informational I-D describes test and evaluation methodology and<br>
measurement techniques for physical network equipment in the data center. It=
's<br>
an useful and clearly written document and it is ready for publication, afte=
r a<br>
number of minor issues and nits will be addressed.<br>
<br>
Major issues:<br>
<br>
Minor issues:<br>
<br>
1. There is one single mention of the scope being benchmarking of 'physical<=
br>
network equipment in the data center'. I assume this is written as opposed t=
o<br>
virtual networking equipment. This is not included in the title or repeated a=
ny<br>
place else. I suggest to make more clear the scope by mentioning 'physical<b=
r>
equipment' at least once more, for example in the Introduction section.<br>
<br>
2. Section 1.2: I am a little concerned about adding additional interpretati=
ons<br>
to the keywords in RFC2119.<br>
<br>
Especially about:<br>
<br>
'MAY: Comprehensive metric for the scenario described'<br></blockquote><div>=
<br></div></div></div><div><b><span style=3D"font-size:12.8px">Thank you, MA=
Y provides an optional metric, I have addressed your comment</span>&nbsp;</b=
></div><span class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0p=
x 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
MAY has a different meaning in 2119, and the fact that a metric is<br>
'comprehensive' (whatever this means) seems to me orthogonal to the 2119<br>=

semantics. The proposed interpretation makes little sense to me, if<br>
'comprehensive' than why not required? or at least as a 'should'?<br>
<br>
3. Section 6.2: 'The intensity of a microburst MAY be varied ...' It would b=
e<br>
useful to make clear what is meant by 'intensity'.<br></blockquote><div><br>=
</div></span><div><div style=3D"font-size:12.8px">The intensity of a microbu=
rst is defined in the definitions document [1] so i referenced it that way</=
div></div><span class=3D""><div>&nbsp;</div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);p=
adding-left:1ex">
<br>
Nits/editorial comments:<br>
<br>
1. The different formatting of the references [1], [2], etc. vs. [RFC2119],<=
br>
etc. is slightly confusing. Unless there is some good reason I suggest to fi=
x<br>
this. Also use of references is inconsistent through the text, sometimes the=
y<br>
are mentioned, sometimes they are not.<br></blockquote><div><br></div></span=
><div>Fixed it for [2] and [3], however kept [1] as [1], as this document is=
 not an RFC yet.</div><span class=3D""><div>&nbsp;</div><blockquote class=3D=
"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex">
<br>
2. Section 2.2: 'Alternatively when a traffic generator CAN NOT be connected=
 '<br>
- this capitalization is not conformant to RFC2119.<br>
<br>
3. 'vlan ZZZ' is not consistent with 'vlan X' and 'vlan Y' previously used.<=
br>
<br>
4. Missing reference - RFC 6985<br>
<br>
5. Missing expansion and references for DSCP and COS<br>
<br>
6. Sections 3.3 and 6.3: s/number of iteration/number of iterations/<br>
<br>
7. The methodology (e.g. in section 5) makes an assumption that the DUT has a=
t<br>
least nine ports. This may be trivial in a DC, but it's worth being mentione=
d<br>
<br></blockquote></span><div><span style=3D"font-size:12.8px">Thanks added t=
his in section 2.3 as it was making more sense to be clarified there</span>&=
nbsp;</div></div><br></div></div>
</blockquote></div><br></div>
</div></blockquote></body></html>=

--Apple-Mail-07F97384-E491-4FE1-AD34-2328613D6C54--


From nobody Mon Jun 12 05:44:45 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A5D9126B6D; Mon, 12 Jun 2017 05:44:37 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149727147725.10260.1123930589293201897@ietfa.amsl.com>
Date: Mon, 12 Jun 2017 05:44:37 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/B6O617fEQZ1DCAkWLV5xC2ReVos>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Jun 2017 12:44:37 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition Technologies
        Authors         : Marius Georgescu
                          Liviu Pislaru
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08.txt
	Pages           : 28
	Date            : 2017-06-12

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a metric for benchmarking load
   scalability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Mon Jun 12 05:48:31 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 640E3126B6D; Mon, 12 Jun 2017 05:48:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level: 
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jzRO4n8_vtsU; Mon, 12 Jun 2017 05:48:26 -0700 (PDT)
Received: from mailproxy3.rcs-rds.ro (mailproxy3.rcs-rds.ro [212.54.120.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6C98012EAB6; Mon, 12 Jun 2017 05:48:14 -0700 (PDT)
Received: from [10.172.5.198] (unknown [10.172.5.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy3.rcs-rds.ro (Postfix) with ESMTPSA id 70E0E1BE712; Mon, 12 Jun 2017 15:48:11 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1497271691; bh=19xCgDV+C5AylBeSTzOFzsIcRBt7NVhPqWC2dqnuoyI=; h=From:Subject:Date:In-Reply-To:Cc:To:References; b=O82qcfryf+TXDkYugqyrB41htd+7dN5iKWRTp1QWeoITAhjy+WyxP2dUa0iW8WaAO cyJ0pySbSNmrzYq6u4uV913yMbhQUnUs+VJumCFXJya58Ltmg7EpzhEBKWPhWPrrwv BgGqtXwsVcx0PUgTomYsstquYrL9sQtU7gh8IAAs=
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
Message-Id: <FBB1F522-D896-407D-809A-56F298596F02@rcs-rds.ro>
Content-Type: multipart/alternative; boundary="Apple-Mail=_3AEE031A-C2F4-43DD-A160-FEAE4A0F4CD8"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Date: Mon, 12 Jun 2017 15:48:09 +0300
In-Reply-To: <CAHw9_iJ8Q+6bmz7=ZXaFNH5ofOweM8uhCh-r=HffUjkGw1dMEg@mail.gmail.com>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org
To: Warren Kumari <warren@kumari.net>
References: <CAHw9_iJ8Q+6bmz7=ZXaFNH5ofOweM8uhCh-r=HffUjkGw1dMEg@mail.gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/t0e-If0oBmh9-oY0AppvD-NQCbw>
Subject: Re: [bmwg] Final edits on draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Jun 2017 12:48:30 -0000

--Apple-Mail=_3AEE031A-C2F4-43DD-A160-FEAE4A0F4CD8
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hello Warren,

Thank you for your help.
We have uploaded version 8, which covers from our perspective the IESG =
comments.
=
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08=
 =
<https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-0=
8>

Best regards,
Marius (for the authors)
> On Jun 8, 2017, at 9:31 PM, Warren Kumari <warren@kumari.net> wrote:
>=20
> Hi there,
>=20
> Your document was on the IESG telechat today, and is now in Approved,
> Point Raised raised state. It doesn't strictly **need** a revised ID,
> but I do think that incorporating these helpful comments /
> clarifications.
>=20
> Alia thinks it could do with an informative reference for baby jumbo
> frames, and says that she has provided a helpful to Al. I think that
> it would be useful to include that, and, while you are editing it,
> also address Alvaro and Adam's points (they are simply
> clarifications).
>=20
> Please explicitly let me know once done, and I can finish the
> approval, and ship it to the RFC ed.
>=20
> W
>=20
> --=20
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>   ---maf
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


--Apple-Mail=_3AEE031A-C2F4-43DD-A160-FEAE4A0F4CD8
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">Hello Warren,<div class=3D""><br class=3D""></div><div =
class=3D"">Thank you for your help.</div><div class=3D"">We have =
uploaded version 8, which covers from our perspective the IESG =
comments.</div><div class=3D""><a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchma=
rking-08" =
class=3D"">https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benc=
hmarking-08</a></div><div class=3D""><br class=3D""></div><div =
class=3D"">Best regards,</div><div class=3D"">Marius (for the =
authors)<br class=3D""><div><blockquote type=3D"cite" class=3D""><div =
class=3D"">On Jun 8, 2017, at 9:31 PM, Warren Kumari &lt;<a =
href=3D"mailto:warren@kumari.net" class=3D"">warren@kumari.net</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div =
class=3D"">Hi there,<br class=3D""><br class=3D"">Your document was on =
the IESG telechat today, and is now in Approved,<br class=3D"">Point =
Raised raised state. It doesn't strictly **need** a revised ID,<br =
class=3D"">but I do think that incorporating these helpful comments /<br =
class=3D"">clarifications.<br class=3D""><br class=3D"">Alia thinks it =
could do with an informative reference for baby jumbo<br =
class=3D"">frames, and says that she has provided a helpful to Al. I =
think that<br class=3D"">it would be useful to include that, and, while =
you are editing it,<br class=3D"">also address Alvaro and Adam's points =
(they are simply<br class=3D"">clarifications).<br class=3D""><br =
class=3D"">Please explicitly let me know once done, and I can finish =
the<br class=3D"">approval, and ship it to the RFC ed.<br class=3D""><br =
class=3D"">W<br class=3D""><br class=3D"">-- <br class=3D"">I don't =
think the execution is relevant when it was obviously a bad<br =
class=3D"">idea in the first place.<br class=3D"">This is like putting =
rabid weasels in your pants, and later expressing<br class=3D"">regret =
at having chosen those particular rabid weasels and that pair<br =
class=3D"">of pants.<br class=3D""> &nbsp;&nbsp;---maf<br class=3D""><br =
class=3D"">_______________________________________________<br =
class=3D"">bmwg mailing list<br class=3D""><a =
href=3D"mailto:bmwg@ietf.org" class=3D"">bmwg@ietf.org</a><br =
class=3D"">https://www.ietf.org/mailman/listinfo/bmwg<br =
class=3D""></div></div></blockquote></div><br =
class=3D""></div></body></html>=

--Apple-Mail=_3AEE031A-C2F4-43DD-A160-FEAE4A0F4CD8--


From nobody Mon Jun 12 09:20:32 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 77E0412940E for <bmwg@ietfa.amsl.com>; Mon, 12 Jun 2017 09:20:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fD5HXWsrU8vC for <bmwg@ietfa.amsl.com>; Mon, 12 Jun 2017 09:20:28 -0700 (PDT)
Received: from mail-ua0-x234.google.com (mail-ua0-x234.google.com [IPv6:2607:f8b0:400c:c08::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5E507129406 for <bmwg@ietf.org>; Mon, 12 Jun 2017 09:20:28 -0700 (PDT)
Received: by mail-ua0-x234.google.com with SMTP id h39so59293067uaa.3 for <bmwg@ietf.org>; Mon, 12 Jun 2017 09:20:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1lSWCyO6HovJQDcf0aEnyFPrPV06DU3RUNodI8cYCnM=; b=VlKJrM6RZZNx9/GO0H3nnxRpdcGhiJAZkOHJO13rmlCpjhbSOO9KG1ovX3L1YqtU/P y5mJMjfGOSX74xWmgU7sfdI6MygCMKYU8g4gYE2InhojtFNn68wcF0gHEzE08NuILMdM dDpNECnhT91nS+3X/GuypXVC0YcEdjGy0vUVJKhzjuuqpYsxw+vzgHa5FrgEhyki948p lOGWy2McR7SwSxMbU9sxeX8W1NnzeF752+oTBTjhcr32EH8UzS9jCsIns7I0kMX67tyW Y3wbRbPa1w49qr+QdAM1wUXUvdfRwXumujcTahsJGXvVlcq0fUtx2p6395LkMK25SBxf ULyA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1lSWCyO6HovJQDcf0aEnyFPrPV06DU3RUNodI8cYCnM=; b=maBCfuupMIe6hkg7se6Z9v5uOhSChurBCsPHFKq/K6ssDwzrGYHqx46dpsGij3Wy/V H0gKtIBLtXko3HWcwI9NJmykPyeSQ8tR9IIkDwBm1pMhlaG88LZ9io/2IJvxxRbG7PE+ F1qTlN3mPoBhMeMvIY9RjIs6zLzLbzLExv9baZ2bXKbdAtp0+LVcQsoJqdQSbBYde1zj EwFncT1zUgjXWgocT9mNndGpMnRQ9XwpBkKBoQ4dp010UyCJyc3JhNhpEuXYuhEdO8AX VzeIzx9A7IVbvUaF7v9jfZpzw8AglbG5YNyI/XXc87BXpVp0oX51g2D/sZMMkSj0J58Q Pbeg==
X-Gm-Message-State: AODbwcDOGfxVuCzMajFkYmpYSDlAVKFwLmj/f70haDIkQ7ahQt/Hgdqn RuyU9SJTE0i8lORy3frFe39zVXqHFwEsWk8=
X-Received: by 10.176.77.230 with SMTP id b38mr33576523uah.76.1497284427443; Mon, 12 Jun 2017 09:20:27 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.82.198 with HTTP; Mon, 12 Jun 2017 09:19:47 -0700 (PDT)
In-Reply-To: <FBB1F522-D896-407D-809A-56F298596F02@rcs-rds.ro>
References: <CAHw9_iJ8Q+6bmz7=ZXaFNH5ofOweM8uhCh-r=HffUjkGw1dMEg@mail.gmail.com> <FBB1F522-D896-407D-809A-56F298596F02@rcs-rds.ro>
From: Warren Kumari <warren@kumari.net>
Date: Mon, 12 Jun 2017 12:19:47 -0400
Message-ID: <CAHw9_iLehig5nFJdr276ujkQ1mSSuvS6A2WFtopvAnJkosXnUA@mail.gmail.com>
To: Marius Georgescu <marius.georgescu@rcs-rds.ro>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/znAeZwjwEaSuBJIuQGokbuo3jVA>
Subject: Re: [bmwg] Final edits on draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Jun 2017 16:20:30 -0000

Thank you very much for the quick turnaround.
I think that the edits have improved the document - I've approved it,
you should see the announcement soon...

W

On Mon, Jun 12, 2017 at 8:48 AM, Marius Georgescu
<marius.georgescu@rcs-rds.ro> wrote:
> Hello Warren,
>
> Thank you for your help.
> We have uploaded version 8, which covers from our perspective the IESG
> comments.
> https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08
>
> Best regards,
> Marius (for the authors)
>
> On Jun 8, 2017, at 9:31 PM, Warren Kumari <warren@kumari.net> wrote:
>
> Hi there,
>
> Your document was on the IESG telechat today, and is now in Approved,
> Point Raised raised state. It doesn't strictly **need** a revised ID,
> but I do think that incorporating these helpful comments /
> clarifications.
>
> Alia thinks it could do with an informative reference for baby jumbo
> frames, and says that she has provided a helpful to Al. I think that
> it would be useful to include that, and, while you are editing it,
> also address Alvaro and Adam's points (they are simply
> clarifications).
>
> Please explicitly let me know once done, and I can finish the
> approval, and ship it to the RFC ed.
>
> W
>
> --
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>   ---maf
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>
>



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Mon Jun 12 09:39:07 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C0C2129408; Mon, 12 Jun 2017 09:38:59 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.0
Auto-Submitted: auto-generated
Precedence: bulk
Cc: The IESG <iesg@ietf.org>, bmwg-chairs@ietf.org, acmorton@att.com, Al Morton <acmorton@att.com>, bmwg@ietf.org, rfc-editor@rfc-editor.org, warren@kumari.net, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149728553915.10251.14714710659997380620.idtracker@ietfa.amsl.com>
Date: Mon, 12 Jun 2017 09:38:59 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/9zByIhAzedWqjBt6RZnHG918cds>
Subject: [bmwg] Document Action: 'Benchmarking Methodology for IPv6 Transition Technologies' to Informational RFC (draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08.txt)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Jun 2017 16:38:59 -0000

The IESG has approved the following document:
- 'Benchmarking Methodology for IPv6 Transition Technologies'
  (draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08.txt) as Informational RFC

This document is the product of the Benchmarking Methodology Working Group.

The IESG contact persons are Warren Kumari and Benoit Claise.

A URL of this Internet Draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/





Technical Summary

   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a metric for benchmarking load
   scalability.

Working Group Summary

   Once the Working Group agreed on the approach taken in this draft,
   development of the draft was straightforward and WG consensus was 
   smooth. No issues raised during IETF LC.

Document Quality

   Document is clear (and provides a nice introduction to
   the technology as well!). 
   Review of key v6 experts was obtained, see Acknowledgements.

Personnel

   Al Morton in the Document Shepherd.
   Warren Kumari is the Responsible AD.


From nobody Tue Jun 13 10:02:01 2017
Return-Path: <sbanks@encrypted.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A1EF11316EF for <bmwg@ietfa.amsl.com>; Tue, 13 Jun 2017 10:01:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z5HdVuUBMTTd for <bmwg@ietfa.amsl.com>; Tue, 13 Jun 2017 10:01:58 -0700 (PDT)
Received: from aws.hosed.org (aws.hosed.org [50.16.104.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4D7961316E8 for <bmwg@ietf.org>; Tue, 13 Jun 2017 10:01:58 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by aws.hosed.org (Postfix) with ESMTP id 977FE803BE; Tue, 13 Jun 2017 13:01:57 -0400 (EDT)
X-Virus-Scanned: Debian amavisd-new at aws.hosed.org
Received: from aws.hosed.org ([127.0.0.1]) by localhost (aws.hosed.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BHfDJWgS6grI; Tue, 13 Jun 2017 13:01:57 -0400 (EDT)
Received: from [172.31.98.233] (193.sub-166-148-94.myvzw.com [166.148.94.193]) by aws.hosed.org (Postfix) with ESMTPSA id 1E5B180366; Tue, 13 Jun 2017 13:01:54 -0400 (EDT)
From: Sarah B <sbanks@encrypted.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Date: Tue, 13 Jun 2017 10:01:47 -0700
Message-Id: <503A3A34-6C12-449F-A0A1-DE0134C3E1C6@encrypted.net>
To: bmwg@ietf.org
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/10zT-85r4_hxda0R0MK4rwA_ZkI>
Subject: [bmwg] SDN Controller and IGP Database size
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Jun 2017 17:02:00 -0000

Hello,
	At our last IETF meeting, I took an action to follow up with the =
rest of my fellow authors on the SDN Controller draft with a question =
around IGP database size, and if it was in scope of the draft. We have =
consensus amongst ourselves (as authors) that this is out of scope; it's =
an interesting question, related, but orthogonal to the intention of the =
draft, and therefore out of scope. Please let us know if you have any =
questions.

Kind regards,
Sarah


From nobody Tue Jun 13 21:00:07 2017
Return-Path: <linuxwolf+ietf@outer-planes.net>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5292B127337; Tue, 13 Jun 2017 21:00:05 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Matthew Miller <linuxwolf+ietf@outer-planes.net>
To: <gen-art@ietf.org>
Cc: ietf@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-dcbench-terminology.all@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149741280522.7481.5692213354540706614@ietfa.amsl.com>
Date: Tue, 13 Jun 2017 21:00:05 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/lgBoPFa5wk_8YJRFMyEtsNxhH2s>
Subject: [bmwg] Genart last call review of draft-ietf-bmwg-dcbench-terminology-10
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 14 Jun 2017 04:00:05 -0000

Reviewer: Matthew Miller
Review result: Almost Ready

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-dcbench-terminology-10
Reviewer: Matthew A. Miller
Review Date: 2017-06-13
IETF LC End Date: 2017-06-13
IESG Telechat date: N/A

Summary:

This document is almost ready; this document is on the path to its
goals set out in the abstract and introduction, but the execution
needs more work.

I think the authors would benefit from another round of proofreading
and submit a new revision.  Below are the most obvious or egregious
issues I have.

Major issues:

* In section 6.2 "Incast", it is unclear what is being measured,
and what the units (if any) for that measurement are.  Is it the
percentage of synchronization?  Is it the synchronous arrival time
(which presumably is measured as a delta over some magnitude of
seconds)? Is it the number of ingress and egress ports?  If it is
the number of ingress and egress ports, then it should define what
"egress" means for the purpose of this measurement

* In Section 8. "Security Considerations", what is the reason for
the "SHOULD" and "SHOULD NOT" in the last paragraph (discussing
special capabilities)?  At the least it seems safest for the
exceptions that rationalize the "SHOULD" and "SHOULD NOT" be
explicitly stated here, or changed to "MUST" and "MUST NOT" if
that rationale cannot be stated.

Minor issues:

* RFC 2119 should be a normative reference, as the reader MUST
understand what the terms as-written mean.

* RFC 5841 needs to be a listed reference; It is explicitly pointed
to in Sections 3.1. and 3.2. From the text is seems a Normative
reference is warranted but it at least needs to be documented in
References.

* RFC 2647 needs to be a listed reference. It is explicitly pointed
to in Section 7.1.; from the text it seems appropriate to be an
Informative reference.

* The definition in Section 1.2. of what the "Measurement Units"
subsections definitions format doesn't seem to be followed in
this document.  Specifically, the "Measurement Units" subsections
rarely ever mention a unit of measure; instead the unit of measure
is almost always specified as part of the "Definition" subsection.
It may be worth revisiting the name for the "Measurement Units"
subsections, or to move the unit of measure out of the
"Definitions" subsections and into the "Measurement Units"
subsections.

* In section 2.3., the use of MUST, MAY, and MUST NOT seems
a little contradictory.  The MUST for point 1 would seem to
already negate point 3, and prohibit point 2.  Would changing
that MUST to SHOULD be acceptable?

Nits/editorial comments:

* It would be helpful to include a reference to what "north-south"
and "east-west" mean, if possible.

* the acronyms DUT and SUT -- at the least -- need to be
expanded on first use, or in Section 1.1. "Terminology".

* In a number of subsections (notably under Section 4. and Section
6.), square brackets ("[" and "]") are used instead of parentheses
("(" and ")"), without any clear reason for the difference.  They
should be switched to parentheses.

* In Section 2.2., the word "for" or "at" should be removed in the
sentence "The change of behavior happens for at specific larger
packet sizes."

* Section 2.3., the phrase "the application commonly need to" should
be changed to "the application commonly needs to".

* In Section 5.2., there is a comma missing between "crystals" and
"or" in "The crystals or clock modules, usually have a specific".

* Also in Section 5.2., the term "devices under test" is used
instead of DUTs the rest of the document seems to use.

* Throughout Section 6.1., the acronyms "cos" and "dcsp" should be
expanded on first use.

* In Section 6.1.1, the definition of "Buffer Size" calls out using
some magnitude of bytes (B, KB, MB, GB), then the example denotes
"Mb" -- which commonly is used for megabits.  From the rest of the
definition, it seems this should be MB, but I have not calculated
if the numerical value preceding the unit needs to be revisited.

* In Section 6.2.1, the Stateful and Stateless terms both use the
phrase "such as for example", which is redundant.  Either "such as"
or "for example" is sufficient, and the other should be removed.

* The formatting in Section 10. "References" is very inconsistent.
There is odd indentation and inconsistent anchor usage.

* The affiliation and email address for Lucien Avramov is Google,
but the street address is mostly that for Cisco Systems, Inc.



From nobody Wed Jun 14 07:04:00 2017
Return-Path: <ietf@kuehlewind.net>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5305712EA96; Wed, 14 Jun 2017 07:03:53 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: =?utf-8?q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-terminology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149744903324.14047.8077563121882638735.idtracker@ietfa.amsl.com>
Date: Wed, 14 Jun 2017 07:03:53 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/mtrs7Igx0ooEiVxnDU4j4ELnRJA>
Subject: [bmwg] =?utf-8?q?Mirja_K=C3=BChlewind=27s_No_Objection_on_draft-i?= =?utf-8?q?etf-bmwg-dcbench-terminology-10=3A_=28with_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 14 Jun 2017 14:03:53 -0000

Mirja Kühlewind has entered the following ballot position for
draft-ietf-bmwg-dcbench-terminology-10: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I didn't put in a discuss because it’s not worth holding up the document as the
content seems fine, BUT I really don't see why this terminology is specified
FOR data centers ONLY. All terminology seem to be applicable in general and as
such this seems to be an extension of RFC1242. Even incast, which is actually a
data center term, is so generally defined that one could use it also in a
different context as well. Further latency is defined in both documents, where
this drafts mentions RFC1242 and mainly extends the description there. That's
in general fine, but I probably would recommend to simply change the title,
remove the work data center there, and make this document update RFC 1242.



From nobody Wed Jun 14 07:16:20 2017
Return-Path: <ietf@kuehlewind.net>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 21F51128CDB; Wed, 14 Jun 2017 07:16:14 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: =?utf-8?q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-terminology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149744977404.13951.15551847308245216174.idtracker@ietfa.amsl.com>
Date: Wed, 14 Jun 2017 07:16:14 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/rOIQpqiO_Q_Q4wQ5O9gHf6Di9w0>
Subject: [bmwg] =?utf-8?q?Mirja_K=C3=BChlewind=27s_No_Objection_on_draft-i?= =?utf-8?q?etf-bmwg-dcbench-terminology-10=3A_=28with_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 14 Jun 2017 14:16:14 -0000

Mirja Kühlewind has entered the following ballot position for
draft-ietf-bmwg-dcbench-terminology-10: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I didn't put in a discuss because it’s not worth holding up the document as the
content seems fine, BUT I really don't see why this terminology is specified
FOR data centers ONLY. All terminology seem to be applicable in general and as
such this seems to be an extension of RFC1242. Even incast, which is actually a
data center term, is so generally defined that one could use it also in a
different context as well. Further latency is defined in both documents, where
this drafts mentions RFC1242 and mainly extends the description there. That's
in general fine, but I probably would recommend to simply change the title,
remove the work data center there, and make this document update RFC 1242. (+
remove the intro text on data centers because the same text is anyway in
draft-ietf-bmwg-dcbench-methodology).



From nobody Wed Jun 14 08:20:29 2017
Return-Path: <ietf@kuehlewind.net>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id CB48F124E15; Wed, 14 Jun 2017 08:20:21 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: =?utf-8?q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149745362178.14099.2258834805567479367.idtracker@ietfa.amsl.com>
Date: Wed, 14 Jun 2017 08:20:21 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Cv6Fdv-6OGNlyvNkWvBSZDMp134>
Subject: [bmwg] =?utf-8?q?Mirja_K=C3=BChlewind=27s_Discuss_on_draft-ietf-b?= =?utf-8?q?mwg-dcbench-methodology-09=3A_=28with_DISCUSS_and_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 14 Jun 2017 15:20:22 -0000

Mirja Kühlewind has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-09: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Again, I don't really understand why any of these tests are specific for data
center equipment. These tests can be applied to any equipment. I think the only
point is that these tests are especially important to run for data center
equipment but that doesn't change the fact that the test itself are not
specific for data center equipment. The reason why it's a discuss this time, is
that I further really don't understand why existing tests have to be specified
again is this document even though a test for that case already exists that
specifies exactly the same thing in another RFC; I'm talking about Section 5 on
Head of Line Blocking that specifies the same test as in section 5.5 of RFC2889
(there called Congestion Control which a confusing term because this doesn't
address end-to-end congestion control, anyway...). I didn't check if that's
also the case for any of the other tests.


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Please provide the appropriate references for DSCP and COS!

Also I find the name of section 6 confusing ("Incast Stateful and Stateless
Traffic ") because your microburst test (section 4) is basically also incast
testing but without TCP cross-traffic. Further the terms stateful and stateless
are also confusing to me; I'd usually use adaptive and constant bit rate
(CBR)/non-adaptive; or is stateful/stateless the commonly-used term in
benchmarking?



From nobody Thu Jun 15 02:49:32 2017
Return-Path: <manoj_ramalingam@live.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C962E12EA56 for <bmwg@ietfa.amsl.com>; Thu, 15 Jun 2017 02:49:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id on0rjZe0PT_E for <bmwg@ietfa.amsl.com>; Thu, 15 Jun 2017 02:49:28 -0700 (PDT)
Received: from APC01-PU1-obe.outbound.protection.outlook.com (mail-oln040092254034.outbound.protection.outlook.com [40.92.254.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F413512EB4C for <bmwg@ietf.org>; Thu, 15 Jun 2017 02:49:27 -0700 (PDT)
Received: from SG2APC01FT053.eop-APC01.prod.protection.outlook.com (10.152.250.54) by SG2APC01HT030.eop-APC01.prod.protection.outlook.com (10.152.251.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1157.12; Thu, 15 Jun 2017 09:49:25 +0000
Received: from PN1PR01MB0127.INDPRD01.PROD.OUTLOOK.COM (10.152.250.60) by SG2APC01FT053.mail.protection.outlook.com (10.152.250.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1075.5 via Frontend Transport; Thu, 15 Jun 2017 09:49:25 +0000
Received: from PN1PR01MB0127.INDPRD01.PROD.OUTLOOK.COM ([10.164.139.16]) by PN1PR01MB0127.INDPRD01.PROD.OUTLOOK.COM ([10.164.139.16]) with mapi id 15.01.1178.016; Thu, 15 Jun 2017 09:49:24 +0000
From: Manoj Ramalingam <manoj_ramalingam@live.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQJBM8mo
Date: Thu, 15 Jun 2017 09:49:24 +0000
Message-ID: <PN1PR01MB0127F0C519F48A755DC376E984C00@PN1PR01MB0127.INDPRD01.PROD.OUTLOOK.COM>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: att.com; dkim=none (message not signed) header.d=none;att.com; dmarc=none action=none header.from=live.com;
x-incomingtopheadermarker: OriginalChecksum:F4012A974933A901B93AEA32234E3F9B552F6E2E0B557D8E422C0B2B09E15BF9; UpperCasedChecksum:F2317F2E140A298E3A644F13592869C641E15652116FCB1F8E18FD74C9429FF4; SizeAsReceived:7275; Count:45
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [wA/hzcEYCPkJ8rOklxMRiG8nS+ShByuS]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; SG2APC01HT030; 7:C55SPU5j/aEgz2CdWYKhy+X2hSX+LpPVBqna0EsAj5YOJaVlHuSY8+ORCA+92Tr2naZPzVJmVWd3Ll8mSJQPjzpFTutPECRCDSEyp+mWDr4ctSMuOl60nPN1LCbqZ0oeRjeQHbh47WK+fczfaqhK2PiHF5cs9Dy1wBaKmfuwK55m9lEop5TjRLU5MEgSAVB1uNzIb1yiFiWMCVf4YYMj/nPyEOWwA7gAphl1RCe1b1PyE6Qp2j1r29cc3b0VlY7IVrBX2UQg9Yp+P+XDfXwZSvGr+P41GVApKX0tFKNv3Hu568EoxQsE/eell4nyM4ebd4POOKV2nvWgElCb4B6d3E9lexUxPO7x3u1+N/YHZ3Ula/ApQSA/YZ1UFQ2s3WXi+rrw02//7Lgx8rOTv0Gf+z7A3G7fndpOXbY6zvDx3G/xWp1sYU9BWB+1nq+iFyzBQaXULE814bhO/QOcWTwlr6dBUc1OHF83Z2Th9/kivxai/jcSEv/jkkXIfa19nmM4udzMEQB5Igq6Y6ooPAygBH84gJFBINFTa18cKhPGNB+CqUooEigZUyCNP6NUljeFybarsnBjxoQWnHJjitaEChnrfaQT595zuqikyMitwiYrCLajnnmmKOKWoHTZc0maWiM8pIpJeglxmWPX6WjQb3C0nJSlLj8FAtipWv1qKNjkQd0Xqu5CrMv/sY4CiBloWwi3QUJVHBhz4Fz0zyJzWTU82/K7b/58SO+4MZ7WDqfj1vz8x66fJ/yupPmKxUhlmO6aTiaZ84kDdMIxsRhhCQ==
x-incomingheadercount: 45
x-eopattributedmessage: 0
x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:SG2APC01HT030; H:PN1PR01MB0127.INDPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; 
x-ms-traffictypediagnostic: SG2APC01HT030:
x-ms-office365-filtering-correlation-id: 286b2ca0-924d-459f-c9ca-08d4b3d3ce79
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322274)(1603101448)(1601125374)(1701031045); SRVR:SG2APC01HT030; 
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(444000031); SRVR:SG2APC01HT030; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SG2APC01HT030; 
x-forefront-prvs: 0339F89554
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_PN1PR01MB0127F0C519F48A755DC376E984C00PN1PR01MB0127INDP_"
MIME-Version: 1.0
X-OriginatorOrg: live.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jun 2017 09:49:24.8860 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT030
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/k-o5-OUsU3H_6jfhdMU31AET7Ug>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 15 Jun 2017 09:49:31 -0000

--_000_PN1PR01MB0127F0C519F48A755DC376E984C00PN1PR01MB0127INDP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,


    Support this Draft for adoption. It provides methodology to benchmark c=
ritical performance parameters for EVPN.


Thanks,

Manoj R


________________________________
From: bmwg <bmwg-bounces@ietf.org> on behalf of MORTON, ALFRED C (AL) <acmo=
rton@att.com>
Sent: Sunday, June 4, 2017 4:04 AM
To: bmwg@ietf.org
Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07

BMWG,

At our Chicago meeting, the BMWG reviewed a version of
the Internet Draft on

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-05

There was sufficient readership present at the meeting
and positive/constructive comments on the work, so the
chairs asked whether support existed to adopt the draft
as a WG item. There was support and no objections. As
always, such decisions are officially taken on the
mailing list.

This message*, therefore, starts a call for adoption on:

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-07

to run until Friday 23 June 2017.

Please reply to bmwg@ietf.org indicating:

(1) whether you support addition of the following milestone to
   the BMWG charter:

date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG

(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
   as the basis document for this milestone

(3) whether you commit to reviewing the WG document if adopted

We are aware that BMWG has nearly completed all its existing
chartered items.  This usually begins discussion of
re-chartering, this time with Warren (our new AD Advisor). The final
decision/schedule for adoption will depend on these discussions.

thanks for your consideration of the draft, and regards,

Al
bmwg co-chair

*using Brian Trammell's recent message to IPPM as a template (thanks!)

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg
bmwg Info Page - Internet Engineering Task Force<https://www.ietf.org/mailm=
an/listinfo/bmwg>
www.ietf.org
To see the collection of prior postings to the list, visit the bmwg Archive=
s or bmwg MHonArc Archives. Using bmwg: To post a message to all the list .=
..



--_000_PN1PR01MB0127F0C519F48A755DC376E984C00PN1PR01MB0127INDP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"><!-- P {margin-top:0;margi=
n-bottom:0;} --></style>
</head>
<body dir=3D"ltr">
<div id=3D"divtagdefaultwrapper" style=3D"font-size:12pt;color:#000000;font=
-family:Calibri,Arial,Helvetica,sans-serif;" dir=3D"ltr">
<p>Hi,</p>
<p><br>
</p>
<p>&nbsp;&nbsp;&nbsp; Support this Draft for adoption. It provides methodol=
ogy to benchmark critical performance parameters for EVPN.</p>
<p><br>
</p>
<p>Thanks,</p>
<p>Manoj R<br>
</p>
<p><br>
</p>
<div style=3D"color: rgb(0, 0, 0);">
<div>
<hr tabindex=3D"-1" style=3D"display:inline-block; width:98%">
<div id=3D"x_divRplyFwdMsg" dir=3D"ltr"><font style=3D"font-size:11pt" face=
=3D"Calibri, sans-serif" color=3D"#000000"><b>From:</b> bmwg &lt;bmwg-bounc=
es@ietf.org&gt; on behalf of MORTON, ALFRED C (AL) &lt;acmorton@att.com&gt;=
<br>
<b>Sent:</b> Sunday, June 4, 2017 4:04 AM<br>
<b>To:</b> bmwg@ietf.org<br>
<b>Subject:</b> [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07</f=
ont>
<div>&nbsp;</div>
</div>
</div>
<font size=3D"2"><span style=3D"font-size:10pt;">
<div class=3D"PlainText">BMWG,<br>
<br>
At our Chicago meeting, the BMWG reviewed a version of<br>
the Internet Draft on<br>
<br>
Benchmarking Methodology for EVPN and PBB-EVPN<br>
draft-kishjac-bmwg-evpntest-05<br>
<br>
There was sufficient readership present at the meeting<br>
and positive/constructive comments on the work, so the<br>
chairs asked whether support existed to adopt the draft<br>
as a WG item. There was support and no objections. As<br>
always, such decisions are officially taken on the <br>
mailing list.<br>
<br>
This message*, therefore, starts a call for adoption on:<br>
<br>
Benchmarking Methodology for EVPN and PBB-EVPN<br>
draft-kishjac-bmwg-evpntest-07<br>
<br>
to run until Friday 23 June 2017. <br>
<br>
Please reply to bmwg@ietf.org indicating:<br>
<br>
(1) whether you support addition of the following milestone to <br>
&nbsp;&nbsp; the BMWG charter:<br>
<br>
date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG<br>
<br>
(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07 <br>
&nbsp;&nbsp; as the basis document for this milestone<br>
<br>
(3) whether you commit to reviewing the WG document if adopted<br>
<br>
We are aware that BMWG has nearly completed all its existing<br>
chartered items.&nbsp; This usually begins discussion of <br>
re-chartering, this time with Warren (our new AD Advisor). The final<br>
decision/schedule for adoption will depend on these discussions. <br>
<br>
thanks for your consideration of the draft, and regards,<br>
<br>
Al<br>
bmwg co-chair<br>
<br>
*using Brian Trammell's recent message to IPPM as a template (thanks!)<br>
<br>
_______________________________________________<br>
bmwg mailing list<br>
bmwg@ietf.org<br>
<a href=3D"https://www.ietf.org/mailman/listinfo/bmwg" id=3D"LPlnk672373" p=
reviewremoved=3D"true">https://www.ietf.org/mailman/listinfo/bmwg</a>
<div id=3D"LPBorder_GT_14975200328330.19132352399900687" style=3D"margin-bo=
ttom: 20px; overflow: auto; width: 100%; text-indent: 0px;">
<table id=3D"LPContainer_14975200328280.23153418797374514" style=3D"width: =
90%; background-color: rgb(255, 255, 255); position: relative; overflow: au=
to; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top: =
1px dotted rgb(200, 200, 200); border-bottom: 1px dotted rgb(200, 200, 200)=
;" role=3D"presentation" cellspacing=3D"0">
<tbody>
<tr style=3D"border-spacing: 0px;" valign=3D"top">
<td id=3D"TextCell_14975200328290.06046269884841737" style=3D"vertical-alig=
n: top; position: relative; padding: 0px; display: table-cell;" colspan=3D"=
2">
<div id=3D"LPRemovePreviewContainer_14975200328290.29732641521036096"></div=
>
<div id=3D"LPTitle_14975200328290.11017330614542487" style=3D"top: 0px; col=
or: rgb(0, 120, 215); font-weight: 400; font-size: 21px; font-family: &quot=
;wf_segoe-ui_light&quot;,&quot;Segoe UI Light&quot;,&quot;Segoe WP Light&qu=
ot;,&quot;Segoe UI&quot;,&quot;Segoe WP&quot;,Tahoma,Arial,sans-serif; line=
-height: 21px;">
<a id=3D"LPUrlAnchor_14975200328300.0691704468715224" style=3D"text-decorat=
ion: none;" href=3D"https://www.ietf.org/mailman/listinfo/bmwg" target=3D"_=
blank">bmwg Info Page - Internet Engineering Task Force</a></div>
<div id=3D"LPMetadata_14975200328310.14759081778033456" style=3D"margin: 10=
px 0px 16px; color: rgb(102, 102, 102); font-weight: 400; font-family: &quo=
t;wf_segoe-ui_normal&quot;,&quot;Segoe UI&quot;,&quot;Segoe WP&quot;,Tahoma=
,Arial,sans-serif; font-size: 14px; line-height: 14px;">
www.ietf.org</div>
<div id=3D"LPDescription_14975200328320.8470412231895204" style=3D"display:=
 block; color: rgb(102, 102, 102); font-weight: 400; font-family: &quot;wf_=
segoe-ui_normal&quot;,&quot;Segoe UI&quot;,&quot;Segoe WP&quot;,Tahoma,Aria=
l,sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; overfl=
ow: hidden;">
To see the collection of prior postings to the list, visit the bmwg Archive=
s or bmwg MHonArc Archives. Using bmwg: To post a message to all the list .=
..</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
</div>
</span></font></div>
</div>
</body>
</html>

--_000_PN1PR01MB0127F0C519F48A755DC376E984C00PN1PR01MB0127INDP_--


From nobody Thu Jun 15 21:15:03 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 171E9126DCA; Thu, 15 Jun 2017 21:15:02 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149758650187.11133.15007295104938063197@ietfa.amsl.com>
Date: Thu, 15 Jun 2017 21:15:01 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/JzI1O0t3Uz1rbX_tHnYJpN26yBg>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-11.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 16 Jun 2017 04:15:02 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-11.txt
	Pages           : 17
	Date            : 2017-06-15

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center. The
terminologies are not only data center specific and can be seen as
updates to [RFC1242], [RFC2432], [RFC2544], [RFC2889] and [RFC3918],
without the intent to cover the all use cases outside of data center.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-11
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-11

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-11


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun 15 21:15:37 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C3E1128D64; Thu, 15 Jun 2017 21:15:29 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149758652940.11200.17772856045391303106@ietfa.amsl.com>
Date: Thu, 15 Jun 2017 21:15:29 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/g5m-8PBCw4seBrbwXefyg3Cod3g>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-10.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 16 Jun 2017 04:15:29 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-10.txt
	Pages           : 18
	Date            : 2017-06-15

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. The methodologies are not only
   data center specific and can be seen as updates to [RFC1242],
   [RFC2432], [RFC2544], [RFC2889], [RFC3918], and [RF6985] without the
   intent to cover the all use cases outside of data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-10
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-10

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-10


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun 15 22:06:59 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 1C8D11200C1; Thu, 15 Jun 2017 22:06:51 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.54.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149758961108.11229.6996832967965295769@ietfa.amsl.com>
Date: Thu, 15 Jun 2017 22:06:51 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/mA7vS-2pJkwq5UwqO_42rYNrOgU>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-12.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 16 Jun 2017 05:06:51 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-12.txt
	Pages           : 17
	Date            : 2017-06-15

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center. The
terminologies are not only data center specific and can be seen as
updates to [RFC1242], [RFC2432], [RFC2544], [RFC2889] and [RFC3918],
without the intent to cover the all use cases outside of data center.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-12
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-12

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-12


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun 15 22:10:15 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 51F47128C83 for <bmwg@ietfa.amsl.com>; Thu, 15 Jun 2017 22:10:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Aa9ueu0RDrUi for <bmwg@ietfa.amsl.com>; Thu, 15 Jun 2017 22:10:03 -0700 (PDT)
Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6A5021200C1 for <bmwg@ietf.org>; Thu, 15 Jun 2017 22:10:03 -0700 (PDT)
Received: by mail-pf0-x22b.google.com with SMTP id s66so17594906pfs.1 for <bmwg@ietf.org>; Thu, 15 Jun 2017 22:10:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1MzW+4GS38pV0WSPQQwPrZiGZ/cfaVXAB1ECkxrRhvI=; b=uIa/LF20NvivIY1u47gVH4D4es1v7HT3WIWhl59dg850RjswJk8XViGaOT0aJkBrLA 3Y4Za1tTEKQOczE6UTi7dAUKwROisJfDhBVH/aA3mjG0cSV5wMmxKdwWVmEl69CztoLy S+WQBsdS7KBKGV7x4dS1OgkhSQ6fdy8tnTWulH6zyTOneTLBmmMGXSZGXC1nunJXZvnE jleYqcegE2CWMMnHQb9O1AQ4CYILzG8xLKiUIo7B7j2ZLAydYjnqrgtp45Emq+M+vzMV 9ddry2xMZ4aEh2aJU5tbI1ZG8Pi0vbrJs7eHhSh48bz+uwx/U99j5U2ZlZL+q/b4sVv6 i8cg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1MzW+4GS38pV0WSPQQwPrZiGZ/cfaVXAB1ECkxrRhvI=; b=B5kKmxfNiqL7Xq6nx4IdvAewMfDL2buXedI7Wl6eTVbX1f+yl9Il+EK0dR5KGqvHkl yNp3hdy2vUZhSKpVkTIXBJBOplexeqxXzk4o3IRSqKDVE6fgYnibAcHbrEEMzg4/qiBa 6W6pgaGdXtLL85L8X0WqT7GVQFR+C/P1nnQRWYwl/FMIfqgpSy56EeuDHAtMgfR3PmIa QDe1Y9RCfoPdL9FUy28Nf7gur24XiDz1hBNE45G5i5eRJszuVTWXP+yKnX3HRIRqcVBI OUW+lv/CXnSXgoVOWYugmfqb7bAyANEv690TqTUx+CUTQYzzkSbhnvhVBylPf2R6Elq8 V7UA==
X-Gm-Message-State: AKS2vOwzBJCIfBo2+KuAaz0mKWcK+gqd3aGGVcYgHhLOSTAGnHTuQtqu lUU74QUCj6pv5ywDXjkBES0mkNHn4Gxq
X-Received: by 10.99.105.4 with SMTP id e4mr9354729pgc.186.1497589802828; Thu, 15 Jun 2017 22:10:02 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Thu, 15 Jun 2017 22:09:42 -0700 (PDT)
In-Reply-To: <149741280522.7481.5692213354540706614@ietfa.amsl.com>
References: <149741280522.7481.5692213354540706614@ietfa.amsl.com>
From: Lucien Avramov <lucienav@google.com>
Date: Thu, 15 Jun 2017 22:09:42 -0700
Message-ID: <CALTEt=DWK5n8xVJ_ZD3j5BZmBT=_id1ZERCHPY3SGEqYL6N8sg@mail.gmail.com>
To: Matthew Miller <linuxwolf+ietf@outer-planes.net>
Cc: gen-art <gen-art@ietf.org>, ietf <ietf@ietf.org>, bmwg@ietf.org,  draft-ietf-bmwg-dcbench-terminology.all@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c140dce380ba605520cccb4"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/6avJqxQqCsm3PX3eswYSMTEORxo>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-dcbench-terminology-10
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 16 Jun 2017 05:10:06 -0000

--94eb2c140dce380ba605520cccb4
Content-Type: text/plain; charset="UTF-8"

Hi Matthew!

Thank you for the review!
We incorporated your feedback and now it's in -12: of the draft
<https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/>

Please see inline more details to each of your feedback.

Thank you for such a detailed write up.

Best,
Lucien




> *Subject:* *Genart last call review of
> draft-ietf-bmwg-dcbench-terminology-10*
>
> Reviewer: Matthew Miller
> Review result: Almost Ready
>
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-bmwg-dcbench-terminology-10
> Reviewer: Matthew A. Miller
> Review Date: 2017-06-13
> IETF LC End Date: 2017-06-13
> IESG Telechat date: N/A
>
> Summary:
>
> This document is almost ready; this document is on the path to its
> goals set out in the abstract and introduction, but the execution
> needs more work.
>
> I think the authors would benefit from another round of proofreading
> and submit a new revision.  Below are the most obvious or egregious
> issues I have.
>
> Major issues:
>
> * In section 6.2 "Incast", it is unclear what is being measured,
> and what the units (if any) for that measurement are.  Is it the
> percentage of synchronization?  Is it the synchronous arrival time
> (which presumably is measured as a delta over some magnitude of
> seconds)? Is it the number of ingress and egress ports?  If it is
> the number of ingress and egress ports, then it should define what
> "egress" means for the purpose of this measurement
>
>
fixed!

>
>
> * In Section 8. "Security Considerations", what is the reason for
> the "SHOULD" and "SHOULD NOT" in the last paragraph (discussing
> special capabilities)?  At the least it seems safest for the
> exceptions that rationalize the "SHOULD" and "SHOULD NOT" be
> explicitly stated here, or changed to "MUST" and "MUST NOT" if
> that rationale cannot be stated.
>
>
These are the standard security considerations all BMWG uses for the
drafts. We want to stay with the considerations we chose.

>
>
> Minor issues:
>
> * RFC 2119 should be a normative reference, as the reader MUST
> understand what the terms as-written mean.
>
> done


>
>
> * RFC 5841 needs to be a listed reference; It is explicitly pointed
> to in Sections 3.1. and 3.2. From the text is seems a Normative
> reference is warranted but it at least needs to be documented in
> References.
>
> done

>
>
> * RFC 2647 needs to be a listed reference. It is explicitly pointed
> to in Section 7.1.; from the text it seems appropriate to be an
> Informative reference.
>
> done

>
>
> * The definition in Section 1.2. of what the "Measurement Units"
> subsections definitions format doesn't seem to be followed in
> this document.  Specifically, the "Measurement Units" subsections
> rarely ever mention a unit of measure; instead the unit of measure
> is almost always specified as part of the "Definition" subsection.
> It may be worth revisiting the name for the "Measurement Units"
> subsections, or to move the unit of measure out of the
> "Definitions" subsections and into the "Measurement Units"
> subsections.
>
> Section 1.2 has the purpose to explain how the document is formatted.
it has 3 subsections each time moving forward as described in 1.2. We use
that third subsection in this document to explain how to measure the topic
discussed.


   - Definition: The specific definition for the term.


   - Discussion: A brief discussion about the term, its application


   - Measurement Units


>
> * In section 2.3., the use of MUST, MAY, and MUST NOT seems
> a little contradictory.  The MUST for point 1 would seem to
> already negate point 3, and prohibit point 2.  Would changing
> that MUST to SHOULD be acceptable?
>
> We wanted to keep MUST , MAY and MUST NOT to explain that : 1) is the way
to go ALWAYS. 2) can be done but it very specific to particular corner use
case. 3) should never be done, although 1) negates 3), we want to call out
3, because people often time do 1) and don't follow 3. (they would do a
LIFO test although they must not do it).



These are the standard security considerations all BMWG uses for the
drafts. We want to stay with the considerations we chose.

>
>
> Nits/editorial comments:
>
> * It would be helpful to include a reference to what "north-south"
> and "east-west" mean, if possible.
>
> done!

>
>
> * the acronyms DUT and SUT -- at the least -- need to be
> expanded on first use, or in Section 1.1. "Terminology".
>
>
done!

>
>
> * In a number of subsections (notably under Section 4. and Section
> 6.), square brackets ("[" and "]") are used instead of parentheses
> ("(" and ")"), without any clear reason for the difference.  They
> should be switched to parentheses.
>
> done!

>
>
> * In Section 2.2., the word "for" or "at" should be removed in the
> sentence "The change of behavior happens for at specific larger
> packet sizes."
>
> done!

>
>
> * Section 2.3., the phrase "the application commonly need to" should
> be changed to "the application commonly needs to".
>
> done!

> * In Section 5.2., there is a comma missing between "crystals" and
> "or" in "The crystals or clock modules, usually have a specific".
>
> done!

>
>
> * Also in Section 5.2., the term "devices under test" is used
> instead of DUTs the rest of the document seems to use.
>
> done!

>
>
> * Throughout Section 6.1., the acronyms "cos" and "dcsp" should be
> expanded on first use.
>
> done!

>
>
> * In Section 6.1.1, the definition of "Buffer Size" calls out using
> some magnitude of bytes (B, KB, MB, GB), then the example denotes
> "Mb" -- which commonly is used for megabits.  From the rest of the
> definition, it seems this should be MB, but I have not calculated
> if the numerical value preceding the unit needs to be revisited.
>
> done!

>
>
> * In Section 6.2.1, the Stateful and Stateless terms both use the
> phrase "such as for example", which is redundant.  Either "such as"
> or "for example" is sufficient, and the other should be removed.
>
> * The formatting in Section 10. "References" is very inconsistent.
> There is odd indentation and inconsistent anchor usage.
>
>
fixed as much as i could, there is a problem with nits in submission tool
when

>
>
> * The affiliation and email address for Lucien Avramov is Google,
> but the street address is mostly that for Cisco Systems, Inc.
>
> done

--94eb2c140dce380ba605520cccb4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Matthew!<div><br></div><div>Thank you for the review!</=
div><div>We incorporated your feedback and now it&#39;s in -12: of the <a h=
ref=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology=
/">draft</a></div><div><br></div><div>Please see inline more details to eac=
h of your feedback.</div><div><br></div><div>Thank you for such a detailed =
write up.</div><div><br></div><div>Best,</div><div>Lucien</div><div><br></d=
iv><div><br><div class=3D"gmail_extra"><br><div class=3D"gmail_quote"><div =
style=3D"min-height:100%"><div class=3D"gmail-nH" style=3D"width:1188px"><d=
iv class=3D"gmail-nH" style=3D""><div class=3D"gmail-nH gmail-bkL"><div cla=
ss=3D"gmail-no"><div class=3D"gmail-nH gmail-bkK gmail-nn" style=3D"width:9=
86px"><div class=3D"gmail-nH"><div class=3D"gmail-nH"><div class=3D"gmail-n=
H gmail-ar4 gmail-z"><div class=3D"gmail-aeI"><div class=3D"gmail-AO"><div =
id=3D"gmail-:4" class=3D"gmail-Tm gmail-aeJ" style=3D"height:551px"><div id=
=3D"gmail-:2" class=3D"gmail-aeF" style=3D"min-height:121px"><div class=3D"=
gmail-nH"><div class=3D"gmail-nH"><div class=3D"gmail-nH gmail-g"><table cl=
ass=3D"gmail-Bs gmail-nH gmail-iY" cellpadding=3D"0" style=3D"width:956px">=
<tbody><tr><td class=3D"gmail-Bu"><div class=3D"gmail-nH gmail-if"><div cla=
ss=3D"gmail-nH gmail-aHU"><div class=3D"gmail-nH gmail-hx"><div class=3D"gm=
ail-nH"><div class=3D"gmail-h7 gmail-hn gmail-ie gmail-nH gmail-oy8Mbf" tab=
index=3D"-1"><div class=3D"gmail-Bk" style=3D"width:705px"><div class=3D"gm=
ail-G3 gmail-G2 gmail-afm"><div id=3D"gmail-:1ph"><div class=3D"gmail-adn g=
mail-ads"><div class=3D"gmail-gs"><div id=3D"gmail-:1qg" class=3D"gmail-ii =
gmail-gt gmail-adO" style=3D"font-size:12.8px"><div id=3D"gmail-:1z2" class=
=3D"gmail-a3s gmail-aXjCH gmail-m15caf4bc03a5f652"><div dir=3D"ltr"><div cl=
ass=3D"gmail_extra"><div class=3D"gmail_quote"><div class=3D"gmail-adM"><di=
v class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div=
 class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-5=
66981276781100764gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div=
 dir=3D"auto"><blockquote type=3D"cite" style=3D"font-size:12.8px"><br><b>S=
ubject:</b>=C2=A0<b>Genart last call review of draft-ietf-bmwg-dcbench-term=
in<wbr>ology-10</b><br><br></blockquote><div style=3D"font-size:12.8px"><di=
v class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><block=
quote type=3D"cite">Reviewer: Matthew Miller<br>Review result: Almost Ready=
<br><br>I am the assigned Gen-ART reviewer for this draft. The General Area=
<br>Review Team (Gen-ART) reviews all IETF documents being processed<br>by =
the IESG for the IETF Chair.=C2=A0 Please treat these comments just<br>like=
 any other last call comments.<br><br>For more information, please see the =
FAQ at<br><br>&lt;<a href=3D"https://trac.ietf.org/trac/gen/wiki/GenArtfaq"=
 target=3D"_blank">https://trac.ietf.org/trac/ge<wbr>n/wiki/GenArtfaq</a>&g=
t;.<br><br>Document: draft-ietf-bmwg-dcbench-termin<wbr>ology-10<br>Reviewe=
r: Matthew A. Miller<br>Review Date: 2017-06-13<br>IETF LC End Date: 2017-0=
6-13<br>IESG Telechat date: N/A<br><br>Summary:<br><br>This document is alm=
ost ready; this document is on the path to its<br>goals set out in the abst=
ract and introduction, but the execution<br>needs more work.<br><br>I think=
 the authors would benefit from another round of proofreading<br>and submit=
 a new revision.=C2=A0 Below are the most obvious or egregious<br>issues I =
have.<br><br>Major issues:<br><br>* In section 6.2 &quot;Incast&quot;, it i=
s unclear what is being measured,<br>and what the units (if any) for that m=
easurement are.=C2=A0 Is it the<br>percentage of synchronization?=C2=A0 Is =
it the synchronous arrival time<br>(which presumably is measured as a delta=
 over some magnitude of<br>seconds)? Is it the number of ingress and egress=
 ports?=C2=A0 If it is<br>the number of ingress and egress ports, then it s=
hould define what<br>&quot;egress&quot; means for the purpose of this measu=
rement</blockquote></div></div></div></div></div></div></div></blockquote><=
div><br></div></div></div><div>fixed!=C2=A0</div><span class=3D"gmail-im"><=
blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l=
eft:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m_-566=
981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764gmail=
-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><div cla=
ss=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockquote=
 type=3D"cite"><br><br>* In Section 8. &quot;Security Considerations&quot;,=
 what is the reason for<br>the &quot;SHOULD&quot; and &quot;SHOULD NOT&quot=
; in the last paragraph (discussing<br>special capabilities)?=C2=A0 At the =
least it seems safest for the<br>exceptions that rationalize the &quot;SHOU=
LD&quot; and &quot;SHOULD NOT&quot; be<br>explicitly stated here, or change=
d to &quot;MUST&quot; and &quot;MUST NOT&quot; if<br>that rationale cannot =
be stated.</blockquote></div></div></div></div></div></div></blockquote><di=
v><br></div></span><div>These are the standard security considerations all =
BMWG uses for the drafts. We want to stay with the considerations we chose.=
 =C2=A0=C2=A0</div><span class=3D"gmail-im"><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204)=
;padding-left:1ex"><div class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><=
div class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr"><div cla=
ss=3D"gmail_quote"><div dir=3D"auto"><div class=3D"gmail-m_-566981276781100=
764gmail-m_8608744719394608261h5"><blockquote type=3D"cite"><br><br>Minor i=
ssues:<br><br>* RFC 2119 should be a normative reference, as the reader MUS=
T<br>understand what the terms as-written mean.</blockquote></div></div></d=
iv></div></div></div></blockquote></span><div>done</div><span class=3D"gmai=
l-im"><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0p=
x 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><d=
iv class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_=
-566981276781100764gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><d=
iv dir=3D"auto"><div class=3D"gmail-m_-566981276781100764gmail-m_8608744719=
394608261h5"><blockquote type=3D"cite"><br><br>* RFC 5841 needs to be a lis=
ted reference; It is explicitly pointed<br>to in Sections 3.1. and 3.2. Fro=
m the text is seems a Normative<br>reference is warranted but it at least n=
eeds to be documented in<br>References.</blockquote></div></div></div></div=
></div></div></blockquote></span><div>done=C2=A0</div><span class=3D"gmail-=
im"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m=
_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764=
gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><di=
v class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><block=
quote type=3D"cite"><br><br>* RFC 2647 needs to be a listed reference. It i=
s explicitly pointed<br>to in Section 7.1.; from the text it seems appropri=
ate to be an<br>Informative reference.</blockquote></div></div></div></div>=
</div></div></blockquote></span><div>done=C2=A0</div><span class=3D"gmail-i=
m"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord=
er-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m_=
-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764g=
mail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><div=
 class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite"><br><br>* The definition in Section 1.2. of what the &qu=
ot;Measurement Units&quot;<br>subsections definitions format doesn&#39;t se=
em to be followed in<br>this document.=C2=A0 Specifically, the &quot;Measur=
ement Units&quot; subsections<br>rarely ever mention a unit of measure; ins=
tead the unit of measure<br>is almost always specified as part of the &quot=
;Definition&quot; subsection.<br>It may be worth revisiting the name for th=
e &quot;Measurement Units&quot;<br>subsections, or to move the unit of meas=
ure out of the<br>&quot;Definitions&quot; subsections and into the &quot;Me=
asurement Units&quot;<br>subsections.</blockquote></div></div></div></div><=
/div></div></blockquote></span><div>Section 1.2 has the purpose to explain =
how the document is formatted.=C2=A0</div><div>it has 3 subsections each ti=
me moving forward as described in 1.2. We use that third subsection in this=
 document to explain how to measure the topic discussed.=C2=A0</div><div><b=
r></div><div><div><ul><li>Definition: The specific definition for the term.=
</li></ul></div><div><ul><li>Discussion: A brief discussion about the term,=
 its application</li></ul></div><div><ul><li>Measurement Units<br></li></ul=
></div></div><span class=3D"gmail-im"><blockquote class=3D"gmail_quote" sty=
le=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddi=
ng-left:1ex"><div class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div cl=
ass=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr"><div class=3D"=
gmail_quote"><div dir=3D"auto"><div class=3D"gmail-m_-566981276781100764gma=
il-m_8608744719394608261h5"><blockquote type=3D"cite"><br><br>* In section =
2.3., the use of MUST, MAY, and MUST NOT seems<br>a little contradictory.=
=C2=A0 The MUST for point 1 would seem to<br>already negate point 3, and pr=
ohibit point 2.=C2=A0 Would changing<br>that MUST to SHOULD be acceptable?<=
/blockquote></div></div></div></div></div></div></blockquote></span><div>We=
 wanted to keep MUST , MAY and MUST NOT to explain that : 1) is the way to =
go ALWAYS. 2) can be done but it very specific to particular corner use cas=
e. 3) should never be done, although 1) negates 3), we want to call out 3, =
because people often time do 1) and don&#39;t follow 3. (they would do a LI=
FO test although they must not do it).=C2=A0</div><div><br></div><div>=C2=
=A0</div><div><br></div><div>These are the standard security considerations=
 all BMWG uses for the drafts. We want to stay with the considerations we c=
hose. =C2=A0</div><span class=3D"gmail-im"><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex"><div class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><d=
iv class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr"><div clas=
s=3D"gmail_quote"><div dir=3D"auto"><div class=3D"gmail-m_-5669812767811007=
64gmail-m_8608744719394608261h5"><blockquote type=3D"cite"><br><br>Nits/edi=
torial comments:<br><br>* It would be helpful to include a reference to wha=
t &quot;north-south&quot;<br>and &quot;east-west&quot; mean, if possible.</=
blockquote></div></div></div></div></div></div></blockquote></span><div>don=
e!=C2=A0</div><span class=3D"gmail-im"><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padd=
ing-left:1ex"><div class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div c=
lass=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr"><div class=3D=
"gmail_quote"><div dir=3D"auto"><div class=3D"gmail-m_-566981276781100764gm=
ail-m_8608744719394608261h5"><blockquote type=3D"cite"><br><br>* the acrony=
ms DUT and SUT -- at the least -- need to be<br>expanded on first use, or i=
n Section 1.1. &quot;Terminology&quot;.</blockquote></div></div></div></div=
></div></div></blockquote><div><br></div></span><div>done!=C2=A0</div><span=
 class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-56=
6981276781100764gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div =
dir=3D"auto"><div class=3D"gmail-m_-566981276781100764gmail-m_8608744719394=
608261h5"><blockquote type=3D"cite"><br><br>* In a number of subsections (n=
otably under Section 4. and Section<br>6.), square brackets (&quot;[&quot; =
and &quot;]&quot;) are used instead of parentheses<br>(&quot;(&quot; and &q=
uot;)&quot;), without any clear reason for the difference.=C2=A0 They<br>sh=
ould be switched to parentheses.</blockquote></div></div></div></div></div>=
</div></blockquote></span><div>done!=C2=A0</div><span class=3D"gmail-im"><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m_-5669=
81276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764gmail-=
h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><div clas=
s=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockquote =
type=3D"cite"><br><br>* In Section 2.2., the word &quot;for&quot; or &quot;=
at&quot; should be removed in the<br>sentence &quot;The change of behavior =
happens for at specific larger<br>packet sizes.&quot;</blockquote></div></d=
iv></div></div></div></div></blockquote></span><div>done!=C2=A0</div><span =
class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div c=
lass=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566=
981276781100764gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div d=
ir=3D"auto"><div class=3D"gmail-m_-566981276781100764gmail-m_86087447193946=
08261h5"><blockquote type=3D"cite"><br><br>* Section 2.3., the phrase &quot=
;the application commonly need to&quot; should<br>be changed to &quot;the a=
pplication commonly needs to&quot;.<br><br></blockquote></div></div></div><=
/div></div></div></blockquote></span><div>done!=C2=A0</div><span class=3D"g=
mail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gm=
ail-m_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-5669812767811=
00764gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto=
"><div class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><=
blockquote type=3D"cite">* In Section 5.2., there is a comma missing betwee=
n &quot;crystals&quot; and<br>&quot;or&quot; in &quot;The crystals or clock=
 modules, usually have a specific&quot;.</blockquote></div></div></div></di=
v></div></div></blockquote></span><div>done!=C2=A0</div><span class=3D"gmai=
l-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b=
order-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail=
-m_-566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-5669812767811007=
64gmail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><=
div class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blo=
ckquote type=3D"cite"><br><br>* Also in Section 5.2., the term &quot;device=
s under test&quot; is used<br>instead of DUTs the rest of the document seem=
s to use.</blockquote></div></div></div></div></div></div></blockquote></sp=
an><div>done!=C2=A0</div><span class=3D"gmail-im"><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20=
4,204);padding-left:1ex"><div class=3D"gmail-m_-566981276781100764gmail-HOE=
nZb"><div class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr"><d=
iv class=3D"gmail_quote"><div dir=3D"auto"><div class=3D"gmail-m_-566981276=
781100764gmail-m_8608744719394608261h5"><blockquote type=3D"cite"><br><br>*=
 Throughout Section 6.1., the acronyms &quot;cos&quot; and &quot;dcsp&quot;=
 should be<br>expanded on first use.</blockquote></div></div></div></div></=
div></div></blockquote></span><div>done!=C2=A0</div><span class=3D"gmail-im=
"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;borde=
r-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m_-=
566981276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764gm=
ail-h5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockqu=
ote type=3D"cite"><br><br>* In Section 6.1.1, the definition of &quot;Buffe=
r Size&quot; calls out using<br>some magnitude of bytes (B, KB, MB, GB), th=
en the example denotes<br>&quot;Mb&quot; -- which commonly is used for mega=
bits.=C2=A0 From the rest of the<br>definition, it seems this should be MB,=
 but I have not calculated<br>if the numerical value preceding the unit nee=
ds to be revisited.</blockquote></div></div></div></div></div></div></block=
quote></span><div>done!=C2=A0</div><span class=3D"gmail-im"><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m_-56698127678110076=
4gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=
=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><div class=3D"gmail-m=
_-566981276781100764gmail-m_8608744719394608261h5"><blockquote type=3D"cite=
"><br><br>* In Section 6.2.1, the Stateful and Stateless terms both use the=
<br>phrase &quot;such as for example&quot;, which is redundant.=C2=A0 Eithe=
r &quot;such as&quot;<br>or &quot;for example&quot; is sufficient, and the =
other should be removed.<br><br>* The formatting in Section 10. &quot;Refer=
ences&quot; is very inconsistent.<br>There is odd indentation and inconsist=
ent anchor usage.</blockquote></div></div></div></div></div></div></blockqu=
ote><div><br></div></span><div>fixed as much as i could, there is a problem=
 with nits in submission tool when =C2=A0</div><span class=3D"gmail-im"><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-lef=
t:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D"gmail-m_-56698=
1276781100764gmail-HOEnZb"><div class=3D"gmail-m_-566981276781100764gmail-h=
5"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"auto"><div class=
=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockquote t=
ype=3D"cite"><br><br>* The affiliation and email address for Lucien Avramov=
 is Google,<br>but the street address is mostly that for Cisco Systems, Inc=
.</blockquote></div></div></div></div></div></div></blockquote></span><div>=
done=C2=A0</div></div></div></div></div></div></div></div></div></div></div=
></div></div></div></div></div></td></tr></tbody></table></div></div></div>=
</div></div></div></div></div></div></div></div></div></div></div></div></d=
iv></div></div></div></div>

--94eb2c140dce380ba605520cccb4--


From nobody Fri Jun 16 19:27:16 2017
Return-Path: <nabeel@nuagenetworks.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F98812EAA7 for <bmwg@ietfa.amsl.com>; Fri, 16 Jun 2017 19:27:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nuagenetworks-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lyeTNRNIo4GM for <bmwg@ietfa.amsl.com>; Fri, 16 Jun 2017 19:27:12 -0700 (PDT)
Received: from mail-qt0-x22c.google.com (mail-qt0-x22c.google.com [IPv6:2607:f8b0:400d:c0d::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4263812E854 for <bmwg@ietf.org>; Fri, 16 Jun 2017 19:27:12 -0700 (PDT)
Received: by mail-qt0-x22c.google.com with SMTP id u12so82303966qth.0 for <bmwg@ietf.org>; Fri, 16 Jun 2017 19:27:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuagenetworks-net.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sCNZRwArPr4E2c0kbvO83RN+z7Eg0YjYB1AGoQgsztQ=; b=cfqT+DFfK4DNp6RO3l/BMloSLvXPRWxXvSQ4pa6sG2wzPkdAYDT0tRjqA60lIpIOa9 UWUThZc+hwZBgR3w6p6nXEslt64gd2LgPmMLpWguJzP5AHYp0Fmj9Ok4FntvcLCBqon9 Had70XKXBnUCl0yEU4zHhyJsYDZDvpUsWkBwKi2qJG+/jjHx5YqsKbN80KIRz3frdBpl kGOhVFa70UAd8aQDcL9DyH4YCOf9gX87bdL10Q6W5VEfsxCSJ37MkkqR0E0PST9kyOvb aRLwnRMjN60Bwhj2h8PHlWsJBj3cvAfN+1Pbbt+MntP9hiugNCy03NvX+n3Rx878Y2Dx JGZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sCNZRwArPr4E2c0kbvO83RN+z7Eg0YjYB1AGoQgsztQ=; b=sv39JftHhg0iI7U6aw28qrA4OVF5f5YWop2zEayrS4Np3uSNtWMYM0GDIaqf3z9/yU 0MpTdGcZ6MR+EZnBsrPNnN7xQm98A8zYMsAhyaAHMQzPRwbVFl81+cTBRxiGpHLDynYS uUwvLSp2hBgwedRXg1zmfYUjaN671UXUfOd7ewdRRz6kJDmTkkESBTQLt34gnE1VCi0z RDnRNhkOXlAEKVTwDC4qqdQzAlkz1rerfvFe2Y+NTl4mhsboeQuhdwCmQ5LqWJVKPuYs zz3u7czoGd+6akzQH8u5NQXKhr65KNUhcSIkb/oY62eXx4iizulM1L/ghLgLZN3BZ/Ja NeWg==
X-Gm-Message-State: AKS2vOwjDpgfgWkLZkUn43iguQKS1/TcE+s/ff77wPmZMcT5PFSCjpjp AhFY0OEhri4EDzkuxKmIbC1+
X-Received: by 10.200.40.46 with SMTP id 43mr16629727qtq.22.1497666431305; Fri, 16 Jun 2017 19:27:11 -0700 (PDT)
Received: from ?IPv6:2600:1017:b821:8be8:24ce:5b66:3046:a7c8? ([2600:1017:b821:8be8:24ce:5b66:3046:a7c8]) by smtp.gmail.com with ESMTPSA id l46sm2829885qtb.21.2017.06.16.19.27.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Jun 2017 19:27:10 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Nabeel Cocker <nabeel@nuagenetworks.net>
X-Mailer: iPhone Mail (14F89)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
Date: Fri, 16 Jun 2017 22:27:09 -0400
Cc: "bmwg@ietf.org" <bmwg@ietf.org>
Content-Transfer-Encoding: 7bit
Message-Id: <B93989E1-6CEA-4878-87BF-0BDF5FA0028C@nuagenetworks.net>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Nu5gd2Y_iZbphtlRckJ5zGaNqgc>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 17 Jun 2017 02:27:14 -0000

Support

Regards,
Nabeel

> On Jun 3, 2017, at 6:34 PM, MORTON, ALFRED C (AL) <acmorton@att.com> wrote:
> 
> BMWG,
> 
> At our Chicago meeting, the BMWG reviewed a version of
> the Internet Draft on
> 
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
> 
> There was sufficient readership present at the meeting
> and positive/constructive comments on the work, so the
> chairs asked whether support existed to adopt the draft
> as a WG item. There was support and no objections. As
> always, such decisions are officially taken on the 
> mailing list.
> 
> This message*, therefore, starts a call for adoption on:
> 
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
> 
> to run until Friday 23 June 2017. 
> 
> Please reply to bmwg@ietf.org indicating:
> 
> (1) whether you support addition of the following milestone to 
>   the BMWG charter:
> 
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
> 
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07 
>   as the basis document for this milestone
> 
> (3) whether you commit to reviewing the WG document if adopted
> 
> We are aware that BMWG has nearly completed all its existing
> chartered items.  This usually begins discussion of 
> re-chartering, this time with Warren (our new AD Advisor). The final
> decision/schedule for adoption will depend on these discussions. 
> 
> thanks for your consideration of the draft, and regards,
> 
> Al
> bmwg co-chair
> 
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
> 
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


From nobody Fri Jun 16 19:28:09 2017
Return-Path: <sob@sobco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0240612EAB9 for <bmwg@ietfa.amsl.com>; Fri, 16 Jun 2017 19:28:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.109
X-Spam-Level: 
X-Spam-Status: No, score=-1.109 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hqM_xTuwEKXJ for <bmwg@ietfa.amsl.com>; Fri, 16 Jun 2017 19:28:06 -0700 (PDT)
Received: from sobco.sobco.com (unknown [136.248.127.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 54A1212EABC for <bmwg@ietf.org>; Fri, 16 Jun 2017 19:28:04 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by sobco.sobco.com (Postfix) with ESMTP id 87BA843F2C69 for <bmwg@ietf.org>; Fri, 16 Jun 2017 22:28:01 -0400 (EDT)
X-Virus-Scanned: amavisd-new at sobco.com
Received: from sobco.sobco.com ([127.0.0.1]) by localhost (sobco.sobco.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JflRUXqruOna for <bmwg@ietf.org>; Fri, 16 Jun 2017 22:28:01 -0400 (EDT)
Received: from [172.20.6.214] (unknown [12.4.55.180]) by sobco.sobco.com (Postfix) with ESMTPSA id 106B043F2C5F for <bmwg@ietf.org>; Fri, 16 Jun 2017 22:28:00 -0400 (EDT)
From: Scott Bradner <sob@sobco.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Message-Id: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com>
Date: Fri, 16 Jun 2017 22:27:59 -0400
To: bmwg@ietf.org
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/IikFI6seTFFZDg8Gy0hlf-0PmGc>
Subject: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 17 Jun 2017 02:28:08 -0000

  IJ have been asked to do a OPS-DIR review of =
draft-ietf-bmwg-dcbench-methodology-07


I quickly For all tests, the percentage of traffic per port capacity =
sent MUST

   be 99.98% at most, with no PPM adjustment to ensure stressing the DUT

   in worst case conditions. Tests results at a lower rate MAY be

   provided for better understanding of performance increase in terms of

   latency and jitter when the rate is lower than 99.98%. The receiving

   rate of the traffic SHOULD be captured during this test in % of line

   rate.=


From nobody Fri Jun 16 19:33:14 2017
Return-Path: <sob@sobco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A0FBC12EAB1; Fri, 16 Jun 2017 19:33:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.109
X-Spam-Level: 
X-Spam-Status: No, score=-1.109 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lbRF_1KNS1WD; Fri, 16 Jun 2017 19:33:11 -0700 (PDT)
Received: from sobco.sobco.com (unknown [136.248.127.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 935E512EAA7; Fri, 16 Jun 2017 19:33:11 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by sobco.sobco.com (Postfix) with ESMTP id 91C1843F2DA7; Fri, 16 Jun 2017 22:33:10 -0400 (EDT)
X-Virus-Scanned: amavisd-new at sobco.com
Received: from sobco.sobco.com ([127.0.0.1]) by localhost (sobco.sobco.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ejhy-a-Yazld; Fri, 16 Jun 2017 22:33:10 -0400 (EDT)
Received: from [172.20.6.214] (unknown [12.4.55.180]) by sobco.sobco.com (Postfix) with ESMTPSA id E020243F2D9C; Fri, 16 Jun 2017 22:33:09 -0400 (EDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Scott Bradner <sob@sobco.com>
In-Reply-To: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com>
Date: Fri, 16 Jun 2017 22:33:07 -0400
Cc: draft-ietf-bmwg-dcbench-methodology.all@ietf.org, bmwg-chairs@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com>
To: bmwg@ietf.org
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/nwn7JHK3c0cdlbMnCW_x5GvtWOE>
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 17 Jun 2017 02:33:13 -0000

try again - my new mac book often sends mail before I want it to


I have been asked to do a OPS-DIR review of =
draft-ietf-bmwg-dcbench-methodology-07

I quickly became confused - a paragraph in section 2.2 reads

For all tests, the percentage of traffic per port capacity sent MUST
  be 99.98% at most, with no PPM adjustment to ensure stressing the DUT
  in worst case conditions. Tests results at a lower rate MAY be
  provided for better understanding of performance increase in terms of
  latency and jitter when the rate is lower than 99.98%. The receiving
  rate of the traffic SHOULD be captured during this test in % of line
  rate.


Thus reads to me like a conformance test not a performance test (saying =
what is =E2=80=9Cpassing=E2=80=9D)  unless
BMWG has morphed a lot since I was last involved conformance and =
pass/fail are not things that BMWG
should be doing -=20

what am I missing?

Scott



From nobody Sat Jun 17 10:49:31 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E6618128954 for <bmwg@ietfa.amsl.com>; Sat, 17 Jun 2017 10:49:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sHc62Bh3JN5m for <bmwg@ietfa.amsl.com>; Sat, 17 Jun 2017 10:49:28 -0700 (PDT)
Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F3FCD129B4D for <bmwg@ietf.org>; Sat, 17 Jun 2017 10:49:25 -0700 (PDT)
Received: by mail-pf0-x22b.google.com with SMTP id 83so35919172pfr.0 for <bmwg@ietf.org>; Sat, 17 Jun 2017 10:49:25 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DDUFLCocuJnrW5czPtw5DJuiYA5asusPxC4gKmJr6yI=; b=pHkeM87sgdPTlEtxjB/y/ooo60HUmGg5mCi7z9dFREOqjOUeKmPD0ulHrlM9u0TCLu cvAMHvbljhLpKRQPOrkQDyTYHVxdxNjl9303kykR+i1aw9Nlhmaa4J7LI8p507Vb9PGE AEgkclkeFXaYx31B19FcqKgcSSD/FoNR5VDe3SBO6WH77UZSgWhv7p2P63DqnhY0lzK7 TsucEBmzf4olxOgnVviXG4oe61HJshZY/Kqa566Fna0/K3hqFc7b4SYbOuOCivIkUix7 k+fPoqh0PF6AygGcE5LddTj3PUKZfdN7nDL/xm3CIe8u2v51K9B1E4NWYT3HGor5O5rt td2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DDUFLCocuJnrW5czPtw5DJuiYA5asusPxC4gKmJr6yI=; b=CycX2PQlcKaX7nCk69su6b2hHC5gEXcH8W0rkhhGYHcdaiPfN2J9sTqJTJoQ1JSulw W5lGX+YS6srGrvQgAJETuRIUpkAqTYIOcdJXSvvgpppH5nJsoBazWUkJrlzJhEHH7l6O 8A/FNe/nO8f0XwUHG3deGxv0mKsDBowBbpPlBEYPX3yAK7bWJiaU31gWxfyaW8NhyZbP Sh2vp55nLlefbN4YPRQkMpFx7ZbodgSy55qS9nubz2sZoi8i4nmmZ7bXs06vPbtFdtF4 6z3dU9IBXVngNLFnPtqsuS+wX700QQ3A6hg588+3WMAF4anwa2fQcAYCYkU2WTNcwlqI SGMA==
X-Gm-Message-State: AKS2vOzd5Aux7cExdLYD9AqmKnhjDNLQBJ/HLZjYzg9ciLVs73RAJ6qT oRaTTs/+ghqMQXTCZlKbqpdM3jPyvZVy
X-Received: by 10.99.106.2 with SMTP id f2mr18023564pgc.46.1497721764935; Sat, 17 Jun 2017 10:49:24 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Sat, 17 Jun 2017 10:49:04 -0700 (PDT)
In-Reply-To: <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com> <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com>
From: Lucien Avramov <lucienav@google.com>
Date: Sat, 17 Jun 2017 10:49:04 -0700
Message-ID: <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com>
To: Scott Bradner <sob@sobco.com>
Cc: bmwg@ietf.org, draft-ietf-bmwg-dcbench-methodology.all@ietf.org,  bmwg-chairs@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c13f244c602b105522b8542"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/M-u2b68aixAdzBwuzfo0hqcNThc>
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 17 Jun 2017 17:49:30 -0000

--94eb2c13f244c602b105522b8542
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Scott!

   - The draft you are reviewing is the ''*methodology*'' draft which
   explains the methodology for line rate performance testing. The exact
   reason onto why it is 99.98% with 0 PPM adjustment is covered extensivel=
y
   in the *terminology draft*, section 5: https://tools.ietf.org/html
   /draft-ietf-bmwg-dcbench-terminology-12#section-5
   <https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-12#sect=
ion-5>.
   That draft dependency is covered in the introduction and also in the
   reference lists and is referenced throughout the document
   - There is a clock drift difference between tester and DUT explained in
   section 5.2 of the *terminology draft*

In our view BMWG has not morphed, our goal has been to be very precise and
accurate with performance testing, and this is not about conformance
testing at all. These definitions that you see, have not changed in 4 years=
 and
have been proven themselves in a very large amount of testing done since we
started publishing.

As you may remember, we have been working on this draft actively for over 4
years now. We've incorporated numerous of your feedbacks on these drafts
over the IETF sessions (since IETF 86 - captured in the meeting minutes of
the IETF sessions).  On April 2016, we had your blessing on the list also
regarding our drafts.

We appreciate as always your feedback, Scott!

Lucien


On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner <sob@sobco.com> wrote:

> try again - my new mac book often sends mail before I want it to
>
>
> I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcbench-
> methodology-07
>
> I quickly became confused - a paragraph in section 2.2 reads
>
> For all tests, the percentage of traffic per port capacity sent MUST
>   be 99.98% at most, with no PPM adjustment to ensure stressing the DUT
>   in worst case conditions. Tests results at a lower rate MAY be
>   provided for better understanding of performance increase in terms of
>   latency and jitter when the rate is lower than 99.98%. The receiving
>   rate of the traffic SHOULD be captured during this test in % of line
>   rate.
>
>
> Thus reads to me like a conformance test not a performance test (saying
> what is =E2=80=9Cpassing=E2=80=9D)  unless
> BMWG has morphed a lot since I was last involved conformance and pass/fai=
l
> are not things that BMWG
> should be doing -
>
> what am I missing?
>
> Scott
>
>
>

--94eb2c13f244c602b105522b8542
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Scott!<div><div style=3D"font-size:12.8px"><ul><li styl=
e=3D"margin-left:15px">The draft you are reviewing is the &#39;&#39;<b>meth=
odology</b>&#39;&#39; draft which explains the methodology for line rate pe=
rformance testing. The exact reason onto why it is 99.98% with 0 PPM adjust=
ment is covered extensively in the=C2=A0<b>terminology draft</b>, section 5=
:=C2=A0<a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-termi=
nology-12#section-5" target=3D"_blank">https://tools.ietf.org/html<wbr>/dra=
ft-ietf-bmwg-dcbench-termi<wbr>nology-12#section-5</a>. That draft dependen=
cy is covered in the introduction and also in the reference lists and is re=
ferenced throughout the document</li><li style=3D"margin-left:15px">There i=
s a clock drift difference between tester and DUT explained in section 5.2 =
of the=C2=A0<b>terminology draft</b></li></ul><div>In our view BMWG has not=
 morphed, our goal has been to be very precise and accurate with performanc=
e testing, and this is not about conformance testing at all. These definiti=
ons that you see, have not changed=C2=A0<span class=3D"gmail-aBn" tabindex=
=3D"0"><span class=3D"gmail-aQJ">in 4 years</span></span>=C2=A0and have bee=
n proven themselves in a very large amount of testing done since we started=
 publishing.</div></div><div style=3D"font-size:12.8px"><br></div><div styl=
e=3D"font-size:12.8px">As you may remember, we have been working on this dr=
aft actively for over 4 years now. We&#39;ve incorporated numerous of your =
feedbacks on these drafts over the IETF sessions (since IETF 86 - captured =
in the meeting minutes of the IETF sessions). =C2=A0<span style=3D"font-siz=
e:12.8px">On April 2016, we had your blessing on the list also regarding ou=
r drafts.</span></div><div style=3D"font-size:12.8px"><br></div><div style=
=3D"font-size:12.8px">We appreciate as always your feedback, Scott!</div></=
div><div style=3D"font-size:12.8px"><br></div><div style=3D"font-size:12.8p=
x">Lucien</div><div style=3D"font-size:12.8px"><br></div><div class=3D"gmai=
l_extra"><br><div class=3D"gmail_quote">On Fri, Jun 16, 2017 at 7:33 PM, Sc=
ott Bradner <span dir=3D"ltr">&lt;<a href=3D"mailto:sob@sobco.com" target=
=3D"_blank">sob@sobco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex">try again - my new mac book often sends mail before I want it to<br>
<br>
<br>
I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcbench-<wbr>me=
thodology-07<br>
<br>
I quickly became confused - a paragraph in section 2.2 reads<br>
<span class=3D""><br>
For all tests, the percentage of traffic per port capacity sent MUST<br>
=C2=A0 be 99.98% at most, with no PPM adjustment to ensure stressing the DU=
T<br>
=C2=A0 in worst case conditions. Tests results at a lower rate MAY be<br>
=C2=A0 provided for better understanding of performance increase in terms o=
f<br>
=C2=A0 latency and jitter when the rate is lower than 99.98%. The receiving=
<br>
=C2=A0 rate of the traffic SHOULD be captured during this test in % of line=
<br>
=C2=A0 rate.<br>
<br>
<br>
</span>Thus reads to me like a conformance test not a performance test (say=
ing what is =E2=80=9Cpassing=E2=80=9D)=C2=A0 unless<br>
BMWG has morphed a lot since I was last involved conformance and pass/fail =
are not things that BMWG<br>
should be doing -<br>
<br>
what am I missing?<br>
<span class=3D"HOEnZb"><font color=3D"#888888"><br>
Scott<br>
<br>
<br>
</font></span></blockquote></div><br></div></div>

--94eb2c13f244c602b105522b8542--


From nobody Sat Jun 17 22:04:43 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 483791296B0; Sat, 17 Jun 2017 22:04:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.411
X-Spam-Level: 
X-Spam-Status: No, score=-3.411 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JG7xfqU9w2Ce; Sat, 17 Jun 2017 22:04:38 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 89C10129649; Sat, 17 Jun 2017 22:04:38 -0700 (PDT)
Received: from pps.filterd (m0049458.ppops.net [127.0.0.1]) by m0049458.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v5I4trrO048865; Sun, 18 Jun 2017 01:04:30 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049458.ppops.net-00191d01. with ESMTP id 2b4yvk1j4m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 18 Jun 2017 01:04:30 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5I54TSj109744; Sun, 18 Jun 2017 00:04:29 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5I54MpQ109700 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 18 Jun 2017 00:04:22 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor); Sun, 18 Jun 2017 05:04:11 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5I54B1s024935; Sun, 18 Jun 2017 00:04:11 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5I540eh024133; Sun, 18 Jun 2017 00:04:00 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id 62D3EE3BF7; Sun, 18 Jun 2017 01:03:50 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Sun, 18 Jun 2017 01:03:59 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Lucien Avramov <lucienav@google.com>, Scott Bradner <sob@sobco.com>
CC: "bmwg@ietf.org" <bmwg@ietf.org>, "draft-ietf-bmwg-dcbench-methodology.all@ietf.org" <draft-ietf-bmwg-dcbench-methodology.all@ietf.org>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>
Thread-Topic: draft-ietf-bmwg-dcbench-methodology-07
Thread-Index: AQHS55IT+1LmjVZvaUuikgc0tDhWFKIpopIQ
Date: Sun, 18 Jun 2017 05:03:58 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FDF58D@njmtexg5.research.att.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com> <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com> <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com>
In-Reply-To: <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [24.136.24.70]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF25FDF58Dnjmtexg5researc_"
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-18_03:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706180085
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2Kf4jlFoR0lIFQtyU6pRistZHso>
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 05:04:41 -0000

--_000_4D7F4AD313D3FC43A053B309F97543CF25FDF58Dnjmtexg5researc_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SGkgTHVjaWVuLCBTY290dCwgYW5kIGFsbCwNCg0KV2hlbiBJIHJlYWQgdGhpcyBtYXRlcmlhbCwg
SSBhc3N1bWVkIHRoZSBnb2FsIHdhcyB0bw0KbGltaXQgdGhlIG1heGltdW0gc2VuZGluZyByYXRl
IG9mIHRoZSB0ZXN0IHRyYWZmaWMgZ2VuZXJhdG9yDQppbiBhY2NvcmRhbmNlIHdpdGggdGhlIEV0
aGVybmV0IGNsb2NrIHRvbGVyYW5jZXMsDQplZmZlY3RpdmVseSBjaG9vc2luZyB0aGUgbG93LWVu
ZCBvZiBjbG9jayB0b2xlcmFuY2UNCmFzIHRoZSBtYXhpbXVtIHNlbmRpbmcgcmF0ZSB3aXRoIGEg
ZmFjdG9yIG9mIDIgbWFyZ2luLg0KOTkuOTglIG9mIHRoZSBub21pbmFsIGxpbmUgcmF0ZSB3b3Vs
ZCBiZSAtMjAwIHBwbSwNCmFuZCBhbnkgY29uZm9ybWluZyBpbnRlcmZhY2UgY2xvY2sgc2hvdWxk
IHdvcmsgZmluZS4NCg0KSWYgSSB1bmRlcnN0b29kIHlvdXIgZ29hbCBjb3JyZWN0bHksDQpJIHdv
bmRlciBpZiBhIHNtYWxsIGNsYXJpZmljYXRpb24gd2lsbCBoZWxwIGZ1dHVyZQ0KcmVhZGVycy4N
Cg0KV2hlbiBzZWN0aW9uIDIuMiBzYXlzOg0KICAgRm9yIGFsbCB0ZXN0cywgdGhlIHBlcmNlbnRh
Z2Ugb2YgdHJhZmZpYyBwZXIgcG9ydCBjYXBhY2l0eSBzZW50IE1VU1QNCiAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5eXl5eXl5eXl5eXl5eXl5eXg0KICAg
YmUgOTkuOTglIGF0IG1vc3QsIHdpdGggbm8gUFBNIGFkanVzdG1lbnQgdG8gZW5zdXJlIHN0cmVz
c2luZyB0aGUgRFVUDQogICAgICArKysrKysrKysrKysrKysNCiAgIGluIHdvcnN0IGNhc2UgY29u
ZGl0aW9ucy4NCg0KWW914oCZdmUgZGVmaW5lZCB0aGUgY29uY2VwdCBvZiBQb3J0IENhcGFjaXR5
IGluIHRlcm1pbm9sb2d5IFNlY3Rpb24gNSwNCnVuZGVyIHRoZSBMaW5lIFJhdGUgZGVmaW5pdGlv
biwgYXMNCg0KICAgVGhlIHRlcm0gInBvcnQgY2FwYWNpdHkiIGRlZmluZXMgdGhlIG1heGltdW0g
c3BlZWQgY2FwYWJpbGl0eSBmb3IgdGhlDQogICBnaXZlbiBwb3J0OyBmb3IgZXhhbXBsZSAxR0Us
IDEwR0UsIDQwR0UsIDEwMEdFIGV0Yy4NCg0KQWN0dWFsbHksIHRoZSBwb3J0IGNhcGFjaXR5IGV4
YW1wbGVzIGFyZSB0aGUgKm5vbWluYWwgdmFsdWUgb2YgTGluZSBSYXRlKiwNCmJlZm9yZSB0aGUg
Y2xvY2sgdG9sZXJhbmNlcyBhcmUgY29uc2lkZXJlZC4NCg0KSGVyZeKAmXMgYSBzdWdnZXN0aW9u
IGZvciBTZWN0aW9uIDIuMiBvZiB0aGUgbWV0aDoNCg0KICAgRm9yIGFsbCB0ZXN0cywgdGhlIHRl
c3QgdHJhZmZpYyBnZW5lcmF0b3Igc2VuZGluZyByYXRlIE1VU1QgYmUNCiAgIGxlc3MgdGhhbiBv
ciBlcXVhbCB0byA5OS45OCUgb2YgdGhlIG5vbWluYWwgdmFsdWUgb2YgTGluZSBSYXRlDQogICAo
d2l0aCBubyBmdXJ0aGVyIFBQTSBhZGp1c3RtZW50IHRvIGFjY291bnQgZm9yIGludGVyZmFjZSBj
bG9jaw0KICAgdG9sZXJhbmNlcyksIHRvIGVuc3VyZSBzdHJlc3NpbmcgdGhlIERVVCBpbiByZWFz
b25hYmxlIHdvcnN0IGNhc2UNCiAgIGNvbmRpdGlvbnMuDQoNCkNoYW5naW5nDQpzL3BvcnQgY2Fw
YWNpdHkvbm9taW5hbCB2YWx1ZSBvZiBMaW5lIFJhdGUvDQpoZXJlLCByZXF1aXJlcyBhbiBhZGRp
dGlvbmFsIHN1YnN0aXR1dGlvbiBpbiBzZWN0aW9uIDIuMyBvZiB0aGUgbWV0aCBkcmFmdCwNCmFu
ZCBhIHNpbmdsZSBzdWJzdGl0dXRpb24gaW4gc2VjdGlvbiA1LjEgb2YgdGhlIHRlcm0gZHJhZnQu
DQoNCkRvZXMgdGhpcyBoZWxwPw0KQWwNCg0KDQpGcm9tOiBMdWNpZW4gQXZyYW1vdiBbbWFpbHRv
Omx1Y2llbmF2QGdvb2dsZS5jb21dDQpTZW50OiBTYXR1cmRheSwgSnVuZSAxNywgMjAxNyAxOjQ5
IFBNDQpUbzogU2NvdHQgQnJhZG5lcg0KQ2M6IGJtd2dAaWV0Zi5vcmc7IGRyYWZ0LWlldGYtYm13
Zy1kY2JlbmNoLW1ldGhvZG9sb2d5LmFsbEBpZXRmLm9yZzsgYm13Zy1jaGFpcnNAaWV0Zi5vcmcN
ClN1YmplY3Q6IFJlOiBkcmFmdC1pZXRmLWJtd2ctZGNiZW5jaC1tZXRob2RvbG9neS0wNw0KDQpI
aSBTY290dCENCsK3ICAgICAgICAgVGhlIGRyYWZ0IHlvdSBhcmUgcmV2aWV3aW5nIGlzIHRoZSAn
J21ldGhvZG9sb2d5JycgZHJhZnQgd2hpY2ggZXhwbGFpbnMgdGhlIG1ldGhvZG9sb2d5IGZvciBs
aW5lIHJhdGUgcGVyZm9ybWFuY2UgdGVzdGluZy4gVGhlIGV4YWN0IHJlYXNvbiBvbnRvIHdoeSBp
dCBpcyA5OS45OCUgd2l0aCAwIFBQTSBhZGp1c3RtZW50IGlzIGNvdmVyZWQgZXh0ZW5zaXZlbHkg
aW4gdGhlIHRlcm1pbm9sb2d5IGRyYWZ0LCBzZWN0aW9uIDU6IGh0dHBzOi8vdG9vbHMuaWV0Zi5v
cmcvaHRtbC9kcmFmdC1pZXRmLWJtd2ctZGNiZW5jaC10ZXJtaW5vbG9neS0xMiNzZWN0aW9uLTU8
aHR0cHM6Ly91cmxkZWZlbnNlLnByb29mcG9pbnQuY29tL3YyL3VybD91PWh0dHBzLTNBX190b29s
cy5pZXRmLm9yZ19odG1sX2RyYWZ0LTJEaWV0Zi0yRGJtd2ctMkRkY2JlbmNoLTJEdGVybWlub2xv
Z3ktMkQxMi0yM3NlY3Rpb24tMkQ1JmQ9RHdNRmFRJmM9TEZZWi1vOV9IVU1lTVRTUWljdmpJZyZy
PU9mc1N1OGtUSWx0VnlEMW9MNzJjQncmbT1YVFNZS01wR2xHLWRyeHUtQ1ZxRGhUTng4WWNWV2RN
bXlwT0FJU2tuTHZjJnM9VWtYY21ydUVzbHdobDdDY0poT0ZFUnpfMjdDWlNmQWZSLTBXSHFabDEz
ZyZlPT4uIFRoYXQgZHJhZnQgZGVwZW5kZW5jeSBpcyBjb3ZlcmVkIGluIHRoZSBpbnRyb2R1Y3Rp
b24gYW5kIGFsc28gaW4gdGhlIHJlZmVyZW5jZSBsaXN0cyBhbmQgaXMgcmVmZXJlbmNlZCB0aHJv
dWdob3V0IHRoZSBkb2N1bWVudA0KwrcgICAgICAgICBUaGVyZSBpcyBhIGNsb2NrIGRyaWZ0IGRp
ZmZlcmVuY2UgYmV0d2VlbiB0ZXN0ZXIgYW5kIERVVCBleHBsYWluZWQgaW4gc2VjdGlvbiA1LjIg
b2YgdGhlIHRlcm1pbm9sb2d5IGRyYWZ0DQpJbiBvdXIgdmlldyBCTVdHIGhhcyBub3QgbW9ycGhl
ZCwgb3VyIGdvYWwgaGFzIGJlZW4gdG8gYmUgdmVyeSBwcmVjaXNlIGFuZCBhY2N1cmF0ZSB3aXRo
IHBlcmZvcm1hbmNlIHRlc3RpbmcsIGFuZCB0aGlzIGlzIG5vdCBhYm91dCBjb25mb3JtYW5jZSB0
ZXN0aW5nIGF0IGFsbC4gVGhlc2UgZGVmaW5pdGlvbnMgdGhhdCB5b3Ugc2VlLCBoYXZlIG5vdCBj
aGFuZ2VkIGluIDQgeWVhcnMgYW5kIGhhdmUgYmVlbiBwcm92ZW4gdGhlbXNlbHZlcyBpbiBhIHZl
cnkgbGFyZ2UgYW1vdW50IG9mIHRlc3RpbmcgZG9uZSBzaW5jZSB3ZSBzdGFydGVkIHB1Ymxpc2hp
bmcuDQoNCkFzIHlvdSBtYXkgcmVtZW1iZXIsIHdlIGhhdmUgYmVlbiB3b3JraW5nIG9uIHRoaXMg
ZHJhZnQgYWN0aXZlbHkgZm9yIG92ZXIgNCB5ZWFycyBub3cuIFdlJ3ZlIGluY29ycG9yYXRlZCBu
dW1lcm91cyBvZiB5b3VyIGZlZWRiYWNrcyBvbiB0aGVzZSBkcmFmdHMgb3ZlciB0aGUgSUVURiBz
ZXNzaW9ucyAoc2luY2UgSUVURiA4NiAtIGNhcHR1cmVkIGluIHRoZSBtZWV0aW5nIG1pbnV0ZXMg
b2YgdGhlIElFVEYgc2Vzc2lvbnMpLiAgT24gQXByaWwgMjAxNiwgd2UgaGFkIHlvdXIgYmxlc3Np
bmcgb24gdGhlIGxpc3QgYWxzbyByZWdhcmRpbmcgb3VyIGRyYWZ0cy4NCg0KV2UgYXBwcmVjaWF0
ZSBhcyBhbHdheXMgeW91ciBmZWVkYmFjaywgU2NvdHQhDQoNCkx1Y2llbg0KDQoNCk9uIEZyaSwg
SnVuIDE2LCAyMDE3IGF0IDc6MzMgUE0sIFNjb3R0IEJyYWRuZXIgPHNvYkBzb2Jjby5jb208bWFp
bHRvOnNvYkBzb2Jjby5jb20+PiB3cm90ZToNCnRyeSBhZ2FpbiAtIG15IG5ldyBtYWMgYm9vayBv
ZnRlbiBzZW5kcyBtYWlsIGJlZm9yZSBJIHdhbnQgaXQgdG8NCg0KDQpJIGhhdmUgYmVlbiBhc2tl
ZCB0byBkbyBhIE9QUy1ESVIgcmV2aWV3IG9mIGRyYWZ0LWlldGYtYm13Zy1kY2JlbmNoLW1ldGhv
ZG9sb2d5LTA3DQoNCkkgcXVpY2tseSBiZWNhbWUgY29uZnVzZWQgLSBhIHBhcmFncmFwaCBpbiBz
ZWN0aW9uIDIuMiByZWFkcw0KDQpGb3IgYWxsIHRlc3RzLCB0aGUgcGVyY2VudGFnZSBvZiB0cmFm
ZmljIHBlciBwb3J0IGNhcGFjaXR5IHNlbnQgTVVTVA0KICBiZSA5OS45OCUgYXQgbW9zdCwgd2l0
aCBubyBQUE0gYWRqdXN0bWVudCB0byBlbnN1cmUgc3RyZXNzaW5nIHRoZSBEVVQNCiAgaW4gd29y
c3QgY2FzZSBjb25kaXRpb25zLiBUZXN0cyByZXN1bHRzIGF0IGEgbG93ZXIgcmF0ZSBNQVkgYmUN
CiAgcHJvdmlkZWQgZm9yIGJldHRlciB1bmRlcnN0YW5kaW5nIG9mIHBlcmZvcm1hbmNlIGluY3Jl
YXNlIGluIHRlcm1zIG9mDQogIGxhdGVuY3kgYW5kIGppdHRlciB3aGVuIHRoZSByYXRlIGlzIGxv
d2VyIHRoYW4gOTkuOTglLiBUaGUgcmVjZWl2aW5nDQogIHJhdGUgb2YgdGhlIHRyYWZmaWMgU0hP
VUxEIGJlIGNhcHR1cmVkIGR1cmluZyB0aGlzIHRlc3QgaW4gJSBvZiBsaW5lDQogIHJhdGUuDQoN
Cg0KVGh1cyByZWFkcyB0byBtZSBsaWtlIGEgY29uZm9ybWFuY2UgdGVzdCBub3QgYSBwZXJmb3Jt
YW5jZSB0ZXN0IChzYXlpbmcgd2hhdCBpcyDigJxwYXNzaW5n4oCdKSAgdW5sZXNzDQpCTVdHIGhh
cyBtb3JwaGVkIGEgbG90IHNpbmNlIEkgd2FzIGxhc3QgaW52b2x2ZWQgY29uZm9ybWFuY2UgYW5k
IHBhc3MvZmFpbCBhcmUgbm90IHRoaW5ncyB0aGF0IEJNV0cNCnNob3VsZCBiZSBkb2luZyAtDQoN
CndoYXQgYW0gSSBtaXNzaW5nPw0KDQpTY290dA0KDQoNCg==

--_000_4D7F4AD313D3FC43A053B309F97543CF25FDF58Dnjmtexg5researc_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTQgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl
PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6
V2luZ2RpbmdzOw0KCXBhbm9zZS0xOjUgMCAwIDAgMCAwIDAgMCAwIDA7fQ0KQGZvbnQtZmFjZQ0K
CXtmb250LWZhbWlseTpXaW5nZGluZ3M7DQoJcGFub3NlLTE6NSAwIDAgMCAwIDAgMCAwIDAgMDt9
DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIg
MiAyIDQgMyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpUYWhvbWE7DQoJcGFub3Nl
LTE6MiAxMSA2IDQgMyA1IDQgNCAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNv
Tm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsNCgltYXJn
aW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGlt
ZXMgTmV3IFJvbWFuIiwic2VyaWYiO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNv
LXN0eWxlLXByaW9yaXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVy
bGluZTt9DQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxl
LXByaW9yaXR5Ojk5Ow0KCWNvbG9yOnB1cnBsZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5l
O30NCnNwYW4uZ21haWwtYWJuDQoJe21zby1zdHlsZS1uYW1lOmdtYWlsLWFibjt9DQpzcGFuLmdt
YWlsLWFxag0KCXttc28tc3R5bGUtbmFtZTpnbWFpbC1hcWo7fQ0Kc3Bhbi5ob2VuemINCgl7bXNv
LXN0eWxlLW5hbWU6aG9lbnpiO30NCnNwYW4uRW1haWxTdHlsZTIwDQoJe21zby1zdHlsZS10eXBl
OnBlcnNvbmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7DQoJY29sb3I6Ymxh
Y2s7fQ0KLk1zb0NocERlZmF1bHQNCgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7DQoJZm9u
dC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjt9DQpAcGFnZSBXb3JkU2VjdGlvbjENCgl7
c2l6ZTo4LjVpbiAxMS4waW47DQoJbWFyZ2luOjEuMGluIDEuMGluIDEuMGluIDEuMGluO30NCmRp
di5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0KLyogTGlzdCBEZWZpbml0aW9u
cyAqLw0KQGxpc3QgbDANCgl7bXNvLWxpc3QtaWQ6MTg2MjQyNzQ4OTsNCgltc28tbGlzdC10ZW1w
bGF0ZS1pZHM6LTE1MjIxMzc5NDY7fQ0KQGxpc3QgbDA6bGV2ZWwxDQoJe21zby1sZXZlbC1udW1i
ZXItZm9ybWF0OmJ1bGxldDsNCgltc28tbGV2ZWwtdGV4dDrvgrc7DQoJbXNvLWxldmVsLXRhYi1z
dG9wOi41aW47DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50
Oi0uMjVpbjsNCgltc28tYW5zaS1mb250LXNpemU6MTAuMHB0Ow0KCWZvbnQtZmFtaWx5OlN5bWJv
bDt9DQpAbGlzdCBsMDpsZXZlbDINCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0Ow0K
CW1zby1sZXZlbC10ZXh0Om87DQoJbXNvLWxldmVsLXRhYi1zdG9wOjEuMGluOw0KCW1zby1sZXZl
bC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDotLjI1aW47DQoJbXNvLWFuc2kt
Zm9udC1zaXplOjEwLjBwdDsNCglmb250LWZhbWlseToiQ291cmllciBOZXciOw0KCW1zby1iaWRp
LWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iO30NCkBsaXN0IGwwOmxldmVsMw0KCXttc28t
bGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674KnOw0KCW1zby1s
ZXZlbC10YWItc3RvcDoxLjVpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJ
dGV4dC1pbmRlbnQ6LS4yNWluOw0KCW1zby1hbnNpLWZvbnQtc2l6ZToxMC4wcHQ7DQoJZm9udC1m
YW1pbHk6V2luZ2RpbmdzO30NCkBsaXN0IGwwOmxldmVsNA0KCXttc28tbGV2ZWwtbnVtYmVyLWZv
cm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674KnOw0KCW1zby1sZXZlbC10YWItc3RvcDoy
LjBpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4y
NWluOw0KCW1zby1hbnNpLWZvbnQtc2l6ZToxMC4wcHQ7DQoJZm9udC1mYW1pbHk6V2luZ2Rpbmdz
O30NCkBsaXN0IGwwOmxldmVsNQ0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJ
bXNvLWxldmVsLXRleHQ674KnOw0KCW1zby1sZXZlbC10YWItc3RvcDoyLjVpbjsNCgltc28tbGV2
ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluOw0KCW1zby1hbnNp
LWZvbnQtc2l6ZToxMC4wcHQ7DQoJZm9udC1mYW1pbHk6V2luZ2RpbmdzO30NCkBsaXN0IGwwOmxl
dmVsNg0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6
74KnOw0KCW1zby1sZXZlbC10YWItc3RvcDozLjBpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0
aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluOw0KCW1zby1hbnNpLWZvbnQtc2l6ZToxMC4w
cHQ7DQoJZm9udC1mYW1pbHk6V2luZ2RpbmdzO30NCkBsaXN0IGwwOmxldmVsNw0KCXttc28tbGV2
ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674KnOw0KCW1zby1sZXZl
bC10YWItc3RvcDozLjVpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4
dC1pbmRlbnQ6LS4yNWluOw0KCW1zby1hbnNpLWZvbnQtc2l6ZToxMC4wcHQ7DQoJZm9udC1mYW1p
bHk6V2luZ2RpbmdzO30NCkBsaXN0IGwwOmxldmVsOA0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1h
dDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674KnOw0KCW1zby1sZXZlbC10YWItc3RvcDo0LjBp
bjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWlu
Ow0KCW1zby1hbnNpLWZvbnQtc2l6ZToxMC4wcHQ7DQoJZm9udC1mYW1pbHk6V2luZ2RpbmdzO30N
CkBsaXN0IGwwOmxldmVsOQ0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNv
LWxldmVsLXRleHQ674KnOw0KCW1zby1sZXZlbC10YWItc3RvcDo0LjVpbjsNCgltc28tbGV2ZWwt
bnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluOw0KCW1zby1hbnNpLWZv
bnQtc2l6ZToxMC4wcHQ7DQoJZm9udC1mYW1pbHk6V2luZ2RpbmdzO30NCm9sDQoJe21hcmdpbi1i
b3R0b206MGluO30NCnVsDQoJe21hcmdpbi1ib3R0b206MGluO30NCi0tPjwvc3R5bGU+PCEtLVtp
ZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWRlZmF1bHRzIHY6ZXh0PSJlZGl0IiBzcGlkbWF4
PSIxMDI2IiAvPg0KPC94bWw+PCFbZW5kaWZdLS0+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+DQo8
bzpzaGFwZWxheW91dCB2OmV4dD0iZWRpdCI+DQo8bzppZG1hcCB2OmV4dD0iZWRpdCIgZGF0YT0i
MSIgLz4NCjwvbzpzaGFwZWxheW91dD48L3htbD48IVtlbmRpZl0tLT4NCjwvaGVhZD4NCjxib2R5
IGxhbmc9IkVOLVVTIiBsaW5rPSJibHVlIiB2bGluaz0icHVycGxlIj4NCjxkaXYgY2xhc3M9Ildv
cmRTZWN0aW9uMSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXpl
OjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+
SGkgTHVjaWVuLCBTY290dCwgYW5kIGFsbCw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom
cXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3Nw
YW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4w
cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPldoZW4g
SSByZWFkIHRoaXMgbWF0ZXJpYWwsIEkgYXNzdW1lZCB0aGUgZ29hbCB3YXMgdG8NCjxvOnA+PC9v
OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNp
emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNr
Ij5saW1pdCB0aGUgbWF4aW11bSBzZW5kaW5nIHJhdGUgb2YgdGhlIHRlc3QgdHJhZmZpYyBnZW5l
cmF0b3I8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBz
dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90
Oztjb2xvcjpibGFjayI+aW4gYWNjb3JkYW5jZSB3aXRoIHRoZSBFdGhlcm5ldCBjbG9jayB0b2xl
cmFuY2VzLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFu
IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1
b3Q7O2NvbG9yOmJsYWNrIj5lZmZlY3RpdmVseSBjaG9vc2luZyB0aGUgbG93LWVuZCBvZiBjbG9j
ayB0b2xlcmFuY2U8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48
c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5l
dyZxdW90Oztjb2xvcjpibGFjayI+YXMgdGhlIG1heGltdW0gc2VuZGluZyByYXRlIHdpdGggYSBm
YWN0b3Igb2YgMiBtYXJnaW4uJm5ic3A7DQo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom
cXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+OTkuOTglIG9mIHRoZSBub21pbmFs
IGxpbmUgcmF0ZSB3b3VsZCBiZSAtMjAwIHBwbSw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls
eTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+YW5kIGFueSBjb25mb3JtaW5n
IGludGVyZmFjZSBjbG9jayBzaG91bGQgd29yayBmaW5lLjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQt
ZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj48bzpwPiZuYnNwOzwv
bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFj
ayI+SWYgSSB1bmRlcnN0b29kIHlvdXIgZ29hbCBjb3JyZWN0bHksDQo8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBw
dDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+SSB3b25k
ZXIgaWYgYSBzbWFsbCBjbGFyaWZpY2F0aW9uIHdpbGwgaGVscCBmdXR1cmUNCjxvOnA+PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj5y
ZWFkZXJzLjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFu
IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1
b3Q7O2NvbG9yOmJsYWNrIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVv
dDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+V2hlbiBzZWN0aW9uIDIuMiBzYXlzOjxv
OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9y
OmJsYWNrIj4mbmJzcDsmbmJzcDsgRm9yIGFsbCB0ZXN0cywgdGhlIHBlcmNlbnRhZ2Ugb2YgdHJh
ZmZpYyBwZXIgcG9ydCBjYXBhY2l0eSBzZW50IE1VU1Q8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8
cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZh
bWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+Jm5ic3A7Jm5ic3A7Jm5i
c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7
Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i
c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7
Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i
c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IF5eXl5eXl5eXl5eXl5eXl5e
XjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxl
PSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2Nv
bG9yOmJsYWNrIj4mbmJzcDsmbmJzcDsgYmUgOTkuOTglIGF0IG1vc3QsIHdpdGggbm8gUFBNIGFk
anVzdG1lbnQgdG8gZW5zdXJlIHN0cmVzc2luZyB0aGUgRFVUPG86cD48L286cD48L3NwYW4+PC9w
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9u
dC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPiZuYnNwOyZuYnNw
OyZuYnNwOyZuYnNwOyZuYnNwOyAmIzQzOyYjNDM7JiM0MzsmIzQzOyYjNDM7JiM0MzsmIzQzOyYj
NDM7JiM0MzsmIzQzOyYjNDM7JiM0MzsmIzQzOyYjNDM7JiM0Mzs8bzpwPjwvbzpwPjwvc3Bhbj48
L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm
b250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+Jm5ic3A7Jm5i
c3A7IGluIHdvcnN0IGNhc2UgY29uZGl0aW9ucy48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls
eTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox
MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPllv
deKAmXZlIGRlZmluZWQgdGhlIGNvbmNlcHQgb2YgUG9ydCBDYXBhY2l0eSBpbiB0ZXJtaW5vbG9n
eSBTZWN0aW9uIDUsPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBO
ZXcmcXVvdDs7Y29sb3I6YmxhY2siPnVuZGVyIHRoZSBMaW5lIFJhdGUgZGVmaW5pdGlvbiwgYXM8
bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i
Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xv
cjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmll
ciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPiZuYnNwOyZuYnNwOyBUaGUgdGVybSAmcXVvdDtwb3J0
IGNhcGFjaXR5JnF1b3Q7IGRlZmluZXMgdGhlIG1heGltdW0gc3BlZWQgY2FwYWJpbGl0eSBmb3Ig
dGhlPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7
Y29sb3I6YmxhY2siPiZuYnNwOyZuYnNwOyBnaXZlbiBwb3J0OyBmb3IgZXhhbXBsZSAxR0UsIDEw
R0UsIDQwR0UsIDEwMEdFIGV0Yy48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD
b3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9w
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9u
dC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPkFjdHVhbGx5LCB0
aGUgcG9ydCBjYXBhY2l0eSBleGFtcGxlcyBhcmUgdGhlICo8Yj5ub21pbmFsIHZhbHVlIG9mIExp
bmUgUmF0ZTwvYj4qLA0KPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmll
ciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPmJlZm9yZSB0aGUgY2xvY2sgdG9sZXJhbmNlcyBhcmUg
Y29uc2lkZXJlZC48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48
c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5l
dyZxdW90Oztjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6
JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPkhlcmXigJlzIGEgc3VnZ2VzdGlv
biBmb3IgU2VjdGlvbiAyLjIgb2YgdGhlIG1ldGg6PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p
bHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPjxvOnA+Jm5ic3A7PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj4m
bmJzcDsmbmJzcDsgRm9yIGFsbCB0ZXN0cywgdGhlIHRlc3QgdHJhZmZpYyBnZW5lcmF0b3Igc2Vu
ZGluZyByYXRlIE1VU1QgYmUNCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0Nv
dXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj4mbmJzcDsmbmJzcDsmbmJzcDtsZXNzIHRoYW4g
b3IgZXF1YWwgdG8gOTkuOTglIG9mIHRoZSBub21pbmFsIHZhbHVlIG9mIExpbmUgUmF0ZTxvOnA+
PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250
LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJs
YWNrIj4mbmJzcDsmbmJzcDsgKHdpdGggbm8gZnVydGhlciBQUE0gYWRqdXN0bWVudCB0byBhY2Nv
dW50IGZvciBpbnRlcmZhY2UgY2xvY2sNCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZx
dW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj4mbmJzcDsmbmJzcDsmbmJzcDt0b2xl
cmFuY2VzKSwgdG8gZW5zdXJlIHN0cmVzc2luZyB0aGUgRFVUIGluIHJlYXNvbmFibGUgd29yc3Qg
Y2FzZQ0KPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4g
c3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVv
dDs7Y29sb3I6YmxhY2siPiZuYnNwOyZuYnNwOyZuYnNwO2NvbmRpdGlvbnMuPG86cD48L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox
MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPjxv
OnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0
eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7
O2NvbG9yOmJsYWNrIj5DaGFuZ2luZw0KPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1
b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPnMvcG9ydCBjYXBhY2l0eS9ub21pbmFs
IHZhbHVlIG9mIExpbmUgUmF0ZS8NCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90
O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj5oZXJlLCByZXF1aXJlcyBhbiBhZGRpdGlv
bmFsIHN1YnN0aXR1dGlvbiBpbiBzZWN0aW9uIDIuMyBvZiB0aGUgbWV0aCBkcmFmdCw8bzpwPjwv
bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFj
ayI+YW5kIGEgc2luZ2xlIHN1YnN0aXR1dGlvbiBpbiBzZWN0aW9uIDUuMSBvZiB0aGUgdGVybSBk
cmFmdC48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBz
dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90
Oztjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z
b05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7
Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPkRvZXMgdGhpcyBoZWxwPzxvOnA+PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj5B
bDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxl
PSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2Nv
bG9yOmJsYWNrIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3Vy
aWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0K
PGRpdiBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgYmx1ZSAxLjVwdDtwYWRk
aW5nOjBpbiAwaW4gMGluIDQuMHB0Ij4NCjxkaXY+DQo8ZGl2IHN0eWxlPSJib3JkZXI6bm9uZTti
b3JkZXItdG9wOnNvbGlkICNCNUM0REYgMS4wcHQ7cGFkZGluZzozLjBwdCAwaW4gMGluIDBpbiI+
DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtm
b250LWZhbWlseTomcXVvdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90OyI+RnJv
bTo8L3NwYW4+PC9iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZx
dW90O1RhaG9tYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7Ij4gTHVjaWVuIEF2cmFtb3Yg
W21haWx0bzpsdWNpZW5hdkBnb29nbGUuY29tXQ0KPGJyPg0KPGI+U2VudDo8L2I+IFNhdHVyZGF5
LCBKdW5lIDE3LCAyMDE3IDE6NDkgUE08YnI+DQo8Yj5Ubzo8L2I+IFNjb3R0IEJyYWRuZXI8YnI+
DQo8Yj5DYzo8L2I+IGJtd2dAaWV0Zi5vcmc7IGRyYWZ0LWlldGYtYm13Zy1kY2JlbmNoLW1ldGhv
ZG9sb2d5LmFsbEBpZXRmLm9yZzsgYm13Zy1jaGFpcnNAaWV0Zi5vcmc8YnI+DQo8Yj5TdWJqZWN0
OjwvYj4gUmU6IGRyYWZ0LWlldGYtYm13Zy1kY2JlbmNoLW1ldGhvZG9sb2d5LTA3PG86cD48L286
cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkhpIFNjb3R0ITxv
OnA+PC9vOnA+PC9wPg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bWFyZ2lu
LWxlZnQ6NDcuMjVwdDt0ZXh0LWluZGVudDotLjI1aW47bXNvLWxpc3Q6bDAgbGV2ZWwxIGxmbzEi
Pg0KPCFbaWYgIXN1cHBvcnRMaXN0c10+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9u
dC1mYW1pbHk6U3ltYm9sIj48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6SWdub3JlIj7CtzxzcGFuIHN0
eWxlPSJmb250OjcuMHB0ICZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyI+Jm5ic3A7Jm5ic3A7
Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7DQo8L3NwYW4+PC9zcGFuPjwvc3Bh
bj48IVtlbmRpZl0+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTo5LjVwdCI+VGhlIGRyYWZ0IHlvdSBh
cmUgcmV2aWV3aW5nIGlzIHRoZSAnJzxiPm1ldGhvZG9sb2d5PC9iPicnIGRyYWZ0IHdoaWNoIGV4
cGxhaW5zIHRoZSBtZXRob2RvbG9neSBmb3IgbGluZSByYXRlIHBlcmZvcm1hbmNlIHRlc3Rpbmcu
IFRoZSBleGFjdCByZWFzb24gb250byB3aHkgaXQgaXMgOTkuOTglIHdpdGggMCBQUE0gYWRqdXN0
bWVudCBpcyBjb3ZlcmVkDQogZXh0ZW5zaXZlbHkgaW4gdGhlJm5ic3A7PGI+dGVybWlub2xvZ3kg
ZHJhZnQ8L2I+LCBzZWN0aW9uIDU6Jm5ic3A7PGEgaHJlZj0iaHR0cHM6Ly91cmxkZWZlbnNlLnBy
b29mcG9pbnQuY29tL3YyL3VybD91PWh0dHBzLTNBX190b29scy5pZXRmLm9yZ19odG1sX2RyYWZ0
LTJEaWV0Zi0yRGJtd2ctMkRkY2JlbmNoLTJEdGVybWlub2xvZ3ktMkQxMi0yM3NlY3Rpb24tMkQ1
JmFtcDtkPUR3TUZhUSZhbXA7Yz1MRllaLW85X0hVTWVNVFNRaWN2aklnJmFtcDtyPU9mc1N1OGtU
SWx0VnlEMW9MNzJjQncmYW1wO209WFRTWUtNcEdsRy1kcnh1LUNWcURoVE54OFljVldkTW15cE9B
SVNrbkx2YyZhbXA7cz1Va1hjbXJ1RXNsd2hsN0NjSmhPRkVSel8yN0NaU2ZBZlItMFdIcVpsMTNn
JmFtcDtlPSIgdGFyZ2V0PSJfYmxhbmsiPmh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFm
dC1pZXRmLWJtd2ctZGNiZW5jaC10ZXJtaW5vbG9neS0xMiNzZWN0aW9uLTU8L2E+Lg0KIFRoYXQg
ZHJhZnQgZGVwZW5kZW5jeSBpcyBjb3ZlcmVkIGluIHRoZSBpbnRyb2R1Y3Rpb24gYW5kIGFsc28g
aW4gdGhlIHJlZmVyZW5jZSBsaXN0cyBhbmQgaXMgcmVmZXJlbmNlZCB0aHJvdWdob3V0IHRoZSBk
b2N1bWVudDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxl
PSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJn
aW4tbGVmdDo0Ny4yNXB0O3RleHQtaW5kZW50Oi0uMjVpbjttc28tbGlzdDpsMCBsZXZlbDEgbGZv
MSI+DQo8IVtpZiAhc3VwcG9ydExpc3RzXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtm
b250LWZhbWlseTpTeW1ib2wiPjxzcGFuIHN0eWxlPSJtc28tbGlzdDpJZ25vcmUiPsK3PHNwYW4g
c3R5bGU9ImZvbnQ6Ny4wcHQgJnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJz
cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9z
cGFuPjwhW2VuZGlmXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjkuNXB0Ij5UaGVyZSBpcyBhIGNs
b2NrIGRyaWZ0IGRpZmZlcmVuY2UgYmV0d2VlbiB0ZXN0ZXIgYW5kIERVVCBleHBsYWluZWQgaW4g
c2VjdGlvbiA1LjIgb2YgdGhlJm5ic3A7PGI+dGVybWlub2xvZ3kgZHJhZnQ8L2I+PG86cD48L286
cD48L3NwYW4+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6OS41cHQiPkluIG91ciB2aWV3IEJNV0cgaGFzIG5vdCBtb3JwaGVkLCBvdXIgZ29h
bCBoYXMgYmVlbiB0byBiZSB2ZXJ5IHByZWNpc2UgYW5kIGFjY3VyYXRlIHdpdGggcGVyZm9ybWFu
Y2UgdGVzdGluZywgYW5kIHRoaXMgaXMgbm90IGFib3V0IGNvbmZvcm1hbmNlIHRlc3RpbmcgYXQg
YWxsLiBUaGVzZSBkZWZpbml0aW9ucyB0aGF0IHlvdSBzZWUsIGhhdmUgbm90IGNoYW5nZWQmbmJz
cDs8c3BhbiBjbGFzcz0iZ21haWwtYXFqIj5pbg0KIDQgeWVhcnM8L3NwYW4+Jm5ic3A7YW5kIGhh
dmUgYmVlbiBwcm92ZW4gdGhlbXNlbHZlcyBpbiBhIHZlcnkgbGFyZ2UgYW1vdW50IG9mIHRlc3Rp
bmcgZG9uZSBzaW5jZSB3ZSBzdGFydGVkIHB1Ymxpc2hpbmcuPG86cD48L286cD48L3NwYW4+PC9w
Pg0KPC9kaXY+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls
ZT0iZm9udC1zaXplOjkuNXB0Ij48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjkuNXB0
Ij5BcyB5b3UgbWF5IHJlbWVtYmVyLCB3ZSBoYXZlIGJlZW4gd29ya2luZyBvbiB0aGlzIGRyYWZ0
IGFjdGl2ZWx5IGZvciBvdmVyIDQgeWVhcnMgbm93LiBXZSd2ZSBpbmNvcnBvcmF0ZWQgbnVtZXJv
dXMgb2YgeW91ciBmZWVkYmFja3Mgb24gdGhlc2UgZHJhZnRzIG92ZXIgdGhlIElFVEYgc2Vzc2lv
bnMgKHNpbmNlIElFVEYgODYgLSBjYXB0dXJlZCBpbiB0aGUgbWVldGluZw0KIG1pbnV0ZXMgb2Yg
dGhlIElFVEYgc2Vzc2lvbnMpLiAmbmJzcDtPbiBBcHJpbCAyMDE2LCB3ZSBoYWQgeW91ciBibGVz
c2luZyBvbiB0aGUgbGlzdCBhbHNvIHJlZ2FyZGluZyBvdXIgZHJhZnRzLjxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxl
PSJmb250LXNpemU6OS41cHQiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0K
PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6OS41cHQi
PldlIGFwcHJlY2lhdGUgYXMgYWx3YXlzIHlvdXIgZmVlZGJhY2ssIFNjb3R0ITxvOnA+PC9vOnA+
PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTo5LjVwdCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9w
Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQt
c2l6ZTo5LjVwdCI+THVjaWVuPG86cD48L286cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTo5LjVwdCI+PG86cD4m
bmJzcDs8L286cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+T24g
RnJpLCBKdW4gMTYsIDIwMTcgYXQgNzozMyBQTSwgU2NvdHQgQnJhZG5lciAmbHQ7PGEgaHJlZj0i
bWFpbHRvOnNvYkBzb2Jjby5jb20iIHRhcmdldD0iX2JsYW5rIj5zb2JAc29iY28uY29tPC9hPiZn
dDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibWFy
Z2luLWJvdHRvbToxMi4wcHQiPnRyeSBhZ2FpbiAtIG15IG5ldyBtYWMgYm9vayBvZnRlbiBzZW5k
cyBtYWlsIGJlZm9yZSBJIHdhbnQgaXQgdG88YnI+DQo8YnI+DQo8YnI+DQpJIGhhdmUgYmVlbiBh
c2tlZCB0byBkbyBhIE9QUy1ESVIgcmV2aWV3IG9mIGRyYWZ0LWlldGYtYm13Zy1kY2JlbmNoLW1l
dGhvZG9sb2d5LTA3PGJyPg0KPGJyPg0KSSBxdWlja2x5IGJlY2FtZSBjb25mdXNlZCAtIGEgcGFy
YWdyYXBoIGluIHNlY3Rpb24gMi4yIHJlYWRzPGJyPg0KPGJyPg0KRm9yIGFsbCB0ZXN0cywgdGhl
IHBlcmNlbnRhZ2Ugb2YgdHJhZmZpYyBwZXIgcG9ydCBjYXBhY2l0eSBzZW50IE1VU1Q8YnI+DQom
bmJzcDsgYmUgOTkuOTglIGF0IG1vc3QsIHdpdGggbm8gUFBNIGFkanVzdG1lbnQgdG8gZW5zdXJl
IHN0cmVzc2luZyB0aGUgRFVUPGJyPg0KJm5ic3A7IGluIHdvcnN0IGNhc2UgY29uZGl0aW9ucy4g
VGVzdHMgcmVzdWx0cyBhdCBhIGxvd2VyIHJhdGUgTUFZIGJlPGJyPg0KJm5ic3A7IHByb3ZpZGVk
IGZvciBiZXR0ZXIgdW5kZXJzdGFuZGluZyBvZiBwZXJmb3JtYW5jZSBpbmNyZWFzZSBpbiB0ZXJt
cyBvZjxicj4NCiZuYnNwOyBsYXRlbmN5IGFuZCBqaXR0ZXIgd2hlbiB0aGUgcmF0ZSBpcyBsb3dl
ciB0aGFuIDk5Ljk4JS4gVGhlIHJlY2VpdmluZzxicj4NCiZuYnNwOyByYXRlIG9mIHRoZSB0cmFm
ZmljIFNIT1VMRCBiZSBjYXB0dXJlZCBkdXJpbmcgdGhpcyB0ZXN0IGluICUgb2YgbGluZTxicj4N
CiZuYnNwOyByYXRlLjxicj4NCjxicj4NCjxicj4NClRodXMgcmVhZHMgdG8gbWUgbGlrZSBhIGNv
bmZvcm1hbmNlIHRlc3Qgbm90IGEgcGVyZm9ybWFuY2UgdGVzdCAoc2F5aW5nIHdoYXQgaXMg4oCc
cGFzc2luZ+KAnSkmbmJzcDsgdW5sZXNzPGJyPg0KQk1XRyBoYXMgbW9ycGhlZCBhIGxvdCBzaW5j
ZSBJIHdhcyBsYXN0IGludm9sdmVkIGNvbmZvcm1hbmNlIGFuZCBwYXNzL2ZhaWwgYXJlIG5vdCB0
aGluZ3MgdGhhdCBCTVdHPGJyPg0Kc2hvdWxkIGJlIGRvaW5nIC08YnI+DQo8YnI+DQp3aGF0IGFt
IEkgbWlzc2luZz88YnI+DQo8c3BhbiBzdHlsZT0iY29sb3I6Izg4ODg4OCI+PGJyPg0KPHNwYW4g
Y2xhc3M9ImhvZW56YiI+U2NvdHQ8L3NwYW4+PGJyPg0KPGJyPg0KPC9zcGFuPjxvOnA+PC9vOnA+
PC9wPg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4N
CjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K

--_000_4D7F4AD313D3FC43A053B309F97543CF25FDF58Dnjmtexg5researc_--


From nobody Sun Jun 18 06:53:28 2017
Return-Path: <sob@sobco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 44205127775; Sun, 18 Jun 2017 06:53:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.109
X-Spam-Level: 
X-Spam-Status: No, score=-1.109 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gBYvYVQxm0qg; Sun, 18 Jun 2017 06:53:24 -0700 (PDT)
Received: from sobco.sobco.com (unknown [136.248.127.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CCEB9127909; Sun, 18 Jun 2017 06:53:22 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by sobco.sobco.com (Postfix) with ESMTP id 7F31D440DB5F; Sun, 18 Jun 2017 09:53:21 -0400 (EDT)
X-Virus-Scanned: amavisd-new at sobco.com
Received: from sobco.sobco.com ([127.0.0.1]) by localhost (sobco.sobco.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d8ayZ60pWO1q; Sun, 18 Jun 2017 09:53:19 -0400 (EDT)
Received: from [192.168.1.100] (unknown [67.143.65.222]) by sobco.sobco.com (Postfix) with ESMTPSA id 002F7440DB4B; Sun, 18 Jun 2017 09:53:08 -0400 (EDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Scott Bradner <sob@sobco.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FDF58D@njmtexg5.research.att.com>
Date: Sun, 18 Jun 2017 09:52:48 -0400
Cc: Lucien Avramov <lucienav@google.com>, "bmwg@ietf.org" <bmwg@ietf.org>, "draft-ietf-bmwg-dcbench-methodology.all@ietf.org" <draft-ietf-bmwg-dcbench-methodology.all@ietf.org>,  "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <C7765B7F-6B7F-45C4-AFC6-A636E8B34810@sobco.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com> <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com> <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF25FDF58D@njmtexg5.research.att.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/f0yBmwl1_jGEuPVcA7k5ekv5J-4>
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 13:53:26 -0000

my issue was that I read the current text as talking about the result of =
the testing not the offered load

so I was just going to suggest coming similar to what Al suggested=20

but I=E2=80=99d add specific pointers to the explanation in the =
terminology document as to why


e.g.

   For all tests, the test traffic generator sending rate MUST be
   less than or equal to 99.98% of the nominal value of Line Rate
   (with no further PPM adjustment to account for interface clock
   tolerances), to ensure stressing the DUT in reasonable worst case
   conditions. (see RFC XXX section ??? - note to RFC editor replace XXX
   with number of terminology RFC)

I have not reviewed the rest of the ID to see if similar misreading is =
possible so that
a similar change would be useful

Scott


> On Jun 18, 2017, at 1:03 AM, MORTON, ALFRED C (AL) <acmorton@att.com> =
wrote:
>=20
> Hi Lucien, Scott, and all,
>=20
> =20
>=20
> When I read this material, I assumed the goal was to
>=20
> limit the maximum sending rate of the test traffic generator
>=20
> in accordance with the Ethernet clock tolerances,
>=20
> effectively choosing the low-end of clock tolerance
>=20
> as the maximum sending rate with a factor of 2 margin.=20
>=20
> 99.98% of the nominal line rate would be -200 ppm,
>=20
> and any conforming interface clock should work fine.
>=20
> =20
>=20
> If I understood your goal correctly,
>=20
> I wonder if a small clarification will help future
>=20
> readers.
>=20
> =20
>=20
> When section 2.2 says:
>=20
>    For all tests, the percentage of traffic per port capacity sent =
MUST
>=20
>                                                 ^^^^^^^^^^^^^^^^^^
>=20
>    be 99.98% at most, with no PPM adjustment to ensure stressing the =
DUT
>=20
>       +++++++++++++++
>=20
>    in worst case conditions.
>=20
> =20
>=20
> You=E2=80=99ve defined the concept of Port Capacity in terminology =
Section 5,
>=20
> under the Line Rate definition, as
>=20
> =20
>=20
>    The term "port capacity" defines the maximum speed capability for =
the
>=20
>    given port; for example 1GE, 10GE, 40GE, 100GE etc.
>=20
> =20
>=20
> Actually, the port capacity examples are the *nominal value of Line =
Rate*,
>=20
> before the clock tolerances are considered.
>=20
> =20
>=20
> Here=E2=80=99s a suggestion for Section 2.2 of the meth:
>=20
> =20
>=20
>    For all tests, the test traffic generator sending rate MUST be
>=20
>    less than or equal to 99.98% of the nominal value of Line Rate
>=20
>    (with no further PPM adjustment to account for interface clock
>=20
>    tolerances), to ensure stressing the DUT in reasonable worst case
>=20
>    conditions.
>=20
> =20
>=20
> Changing
>=20
> s/port capacity/nominal value of Line Rate/
>=20
> here, requires an additional substitution in section 2.3 of the meth =
draft,
>=20
> and a single substitution in section 5.1 of the term draft.
>=20
> =20
>=20
> Does this help?
>=20
> Al
>=20
> =20
>=20
> =20
>=20
> From: Lucien Avramov [mailto:lucienav@google.com]=20
> Sent: Saturday, June 17, 2017 1:49 PM
> To: Scott Bradner
> Cc: bmwg@ietf.org; draft-ietf-bmwg-dcbench-methodology.all@ietf.org; =
bmwg-chairs@ietf.org
> Subject: Re: draft-ietf-bmwg-dcbench-methodology-07
>=20
> =20
>=20
> Hi Scott!
>=20
> =C2=B7         The draft you are reviewing is the ''methodology'' =
draft which explains the methodology for line rate performance testing. =
The exact reason onto why it is 99.98% with 0 PPM adjustment is covered =
extensively in the terminology draft, section 5: =
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-12#section=
-5. That draft dependency is covered in the introduction and also in the =
reference lists and is referenced throughout the document
>=20
> =C2=B7         There is a clock drift difference between tester and =
DUT explained in section 5.2 of the terminology draft
>=20
> In our view BMWG has not morphed, our goal has been to be very precise =
and accurate with performance testing, and this is not about conformance =
testing at all. These definitions that you see, have not changed in 4 =
years and have been proven themselves in a very large amount of testing =
done since we started publishing.
>=20
> =20
>=20
> As you may remember, we have been working on this draft actively for =
over 4 years now. We've incorporated numerous of your feedbacks on these =
drafts over the IETF sessions (since IETF 86 - captured in the meeting =
minutes of the IETF sessions).  On April 2016, we had your blessing on =
the list also regarding our drafts.
>=20
> =20
>=20
> We appreciate as always your feedback, Scott!
>=20
> =20
>=20
> Lucien
>=20
> =20
>=20
> =20
>=20
> On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner <sob@sobco.com> wrote:
>=20
> try again - my new mac book often sends mail before I want it to
>=20
>=20
> I have been asked to do a OPS-DIR review of =
draft-ietf-bmwg-dcbench-methodology-07
>=20
> I quickly became confused - a paragraph in section 2.2 reads
>=20
> For all tests, the percentage of traffic per port capacity sent MUST
>   be 99.98% at most, with no PPM adjustment to ensure stressing the =
DUT
>   in worst case conditions. Tests results at a lower rate MAY be
>   provided for better understanding of performance increase in terms =
of
>   latency and jitter when the rate is lower than 99.98%. The receiving
>   rate of the traffic SHOULD be captured during this test in % of line
>   rate.
>=20
>=20
> Thus reads to me like a conformance test not a performance test =
(saying what is =E2=80=9Cpassing=E2=80=9D)  unless
> BMWG has morphed a lot since I was last involved conformance and =
pass/fail are not things that BMWG
> should be doing -
>=20
> what am I missing?
>=20
> Scott
>=20
>=20
> =20
>=20


From nobody Sun Jun 18 09:52:14 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3ADA312947D; Sun, 18 Jun 2017 09:52:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WLYQkIsiiBpt; Sun, 18 Jun 2017 09:52:02 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B6BBC1270FC; Sun, 18 Jun 2017 09:52:02 -0700 (PDT)
Received: from pps.filterd (m0049295.ppops.net [127.0.0.1]) by m0049295.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v5IGjHNl014894; Sun, 18 Jun 2017 12:51:59 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049295.ppops.net-00191d01. with ESMTP id 2b50d53g3v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 18 Jun 2017 12:51:59 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5IGpw5X092368; Sun, 18 Jun 2017 11:51:58 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5IGpuQm092364 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 18 Jun 2017 11:51:56 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor); Sun, 18 Jun 2017 16:51:40 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5IGpeWj025577; Sun, 18 Jun 2017 11:51:40 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5IGpVbk025310; Sun, 18 Jun 2017 11:51:35 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 7EE51E0428; Sun, 18 Jun 2017 12:51:30 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Sun, 18 Jun 2017 12:51:30 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Scott Bradner <sob@sobco.com>, "ops-dir@ietf.org" <ops-dir@ietf.org>
CC: "draft-ietf-bmwg-dcbench-methodology.all@ietf.org" <draft-ietf-bmwg-dcbench-methodology.all@ietf.org>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
Thread-Index: AQHS55IT+1LmjVZvaUuikgc0tDhWFKIpopIQgAFFrgD//+3SoA==
Date: Sun, 18 Jun 2017 16:51:29 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FDF6E2@njmtexg5.research.att.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com> <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com> <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF25FDF58D@njmtexg5.research.att.com> <C7765B7F-6B7F-45C4-AFC6-A636E8B34810@sobco.com>
In-Reply-To: <C7765B7F-6B7F-45C4-AFC6-A636E8B34810@sobco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.70.159.40]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-18_11:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706180301
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dtsffcO4DC1JNE8MhBAnhA-AxLA>
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 16:52:05 -0000

YWRkaW5nIE9QUy1ESVIsDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGJt
d2cgW21haWx0bzpibXdnLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBTY290dCBCcmFk
bmVyDQo+IFNlbnQ6IFN1bmRheSwgSnVuZSAxOCwgMjAxNyA5OjUzIEFNDQo+IFRvOiBNT1JUT04s
IEFMRlJFRCBDIChBTCkNCj4gQ2M6IGRyYWZ0LWlldGYtYm13Zy1kY2JlbmNoLW1ldGhvZG9sb2d5
LmFsbEBpZXRmLm9yZzsgYm13Zy0NCj4gY2hhaXJzQGlldGYub3JnOyBibXdnQGlldGYub3JnDQo+
IFN1YmplY3Q6IFJlOiBbYm13Z10gZHJhZnQtaWV0Zi1ibXdnLWRjYmVuY2gtbWV0aG9kb2xvZ3kt
MDcNCj4gDQo+IG15IGlzc3VlIHdhcyB0aGF0IEkgcmVhZCB0aGUgY3VycmVudCB0ZXh0IGFzIHRh
bGtpbmcgYWJvdXQgdGhlIHJlc3VsdCBvZg0KPiB0aGUgdGVzdGluZyBub3QgdGhlIG9mZmVyZWQg
bG9hZA0KPiANCj4gc28gSSB3YXMganVzdCBnb2luZyB0byBzdWdnZXN0IGNvbWluZyBzaW1pbGFy
IHRvIHdoYXQgQWwgc3VnZ2VzdGVkDQo+IA0KPiBidXQgSeKAmWQgYWRkIHNwZWNpZmljIHBvaW50
ZXJzIHRvIHRoZSBleHBsYW5hdGlvbiBpbiB0aGUgdGVybWlub2xvZ3kNCj4gZG9jdW1lbnQgYXMg
dG8gd2h5DQo+IA0KPiANCj4gZS5nLg0KPiANCj4gICAgRm9yIGFsbCB0ZXN0cywgdGhlIHRlc3Qg
dHJhZmZpYyBnZW5lcmF0b3Igc2VuZGluZyByYXRlIE1VU1QgYmUNCj4gICAgbGVzcyB0aGFuIG9y
IGVxdWFsIHRvIDk5Ljk4JSBvZiB0aGUgbm9taW5hbCB2YWx1ZSBvZiBMaW5lIFJhdGUNCj4gICAg
KHdpdGggbm8gZnVydGhlciBQUE0gYWRqdXN0bWVudCB0byBhY2NvdW50IGZvciBpbnRlcmZhY2Ug
Y2xvY2sNCj4gICAgdG9sZXJhbmNlcyksIHRvIGVuc3VyZSBzdHJlc3NpbmcgdGhlIERVVCBpbiBy
ZWFzb25hYmxlIHdvcnN0IGNhc2UNCj4gICAgY29uZGl0aW9ucy4gKHNlZSBSRkMgWFhYIHNlY3Rp
b24gPz8/IC0gbm90ZSB0byBSRkMgZWRpdG9yIHJlcGxhY2UgWFhYDQo+ICAgIHdpdGggbnVtYmVy
IG9mIHRlcm1pbm9sb2d5IFJGQykNCj4gDQpbQUNNXSANCmdvb2QgYWRkaXRpb24sIFNjb3R0Lg0K
DQo+IEkgaGF2ZSBub3QgcmV2aWV3ZWQgdGhlIHJlc3Qgb2YgdGhlIElEIHRvIHNlZSBpZiBzaW1p
bGFyIG1pc3JlYWRpbmcgaXMNCj4gcG9zc2libGUgc28gdGhhdA0KPiBhIHNpbWlsYXIgY2hhbmdl
IHdvdWxkIGJlIHVzZWZ1bA0KW0FDTV0gDQpMZXQncyBhc3N1bWUgZm9yIG5vdyB0aGF0IG90aGVy
IEFyZWEgcmV2aWV3ZXJzIGFuZCBJRVNHIHJldmlldyB3aWxsDQpjYXRjaCBzb21lIG1vcmUgb2Yg
dGhlc2UsIGlmIG5lZWRlZC4gDQoNCnRoYW5rcywNCkFsDQoNCj4gDQo+IFNjb3R0DQo+IA0KPiAN
Cj4gPiBPbiBKdW4gMTgsIDIwMTcsIGF0IDE6MDMgQU0sIE1PUlRPTiwgQUxGUkVEIEMgKEFMKSA8
YWNtb3J0b25AYXR0LmNvbT4NCj4gd3JvdGU6DQo+ID4NCj4gPiBIaSBMdWNpZW4sIFNjb3R0LCBh
bmQgYWxsLA0KPiA+DQo+ID4NCj4gPg0KPiA+IFdoZW4gSSByZWFkIHRoaXMgbWF0ZXJpYWwsIEkg
YXNzdW1lZCB0aGUgZ29hbCB3YXMgdG8NCj4gPg0KPiA+IGxpbWl0IHRoZSBtYXhpbXVtIHNlbmRp
bmcgcmF0ZSBvZiB0aGUgdGVzdCB0cmFmZmljIGdlbmVyYXRvcg0KPiA+DQo+ID4gaW4gYWNjb3Jk
YW5jZSB3aXRoIHRoZSBFdGhlcm5ldCBjbG9jayB0b2xlcmFuY2VzLA0KPiA+DQo+ID4gZWZmZWN0
aXZlbHkgY2hvb3NpbmcgdGhlIGxvdy1lbmQgb2YgY2xvY2sgdG9sZXJhbmNlDQo+ID4NCj4gPiBh
cyB0aGUgbWF4aW11bSBzZW5kaW5nIHJhdGUgd2l0aCBhIGZhY3RvciBvZiAyIG1hcmdpbi4NCj4g
Pg0KPiA+IDk5Ljk4JSBvZiB0aGUgbm9taW5hbCBsaW5lIHJhdGUgd291bGQgYmUgLTIwMCBwcG0s
DQo+ID4NCj4gPiBhbmQgYW55IGNvbmZvcm1pbmcgaW50ZXJmYWNlIGNsb2NrIHNob3VsZCB3b3Jr
IGZpbmUuDQo+ID4NCj4gPg0KPiA+DQo+ID4gSWYgSSB1bmRlcnN0b29kIHlvdXIgZ29hbCBjb3Jy
ZWN0bHksDQo+ID4NCj4gPiBJIHdvbmRlciBpZiBhIHNtYWxsIGNsYXJpZmljYXRpb24gd2lsbCBo
ZWxwIGZ1dHVyZQ0KPiA+DQo+ID4gcmVhZGVycy4NCj4gPg0KPiA+DQo+ID4NCj4gPiBXaGVuIHNl
Y3Rpb24gMi4yIHNheXM6DQo+ID4NCj4gPiAgICBGb3IgYWxsIHRlc3RzLCB0aGUgcGVyY2VudGFn
ZSBvZiB0cmFmZmljIHBlciBwb3J0IGNhcGFjaXR5IHNlbnQNCj4gTVVTVA0KPiA+DQo+ID4gICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXl5eXl5eXl5eXl5e
Xl5eXl5eDQo+ID4NCj4gPiAgICBiZSA5OS45OCUgYXQgbW9zdCwgd2l0aCBubyBQUE0gYWRqdXN0
bWVudCB0byBlbnN1cmUgc3RyZXNzaW5nIHRoZQ0KPiBEVVQNCj4gPg0KPiA+ICAgICAgICsrKysr
KysrKysrKysrKw0KPiA+DQo+ID4gICAgaW4gd29yc3QgY2FzZSBjb25kaXRpb25zLg0KPiA+DQo+
ID4NCj4gPg0KPiA+IFlvdeKAmXZlIGRlZmluZWQgdGhlIGNvbmNlcHQgb2YgUG9ydCBDYXBhY2l0
eSBpbiB0ZXJtaW5vbG9neSBTZWN0aW9uIDUsDQo+ID4NCj4gPiB1bmRlciB0aGUgTGluZSBSYXRl
IGRlZmluaXRpb24sIGFzDQo+ID4NCj4gPg0KPiA+DQo+ID4gICAgVGhlIHRlcm0gInBvcnQgY2Fw
YWNpdHkiIGRlZmluZXMgdGhlIG1heGltdW0gc3BlZWQgY2FwYWJpbGl0eSBmb3INCj4gdGhlDQo+
ID4NCj4gPiAgICBnaXZlbiBwb3J0OyBmb3IgZXhhbXBsZSAxR0UsIDEwR0UsIDQwR0UsIDEwMEdF
IGV0Yy4NCj4gPg0KPiA+DQo+ID4NCj4gPiBBY3R1YWxseSwgdGhlIHBvcnQgY2FwYWNpdHkgZXhh
bXBsZXMgYXJlIHRoZSAqbm9taW5hbCB2YWx1ZSBvZiBMaW5lDQo+IFJhdGUqLA0KPiA+DQo+ID4g
YmVmb3JlIHRoZSBjbG9jayB0b2xlcmFuY2VzIGFyZSBjb25zaWRlcmVkLg0KPiA+DQo+ID4NCj4g
Pg0KPiA+IEhlcmXigJlzIGEgc3VnZ2VzdGlvbiBmb3IgU2VjdGlvbiAyLjIgb2YgdGhlIG1ldGg6
DQo+ID4NCj4gPg0KPiA+DQo+ID4gICAgRm9yIGFsbCB0ZXN0cywgdGhlIHRlc3QgdHJhZmZpYyBn
ZW5lcmF0b3Igc2VuZGluZyByYXRlIE1VU1QgYmUNCj4gPg0KPiA+ICAgIGxlc3MgdGhhbiBvciBl
cXVhbCB0byA5OS45OCUgb2YgdGhlIG5vbWluYWwgdmFsdWUgb2YgTGluZSBSYXRlDQo+ID4NCj4g
PiAgICAod2l0aCBubyBmdXJ0aGVyIFBQTSBhZGp1c3RtZW50IHRvIGFjY291bnQgZm9yIGludGVy
ZmFjZSBjbG9jaw0KPiA+DQo+ID4gICAgdG9sZXJhbmNlcyksIHRvIGVuc3VyZSBzdHJlc3Npbmcg
dGhlIERVVCBpbiByZWFzb25hYmxlIHdvcnN0IGNhc2UNCj4gPg0KPiA+ICAgIGNvbmRpdGlvbnMu
DQo+ID4NCj4gPg0KPiA+DQo+ID4gQ2hhbmdpbmcNCj4gPg0KPiA+IHMvcG9ydCBjYXBhY2l0eS9u
b21pbmFsIHZhbHVlIG9mIExpbmUgUmF0ZS8NCj4gPg0KPiA+IGhlcmUsIHJlcXVpcmVzIGFuIGFk
ZGl0aW9uYWwgc3Vic3RpdHV0aW9uIGluIHNlY3Rpb24gMi4zIG9mIHRoZSBtZXRoDQo+IGRyYWZ0
LA0KPiA+DQo+ID4gYW5kIGEgc2luZ2xlIHN1YnN0aXR1dGlvbiBpbiBzZWN0aW9uIDUuMSBvZiB0
aGUgdGVybSBkcmFmdC4NCj4gPg0KPiA+DQo+ID4NCj4gPiBEb2VzIHRoaXMgaGVscD8NCj4gPg0K
PiA+IEFsDQo+ID4NCj4gPg0KPiA+DQo+ID4NCj4gPg0KPiA+IEZyb206IEx1Y2llbiBBdnJhbW92
IFttYWlsdG86bHVjaWVuYXZAZ29vZ2xlLmNvbV0NCj4gPiBTZW50OiBTYXR1cmRheSwgSnVuZSAx
NywgMjAxNyAxOjQ5IFBNDQo+ID4gVG86IFNjb3R0IEJyYWRuZXINCj4gPiBDYzogYm13Z0BpZXRm
Lm9yZzsgZHJhZnQtaWV0Zi1ibXdnLWRjYmVuY2gtbWV0aG9kb2xvZ3kuYWxsQGlldGYub3JnOw0K
PiBibXdnLWNoYWlyc0BpZXRmLm9yZw0KPiA+IFN1YmplY3Q6IFJlOiBkcmFmdC1pZXRmLWJtd2ct
ZGNiZW5jaC1tZXRob2RvbG9neS0wNw0KPiA+DQo+ID4NCj4gPg0KPiA+IEhpIFNjb3R0IQ0KPiA+
DQo+ID4gwrcgICAgICAgICBUaGUgZHJhZnQgeW91IGFyZSByZXZpZXdpbmcgaXMgdGhlICcnbWV0
aG9kb2xvZ3knJyBkcmFmdA0KPiB3aGljaCBleHBsYWlucyB0aGUgbWV0aG9kb2xvZ3kgZm9yIGxp
bmUgcmF0ZSBwZXJmb3JtYW5jZSB0ZXN0aW5nLiBUaGUNCj4gZXhhY3QgcmVhc29uIG9udG8gd2h5
IGl0IGlzIDk5Ljk4JSB3aXRoIDAgUFBNIGFkanVzdG1lbnQgaXMgY292ZXJlZA0KPiBleHRlbnNp
dmVseSBpbiB0aGUgdGVybWlub2xvZ3kgZHJhZnQsIHNlY3Rpb24gNToNCj4gaHR0cHM6Ly91cmxk
ZWZlbnNlLnByb29mcG9pbnQuY29tL3YyL3VybD91PWh0dHBzLQ0KPiAzQV9fdG9vbHMuaWV0Zi5v
cmdfaHRtbF9kcmFmdC0yRGlldGYtMkRibXdnLTJEZGNiZW5jaC0yRHRlcm1pbm9sb2d5LQ0KPiAy
RDEyLTIzc2VjdGlvbi0yRDUmZD1Ed0lHYVEmYz1MRllaLQ0KPiBvOV9IVU1lTVRTUWljdmpJZyZy
PU9mc1N1OGtUSWx0VnlEMW9MNzJjQncmbT1jWTJlSVdmQWF1anpQcjdBNDlsV0NqdUpfVngNCj4g
MHl3aEVERG1zV1BCY1RKQSZzPTk2VDNLcXhobTNjLWdBU0RhMXdfWERSQWNYWWgzMG03ZnlqUEVt
MjhIQ0UmZT0gLiBUaGF0DQo+IGRyYWZ0IGRlcGVuZGVuY3kgaXMgY292ZXJlZCBpbiB0aGUgaW50
cm9kdWN0aW9uIGFuZCBhbHNvIGluIHRoZQ0KPiByZWZlcmVuY2UgbGlzdHMgYW5kIGlzIHJlZmVy
ZW5jZWQgdGhyb3VnaG91dCB0aGUgZG9jdW1lbnQNCj4gPg0KPiA+IMK3ICAgICAgICAgVGhlcmUg
aXMgYSBjbG9jayBkcmlmdCBkaWZmZXJlbmNlIGJldHdlZW4gdGVzdGVyIGFuZCBEVVQNCj4gZXhw
bGFpbmVkIGluIHNlY3Rpb24gNS4yIG9mIHRoZSB0ZXJtaW5vbG9neSBkcmFmdA0KPiA+DQo+ID4g
SW4gb3VyIHZpZXcgQk1XRyBoYXMgbm90IG1vcnBoZWQsIG91ciBnb2FsIGhhcyBiZWVuIHRvIGJl
IHZlcnkgcHJlY2lzZQ0KPiBhbmQgYWNjdXJhdGUgd2l0aCBwZXJmb3JtYW5jZSB0ZXN0aW5nLCBh
bmQgdGhpcyBpcyBub3QgYWJvdXQgY29uZm9ybWFuY2UNCj4gdGVzdGluZyBhdCBhbGwuIFRoZXNl
IGRlZmluaXRpb25zIHRoYXQgeW91IHNlZSwgaGF2ZSBub3QgY2hhbmdlZCBpbiA0DQo+IHllYXJz
IGFuZCBoYXZlIGJlZW4gcHJvdmVuIHRoZW1zZWx2ZXMgaW4gYSB2ZXJ5IGxhcmdlIGFtb3VudCBv
ZiB0ZXN0aW5nDQo+IGRvbmUgc2luY2Ugd2Ugc3RhcnRlZCBwdWJsaXNoaW5nLg0KPiA+DQo+ID4N
Cj4gPg0KPiA+IEFzIHlvdSBtYXkgcmVtZW1iZXIsIHdlIGhhdmUgYmVlbiB3b3JraW5nIG9uIHRo
aXMgZHJhZnQgYWN0aXZlbHkgZm9yDQo+IG92ZXIgNCB5ZWFycyBub3cuIFdlJ3ZlIGluY29ycG9y
YXRlZCBudW1lcm91cyBvZiB5b3VyIGZlZWRiYWNrcyBvbiB0aGVzZQ0KPiBkcmFmdHMgb3ZlciB0
aGUgSUVURiBzZXNzaW9ucyAoc2luY2UgSUVURiA4NiAtIGNhcHR1cmVkIGluIHRoZSBtZWV0aW5n
DQo+IG1pbnV0ZXMgb2YgdGhlIElFVEYgc2Vzc2lvbnMpLiAgT24gQXByaWwgMjAxNiwgd2UgaGFk
IHlvdXIgYmxlc3Npbmcgb24NCj4gdGhlIGxpc3QgYWxzbyByZWdhcmRpbmcgb3VyIGRyYWZ0cy4N
Cj4gPg0KPiA+DQo+ID4NCj4gPiBXZSBhcHByZWNpYXRlIGFzIGFsd2F5cyB5b3VyIGZlZWRiYWNr
LCBTY290dCENCj4gPg0KPiA+DQo+ID4NCj4gPiBMdWNpZW4NCj4gPg0KPiA+DQo+ID4NCj4gPg0K
PiA+DQo+ID4gT24gRnJpLCBKdW4gMTYsIDIwMTcgYXQgNzozMyBQTSwgU2NvdHQgQnJhZG5lciA8
c29iQHNvYmNvLmNvbT4gd3JvdGU6DQo+ID4NCj4gPiB0cnkgYWdhaW4gLSBteSBuZXcgbWFjIGJv
b2sgb2Z0ZW4gc2VuZHMgbWFpbCBiZWZvcmUgSSB3YW50IGl0IHRvDQo+ID4NCj4gPg0KPiA+IEkg
aGF2ZSBiZWVuIGFza2VkIHRvIGRvIGEgT1BTLURJUiByZXZpZXcgb2YgZHJhZnQtaWV0Zi1ibXdn
LWRjYmVuY2gtDQo+IG1ldGhvZG9sb2d5LTA3DQo+ID4NCj4gPiBJIHF1aWNrbHkgYmVjYW1lIGNv
bmZ1c2VkIC0gYSBwYXJhZ3JhcGggaW4gc2VjdGlvbiAyLjIgcmVhZHMNCj4gPg0KPiA+IEZvciBh
bGwgdGVzdHMsIHRoZSBwZXJjZW50YWdlIG9mIHRyYWZmaWMgcGVyIHBvcnQgY2FwYWNpdHkgc2Vu
dCBNVVNUDQo+ID4gICBiZSA5OS45OCUgYXQgbW9zdCwgd2l0aCBubyBQUE0gYWRqdXN0bWVudCB0
byBlbnN1cmUgc3RyZXNzaW5nIHRoZQ0KPiBEVVQNCj4gPiAgIGluIHdvcnN0IGNhc2UgY29uZGl0
aW9ucy4gVGVzdHMgcmVzdWx0cyBhdCBhIGxvd2VyIHJhdGUgTUFZIGJlDQo+ID4gICBwcm92aWRl
ZCBmb3IgYmV0dGVyIHVuZGVyc3RhbmRpbmcgb2YgcGVyZm9ybWFuY2UgaW5jcmVhc2UgaW4gdGVy
bXMNCj4gb2YNCj4gPiAgIGxhdGVuY3kgYW5kIGppdHRlciB3aGVuIHRoZSByYXRlIGlzIGxvd2Vy
IHRoYW4gOTkuOTglLiBUaGUgcmVjZWl2aW5nDQo+ID4gICByYXRlIG9mIHRoZSB0cmFmZmljIFNI
T1VMRCBiZSBjYXB0dXJlZCBkdXJpbmcgdGhpcyB0ZXN0IGluICUgb2YgbGluZQ0KPiA+ICAgcmF0
ZS4NCj4gPg0KPiA+DQo+ID4gVGh1cyByZWFkcyB0byBtZSBsaWtlIGEgY29uZm9ybWFuY2UgdGVz
dCBub3QgYSBwZXJmb3JtYW5jZSB0ZXN0DQo+IChzYXlpbmcgd2hhdCBpcyDigJxwYXNzaW5n4oCd
KSAgdW5sZXNzDQo+ID4gQk1XRyBoYXMgbW9ycGhlZCBhIGxvdCBzaW5jZSBJIHdhcyBsYXN0IGlu
dm9sdmVkIGNvbmZvcm1hbmNlIGFuZA0KPiBwYXNzL2ZhaWwgYXJlIG5vdCB0aGluZ3MgdGhhdCBC
TVdHDQo+ID4gc2hvdWxkIGJlIGRvaW5nIC0NCj4gPg0KPiA+IHdoYXQgYW0gSSBtaXNzaW5nPw0K
PiA+DQo+ID4gU2NvdHQNCj4gPg0KPiA+DQo+ID4NCj4gPg0KPiANCj4gX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gYm13ZyBtYWlsaW5nIGxpc3QNCj4g
Ym13Z0BpZXRmLm9yZw0KPiBodHRwczovL3VybGRlZmVuc2UucHJvb2Zwb2ludC5jb20vdjIvdXJs
P3U9aHR0cHMtDQo+IDNBX193d3cuaWV0Zi5vcmdfbWFpbG1hbl9saXN0aW5mb19ibXdnJmQ9RHdJ
R2FRJmM9TEZZWi0NCj4gbzlfSFVNZU1UU1FpY3ZqSWcmcj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3
Jm09Y1kyZUlXZkFhdWp6UHI3QTQ5bFdDanVKX1Z4DQo+IDB5d2hFRERtc1dQQmNUSkEmcz1Od05j
endUVU5feVFxamw0QXc0akJUcWszaU1iSnBUckpNRGl1RlQ4ZzIwJmU9DQo=


From nobody Sun Jun 18 09:57:29 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 497AC129482; Sun, 18 Jun 2017 09:57:20 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149780504024.10668.9284122758193445874@ietfa.amsl.com>
Date: Sun, 18 Jun 2017 09:57:20 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dZ1MtbHL7Mtd4AnaA9X5DnrxLCk>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-11.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 16:57:20 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-11.txt
	Pages           : 18
	Date            : 2017-06-18

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. Many of these terms and methods
   may be applicable beyond this publication's scope as the technologies
   originally applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-11
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-11

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-11


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Sun Jun 18 09:57:42 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 9B2001294B2; Sun, 18 Jun 2017 09:57:31 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149780505159.10664.7052393026706707129@ietfa.amsl.com>
Date: Sun, 18 Jun 2017 09:57:31 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/QAGmeL2M_AufK7h68SNQ1Ar7xA4>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-13.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 16:57:32 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-13.txt
	Pages           : 17
	Date            : 2017-06-18

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center. Many of
these terms and methods may be applicable beyond this publication's
scope as the technologies originally applied in the data center are
deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-13
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-13

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-13


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Sun Jun 18 10:01:06 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EACA21294A6 for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 10:01:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HLcaSiTdHIEQ for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 10:01:01 -0700 (PDT)
Received: from mail-pg0-x22d.google.com (mail-pg0-x22d.google.com [IPv6:2607:f8b0:400e:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A5EB3128CDB for <bmwg@ietf.org>; Sun, 18 Jun 2017 10:01:01 -0700 (PDT)
Received: by mail-pg0-x22d.google.com with SMTP id u62so19555176pgb.3 for <bmwg@ietf.org>; Sun, 18 Jun 2017 10:01:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6dV8mSlubpzBFrpb2TepohF7CxBVqa4M57c/jJJmVKE=; b=dorlCz7bEePV/AGoZyaQ5Uu2zWmkcqmSp41OQwVDz2mhoty+fv+ZZYRr/g9oPHJ0YC ncYw4LbRDGf4byaGzqBotUx9YeGA5RNGEpj+34/nfUuvKJ6tP6UZp7p9n9GM90qBKOL9 5rAXt9wGEPMJM4KCaHrK+Wa7VxllVyD0xGpHPFXmDwMrLtCSMrbIZwLDP3uK0xLjSBRR tg3+uxXPRajCzLnjp3y0PNDIHUqOAgmtcSdNO9MG1xSvf4MLTFUJmSS6wMdibzAMCaQY 3Vpwcm+eyPPNWH8z74W8i7pwie+eHygL/6EYP0mBTxZxQHk64QsvwPyssbSYNMGblcLr Uovg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6dV8mSlubpzBFrpb2TepohF7CxBVqa4M57c/jJJmVKE=; b=fcVomfOqn9oqW8c2AR44WpsobYNMJ6x6e60Hp55fKC3RCJvPFB4me0abkC8s0r/E9g qHO1TnZNLs3jLPfTqDGcJ0turuGFhzIOtjG/WU6y7JmorHpixz+mHvRwqzRmV6EAdYec PBpyM40Dz5bv3TVamJPAPwnkR2AiLORT8858kBUhHBvFSsGSfio/LXPBDs1f60fU/lkL XAbmzP7v6uhcI1JTVaU25bPCjSPT8KnD3G5Mn+Gj/DmWo/pfKvq0+keVqAgrEeSB1MAo pq4GCk+nYw/UTByPtK4obdHrTpLaUaxw02hxkDPPukJv7drBhfZUR81m+FELYzX+nD/B tCtg==
X-Gm-Message-State: AKS2vOyhYXyXIe6G38kbiBp9ao295alsY08SILgStMn8BEcTHfkIHmXb MCXTCYhEMd1PUfoAQbWbcc+SOLDjwiJo
X-Received: by 10.84.238.206 with SMTP id l14mr4252566pln.280.1497805261149; Sun, 18 Jun 2017 10:01:01 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Sun, 18 Jun 2017 10:00:40 -0700 (PDT)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FDF6E2@njmtexg5.research.att.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com> <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com> <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF25FDF58D@njmtexg5.research.att.com> <C7765B7F-6B7F-45C4-AFC6-A636E8B34810@sobco.com> <4D7F4AD313D3FC43A053B309F97543CF25FDF6E2@njmtexg5.research.att.com>
From: Lucien Avramov <lucienav@google.com>
Date: Sun, 18 Jun 2017 10:00:40 -0700
Message-ID: <CALTEt=ByHULaoXkpKTr30fcCcCnGzNeFTr+fVqfrgVV-Rr1euw@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Cc: Scott Bradner <sob@sobco.com>, "ops-dir@ietf.org" <ops-dir@ietf.org>,  "draft-ietf-bmwg-dcbench-methodology.all@ietf.org" <draft-ietf-bmwg-dcbench-methodology.all@ietf.org>,  "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="f403045fdf8488f03a05523ef6ec"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/nnoagZyWWAevSOh32E0x32Bn4lw>
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 17:01:05 -0000

--f403045fdf8488f03a05523ef6ec
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Al, Scott

thanks for the comments, updated both drafts just now with all comments!

Thanks,
Lucien

On Sun, Jun 18, 2017 at 9:51 AM, MORTON, ALFRED C (AL) <acmorton@att.com>
wrote:

> adding OPS-DIR,
> > -----Original Message-----
> > From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Scott Bradner
> > Sent: Sunday, June 18, 2017 9:53 AM
> > To: MORTON, ALFRED C (AL)
> > Cc: draft-ietf-bmwg-dcbench-methodology.all@ietf.org; bmwg-
> > chairs@ietf.org; bmwg@ietf.org
> > Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
> >
> > my issue was that I read the current text as talking about the result o=
f
> > the testing not the offered load
> >
> > so I was just going to suggest coming similar to what Al suggested
> >
> > but I=E2=80=99d add specific pointers to the explanation in the termino=
logy
> > document as to why
> >
> >
> > e.g.
> >
> >    For all tests, the test traffic generator sending rate MUST be
> >    less than or equal to 99.98% of the nominal value of Line Rate
> >    (with no further PPM adjustment to account for interface clock
> >    tolerances), to ensure stressing the DUT in reasonable worst case
> >    conditions. (see RFC XXX section ??? - note to RFC editor replace XX=
X
> >    with number of terminology RFC)
> >
> [ACM]
> good addition, Scott.
>
> > I have not reviewed the rest of the ID to see if similar misreading is
> > possible so that
> > a similar change would be useful
> [ACM]
> Let's assume for now that other Area reviewers and IESG review will
> catch some more of these, if needed.
>
> thanks,
> Al
>
> >
> > Scott
> >
> >
> > > On Jun 18, 2017, at 1:03 AM, MORTON, ALFRED C (AL) <acmorton@att.com>
> > wrote:
> > >
> > > Hi Lucien, Scott, and all,
> > >
> > >
> > >
> > > When I read this material, I assumed the goal was to
> > >
> > > limit the maximum sending rate of the test traffic generator
> > >
> > > in accordance with the Ethernet clock tolerances,
> > >
> > > effectively choosing the low-end of clock tolerance
> > >
> > > as the maximum sending rate with a factor of 2 margin.
> > >
> > > 99.98% of the nominal line rate would be -200 ppm,
> > >
> > > and any conforming interface clock should work fine.
> > >
> > >
> > >
> > > If I understood your goal correctly,
> > >
> > > I wonder if a small clarification will help future
> > >
> > > readers.
> > >
> > >
> > >
> > > When section 2.2 says:
> > >
> > >    For all tests, the percentage of traffic per port capacity sent
> > MUST
> > >
> > >                                                 ^^^^^^^^^^^^^^^^^^
> > >
> > >    be 99.98% at most, with no PPM adjustment to ensure stressing the
> > DUT
> > >
> > >       +++++++++++++++
> > >
> > >    in worst case conditions.
> > >
> > >
> > >
> > > You=E2=80=99ve defined the concept of Port Capacity in terminology Se=
ction 5,
> > >
> > > under the Line Rate definition, as
> > >
> > >
> > >
> > >    The term "port capacity" defines the maximum speed capability for
> > the
> > >
> > >    given port; for example 1GE, 10GE, 40GE, 100GE etc.
> > >
> > >
> > >
> > > Actually, the port capacity examples are the *nominal value of Line
> > Rate*,
> > >
> > > before the clock tolerances are considered.
> > >
> > >
> > >
> > > Here=E2=80=99s a suggestion for Section 2.2 of the meth:
> > >
> > >
> > >
> > >    For all tests, the test traffic generator sending rate MUST be
> > >
> > >    less than or equal to 99.98% of the nominal value of Line Rate
> > >
> > >    (with no further PPM adjustment to account for interface clock
> > >
> > >    tolerances), to ensure stressing the DUT in reasonable worst case
> > >
> > >    conditions.
> > >
> > >
> > >
> > > Changing
> > >
> > > s/port capacity/nominal value of Line Rate/
> > >
> > > here, requires an additional substitution in section 2.3 of the meth
> > draft,
> > >
> > > and a single substitution in section 5.1 of the term draft.
> > >
> > >
> > >
> > > Does this help?
> > >
> > > Al
> > >
> > >
> > >
> > >
> > >
> > > From: Lucien Avramov [mailto:lucienav@google.com]
> > > Sent: Saturday, June 17, 2017 1:49 PM
> > > To: Scott Bradner
> > > Cc: bmwg@ietf.org; draft-ietf-bmwg-dcbench-methodology.all@ietf.org;
> > bmwg-chairs@ietf.org
> > > Subject: Re: draft-ietf-bmwg-dcbench-methodology-07
> > >
> > >
> > >
> > > Hi Scott!
> > >
> > > =C2=B7         The draft you are reviewing is the ''methodology'' dra=
ft
> > which explains the methodology for line rate performance testing. The
> > exact reason onto why it is 99.98% with 0 PPM adjustment is covered
> > extensively in the terminology draft, section 5:
> > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> > 3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Ddcbench-2Dterminology-
> > 2D12-23section-2D5&d=3DDwIGaQ&c=3DLFYZ-
> > o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DcY2eIWfAaujzPr7A49lWCj=
uJ_Vx
> > 0ywhEDDmsWPBcTJA&s=3D96T3Kqxhm3c-gASDa1w_XDRAcXYh30m7fyjPEm28HCE&e=3D .=
 That
> > draft dependency is covered in the introduction and also in the
> > reference lists and is referenced throughout the document
> > >
> > > =C2=B7         There is a clock drift difference between tester and D=
UT
> > explained in section 5.2 of the terminology draft
> > >
> > > In our view BMWG has not morphed, our goal has been to be very precis=
e
> > and accurate with performance testing, and this is not about conformanc=
e
> > testing at all. These definitions that you see, have not changed in 4
> > years and have been proven themselves in a very large amount of testing
> > done since we started publishing.
> > >
> > >
> > >
> > > As you may remember, we have been working on this draft actively for
> > over 4 years now. We've incorporated numerous of your feedbacks on thes=
e
> > drafts over the IETF sessions (since IETF 86 - captured in the meeting
> > minutes of the IETF sessions).  On April 2016, we had your blessing on
> > the list also regarding our drafts.
> > >
> > >
> > >
> > > We appreciate as always your feedback, Scott!
> > >
> > >
> > >
> > > Lucien
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner <sob@sobco.com> wrote:
> > >
> > > try again - my new mac book often sends mail before I want it to
> > >
> > >
> > > I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcbench-
> > methodology-07
> > >
> > > I quickly became confused - a paragraph in section 2.2 reads
> > >
> > > For all tests, the percentage of traffic per port capacity sent MUST
> > >   be 99.98% at most, with no PPM adjustment to ensure stressing the
> > DUT
> > >   in worst case conditions. Tests results at a lower rate MAY be
> > >   provided for better understanding of performance increase in terms
> > of
> > >   latency and jitter when the rate is lower than 99.98%. The receivin=
g
> > >   rate of the traffic SHOULD be captured during this test in % of lin=
e
> > >   rate.
> > >
> > >
> > > Thus reads to me like a conformance test not a performance test
> > (saying what is =E2=80=9Cpassing=E2=80=9D)  unless
> > > BMWG has morphed a lot since I was last involved conformance and
> > pass/fail are not things that BMWG
> > > should be doing -
> > >
> > > what am I missing?
> > >
> > > Scott
> > >
> > >
> > >
> > >
> >
> > _______________________________________________
> > bmwg mailing list
> > bmwg@ietf.org
> > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> > 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwIGaQ&c=3DLFYZ-
> > o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DcY2eIWfAaujzPr7A49lWCj=
uJ_Vx
> > 0ywhEDDmsWPBcTJA&s=3DNwNczwTUN_yQqjl4Aw4jBTqk3iMbJpTrJMDiuFT8g20&e=3D
>

--f403045fdf8488f03a05523ef6ec
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Al, Scott<div><br></div><div>thanks for the comments, u=
pdated both drafts just now with all comments!</div><div><br></div><div>Tha=
nks,</div><div>Lucien</div></div><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Sun, Jun 18, 2017 at 9:51 AM, MORTON, ALFRED C (AL) <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:acmorton@att.com" target=3D"_blank">ac=
morton@att.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">addi=
ng OPS-DIR,<br>
<span class=3D"">&gt; -----Original Message-----<br>
&gt; From: bmwg [mailto:<a href=3D"mailto:bmwg-bounces@ietf.org">bmwg-bounc=
es@ietf.org</a>] On Behalf Of Scott Bradner<br>
&gt; Sent: Sunday, June 18, 2017 9:53 AM<br>
&gt; To: MORTON, ALFRED C (AL)<br>
</span><span class=3D"">&gt; Cc: <a href=3D"mailto:draft-ietf-bmwg-dcbench-=
methodology.all@ietf.org">draft-ietf-bmwg-dcbench-<wbr>methodology.all@ietf=
.org</a>; bmwg-<br>
&gt; <a href=3D"mailto:chairs@ietf.org">chairs@ietf.org</a>; <a href=3D"mai=
lto:bmwg@ietf.org">bmwg@ietf.org</a><br>
&gt; Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-<wbr>methodology-07<br>
&gt;<br>
&gt; my issue was that I read the current text as talking about the result =
of<br>
&gt; the testing not the offered load<br>
&gt;<br>
&gt; so I was just going to suggest coming similar to what Al suggested<br>
&gt;<br>
&gt; but I=E2=80=99d add specific pointers to the explanation in the termin=
ology<br>
&gt; document as to why<br>
&gt;<br>
&gt;<br>
&gt; e.g.<br>
&gt;<br>
&gt;=C2=A0 =C2=A0 For all tests, the test traffic generator sending rate MU=
ST be<br>
&gt;=C2=A0 =C2=A0 less than or equal to 99.98% of the nominal value of Line=
 Rate<br>
&gt;=C2=A0 =C2=A0 (with no further PPM adjustment to account for interface =
clock<br>
&gt;=C2=A0 =C2=A0 tolerances), to ensure stressing the DUT in reasonable wo=
rst case<br>
&gt;=C2=A0 =C2=A0 conditions. (see RFC XXX section ??? - note to RFC editor=
 replace XXX<br>
&gt;=C2=A0 =C2=A0 with number of terminology RFC)<br>
&gt;<br>
</span>[ACM]<br>
good addition, Scott.<br>
<span class=3D""><br>
&gt; I have not reviewed the rest of the ID to see if similar misreading is=
<br>
&gt; possible so that<br>
&gt; a similar change would be useful<br>
</span>[ACM]<br>
Let&#39;s assume for now that other Area reviewers and IESG review will<br>
catch some more of these, if needed.<br>
<br>
thanks,<br>
Al<br>
<div><div class=3D"h5"><br>
&gt;<br>
&gt; Scott<br>
&gt;<br>
&gt;<br>
&gt; &gt; On Jun 18, 2017, at 1:03 AM, MORTON, ALFRED C (AL) &lt;<a href=3D=
"mailto:acmorton@att.com">acmorton@att.com</a>&gt;<br>
&gt; wrote:<br>
&gt; &gt;<br>
&gt; &gt; Hi Lucien, Scott, and all,<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; When I read this material, I assumed the goal was to<br>
&gt; &gt;<br>
&gt; &gt; limit the maximum sending rate of the test traffic generator<br>
&gt; &gt;<br>
&gt; &gt; in accordance with the Ethernet clock tolerances,<br>
&gt; &gt;<br>
&gt; &gt; effectively choosing the low-end of clock tolerance<br>
&gt; &gt;<br>
&gt; &gt; as the maximum sending rate with a factor of 2 margin.<br>
&gt; &gt;<br>
&gt; &gt; 99.98% of the nominal line rate would be -200 ppm,<br>
&gt; &gt;<br>
&gt; &gt; and any conforming interface clock should work fine.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; If I understood your goal correctly,<br>
&gt; &gt;<br>
&gt; &gt; I wonder if a small clarification will help future<br>
&gt; &gt;<br>
&gt; &gt; readers.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; When section 2.2 says:<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 For all tests, the percentage of traffic per port ca=
pacity sent<br>
&gt; MUST<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^^^^^^^^^^^^^^^^^^<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 be 99.98% at most, with no PPM adjustment to ensure =
stressing the<br>
&gt; DUT<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0+++++++++++++++<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 in worst case conditions.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; You=E2=80=99ve defined the concept of Port Capacity in terminolog=
y Section 5,<br>
&gt; &gt;<br>
&gt; &gt; under the Line Rate definition, as<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 The term &quot;port capacity&quot; defines the maxim=
um speed capability for<br>
&gt; the<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 given port; for example 1GE, 10GE, 40GE, 100GE etc.<=
br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Actually, the port capacity examples are the *nominal value of Li=
ne<br>
&gt; Rate*,<br>
&gt; &gt;<br>
&gt; &gt; before the clock tolerances are considered.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Here=E2=80=99s a suggestion for Section 2.2 of the meth:<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 For all tests, the test traffic generator sending ra=
te MUST be<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 less than or equal to 99.98% of the nominal value of=
 Line Rate<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 (with no further PPM adjustment to account for inter=
face clock<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 tolerances), to ensure stressing the DUT in reasonab=
le worst case<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 conditions.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Changing<br>
&gt; &gt;<br>
&gt; &gt; s/port capacity/nominal value of Line Rate/<br>
&gt; &gt;<br>
&gt; &gt; here, requires an additional substitution in section 2.3 of the m=
eth<br>
&gt; draft,<br>
&gt; &gt;<br>
&gt; &gt; and a single substitution in section 5.1 of the term draft.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Does this help?<br>
&gt; &gt;<br>
&gt; &gt; Al<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; From: Lucien Avramov [mailto:<a href=3D"mailto:lucienav@google.co=
m">lucienav@google.com</a>]<br>
&gt; &gt; Sent: Saturday, June 17, 2017 1:49 PM<br>
&gt; &gt; To: Scott Bradner<br>
&gt; &gt; Cc: <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a>; <a href=
=3D"mailto:draft-ietf-bmwg-dcbench-methodology.all@ietf.org">draft-ietf-bmw=
g-dcbench-<wbr>methodology.all@ietf.org</a>;<br>
&gt; <a href=3D"mailto:bmwg-chairs@ietf.org">bmwg-chairs@ietf.org</a><br>
&gt; &gt; Subject: Re: draft-ietf-bmwg-dcbench-<wbr>methodology-07<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Hi Scott!<br>
&gt; &gt;<br>
&gt; &gt; =C2=B7=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0The draft you are reviewi=
ng is the &#39;&#39;methodology&#39;&#39; draft<br>
&gt; which explains the methodology for line rate performance testing. The<=
br>
&gt; exact reason onto why it is 99.98% with 0 PPM adjustment is covered<br=
>
&gt; extensively in the terminology draft, section 5:<br>
</div></div>&gt; <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dht=
tps-" rel=3D"noreferrer" target=3D"_blank">https://urldefense.proofpoint.<w=
br>com/v2/url?u=3Dhttps-</a><br>
&gt; 3A__tools.ietf.org_html_draft-<wbr>2Dietf-2Dbmwg-2Ddcbench-<wbr>2Dterm=
inology-<br>
&gt; 2D12-23section-2D5&amp;d=3DDwIGaQ&amp;c=3D<wbr>LFYZ-<br>
&gt; o9_HUMeMTSQicvjIg&amp;r=3D<wbr>OfsSu8kTIltVyD1oL72cBw&amp;m=3D<wbr>cY2=
eIWfAaujzPr7A49lWCjuJ_Vx<br>
&gt; 0ywhEDDmsWPBcTJA&amp;s=3D<wbr>96T3Kqxhm3c-gASDa1w_<wbr>XDRAcXYh30m7fyj=
PEm28HCE&amp;e=3D . That<br>
<div><div class=3D"h5">&gt; draft dependency is covered in the introduction=
 and also in the<br>
&gt; reference lists and is referenced throughout the document<br>
&gt; &gt;<br>
&gt; &gt; =C2=B7=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0There is a clock drift di=
fference between tester and DUT<br>
&gt; explained in section 5.2 of the terminology draft<br>
&gt; &gt;<br>
&gt; &gt; In our view BMWG has not morphed, our goal has been to be very pr=
ecise<br>
&gt; and accurate with performance testing, and this is not about conforman=
ce<br>
&gt; testing at all. These definitions that you see, have not changed in 4<=
br>
&gt; years and have been proven themselves in a very large amount of testin=
g<br>
&gt; done since we started publishing.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; As you may remember, we have been working on this draft actively =
for<br>
&gt; over 4 years now. We&#39;ve incorporated numerous of your feedbacks on=
 these<br>
&gt; drafts over the IETF sessions (since IETF 86 - captured in the meeting=
<br>
&gt; minutes of the IETF sessions).=C2=A0 On April 2016, we had your blessi=
ng on<br>
&gt; the list also regarding our drafts.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; We appreciate as always your feedback, Scott!<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Lucien<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner &lt;<a href=3D"mai=
lto:sob@sobco.com">sob@sobco.com</a>&gt; wrote:<br>
&gt; &gt;<br>
&gt; &gt; try again - my new mac book often sends mail before I want it to<=
br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcben=
ch-<br>
&gt; methodology-07<br>
&gt; &gt;<br>
&gt; &gt; I quickly became confused - a paragraph in section 2.2 reads<br>
&gt; &gt;<br>
&gt; &gt; For all tests, the percentage of traffic per port capacity sent M=
UST<br>
&gt; &gt;=C2=A0 =C2=A0be 99.98% at most, with no PPM adjustment to ensure s=
tressing the<br>
&gt; DUT<br>
&gt; &gt;=C2=A0 =C2=A0in worst case conditions. Tests results at a lower ra=
te MAY be<br>
&gt; &gt;=C2=A0 =C2=A0provided for better understanding of performance incr=
ease in terms<br>
&gt; of<br>
&gt; &gt;=C2=A0 =C2=A0latency and jitter when the rate is lower than 99.98%=
. The receiving<br>
&gt; &gt;=C2=A0 =C2=A0rate of the traffic SHOULD be captured during this te=
st in % of line<br>
&gt; &gt;=C2=A0 =C2=A0rate.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Thus reads to me like a conformance test not a performance test<b=
r>
&gt; (saying what is =E2=80=9Cpassing=E2=80=9D)=C2=A0 unless<br>
&gt; &gt; BMWG has morphed a lot since I was last involved conformance and<=
br>
&gt; pass/fail are not things that BMWG<br>
&gt; &gt; should be doing -<br>
&gt; &gt;<br>
&gt; &gt; what am I missing?<br>
&gt; &gt;<br>
&gt; &gt; Scott<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt;<br>
</div></div><span class=3D"">&gt; ______________________________<wbr>______=
___________<br>
&gt; bmwg mailing list<br>
&gt; <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
</span>&gt; <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-"=
 rel=3D"noreferrer" target=3D"_blank">https://urldefense.proofpoint.<wbr>co=
m/v2/url?u=3Dhttps-</a><br>
&gt; 3A__www.ietf.org_mailman_<wbr>listinfo_bmwg&amp;d=3DDwIGaQ&amp;c=3DLFY=
Z-<br>
&gt; o9_HUMeMTSQicvjIg&amp;r=3D<wbr>OfsSu8kTIltVyD1oL72cBw&amp;m=3D<wbr>cY2=
eIWfAaujzPr7A49lWCjuJ_Vx<br>
&gt; 0ywhEDDmsWPBcTJA&amp;s=3DNwNczwTUN_<wbr>yQqjl4Aw4jBTqk3iMbJpTrJMDiuFT8=
<wbr>g20&amp;e=3D<br>
</blockquote></div><br></div>

--f403045fdf8488f03a05523ef6ec--


From nobody Sun Jun 18 10:05:07 2017
Return-Path: <saurabhchattopadhya@hcl.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DDBED1294A2 for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 10:05:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2KfNyJRy6PNn for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 10:05:02 -0700 (PDT)
Received: from GWS05.hcl.com (gws05.avitas.hcl.com [203.105.185.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D76CD1294A6 for <bmwg@ietf.org>; Sun, 18 Jun 2017 10:05:00 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="5.39,357,1493663400";  d="scan'208,146";a="196323493"
Received: from unknown (HELO NDA-CORP-HT02.CORP.HCL.IN) ([10.248.64.34]) by GWS05.hcl.com with ESMTP/TLS/AES128-SHA; 18 Jun 2017 22:04:41 +0530
Received: from NDA-CORP-EDGE2.CORP.HCL.COM (10.248.64.160) by NDA-CORP-HT02.CORP.HCL.IN (10.248.64.34) with Microsoft SMTP Server (TLS) id 14.3.339.0; Sun, 18 Jun 2017 22:34:56 +0530
Received: from APC01-PU1-obe.outbound.protection.outlook.com (65.55.88.24) by transportedge.hcl.com (10.248.64.160) with Microsoft SMTP Server (TLS) id 14.3.339.0; Sun, 18 Jun 2017 22:34:41 +0530
Received: from PS1PR0401MB1594.apcprd04.prod.outlook.com (10.165.209.149) by PS1PR0401MB1594.apcprd04.prod.outlook.com (10.165.209.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14; Sun, 18 Jun 2017 17:04:53 +0000
Received: from PS1PR0401MB1594.apcprd04.prod.outlook.com ([fe80::ec14:a6e3:3a2c:5e39]) by PS1PR0401MB1594.apcprd04.prod.outlook.com ([fe80::ec14:a6e3:3a2c:5e39%13]) with mapi id 15.01.1178.016; Sun, 18 Jun 2017 17:04:53 +0000
From: "Saurabh Chattopadhyay - ERS, HCL Tech" <saurabhchattopadhya@hcl.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQLm9lPA
Date: Sun, 18 Jun 2017 17:04:53 +0000
Message-ID: <PS1PR0401MB15945C604A113D571107E2E4ACC70@PS1PR0401MB1594.apcprd04.prod.outlook.com>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: att.com; dkim=none (message not signed) header.d=none;att.com; dmarc=none action=none header.from=hcl.com;
x-originating-ip: [182.69.230.56]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; PS1PR0401MB1594; 7:OQlN9vJn8nF4r6gtehdecEJFjPKagHEuwvEKFaQ0S8CvqzilzuXmrZXYmCHg+qKBRLKyQ5itQI+V//S3rZNQdQW2pV3vicuqFJWM5LsFf2GIHcqpKf6tyHo17fhEGHwy3hXrflz9LddvN44JGAj/xHPSzhofWqUhnR+x9ln6GQNDg9zguTd4u7WCD+1DBw5YXY33QSIVY17ZagQdUbVQaQnCrzceM+bGBR6By2cZ3dv0K9pGMLpozXRS4TMAlAOIHI2PA+GcfXQiPa2Czh0iYBRxkRb+Hit424zkK8w79Y3MLS8XTFrLgD8JXuL5BCUK31/476b+0WgjfNKjIPFfjcXKH2TbxhPyaTAdtqtjizMXHxz2VA6NGiTAO5ywbUpzdyL+9X/C3+2T3E7pKCLunNWgpHb9wEwhaXKhcBmy8Xd8KoR1Z4ztgJyLgHn6Fvk1M8egiLllMPhvTyCC3rLH6vpFnUdT3OZQJDTZzELfs1C7TP2v/wsAYS1dWmjE/5FfI+lvpd2KZ5cA1jTM8SD19T/OrpWO/RvuMuJbmJN9xCDwfAbl4r7CsApPF//Oi5dcFgbnunlp0S5t4nPv0t1re57w6vEOHL9vQEm77qDWVqvNAyVf28BcdrCYZRI65MZd8WKTehEsBVoTKnFI+VDEMSl89I23z5QEGdL/3dNg7PORwgMOirYgnQnOO/pwnid34ekthRCWzVn57tE0ibjWkNM10d8u3LS/gsol+Q/X6s2lsYpnLMNVsL7a2foHyO5X3KUo21e9icNlfINkbmX0qT9ejJ2q/c4/tPj+aYWHCWo=
x-ms-office365-filtering-correlation-id: e5102c99-db55-48f4-de04-08d4b66c2371
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(201703131423075)(201703031133081)(201702281549075); SRVR:PS1PR0401MB1594; 
x-ms-traffictypediagnostic: PS1PR0401MB1594:
x-microsoft-antispam-prvs: <PS1PR0401MB15940CB732928B47CF5788F2ACC70@PS1PR0401MB1594.apcprd04.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:;
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(6041248)(20161123562025)(20161123560025)(20161123564025)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:PS1PR0401MB1594; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:PS1PR0401MB1594; 
x-forefront-prvs: 034215E98F
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39400400002)(39410400002)(39840400002)(39850400002)(39450400003)(377454003)(13464003)(9686003)(189998001)(5660300001)(66066001)(99286003)(305945005)(33656002)(3846002)(14454004)(6306002)(55016002)(2501003)(6116002)(2900100001)(2950100002)(81166006)(6436002)(102836003)(76176999)(50986999)(8936002)(86362001)(966005)(53546009)(5250100002)(53936002)(230783001)(25786009)(2906002)(6246003)(38730400002)(3280700002)(74316002)(7696004)(54356999)(7736002)(3660700001)(6506006)(8676002)(478600001); DIR:OUT; SFP:1102; SCL:1; SRVR:PS1PR0401MB1594; H:PS1PR0401MB1594.apcprd04.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jun 2017 17:04:53.1283 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 189de737-c93a-4f5a-8b68-6f4ca9941912
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PS1PR0401MB1594
X-OriginatorOrg: HCL.COM
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CqCB0YtoXu_2T5ijUGTF1ABNKsw>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 17:05:06 -0000

Support this adoption. The draft can be developed further to formalize the =
benchmarking methodologies for EVPN & PBB-EVPN, while extending from benchm=
arking methodologies of overlapped functionalities as defined in existing R=
FCs.

Regards,
Saurabh

-----Original Message-----
From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED C (AL)
Sent: Sunday, June 04, 2017 4:04 AM
To: bmwg@ietf.org
Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07

BMWG,

At our Chicago meeting, the BMWG reviewed a version of the Internet Draft on

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-05

There was sufficient readership present at the meeting and positive/constru=
ctive comments on the work, so the chairs asked whether support existed to =
adopt the draft as a WG item. There was support and no objections. As alway=
s, such decisions are officially taken on the mailing list.

This message*, therefore, starts a call for adoption on:

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-07

to run until Friday 23 June 2017. =


Please reply to bmwg@ietf.org indicating:

(1) whether you support addition of the following milestone to =

   the BMWG charter:

date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG

(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07 =

   as the basis document for this milestone

(3) whether you commit to reviewing the WG document if adopted

We are aware that BMWG has nearly completed all its existing chartered item=
s.  This usually begins discussion of re-chartering, this time with Warren =
(our new AD Advisor). The final decision/schedule for adoption will depend =
on these discussions. =


thanks for your consideration of the draft, and regards,

Al
bmwg co-chair

*using Brian Trammell's recent message to IPPM as a template (thanks!)

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


::DISCLAIMER::
---------------------------------------------------------------------------=
-------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and inte=
nded for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as informa=
tion could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in trans=
mission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability =
on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the =
author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, disse=
mination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written=
 consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please=
 delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses =
and other defects.

---------------------------------------------------------------------------=
-------------------------------------------------------------------------


From nobody Sun Jun 18 16:52:49 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A98B7124D37; Sun, 18 Jun 2017 16:52:40 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149782996056.10747.12049944084414492220@ietfa.amsl.com>
Date: Sun, 18 Jun 2017 16:52:40 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/qGoNQET8R9bF-Pw-5A7sWl3zGXA>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-12.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Jun 2017 23:52:41 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-12.txt
	Pages           : 19
	Date            : 2017-06-18

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. Many of these terms and methods
   may be applicable beyond this publication's scope as the technologies
   originally applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-12
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-12

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-12


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Sun Jun 18 18:46:55 2017
Return-Path: <rukeshd@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BE080126B6D for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 18:46:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.021
X-Spam-Level: 
X-Spam-Status: No, score=-2.021 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ur14MKLsMT2A for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 18:46:51 -0700 (PDT)
Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0138.outbound.protection.outlook.com [104.47.40.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DE280126B71 for <bmwg@ietf.org>; Sun, 18 Jun 2017 18:46:50 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=E8pjNbfxWsCVxiRQWgLhcf0GfOidVAARTNYFRLa94UM=; b=gQNVUpp3obko6VEWSCIi6TwiMaPLlZ04PTPtPPGqrATJKFU6b0X4LFSWL1o0mhwIzyQOkPbSXZWx22D+7KZwKtpobXUfe01+vWJ9vrDtI5Lmj49iL3ZujlvE1aY2eMf1/qqjBPC+6VnF7jgsz++65J/qKCwOljxOHFjfGZGrbMM=
Received: from MWHPR05MB2815.namprd05.prod.outlook.com (10.168.245.9) by MWHPR05MB2942.namprd05.prod.outlook.com (10.168.246.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Mon, 19 Jun 2017 01:46:49 +0000
Received: from MWHPR05MB2815.namprd05.prod.outlook.com ([10.168.245.9]) by MWHPR05MB2815.namprd05.prod.outlook.com ([10.168.245.9]) with mapi id 15.01.1199.007; Mon, 19 Jun 2017 01:46:49 +0000
From: Rukesh D <rukeshd@juniper.net>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQLm9lPAABKVUnA=
Date: Mon, 19 Jun 2017 01:46:49 +0000
Message-ID: <MWHPR05MB28153250C6BFD657F291C9A0A2C40@MWHPR05MB2815.namprd05.prod.outlook.com>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com> <PS1PR0401MB15945C604A113D571107E2E4ACC70@PS1PR0401MB1594.apcprd04.prod.outlook.com>
In-Reply-To: <PS1PR0401MB15945C604A113D571107E2E4ACC70@PS1PR0401MB1594.apcprd04.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: att.com; dkim=none (message not signed) header.d=none;att.com; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [66.129.239.14]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MWHPR05MB2942; 7:EM4Gg7j1uIGmcc7ELsrV3VapqqakqJX1E7RvKiwMHwfeoYppS7YgFIPzAp79F4A98V6NP+LSAGTDKlDokFxh9DXFz53LAsK54VjJ/04gP3+WRl3Q1JlGvqSDs4cOWPPQJZoAw4JVsagQjXjWjn7+vwGSll6LYon8OZClWslHXa3gkKhDPbkBiip0O6nHfgFQYZ/FhEkmfUicGunONsuLrAKs58+Q5AKpv4ClZw9IV6F0Urnc8Y0FWgH9w4dQQj8pZ+zZPJVPICxziLvCQ8e8GD4SZ9/yMDjkT8Nyg79RrNrBAUumMx6rx/4OAktbllX1XXcWSRvI39eLoZymC0tGd82xom7ZwPRRzqXG/EfTcJ9VWe6ef1xuoAGjePV55QJ7WqPqqthe9+kJqsJeY7jfzr0QS+ofx/FRbQ7jQG4M7TUNqSq+LOYWG52yYxVHAJSZF3Bwln0SEihZyFTXr1mLcPsKzG5aF5sbySRTlO/xmev70oOd4TvMShkgOuymgK3/x1pJIClgafGEe6n5WAac1EGVn6FvpcS/TMYzGEjKdh+vCcYfT5KotSaFyPKvBcUnfif3X4AH6HN3/YO6UuBI1vxBmS45fIO4ZrdIRI1N0baUhFrWH2727GExEFMQ8sr5KOxqyIOBw5mouUU/f13qGSTfOMj0gV3im0QrH2xp98c5O/aoW4NNsicAUR4k5GdcChTudMzJ5hPEcOLgTqaAF+3f+aBvkXbklKoYEs3t1FfPgAoGyntRMAQaZV0UMBK0jwliDdvU/8ySO4r+6rJ3bgy6ODfMWUKwRUdt+G7lLW0=
x-ms-office365-filtering-correlation-id: 25850842-ebd7-46b6-f7e1-08d4b6b50d5e
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500041)(300135000095)(300000501041)(300135300095)(22001)(300000502041)(300135100095)(2017030254075)(300000503041)(300135400095)(48565401081)(201703131423075)(201703031133081)(201702281549075)(300000504041)(300135200095)(300000505041)(300135600095)(300000506037)(300135500095); SRVR:MWHPR05MB2942; 
x-ms-traffictypediagnostic: MWHPR05MB2942:
x-microsoft-antispam-prvs: <MWHPR05MB29422DD88893E8CABB2CBDD0A2C40@MWHPR05MB2942.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:;
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123558100)(20161123564025)(20161123555025)(20161123560025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR05MB2942; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR05MB2942; 
x-forefront-prvs: 0343AC1D30
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39860400002)(39410400002)(39400400002)(39850400002)(39450400003)(39840400002)(38564003)(377454003)(13464003)(305945005)(66066001)(7696004)(230783001)(8676002)(81166006)(5660300001)(7736002)(8936002)(3846002)(6116002)(102836003)(76176999)(54356999)(2906002)(50986999)(3660700001)(3280700002)(53546009)(6506006)(74316002)(6436002)(2900100001)(55016002)(2950100002)(122556002)(25786009)(189998001)(53936002)(6306002)(38730400002)(33656002)(9686003)(5890100001)(86362001)(229853002)(99286003)(966005)(2501003)(77096006)(478600001)(14454004); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR05MB2942; H:MWHPR05MB2815.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jun 2017 01:46:49.5121 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB2942
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CiPiI9Oc8Bmfb1iwMw0-sw0cIKA>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 19 Jun 2017 01:46:54 -0000

This draft covers all the important benchmarking methodologies for EVPN. Su=
pport this for adoption.

Thanks,
Rukesh
-----Original Message-----
From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED C (AL=
)
Sent: Sunday, June 04, 2017 4:04 AM
To: bmwg@ietf.org
Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07

BMWG,

At our Chicago meeting, the BMWG reviewed a version of the Internet Draft o=
n

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-05

There was sufficient readership present at the meeting and positive/constru=
ctive comments on the work, so the chairs asked whether support existed to =
adopt the draft as a WG item. There was support and no objections. As alway=
s, such decisions are officially taken on the mailing list.

This message*, therefore, starts a call for adoption on:

Benchmarking Methodology for EVPN and PBB-EVPN
draft-kishjac-bmwg-evpntest-07

to run until Friday 23 June 2017.=20

Please reply to bmwg@ietf.org indicating:

(1) whether you support addition of the following milestone to=20
   the BMWG charter:

date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG

(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07=20
   as the basis document for this milestone

(3) whether you commit to reviewing the WG document if adopted

We are aware that BMWG has nearly completed all its existing chartered item=
s.  This usually begins discussion of re-chartering, this time with Warren =
(our new AD Advisor). The final decision/schedule for adoption will depend =
on these discussions.=20

thanks for your consideration of the draft, and regards,

Al
bmwg co-chair

*using Brian Trammell's recent message to IPPM as a template (thanks!)

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


::DISCLAIMER::
---------------------------------------------------------------------------=
-------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and inte=
nded for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as informa=
tion could be intercepted, corrupted, lost, destroyed, arrive late or incom=
plete, or may contain viruses in transmission. The e mail and its contents =
(with or without referred errors) shall therefore not attach any liability =
on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the =
author and may not necessarily reflect the views or opinions of HCL or its =
affiliates. Any form of reproduction, dissemination, copying, disclosure, m=
odification, distribution and / or publication of this message without the =
prior written consent of authorized representative of HCL is strictly prohi=
bited. If you have received this email in error please delete it and notify=
 the sender immediately.
Before opening any email and/or attachments, please check them for viruses =
and other defects.

---------------------------------------------------------------------------=
-------------------------------------------------------------------------

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


From nobody Sun Jun 18 22:53:01 2017
Return-Path: <bill.wu@huawei.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E008128B88 for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 22:53:00 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.22
X-Spam-Level: 
X-Spam-Status: No, score=-4.22 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8-7CU3IhUgk2 for <bmwg@ietfa.amsl.com>; Sun, 18 Jun 2017 22:52:57 -0700 (PDT)
Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 602CF128B8D for <bmwg@ietf.org>; Sun, 18 Jun 2017 22:52:56 -0700 (PDT)
Received: from 172.18.7.190 (EHLO lhreml702-cah.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id DIT49795; Mon, 19 Jun 2017 05:52:54 +0000 (GMT)
Received: from NKGEML413-HUB.china.huawei.com (10.98.56.74) by lhreml702-cah.china.huawei.com (10.201.108.43) with Microsoft SMTP Server (TLS) id 14.3.301.0; Mon, 19 Jun 2017 06:52:45 +0100
Received: from NKGEML513-MBX.china.huawei.com ([169.254.1.25]) by NKGEML413-HUB.china.huawei.com ([10.98.56.74]) with mapi id 14.03.0235.001; Mon, 19 Jun 2017 13:52:28 +0800
From: Qin Wu <bill.wu@huawei.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: Sudhin Jacob <sjacob@juniper.net>
Thread-Topic: RE: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLowDvPXIyG+SRaTPO7O2uavuR4DA==
Date: Mon, 19 Jun 2017 05:52:28 +0000
Message-ID: <B8F9A780D330094D99AF023C5877DABA9A961079@nkgeml513-mbx.china.huawei.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.136.78.218]
Content-Type: multipart/alternative; boundary="_000_B8F9A780D330094D99AF023C5877DABA9A961079nkgeml513mbxchi_"
MIME-Version: 1.0
X-CFilter-Loop: Reflected
X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020202.594766B6.00F0, ss=1, re=0.000, recu=0.000, reip=0.000,  cl=1, cld=1, fgs=0, ip=169.254.1.25, so=2013-06-18 04:22:30, dmn=2013-03-21 17:37:32
X-Mirapoint-Loop-Id: e6ff820794f8b9edc32dab8033cb8858
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/x34mE28DuAhyM9Meh0Br-25ul3k>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 19 Jun 2017 05:53:00 -0000

--_000_B8F9A780D330094D99AF023C5877DABA9A961079nkgeml513mbxchi_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Support adoption of this draft.



Qin

-----Original Message-----

From: bmwg [mailto:bmwg-bounces at ietf.org] On Behalf Of MORTON, ALFRED C =
(AL)

Sent: Sunday, June 04, 2017 4:04 AM

To: bmwg at ietf.org

Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07



BMWG,



At our Chicago meeting, the BMWG reviewed a version of the Internet Draft o=
n



Benchmarking Methodology for EVPN and PBB-EVPN

draft-kishjac-bmwg-evpntest-05



There was sufficient readership present at the meeting and positive/constru=
ctive comments on the work, so the chairs asked whether support existed to =
adopt the draft as a WG item. There was support and no objections. As alway=
s, such decisions are officially taken on the mailing list.



This message*, therefore, starts a call for adoption on:



Benchmarking Methodology for EVPN and PBB-EVPN

draft-kishjac-bmwg-evpntest-07



to run until Friday 23 June 2017.



Please reply to bmwg at ietf.org indicating:



(1) whether you support addition of the following milestone to

   the BMWG charter:



date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG



(2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07

   as the basis document for this milestone



(3) whether you commit to reviewing the WG document if adopted



We are aware that BMWG has nearly completed all its existing chartered item=
s.  This usually begins discussion of re-chartering, this time with Warren =
(our new AD Advisor). The final decision/schedule for adoption will depend =
on these discussions.



thanks for your consideration of the draft, and regards,



Al

bmwg co-chair



*using Brian Trammell's recent message to IPPM as a template (thanks!)



_______________________________________________

bmwg mailing list

bmwg at ietf.org

https://www.ietf.org/mailman/listinfo/bmwg






--_000_B8F9A780D330094D99AF023C5877DABA9A961079nkgeml513mbxchi_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:\5B8B\4F53;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@\5B8B\4F53";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	text-align:justify;
	text-justify:inter-ideograph;
	font-size:10.5pt;
	font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
pre
	{mso-style-priority:99;
	mso-style-link:"HTML \9884\8BBE\683C\5F0F Char";
	margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:\5B8B\4F53;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
span.HTMLChar
	{mso-style-name:"HTML \9884\8BBE\683C\5F0F Char";
	mso-style-priority:99;
	mso-style-link:"HTML \9884\8BBE\683C\5F0F";
	font-family:\5B8B\4F53;}
.MsoChpDefault
	{mso-style-type:export-only;}
/* Page Definitions */
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"ZH-CN" link=3D"blue" vlink=3D"purple" style=3D"text-justify-t=
rim:punctuation">
<div class=3D"WordSection1">
<pre><span lang=3D"EN-US">Support adoption of this draft.<o:p></o:p></span>=
</pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">Qin<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US">-----Original Message-----<o:p></o:p></span></pre=
>
<pre><span lang=3D"EN-US">From: bmwg [<a href=3D"mailto:bmwg-bounces">mailt=
o:bmwg-bounces</a> at ietf.org] On Behalf Of MORTON, ALFRED C (AL)<o:p></o:=
p></span></pre>
<pre><span lang=3D"EN-US">Sent: Sunday, June 04, 2017 4:04 AM<o:p></o:p></s=
pan></pre>
<pre><span lang=3D"EN-US">To: bmwg at ietf.org<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US">Subject: [bmwg] Call For Adoption: draft-kishjac-=
bmwg-evpntest-07<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">BMWG,<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">At our Chicago meeting, the BMWG reviewed a versi=
on of the Internet Draft on<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">Benchmarking Methodology for EVPN and PBB-EVPN<o:=
p></o:p></span></pre>
<pre><span lang=3D"EN-US">draft-kishjac-bmwg-evpntest-05<o:p></o:p></span><=
/pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">There was sufficient readership present at the me=
eting and positive/constructive comments on the work, so the chairs asked w=
hether support existed to adopt the draft as a WG item. There was support a=
nd no objections. As always, such decisions are officially taken on the mai=
ling list.<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">This message*, therefore, starts a call for adopt=
ion on:<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">Benchmarking Methodology for EVPN and PBB-EVPN<o:=
p></o:p></span></pre>
<pre><span lang=3D"EN-US">draft-kishjac-bmwg-evpntest-07<o:p></o:p></span><=
/pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">to run until Friday 23 June 2017. <o:p></o:p></sp=
an></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">Please reply to bmwg at ietf.org indicating:<o:p>=
</o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">(1) whether you support addition of the following=
 milestone to <o:p></o:p></span></pre>
<pre><span lang=3D"EN-US">&nbsp;&nbsp;&nbsp;the BMWG charter:<o:p></o:p></s=
pan></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">date TBD: Benchmarking Methodology for EVPN and P=
BB-EVPN to the IESG<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">(2) whether you support the adoption of draft-kis=
hjac-bmwg-evpntest-07 <o:p></o:p></span></pre>
<pre><span lang=3D"EN-US">&nbsp;&nbsp;&nbsp;as the basis document for this =
milestone<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">(3) whether you commit to reviewing the WG docume=
nt if adopted<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">We are aware that BMWG has nearly completed all i=
ts existing chartered items.&nbsp; This usually begins discussion of re-cha=
rtering, this time with Warren (our new AD Advisor). The final decision/sch=
edule for adoption will depend on these discussions. <o:p></o:p></span></pr=
e>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">thanks for your consideration of the draft, and r=
egards,<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">Al<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US">bmwg co-chair<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">*using Brian Trammell's recent message to IPPM as=
 a template (thanks!)<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US">_______________________________________________<o=
:p></o:p></span></pre>
<pre><span lang=3D"EN-US">bmwg mailing list<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US">bmwg at ietf.org<o:p></o:p></span></pre>
<pre><span lang=3D"EN-US"><a href=3D"https://www.ietf.org/mailman/listinfo/=
bmwg">https://www.ietf.org/mailman/listinfo/bmwg</a><o:p></o:p></span></pre=
>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<pre><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></pre>
<p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p>&nbsp;</o:p></span></p>
</div>
</body>
</html>

--_000_B8F9A780D330094D99AF023C5877DABA9A961079nkgeml513mbxchi_--


From nobody Mon Jun 19 02:32:40 2017
Return-Path: <dromasca@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 433F112943A; Mon, 19 Jun 2017 02:32:38 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Dan Romascanu <dromasca@gmail.com>
To: <gen-art@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology.all@ietf.org, ietf@ietf.org, bmwg@ietf.org, dromasca@gmail.com
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149786475813.10715.10157714153398648057@ietfa.amsl.com>
Date: Mon, 19 Jun 2017 02:32:38 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/tiXo4o0dQT7V3Ecmek3yDGYGa0s>
Subject: [bmwg] Genart telechat review of draft-ietf-bmwg-dcbench-methodology-12
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 19 Jun 2017 09:32:38 -0000

Reviewer: Dan Romascanu
Review result: Ready

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair. Please wait for direction from your
document shepherd or AD before posting a new version of the draft.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-dcbench-methodology-??
Reviewer: Dan Romascanu
Review Date: 2017-06-19
IETF LC End Date: 2017-06-13
IESG Telechat date: 2017-06-22

Summary:

This Informational I-D describes test and evaluation methodology and
measurement techniques for physical network equipment in the data center. It's
an useful and clearly written document and it is ready for publication. A few
minor issues and nits that were brought up in my LC review were addressed by
the authors - thanks!

Major issues:

Minor issues:

Nits/editorial comments:



From nobody Mon Jun 19 14:30:58 2017
Return-Path: <aretana@cisco.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id AFEE91201F8; Mon, 19 Jun 2017 14:30:51 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alvaro Retana <aretana@cisco.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-terminology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149790785163.10643.5101366875633212921.idtracker@ietfa.amsl.com>
Date: Mon, 19 Jun 2017 14:30:51 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/100RKtoCGH61d37OFYLJzmXJbWA>
Subject: [bmwg] Alvaro Retana's No Objection on draft-ietf-bmwg-dcbench-terminology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 19 Jun 2017 21:30:52 -0000

Alvaro Retana has entered the following ballot position for
draft-ietf-bmwg-dcbench-terminology-13: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I agree with Mirja.  Given that the text (in the Abstract) already recognize
the broader applicability, I would like to see that reflected in a clear way on
the title and the contents.



From nobody Mon Jun 19 14:32:25 2017
Return-Path: <aretana@cisco.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E7DE1201F8; Mon, 19 Jun 2017 14:32:22 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Alvaro Retana <aretana@cisco.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com>
Date: Mon, 19 Jun 2017 14:32:22 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/TnOmDkK_nKmicmM7YKBPydSPaAA>
Subject: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 19 Jun 2017 21:32:22 -0000

Alvaro Retana has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-12: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

This document contains both the RFC2119 boilerplate and specific definitions
for MUST, SHOULD/RECOMMENDED and MAY. Even though the additional definitions
are close enough to RFC2119, I think that one of them should be taken off to
avoid any type of confusion.

Note that the (new) definitions in this document are focused around "metrics",
which would not apply to text such as (from 2.2) "A traffic generator SHOULD be
connected to all ports on the DUT. Two tests MUST be conducted...".


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I support Mirja's DISCUSS.  The authors recognize that the applicability of the
tests described may go beyond the DC, so it should be reflected appropriately
(starting with the title of the document).  From the Abstract:

   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. Many of these terms and methods
   may be applicable beyond this publication's scope as the technologies
   originally applied in the data center are deployed elsewhere.



From nobody Mon Jun 19 18:14:55 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF0C8131868 for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:14:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xsSn7KXSlUBY for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:14:44 -0700 (PDT)
Received: from mail-pg0-x22f.google.com (mail-pg0-x22f.google.com [IPv6:2607:f8b0:400e:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 666D5131869 for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:14:44 -0700 (PDT)
Received: by mail-pg0-x22f.google.com with SMTP id f185so55641274pgc.0 for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:14:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vfqrLEDdElap3Mz/cSKhvEDVqOpBLJ4efL0Bp7XFRjU=; b=IXmzHZlE10xNRCF4Z7eSlcI+pR8cG16Vs74aevoIPsdQfxxTxU9SiiQ+jR5A/fgneU 6t8DBNTd2QsqGHdjv2ZXsPfJC9IVL7cSmoI7VHgDubuksgtPGClqWVT3eJgOgGJOq0JE PxTWtk1G91EMRMsxkuJrBGIQ0f0pawqamrkFVNEzbxGjaXv2p5JNxAopG3+O14+WGTMh NBxNXesQ6DcK23mtKFoI+6aXlw0gvdt5imZgJ46uESzaPcBNjjsAPzGD7oiyFVsjjiCE Jh+0ygdSsiE734RIXpP3f8Tzx6dUhkca1NX3ekFih1mr2Tds7WFGK/VhtkRz1CGGW8yx Pc/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vfqrLEDdElap3Mz/cSKhvEDVqOpBLJ4efL0Bp7XFRjU=; b=Bc6oZt/hue2EAQtp35yPl2VB+GiRWTOfy0mxiJdGHSTbDocaIyMd1NKdch8k56+0Ec npaHLySLh5hD30tQikCKZGz8mpLIHlLDCDnjffEM81O6AQc6cgeJZ4mQJhJREHSxMw0l GIqUP4e7CAhy259Wk/49LY3O3xhOP7gFfpOHkvW49QNrUZWl1cLaRy2xMcoZwcgQ+JDW Sf0KhCVLEyuON2ad2j4CmdPSUc7plCqJO7NqodXySeeb0iczzEHRWEthaWElgg+B7C6b 1PsG8LzLg4mRLMnbFUVlr4wTi7ZU3f9c8gt5ZFZLi9vWi6YnoTaa76OlWo6/lNwDrapI 4DqQ==
X-Gm-Message-State: AKS2vOxPR20z4Oh+JtS10hC152wFa6E1wu+E/xjN+EyIuDlhFLsK6w+Z GJbD/0qGUOCbNIampB0T8ENe5j1WxiRP
X-Received: by 10.99.106.2 with SMTP id f2mr29594120pgc.46.1497921283799; Mon, 19 Jun 2017 18:14:43 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Mon, 19 Jun 2017 18:14:23 -0700 (PDT)
In-Reply-To: <149786475813.10715.10157714153398648057@ietfa.amsl.com>
References: <149786475813.10715.10157714153398648057@ietfa.amsl.com>
From: Lucien Avramov <lucienav@google.com>
Date: Mon, 19 Jun 2017 18:14:23 -0700
Message-ID: <CALTEt=DJJmHMnOY4puDB+Qf9q2Rm_-Pj52VS2vwJq=sLqGVoew@mail.gmail.com>
To: Dan Romascanu <dromasca@gmail.com>
Cc: gen-art <gen-art@ietf.org>, draft-ietf-bmwg-dcbench-methodology.all@ietf.org,  ietf <ietf@ietf.org>, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c13f244069026055259fa2b"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/HOBQhZ1Eic1NauaQFtZxcdq7xCg>
Subject: Re: [bmwg] Genart telechat review of draft-ietf-bmwg-dcbench-methodology-12
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 01:14:47 -0000

--94eb2c13f244069026055259fa2b
Content-Type: text/plain; charset="UTF-8"

Thank you Dan for your review and support!

On Mon, Jun 19, 2017 at 2:32 AM, Dan Romascanu <dromasca@gmail.com> wrote:

> Reviewer: Dan Romascanu
> Review result: Ready
>
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair. Please wait for direction from your
> document shepherd or AD before posting a new version of the draft.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-bmwg-dcbench-methodology-??
> Reviewer: Dan Romascanu
> Review Date: 2017-06-19
> IETF LC End Date: 2017-06-13
> IESG Telechat date: 2017-06-22
>
> Summary:
>
> This Informational I-D describes test and evaluation methodology and
> measurement techniques for physical network equipment in the data center.
> It's
> an useful and clearly written document and it is ready for publication. A
> few
> minor issues and nits that were brought up in my LC review were addressed
> by
> the authors - thanks!
>
> Major issues:
>
> Minor issues:
>
> Nits/editorial comments:
>
>
>

--94eb2c13f244069026055259fa2b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thank you Dan for your review and support!</div><div class=
=3D"gmail_extra"><br><div class=3D"gmail_quote">On Mon, Jun 19, 2017 at 2:3=
2 AM, Dan Romascanu <span dir=3D"ltr">&lt;<a href=3D"mailto:dromasca@gmail.=
com" target=3D"_blank">dromasca@gmail.com</a>&gt;</span> wrote:<br><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc s=
olid;padding-left:1ex">Reviewer: Dan Romascanu<br>
Review result: Ready<br>
<br>
I am the assigned Gen-ART reviewer for this draft. The General Area<br>
Review Team (Gen-ART) reviews all IETF documents being processed<br>
by the IESG for the IETF Chair. Please wait for direction from your<br>
document shepherd or AD before posting a new version of the draft.<br>
<br>
For more information, please see the FAQ at<br>
<br>
&lt;<a href=3D"https://trac.ietf.org/trac/gen/wiki/GenArtfaq" rel=3D"norefe=
rrer" target=3D"_blank">https://trac.ietf.org/trac/<wbr>gen/wiki/GenArtfaq<=
/a>&gt;.<br>
<br>
Document: draft-ietf-bmwg-dcbench-<wbr>methodology-??<br>
Reviewer: Dan Romascanu<br>
Review Date: 2017-06-19<br>
IETF LC End Date: 2017-06-13<br>
IESG Telechat date: 2017-06-22<br>
<br>
Summary:<br>
<br>
This Informational I-D describes test and evaluation methodology and<br>
measurement techniques for physical network equipment in the data center. I=
t&#39;s<br>
an useful and clearly written document and it is ready for publication. A f=
ew<br>
minor issues and nits that were brought up in my LC review were addressed b=
y<br>
the authors - thanks!<br>
<br>
Major issues:<br>
<br>
Minor issues:<br>
<br>
Nits/editorial comments:<br>
<br>
<br>
</blockquote></div><br></div>

--94eb2c13f244069026055259fa2b--


From nobody Mon Jun 19 18:16:55 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E7BBE127B60 for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:16:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mgyiQEIsekhR for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:16:47 -0700 (PDT)
Received: from mail-pg0-x22c.google.com (mail-pg0-x22c.google.com [IPv6:2607:f8b0:400e:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2C98E13186B for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:16:45 -0700 (PDT)
Received: by mail-pg0-x22c.google.com with SMTP id f185so55661790pgc.0 for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:16:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ba/5yClDbw03z0u+v7MVqVKh1F1MbzBWGkP3F526p9c=; b=F21ptqBvNh0EBgN4/8RBgMIT2yjyUbQAWsSg5D9rWRdODRCW5UThCgtyMvDpkbYlOe d6XHinzR7kOYwt8M58O1y4xperATasj7rWzJt0qdvL6+7Jwd+Sicpxu0RbrAYiOMEXtw odTK0BqqZeJrvjxQHZOWvQGZVyxKAbD/1u+Z8NuCuYYeWWLgqWFwljQO21G+3rT0c6SF 9N1Za2lx6IrWVfgs7OkhHWeJR9pdW9lJGFL70egRMHPgWDwGeMq0CH/5JuM/HazL+tow G+44dveo0Z0je+viYZzvx7rJx705OOW/VlSojP07tptSs9X/9Yiv5omdVxCeGpHHB85C UHPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ba/5yClDbw03z0u+v7MVqVKh1F1MbzBWGkP3F526p9c=; b=iv5vU+Xro4XFKsJ3ojaYkCHz1zToWSZnFBO/hEYkf2CC/GyHZAlnCGaDPELZL0D1eB 8yNG1rh8T4+nR4QPLA5M7o4sC5qlGOxPZlYHfXxgm0CDpo9tSHO2iSH4MMRsEjFkRzZO UyglbJvxt6/Q2FbRxEVJE0J7lwMFtFP2YkYmaTXunQ39gTOzmdhiHnoYddNA99YGnthc gDP6hxNUVRbBW6LAw125sgg1O0laEstLCETCTZ3ewz+yz4BdHrdkZ8Svkp3krYgHOMWo XDgtM64V5qX2aXyDNdHLQzA5bWcdkAvZE2yGq46p5jWYt58hlUDzOeNBLgegTgHgaX2F ZnqA==
X-Gm-Message-State: AKS2vOyPmF+2hQGUxQhK7bqAPJ5vBrpLJ4K/UQQelhdw9ipa2x3+ymTX RvcUe1zDte5Y2t0G68Wgn6LxF4EQBYPO
X-Received: by 10.84.224.199 with SMTP id k7mr2144572pln.207.1497921404550; Mon, 19 Jun 2017 18:16:44 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Mon, 19 Jun 2017 18:16:24 -0700 (PDT)
In-Reply-To: <149790785163.10643.5101366875633212921.idtracker@ietfa.amsl.com>
References: <149790785163.10643.5101366875633212921.idtracker@ietfa.amsl.com>
From: Lucien Avramov <lucienav@google.com>
Date: Mon, 19 Jun 2017 18:16:24 -0700
Message-ID: <CALTEt=A_KM1oAXLORcYP+vYbrDijpKHsP=XgVExo-Bm8fBxYSQ@mail.gmail.com>
To: Alvaro Retana <aretana@cisco.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="f4030436191c3893b905525a01f6"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/-yXk8Y_1YPN1BzN-yvgMvp4Rgus>
Subject: Re: [bmwg] Alvaro Retana's No Objection on draft-ietf-bmwg-dcbench-terminology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 01:16:49 -0000

--f4030436191c3893b905525a01f6
Content-Type: text/plain; charset="UTF-8"

Hi Alvaro,

Thank you for the comment. We (authors and chair) agree all with Mirja and
made an effort to clarify the text in the outline and in the content (the
latest version reflect those changes).

I object to changing the tittle, this is the essence of this testing and
want to keep it as it. Our efforts to clarify the publication content and
tests to perform clarify the ask.

Thanks for the feedback.

Cheers,
Lucien

On Mon, Jun 19, 2017 at 2:30 PM, Alvaro Retana <aretana@cisco.com> wrote:

> Alvaro Retana has entered the following ballot position for
> draft-ietf-bmwg-dcbench-terminology-13: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> I agree with Mirja.  Given that the text (in the Abstract) already
> recognize
> the broader applicability, I would like to see that reflected in a clear
> way on
> the title and the contents.
>
>
>

--f4030436191c3893b905525a01f6
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Alvaro,=C2=A0<div><br></div><div>Thank you for the comm=
ent. We (authors and chair) agree all with Mirja and made an effort to clar=
ify the text in the outline and in the content (the latest version reflect =
those changes).=C2=A0</div><div><br></div><div>I object to changing the tit=
tle, this is the essence of this testing and want to keep it as it. Our eff=
orts to clarify the publication content and tests to perform clarify the as=
k.=C2=A0</div><div><br></div><div>Thanks for the feedback.</div><div><br></=
div><div>Cheers,</div><div>Lucien</div></div><div class=3D"gmail_extra"><br=
><div class=3D"gmail_quote">On Mon, Jun 19, 2017 at 2:30 PM, Alvaro Retana =
<span dir=3D"ltr">&lt;<a href=3D"mailto:aretana@cisco.com" target=3D"_blank=
">aretana@cisco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quo=
te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=
>Alvaro Retana has entered the following ballot position for<br>
draft-ietf-bmwg-dcbench-<wbr>terminology-13: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-termino=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>terminology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
I agree with Mirja.=C2=A0 Given that the text (in the Abstract) already rec=
ognize<br>
the broader applicability, I would like to see that reflected in a clear wa=
y on<br>
the title and the contents.<br>
<br>
<br>
</blockquote></div><br></div>

--f4030436191c3893b905525a01f6--


From nobody Mon Jun 19 18:18:14 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F36E113186B for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:18:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A_TFAWYO0Mtp for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:18:10 -0700 (PDT)
Received: from mail-pf0-x232.google.com (mail-pf0-x232.google.com [IPv6:2607:f8b0:400e:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 91BB713186E for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:18:10 -0700 (PDT)
Received: by mail-pf0-x232.google.com with SMTP id c73so1598863pfk.2 for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:18:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=E0eXy7nHJi1MAu9+VdPYkjkFd8dHxR2aYpuxdYI8wpI=; b=uCg5B0G1KWYvt+/A62wqbOJW88N+Z4klM80sYsKsyXBMYSpWI9bd4L/FKl10CLjmTd YvQtPeoeDzqdqfSp8p3qU1zTe1GtnzsMN7Bd8QbO6XEmLOxQLLhzL40P71x1/RnPLrgI yZMbmJ5Vn1mU/P5FLY9I3TmSjK9lg57+5/yV/K4QMehEUq5D8+yfdoNYT5LI/RyL5TD0 TvyS57UYxmwhCgnMao+Pino0B+tKpO3Q3cKPF2XLeXqO7ETfK5knphoXEst5D8IlWlYA S0AXNzuETRRAANbLVTzkrHLDLfs3X3Xf6XwmYCjoCRJ9dnOXv33Q0HDYJeFv8377fb0K QZRg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=E0eXy7nHJi1MAu9+VdPYkjkFd8dHxR2aYpuxdYI8wpI=; b=ajL9rxqTRFA6GuhqCjVHZBWH6eaGMT1/Y5khEuzoo0moan6OEqTTE2F2mNyRd1rNtK pFdkAe3QDITD+MLacrbznYB1YUFDykZ/fz3lQ0h40GhGSdMLh2kO7HQNrLg53qfXJYFR 5HoONVbAjGRf3tnjvJayg0mK6DQB2PULlAjNjixzzlJcM3TOiyVFn9K+cvRZN7jd0Ey+ bMl1Ano406va5BpnGJEODjg+nEUOiLDAaLe+lbeK7f4X+sQ3GzOp2pG5kRCCv6eGWxtQ N/RHUXI/RcGIoT0L6XaXoIVbpr7pZr+rHsvicZCOuIwG2F4rSHXPUF0aWkmnZK9bx1+l Jlcg==
X-Gm-Message-State: AKS2vOx7exemaHUbDMR5FGyx9ByrmrO8SJsFSK0zCH5A9AnvCpxaXdM6 /9dGy4O6HsYSlETnmepBFrP6LUiHetom
X-Received: by 10.99.51.1 with SMTP id z1mr4318322pgz.227.1497921490069; Mon, 19 Jun 2017 18:18:10 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Mon, 19 Jun 2017 18:17:49 -0700 (PDT)
In-Reply-To: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com>
References: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com>
From: Lucien Avramov <lucienav@google.com>
Date: Mon, 19 Jun 2017 18:17:49 -0700
Message-ID: <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com>
To: Alvaro Retana <aretana@cisco.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c0d957c51d08205525a0670"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/xXcFWWqdf8DO-1CWW6lbdsUuKHA>
Subject: Re: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 01:18:13 -0000

--94eb2c0d957c51d08205525a0670
Content-Type: text/plain; charset="UTF-8"

Hi Alvaro,

In the context of the tests we describe in this publication, it's key to
keep the words we have chosen, therefore I don't plan to make further
changes on that.

Thanks!
Lucien

On Mon, Jun 19, 2017 at 2:32 PM, Alvaro Retana <aretana@cisco.com> wrote:

> Alvaro Retana has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-12: Discuss
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>
>
>
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> This document contains both the RFC2119 boilerplate and specific
> definitions
> for MUST, SHOULD/RECOMMENDED and MAY. Even though the additional
> definitions
> are close enough to RFC2119, I think that one of them should be taken off
> to
> avoid any type of confusion.
>
> Note that the (new) definitions in this document are focused around
> "metrics",
> which would not apply to text such as (from 2.2) "A traffic generator
> SHOULD be
> connected to all ports on the DUT. Two tests MUST be conducted...".
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> I support Mirja's DISCUSS.  The authors recognize that the applicability
> of the
> tests described may go beyond the DC, so it should be reflected
> appropriately
> (starting with the title of the document).  From the Abstract:
>
>    The purpose of this informational document is to establish test and
>    evaluation methodology and measurement techniques for physical
>    network equipment in the data center. Many of these terms and methods
>    may be applicable beyond this publication's scope as the technologies
>    originally applied in the data center are deployed elsewhere.
>
>
>

--94eb2c0d957c51d08205525a0670
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Alvaro,=C2=A0<div><br></div><div>In the context of the =
tests we describe in this publication, it&#39;s key to keep the words we ha=
ve chosen, therefore I don&#39;t plan to make further changes on that.</div=
><div><br></div><div>Thanks!</div><div>Lucien</div><div class=3D"gmail_extr=
a"><br><div class=3D"gmail_quote">On Mon, Jun 19, 2017 at 2:32 PM, Alvaro R=
etana <span dir=3D"ltr">&lt;<a href=3D"mailto:aretana@cisco.com" target=3D"=
_blank">aretana@cisco.com</a>&gt;</span> wrote:<br><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex">Alvaro Retana has entered the following ballot position for<br>
draft-ietf-bmwg-dcbench-<wbr>methodology-12: Discuss<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodo=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>methodology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
DISCUSS:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
This document contains both the RFC2119 boilerplate and specific definition=
s<br>
for MUST, SHOULD/RECOMMENDED and MAY. Even though the additional definition=
s<br>
are close enough to RFC2119, I think that one of them should be taken off t=
o<br>
avoid any type of confusion.<br>
<br>
Note that the (new) definitions in this document are focused around &quot;m=
etrics&quot;,<br>
which would not apply to text such as (from 2.2) &quot;A traffic generator =
SHOULD be<br>
connected to all ports on the DUT. Two tests MUST be conducted...&quot;.<br=
>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
I support Mirja&#39;s DISCUSS.=C2=A0 The authors recognize that the applica=
bility of the<br>
tests described may go beyond the DC, so it should be reflected appropriate=
ly<br>
(starting with the title of the document).=C2=A0 From the Abstract:<br>
<br>
=C2=A0 =C2=A0The purpose of this informational document is to establish tes=
t and<br>
=C2=A0 =C2=A0evaluation methodology and measurement techniques for physical=
<br>
=C2=A0 =C2=A0network equipment in the data center. Many of these terms and =
methods<br>
=C2=A0 =C2=A0may be applicable beyond this publication&#39;s scope as the t=
echnologies<br>
=C2=A0 =C2=A0originally applied in the data center are deployed elsewhere.<=
br>
<br>
<br>
</blockquote></div><br></div></div>

--94eb2c0d957c51d08205525a0670--


From nobody Mon Jun 19 18:29:45 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D65741318A1 for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:29:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PaW_Xdtkm-2y for <bmwg@ietfa.amsl.com>; Mon, 19 Jun 2017 18:29:34 -0700 (PDT)
Received: from mail-pg0-x235.google.com (mail-pg0-x235.google.com [IPv6:2607:f8b0:400e:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 048161318A9 for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:29:33 -0700 (PDT)
Received: by mail-pg0-x235.google.com with SMTP id e187so3443126pgc.1 for <bmwg@ietf.org>; Mon, 19 Jun 2017 18:29:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Rug11WbhigLQDUfItamseJM3v4nQ/CbmxXbkiekxDn4=; b=oa64aJrfxZEr8Q1CoirJ/rLkETNgOKJqPCjvMaEmOZYvUhBNL5Tf2ANIaucqkhMMEp Hp3f/Aa0nM79zVLtVMgn9likqdTwMJEAFtDMMzgVTxqm2KptFBkE5uhKOoepHchLEKcT qK5bqkS5oAB4N6mJMJvlEOmEONOVWVpKwgg43E3pdUKc/P9hQHcBIqlzWG48kU3NC7cS K8CYn9IGfmo5/81qOqKtkRIaoycVPEKVh543QbeWbPDeDJlyTAY9FJ+nUDgkpYy2izf1 hl2xK2Sh+eKAX6erRQ5YISnDNNfI2JNchlOR/1fAOMEglsdhFUWMZ3s8uG9krC6ayx77 7Z2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Rug11WbhigLQDUfItamseJM3v4nQ/CbmxXbkiekxDn4=; b=L9vRU/Rg8PpuZi/xO6irj0yAjjO/+qqymdArhPbf1kxT/YEJ6nCh0M9rU7hNUFxpXM zYU46CSTg5yZnn1WTStguMyGE2D345G7NGTeha7K5QHXsi3+A8Gobi2rrjjqArxbqwTO lvnlv6huq8chr0Akg+s1URz7vLYwn4QGhUZ464oy4+URio8mPK0EWUIL4izngbaxf5dw QvKlmxNUGTtWUAOgGGhcIzzD+M9KOIG08CPwii021osP09XpMRcqDiLqKoolH2nuibA0 ii8S132/Yphu4uiNq29U8F++6NAN0vxCW/vkfKiFBCqn7DBkzj/ozrgTW9M/4Y9Bg+Qy aoSg==
X-Gm-Message-State: AKS2vOzT7Le1y+uc9PNejEL+Iu0P92gYh0aMrULjQ6cZPCv6XXKDKXIl o9JOJi9bLv7wk9UBeu3AGLgPQlU8R1mc
X-Received: by 10.98.92.3 with SMTP id q3mr7911139pfb.65.1497922173372; Mon, 19 Jun 2017 18:29:33 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Mon, 19 Jun 2017 18:29:12 -0700 (PDT)
In-Reply-To: <149744977404.13951.15551847308245216174.idtracker@ietfa.amsl.com>
References: <149744977404.13951.15551847308245216174.idtracker@ietfa.amsl.com>
From: Lucien Avramov <lucienav@google.com>
Date: Mon, 19 Jun 2017 18:29:12 -0700
Message-ID: <CALTEt=DdJ-_h7Pmd+r94Gac1dL28J=pAG6FY2jbprS2yWAOP9g@mail.gmail.com>
To: =?UTF-8?Q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c03e1d40c207e05525a2f5c"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/4u3Hq3K2CismrzX2S9Kbo6CEAbI>
Subject: Re: [bmwg]  =?utf-8?q?Mirja_K=C3=BChlewind=27s_No_Objection_on_draft-?= =?utf-8?q?ietf-bmwg-dcbench-terminology-10=3A_=28with_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 01:29:37 -0000

--94eb2c03e1d40c207e05525a2f5c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Mirja,

I believe we have resolved all the comments and suggestions so far.

Can you please let us know what you think?

Thanks!
Lucien

On Wed, Jun 14, 2017 at 7:16 AM, Mirja K=C3=BChlewind <ietf@kuehlewind.net>
wrote:

> Mirja K=C3=BChlewind has entered the following ballot position for
> draft-ietf-bmwg-dcbench-terminology-10: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> I didn't put in a discuss because it=E2=80=99s not worth holding up the d=
ocument
> as the
> content seems fine, BUT I really don't see why this terminology is
> specified
> FOR data centers ONLY. All terminology seem to be applicable in general
> and as
> such this seems to be an extension of RFC1242. Even incast, which is
> actually a
> data center term, is so generally defined that one could use it also in a
> different context as well. Further latency is defined in both documents,
> where
> this drafts mentions RFC1242 and mainly extends the description there.
> That's
> in general fine, but I probably would recommend to simply change the titl=
e,
> remove the work data center there, and make this document update RFC 1242=
.
> (+
> remove the intro text on data centers because the same text is anyway in
> draft-ietf-bmwg-dcbench-methodology).
>
>
>

--94eb2c03e1d40c207e05525a2f5c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Mirja,<div><br></div><div>I believe we have resolved al=
l the comments and suggestions so far.=C2=A0</div><div><br></div><div>Can y=
ou please let us know what you think?</div><div><br></div><div>Thanks!</div=
><div>Lucien</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_=
quote">On Wed, Jun 14, 2017 at 7:16 AM, Mirja K=C3=BChlewind <span dir=3D"l=
tr">&lt;<a href=3D"mailto:ietf@kuehlewind.net" target=3D"_blank">ietf@kuehl=
ewind.net</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div cla=
ss=3D"HOEnZb"><div class=3D"h5">Mirja K=C3=BChlewind has entered the follow=
ing ballot position for<br>
draft-ietf-bmwg-dcbench-<wbr>terminology-10: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-termino=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>terminology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
I didn&#39;t put in a discuss because it=E2=80=99s not worth holding up the=
 document as the<br>
content seems fine, BUT I really don&#39;t see why this terminology is spec=
ified<br>
FOR data centers ONLY. All terminology seem to be applicable in general and=
 as<br>
such this seems to be an extension of RFC1242. Even incast, which is actual=
ly a<br>
data center term, is so generally defined that one could use it also in a<b=
r>
different context as well. Further latency is defined in both documents, wh=
ere<br>
this drafts mentions RFC1242 and mainly extends the description there. That=
&#39;s<br>
in general fine, but I probably would recommend to simply change the title,=
<br>
</div></div>remove the work data center there, and make this document updat=
e RFC 1242. (+<br>
remove the intro text on data centers because the same text is anyway in<br=
>
draft-ietf-bmwg-dcbench-<wbr>methodology).<br>
<br>
<br>
</blockquote></div><br></div>

--94eb2c03e1d40c207e05525a2f5c--


From nobody Tue Jun 20 05:35:57 2017
Return-Path: <sjacob@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A6A7E12EB12 for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 05:35:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.021
X-Spam-Level: 
X-Spam-Status: No, score=-2.021 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LqBH9nt0ksKD for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 05:35:45 -0700 (PDT)
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0106.outbound.protection.outlook.com [104.47.36.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 84FC112EB06 for <bmwg@ietf.org>; Tue, 20 Jun 2017 05:35:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ISlabjNbgKFytOnD0DHPwCEbc9ufwsfiHVCLgYmBSow=; b=LH4g4l1sXuYE+ID9sFfa99PUmohbNTZtUrTymbUkZPM8fz4lnPIOIO+poATp6e5I17vGpfmrHY5P5NPlS0Z5AjKOO6rurB8RmqNbBJOe5Z7uZ/CeHwXUk6bcQmBrEz+lVE5WaqOgI2/X4Es/m/4DUCOH7MeXTorDzpySmMO7kQY=
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com (10.160.200.24) by BY1PR0501MB1525.namprd05.prod.outlook.com (10.160.203.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Tue, 20 Jun 2017 12:35:22 +0000
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) by BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) with mapi id 15.01.1199.012; Tue, 20 Jun 2017 12:35:22 +0000
From: Sudhin Jacob <sjacob@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: REG:draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLpwPH4qBN4PZCDQI2/HEM5GQVahg==
Date: Tue, 20 Jun 2017 12:35:22 +0000
Message-ID: <BY1PR0501MB13524A41D0E1B196F1E9E97AC2C50@BY1PR0501MB1352.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.11]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BY1PR0501MB1525; 7:sy2CSkPrjZ13M9DNS1Xy8Noy6i6Kqs3p1QDBwa5mceSDc+IzxUm6h8+v/SzRrnMBflgfsjPMN2qJpaWtKTwKrM47OfW9s/F5gilyVLrcGtbnxSoCguJNddKQYrsAIfZ61fu6zrH2jfMOlSF0ZaEZbrQPM3PXA50WBdbeq0DC3gi1nxPxk8k6pqGmohOU85pidx6JWQJ96DOT9qvXixCDWeJFfczsZAg7lRcp63kH7Y42bAQxHl4ka50zQsO5/Gtcu+IZLuUQHD7WUrrZKFF1vP2DeKtt/ZQIksop6XJlSHFbWFGSUfLcVZYqix/27ZUSRj2c9E0uC9BhW1l2H2BeJ+lyqhPpA3icQS1u1p/gqh12l18HwBvt+/CM56Sai9Wy1a+Hc6akLWczIzSdI/mwQyGNEMn6D8BPAj8YvuwAHtX/6yvmTv1eaEJr6PPJBqCOIdFf9nB2CY1OShmVl3QKyPSyNkoig7lrd9mlO3fTvKD3fMT+me4NbDKzIjb3UMpKRrfEFxvivBCGQAsQ7OAPQXaq2XL7mYpELGPFWoh3G8XjS9sxN1pkbRxwUpDevS35OJD6BMjQ59nWboK5ScZZ/IDJNyRqqVm5RyC4+wjfILpWUh87IkpaRUfu73HYIAYfg09SxHfJjILrLbVoIt6rfaNC8zYT4tx3kCD2adB6nTRrugLawiwxJhZHCFh+KYbUzsz5t8l82Wr2yVIDHr5q14LDXDHXUNMhayXFodeSim4rg0cZFbucMTy3Mc6Pzo9JE2dRrmEUI0Pd/Bb72Jj9XAAK5EDzAUFsYQwYf4sKwyg=
x-ms-office365-filtering-correlation-id: 9440aa4b-2e59-4c41-fc74-08d4b7d8d17e
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500055)(300135000095)(300000501055)(300135300095)(300000502055)(300135100095)(22001)(2017030254075)(300000503055)(300135400095)(48565401081)(201703131423075)(201703031133081)(201702281549075)(300000504055)(300135200095)(300000505055)(300135600095)(300000506048)(300135500095); SRVR:BY1PR0501MB1525; 
x-ms-traffictypediagnostic: BY1PR0501MB1525:
x-microsoft-antispam-prvs: <BY1PR0501MB1525537F68AD0F9ECBED279CC2C50@BY1PR0501MB1525.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(3002001)(6055026)(6041248)(20161123558100)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY1PR0501MB1525; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY1PR0501MB1525; 
x-forefront-prvs: 03449D5DD1
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39840400002)(39860400002)(39410400002)(39400400002)(39850400002)(39450400003)(53754006)(102836003)(3660700001)(6116002)(3280700002)(2351001)(6916009)(3846002)(33656002)(2501003)(230783001)(2900100001)(25786009)(2906002)(14454004)(189998001)(8936002)(38730400002)(110136004)(7696004)(81166006)(6436002)(122556002)(8676002)(1730700003)(9686003)(66066001)(99286003)(77096006)(55016002)(6306002)(53936002)(305945005)(478600001)(6506006)(54356999)(50986999)(966005)(74316002)(5660300001)(5640700003); DIR:OUT; SFP:1102; SCL:1; SRVR:BY1PR0501MB1525; H:BY1PR0501MB1352.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_BY1PR0501MB13524A41D0E1B196F1E9E97AC2C50BY1PR0501MB1352_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jun 2017 12:35:22.0724 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0501MB1525
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/A_ALCug9PYq_jZGH8_IKucUFFjs>
Subject: [bmwg] REG:draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 12:35:57 -0000

--_000_BY1PR0501MB13524A41D0E1B196F1E9E97AC2C50BY1PR0501MB1352_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi All,

Let me thank you all for supporting this draft.  I take the opportunity to =
thank everyone for their valuable feedback. As a an author I support this d=
raft for adoption.
I humbly request workgroup chair for adopting this draft.

Regards,
Sudhin


--_000_BY1PR0501MB13524A41D0E1B196F1E9E97AC2C50BY1PR0501MB1352_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;">
<div>Hi All,</div>
<div>&nbsp;</div>
<div>Let me thank you all for supporting this draft.&nbsp; I take the oppor=
tunity to thank everyone for their valuable feedback. As a an author I supp=
ort this draft for adoption.</div>
<div>I humbly request workgroup chair for adopting this draft.</div>
<div>&nbsp;</div>
<div>Regards,</div>
<div>Sudhin</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_BY1PR0501MB13524A41D0E1B196F1E9E97AC2C50BY1PR0501MB1352_--


From nobody Tue Jun 20 08:49:51 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E0B1131B15; Tue, 20 Jun 2017 08:49:50 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149797379047.25071.8756000179040082044@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 08:49:50 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/FS_fnQxZeWtgrQk_eEAMEfRVyK0>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-13.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 15:49:50 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-13.txt
	Pages           : 20
	Date            : 2017-06-20

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. Many of these terms and methods
   may be applicable beyond this publication's scope as the technologies
   originally applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-13
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-13

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-13


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Jun 20 08:50:31 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5B17A131BBA; Tue, 20 Jun 2017 08:50:22 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149797382233.24998.5873925255849449371@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 08:50:22 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/G_M1dbiacv1aiqDA7PC9SIYksNc>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-14.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 15:50:22 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-14.txt
	Pages           : 17
	Date            : 2017-06-20

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center. Many of
these terms and methods may be applicable beyond this publication's
scope as the technologies originally applied in the data center are
deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-14
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-14

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-14


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Jun 20 09:35:51 2017
Return-Path: <aretana@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 707881293D9; Tue, 20 Jun 2017 09:35:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.522
X-Spam-Level: 
X-Spam-Status: No, score=-14.522 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id twe2p0iPrZrp; Tue, 20 Jun 2017 09:35:42 -0700 (PDT)
Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4D249131B9D; Tue, 20 Jun 2017 09:35:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=5832; q=dns/txt; s=iport; t=1497976527; x=1499186127; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=asYkaCZc1XyMjcaegvPj75kFvmKteTxdpbd+uRd9VO4=; b=ZrccDElIrU5YSPAu6mIW0HhU1MHOABntTlR3WOmuEHZxkJB6+BNF712R NopuQUOX5RcWYKEOs0cbSOpM2sPTHK6rzR5/lBvuMpLL4pIGZkvVMJPgP /oq6c0i3XhPxzX/bhCH5x+5CDU4kztTji3DHxif8ljGea5+by58uXIvJY M=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0B1AQAdTklZ/4cNJK1dGgEBAQECAQEBA?= =?us-ascii?q?QgBAQEBgm9pYoENB4NkihmRf5BOhSqBMgNchiQCGoJMPxgBAgEBAQEBAQFrKIU?= =?us-ascii?q?ZAQQBI1YFCwIBCD8DAgICMBQRAgQOBYlIXAisDYImKosvAQEBAQEBAQEBAQEBA?= =?us-ascii?q?QEBAQEBAQEBHYZsgWArgnmHezCCMQWXHIdFApNgkg6VDAEfOEw+dBVbAYUvgU1?= =?us-ascii?q?2iEyBDQEBAQ?=
X-IronPort-AV: E=Sophos;i="5.39,364,1493683200";  d="scan'208,217";a="260263500"
Received: from alln-core-2.cisco.com ([173.36.13.135]) by rcdn-iport-4.cisco.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 20 Jun 2017 16:35:26 +0000
Received: from XCH-ALN-005.cisco.com (xch-aln-005.cisco.com [173.36.7.15]) by alln-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id v5KGZQ3E031579 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 20 Jun 2017 16:35:26 GMT
Received: from xch-aln-002.cisco.com (173.36.7.12) by XCH-ALN-005.cisco.com (173.36.7.15) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Tue, 20 Jun 2017 11:35:25 -0500
Received: from xch-aln-002.cisco.com ([173.36.7.12]) by XCH-ALN-002.cisco.com ([173.36.7.12]) with mapi id 15.00.1210.000; Tue, 20 Jun 2017 11:35:25 -0500
From: "Alvaro Retana (aretana)" <aretana@cisco.com>
To: Lucien Avramov <lucienav@google.com>
CC: The IESG <iesg@ietf.org>, "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
Thread-Index: AQHS6UORi83ftabEaky9N6fUtwLWHqItR1qAgAEh4wA=
Date: Tue, 20 Jun 2017 16:35:25 +0000
Message-ID: <D78D2206-5A27-40E3-9E29-8C4A8D1ABD01@cisco.com>
References: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com> <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com>
In-Reply-To: <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/f.20.0.170309
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.61.241.68]
Content-Type: multipart/alternative; boundary="_000_D78D22065A2740E39E298C4A8D1ABD01ciscocom_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2Bol5-u6-SkHnrHx_blXbrGojNo>
Subject: Re: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 16:35:44 -0000

--_000_D78D22065A2740E39E298C4A8D1ABD01ciscocom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

T24gNi8yMC8xNywgMzoxNyBBTSwgIkx1Y2llbiBBdnJhbW92IiA8bHVjaWVuYXZAZ29vZ2xlLmNv
bTxtYWlsdG86bHVjaWVuYXZAZ29vZ2xlLmNvbT4+IHdyb3RlOg0KDQpMdWNpZW46DQoNCkhpIQ0K
DQo+IEluIHRoZSBjb250ZXh0IG9mIHRoZSB0ZXN0cyB3ZSBkZXNjcmliZSBpbiB0aGlzIHB1Ymxp
Y2F0aW9uLCBpdCdzIGtleSB0byBrZWVwDQo+IHRoZSB3b3JkcyB3ZSBoYXZlIGNob3NlbuKApg0K
DQpJIGRvbuKAmXQgaGF2ZSBhIHByb2JsZW0gd2l0aCB0aGUgd29yZHMgdGhlbXNlbHZlcyDigJMg
dGhlIHBvaW50IG9mIG15IERJU0NVU1MgaXMgdGhlIGZhY3QgdGhhdCBlYWNoIHdvcmQgKOKAnE1V
U1TigJ0sIGZvciBleGFtcGxlKSBoYXMgdHdvIGRpZmZlcmVudCBkZWZpbml0aW9ucy4gIFllcywg
dGhleSBhcmUgY2xvc2UsIGJ1dCBkaWZmZXJlbnQuICBJbiBmYWN0LCB0aGV5IGFyZSBzbyBjbG9z
ZSB0aGF0IEkgdGhpbmsgeW91IGNvdWxkIHRha2Ugb3V0IHRoZSBkZWZpbml0aW9ucyBpbiAxLjIg
YW5kIG5vdCBsb3NlIGFueXRoaW5nLg0KDQpSZWdhcmRzLA0KDQpBbHZhcm8uDQo=

--_000_D78D22065A2740E39E298C4A8D1ABD01ciscocom_
Content-Type: text/html; charset="utf-8"
Content-ID: <C38C13CE25C932418049EE5F11621722@emea.cisco.com>
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4
bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo
dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo
dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp
dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l
dGEgbmFtZT0iVGl0bGUiIGNvbnRlbnQ9IiI+DQo8bWV0YSBuYW1lPSJLZXl3b3JkcyIgY29udGVu
dD0iIj4NCjxtZXRhIG5hbWU9IkdlbmVyYXRvciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUg
KGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxlPjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8N
CkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0
IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJ
cGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8N
CnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsN
CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWls
eToiVGltZXMgTmV3IFJvbWFuIjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1z
dHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxp
bmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1w
cmlvcml0eTo5OTsNCgljb2xvcjpwdXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9
DQpzcGFuLkVtYWlsU3R5bGUxNw0KCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbC1yZXBseTsNCglm
b250LWZhbWlseTpDYWxpYnJpOw0KCWNvbG9yOndpbmRvd3RleHQ7DQoJZm9udC13ZWlnaHQ6bm9y
bWFsOw0KCWZvbnQtc3R5bGU6bm9ybWFsO30NCnNwYW4ubXNvSW5zDQoJe21zby1zdHlsZS10eXBl
OmV4cG9ydC1vbmx5Ow0KCW1zby1zdHlsZS1uYW1lOiIiOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRl
cmxpbmU7DQoJY29sb3I6dGVhbDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlwZTpl
eHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtz
aXplOjguNWluIDExLjBpbjsNCgltYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW47fQ0KZGl2
LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQotLT48L3N0eWxlPg0KPC9oZWFk
Pg0KPGJvZHkgYmdjb2xvcj0id2hpdGUiIGxhbmc9IkVOLVVTIiBsaW5rPSJibHVlIiB2bGluaz0i
cHVycGxlIj4NCjxkaXYgY2xhc3M9IldvcmRTZWN0aW9uMSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSI+T24gNi8yMC8xNywgMzoxNyBBTSwg
JnF1b3Q7THVjaWVuIEF2cmFtb3YmcXVvdDsgJmx0OzxhIGhyZWY9Im1haWx0bzpsdWNpZW5hdkBn
b29nbGUuY29tIj5sdWNpZW5hdkBnb29nbGUuY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48
L3NwYW4+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250
LWZhbWlseTpDYWxpYnJpIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSI+THVjaWVuOjxvOnA+
PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250
LWZhbWlseTpDYWxpYnJpIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSI+SGkhPG86cD48L286
cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFt
aWx5OkNhbGlicmkiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGlicmk7Y29sb3I6Ymxh
Y2siPiZndDsgSW4gdGhlIGNvbnRleHQgb2YgdGhlIHRlc3RzIHdlIGRlc2NyaWJlIGluIHRoaXMg
cHVibGljYXRpb24sIGl0J3Mga2V5IHRvIGtlZXANCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJpO2NvbG9y
OmJsYWNrIj4mZ3Q7IHRoZSB3b3JkcyB3ZSBoYXZlIGNob3NlbuKApjxvOnA+PC9vOnA+PC9zcGFu
PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxp
YnJpO2NvbG9yOmJsYWNrIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaTtjb2xvcjpibGFjayI+
SSBkb27igJl0IGhhdmUgYSBwcm9ibGVtIHdpdGggdGhlIHdvcmRzIHRoZW1zZWx2ZXMg4oCTIHRo
ZSBwb2ludCBvZiBteSBESVNDVVNTIGlzIHRoZSBmYWN0IHRoYXQgZWFjaCB3b3JkICjigJxNVVNU
4oCdLCBmb3IgZXhhbXBsZSkgaGFzIHR3byBkaWZmZXJlbnQgZGVmaW5pdGlvbnMuJm5ic3A7IFll
cywgdGhleSBhcmUgY2xvc2UsIGJ1dCBkaWZmZXJlbnQuJm5ic3A7DQogSW4gZmFjdCwgdGhleSBh
cmUgc28gY2xvc2UgdGhhdCBJIHRoaW5rIHlvdSBjb3VsZCB0YWtlIG91dCB0aGUgZGVmaW5pdGlv
bnMgaW4gMS4yIGFuZCBub3QgbG9zZSBhbnl0aGluZy4NCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJpO2Nv
bG9yOmJsYWNrIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaTtjb2xvcjpibGFjayI+UmVnYXJk
cyw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls
ZT0iZm9udC1mYW1pbHk6Q2FsaWJyaTtjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3Nw
YW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNh
bGlicmk7Y29sb3I6YmxhY2siPkFsdmFyby48L3NwYW4+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5
OkNhbGlicmkiPjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1s
Pg0K

--_000_D78D22065A2740E39E298C4A8D1ABD01ciscocom_--


From nobody Tue Jun 20 10:05:10 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C950131A60 for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 08:57:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZE__RZ2yemhV for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 08:57:39 -0700 (PDT)
Received: from mail-pg0-x22b.google.com (mail-pg0-x22b.google.com [IPv6:2607:f8b0:400e:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 743F0131B03 for <bmwg@ietf.org>; Tue, 20 Jun 2017 08:55:44 -0700 (PDT)
Received: by mail-pg0-x22b.google.com with SMTP id e187so12024897pgc.1 for <bmwg@ietf.org>; Tue, 20 Jun 2017 08:55:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=3q3sslFXB2SkqqLSCP/EJLbo48XuTjU2SNOdF+HRBtk=; b=b+dNNS24bNsxA1xGXVDUNICkGMerHXpXrwYn0S9LGpPoQeDZYImkaTQ/Zw8UNqm9M6 SU2tKMeTRI9AeZnvq8Cb5BZ6ZoYaCY7JWEXo584pMPdn9MOOWwu+oxNYKm4jdxQD+hjB HOeHwvESNGJMxnqiQOxjKXIaop1U7bSBuH3tTz6noml3/pEVDoiEB6oArfFdPT4x/m4M 2y25r46GckzoH23nGQkdGOAYr5QN3BcisbzZMMJEoSHsK42HUI4DIAVotaTBhlbp3UGa dZR8GtmI1nKGjOspiQCo01VggLLaCtRFlluvz1sGO8ZeAL0CY+XJ3gzyLOSSpcULpcbj t3SQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=3q3sslFXB2SkqqLSCP/EJLbo48XuTjU2SNOdF+HRBtk=; b=XiiH6vOKQdpsObFkJezsbinJXZdFBU07m5wljh4TEmGrSYYGp1DA8Hm39w69rdDtFE J45FmFAZIBSlltxwYFAuYN2l2fVV4QF81qyz7XUAPBLKpFFfd2HM5rZq3MNBGfVIOWzG 0ylyw5g6ecivsDGMvTeu+5Bc+kxymrtKnpHo8jgMH/DlIjU5W085Ziethp2SEBlT55VY N+XuHL49xEbsJzLltNyRIjrRX65Uu15JfveQbAZ3q4CEezOkNCAlzPHMwis1C/3fZ6un ohndRaMUc1xTtFaFHZScCxuij/+VEBMYhhCiPsdQEhy7Nl2OJBNT9CJDOAaehgOU1hGz figA==
X-Gm-Message-State: AKS2vOz3FivIrzqSum1iBQKT4cZa6Hm1/MFABbpwJHo36wdUCiguTNPh kJyHkDjK4q9NVAcxaqavStiw2i1EvXj2
X-Received: by 10.98.92.3 with SMTP id q3mr11191183pfb.65.1497974143593; Tue, 20 Jun 2017 08:55:43 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Tue, 20 Jun 2017 08:55:23 -0700 (PDT)
In-Reply-To: <CALTEt=ByHULaoXkpKTr30fcCcCnGzNeFTr+fVqfrgVV-Rr1euw@mail.gmail.com>
References: <E10D882D-B040-4CFF-BCC6-052EEB2548B7@sobco.com> <9C9527FA-33F1-47B2-B1AD-F17BCA10DB71@sobco.com> <CALTEt=DdAh5No3cXJzHQ_XGza8tijUATT7DctkE2VUi_gkEGhg@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF25FDF58D@njmtexg5.research.att.com> <C7765B7F-6B7F-45C4-AFC6-A636E8B34810@sobco.com> <4D7F4AD313D3FC43A053B309F97543CF25FDF6E2@njmtexg5.research.att.com> <CALTEt=ByHULaoXkpKTr30fcCcCnGzNeFTr+fVqfrgVV-Rr1euw@mail.gmail.com>
From: Lucien Avramov <lucienav@google.com>
Date: Tue, 20 Jun 2017 08:55:23 -0700
Message-ID: <CALTEt=AH=+UrrU+GBS9OJsF5mRY2-Bq1C4Ji27u5ge_FOceMyw@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, =?UTF-8?Q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>,  Adam Montville <adam.w.montville@gmail.com>, Dan Romascanu <dromasca@gmail.com>, Jacob Rapp <jrapp@vmware.com>, Sarah B <sbanks@encrypted.net>
Cc: Scott Bradner <sob@sobco.com>, "ops-dir@ietf.org" <ops-dir@ietf.org>,  "draft-ietf-bmwg-dcbench-methodology.all@ietf.org" <draft-ietf-bmwg-dcbench-methodology.all@ietf.org>,  "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c03e1d4b6af2c05526648f1"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/B8wntkywaOepZF7IuwFEAoUok2s>
X-Mailman-Approved-At: Tue, 20 Jun 2017 10:05:08 -0700
Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 15:57:43 -0000

--94eb2c03e1d4b6af2c05526648f1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Scott, Al, IESG reviewers, BMWG!

After a good amount of work and feedback from Scott yesterday, I am pleased
to report that we have posted the final versions now.

The changes to look for in the methodology draft are about:

1) clarifying terms and pointing to the terminology draft when appropriate
when new terms were defined in the terminology draft (such as for example
intensity )
2) clarifying the iteration process for all tests, explaining briefly how
to read the test sequence each time
3) clarifying when TCP type of testing and UDP type of testing is needed
(for goodput type of testing)

Many thanks for all the work from so many folks getting us together to
mature documents.

Regards,
Lucien

On Sun, Jun 18, 2017 at 10:00 AM, Lucien Avramov <lucienav@google.com>
wrote:

> Hi Al, Scott
>
> thanks for the comments, updated both drafts just now with all comments!
>
> Thanks,
> Lucien
>
> On Sun, Jun 18, 2017 at 9:51 AM, MORTON, ALFRED C (AL) <acmorton@att.com>
> wrote:
>
>> adding OPS-DIR,
>> > -----Original Message-----
>> > From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Scott Bradner
>> > Sent: Sunday, June 18, 2017 9:53 AM
>> > To: MORTON, ALFRED C (AL)
>> > Cc: draft-ietf-bmwg-dcbench-methodology.all@ietf.org; bmwg-
>> > chairs@ietf.org; bmwg@ietf.org
>> > Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
>> >
>> > my issue was that I read the current text as talking about the result =
of
>> > the testing not the offered load
>> >
>> > so I was just going to suggest coming similar to what Al suggested
>> >
>> > but I=E2=80=99d add specific pointers to the explanation in the termin=
ology
>> > document as to why
>> >
>> >
>> > e.g.
>> >
>> >    For all tests, the test traffic generator sending rate MUST be
>> >    less than or equal to 99.98% of the nominal value of Line Rate
>> >    (with no further PPM adjustment to account for interface clock
>> >    tolerances), to ensure stressing the DUT in reasonable worst case
>> >    conditions. (see RFC XXX section ??? - note to RFC editor replace X=
XX
>> >    with number of terminology RFC)
>> >
>> [ACM]
>> good addition, Scott.
>>
>> > I have not reviewed the rest of the ID to see if similar misreading is
>> > possible so that
>> > a similar change would be useful
>> [ACM]
>> Let's assume for now that other Area reviewers and IESG review will
>> catch some more of these, if needed.
>>
>> thanks,
>> Al
>>
>> >
>> > Scott
>> >
>> >
>> > > On Jun 18, 2017, at 1:03 AM, MORTON, ALFRED C (AL) <acmorton@att.com=
>
>> > wrote:
>> > >
>> > > Hi Lucien, Scott, and all,
>> > >
>> > >
>> > >
>> > > When I read this material, I assumed the goal was to
>> > >
>> > > limit the maximum sending rate of the test traffic generator
>> > >
>> > > in accordance with the Ethernet clock tolerances,
>> > >
>> > > effectively choosing the low-end of clock tolerance
>> > >
>> > > as the maximum sending rate with a factor of 2 margin.
>> > >
>> > > 99.98% of the nominal line rate would be -200 ppm,
>> > >
>> > > and any conforming interface clock should work fine.
>> > >
>> > >
>> > >
>> > > If I understood your goal correctly,
>> > >
>> > > I wonder if a small clarification will help future
>> > >
>> > > readers.
>> > >
>> > >
>> > >
>> > > When section 2.2 says:
>> > >
>> > >    For all tests, the percentage of traffic per port capacity sent
>> > MUST
>> > >
>> > >                                                 ^^^^^^^^^^^^^^^^^^
>> > >
>> > >    be 99.98% at most, with no PPM adjustment to ensure stressing the
>> > DUT
>> > >
>> > >       +++++++++++++++
>> > >
>> > >    in worst case conditions.
>> > >
>> > >
>> > >
>> > > You=E2=80=99ve defined the concept of Port Capacity in terminology S=
ection 5,
>> > >
>> > > under the Line Rate definition, as
>> > >
>> > >
>> > >
>> > >    The term "port capacity" defines the maximum speed capability for
>> > the
>> > >
>> > >    given port; for example 1GE, 10GE, 40GE, 100GE etc.
>> > >
>> > >
>> > >
>> > > Actually, the port capacity examples are the *nominal value of Line
>> > Rate*,
>> > >
>> > > before the clock tolerances are considered.
>> > >
>> > >
>> > >
>> > > Here=E2=80=99s a suggestion for Section 2.2 of the meth:
>> > >
>> > >
>> > >
>> > >    For all tests, the test traffic generator sending rate MUST be
>> > >
>> > >    less than or equal to 99.98% of the nominal value of Line Rate
>> > >
>> > >    (with no further PPM adjustment to account for interface clock
>> > >
>> > >    tolerances), to ensure stressing the DUT in reasonable worst case
>> > >
>> > >    conditions.
>> > >
>> > >
>> > >
>> > > Changing
>> > >
>> > > s/port capacity/nominal value of Line Rate/
>> > >
>> > > here, requires an additional substitution in section 2.3 of the meth
>> > draft,
>> > >
>> > > and a single substitution in section 5.1 of the term draft.
>> > >
>> > >
>> > >
>> > > Does this help?
>> > >
>> > > Al
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > From: Lucien Avramov [mailto:lucienav@google.com]
>> > > Sent: Saturday, June 17, 2017 1:49 PM
>> > > To: Scott Bradner
>> > > Cc: bmwg@ietf.org; draft-ietf-bmwg-dcbench-methodology.all@ietf.org;
>> > bmwg-chairs@ietf.org
>> > > Subject: Re: draft-ietf-bmwg-dcbench-methodology-07
>> > >
>> > >
>> > >
>> > > Hi Scott!
>> > >
>> > > =C2=B7         The draft you are reviewing is the ''methodology'' dr=
aft
>> > which explains the methodology for line rate performance testing. The
>> > exact reason onto why it is 99.98% with 0 PPM adjustment is covered
>> > extensively in the terminology draft, section 5:
>> > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
>> > 3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Ddcbench-2Dterminology-
>> > 2D12-23section-2D5&d=3DDwIGaQ&c=3DLFYZ-
>> > o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DcY2eIWfAaujzPr7
>> A49lWCjuJ_Vx
>> > 0ywhEDDmsWPBcTJA&s=3D96T3Kqxhm3c-gASDa1w_XDRAcXYh30m7fyjPEm28HCE&e=3D =
.
>> That
>> > draft dependency is covered in the introduction and also in the
>> > reference lists and is referenced throughout the document
>> > >
>> > > =C2=B7         There is a clock drift difference between tester and =
DUT
>> > explained in section 5.2 of the terminology draft
>> > >
>> > > In our view BMWG has not morphed, our goal has been to be very preci=
se
>> > and accurate with performance testing, and this is not about conforman=
ce
>> > testing at all. These definitions that you see, have not changed in 4
>> > years and have been proven themselves in a very large amount of testin=
g
>> > done since we started publishing.
>> > >
>> > >
>> > >
>> > > As you may remember, we have been working on this draft actively for
>> > over 4 years now. We've incorporated numerous of your feedbacks on the=
se
>> > drafts over the IETF sessions (since IETF 86 - captured in the meeting
>> > minutes of the IETF sessions).  On April 2016, we had your blessing on
>> > the list also regarding our drafts.
>> > >
>> > >
>> > >
>> > > We appreciate as always your feedback, Scott!
>> > >
>> > >
>> > >
>> > > Lucien
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner <sob@sobco.com> wrote=
:
>> > >
>> > > try again - my new mac book often sends mail before I want it to
>> > >
>> > >
>> > > I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcbench-
>> > methodology-07
>> > >
>> > > I quickly became confused - a paragraph in section 2.2 reads
>> > >
>> > > For all tests, the percentage of traffic per port capacity sent MUST
>> > >   be 99.98% at most, with no PPM adjustment to ensure stressing the
>> > DUT
>> > >   in worst case conditions. Tests results at a lower rate MAY be
>> > >   provided for better understanding of performance increase in terms
>> > of
>> > >   latency and jitter when the rate is lower than 99.98%. The receivi=
ng
>> > >   rate of the traffic SHOULD be captured during this test in % of li=
ne
>> > >   rate.
>> > >
>> > >
>> > > Thus reads to me like a conformance test not a performance test
>> > (saying what is =E2=80=9Cpassing=E2=80=9D)  unless
>> > > BMWG has morphed a lot since I was last involved conformance and
>> > pass/fail are not things that BMWG
>> > > should be doing -
>> > >
>> > > what am I missing?
>> > >
>> > > Scott
>> > >
>> > >
>> > >
>> > >
>> >
>> > _______________________________________________
>> > bmwg mailing list
>> > bmwg@ietf.org
>> > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
>> > 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwIGaQ&c=3DLFYZ-
>> > o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DcY2eIWfAaujzPr7
>> A49lWCjuJ_Vx
>> > 0ywhEDDmsWPBcTJA&s=3DNwNczwTUN_yQqjl4Aw4jBTqk3iMbJpTrJMDiuFT8g20&e=3D
>>
>
>

--94eb2c03e1d4b6af2c05526648f1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Scott, Al, IESG reviewers, BMWG!<div><br></div><div>Aft=
er a good amount of work and feedback from Scott yesterday, I am pleased to=
 report that we have posted the final versions now.=C2=A0</div><div><br></d=
iv><div>The changes to look for in the methodology draft are about:</div><d=
iv><br></div><div>1) clarifying terms and pointing to the terminology draft=
 when appropriate when new terms were defined in the terminology draft (suc=
h as for example intensity )</div><div>2) clarifying the iteration process =
for all tests, explaining briefly how to read the test sequence each time</=
div><div>3) clarifying when TCP type of testing and UDP type of testing is =
needed (for goodput type of testing)</div><div><br></div><div>Many thanks f=
or all the work from so many folks getting us together to mature documents.=
</div><div><br></div><div>Regards,</div><div>Lucien</div></div><div class=
=3D"gmail_extra"><br><div class=3D"gmail_quote">On Sun, Jun 18, 2017 at 10:=
00 AM, Lucien Avramov <span dir=3D"ltr">&lt;<a href=3D"mailto:lucienav@goog=
le.com" target=3D"_blank">lucienav@google.com</a>&gt;</span> wrote:<br><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex"><div dir=3D"ltr">Hi Al, Scott<div><br></div><div=
>thanks for the comments, updated both drafts just now with all comments!</=
div><div><br></div><div>Thanks,</div><div>Lucien</div></div><div class=3D"H=
OEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"gmail=
_quote">On Sun, Jun 18, 2017 at 9:51 AM, MORTON, ALFRED C (AL) <span dir=3D=
"ltr">&lt;<a href=3D"mailto:acmorton@att.com" target=3D"_blank">acmorton@at=
t.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">adding OPS-DI=
R,<br>
<span>&gt; -----Original Message-----<br>
&gt; From: bmwg [mailto:<a href=3D"mailto:bmwg-bounces@ietf.org" target=3D"=
_blank">bmwg-bounces@ietf.org</a>] On Behalf Of Scott Bradner<br>
&gt; Sent: Sunday, June 18, 2017 9:53 AM<br>
&gt; To: MORTON, ALFRED C (AL)<br>
</span><span>&gt; Cc: <a href=3D"mailto:draft-ietf-bmwg-dcbench-methodology=
.all@ietf.org" target=3D"_blank">draft-ietf-bmwg-dcbench-method<wbr>ology.a=
ll@ietf.org</a>; bmwg-<br>
&gt; <a href=3D"mailto:chairs@ietf.org" target=3D"_blank">chairs@ietf.org</=
a>; <a href=3D"mailto:bmwg@ietf.org" target=3D"_blank">bmwg@ietf.org</a><br=
>
&gt; Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-method<wbr>ology-07<br>
&gt;<br>
&gt; my issue was that I read the current text as talking about the result =
of<br>
&gt; the testing not the offered load<br>
&gt;<br>
&gt; so I was just going to suggest coming similar to what Al suggested<br>
&gt;<br>
&gt; but I=E2=80=99d add specific pointers to the explanation in the termin=
ology<br>
&gt; document as to why<br>
&gt;<br>
&gt;<br>
&gt; e.g.<br>
&gt;<br>
&gt;=C2=A0 =C2=A0 For all tests, the test traffic generator sending rate MU=
ST be<br>
&gt;=C2=A0 =C2=A0 less than or equal to 99.98% of the nominal value of Line=
 Rate<br>
&gt;=C2=A0 =C2=A0 (with no further PPM adjustment to account for interface =
clock<br>
&gt;=C2=A0 =C2=A0 tolerances), to ensure stressing the DUT in reasonable wo=
rst case<br>
&gt;=C2=A0 =C2=A0 conditions. (see RFC XXX section ??? - note to RFC editor=
 replace XXX<br>
&gt;=C2=A0 =C2=A0 with number of terminology RFC)<br>
&gt;<br>
</span>[ACM]<br>
good addition, Scott.<br>
<span><br>
&gt; I have not reviewed the rest of the ID to see if similar misreading is=
<br>
&gt; possible so that<br>
&gt; a similar change would be useful<br>
</span>[ACM]<br>
Let&#39;s assume for now that other Area reviewers and IESG review will<br>
catch some more of these, if needed.<br>
<br>
thanks,<br>
Al<br>
<div><div class=3D"m_-718572519514540372h5"><br>
&gt;<br>
&gt; Scott<br>
&gt;<br>
&gt;<br>
&gt; &gt; On Jun 18, 2017, at 1:03 AM, MORTON, ALFRED C (AL) &lt;<a href=3D=
"mailto:acmorton@att.com" target=3D"_blank">acmorton@att.com</a>&gt;<br>
&gt; wrote:<br>
&gt; &gt;<br>
&gt; &gt; Hi Lucien, Scott, and all,<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; When I read this material, I assumed the goal was to<br>
&gt; &gt;<br>
&gt; &gt; limit the maximum sending rate of the test traffic generator<br>
&gt; &gt;<br>
&gt; &gt; in accordance with the Ethernet clock tolerances,<br>
&gt; &gt;<br>
&gt; &gt; effectively choosing the low-end of clock tolerance<br>
&gt; &gt;<br>
&gt; &gt; as the maximum sending rate with a factor of 2 margin.<br>
&gt; &gt;<br>
&gt; &gt; 99.98% of the nominal line rate would be -200 ppm,<br>
&gt; &gt;<br>
&gt; &gt; and any conforming interface clock should work fine.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; If I understood your goal correctly,<br>
&gt; &gt;<br>
&gt; &gt; I wonder if a small clarification will help future<br>
&gt; &gt;<br>
&gt; &gt; readers.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; When section 2.2 says:<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 For all tests, the percentage of traffic per port ca=
pacity sent<br>
&gt; MUST<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^^^^^^^^^^^^^^^^^^<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 be 99.98% at most, with no PPM adjustment to ensure =
stressing the<br>
&gt; DUT<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0+++++++++++++++<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 in worst case conditions.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; You=E2=80=99ve defined the concept of Port Capacity in terminolog=
y Section 5,<br>
&gt; &gt;<br>
&gt; &gt; under the Line Rate definition, as<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 The term &quot;port capacity&quot; defines the maxim=
um speed capability for<br>
&gt; the<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 given port; for example 1GE, 10GE, 40GE, 100GE etc.<=
br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Actually, the port capacity examples are the *nominal value of Li=
ne<br>
&gt; Rate*,<br>
&gt; &gt;<br>
&gt; &gt; before the clock tolerances are considered.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Here=E2=80=99s a suggestion for Section 2.2 of the meth:<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 For all tests, the test traffic generator sending ra=
te MUST be<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 less than or equal to 99.98% of the nominal value of=
 Line Rate<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 (with no further PPM adjustment to account for inter=
face clock<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 tolerances), to ensure stressing the DUT in reasonab=
le worst case<br>
&gt; &gt;<br>
&gt; &gt;=C2=A0 =C2=A0 conditions.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Changing<br>
&gt; &gt;<br>
&gt; &gt; s/port capacity/nominal value of Line Rate/<br>
&gt; &gt;<br>
&gt; &gt; here, requires an additional substitution in section 2.3 of the m=
eth<br>
&gt; draft,<br>
&gt; &gt;<br>
&gt; &gt; and a single substitution in section 5.1 of the term draft.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Does this help?<br>
&gt; &gt;<br>
&gt; &gt; Al<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; From: Lucien Avramov [mailto:<a href=3D"mailto:lucienav@google.co=
m" target=3D"_blank">lucienav@google.com</a>]<br>
&gt; &gt; Sent: Saturday, June 17, 2017 1:49 PM<br>
&gt; &gt; To: Scott Bradner<br>
&gt; &gt; Cc: <a href=3D"mailto:bmwg@ietf.org" target=3D"_blank">bmwg@ietf.=
org</a>; <a href=3D"mailto:draft-ietf-bmwg-dcbench-methodology.all@ietf.org=
" target=3D"_blank">draft-ietf-bmwg-dcbench-method<wbr>ology.all@ietf.org</=
a>;<br>
&gt; <a href=3D"mailto:bmwg-chairs@ietf.org" target=3D"_blank">bmwg-chairs@=
ietf.org</a><br>
&gt; &gt; Subject: Re: draft-ietf-bmwg-dcbench-method<wbr>ology-07<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Hi Scott!<br>
&gt; &gt;<br>
&gt; &gt; =C2=B7=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0The draft you are reviewi=
ng is the &#39;&#39;methodology&#39;&#39; draft<br>
&gt; which explains the methodology for line rate performance testing. The<=
br>
&gt; exact reason onto why it is 99.98% with 0 PPM adjustment is covered<br=
>
&gt; extensively in the terminology draft, section 5:<br>
</div></div>&gt; <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dht=
tps-" rel=3D"noreferrer" target=3D"_blank">https://urldefense.proofpoint.<w=
br>com/v2/url?u=3Dhttps-</a><br>
&gt; 3A__tools.ietf.org_html_draft-<wbr>2Dietf-2Dbmwg-2Ddcbench-2Dterm<wbr>=
inology-<br>
&gt; 2D12-23section-2D5&amp;d=3DDwIGaQ&amp;c=3D<wbr>LFYZ-<br>
&gt; o9_HUMeMTSQicvjIg&amp;r=3DOfsSu8kTIl<wbr>tVyD1oL72cBw&amp;m=3DcY2eIWfA=
aujzPr7<wbr>A49lWCjuJ_Vx<br>
&gt; 0ywhEDDmsWPBcTJA&amp;s=3D96T3Kqxhm3c<wbr>-gASDa1w_XDRAcXYh30m7fyjPEm28=
H<wbr>CE&amp;e=3D . That<br>
<div><div class=3D"m_-718572519514540372h5">&gt; draft dependency is covere=
d in the introduction and also in the<br>
&gt; reference lists and is referenced throughout the document<br>
&gt; &gt;<br>
&gt; &gt; =C2=B7=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0There is a clock drift di=
fference between tester and DUT<br>
&gt; explained in section 5.2 of the terminology draft<br>
&gt; &gt;<br>
&gt; &gt; In our view BMWG has not morphed, our goal has been to be very pr=
ecise<br>
&gt; and accurate with performance testing, and this is not about conforman=
ce<br>
&gt; testing at all. These definitions that you see, have not changed in 4<=
br>
&gt; years and have been proven themselves in a very large amount of testin=
g<br>
&gt; done since we started publishing.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; As you may remember, we have been working on this draft actively =
for<br>
&gt; over 4 years now. We&#39;ve incorporated numerous of your feedbacks on=
 these<br>
&gt; drafts over the IETF sessions (since IETF 86 - captured in the meeting=
<br>
&gt; minutes of the IETF sessions).=C2=A0 On April 2016, we had your blessi=
ng on<br>
&gt; the list also regarding our drafts.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; We appreciate as always your feedback, Scott!<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Lucien<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner &lt;<a href=3D"mai=
lto:sob@sobco.com" target=3D"_blank">sob@sobco.com</a>&gt; wrote:<br>
&gt; &gt;<br>
&gt; &gt; try again - my new mac book often sends mail before I want it to<=
br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcben=
ch-<br>
&gt; methodology-07<br>
&gt; &gt;<br>
&gt; &gt; I quickly became confused - a paragraph in section 2.2 reads<br>
&gt; &gt;<br>
&gt; &gt; For all tests, the percentage of traffic per port capacity sent M=
UST<br>
&gt; &gt;=C2=A0 =C2=A0be 99.98% at most, with no PPM adjustment to ensure s=
tressing the<br>
&gt; DUT<br>
&gt; &gt;=C2=A0 =C2=A0in worst case conditions. Tests results at a lower ra=
te MAY be<br>
&gt; &gt;=C2=A0 =C2=A0provided for better understanding of performance incr=
ease in terms<br>
&gt; of<br>
&gt; &gt;=C2=A0 =C2=A0latency and jitter when the rate is lower than 99.98%=
. The receiving<br>
&gt; &gt;=C2=A0 =C2=A0rate of the traffic SHOULD be captured during this te=
st in % of line<br>
&gt; &gt;=C2=A0 =C2=A0rate.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Thus reads to me like a conformance test not a performance test<b=
r>
&gt; (saying what is =E2=80=9Cpassing=E2=80=9D)=C2=A0 unless<br>
&gt; &gt; BMWG has morphed a lot since I was last involved conformance and<=
br>
&gt; pass/fail are not things that BMWG<br>
&gt; &gt; should be doing -<br>
&gt; &gt;<br>
&gt; &gt; what am I missing?<br>
&gt; &gt;<br>
&gt; &gt; Scott<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt;<br>
</div></div><span>&gt; ______________________________<wbr>_________________=
<br>
&gt; bmwg mailing list<br>
&gt; <a href=3D"mailto:bmwg@ietf.org" target=3D"_blank">bmwg@ietf.org</a><b=
r>
</span>&gt; <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-"=
 rel=3D"noreferrer" target=3D"_blank">https://urldefense.proofpoint.<wbr>co=
m/v2/url?u=3Dhttps-</a><br>
&gt; 3A__www.ietf.org_mailman_listi<wbr>nfo_bmwg&amp;d=3DDwIGaQ&amp;c=3DLFY=
Z-<br>
&gt; o9_HUMeMTSQicvjIg&amp;r=3DOfsSu8kTIl<wbr>tVyD1oL72cBw&amp;m=3DcY2eIWfA=
aujzPr7<wbr>A49lWCjuJ_Vx<br>
&gt; 0ywhEDDmsWPBcTJA&amp;s=3DNwNczwTUN_y<wbr>Qqjl4Aw4jBTqk3iMbJpTrJMDiuFT8=
g<wbr>20&amp;e=3D<br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--94eb2c03e1d4b6af2c05526648f1--


From lucien.avramov@gmail.com  Tue Jun 20 09:41:19 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F3FF131BCC; Tue, 20 Jun 2017 09:41:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0oZ5RSaCemuv; Tue, 20 Jun 2017 09:41:17 -0700 (PDT)
Received: from mail-yw0-x235.google.com (mail-yw0-x235.google.com [IPv6:2607:f8b0:4002:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 43F0A12944A; Tue, 20 Jun 2017 09:40:51 -0700 (PDT)
Received: by mail-yw0-x235.google.com with SMTP id l75so54575853ywc.3; Tue, 20 Jun 2017 09:40:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xlpbpPjTkdjlP61TQA/i9cFm63AQjZkVFlQo76FSjcE=; b=Nj0ScyR5ay3TUGoEK3aLJ5C4AQe1hcRFCP9RClzW3o3/h5n5jBQ3Fu+iwCdOkyawnD lOIEqMaIQr5PrL0gK80ESoxeuZOr2ca4qWf7RzjyXNfGqXl39A85fMVhXgVRO3rp2G1X JgXYmJGg07dTkiDiF2lkv+LdCKR5rTHqoO2WkgcWC6ncnW7EBFXZgeD7JKFvdR2Nbgu8 V7Pb20nAgcQQPHpFr1YAeOoLdWrpNMwOdAR052s7r7DkZ4ncqISejO4d31U/rpe60QDN EI3J8CunlFcMNmqGc3Ckctf43Kvf9dI8Y4azEqfHZL8vtAjM+lXczMvCmvXHqd8zKc8A QoOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xlpbpPjTkdjlP61TQA/i9cFm63AQjZkVFlQo76FSjcE=; b=ALvNb9kN3KTPrdQvM7SWlTz94MFOrCrPUWKNkNNZUksUkz3wHqBXd9wWtcwVAxN2nh AjhRxz+s10c9+cFgqinvmwxSzoRUrMlTQyH4nUbizfIWb39lbKp15qgt4myDqv0RniNW 9P3CRdzpCKIzd1rKyVBUuHE6eWb9fW4oEe6jMCUGd938RBNlopcvf1MzkVrzAYa21bfL ajFrTB27IG3LSpOc3vCf3mM9z6hSboG+IU2pmqM/eC1+cPJoSEKhY+SHt260VMotUaJ1 w0BpYo8hnCV7swEFHprNMCCiD1G6Omy1VSAGQaEyLpox8NTYwNM3ziDmz+2+1ErDtoep t++g==
X-Gm-Message-State: AKS2vOw5m96wk/DFufsIcpP/33Q54uyPNs/Y1o4CwRYLVc/06Y31PTaG KAsP2vlStraTCUYFqI+0nAnjuxa+Aw==
X-Received: by 10.13.232.136 with SMTP id r130mr1925080ywe.161.1497976850365;  Tue, 20 Jun 2017 09:40:50 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Tue, 20 Jun 2017 09:40:49 -0700 (PDT)
In-Reply-To: <D78D2206-5A27-40E3-9E29-8C4A8D1ABD01@cisco.com>
References: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com> <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com> <D78D2206-5A27-40E3-9E29-8C4A8D1ABD01@cisco.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Tue, 20 Jun 2017 09:40:49 -0700
Message-ID: <CAArZqeWN2suTLp5DUNsqEK5uCP7pa7-rtu7510c508jQMwYTeg@mail.gmail.com>
To: "Alvaro Retana (aretana)" <aretana@cisco.com>
Cc: Lucien Avramov <lucienav@google.com>, The IESG <iesg@ietf.org>,  "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>,  Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>,  "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c08712c0c44b7055266eac6"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/fF67Rpczuk4Ukt3jAhMjdpVvSPs>
X-Mailman-Approved-At: Tue, 20 Jun 2017 10:05:14 -0700
Subject: Re: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 16:43:09 -0000

--94eb2c08712c0c44b7055266eac6
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks for clarifying Alvaro!! Much appreciate it.

Got it, i'd like to keep 1.2 add-on as it makes it more clear for the
reader to understand what I want to convey when I use this language in the
doc, instead of having them go read another RFC.

On Tue, Jun 20, 2017 at 9:35 AM, Alvaro Retana (aretana) <aretana@cisco.com=
>
wrote:

> On 6/20/17, 3:17 AM, "Lucien Avramov" <lucienav@google.com> wrote:
>
>
>
> Lucien:
>
>
>
> Hi!
>
>
>
> > In the context of the tests we describe in this publication, it's key t=
o
> keep
>
> > the words we have chosen=E2=80=A6
>
>
>
> I don=E2=80=99t have a problem with the words themselves =E2=80=93 the po=
int of my DISCUSS
> is the fact that each word (=E2=80=9CMUST=E2=80=9D, for example) has two =
different
> definitions.  Yes, they are close, but different.  In fact, they are so
> close that I think you could take out the definitions in 1.2 and not lose
> anything.
>
>
>
> Regards,
>
>
>
> Alvaro.
>

--94eb2c08712c0c44b7055266eac6
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks for clarifying Alvaro!! Much appreciate it.=C2=A0<d=
iv><br></div><div><div>Got it, i&#39;d like to keep 1.2 add-on as it makes =
it more clear for the reader to understand what I want to convey when I use=
 this language in the doc, instead of having them go read another RFC.=C2=
=A0</div></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quo=
te">On Tue, Jun 20, 2017 at 9:35 AM, Alvaro Retana (aretana) <span dir=3D"l=
tr">&lt;<a href=3D"mailto:aretana@cisco.com" target=3D"_blank">aretana@cisc=
o.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">







<div bgcolor=3D"white" lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"m_-7067659852252104260WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">On 6/20/17, 3:17=
 AM, &quot;Lucien Avramov&quot; &lt;<a href=3D"mailto:lucienav@google.com" =
target=3D"_blank">lucienav@google.com</a>&gt; wrote:<u></u><u></u></span></=
p>
<div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Lucien:<u></u><u=
></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Hi!<u></u><u></u=
></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
</div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
 In the context of the tests we describe in this publication, it&#39;s key =
to keep
<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
 the words we have chosen=E2=80=A6<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black"><u><=
/u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">I do=
n=E2=80=99t have a problem with the words themselves =E2=80=93 the point of=
 my DISCUSS is the fact that each word (=E2=80=9CMUST=E2=80=9D, for example=
) has two different definitions.=C2=A0 Yes, they are close, but different.=
=C2=A0
 In fact, they are so close that I think you could take out the definitions=
 in 1.2 and not lose anything.
<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black"><u><=
/u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">Rega=
rds,<span class=3D"HOEnZb"><font color=3D"#888888"><u></u><u></u></font></s=
pan></span></p><span class=3D"HOEnZb"><font color=3D"#888888">
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black"><u><=
/u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">Alva=
ro.</span><span style=3D"font-family:Calibri"><u></u><u></u></span></p>
</font></span></div>
</div>

</blockquote></div><br></div>

--94eb2c08712c0c44b7055266eac6--


From nobody Tue Jun 20 11:43:51 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BE8F51315D8 for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 11:43:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6ThxTjOHV4gO for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 11:43:48 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A6FCE131589 for <bmwg@ietf.org>; Tue, 20 Jun 2017 11:43:48 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v5KIfV5q025405 for <bmwg@ietf.org>; Tue, 20 Jun 2017 14:43:46 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by mx0a-00191d01.pphosted.com with ESMTP id 2b767n81nm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Tue, 20 Jun 2017 14:43:46 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KIhjgV042842 for <bmwg@ietf.org>; Tue, 20 Jun 2017 13:43:45 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KIhe4S042822 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Tue, 20 Jun 2017 13:43:40 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Tue, 20 Jun 2017 18:43:23 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KIhMAL013873 for <bmwg@ietf.org>; Tue, 20 Jun 2017 13:43:23 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KIhIU9013646 for <bmwg@ietf.org>; Tue, 20 Jun 2017 13:43:18 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 6E10AF049C for <bmwg@ietf.org>; Tue, 20 Jun 2017 14:43:18 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 20 Jun 2017 14:43:18 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQNPDsMQ
Date: Tue, 20 Jun 2017 18:43:18 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.att.com>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.28.111.206]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-20_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706200320
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0CzMYB9uc5rq_7dnGQbqce0iuDo>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 18:43:51 -0000

BMWG,

So far, I count about eight people who offered=20
support for the draft, and one person who made
a technical comment along with their message.

*No one* has answered this question:
>=20
> (3) whether you commit to reviewing the WG document if adopted
>

With no commitments to review the draft and
provide the necessary expertise to develop=20
appropriate benchmarks for our industry,
it will not be appropriate to add a milestone
to our charter (or re-charter, if necessary).

There is still time for everyone to reconsider
their position, and provide an answer to the=20
question above.  The WGLC closes on Friday 23 June 2017.

thanks for your consideration,
Al
bmwg co-chair


> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED C
> (AL)
> Sent: Saturday, June 03, 2017 6:34 PM
> To: bmwg@ietf.org
> Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
>=20
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more
> information.
>=20
> BMWG,
>=20
> At our Chicago meeting, the BMWG reviewed a version of
> the Internet Draft on
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
>=20
> There was sufficient readership present at the meeting
> and positive/constructive comments on the work, so the
> chairs asked whether support existed to adopt the draft
> as a WG item. There was support and no objections. As
> always, such decisions are officially taken on the
> mailing list.
>=20
> This message*, therefore, starts a call for adoption on:
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
>=20
> to run until Friday 23 June 2017.
>=20
> Please reply to bmwg@ietf.org indicating:
>=20
> (1) whether you support addition of the following milestone to
>    the BMWG charter:
>=20
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
>=20
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>    as the basis document for this milestone
>=20
> (3) whether you commit to reviewing the WG document if adopted
>=20
> We are aware that BMWG has nearly completed all its existing
> chartered items.  This usually begins discussion of
> re-chartering, this time with Warren (our new AD Advisor). The final
> decision/schedule for adoption will depend on these discussions.
>=20
> thanks for your consideration of the draft, and regards,
>=20
> Al
> bmwg co-chair
>=20
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D3tXKQNBR8Q4HC7BBx6pc5me8=
u9z
> Ne9QVo_owOF5-ciM&s=3DxIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=3D


From nobody Tue Jun 20 13:29:03 2017
Return-Path: <spencerdawkins.ietf@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 2028D1294E6; Tue, 20 Jun 2017 13:28:55 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Spencer Dawkins <spencerdawkins.ietf@gmail.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 13:28:55 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/5GU7TcDLFFWTswOwANvhqbmB9OY>
Subject: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 20:28:55 -0000

Spencer Dawkins has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-13: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

(I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and
draft-ietf-bmwg-dcbench-methodology)

I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology and
draft-ietf-bmwg-dcbench-methodology that assert these documents aren't specific
to "data centers". That wouldn't surprise me, but I'm not seeing a definition
of "data center" in either document - did I miss it?

I suspect that the authors have specific technical characteristics in mind,
that happen to map onto what data centers look like today, but may not in the
future ("RFCs last forever"). Is it possible to tease those characteristics out?

(Full disclosure: my first working group in the IETF could have been called
"TCP over cellular links", but it turned out that when we said "cellular
links", we meant "low-speed links with high loss rates and asymmetric
bandwidth". "Cellular links" in the late 1990s didn't have the same
characteristics that "cellular links" have in 2017, but there are other link
types with those characteristics, so the documents ended up being useful in
places like CORE. I'm not suggesting anything like a restructure of the
document(s), only that they be clear about how future readers would know
whether they should be reading them in 2027)



From nobody Tue Jun 20 13:38:51 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8CB2B129494; Tue, 20 Jun 2017 13:38:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level: 
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LdWJtbvtWJT5; Tue, 20 Jun 2017 13:38:47 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ADDA3128954; Tue, 20 Jun 2017 13:38:47 -0700 (PDT)
Received: from pps.filterd (m0049297.ppops.net [127.0.0.1]) by m0049297.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v5KKZQeH017222; Tue, 20 Jun 2017 16:38:43 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049297.ppops.net-00191d01. with ESMTP id 2b79fd3yye-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Jun 2017 16:38:42 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KKcfFL126869; Tue, 20 Jun 2017 15:38:42 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KKcYsV126841 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 20 Jun 2017 15:38:34 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor); Tue, 20 Jun 2017 20:38:19 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KKcJnt018292; Tue, 20 Jun 2017 15:38:19 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5KKcBNp017887; Tue, 20 Jun 2017 15:38:11 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 261C3E0686; Tue, 20 Jun 2017 16:38:11 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 20 Jun 2017 16:38:11 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Spencer Dawkins <spencerdawkins.ietf@gmail.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
Thread-Index: AQHS6gPb+6Fixbn350KYLzGljL1Zy6IuNVXA
Date: Tue, 20 Jun 2017 20:38:10 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FE1243@njmtexg5.research.att.com>
References: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com>
In-Reply-To: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [104.129.194.133]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-20_10:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706200355
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/s11oG_eDcY5SdHSiwbpRZjWrdBc>
Subject: Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 20:38:49 -0000

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTcGVuY2VyIERhd2tpbnMgW21h
aWx0bzpzcGVuY2VyZGF3a2lucy5pZXRmQGdtYWlsLmNvbV0NCj4gU2VudDogVHVlc2RheSwgSnVu
ZSAyMCwgMjAxNyA0OjI5IFBNDQo+IFRvOiBUaGUgSUVTRw0KPiBDYzogZHJhZnQtaWV0Zi1ibXdn
LWRjYmVuY2gtbWV0aG9kb2xvZ3lAaWV0Zi5vcmc7IFNhcmFoIEJhbmtzOyBibXdnLQ0KPiBjaGFp
cnNAaWV0Zi5vcmc7IHNiYW5rc0BlbmNyeXB0ZWQubmV0OyBibXdnQGlldGYub3JnDQo+IFN1Ympl
Y3Q6IFNwZW5jZXIgRGF3a2lucycgTm8gT2JqZWN0aW9uIG9uIGRyYWZ0LWlldGYtYm13Zy1kY2Jl
bmNoLQ0KPiBtZXRob2RvbG9neS0xMzogKHdpdGggQ09NTUVOVCkNCj4gDQo+IFNwZW5jZXIgRGF3
a2lucyBoYXMgZW50ZXJlZCB0aGUgZm9sbG93aW5nIGJhbGxvdCBwb3NpdGlvbiBmb3INCj4gZHJh
ZnQtaWV0Zi1ibXdnLWRjYmVuY2gtbWV0aG9kb2xvZ3ktMTM6IE5vIE9iamVjdGlvbg0KPiANCi4u
Lg0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tDQo+IENPTU1FTlQ6DQo+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gDQo+IChJ
J20gbWFraW5nIHRoZSBzYW1lIGNvbW1lbnQgb24gZHJhZnQtaWV0Zi1ibXdnLWRjYmVuY2gtdGVy
bWlub2xvZ3kgYW5kDQo+IGRyYWZ0LWlldGYtYm13Zy1kY2JlbmNoLW1ldGhvZG9sb2d5KQ0KPiAN
Cj4gSSdtIGxvb2tpbmcgYXQgdGhlIGJhbGxvdCBwb3NpdGlvbnMgb24gZHJhZnQtaWV0Zi1ibXdn
LWRjYmVuY2gtDQo+IHRlcm1pbm9sb2d5IGFuZA0KPiBkcmFmdC1pZXRmLWJtd2ctZGNiZW5jaC1t
ZXRob2RvbG9neSB0aGF0IGFzc2VydCB0aGVzZSBkb2N1bWVudHMgYXJlbid0DQo+IHNwZWNpZmlj
DQo+IHRvICJkYXRhIGNlbnRlcnMiLiBUaGF0IHdvdWxkbid0IHN1cnByaXNlIG1lLCBidXQgSSdt
IG5vdCBzZWVpbmcgYQ0KPiBkZWZpbml0aW9uDQo+IG9mICJkYXRhIGNlbnRlciIgaW4gZWl0aGVy
IGRvY3VtZW50IC0gZGlkIEkgbWlzcyBpdD8NCj4gDQo+IEkgc3VzcGVjdCB0aGF0IHRoZSBhdXRo
b3JzIGhhdmUgc3BlY2lmaWMgdGVjaG5pY2FsIGNoYXJhY3RlcmlzdGljcyBpbg0KPiBtaW5kLA0K
PiB0aGF0IGhhcHBlbiB0byBtYXAgb250byB3aGF0IGRhdGEgY2VudGVycyBsb29rIGxpa2UgdG9k
YXksIGJ1dCBtYXkgbm90DQo+IGluIHRoZQ0KPiBmdXR1cmUgKCJSRkNzIGxhc3QgZm9yZXZlciIp
LiBJcyBpdCBwb3NzaWJsZSB0byB0ZWFzZSB0aG9zZQ0KPiBjaGFyYWN0ZXJpc3RpY3Mgb3V0Pw0K
PiANCj4gKEZ1bGwgZGlzY2xvc3VyZTogbXkgZmlyc3Qgd29ya2luZyBncm91cCBpbiB0aGUgSUVU
RiBjb3VsZCBoYXZlIGJlZW4NCj4gY2FsbGVkDQo+ICJUQ1Agb3ZlciBjZWxsdWxhciBsaW5rcyIs
IGJ1dCBpdCB0dXJuZWQgb3V0IHRoYXQgd2hlbiB3ZSBzYWlkICJjZWxsdWxhcg0KPiBsaW5rcyIs
IHdlIG1lYW50ICJsb3ctc3BlZWQgbGlua3Mgd2l0aCBoaWdoIGxvc3MgcmF0ZXMgYW5kIGFzeW1t
ZXRyaWMNCj4gYmFuZHdpZHRoIi4gIkNlbGx1bGFyIGxpbmtzIiBpbiB0aGUgbGF0ZSAxOTkwcyBk
aWRuJ3QgaGF2ZSB0aGUgc2FtZQ0KPiBjaGFyYWN0ZXJpc3RpY3MgdGhhdCAiY2VsbHVsYXIgbGlu
a3MiIGhhdmUgaW4gMjAxNywgYnV0IHRoZXJlIGFyZSBvdGhlcg0KPiBsaW5rDQo+IHR5cGVzIHdp
dGggdGhvc2UgY2hhcmFjdGVyaXN0aWNzLCBzbyB0aGUgZG9jdW1lbnRzIGVuZGVkIHVwIGJlaW5n
IHVzZWZ1bA0KPiBpbg0KPiBwbGFjZXMgbGlrZSBDT1JFLiBJJ20gbm90IHN1Z2dlc3RpbmcgYW55
dGhpbmcgbGlrZSBhIHJlc3RydWN0dXJlIG9mIHRoZQ0KPiBkb2N1bWVudChzKSwgb25seSB0aGF0
IHRoZXkgYmUgY2xlYXIgYWJvdXQgaG93IGZ1dHVyZSByZWFkZXJzIHdvdWxkIGtub3cNCj4gd2hl
dGhlciB0aGV5IHNob3VsZCBiZSByZWFkaW5nIHRoZW0gaW4gMjAyNykNCj4gDQpbQUNNXSB0aGFu
a3MgU3BlbmNlci4NCg0KSSdtIHJlbWluZGVkIHRoYXQgd2UgdXNlZCB0byBjYWxsIHRoZXNlIHBs
YWNlcyANCiJjb21wdXRlciByb29tcyIsIGFuZCB0aGVuICJjb21wdXRlciBjZW50ZXJzIi4NCmh0
dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0RhdGFfY2VudGVyDQoNCkkgYWdyZWUsIHRoZSBt
ZW1vIHNob3VsZCBzYXkgd2hhdCB3ZSBtZWFuIGZvciB0aGlzDQphcGV4LWxldmVsIHRlcm0sIGFu
ZCB0aGUgYXJ0aWNsZSBhYm92ZSBtYXkgaGVscCB0aGUgDQphdXRob3JzIGV4cHJlc3MgaXQuDQoN
CnJlZ2FyZHMsDQpBbA0KDQo=


From nobody Tue Jun 20 13:44:17 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 629DA131640 for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 13:44:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.002
X-Spam-Level: 
X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9S_dMwx_BnCN for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 13:44:13 -0700 (PDT)
Received: from mail-pf0-x22a.google.com (mail-pf0-x22a.google.com [IPv6:2607:f8b0:400e:c00::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3F74113163B for <bmwg@ietf.org>; Tue, 20 Jun 2017 13:44:13 -0700 (PDT)
Received: by mail-pf0-x22a.google.com with SMTP id 83so74700847pfr.0 for <bmwg@ietf.org>; Tue, 20 Jun 2017 13:44:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sYPLNeovXoK5sQO/WMLEihf/M7dAit2ShTsdBOX9QF8=; b=DBemqhqqtY6UWIsyKjwL/4sv2TWWM5UHSTv5alRJV2YyCyy10WPs4J2H6FiM/6fwXg wusSj2fr5O7Xx9d10cB/44Ri0a2dYihSnHuO+WCtDRTd+/4nyL5vdAJRzHFcEvEekhRe 37L9wxq+OfOLWiVo9Zzj9nuJXs+2InGF1YRFiOsrUov3gSX5DI7NY0vVwXYvAbbuDH0o 54aC7DSblUsNpz0NxFDeTLbjwHBbv2p25TAY3tjbgVKNRHoZPjjYewOZ9WKbqW8c6Do5 sflCfZxUd8RBwgpn73gRuzvQ1eKbZsoh7K8VxLW+aZ93lt5V54lKZe7x+doap7EoTgFH Lbqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sYPLNeovXoK5sQO/WMLEihf/M7dAit2ShTsdBOX9QF8=; b=AgUpnNxO9BuW8/0GkgoJ4qDoVGJOBhpBCnAPYRHPYm5vykBOdq4Bls1fu8HD6c+iQX vzxAUZqphHKdbRKV3/NVDZp6okJqjvtozgmU/yO/7Mqb0jw4IgJYRBOeI3qUD8ZU295t q1CW1e2aGGh4HXj5Z4+diQWup88Be32TwgMu1d19QiGRfQvXiQZXfq3zd3IpSRumApzP KTQzJVq0OkLOHCnmNeEEe0EEhz9oAYIzAMbfLgnrSZQyB+1LL0gyTw9A/1wDLlVDlqKT oHTOQZIpo0+XESqipGd9++JVvL3ndPg/kv7f5otXlyY6itfdPMQqnGaOY2BDTwQVp83b xlIw==
X-Gm-Message-State: AKS2vOyvdGYrCMlt+P2OukRN+EIyXqyHYW9BQ9K+PGHxvj1V1TP1vwI9 rKF+dwlQ4ly+yEZ1
X-Received: by 10.99.126.67 with SMTP id o3mr33136628pgn.36.1497991452637; Tue, 20 Jun 2017 13:44:12 -0700 (PDT)
Received: from ?IPv6:2620::100e:fd00:55db:f4b:7d01:12b2? ([2620:0:100e:fd00:55db:f4b:7d01:12b2]) by smtp.gmail.com with ESMTPSA id k11sm30043913pfj.48.2017.06.20.13.44.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Jun 2017 13:44:11 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Lucien Avramov <lucienav@google.com>
X-Mailer: iPhone Mail (14E304)
In-Reply-To: <8BA949EB-B14A-49BA-992F-34180343FB92@gmail.com>
Date: Tue, 20 Jun 2017 13:44:11 -0700
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <D1FF3AD3-2249-45DB-8A93-32DEA3D5E503@google.com>
References: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com> <8BA949EB-B14A-49BA-992F-34180343FB92@gmail.com>
To: Spencer Dawkins <spencerdawkins.ietf@gmail.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/1HvrlMIDii7KilZpcBZghDK0UvM>
Subject: Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 20:44:16 -0000

>=20
> Definitely agreed, will place a statement in both documents today to expla=
in what datacenter switches are at time of writing.=20
>=20
> Thank you,
> Lucien
>=20
>> On Jun 20, 2017, at 13:28, Spencer Dawkins <spencerdawkins.ietf@gmail.com=
> wrote:
>>=20
>> Spencer Dawkins has entered the following ballot position for
>> draft-ietf-bmwg-dcbench-methodology-13: No Objection
>>=20
>> When responding, please keep the subject line intact and reply to all
>> email addresses included in the To and CC lines. (Feel free to cut this
>> introductory paragraph, however.)
>>=20
>>=20
>> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html=

>> for more information about IESG DISCUSS and COMMENT positions.
>>=20
>>=20
>> The document, along with other ballot positions, can be found here:
>> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>>=20
>>=20
>>=20
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>=20
>> (I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and
>> draft-ietf-bmwg-dcbench-methodology)
>>=20
>> I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminolog=
y and
>> draft-ietf-bmwg-dcbench-methodology that assert these documents aren't sp=
ecific
>> to "data centers". That wouldn't surprise me, but I'm not seeing a defini=
tion
>> of "data center" in either document - did I miss it?
>>=20
>> I suspect that the authors have specific technical characteristics in min=
d,
>> that happen to map onto what data centers look like today, but may not in=
 the
>> future ("RFCs last forever"). Is it possible to tease those characteristi=
cs out?
>>=20
>> (Full disclosure: my first working group in the IETF could have been call=
ed
>> "TCP over cellular links", but it turned out that when we said "cellular
>> links", we meant "low-speed links with high loss rates and asymmetric
>> bandwidth". "Cellular links" in the late 1990s didn't have the same
>> characteristics that "cellular links" have in 2017, but there are other l=
ink
>> types with those characteristics, so the documents ended up being useful i=
n
>> places like CORE. I'm not suggesting anything like a restructure of the
>> document(s), only that they be clear about how future readers would know
>> whether they should be reading them in 2027)
>>=20
>>=20


From nobody Tue Jun 20 13:56:41 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1854D13163B; Tue, 20 Jun 2017 13:42:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m1cM4PyKJnfx; Tue, 20 Jun 2017 13:42:43 -0700 (PDT)
Received: from mail-pf0-x241.google.com (mail-pf0-x241.google.com [IPv6:2607:f8b0:400e:c00::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D690D131633; Tue, 20 Jun 2017 13:42:43 -0700 (PDT)
Received: by mail-pf0-x241.google.com with SMTP id y7so25820153pfd.3; Tue, 20 Jun 2017 13:42:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fRP48C7c2AznNBboNOMJjAlj9XdtIJVzKfTQ7KaJIs4=; b=uTi95cm6SIBn+In/Zy8aBz7esT5r+JtYF0wkxAm9BilzF8chtlhQBBcJ/1di4pYgcn +qoy8k5O/6t/KhGuYhXTyG89WnKT8eFt+J3ClekTu/X0b7iqzSjgpejeiTiTQXv7EPWL JU3v2rf4Cqjqy8yV3rnCDVmmTsLPoY5JPzVWKkbwVagOUWXs9IApFQWJJJNLut5nN0h2 9nj/vkD9ELnWmyxb1Dk83hFKI9yg0Ize8EKCXASOcO5QsULv6TF7doi4gykOmmO/3cbK WNckK8glSqOb0xZ7IwbQr3rRspF7VHsYKyEqJsmakfQhvjqKWX2UDXAtj/aK2tDkGOWp Ge2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fRP48C7c2AznNBboNOMJjAlj9XdtIJVzKfTQ7KaJIs4=; b=AVsItjEh9y9IjIcJ06e3kbkIJgigzUtWNqYCOUw+M5d93BJUgVElJLTl816GZ10oRY 7PhdxrmzxpTzPij6Qg6E9/LIqRoJ4qMtbUN8gKDdprJ5vOzWRU1I664NHkESOcAnCAG/ cwMZ5gG15hODgI0eCcjyJPriob5+uNczRj5HZXdsMvmEp8FgfdGvx/CIRFx70KqV/O/j EGor45fCp3c14vNVXWF1soPpvuiRl3Cj1k+27JLE+Yd8+04w1nUiFTxS4nwA3lZ5K9I8 JW14AyvQEUahW7uyIvoVv9UbX5dN/Cr0rhtj8G+QKMhYQzlctORbqQEYuja5wWWhtGfh t++A==
X-Gm-Message-State: AKS2vOxneJlu2kNBqABHZfUH/qdUQUQV+bIhIuIzwuVBpfCPiBv3TDqF sLDdOfhVuqIv6MDm
X-Received: by 10.98.18.143 with SMTP id 15mr4472530pfs.163.1497991363172; Tue, 20 Jun 2017 13:42:43 -0700 (PDT)
Received: from ?IPv6:2620::100e:fd00:55db:f4b:7d01:12b2? ([2620:0:100e:fd00:55db:f4b:7d01:12b2]) by smtp.gmail.com with ESMTPSA id n78sm31063116pfh.128.2017.06.20.13.42.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Jun 2017 13:42:42 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Lucien Avramov <lucien.avramov@gmail.com>
X-Mailer: iPhone Mail (14E304)
In-Reply-To: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 13:42:42 -0700
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <8BA949EB-B14A-49BA-992F-34180343FB92@gmail.com>
References: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com>
To: Spencer Dawkins <spencerdawkins.ietf@gmail.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/v13tp1ZapuUXLZH-zMLdDZQVyHQ>
X-Mailman-Approved-At: Tue, 20 Jun 2017 13:56:40 -0700
Subject: Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Jun 2017 20:42:46 -0000

Definitely agreed, will place a statement in both documents today to explain=
 what datacenter switches are at time of writing.=20

Thank you,
Lucien

> On Jun 20, 2017, at 13:28, Spencer Dawkins <spencerdawkins.ietf@gmail.com>=
 wrote:
>=20
> Spencer Dawkins has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-13: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>=20
>=20
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> (I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and
> draft-ietf-bmwg-dcbench-methodology)
>=20
> I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology=
 and
> draft-ietf-bmwg-dcbench-methodology that assert these documents aren't spe=
cific
> to "data centers". That wouldn't surprise me, but I'm not seeing a definit=
ion
> of "data center" in either document - did I miss it?
>=20
> I suspect that the authors have specific technical characteristics in mind=
,
> that happen to map onto what data centers look like today, but may not in t=
he
> future ("RFCs last forever"). Is it possible to tease those characteristic=
s out?
>=20
> (Full disclosure: my first working group in the IETF could have been calle=
d
> "TCP over cellular links", but it turned out that when we said "cellular
> links", we meant "low-speed links with high loss rates and asymmetric
> bandwidth". "Cellular links" in the late 1990s didn't have the same
> characteristics that "cellular links" have in 2017, but there are other li=
nk
> types with those characteristics, so the documents ended up being useful i=
n
> places like CORE. I'm not suggesting anything like a restructure of the
> document(s), only that they be clear about how future readers would know
> whether they should be reading them in 2027)
>=20
>=20


From nobody Tue Jun 20 19:48:39 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E49701293EC; Tue, 20 Jun 2017 19:48:37 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149801331789.15559.12729542047109038580@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 19:48:37 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/LCusiUiWjepkM-tvkhDm14DXHgc>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-15.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 02:48:38 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-15.txt
	Pages           : 17
	Date            : 2017-06-20

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to performance
evaluations of data center network equipment. This document establishes
the important concepts for benchmarking network switches and routers in
the data center and, is a pre-requisite to the test methodology
publication [1]. Many of these terms and methods may be applicable to
network equipment beyond this publication's scope as the technologies
originally applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-15
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-15

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-15


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Jun 20 19:53:05 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C88BB126B6E; Tue, 20 Jun 2017 19:52:56 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149801357680.15752.15639403642601267648@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 19:52:56 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/9njLlgdf-qC-H83VOC8WLXt5PNc>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-14.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 02:52:57 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-14.txt
	Pages           : 20
	Date            : 2017-06-20

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. A pre-requisite to this
   publication is the terminology document [1]. Many of these terms and
   methods may be applicable beyond this publication's scope as the
   technologies originally applied in the data center are deployed
   elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-14
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-14

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-14


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Jun 20 19:57:39 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F66C131587; Tue, 20 Jun 2017 19:57:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level: 
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pHxnTdbHBbTa; Tue, 20 Jun 2017 19:57:35 -0700 (PDT)
Received: from mail-yw0-x22a.google.com (mail-yw0-x22a.google.com [IPv6:2607:f8b0:4002:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 00E0A131560; Tue, 20 Jun 2017 19:57:34 -0700 (PDT)
Received: by mail-yw0-x22a.google.com with SMTP id 63so59482157ywr.0; Tue, 20 Jun 2017 19:57:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=G9l7dAxr/6VYhf6ohLt3lIFKcmPPqlgFYV3Ch9MY/ok=; b=eNJsw6XxhNps5T2QbWY1KR0LhC0KxQXRaozM24vlrrgWk2l4QvN+9NEXQTZ9OS6f7H 3LZv11ZzG/a79KbyV20hBfmYaGRjucEi2UvZU1xFeuHlhgRH9VKbQC8EbDKZrZOgzgVE GTiyuVbnVyC3FXjdvsTUXEnqGTZqLcLduECuv3ssUvCQGKtwie6jpsGlPZpK8inMoS2/ NAenfyhL2qa+sUGdp2pIKIV9niBrfoUv5+NpQjogqIC738hDTLr6YFFMNbKea2A7BaJY VAfkofb7SJMNVo5H9nwT6cyyR0wD6xvKMZmUcmRXpzysJ2cTvMMl+hfYunofu41vnGwy EZ4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=G9l7dAxr/6VYhf6ohLt3lIFKcmPPqlgFYV3Ch9MY/ok=; b=Wmyf7mv4AQ/jv9roWSvVkWDyBvNk3kDgc8Iqtj9bz2U68XN9YF7hOY6tHymOVdHgZN Ql0JjoSNJwHyMN6Z9ABdtx/i1+4O5Oz4qs7McEpZO8ik2+RLxdDATRpmznCRB/vVbSCz V4TRg5zHHFd7+wV+HKdDEr+LLPhJ7HCxkyW5UI/DJ3mgIAbmwqBwArAWp/87lg4jOc35 n7lwlrhJhD1hvgUDHetyJvi8FQkq5VhjfoEziJDhnHtBfn8Bklye6QmZ070p+yFndld+ oR7RZsd6gZjqYORYVrvVYTdGcC+vvQvWk6zVmpayZDJiSPxOoqosvrU94Y6xjjLDko3p WtyA==
X-Gm-Message-State: AKS2vOyh3Jc9sx2LutFPBOtt6tTLXaqdaudlKquCmIbeMMyTX8Fr56lY aohuoMGx5pDUuZd0XUO3zMo/7MBwbQ==
X-Received: by 10.13.205.66 with SMTP id p63mr899976ywd.318.1498013854081; Tue, 20 Jun 2017 19:57:34 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Tue, 20 Jun 2017 19:57:33 -0700 (PDT)
In-Reply-To: <8BA949EB-B14A-49BA-992F-34180343FB92@gmail.com>
References: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com> <8BA949EB-B14A-49BA-992F-34180343FB92@gmail.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Tue, 20 Jun 2017 19:57:33 -0700
Message-ID: <CAArZqeUwPmaeMRBv8EdrK9goEDrpSZNPtUtreK1Zz3UnpkgqYg@mail.gmail.com>
To: Spencer Dawkins <spencerdawkins.ietf@gmail.com>, sob@sobco.com, warren@kumari.net
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a114e58a8a4310605526f8796"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/YC7n9YY93ohYQdRXl_BvCJyja64>
Subject: Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 02:57:38 -0000

--001a114e58a8a4310605526f8796
Content-Type: text/plain; charset="UTF-8"

Hi Spencer, Al (+Scott, +Warren), and all,

Thank you for the suggestion. Both drafts have been updated with explaining
what a data center network device unter test is for these tests.

Besides adding more clarity on both abstracts, and introduction, I added
the following text in the introduction section of each draft to specify
what a data center Network DUT is for these drafts.

I hope Scott will appreciate the low buffer comment, as the high
performance data center switches today have significantly less buffer than
other networking devices.

Currently, typical data Center networking devices are characterized
by:

-High port density (48 ports of more)

-High speed (up to 100 Gb/s currently per port)

-High throughput (line rate on all ports for Layer 2 and/or Layer 3)

-Low latency (in the microsecond or nanosecond range)

-Low amount of buffer (in the Mb range)

-Layer 2 and Layer 3 forwarding capability (Layer 3 not mandatory)

On Tue, Jun 20, 2017 at 1:42 PM, Lucien Avramov <lucien.avramov@gmail.com>
wrote:

> Definitely agreed, will place a statement in both documents today to
> explain what datacenter switches are at time of writing.
>
> Thank you,
> Lucien
>
> > On Jun 20, 2017, at 13:28, Spencer Dawkins <
> spencerdawkins.ietf@gmail.com> wrote:
> >
> > Spencer Dawkins has entered the following ballot position for
> > draft-ietf-bmwg-dcbench-methodology-13: No Objection
> >
> > When responding, please keep the subject line intact and reply to all
> > email addresses included in the To and CC lines. (Feel free to cut this
> > introductory paragraph, however.)
> >
> >
> > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.
> html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
> >
> >
> >
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> >
> > (I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and
> > draft-ietf-bmwg-dcbench-methodology)
> >
> > I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology
> and
> > draft-ietf-bmwg-dcbench-methodology that assert these documents aren't
> specific
> > to "data centers". That wouldn't surprise me, but I'm not seeing a
> definition
> > of "data center" in either document - did I miss it?
> >
> > I suspect that the authors have specific technical characteristics in
> mind,
> > that happen to map onto what data centers look like today, but may not
> in the
> > future ("RFCs last forever"). Is it possible to tease those
> characteristics out?
> >
> > (Full disclosure: my first working group in the IETF could have been
> called
> > "TCP over cellular links", but it turned out that when we said "cellular
> > links", we meant "low-speed links with high loss rates and asymmetric
> > bandwidth". "Cellular links" in the late 1990s didn't have the same
> > characteristics that "cellular links" have in 2017, but there are other
> link
> > types with those characteristics, so the documents ended up being useful
> in
> > places like CORE. I'm not suggesting anything like a restructure of the
> > document(s), only that they be clear about how future readers would know
> > whether they should be reading them in 2027)
> >
> >
>

--001a114e58a8a4310605526f8796
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Spencer, Al (+Scott, +Warren), and all,=C2=A0<div><br><=
/div><div>Thank you for the suggestion. Both drafts have been updated with =
explaining what a data center network device unter test is for these tests.=
=C2=A0</div><div><br></div><div>Besides adding more clarity on both abstrac=
ts, and introduction, I added the following text in the introduction sectio=
n of each draft to specify what a data center Network DUT is for these draf=
ts.=C2=A0</div><div><br></div><div>I hope Scott will appreciate the low buf=
fer comment, as the high performance data center switches today have signif=
icantly less buffer than other networking devices.=C2=A0</div><div><br></di=
v><div><div>Currently, typical data Center networking devices are character=
ized</div><div>by:</div><div><br></div><div>-High port density (48 ports of=
 more)</div><div><br></div><div>-High speed (up to 100 Gb/s currently per p=
ort)</div><div><br></div><div>-High throughput (line rate on all ports for =
Layer 2 and/or Layer 3)</div><div><br></div><div>-Low latency (in the micro=
second or nanosecond range)</div><div><br></div><div>-Low amount of buffer =
(in the Mb range)</div><div><br></div><div>-Layer 2 and Layer 3 forwarding =
capability (Layer 3 not mandatory)</div></div></div><div class=3D"gmail_ext=
ra"><br><div class=3D"gmail_quote">On Tue, Jun 20, 2017 at 1:42 PM, Lucien =
Avramov <span dir=3D"ltr">&lt;<a href=3D"mailto:lucien.avramov@gmail.com" t=
arget=3D"_blank">lucien.avramov@gmail.com</a>&gt;</span> wrote:<br><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc s=
olid;padding-left:1ex">Definitely agreed, will place a statement in both do=
cuments today to explain what datacenter switches are at time of writing.<b=
r>
<br>
Thank you,<br>
Lucien<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
&gt; On Jun 20, 2017, at 13:28, Spencer Dawkins &lt;<a href=3D"mailto:spenc=
erdawkins.ietf@gmail.com">spencerdawkins.ietf@gmail.com</a><wbr>&gt; wrote:=
<br>
&gt;<br>
&gt; Spencer Dawkins has entered the following ballot position for<br>
&gt; draft-ietf-bmwg-dcbench-<wbr>methodology-13: No Objection<br>
&gt;<br>
&gt; When responding, please keep the subject line intact and reply to all<=
br>
&gt; email addresses included in the To and CC lines. (Feel free to cut thi=
s<br>
&gt; introductory paragraph, however.)<br>
&gt;<br>
&gt;<br>
&gt; Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss=
-criteria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/i=
esg/<wbr>statement/discuss-criteria.<wbr>html</a><br>
&gt; for more information about IESG DISCUSS and COMMENT positions.<br>
&gt;<br>
&gt;<br>
&gt; The document, along with other ballot positions, can be found here:<br=
>
&gt; <a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-me=
thodology/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.o=
rg/<wbr>doc/draft-ietf-bmwg-dcbench-<wbr>methodology/</a><br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt; COMMENT:<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt;<br>
&gt; (I&#39;m making the same comment on draft-ietf-bmwg-dcbench-<wbr>termi=
nology and<br>
&gt; draft-ietf-bmwg-dcbench-<wbr>methodology)<br>
&gt;<br>
&gt; I&#39;m looking at the ballot positions on draft-ietf-bmwg-dcbench-<wb=
r>terminology and<br>
&gt; draft-ietf-bmwg-dcbench-<wbr>methodology that assert these documents a=
ren&#39;t specific<br>
&gt; to &quot;data centers&quot;. That wouldn&#39;t surprise me, but I&#39;=
m not seeing a definition<br>
&gt; of &quot;data center&quot; in either document - did I miss it?<br>
&gt;<br>
&gt; I suspect that the authors have specific technical characteristics in =
mind,<br>
&gt; that happen to map onto what data centers look like today, but may not=
 in the<br>
&gt; future (&quot;RFCs last forever&quot;). Is it possible to tease those =
characteristics out?<br>
&gt;<br>
&gt; (Full disclosure: my first working group in the IETF could have been c=
alled<br>
&gt; &quot;TCP over cellular links&quot;, but it turned out that when we sa=
id &quot;cellular<br>
&gt; links&quot;, we meant &quot;low-speed links with high loss rates and a=
symmetric<br>
&gt; bandwidth&quot;. &quot;Cellular links&quot; in the late 1990s didn&#39=
;t have the same<br>
&gt; characteristics that &quot;cellular links&quot; have in 2017, but ther=
e are other link<br>
&gt; types with those characteristics, so the documents ended up being usef=
ul in<br>
&gt; places like CORE. I&#39;m not suggesting anything like a restructure o=
f the<br>
&gt; document(s), only that they be clear about how future readers would kn=
ow<br>
&gt; whether they should be reading them in 2027)<br>
&gt;<br>
&gt;<br>
</div></div></blockquote></div><br></div>

--001a114e58a8a4310605526f8796--


From nobody Tue Jun 20 20:02:18 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 15ED6126B6E; Tue, 20 Jun 2017 20:02:17 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149801413705.15924.9729654651455535125@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 20:02:17 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/pC95-WmCe4rLRDjbBvtbbXBA9C8>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-15.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 03:02:17 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-15.txt
	Pages           : 20
	Date            : 2017-06-20

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. A pre-requisite to this
   publication is the terminology document [1]. Many of these terms and
   methods may be applicable beyond this publication's scope as the
   technologies originally applied in the data center are deployed
   elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-15
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-15

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-15


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Jun 20 20:02:45 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DAFA13164D; Tue, 20 Jun 2017 20:02:38 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149801415853.15868.16183540846648407556@ietfa.amsl.com>
Date: Tue, 20 Jun 2017 20:02:38 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/u-b_IKmDpbU43Vo72RKru4G615s>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-16.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 03:02:39 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-16.txt
	Pages           : 17
	Date            : 2017-06-20

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to performance
evaluations of data center network equipment. This document establishes
the important concepts for benchmarking network switches and routers in
the data center and, is a pre-requisite to the test methodology
publication [1]. Many of these terms and methods may be applicable to
network equipment beyond this publication's scope as the technologies
originally applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-16
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-16

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-16


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Jun 20 20:17:50 2017
Return-Path: <sjacob@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9881712949F for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 20:17:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.021
X-Spam-Level: 
X-Spam-Status: No, score=-2.021 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dh3UcIGnp9hB for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 20:17:47 -0700 (PDT)
Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0137.outbound.protection.outlook.com [104.47.33.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC2BF126CC4 for <bmwg@ietf.org>; Tue, 20 Jun 2017 20:17:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=eUbWo7OWtoy5LkPeR+3q/SHl0HoZ/METBB9tjJF5l24=; b=EekoWF8WNGkp5VRRK+52DyuSrYKo5sStduVDPGFhSJlP+aqbYpo+9N/mFgYlno88mQ4MDLRpJS2stMTpWdGk3YhAelO3lw7BWjmdNyRdyAWUiVBOlfVzu+fpjmzTvjwTOkCFdbWu7c8xQzufKzpNsZXIS7WN2v+VPisnzHcKzHI=
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com (10.160.200.24) by BY1PR0501MB1367.namprd05.prod.outlook.com (10.160.200.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Wed, 21 Jun 2017 03:17:45 +0000
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) by BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) with mapi id 15.01.1199.012; Wed, 21 Jun 2017 03:17:45 +0000
From: Sudhin Jacob <sjacob@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: "MORTON, ALFRED C (AL)" <acmorton@att.com>, Sarah B <sbanks@encrypted.net>
Thread-Topic: bmwg Digest, Vol 153, Issue 25
Thread-Index: AQHS6fePT+vpxS4fPUa2Gn6Fr+FK+aIupZnQ
Date: Wed, 21 Jun 2017 03:17:44 +0000
Message-ID: <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
References: <mailman.89.1497985213.8807.bmwg@ietf.org>
In-Reply-To: <mailman.89.1497985213.8807.bmwg@ietf.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.12]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BY1PR0501MB1367; 7:zA3x0yhqYb3qUA65GSCnip5BEui8vISL+p9jKZOJX/rm5R38AaPC88SC8Vqv+59KIRXpvX6tj32POXnXeVlrH+Uz53RMSYe8Hq3taG777riA1hRlFkbwH9fs1EdKwdGYeVAsAyENap+dEJOvnbCSTVwT3pOW7hk57SMu8bSGBGb7PSquHNxUdA2xFI/41S7YWomdb7o8IfokCJY2Z6BiLk5jVs/cKTN4tDgrVuIKOKddho3ZoTatiRvrylVeYWutrSGle9rZly2JnazwT6S9v9L6w17kx5bRjOq6GtA2sQfusfeRtMNZoKXs6Yk4hDcZHE5LA2dHt4PHIyGd6AqDoHQZmvzDHaM7HNGM5y8ZRcyj6B44owWTbXCxHKIheH7ewM3GvIs/MN3O5Z84FuI9PkrolN/rOcaOV2qANrkZ+6VSYKdamydH09eeQx8MnDaTB0k8YQ7grg1qc2Tx9teMh6R+PCQxCYAev8005EQfpCHMEVi4TAsb/9fw7grXziRc0WPXOE3K74SkAv/OQR08jt0RVtNL9jpyAEgxterWOj1aEd86BwnL9HNGZVmvk5755YBPWwzwMyOxPzi0i42yoJnskVKRvvpnqvOTeQ4HribpR+Vdo6VHAEPiZKuU2M0EKD830AHzKAQ5kJEhWJq901ywqocB/d1PuGsdBxobo1t8yLnuHT59mPaYSacVf7qhgLXmcfyTOIxCDnYb27ByY0ldzphy4aTrDgdtfdxLa5JHqvc8Q1bbDn3X2mBUPJYz1SNm8JhT18KGMHwsNirf+QCa7S/qutOvOP5jn/YZ+24=
x-ms-office365-filtering-correlation-id: c8eb1947-8ec5-4f26-6d8c-08d4b85415e3
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081);  SRVR:BY1PR0501MB1367; 
x-ms-traffictypediagnostic: BY1PR0501MB1367:
x-microsoft-antispam-prvs: <BY1PR0501MB13673F0618EBE0B74628BDC1C2DA0@BY1PR0501MB1367.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(192374486261705)(97927398514766); 
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93001095)(6055026)(6041248)(20161123560025)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY1PR0501MB1367; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY1PR0501MB1367; 
x-forefront-prvs: 0345CFD558
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39840400002)(39450400003)(39410400002)(39400400002)(39860400002)(13464003)(18543002)(10533003)(377454003)(27574002)(5403001)(25786009)(38730400002)(6506006)(6306002)(55016002)(5660300001)(6436002)(8936002)(99286003)(54906002)(74316002)(2900100001)(2950100002)(966005)(6246003)(3846002)(2501003)(6916009)(305945005)(102836003)(1730700003)(14454004)(478600001)(4326008)(9686003)(6116002)(110136004)(7696004)(81166006)(8676002)(53936002)(7736002)(5640700003)(76176999)(512954002)(50986999)(54356999)(575784001)(229853002)(122556002)(33656002)(3280700002)(3660700001)(66066001)(86362001)(2351001)(189998001)(77096006)(2906002)(53546010); DIR:OUT; SFP:1102; SCL:1; SRVR:BY1PR0501MB1367; H:BY1PR0501MB1352.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2017 03:17:44.8666 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0501MB1367
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/JwjW8LiY3sgNLLONgjOMqqcJyII>
Subject: Re: [bmwg] bmwg Digest, Vol 153, Issue 25
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 03:17:50 -0000

Hi Al,

As an author of the draft. Let me answer the question.

whether you commit to reviewing the WG document if adopted?

Yes. I am committed to review the draft, if adopted.

Regards,
Sudhin


-----Original Message-----
From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmwg-request@ietf.or=
g
Sent: Wednesday, June 21, 2017 12:30 AM
To: bmwg@ietf.org
Subject: bmwg Digest, Vol 153, Issue 25

Send bmwg mailing list submissions to
	bmwg@ietf.org

To subscribe or unsubscribe via the World Wide Web, visit
	https://www.ietf.org/mailman/listinfo/bmwg
or, via email, send a message with subject or body 'help' to
	bmwg-request@ietf.org

You can reach the person managing the list at
	bmwg-owner@ietf.org

When replying, please edit your Subject line so it is more specific than "R=
e: Contents of bmwg digest..."


Today's Topics:

   1. Re: Call For Adoption: draft-kishjac-bmwg-evpntest-07
      (MORTON, ALFRED C (AL))


----------------------------------------------------------------------

Message: 1
Date: Tue, 20 Jun 2017 18:43:18 +0000
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
Message-ID:
	<4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.att.com>
Content-Type: text/plain; charset=3D"us-ascii"

BMWG,

So far, I count about eight people who offered support for the draft, and o=
ne person who made a technical comment along with their message.

*No one* has answered this question:
>=20
> (3) whether you commit to reviewing the WG document if adopted
>

With no commitments to review the draft and provide the necessary expertise=
 to develop appropriate benchmarks for our industry, it will not be appropr=
iate to add a milestone to our charter (or re-charter, if necessary).

There is still time for everyone to reconsider their position, and provide =
an answer to the question above.  The WGLC closes on Friday 23 June 2017.

thanks for your consideration,
Al
bmwg co-chair


> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED=20
> C
> (AL)
> Sent: Saturday, June 03, 2017 6:34 PM
> To: bmwg@ietf.org
> Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
>=20
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more=20
> information.
>=20
> BMWG,
>=20
> At our Chicago meeting, the BMWG reviewed a version of the Internet=20
> Draft on
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
>=20
> There was sufficient readership present at the meeting and=20
> positive/constructive comments on the work, so the chairs asked=20
> whether support existed to adopt the draft as a WG item. There was=20
> support and no objections. As always, such decisions are officially=20
> taken on the mailing list.
>=20
> This message*, therefore, starts a call for adoption on:
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
>=20
> to run until Friday 23 June 2017.
>=20
> Please reply to bmwg@ietf.org indicating:
>=20
> (1) whether you support addition of the following milestone to
>    the BMWG charter:
>=20
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
>=20
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>    as the basis document for this milestone
>=20
> (3) whether you commit to reviewing the WG document if adopted
>=20
> We are aware that BMWG has nearly completed all its existing chartered=20
> items.  This usually begins discussion of re-chartering, this time=20
> with Warren (our new AD Advisor). The final decision/schedule for=20
> adoption will depend on these discussions.
>=20
> thanks for your consideration of the draft, and regards,
>=20
> Al
> bmwg co-chair
>=20
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D3tXKQNBR8Q4HC7BBx6pc5me8=
u
> 9z Ne9QVo_owOF5-ciM&s=3DxIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=3D



------------------------------

Subject: Digest Footer

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


------------------------------

End of bmwg Digest, Vol 153, Issue 25
*************************************


From nobody Tue Jun 20 20:44:04 2017
Return-Path: <shabeer@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6B66B126DED for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 20:44:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.021
X-Spam-Level: 
X-Spam-Status: No, score=-2.021 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bkPPVHjZtZjW for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 20:44:00 -0700 (PDT)
Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0109.outbound.protection.outlook.com [104.47.32.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EE7F71204DA for <bmwg@ietf.org>; Tue, 20 Jun 2017 20:43:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ASXife95QrPXq+zkt6OhKQFSSBBCyf+NSI90UNWPziM=; b=VPqkaxZi14iUJV+uL8uKj8Jd2sSqPVn1j+MiBNszBevqkULjIF2Kb1W8Q6jqglSz9uEK7X4yYp9pbZ7WWNagrwGufQ4uos6uDz3UsbRa1Izk0w9xLrU7iaEADAULhXkDMFBXJJsfxbXK26jkUCDBPkoJ9kgYhno+W4hAkPzfw2w=
Received: from MWHPR05MB3264.namprd05.prod.outlook.com (10.173.230.15) by MWHPR05MB2877.namprd05.prod.outlook.com (10.168.245.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Wed, 21 Jun 2017 03:43:58 +0000
Received: from MWHPR05MB3264.namprd05.prod.outlook.com ([10.173.230.15]) by MWHPR05MB3264.namprd05.prod.outlook.com ([10.173.230.15]) with mapi id 15.01.1199.015; Wed, 21 Jun 2017 03:43:58 +0000
From: Mohammed Shabeer A S <shabeer@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: bmwg Digest, Vol 153, Issue 25
Thread-Index: AQHS6fePT+vpxS4fPUa2Gn6Fr+FK+aIupZnQgAAHG7A=
Date: Wed, 21 Jun 2017 03:43:58 +0000
Message-ID: <MWHPR05MB3264C8E8B3C361D5563B49F9BEDA0@MWHPR05MB3264.namprd05.prod.outlook.com>
References: <mailman.89.1497985213.8807.bmwg@ietf.org> <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
In-Reply-To: <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.14]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MWHPR05MB2877; 7:ldycrF9vM8hkgJisGsS3JTerNuHRUia2UrQvwusUTri5pK+vgsCVUYXcAZkEWOVIpRFK2MJ6uuzg2UInwNKoPYl2sOhFyL3jEaYgfz78Jrd0eLw7ilM11vxm6Pl53GBkZ0cu0rTYsWTpvp1b21kwxZUlF9Ii9nnnUysNo46Yvzst67anWaHYy0EiXUQaBf9jA1q5FA24qOaWr7CNuu8o8DWdJP4P0OEwBQdEtSFi0kZIFCDORLTaiBg4RnSdcJQzn5TnC0+EGym4Dtvug1Oej3aS8ok6Y2UXIFcVcUyBVl7zkAGSNd073wVwP+L9Cu4gha2WkrQ/4FwbeVzLKucj2KFQaDAoYE8+nZ1QC3a+RHcFkC/Nmu2EWBrjZkW7WvA9Knb9d/ItVyGo/XuB1Vrv4aymo1nQYdEwZU0FslH3EG/jKTWhW/1O4N58hSLEj92E5e+eVW1tJ+n/gCjZsMmWB4drXxEg6eAfQJNGIFxqVMPxjbHmTPZ2AL0TZFCgktQ8AeSk1Fw2VQbPFHJXb3rnLYKwcQyANVwFJz8gse/Fiyx259xjqEOTg3eXfG/5LY27CfFmeD+Nuu0F4mWhf2O55Eyu+QiTkj6s2ws7CcbdEFvTy3KmDkL4shr5J2WOJZCIBMzLeyHGH7qvQmY+kXCZbAzTA8Lu0fHTxM2+auEiQTxcHGlzDwGgw+tffmoodFJaAlLuYLWdQp9yKv64jPsN+spFm0QWw+Cd5Y5r2cXFq6sXw88P4DQm4ODYfHkoD2EPtvOdL0PaJtJIXcLenKuAtJmsmsTnQdqo75Zg/fMn7ig=
x-ms-office365-filtering-correlation-id: 3c272b06-eee6-4d2d-cb48-08d4b857bfbe
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081);  SRVR:MWHPR05MB2877; 
x-ms-traffictypediagnostic: MWHPR05MB2877:
x-microsoft-antispam-prvs: <MWHPR05MB287703BA0C15172F56674B8EBEDA0@MWHPR05MB2877.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(192374486261705)(97927398514766); 
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(6055026)(6041248)(20161123555025)(20161123560025)(20161123562025)(20161123564025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR05MB2877; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR05MB2877; 
x-forefront-prvs: 0345CFD558
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39410400002)(39860400002)(39840400002)(39450400003)(39400400002)(39850400002)(10533003)(18543002)(13464003)(377454003)(5403001)(27574002)(2906002)(74316002)(5660300001)(7736002)(2900100001)(8936002)(305945005)(512954002)(1730700003)(8676002)(81166006)(14454004)(478600001)(66066001)(2351001)(2501003)(966005)(7696004)(6436002)(6116002)(3846002)(99286003)(55016002)(6506006)(102836003)(25786009)(38730400002)(110136004)(3660700001)(3280700002)(122556002)(77096006)(33656002)(6306002)(53936002)(189998001)(9686003)(5640700003)(76176999)(6916009)(50986999)(86362001)(2950100002)(54356999)(575784001)(229853002)(53546010); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR05MB2877; H:MWHPR05MB3264.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2017 03:43:58.3762 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB2877
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/nmocS3XZdsv-TvcKO6E92KYWS0M>
Subject: Re: [bmwg] bmwg Digest, Vol 153, Issue 25
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 03:44:02 -0000

HI Al,

Yes ,I am committed to review this draft if adopted. Since PBB-EVPN and EVP=
N is widely deployed in networks it is very much essential to benchmark it.


Regards
Shabeer

or, via email, send a message with subject or body 'help' to
	bmwg-request@ietf.org

You can reach the person managing the list at
	bmwg-owner@ietf.org

When replying, please edit your Subject line so it is more specific than "R=
e: Contents of bmwg digest..."


Today's Topics:

   1. Re: Call For Adoption: draft-kishjac-bmwg-evpntest-07
      (MORTON, ALFRED C (AL))


----------------------------------------------------------------------

Message: 1
Date: Tue, 20 Jun 2017 18:43:18 +0000
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
Message-ID:
	<4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.att.com>
Content-Type: text/plain; charset=3D"us-ascii"

BMWG,

So far, I count about eight people who offered support for the draft, and o=
ne person who made a technical comment along with their message.

*No one* has answered this question:
>=20
> (3) whether you commit to reviewing the WG document if adopted
>

With no commitments to review the draft and provide the necessary expertise=
 to develop appropriate benchmarks for our industry, it will not be appropr=
iate to add a milestone to our charter (or re-charter, if necessary).

There is still time for everyone to reconsider their position, and provide =
an answer to the question above.  The WGLC closes on Friday 23 June 2017.

thanks for your consideration,
Al
bmwg co-chair


> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED=20
> C
> (AL)
> Sent: Saturday, June 03, 2017 6:34 PM
> To: bmwg@ietf.org
> Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
>=20
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more=20
> information.
>=20
> BMWG,
>=20
> At our Chicago meeting, the BMWG reviewed a version of the Internet=20
> Draft on
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
>=20
> There was sufficient readership present at the meeting and=20
> positive/constructive comments on the work, so the chairs asked=20
> whether support existed to adopt the draft as a WG item. There was=20
> support and no objections. As always, such decisions are officially=20
> taken on the mailing list.
>=20
> This message*, therefore, starts a call for adoption on:
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
>=20
> to run until Friday 23 June 2017.
>=20
> Please reply to bmwg@ietf.org indicating:
>=20
> (1) whether you support addition of the following milestone to
>    the BMWG charter:
>=20
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
>=20
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>    as the basis document for this milestone
>=20
> (3) whether you commit to reviewing the WG document if adopted
>=20
> We are aware that BMWG has nearly completed all its existing chartered=20
> items.  This usually begins discussion of re-chartering, this time=20
> with Warren (our new AD Advisor). The final decision/schedule for=20
> adoption will depend on these discussions.
>=20
> thanks for your consideration of the draft, and regards,
>=20
> Al
> bmwg co-chair
>=20
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D3tXKQNBR8Q4HC7BBx6pc5me8=
u
> 9z Ne9QVo_owOF5-ciM&s=3DxIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=3D



------------------------------

Subject: Digest Footer

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


------------------------------

End of bmwg Digest, Vol 153, Issue 25
*************************************

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


From nobody Tue Jun 20 21:48:11 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E47791242F5 for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 21:48:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1K3K-mYDEbUI for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 21:48:06 -0700 (PDT)
Received: from mail-pf0-x232.google.com (mail-pf0-x232.google.com [IPv6:2607:f8b0:400e:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D62C11315E1 for <bmwg@ietf.org>; Tue, 20 Jun 2017 21:48:06 -0700 (PDT)
Received: by mail-pf0-x232.google.com with SMTP id x63so79552571pff.3 for <bmwg@ietf.org>; Tue, 20 Jun 2017 21:48:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=JbqkTQbEZn3eaLInz4pmrWuWZxiZ2/td4ZxtvUD8LtE=; b=HzvJYwZ0w1LaWKXdO5AryOB1inYTB7PU24Kcv8GPTHxO3Ohj65J0af+tHi1h/Q91RW U+xAmEEleQbQoyLML2z9Zlag9hba5mwnyG4RuJlM3RD5c5gLHE+zJ32HqpSlv7IlpVHF PngrFCpdY6tiDTC5Z6BdmDhWzMzXuPM6CPiOvOxkUImEBwe3OyRKCVf44fRSiISCGc0Q HgZwKgEAt7VDFfNJPYASl0gHw3WVx/n38I57adth0hJoLFw6OEtkm/MSd9Td40iXxQJj rztNAjdJ7dpZiWocaYClEmiTxHFkwfm2NB52la7+5SDBO+OrIQu6V87J+4qZzW0iqXAd ksTw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=JbqkTQbEZn3eaLInz4pmrWuWZxiZ2/td4ZxtvUD8LtE=; b=qPuVc/2TXLup/gKRdSwSjsAcBTMYEGva2Aa6jf1T9BPWNajjg3J9GDwgWvmywKNbKd DTbmcvSPU8co+49/35kPIB8dFn3F6XIwjL36VL2aRS8VlTgkcDjiEmiaT0G4PYj1C8ZK B+eMPxNFl1B1bT4ZlOa4wdQ/beuD8F/d/QEy72jIZy7YLUfST7IhEReGhvRbDdpOTETO MjWcI5vVif3OL9/sgPT7CV0ATnafubVMq/LK1okJhmEMFxXYx6Vj8mNaix3O2r8usnzQ NhV4JmH3Hf4UfNv9uFt2u6dxqs//3yykYbYHPPFe9mrpn8kIk34oCAvT3wvFyM1bhIeQ UOuQ==
X-Gm-Message-State: AKS2vOwz4VDcFmVP7VDgj3kfEl31mW000Rn7x/fDbarLXwzqYUzpIyRr fQhRNERUXKL42b9XwJOxQdNguJQKI+NkNqU=
X-Received: by 10.84.231.15 with SMTP id f15mr40334555plk.131.1498020486266; Tue, 20 Jun 2017 21:48:06 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.168.5 with HTTP; Tue, 20 Jun 2017 21:47:45 -0700 (PDT)
In-Reply-To: <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
References: <mailman.89.1497985213.8807.bmwg@ietf.org> <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
From: Lucien Avramov <lucienav@google.com>
Date: Tue, 20 Jun 2017 21:47:45 -0700
Message-ID: <CALTEt=AchbjC+ik2-3ZBs=gXfReODhe_x9BubVk-WtUjYz2OvA@mail.gmail.com>
To: Sudhin Jacob <sjacob@juniper.net>
Cc: "bmwg@ietf.org" <bmwg@ietf.org>, "MORTON, ALFRED C (AL)" <acmorton@att.com>
Content-Type: multipart/alternative; boundary="f403043607d8f398f1055271122f"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/43N8-tkZY4Bwe4ql6B5TYWi_r-A>
Subject: Re: [bmwg] bmwg Digest, Vol 153, Issue 25
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 04:48:10 -0000

--f403043607d8f398f1055271122f
Content-Type: text/plain; charset="UTF-8"

One would hope that an author is committed to review what they write :-)

I am personally on the fence for this draft at this point.

Why would EVPN need a specific benchmark to start with, that is different
from standard benchmarking tools and techniques?



On Tue, Jun 20, 2017 at 8:17 PM, Sudhin Jacob <sjacob@juniper.net> wrote:

> Hi Al,
>
> As an author of the draft. Let me answer the question.
>
> whether you commit to reviewing the WG document if adopted?
>
> Yes. I am committed to review the draft, if adopted.
>
> Regards,
> Sudhin
>
>
> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of
> bmwg-request@ietf.org
> Sent: Wednesday, June 21, 2017 12:30 AM
> To: bmwg@ietf.org
> Subject: bmwg Digest, Vol 153, Issue 25
>
> Send bmwg mailing list submissions to
>         bmwg@ietf.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://www.ietf.org/mailman/listinfo/bmwg
> or, via email, send a message with subject or body 'help' to
>         bmwg-request@ietf.org
>
> You can reach the person managing the list at
>         bmwg-owner@ietf.org
>
> When replying, please edit your Subject line so it is more specific than
> "Re: Contents of bmwg digest..."
>
>
> Today's Topics:
>
>    1. Re: Call For Adoption: draft-kishjac-bmwg-evpntest-07
>       (MORTON, ALFRED C (AL))
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 20 Jun 2017 18:43:18 +0000
> From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
> To: "bmwg@ietf.org" <bmwg@ietf.org>
> Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
> Message-ID:
>         <4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.
> att.com>
> Content-Type: text/plain; charset="us-ascii"
>
> BMWG,
>
> So far, I count about eight people who offered support for the draft, and
> one person who made a technical comment along with their message.
>
> *No one* has answered this question:
> >
> > (3) whether you commit to reviewing the WG document if adopted
> >
>
> With no commitments to review the draft and provide the necessary
> expertise to develop appropriate benchmarks for our industry, it will not
> be appropriate to add a milestone to our charter (or re-charter, if
> necessary).
>
> There is still time for everyone to reconsider their position, and provide
> an answer to the question above.  The WGLC closes on Friday 23 June 2017.
>
> thanks for your consideration,
> Al
> bmwg co-chair
>
>
> > -----Original Message-----
> > From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED
> > C
> > (AL)
> > Sent: Saturday, June 03, 2017 6:34 PM
> > To: bmwg@ietf.org
> > Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
> >
> > *** Security Advisory: This Message Originated Outside of AT&T ***.
> > Reference http://cso.att.com/EmailSecurity/IDSP.html for more
> > information.
> >
> > BMWG,
> >
> > At our Chicago meeting, the BMWG reviewed a version of the Internet
> > Draft on
> >
> > Benchmarking Methodology for EVPN and PBB-EVPN
> > draft-kishjac-bmwg-evpntest-05
> >
> > There was sufficient readership present at the meeting and
> > positive/constructive comments on the work, so the chairs asked
> > whether support existed to adopt the draft as a WG item. There was
> > support and no objections. As always, such decisions are officially
> > taken on the mailing list.
> >
> > This message*, therefore, starts a call for adoption on:
> >
> > Benchmarking Methodology for EVPN and PBB-EVPN
> > draft-kishjac-bmwg-evpntest-07
> >
> > to run until Friday 23 June 2017.
> >
> > Please reply to bmwg@ietf.org indicating:
> >
> > (1) whether you support addition of the following milestone to
> >    the BMWG charter:
> >
> > date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
> >
> > (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
> >    as the basis document for this milestone
> >
> > (3) whether you commit to reviewing the WG document if adopted
> >
> > We are aware that BMWG has nearly completed all its existing chartered
> > items.  This usually begins discussion of re-chartering, this time
> > with Warren (our new AD Advisor). The final decision/schedule for
> > adoption will depend on these discussions.
> >
> > thanks for your consideration of the draft, and regards,
> >
> > Al
> > bmwg co-chair
> >
> > *using Brian Trammell's recent message to IPPM as a template (thanks!)
> >
> > _______________________________________________
> > bmwg mailing list
> > bmwg@ietf.org
> > https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__www.ietf.org_mailman_listinfo_bmwg&d=DwICAg&c=LFYZ-
> > o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=3tXKQNBR8Q4HC7BBx6pc5me8u
> > 9z Ne9QVo_owOF5-ciM&s=xIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>
>
> ------------------------------
>
> End of bmwg Digest, Vol 153, Issue 25
> *************************************
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>

--f403043607d8f398f1055271122f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">One would hope that an author is committed to review what =
they write :-)<div><br></div><div>I am personally on the fence for this dra=
ft at this point.</div><div><br></div><div>Why would EVPN need a specific b=
enchmark to start with, that is different from standard benchmarking tools =
and techniques?</div><div><br></div><div><br></div></div><div class=3D"gmai=
l_extra"><br><div class=3D"gmail_quote">On Tue, Jun 20, 2017 at 8:17 PM, Su=
dhin Jacob <span dir=3D"ltr">&lt;<a href=3D"mailto:sjacob@juniper.net" targ=
et=3D"_blank">sjacob@juniper.net</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">Hi Al,<br>
<br>
As an author of the draft. Let me answer the question.<br>
<br>
whether you commit to reviewing the WG document if adopted?<br>
<br>
Yes. I am committed to review the draft, if adopted.<br>
<br>
Regards,<br>
Sudhin<br>
<br>
<br>
-----Original Message-----<br>
From: bmwg [mailto:<a href=3D"mailto:bmwg-bounces@ietf.org">bmwg-bounces@ie=
tf.org</a>] On Behalf Of <a href=3D"mailto:bmwg-request@ietf.org">bmwg-requ=
est@ietf.org</a><br>
Sent: Wednesday, June 21, 2017 12:30 AM<br>
To: <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
Subject: bmwg Digest, Vol 153, Issue 25<br>
<br>
Send bmwg mailing list submissions to<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org<=
/a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"https://www.ietf.org/mailman/listinf=
o/bmwg" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/mailman/<=
wbr>listinfo/bmwg</a><br>
or, via email, send a message with subject or body &#39;help&#39; to<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"mailto:bmwg-request@ietf.org">bmwg-r=
equest@ietf.org</a><br>
<br>
You can reach the person managing the list at<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"mailto:bmwg-owner@ietf.org">bmwg-own=
er@ietf.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific than &q=
uot;Re: Contents of bmwg digest...&quot;<br>
<br>
<br>
Today&#39;s Topics:<br>
<br>
=C2=A0 =C2=A01. Re: Call For Adoption: draft-kishjac-bmwg-evpntest-07<br>
=C2=A0 =C2=A0 =C2=A0 (MORTON, ALFRED C (AL))<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
Message: 1<br>
Date: Tue, 20 Jun 2017 18:43:18 +0000<br>
From: &quot;MORTON, ALFRED C (AL)&quot; &lt;<a href=3D"mailto:acmorton@att.=
com">acmorton@att.com</a>&gt;<br>
To: &quot;<a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a>&quot; &lt;<a h=
ref=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a>&gt;<br>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07<br>
Message-ID:<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;<a href=3D"mailto:4D7F4AD313D3FC43A053B309F=
97543CF25FE116A@njmtexg5.research.att.com">4D7F4AD313D3FC43A053B309F9754<wb=
r>3CF25FE116A@njmtexg5.research.<wbr>att.com</a>&gt;<br>
Content-Type: text/plain; charset=3D&quot;us-ascii&quot;<br>
<br>
BMWG,<br>
<br>
So far, I count about eight people who offered support for the draft, and o=
ne person who made a technical comment along with their message.<br>
<br>
*No one* has answered this question:<br>
&gt;<br>
&gt; (3) whether you commit to reviewing the WG document if adopted<br>
&gt;<br>
<br>
With no commitments to review the draft and provide the necessary expertise=
 to develop appropriate benchmarks for our industry, it will not be appropr=
iate to add a milestone to our charter (or re-charter, if necessary).<br>
<br>
There is still time for everyone to reconsider their position, and provide =
an answer to the question above.=C2=A0 The WGLC closes on Friday 23 June 20=
17.<br>
<br>
thanks for your consideration,<br>
Al<br>
bmwg co-chair<br>
<br>
<br>
&gt; -----Original Message-----<br>
&gt; From: bmwg [mailto:<a href=3D"mailto:bmwg-bounces@ietf.org">bmwg-bounc=
es@ietf.org</a>] On Behalf Of MORTON, ALFRED<br>
&gt; C<br>
&gt; (AL)<br>
&gt; Sent: Saturday, June 03, 2017 6:34 PM<br>
&gt; To: <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
&gt; Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07<br>
&gt;<br>
&gt; *** Security Advisory: This Message Originated Outside of AT&amp;T ***=
.<br>
&gt; Reference <a href=3D"http://cso.att.com/EmailSecurity/IDSP.html" rel=
=3D"noreferrer" target=3D"_blank">http://cso.att.com/<wbr>EmailSecurity/IDS=
P.html</a> for more<br>
&gt; information.<br>
&gt;<br>
&gt; BMWG,<br>
&gt;<br>
&gt; At our Chicago meeting, the BMWG reviewed a version of the Internet<br=
>
&gt; Draft on<br>
&gt;<br>
&gt; Benchmarking Methodology for EVPN and PBB-EVPN<br>
&gt; draft-kishjac-bmwg-evpntest-05<br>
&gt;<br>
&gt; There was sufficient readership present at the meeting and<br>
&gt; positive/constructive comments on the work, so the chairs asked<br>
&gt; whether support existed to adopt the draft as a WG item. There was<br>
&gt; support and no objections. As always, such decisions are officially<br=
>
&gt; taken on the mailing list.<br>
&gt;<br>
&gt; This message*, therefore, starts a call for adoption on:<br>
&gt;<br>
&gt; Benchmarking Methodology for EVPN and PBB-EVPN<br>
&gt; draft-kishjac-bmwg-evpntest-07<br>
&gt;<br>
&gt; to run until Friday 23 June 2017.<br>
&gt;<br>
&gt; Please reply to <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a> ind=
icating:<br>
&gt;<br>
&gt; (1) whether you support addition of the following milestone to<br>
&gt;=C2=A0 =C2=A0 the BMWG charter:<br>
&gt;<br>
&gt; date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG<b=
r>
&gt;<br>
&gt; (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07=
<br>
&gt;=C2=A0 =C2=A0 as the basis document for this milestone<br>
&gt;<br>
&gt; (3) whether you commit to reviewing the WG document if adopted<br>
&gt;<br>
&gt; We are aware that BMWG has nearly completed all its existing chartered=
<br>
&gt; items.=C2=A0 This usually begins discussion of re-chartering, this tim=
e<br>
&gt; with Warren (our new AD Advisor). The final decision/schedule for<br>
&gt; adoption will depend on these discussions.<br>
&gt;<br>
&gt; thanks for your consideration of the draft, and regards,<br>
&gt;<br>
&gt; Al<br>
&gt; bmwg co-chair<br>
&gt;<br>
&gt; *using Brian Trammell&#39;s recent message to IPPM as a template (than=
ks!)<br>
&gt;<br>
&gt; ______________________________<wbr>_________________<br>
&gt; bmwg mailing list<br>
&gt; <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
&gt; <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-" rel=3D=
"noreferrer" target=3D"_blank">https://urldefense.proofpoint.<wbr>com/v2/ur=
l?u=3Dhttps-</a><br>
&gt; 3A__www.ietf.org_mailman_<wbr>listinfo_bmwg&amp;d=3DDwICAg&amp;c=3DLFY=
Z-<br>
&gt; o9_HUMeMTSQicvjIg&amp;r=3D<wbr>OfsSu8kTIltVyD1oL72cBw&amp;m=3D<wbr>3tX=
KQNBR8Q4HC7BBx6pc5me8u<br>
&gt; 9z Ne9QVo_owOF5-ciM&amp;s=3D<wbr>xIKgDP2zI9F0IvfisGlEBjIij3I_<wbr>H95z=
kVWWJGP8zlg&amp;e=3D<br>
<br>
<br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
______________________________<wbr>_________________<br>
bmwg mailing list<br>
<a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/bmwg" rel=3D"noreferrer" t=
arget=3D"_blank">https://www.ietf.org/mailman/<wbr>listinfo/bmwg</a><br>
<br>
<br>
------------------------------<br>
<br>
End of bmwg Digest, Vol 153, Issue 25<br>
******************************<wbr>*******<br>
<br>
______________________________<wbr>_________________<br>
bmwg mailing list<br>
<a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
<a href=3D"https://www.ietf.org/mailman/listinfo/bmwg" rel=3D"noreferrer" t=
arget=3D"_blank">https://www.ietf.org/mailman/<wbr>listinfo/bmwg</a><br>
</blockquote></div><br></div>

--f403043607d8f398f1055271122f--


From nobody Tue Jun 20 22:24:31 2017
Return-Path: <sjacob@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 082F9126CC7 for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 22:24:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.02
X-Spam-Level: 
X-Spam-Status: No, score=-2.02 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zbvhWFkma_nC for <bmwg@ietfa.amsl.com>; Tue, 20 Jun 2017 22:24:26 -0700 (PDT)
Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0132.outbound.protection.outlook.com [104.47.33.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 96CE51201F2 for <bmwg@ietf.org>; Tue, 20 Jun 2017 22:24:25 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=up3jcGobwtT/XqT8hQlw+IykaI3EqkUbFJfee2FAJPc=; b=drQTkOUShdVr2QehJF1wsYhZ2+VoOZxYykfSGiIsE9jH1OjYvUsZbUkkZnwwlutboYVJZyGf+mTF7YggHYAAMWjUS9JUT7tidwHRaISA1rPi16ifPynLhgb/nllWpckV0Ft67zw6i8QB4MH6m6DJg0DVW7sHSwHG/zNVa12ATQ0=
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com (10.160.200.24) by BY1PR0501MB1575.namprd05.prod.outlook.com (10.160.203.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Wed, 21 Jun 2017 05:24:24 +0000
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) by BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) with mapi id 15.01.1199.012; Wed, 21 Jun 2017 05:24:23 +0000
From: Sudhin Jacob <sjacob@juniper.net>
To: Lucien Avramov <lucienav@google.com>
CC: "bmwg@ietf.org" <bmwg@ietf.org>, "MORTON, ALFRED C (AL)" <acmorton@att.com>
Thread-Topic: [bmwg] bmwg Digest, Vol 153, Issue 25
Thread-Index: AQHS6fePT+vpxS4fPUa2Gn6Fr+FK+aIupZnQgAAZhICAAAePEA==
Date: Wed, 21 Jun 2017 05:24:23 +0000
Message-ID: <BY1PR0501MB13525E3A520D0117F36FAEF9C2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
References: <mailman.89.1497985213.8807.bmwg@ietf.org> <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com> <CALTEt=AchbjC+ik2-3ZBs=gXfReODhe_x9BubVk-WtUjYz2OvA@mail.gmail.com>
In-Reply-To: <CALTEt=AchbjC+ik2-3ZBs=gXfReODhe_x9BubVk-WtUjYz2OvA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: google.com; dkim=none (message not signed) header.d=none;google.com; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.12]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BY1PR0501MB1575; 7:90yGq9R2r1izng5kyCIc5wTXLTYntFeZN0aDwlYehtYGtc8UfSN4E/pToL2mq4tmWRKSwKEnuq7N0+nal98VzIyaJ0H1/y7mUwYPGbLh8ITnKLjNF3y6fuJDJBIWzHfjOLN/wuBI4S1TP48SRuucrV0eQmknFCHmQlR/KGD9T7D5dCoq/rx+MX6nslPofKXdzXqKDyeFxWr3Ldi+18YpYYPwsU3Hh4q3v+mnyPnw9mM2lGpYNbsNvCBho3HnPHWHI85brl/V+pe8aZXGGV5W8O3Z64+rooIYuYLcPqRSHwMW4ZeQkS2rLmPpzNx4vsG/56aXqXZH5q6VRreCT8R4AheZLmb/Uf9AagK5ZalNQKs62VcQbayfNqez4HyBiCzT6b2Pu6rYEgzAxGmmrbQ0yFPGNO/3gAMVOs+BKi1lqDgP7xH3QI/c5C82Y+FvzUGyLiKo/S9mzWmeZ7ZXajdr9Z++vPpeujr99WUc7JAYlLGVAGo6Va5YFQVLrp4/sG+o0RYoEcZZ1MT/D0W4nZBxtoHiun7FaBM9TXu3bbe1UjxXNWGZFNlXFrldmUqsbOx65xnSlalmHnPwRV+hpqh+6V9oaRSv2N1wyV5Ixdo9Yk3g30qLHURKuttn01Cf4rm1nUW9gZfvt8owGPnClJD/VoQhjkutHhqm1ihaVfvrDyONr7n9Zow7vVlMlZdxnIlT9eppx1HVOJNanwVch9JYgkHH+VNa8WGJ0LeRWrNxnn33hQyJYTUzOHAtHpT3LT8TzyhoBoVhjtbdqNCt3ikjriMJNfG5zHklin+tar806aE=
x-ms-office365-filtering-correlation-id: b521b104-d320-4d80-4201-08d4b865c73c
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081)(201702281549075); SRVR:BY1PR0501MB1575; 
x-ms-traffictypediagnostic: BY1PR0501MB1575:
x-microsoft-antispam-prvs: <BY1PR0501MB1575941B3D9745622391DDACC2DA0@BY1PR0501MB1575.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(192374486261705)(138986009662008)(97927398514766)(211936372134217)(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123562025)(20161123555025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY1PR0501MB1575; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY1PR0501MB1575; 
x-forefront-prvs: 0345CFD558
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39450400003)(39850400002)(39840400002)(39400400002)(39410400002)(39860400002)(13464003)(27574002)(18543002)(24454002)(10533003)(5403001)(377454003)(54896002)(236005)(6306002)(9686003)(606005)(6436002)(55016002)(99286003)(25786009)(53936002)(76176999)(54356999)(4326008)(38730400002)(77096006)(110136004)(50986999)(7696004)(9326002)(3660700001)(8676002)(81166006)(74316002)(3280700002)(8936002)(512954002)(6116002)(102836003)(790700001)(3846002)(2900100001)(6506006)(189998001)(33656002)(7906003)(14454004)(5660300001)(122556002)(2906002)(966005)(66066001)(7736002)(86362001)(575784001)(478600001)(6916009)(2950100002)(229853002)(53546010)(19609705001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY1PR0501MB1575; H:BY1PR0501MB1352.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_BY1PR0501MB13525E3A520D0117F36FAEF9C2DA0BY1PR0501MB1352_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2017 05:24:23.8655 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0501MB1575
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/VqmtK7r483QME33hBgM-LVeWBCM>
Subject: Re: [bmwg] bmwg Digest, Vol 153, Issue 25
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 05:24:30 -0000

--_000_BY1PR0501MB13525E3A520D0117F36FAEF9C2DA0BY1PR0501MB1352_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SGkgTHVjaWVuLA0KDQpMZXQgbWUgZXhwbGFpbiBteSB2aWV3LiBJIHNhaWQgSSBhbSBjb21taXR0
ZWQgZm9yIGVkaXRpbmcgY2hhbmdlcywgcHVibGlzaGluZyBldGMuIEFzIGFuIGF1dGhvciBJIGV4
cHJlc3NlZCBteSBjb21taXRtZW50IHRvIHRoZSBjb21tdW5pdHkuICBJIHRoaW5rIEkgaGF2ZSB0
aGUgbGliZXJ0eSB0byAgc2hhcmUgbXkgdmlldyBjYW4gSSA/LiAgS2luZGx5IGNvcnJlY3QgbWUg
aWYgSSBhbSB3cm9uZyBpcyB0aGF0IGFuIG9mZmVuY2U/Lg0KDQpBcyB5b3VyIHNlY29uZCBxdWVz
dGlvbiBLaW5kbHkgc2VlIHRoZSBhbnN3ZXIgaW5saW5lLg0KDQpSZWdhcmRzLA0KU3VkaGluDQoN
CkZyb206IEx1Y2llbiBBdnJhbW92IFttYWlsdG86bHVjaWVuYXZAZ29vZ2xlLmNvbV0NClNlbnQ6
IFdlZG5lc2RheSwgSnVuZSAyMSwgMjAxNyAxMDoxOCBBTQ0KVG86IFN1ZGhpbiBKYWNvYiA8c2ph
Y29iQGp1bmlwZXIubmV0Pg0KQ2M6IGJtd2dAaWV0Zi5vcmc7IE1PUlRPTiwgQUxGUkVEIEMgKEFM
KSA8YWNtb3J0b25AYXR0LmNvbT4NClN1YmplY3Q6IFJlOiBbYm13Z10gYm13ZyBEaWdlc3QsIFZv
bCAxNTMsIElzc3VlIDI1DQoNCk9uZSB3b3VsZCBob3BlIHRoYXQgYW4gYXV0aG9yIGlzIGNvbW1p
dHRlZCB0byByZXZpZXcgd2hhdCB0aGV5IHdyaXRlIDotKQ0KDQpJIGFtIHBlcnNvbmFsbHkgb24g
dGhlIGZlbmNlIGZvciB0aGlzIGRyYWZ0IGF0IHRoaXMgcG9pbnQuDQoNCldoeSB3b3VsZCBFVlBO
IG5lZWQgYSBzcGVjaWZpYyBiZW5jaG1hcmsgdG8gc3RhcnQgd2l0aCwgdGhhdCBpcyBkaWZmZXJl
bnQgZnJvbSBzdGFuZGFyZCBiZW5jaG1hcmtpbmcgdG9vbHMgYW5kIHRlY2huaXF1ZXM/DQoNClN1
ZGhpbj4+Pj4gRVZQTiBhbmQgUEJCLUVWUE4gIGlzICByZWxhdGl2ZWx5IG5ldyBSRkMgd2hpY2gg
Y2FtZSBhIGNvdXBsZSBvZiB5ZWFycyBiYWNrIGFuZCBpdCBpcyB3aWRlbHkgZGVwbG95ZWQgYW5k
IGR1ZSB0byBnYXAgaW4gdGhlIGN1cnJlbnQgdGVjaG5pcXVlIGl0IHdhcyBuZWVkZWQgdG8gYmVu
Y2htYXJrIHRoZXNlIGZvciB0aGUgYmVuZWZpdCBvZiBlbmQgY3VzdG9tZXJzLg0KTWF5IEkgcmVx
dWVzdCB5b3UgdG8gZ28gdGhyb3VnaCB0aGUgZHJhZnQgYW5kIHBsZWFzZSBmZWVsIGZyZWUgdG8g
cmVhY2ggb3V0IHRvIG1lIGZvciBhbnkgcXVlcmllcywgSSBsb3ZlIHRvIGV4cGxhaW4gaXQuDQoN
ClJlZ2FyZHMsDQpTdWRoaW4NCg0KDQoNCk9uIFR1ZSwgSnVuIDIwLCAyMDE3IGF0IDg6MTcgUE0s
IFN1ZGhpbiBKYWNvYiA8c2phY29iQGp1bmlwZXIubmV0PG1haWx0bzpzamFjb2JAanVuaXBlci5u
ZXQ+PiB3cm90ZToNCkhpIEFsLA0KDQpBcyBhbiBhdXRob3Igb2YgdGhlIGRyYWZ0LiBMZXQgbWUg
YW5zd2VyIHRoZSBxdWVzdGlvbi4NCg0Kd2hldGhlciB5b3UgY29tbWl0IHRvIHJldmlld2luZyB0
aGUgV0cgZG9jdW1lbnQgaWYgYWRvcHRlZD8NCg0KWWVzLiBJIGFtIGNvbW1pdHRlZCB0byByZXZp
ZXcgdGhlIGRyYWZ0LCBpZiBhZG9wdGVkLg0KDQpSZWdhcmRzLA0KU3VkaGluDQoNCg0KLS0tLS1P
cmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IGJtd2cgW21haWx0bzpibXdnLWJvdW5jZXNAaWV0
Zi5vcmc8bWFpbHRvOmJtd2ctYm91bmNlc0BpZXRmLm9yZz5dIE9uIEJlaGFsZiBPZiBibXdnLXJl
cXVlc3RAaWV0Zi5vcmc8bWFpbHRvOmJtd2ctcmVxdWVzdEBpZXRmLm9yZz4NClNlbnQ6IFdlZG5l
c2RheSwgSnVuZSAyMSwgMjAxNyAxMjozMCBBTQ0KVG86IGJtd2dAaWV0Zi5vcmc8bWFpbHRvOmJt
d2dAaWV0Zi5vcmc+DQpTdWJqZWN0OiBibXdnIERpZ2VzdCwgVm9sIDE1MywgSXNzdWUgMjUNCg0K
U2VuZCBibXdnIG1haWxpbmcgbGlzdCBzdWJtaXNzaW9ucyB0bw0KICAgICAgICBibXdnQGlldGYu
b3JnPG1haWx0bzpibXdnQGlldGYub3JnPg0KDQpUbyBzdWJzY3JpYmUgb3IgdW5zdWJzY3JpYmUg
dmlhIHRoZSBXb3JsZCBXaWRlIFdlYiwgdmlzaXQNCiAgICAgICAgaHR0cHM6Ly93d3cuaWV0Zi5v
cmcvbWFpbG1hbi9saXN0aW5mby9ibXdnDQpvciwgdmlhIGVtYWlsLCBzZW5kIGEgbWVzc2FnZSB3
aXRoIHN1YmplY3Qgb3IgYm9keSAnaGVscCcgdG8NCiAgICAgICAgYm13Zy1yZXF1ZXN0QGlldGYu
b3JnPG1haWx0bzpibXdnLXJlcXVlc3RAaWV0Zi5vcmc+DQoNCllvdSBjYW4gcmVhY2ggdGhlIHBl
cnNvbiBtYW5hZ2luZyB0aGUgbGlzdCBhdA0KICAgICAgICBibXdnLW93bmVyQGlldGYub3JnPG1h
aWx0bzpibXdnLW93bmVyQGlldGYub3JnPg0KDQpXaGVuIHJlcGx5aW5nLCBwbGVhc2UgZWRpdCB5
b3VyIFN1YmplY3QgbGluZSBzbyBpdCBpcyBtb3JlIHNwZWNpZmljIHRoYW4gIlJlOiBDb250ZW50
cyBvZiBibXdnIGRpZ2VzdC4uLiINCg0KDQpUb2RheSdzIFRvcGljczoNCg0KICAgMS4gUmU6IENh
bGwgRm9yIEFkb3B0aW9uOiBkcmFmdC1raXNoamFjLWJtd2ctZXZwbnRlc3QtMDcNCiAgICAgIChN
T1JUT04sIEFMRlJFRCBDIChBTCkpDQoNCg0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQpNZXNzYWdlOiAxDQpE
YXRlOiBUdWUsIDIwIEp1biAyMDE3IDE4OjQzOjE4ICswMDAwDQpGcm9tOiAiTU9SVE9OLCBBTEZS
RUQgQyAoQUwpIiA8YWNtb3J0b25AYXR0LmNvbTxtYWlsdG86YWNtb3J0b25AYXR0LmNvbT4+DQpU
bzogImJtd2dAaWV0Zi5vcmc8bWFpbHRvOmJtd2dAaWV0Zi5vcmc+IiA8Ym13Z0BpZXRmLm9yZzxt
YWlsdG86Ym13Z0BpZXRmLm9yZz4+DQpTdWJqZWN0OiBSZTogW2Jtd2ddIENhbGwgRm9yIEFkb3B0
aW9uOiBkcmFmdC1raXNoamFjLWJtd2ctZXZwbnRlc3QtMDcNCk1lc3NhZ2UtSUQ6DQogICAgICAg
IDw0RDdGNEFEMzEzRDNGQzQzQTA1M0IzMDlGOTc1NDNDRjI1RkUxMTZBQG5qbXRleGc1LnJlc2Vh
cmNoLmF0dC5jb208bWFpbHRvOjREN0Y0QUQzMTNEM0ZDNDNBMDUzQjMwOUY5NzU0M0NGMjVGRTEx
NkFAbmptdGV4ZzUucmVzZWFyY2guYXR0LmNvbT4+DQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47
IGNoYXJzZXQ9InVzLWFzY2lpIg0KDQpCTVdHLA0KDQpTbyBmYXIsIEkgY291bnQgYWJvdXQgZWln
aHQgcGVvcGxlIHdobyBvZmZlcmVkIHN1cHBvcnQgZm9yIHRoZSBkcmFmdCwgYW5kIG9uZSBwZXJz
b24gd2hvIG1hZGUgYSB0ZWNobmljYWwgY29tbWVudCBhbG9uZyB3aXRoIHRoZWlyIG1lc3NhZ2Uu
DQoNCipObyBvbmUqIGhhcyBhbnN3ZXJlZCB0aGlzIHF1ZXN0aW9uOg0KPg0KPiAoMykgd2hldGhl
ciB5b3UgY29tbWl0IHRvIHJldmlld2luZyB0aGUgV0cgZG9jdW1lbnQgaWYgYWRvcHRlZA0KPg0K
DQpXaXRoIG5vIGNvbW1pdG1lbnRzIHRvIHJldmlldyB0aGUgZHJhZnQgYW5kIHByb3ZpZGUgdGhl
IG5lY2Vzc2FyeSBleHBlcnRpc2UgdG8gZGV2ZWxvcCBhcHByb3ByaWF0ZSBiZW5jaG1hcmtzIGZv
ciBvdXIgaW5kdXN0cnksIGl0IHdpbGwgbm90IGJlIGFwcHJvcHJpYXRlIHRvIGFkZCBhIG1pbGVz
dG9uZSB0byBvdXIgY2hhcnRlciAob3IgcmUtY2hhcnRlciwgaWYgbmVjZXNzYXJ5KS4NCg0KVGhl
cmUgaXMgc3RpbGwgdGltZSBmb3IgZXZlcnlvbmUgdG8gcmVjb25zaWRlciB0aGVpciBwb3NpdGlv
biwgYW5kIHByb3ZpZGUgYW4gYW5zd2VyIHRvIHRoZSBxdWVzdGlvbiBhYm92ZS4gIFRoZSBXR0xD
IGNsb3NlcyBvbiBGcmlkYXkgMjMgSnVuZSAyMDE3Lg0KDQp0aGFua3MgZm9yIHlvdXIgY29uc2lk
ZXJhdGlvbiwNCkFsDQpibXdnIGNvLWNoYWlyDQoNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut
LS0tLQ0KPiBGcm9tOiBibXdnIFttYWlsdG86Ym13Zy1ib3VuY2VzQGlldGYub3JnPG1haWx0bzpi
bXdnLWJvdW5jZXNAaWV0Zi5vcmc+XSBPbiBCZWhhbGYgT2YgTU9SVE9OLCBBTEZSRUQNCj4gQw0K
PiAoQUwpDQo+IFNlbnQ6IFNhdHVyZGF5LCBKdW5lIDAzLCAyMDE3IDY6MzQgUE0NCj4gVG86IGJt
d2dAaWV0Zi5vcmc8bWFpbHRvOmJtd2dAaWV0Zi5vcmc+DQo+IFN1YmplY3Q6IFtibXdnXSBDYWxs
IEZvciBBZG9wdGlvbjogZHJhZnQta2lzaGphYy1ibXdnLWV2cG50ZXN0LTA3DQo+DQo+ICoqKiBT
ZWN1cml0eSBBZHZpc29yeTogVGhpcyBNZXNzYWdlIE9yaWdpbmF0ZWQgT3V0c2lkZSBvZiBBVCZU
ICoqKi4NCj4gUmVmZXJlbmNlIGh0dHA6Ly9jc28uYXR0LmNvbS9FbWFpbFNlY3VyaXR5L0lEU1Au
aHRtbCBmb3IgbW9yZQ0KPiBpbmZvcm1hdGlvbi4NCj4NCj4gQk1XRywNCj4NCj4gQXQgb3VyIENo
aWNhZ28gbWVldGluZywgdGhlIEJNV0cgcmV2aWV3ZWQgYSB2ZXJzaW9uIG9mIHRoZSBJbnRlcm5l
dA0KPiBEcmFmdCBvbg0KPg0KPiBCZW5jaG1hcmtpbmcgTWV0aG9kb2xvZ3kgZm9yIEVWUE4gYW5k
IFBCQi1FVlBODQo+IGRyYWZ0LWtpc2hqYWMtYm13Zy1ldnBudGVzdC0wNQ0KPg0KPiBUaGVyZSB3
YXMgc3VmZmljaWVudCByZWFkZXJzaGlwIHByZXNlbnQgYXQgdGhlIG1lZXRpbmcgYW5kDQo+IHBv
c2l0aXZlL2NvbnN0cnVjdGl2ZSBjb21tZW50cyBvbiB0aGUgd29yaywgc28gdGhlIGNoYWlycyBh
c2tlZA0KPiB3aGV0aGVyIHN1cHBvcnQgZXhpc3RlZCB0byBhZG9wdCB0aGUgZHJhZnQgYXMgYSBX
RyBpdGVtLiBUaGVyZSB3YXMNCj4gc3VwcG9ydCBhbmQgbm8gb2JqZWN0aW9ucy4gQXMgYWx3YXlz
LCBzdWNoIGRlY2lzaW9ucyBhcmUgb2ZmaWNpYWxseQ0KPiB0YWtlbiBvbiB0aGUgbWFpbGluZyBs
aXN0Lg0KPg0KPiBUaGlzIG1lc3NhZ2UqLCB0aGVyZWZvcmUsIHN0YXJ0cyBhIGNhbGwgZm9yIGFk
b3B0aW9uIG9uOg0KPg0KPiBCZW5jaG1hcmtpbmcgTWV0aG9kb2xvZ3kgZm9yIEVWUE4gYW5kIFBC
Qi1FVlBODQo+IGRyYWZ0LWtpc2hqYWMtYm13Zy1ldnBudGVzdC0wNw0KPg0KPiB0byBydW4gdW50
aWwgRnJpZGF5IDIzIEp1bmUgMjAxNy4NCj4NCj4gUGxlYXNlIHJlcGx5IHRvIGJtd2dAaWV0Zi5v
cmc8bWFpbHRvOmJtd2dAaWV0Zi5vcmc+IGluZGljYXRpbmc6DQo+DQo+ICgxKSB3aGV0aGVyIHlv
dSBzdXBwb3J0IGFkZGl0aW9uIG9mIHRoZSBmb2xsb3dpbmcgbWlsZXN0b25lIHRvDQo+ICAgIHRo
ZSBCTVdHIGNoYXJ0ZXI6DQo+DQo+IGRhdGUgVEJEOiBCZW5jaG1hcmtpbmcgTWV0aG9kb2xvZ3kg
Zm9yIEVWUE4gYW5kIFBCQi1FVlBOIHRvIHRoZSBJRVNHDQo+DQo+ICgyKSB3aGV0aGVyIHlvdSBz
dXBwb3J0IHRoZSBhZG9wdGlvbiBvZiBkcmFmdC1raXNoamFjLWJtd2ctZXZwbnRlc3QtMDcNCj4g
ICAgYXMgdGhlIGJhc2lzIGRvY3VtZW50IGZvciB0aGlzIG1pbGVzdG9uZQ0KPg0KPiAoMykgd2hl
dGhlciB5b3UgY29tbWl0IHRvIHJldmlld2luZyB0aGUgV0cgZG9jdW1lbnQgaWYgYWRvcHRlZA0K
Pg0KPiBXZSBhcmUgYXdhcmUgdGhhdCBCTVdHIGhhcyBuZWFybHkgY29tcGxldGVkIGFsbCBpdHMg
ZXhpc3RpbmcgY2hhcnRlcmVkDQo+IGl0ZW1zLiAgVGhpcyB1c3VhbGx5IGJlZ2lucyBkaXNjdXNz
aW9uIG9mIHJlLWNoYXJ0ZXJpbmcsIHRoaXMgdGltZQ0KPiB3aXRoIFdhcnJlbiAob3VyIG5ldyBB
RCBBZHZpc29yKS4gVGhlIGZpbmFsIGRlY2lzaW9uL3NjaGVkdWxlIGZvcg0KPiBhZG9wdGlvbiB3
aWxsIGRlcGVuZCBvbiB0aGVzZSBkaXNjdXNzaW9ucy4NCj4NCj4gdGhhbmtzIGZvciB5b3VyIGNv
bnNpZGVyYXRpb24gb2YgdGhlIGRyYWZ0LCBhbmQgcmVnYXJkcywNCj4NCj4gQWwNCj4gYm13ZyBj
by1jaGFpcg0KPg0KPiAqdXNpbmcgQnJpYW4gVHJhbW1lbGwncyByZWNlbnQgbWVzc2FnZSB0byBJ
UFBNIGFzIGEgdGVtcGxhdGUgKHRoYW5rcyEpDQo+DQo+IF9fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fDQo+IGJtd2cgbWFpbGluZyBsaXN0DQo+IGJtd2dAaWV0
Zi5vcmc8bWFpbHRvOmJtd2dAaWV0Zi5vcmc+DQo+IGh0dHBzOi8vdXJsZGVmZW5zZS5wcm9vZnBv
aW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0FfX3d3dy5pZXRmLm9yZ19tYWlsbWFuX2xpc3Rp
bmZvX2Jtd2cmZD1Ed0lDQWcmYz1MRllaLQ0KPiBvOV9IVU1lTVRTUWljdmpJZyZyPU9mc1N1OGtU
SWx0VnlEMW9MNzJjQncmbT0zdFhLUU5CUjhRNEhDN0JCeDZwYzVtZTh1DQo+IDl6IE5lOVFWb19v
d09GNS1jaU0mcz14SUtnRFAyekk5RjBJdmZpc0dsRUJqSWlqM0lfSDk1emtWV1dKR1A4emxnJmU9
DQoNCg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KU3ViamVjdDogRGlnZXN0
IEZvb3Rlcg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
Xw0KYm13ZyBtYWlsaW5nIGxpc3QNCmJtd2dAaWV0Zi5vcmc8bWFpbHRvOmJtd2dAaWV0Zi5vcmc+
DQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Jtd2cNCg0KDQotLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KRW5kIG9mIGJtd2cgRGlnZXN0LCBWb2wgMTUzLCBJ
c3N1ZSAyNQ0KKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKg0KDQpfX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KYm13ZyBtYWlsaW5nIGxp
c3QNCmJtd2dAaWV0Zi5vcmc8bWFpbHRvOmJtd2dAaWV0Zi5vcmc+DQpodHRwczovL3d3dy5pZXRm
Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Jtd2cNCg0K

--_000_BY1PR0501MB13525E3A520D0117F36FAEF9C2DA0BY1PR0501MB1352_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl
PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6
IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m
YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy
IDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWws
IGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsNCgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJ
Zm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9
DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCglj
b2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KYTp2aXNpdGVkLCBzcGFu
Lk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpw
dXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQpwLm1zb25vcm1hbDAsIGxpLm1z
b25vcm1hbDAsIGRpdi5tc29ub3JtYWwwDQoJe21zby1zdHlsZS1uYW1lOm1zb25vcm1hbDsNCglt
c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzsNCgltYXJnaW4tcmlnaHQ6MGluOw0KCW1zby1tYXJnaW4t
Ym90dG9tLWFsdDphdXRvOw0KCW1hcmdpbi1sZWZ0OjBpbjsNCglmb250LXNpemU6MTIuMHB0Ow0K
CWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmO30NCnNwYW4uRW1haWxTdHlsZTE4
DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJp
IixzYW5zLXNlcmlmOw0KCWNvbG9yOiMxRjQ5N0Q7fQ0KLk1zb0NocERlZmF1bHQNCgl7bXNvLXN0
eWxlLXR5cGU6ZXhwb3J0LW9ubHk7DQoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7
fQ0KQHBhZ2UgV29yZFNlY3Rpb24xDQoJe3NpemU6OC41aW4gMTEuMGluOw0KCW1hcmdpbjoxLjBp
biAxLjBpbiAxLjBpbiAxLjBpbjt9DQpkaXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rp
b24xO30NCi0tPjwvc3R5bGU+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWRlZmF1
bHRzIHY6ZXh0PSJlZGl0IiBzcGlkbWF4PSIxMDI2IiAvPg0KPC94bWw+PCFbZW5kaWZdLS0+PCEt
LVtpZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWxheW91dCB2OmV4dD0iZWRpdCI+DQo8bzpp
ZG1hcCB2OmV4dD0iZWRpdCIgZGF0YT0iMSIgLz4NCjwvbzpzaGFwZWxheW91dD48L3htbD48IVtl
bmRpZl0tLT4NCjwvaGVhZD4NCjxib2R5IGxhbmc9IkVOLVVTIiBsaW5rPSJibHVlIiB2bGluaz0i
cHVycGxlIj4NCjxkaXYgY2xhc3M9IldvcmRTZWN0aW9uMSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJp
JnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+SGkgTHVjaWVuLDxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEu
MHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0
OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48
c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1
b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+TGV0IG1lIGV4cGxhaW4gbXkgdmlldy4gSSBz
YWlkIEkgYW0gY29tbWl0dGVkIGZvciBlZGl0aW5nIGNoYW5nZXMsIHB1Ymxpc2hpbmcgZXRjLiBB
cyBhbiBhdXRob3IgSSBleHByZXNzZWQgbXkgY29tbWl0bWVudCB0byB0aGUgY29tbXVuaXR5LiZu
YnNwOyBJIHRoaW5rIEkgaGF2ZSB0aGUNCiBsaWJlcnR5IHRvICZuYnNwO3NoYXJlIG15IHZpZXcg
Y2FuIEkgPy4gJm5ic3A7S2luZGx5IGNvcnJlY3QgbWUgaWYgSSBhbSB3cm9uZyBpcyB0aGF0IGFu
IG9mZmVuY2U/LjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz
cGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVv
dDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+
DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250
LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+QXMg
eW91ciBzZWNvbmQgcXVlc3Rpb24gS2luZGx5IHNlZSB0aGUgYW5zd2VyIGlubGluZS48bzpwPjwv
bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29s
b3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2Fs
aWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0QiPlJlZ2FyZHMsPG86cD48L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox
MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMx
RjQ5N0QiPlN1ZGhpbjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkm
cXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48
L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBw
dDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPkZyb206PC9zcGFu
PjwvYj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxp
YnJpJnF1b3Q7LHNhbnMtc2VyaWYiPiBMdWNpZW4gQXZyYW1vdiBbbWFpbHRvOmx1Y2llbmF2QGdv
b2dsZS5jb21dDQo8YnI+DQo8Yj5TZW50OjwvYj4gV2VkbmVzZGF5LCBKdW5lIDIxLCAyMDE3IDEw
OjE4IEFNPGJyPg0KPGI+VG86PC9iPiBTdWRoaW4gSmFjb2IgJmx0O3NqYWNvYkBqdW5pcGVyLm5l
dCZndDs8YnI+DQo8Yj5DYzo8L2I+IGJtd2dAaWV0Zi5vcmc7IE1PUlRPTiwgQUxGUkVEIEMgKEFM
KSAmbHQ7YWNtb3J0b25AYXR0LmNvbSZndDs8YnI+DQo8Yj5TdWJqZWN0OjwvYj4gUmU6IFtibXdn
XSBibXdnIERpZ2VzdCwgVm9sIDE1MywgSXNzdWUgMjU8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8
cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj5PbmUgd291bGQgaG9wZSB0aGF0IGFuIGF1dGhvciBpcyBjb21taXR0ZWQg
dG8gcmV2aWV3IHdoYXQgdGhleSB3cml0ZSA6LSk8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPkkgYW0gcGVyc29uYWxseSBvbiB0aGUgZmVuY2UgZm9yIHRoaXMg
ZHJhZnQgYXQgdGhpcyBwb2ludC48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNs
YXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCI+V2h5IHdvdWxkIEVWUE4gbmVlZCBhIHNwZWNpZmljIGJlbmNobWFy
ayB0byBzdGFydCB3aXRoLCB0aGF0IGlzIGRpZmZlcmVudCBmcm9tIHN0YW5kYXJkIGJlbmNobWFy
a2luZyB0b29scyBhbmQgdGVjaG5pcXVlcz88bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0Nh
bGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwv
c3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEx
LjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFG
NDk3RCI+U3VkaGluJmd0OyZndDsmZ3Q7Jmd0OyBFVlBOIGFuZCBQQkItRVZQTiAmbmJzcDtpcyZu
YnNwOyByZWxhdGl2ZWx5IG5ldyBSRkMgd2hpY2ggY2FtZSBhIGNvdXBsZSBvZiB5ZWFycyBiYWNr
IGFuZCBpdCBpcyB3aWRlbHkgZGVwbG95ZWQgYW5kIGR1ZSB0byBnYXAgaW4gdGhlIGN1cnJlbnQg
dGVjaG5pcXVlIGl0IHdhcw0KIG5lZWRlZCB0byBiZW5jaG1hcmsgdGhlc2UgZm9yIHRoZSBiZW5l
Zml0IG9mIGVuZCBjdXN0b21lcnMuPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z
b05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7
Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0QiPk1heSBJIHJlcXVlc3QgeW91
IHRvIGdvIHRocm91Z2ggdGhlIGRyYWZ0IGFuZCBwbGVhc2UgZmVlbCBmcmVlIHRvIHJlYWNoIG91
dCB0byBtZSBmb3IgYW55IHF1ZXJpZXMsIEkgbG92ZSB0byBleHBsYWluIGl0LjxvOnA+PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjoj
MUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJp
JnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+UmVnYXJkcyw8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBw
dDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3
RCI+U3VkaGluPG86cD48L286cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCI+T24gVHVlLCBKdW4gMjAsIDIwMTcgYXQgODoxNyBQTSwgU3VkaGlu
IEphY29iICZsdDs8YSBocmVmPSJtYWlsdG86c2phY29iQGp1bmlwZXIubmV0IiB0YXJnZXQ9Il9i
bGFuayI+c2phY29iQGp1bmlwZXIubmV0PC9hPiZndDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8
YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAx
LjBwdDtwYWRkaW5nOjBpbiAwaW4gMGluIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1y
aWdodDowaW4iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+SGkgQWwsPGJyPg0KPGJyPg0KQXMgYW4g
YXV0aG9yIG9mIHRoZSBkcmFmdC4gTGV0IG1lIGFuc3dlciB0aGUgcXVlc3Rpb24uPGJyPg0KPGJy
Pg0Kd2hldGhlciB5b3UgY29tbWl0IHRvIHJldmlld2luZyB0aGUgV0cgZG9jdW1lbnQgaWYgYWRv
cHRlZD88YnI+DQo8YnI+DQpZZXMuIEkgYW0gY29tbWl0dGVkIHRvIHJldmlldyB0aGUgZHJhZnQs
IGlmIGFkb3B0ZWQuPGJyPg0KPGJyPg0KUmVnYXJkcyw8YnI+DQpTdWRoaW48YnI+DQo8YnI+DQo8
YnI+DQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLTxicj4NCkZyb206IGJtd2cgW21haWx0bzo8
YSBocmVmPSJtYWlsdG86Ym13Zy1ib3VuY2VzQGlldGYub3JnIj5ibXdnLWJvdW5jZXNAaWV0Zi5v
cmc8L2E+XSBPbiBCZWhhbGYgT2YNCjxhIGhyZWY9Im1haWx0bzpibXdnLXJlcXVlc3RAaWV0Zi5v
cmciPmJtd2ctcmVxdWVzdEBpZXRmLm9yZzwvYT48YnI+DQpTZW50OiBXZWRuZXNkYXksIEp1bmUg
MjEsIDIwMTcgMTI6MzAgQU08YnI+DQpUbzogPGEgaHJlZj0ibWFpbHRvOmJtd2dAaWV0Zi5vcmci
PmJtd2dAaWV0Zi5vcmc8L2E+PGJyPg0KU3ViamVjdDogYm13ZyBEaWdlc3QsIFZvbCAxNTMsIElz
c3VlIDI1PGJyPg0KPGJyPg0KU2VuZCBibXdnIG1haWxpbmcgbGlzdCBzdWJtaXNzaW9ucyB0bzxi
cj4NCiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyA8YSBocmVmPSJtYWlsdG86Ym13Z0BpZXRm
Lm9yZyI+Ym13Z0BpZXRmLm9yZzwvYT48YnI+DQo8YnI+DQpUbyBzdWJzY3JpYmUgb3IgdW5zdWJz
Y3JpYmUgdmlhIHRoZSBXb3JsZCBXaWRlIFdlYiwgdmlzaXQ8YnI+DQombmJzcDsgJm5ic3A7ICZu
YnNwOyAmbmJzcDsgPGEgaHJlZj0iaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5m
by9ibXdnIiB0YXJnZXQ9Il9ibGFuayI+aHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0
aW5mby9ibXdnPC9hPjxicj4NCm9yLCB2aWEgZW1haWwsIHNlbmQgYSBtZXNzYWdlIHdpdGggc3Vi
amVjdCBvciBib2R5ICdoZWxwJyB0bzxicj4NCiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyA8
YSBocmVmPSJtYWlsdG86Ym13Zy1yZXF1ZXN0QGlldGYub3JnIj5ibXdnLXJlcXVlc3RAaWV0Zi5v
cmc8L2E+PGJyPg0KPGJyPg0KWW91IGNhbiByZWFjaCB0aGUgcGVyc29uIG1hbmFnaW5nIHRoZSBs
aXN0IGF0PGJyPg0KJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDxhIGhyZWY9Im1haWx0bzpi
bXdnLW93bmVyQGlldGYub3JnIj5ibXdnLW93bmVyQGlldGYub3JnPC9hPjxicj4NCjxicj4NCldo
ZW4gcmVwbHlpbmcsIHBsZWFzZSBlZGl0IHlvdXIgU3ViamVjdCBsaW5lIHNvIGl0IGlzIG1vcmUg
c3BlY2lmaWMgdGhhbiAmcXVvdDtSZTogQ29udGVudHMgb2YgYm13ZyBkaWdlc3QuLi4mcXVvdDs8
YnI+DQo8YnI+DQo8YnI+DQpUb2RheSdzIFRvcGljczo8YnI+DQo8YnI+DQombmJzcDsgJm5ic3A7
MS4gUmU6IENhbGwgRm9yIEFkb3B0aW9uOiBkcmFmdC1raXNoamFjLWJtd2ctZXZwbnRlc3QtMDc8
YnI+DQombmJzcDsgJm5ic3A7ICZuYnNwOyAoTU9SVE9OLCBBTEZSRUQgQyAoQUwpKTxicj4NCjxi
cj4NCjxicj4NCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+DQo8YnI+DQpNZXNzYWdlOiAxPGJyPg0KRGF0ZTog
VHVlLCAyMCBKdW4gMjAxNyAxODo0MzoxOCAmIzQzOzAwMDA8YnI+DQpGcm9tOiAmcXVvdDtNT1JU
T04sIEFMRlJFRCBDIChBTCkmcXVvdDsgJmx0OzxhIGhyZWY9Im1haWx0bzphY21vcnRvbkBhdHQu
Y29tIj5hY21vcnRvbkBhdHQuY29tPC9hPiZndDs8YnI+DQpUbzogJnF1b3Q7PGEgaHJlZj0ibWFp
bHRvOmJtd2dAaWV0Zi5vcmciPmJtd2dAaWV0Zi5vcmc8L2E+JnF1b3Q7ICZsdDs8YSBocmVmPSJt
YWlsdG86Ym13Z0BpZXRmLm9yZyI+Ym13Z0BpZXRmLm9yZzwvYT4mZ3Q7PGJyPg0KU3ViamVjdDog
UmU6IFtibXdnXSBDYWxsIEZvciBBZG9wdGlvbjogZHJhZnQta2lzaGphYy1ibXdnLWV2cG50ZXN0
LTA3PGJyPg0KTWVzc2FnZS1JRDo8YnI+DQombmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJmx0
OzxhIGhyZWY9Im1haWx0bzo0RDdGNEFEMzEzRDNGQzQzQTA1M0IzMDlGOTc1NDNDRjI1RkUxMTZB
QG5qbXRleGc1LnJlc2VhcmNoLmF0dC5jb20iPjREN0Y0QUQzMTNEM0ZDNDNBMDUzQjMwOUY5NzU0
M0NGMjVGRTExNkFAbmptdGV4ZzUucmVzZWFyY2guYXR0LmNvbTwvYT4mZ3Q7PGJyPg0KQ29udGVu
dC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PSZxdW90O3VzLWFzY2lpJnF1b3Q7PGJyPg0KPGJy
Pg0KQk1XRyw8YnI+DQo8YnI+DQpTbyBmYXIsIEkgY291bnQgYWJvdXQgZWlnaHQgcGVvcGxlIHdo
byBvZmZlcmVkIHN1cHBvcnQgZm9yIHRoZSBkcmFmdCwgYW5kIG9uZSBwZXJzb24gd2hvIG1hZGUg
YSB0ZWNobmljYWwgY29tbWVudCBhbG9uZyB3aXRoIHRoZWlyIG1lc3NhZ2UuPGJyPg0KPGJyPg0K
Kk5vIG9uZSogaGFzIGFuc3dlcmVkIHRoaXMgcXVlc3Rpb246PGJyPg0KJmd0Ozxicj4NCiZndDsg
KDMpIHdoZXRoZXIgeW91IGNvbW1pdCB0byByZXZpZXdpbmcgdGhlIFdHIGRvY3VtZW50IGlmIGFk
b3B0ZWQ8YnI+DQomZ3Q7PGJyPg0KPGJyPg0KV2l0aCBubyBjb21taXRtZW50cyB0byByZXZpZXcg
dGhlIGRyYWZ0IGFuZCBwcm92aWRlIHRoZSBuZWNlc3NhcnkgZXhwZXJ0aXNlIHRvIGRldmVsb3Ag
YXBwcm9wcmlhdGUgYmVuY2htYXJrcyBmb3Igb3VyIGluZHVzdHJ5LCBpdCB3aWxsIG5vdCBiZSBh
cHByb3ByaWF0ZSB0byBhZGQgYSBtaWxlc3RvbmUgdG8gb3VyIGNoYXJ0ZXIgKG9yIHJlLWNoYXJ0
ZXIsIGlmIG5lY2Vzc2FyeSkuPGJyPg0KPGJyPg0KVGhlcmUgaXMgc3RpbGwgdGltZSBmb3IgZXZl
cnlvbmUgdG8gcmVjb25zaWRlciB0aGVpciBwb3NpdGlvbiwgYW5kIHByb3ZpZGUgYW4gYW5zd2Vy
IHRvIHRoZSBxdWVzdGlvbiBhYm92ZS4mbmJzcDsgVGhlIFdHTEMgY2xvc2VzIG9uIEZyaWRheSAy
MyBKdW5lIDIwMTcuPGJyPg0KPGJyPg0KdGhhbmtzIGZvciB5b3VyIGNvbnNpZGVyYXRpb24sPGJy
Pg0KQWw8YnI+DQpibXdnIGNvLWNoYWlyPGJyPg0KPGJyPg0KPGJyPg0KJmd0OyAtLS0tLU9yaWdp
bmFsIE1lc3NhZ2UtLS0tLTxicj4NCiZndDsgRnJvbTogYm13ZyBbbWFpbHRvOjxhIGhyZWY9Im1h
aWx0bzpibXdnLWJvdW5jZXNAaWV0Zi5vcmciPmJtd2ctYm91bmNlc0BpZXRmLm9yZzwvYT5dIE9u
IEJlaGFsZiBPZiBNT1JUT04sIEFMRlJFRDxicj4NCiZndDsgQzxicj4NCiZndDsgKEFMKTxicj4N
CiZndDsgU2VudDogU2F0dXJkYXksIEp1bmUgMDMsIDIwMTcgNjozNCBQTTxicj4NCiZndDsgVG86
IDxhIGhyZWY9Im1haWx0bzpibXdnQGlldGYub3JnIj5ibXdnQGlldGYub3JnPC9hPjxicj4NCiZn
dDsgU3ViamVjdDogW2Jtd2ddIENhbGwgRm9yIEFkb3B0aW9uOiBkcmFmdC1raXNoamFjLWJtd2ct
ZXZwbnRlc3QtMDc8YnI+DQomZ3Q7PGJyPg0KJmd0OyAqKiogU2VjdXJpdHkgQWR2aXNvcnk6IFRo
aXMgTWVzc2FnZSBPcmlnaW5hdGVkIE91dHNpZGUgb2YgQVQmYW1wO1QgKioqLjxicj4NCiZndDsg
UmVmZXJlbmNlIDxhIGhyZWY9Imh0dHA6Ly9jc28uYXR0LmNvbS9FbWFpbFNlY3VyaXR5L0lEU1Au
aHRtbCIgdGFyZ2V0PSJfYmxhbmsiPg0KaHR0cDovL2Nzby5hdHQuY29tL0VtYWlsU2VjdXJpdHkv
SURTUC5odG1sPC9hPiBmb3IgbW9yZTxicj4NCiZndDsgaW5mb3JtYXRpb24uPGJyPg0KJmd0Ozxi
cj4NCiZndDsgQk1XRyw8YnI+DQomZ3Q7PGJyPg0KJmd0OyBBdCBvdXIgQ2hpY2FnbyBtZWV0aW5n
LCB0aGUgQk1XRyByZXZpZXdlZCBhIHZlcnNpb24gb2YgdGhlIEludGVybmV0PGJyPg0KJmd0OyBE
cmFmdCBvbjxicj4NCiZndDs8YnI+DQomZ3Q7IEJlbmNobWFya2luZyBNZXRob2RvbG9neSBmb3Ig
RVZQTiBhbmQgUEJCLUVWUE48YnI+DQomZ3Q7IGRyYWZ0LWtpc2hqYWMtYm13Zy1ldnBudGVzdC0w
NTxicj4NCiZndDs8YnI+DQomZ3Q7IFRoZXJlIHdhcyBzdWZmaWNpZW50IHJlYWRlcnNoaXAgcHJl
c2VudCBhdCB0aGUgbWVldGluZyBhbmQ8YnI+DQomZ3Q7IHBvc2l0aXZlL2NvbnN0cnVjdGl2ZSBj
b21tZW50cyBvbiB0aGUgd29yaywgc28gdGhlIGNoYWlycyBhc2tlZDxicj4NCiZndDsgd2hldGhl
ciBzdXBwb3J0IGV4aXN0ZWQgdG8gYWRvcHQgdGhlIGRyYWZ0IGFzIGEgV0cgaXRlbS4gVGhlcmUg
d2FzPGJyPg0KJmd0OyBzdXBwb3J0IGFuZCBubyBvYmplY3Rpb25zLiBBcyBhbHdheXMsIHN1Y2gg
ZGVjaXNpb25zIGFyZSBvZmZpY2lhbGx5PGJyPg0KJmd0OyB0YWtlbiBvbiB0aGUgbWFpbGluZyBs
aXN0Ljxicj4NCiZndDs8YnI+DQomZ3Q7IFRoaXMgbWVzc2FnZSosIHRoZXJlZm9yZSwgc3RhcnRz
IGEgY2FsbCBmb3IgYWRvcHRpb24gb246PGJyPg0KJmd0Ozxicj4NCiZndDsgQmVuY2htYXJraW5n
IE1ldGhvZG9sb2d5IGZvciBFVlBOIGFuZCBQQkItRVZQTjxicj4NCiZndDsgZHJhZnQta2lzaGph
Yy1ibXdnLWV2cG50ZXN0LTA3PGJyPg0KJmd0Ozxicj4NCiZndDsgdG8gcnVuIHVudGlsIEZyaWRh
eSAyMyBKdW5lIDIwMTcuPGJyPg0KJmd0Ozxicj4NCiZndDsgUGxlYXNlIHJlcGx5IHRvIDxhIGhy
ZWY9Im1haWx0bzpibXdnQGlldGYub3JnIj5ibXdnQGlldGYub3JnPC9hPiBpbmRpY2F0aW5nOjxi
cj4NCiZndDs8YnI+DQomZ3Q7ICgxKSB3aGV0aGVyIHlvdSBzdXBwb3J0IGFkZGl0aW9uIG9mIHRo
ZSBmb2xsb3dpbmcgbWlsZXN0b25lIHRvPGJyPg0KJmd0OyZuYnNwOyAmbmJzcDsgdGhlIEJNV0cg
Y2hhcnRlcjo8YnI+DQomZ3Q7PGJyPg0KJmd0OyBkYXRlIFRCRDogQmVuY2htYXJraW5nIE1ldGhv
ZG9sb2d5IGZvciBFVlBOIGFuZCBQQkItRVZQTiB0byB0aGUgSUVTRzxicj4NCiZndDs8YnI+DQom
Z3Q7ICgyKSB3aGV0aGVyIHlvdSBzdXBwb3J0IHRoZSBhZG9wdGlvbiBvZiBkcmFmdC1raXNoamFj
LWJtd2ctZXZwbnRlc3QtMDc8YnI+DQomZ3Q7Jm5ic3A7ICZuYnNwOyBhcyB0aGUgYmFzaXMgZG9j
dW1lbnQgZm9yIHRoaXMgbWlsZXN0b25lPGJyPg0KJmd0Ozxicj4NCiZndDsgKDMpIHdoZXRoZXIg
eW91IGNvbW1pdCB0byByZXZpZXdpbmcgdGhlIFdHIGRvY3VtZW50IGlmIGFkb3B0ZWQ8YnI+DQom
Z3Q7PGJyPg0KJmd0OyBXZSBhcmUgYXdhcmUgdGhhdCBCTVdHIGhhcyBuZWFybHkgY29tcGxldGVk
IGFsbCBpdHMgZXhpc3RpbmcgY2hhcnRlcmVkPGJyPg0KJmd0OyBpdGVtcy4mbmJzcDsgVGhpcyB1
c3VhbGx5IGJlZ2lucyBkaXNjdXNzaW9uIG9mIHJlLWNoYXJ0ZXJpbmcsIHRoaXMgdGltZTxicj4N
CiZndDsgd2l0aCBXYXJyZW4gKG91ciBuZXcgQUQgQWR2aXNvcikuIFRoZSBmaW5hbCBkZWNpc2lv
bi9zY2hlZHVsZSBmb3I8YnI+DQomZ3Q7IGFkb3B0aW9uIHdpbGwgZGVwZW5kIG9uIHRoZXNlIGRp
c2N1c3Npb25zLjxicj4NCiZndDs8YnI+DQomZ3Q7IHRoYW5rcyBmb3IgeW91ciBjb25zaWRlcmF0
aW9uIG9mIHRoZSBkcmFmdCwgYW5kIHJlZ2FyZHMsPGJyPg0KJmd0Ozxicj4NCiZndDsgQWw8YnI+
DQomZ3Q7IGJtd2cgY28tY2hhaXI8YnI+DQomZ3Q7PGJyPg0KJmd0OyAqdXNpbmcgQnJpYW4gVHJh
bW1lbGwncyByZWNlbnQgbWVzc2FnZSB0byBJUFBNIGFzIGEgdGVtcGxhdGUgKHRoYW5rcyEpPGJy
Pg0KJmd0Ozxicj4NCiZndDsgX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX188YnI+DQomZ3Q7IGJtd2cgbWFpbGluZyBsaXN0PGJyPg0KJmd0OyA8YSBocmVmPSJt
YWlsdG86Ym13Z0BpZXRmLm9yZyI+Ym13Z0BpZXRmLm9yZzwvYT48YnI+DQomZ3Q7IDxhIGhyZWY9
Imh0dHBzOi8vdXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0iIHRhcmdl
dD0iX2JsYW5rIj5odHRwczovL3VybGRlZmVuc2UucHJvb2Zwb2ludC5jb20vdjIvdXJsP3U9aHR0
cHMtPC9hPjxicj4NCiZndDsgM0FfX3d3dy5pZXRmLm9yZ19tYWlsbWFuX2xpc3RpbmZvX2Jtd2cm
YW1wO2Q9RHdJQ0FnJmFtcDtjPUxGWVotPGJyPg0KJmd0OyBvOV9IVU1lTVRTUWljdmpJZyZhbXA7
cj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3JmFtcDttPTN0WEtRTkJSOFE0SEM3QkJ4NnBjNW1lOHU8
YnI+DQomZ3Q7IDl6IE5lOVFWb19vd09GNS1jaU0mYW1wO3M9eElLZ0RQMnpJOUYwSXZmaXNHbEVC
aklpajNJX0g5NXprVldXSkdQOHpsZyZhbXA7ZT08YnI+DQo8YnI+DQo8YnI+DQo8YnI+DQotLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+DQo8YnI+DQpTdWJqZWN0OiBEaWdlc3QgRm9v
dGVyPGJyPg0KPGJyPg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX188YnI+DQpibXdnIG1haWxpbmcgbGlzdDxicj4NCjxhIGhyZWY9Im1haWx0bzpibXdnQGll
dGYub3JnIj5ibXdnQGlldGYub3JnPC9hPjxicj4NCjxhIGhyZWY9Imh0dHBzOi8vd3d3LmlldGYu
b3JnL21haWxtYW4vbGlzdGluZm8vYm13ZyIgdGFyZ2V0PSJfYmxhbmsiPmh0dHBzOi8vd3d3Lmll
dGYub3JnL21haWxtYW4vbGlzdGluZm8vYm13ZzwvYT48YnI+DQo8YnI+DQo8YnI+DQotLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+DQo8YnI+DQpFbmQgb2YgYm13ZyBEaWdlc3QsIFZv
bCAxNTMsIElzc3VlIDI1PGJyPg0KKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq
Kjxicj4NCjxicj4NCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fPGJyPg0KYm13ZyBtYWlsaW5nIGxpc3Q8YnI+DQo8YSBocmVmPSJtYWlsdG86Ym13Z0BpZXRm
Lm9yZyI+Ym13Z0BpZXRmLm9yZzwvYT48YnI+DQo8YSBocmVmPSJodHRwczovL3d3dy5pZXRmLm9y
Zy9tYWlsbWFuL2xpc3RpbmZvL2Jtd2ciIHRhcmdldD0iX2JsYW5rIj5odHRwczovL3d3dy5pZXRm
Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Jtd2c8L2E+PG86cD48L286cD48L3A+DQo8L2Jsb2NrcXVv
dGU+DQo8L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0K
PC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg==

--_000_BY1PR0501MB13525E3A520D0117F36FAEF9C2DA0BY1PR0501MB1352_--


From nobody Wed Jun 21 01:26:46 2017
Return-Path: <giuseppe.fioccola@telecomitalia.it>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E97D1318E9 for <bmwg@ietfa.amsl.com>; Wed, 21 Jun 2017 01:26:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level: 
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fx0czucLUaZS for <bmwg@ietfa.amsl.com>; Wed, 21 Jun 2017 01:26:39 -0700 (PDT)
Received: from mx01.telecomitalia.it (mx01.telecomitalia.it [217.169.121.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AE35E1318AF for <bmwg@ietf.org>; Wed, 21 Jun 2017 01:25:58 -0700 (PDT)
X-AuditID: d9a9790a-8a3ff70000003804-2a-594a2d942334
Received: from TELMBXA02RM001.telecomitalia.local ( [10.14.252.26]) (using TLS with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client did not present a certificate) by mx01.telecomitalia.it () with SMTP id 86.02.14340.49D2A495; Wed, 21 Jun 2017 10:25:56 +0200 (CEST)
From: Fioccola Giuseppe <giuseppe.fioccola@telecomitalia.it>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLct41m0dFmKKXrQM2LQ9IfZznSGQNPDsMQABzpekA=
Date: Wed, 21 Jun 2017 08:25:56 +0000
Message-ID: <48564d4fe6654af28a3535d00fec0a61@TELMBXB02RM001.telecomitalia.local>
References: <4D7F4AD313D3FC43A053B309F97543CF25FD3680@njmtexg5.research.att.com> <4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.att.com>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.14.252.236]
x-ti-disclaimer: Disclaimer1
Content-Type: text/plain; charset="iso-8859-1"
content-transfer-encoding: quoted-printable
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrGKsWRmVeSWpSXmKPExsXCxfdHSneKrlekwcK7GhZbj01ktOj/epPN gcnjZf8cRo8lS34yBTBFNTDaJObl5ZcklqQqpKQWJ9squWQWJ+ckZuamFimEpOakJufnKilk ptgqGSspFOQkJqfmpuaV2ColFhSk5qUo2XEpYAAboLLMPIXUvOT8lMy8dFslz2B/XQsLU0td QyW7wNLU4pJ8hdzU4uLE9PTMfIXUhPWCGWfXnmIp2CNdMXflKsYGxrtiXYycHBICJhLL1m1l 7GLk4hASmMokMXXJKRaQBJuAjcTBVyfYQGwRgUCJye8+M4HYwgK2Es8bG5gg4vYSu65sh7Kt JI5OaAezWQRUJWaf+MAMYvMC9S5/f5YZYsF0RoklX/vAFnACJf5euAxmMwrISkzYvYgRxGYW EJd4Mf0EO8R1AhJL9pxnhrBFJV4+/scKYRtIbF26jwXCVpTY/vU4VFxGYuGRyawQc/Qkbkyd wgZha0ssW/ga6iBBiZMzn7BMYBSdhWTdLCQts5C0zELSsoCRZRWjaG6FgaFeCSTSMksSczIT 9TJLNjECE8PNlZVcOxhfr3I+xCjAwajEw7tV3CtSiDWxrLgy9xCjBAezkghvsAZQiDclsbIq tSg/vqg0J7X4EKMPMMgmMkuJJucDk1ZeSbyhiYWlobGFhZGhhZkpDmElcd7pakCzBNKBqSo7 NbUgtQhmHBMHp1QD44Hn5S//RycU5p9XnHvy+ZyS7f+4DX5G+rt9M+Mz5ktW4j2WUVumx/3K kYVx97oLt3RE+GIdoyKmOdX72B0/tuenbobaVfNH+VbP5r454srz/4LExsdnO8sUmfRn8M/I 65v2he9pVeOrL5o9VXx7D6p+titXnDlP3P1Aw4nnu/oubPh+6cS3ciWW4oxEQy3mouJEAN62 qwo5AwAA
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/25EIPqAfEV1Ee2TY3_mrzWJ4LEs>
Subject: [bmwg] R: Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 08:26:41 -0000

I will also review the WG document if adopted.
Thanks,

Giuseppe

-----Messaggio originale-----
Da: bmwg [mailto:bmwg-bounces@ietf.org] Per conto di MORTON, ALFRED C (AL)
Inviato: marted=EC 20 giugno 2017 20:43
A: bmwg@ietf.org
Oggetto: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07

BMWG,

So far, I count about eight people who offered support for the draft, and on=
e person who made a technical comment along with their message.

*No one* has answered this question:
> 
> (3) whether you commit to reviewing the WG document if adopted
>

With no commitments to review the draft and provide the necessary expertise=
 to develop appropriate benchmarks for our industry, it will not be appropri=
ate to add a milestone to our charter (or re-charter, if necessary).

There is still time for everyone to reconsider their position, and provide a=
n answer to the question above.  The WGLC closes on Friday 23 June 2017.

thanks for your consideration,
Al
bmwg co-chair


> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED 
> C
> (AL)
> Sent: Saturday, June 03, 2017 6:34 PM
> To: bmwg@ietf.org
> Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
> 
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more 
> information.
> 
> BMWG,
> 
> At our Chicago meeting, the BMWG reviewed a version of the Internet 
> Draft on
> 
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
> 
> There was sufficient readership present at the meeting and 
> positive/constructive comments on the work, so the chairs asked 
> whether support existed to adopt the draft as a WG item. There was 
> support and no objections. As always, such decisions are officially 
> taken on the mailing list.
> 
> This message*, therefore, starts a call for adoption on:
> 
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
> 
> to run until Friday 23 June 2017.
> 
> Please reply to bmwg@ietf.org indicating:
> 
> (1) whether you support addition of the following milestone to
>    the BMWG charter:
> 
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
> 
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>    as the basis document for this milestone
> 
> (3) whether you commit to reviewing the WG document if adopted
> 
> We are aware that BMWG has nearly completed all its existing chartered 
> items.  This usually begins discussion of re-chartering, this time 
> with Warren (our new AD Advisor). The final decision/schedule for 
> adoption will depend on these discussions.
> 
> thanks for your consideration of the draft, and regards,
> 
> Al
> bmwg co-chair
> 
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
> 
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D3tXKQNBR8Q4HC7BBx6pc5me8u
> 9z Ne9QVo_owOF5-ciM&s=3DxIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=3D

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle pers=
one indicate. La diffusione, copia o qualsiasi altra azione derivante dalla=
 conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia=
te ricevuto questo documento per errore siete cortesemente pregati di darne=
 immediata comunicazione al mittente e di provvedere alla sua distruzione, G=
razie. 

This e-mail and any attachments is confidential and may contain privileged i=
nformation intended for the addressee(s) only. Dissemination, copying, print=
ing or use by anybody else is unauthorised. If you are not the intended reci=
pient, please delete this message and any attachments and advise the sender=
 by return e-mail, Thanks. 

Rispetta l'ambiente. Non stampare questa mail se non =E8 necessario.


From nobody Wed Jun 21 06:08:57 2017
Return-Path: <spencerdawkins.ietf@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 726C7129AD5; Wed, 21 Jun 2017 06:08:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level: 
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 37CIvgc5B-nL; Wed, 21 Jun 2017 06:08:53 -0700 (PDT)
Received: from mail-yb0-x236.google.com (mail-yb0-x236.google.com [IPv6:2607:f8b0:4002:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 32AF6129ACC; Wed, 21 Jun 2017 06:08:53 -0700 (PDT)
Received: by mail-yb0-x236.google.com with SMTP id e201so42341968ybb.1; Wed, 21 Jun 2017 06:08:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=f7CzcVIlFelhsZ0WsnI9KZXlvQndCHZ/dOS7LZOKj40=; b=iQqBZ9ny50w/5bpsdLS2WZjDyiVJfX7WdNjeD6JvgBQzIXLm/S8SGg08B4EVXjx4h0 EvKl/dqyZx4mgVnbfiRJqeJPFocDJU2TTGtSZSlrgnvVk/YYvHi2/f2tJJii/WhC7iLK k9UvQT3ps/gKDIZ1wKVf5wJDWyau+o2Xbw+CA39mKYCtAvDPM3Knphmwu6W1I9bJlEYN aT8H8geq/ouG14DbfnWkvuzNjEUa78qYeDa+uiHCpG4MvLPDAhuyIZE7uMg87ffr1YlN 51dqQj8t/aYtSpyv61PADLy/e9gZhTGeLBtj9sHWi8IXjJjUS5ILZnI3Lg2Nnh9kBfet wx3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=f7CzcVIlFelhsZ0WsnI9KZXlvQndCHZ/dOS7LZOKj40=; b=mbpoiwoBUraNK0gtRABjB71fPXIKnQVSgBAUqRTZHFv0ClQDCVaYiuRICkHA/GHUyO VrsvR+vfTwMSh6KfXjQvT0DCoLMUwTT/JxmuOFZ/EuTP6yeHXTaIwG39mkm7gumwp4kt fuDqikELv2SqnlSICYrEUzssbVRi8oy+IHTMAuYRE693jL7+ZuTe2kr5yigpTzomB/8G H/8iSeqJ1QNQF8ihI2M3J3uNUZq+YpZ0qMUGn2l9ZDXg4QtlH2C/e6WjkZEIw/FI+dMw x4aiGuvM3gNJyj9oSkr5u3sAHbp52XYfm/ffhfXceWXGuDee8QTPpYzLG6ZA2q4bNFyU H5ow==
X-Gm-Message-State: AKS2vOxeaNv7MqWNcIJORWJ7ZeIpbFduMt9TtEgXu03gofTrhOPT3aWy 8BRdiY0B/56bscsKKusYvwLsjUQS0w==
X-Received: by 10.37.230.197 with SMTP id d188mr22840667ybh.37.1498050532350;  Wed, 21 Jun 2017 06:08:52 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.37.104.144 with HTTP; Wed, 21 Jun 2017 06:08:51 -0700 (PDT)
Received: by 10.37.104.144 with HTTP; Wed, 21 Jun 2017 06:08:51 -0700 (PDT)
In-Reply-To: <CAArZqeUwPmaeMRBv8EdrK9goEDrpSZNPtUtreK1Zz3UnpkgqYg@mail.gmail.com>
References: <149799053504.24974.8498552938277597302.idtracker@ietfa.amsl.com> <8BA949EB-B14A-49BA-992F-34180343FB92@gmail.com> <CAArZqeUwPmaeMRBv8EdrK9goEDrpSZNPtUtreK1Zz3UnpkgqYg@mail.gmail.com>
From: Spencer Dawkins at IETF <spencerdawkins.ietf@gmail.com>
Date: Wed, 21 Jun 2017 08:08:51 -0500
Message-ID: <CAKKJt-eZhSoeaWaf=scEOQPn2C3nmkLrct8uXkQqmckCcmJ+YA@mail.gmail.com>
To: Lucien <lucien.avramov@gmail.com>
Cc: iesg@ietf.org, Warren Kumari <warren@kumari.net>, Sarah Banks <sbanks@encrypted.net>,  sob@sobco.com, bmwg-chairs@ietf.org, bmwg@ietf.org,  draft-ietf-bmwg-dcbench-methodology@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c0afb0ad625a30552781151"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/UGUnIsNwFPGfv6t4nMt1X6r-u80>
Subject: Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-dcbench-methodology-13: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 13:08:55 -0000

--94eb2c0afb0ad625a30552781151
Content-Type: text/plain; charset="UTF-8"

Hi, Lucien,

On Jun 20, 2017 21:57, "Lucien" <lucien.avramov@gmail.com> wrote:

Hi Spencer, Al (+Scott, +Warren), and all,

Thank you for the suggestion. Both drafts have been updated with explaining
what a data center network device unter test is for these tests.

Besides adding more clarity on both abstracts, and introduction, I added
the following text in the introduction section of each draft to specify
what a data center Network DUT is for these drafts.


That's great, and thanks!

Spencer


I hope Scott will appreciate the low buffer comment, as the high
performance data center switches today have significantly less buffer than
other networking devices.

Currently, typical data Center networking devices are characterized
by:

-High port density (48 ports of more)

-High speed (up to 100 Gb/s currently per port)

-High throughput (line rate on all ports for Layer 2 and/or Layer 3)

-Low latency (in the microsecond or nanosecond range)

-Low amount of buffer (in the Mb range)

-Layer 2 and Layer 3 forwarding capability (Layer 3 not mandatory)

On Tue, Jun 20, 2017 at 1:42 PM, Lucien Avramov <lucien.avramov@gmail.com>
wrote:

> Definitely agreed, will place a statement in both documents today to
> explain what datacenter switches are at time of writing.
>
> Thank you,
> Lucien
>
> > On Jun 20, 2017, at 13:28, Spencer Dawkins <
> spencerdawkins.ietf@gmail.com> wrote:
> >
> > Spencer Dawkins has entered the following ballot position for
> > draft-ietf-bmwg-dcbench-methodology-13: No Objection
> >
> > When responding, please keep the subject line intact and reply to all
> > email addresses included in the To and CC lines. (Feel free to cut this
> > introductory paragraph, however.)
> >
> >
> > Please refer to https://www.ietf.org/iesg/stat
> ement/discuss-criteria.html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
> >
> >
> >
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> >
> > (I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and
> > draft-ietf-bmwg-dcbench-methodology)
> >
> > I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology
> and
> > draft-ietf-bmwg-dcbench-methodology that assert these documents aren't
> specific
> > to "data centers". That wouldn't surprise me, but I'm not seeing a
> definition
> > of "data center" in either document - did I miss it?
> >
> > I suspect that the authors have specific technical characteristics in
> mind,
> > that happen to map onto what data centers look like today, but may not
> in the
> > future ("RFCs last forever"). Is it possible to tease those
> characteristics out?
> >
> > (Full disclosure: my first working group in the IETF could have been
> called
> > "TCP over cellular links", but it turned out that when we said "cellular
> > links", we meant "low-speed links with high loss rates and asymmetric
> > bandwidth". "Cellular links" in the late 1990s didn't have the same
> > characteristics that "cellular links" have in 2017, but there are other
> link
> > types with those characteristics, so the documents ended up being useful
> in
> > places like CORE. I'm not suggesting anything like a restructure of the
> > document(s), only that they be clear about how future readers would know
> > whether they should be reading them in 2027)
> >
> >
>

--94eb2c0afb0ad625a30552781151
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div>Hi, Lucien,<br><div class=3D"gmail_extra"><br><div c=
lass=3D"gmail_quote">On Jun 20, 2017 21:57, &quot;Lucien&quot; &lt;<a href=
=3D"mailto:lucien.avramov@gmail.com">lucien.avramov@gmail.com</a>&gt; wrote=
:<br type=3D"attribution"><blockquote class=3D"quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi Spen=
cer, Al (+Scott, +Warren), and all,=C2=A0<div><br></div><div>Thank you for =
the suggestion. Both drafts have been updated with explaining what a data c=
enter network device unter test is for these tests.=C2=A0</div><div><br></d=
iv><div>Besides adding more clarity on both abstracts, and introduction, I =
added the following text in the introduction section of each draft to speci=
fy what a data center Network DUT is for these drafts.=C2=A0</div></div></b=
lockquote></div></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">T=
hat&#39;s great, and thanks!</div><div dir=3D"auto"><br></div><div dir=3D"a=
uto">Spencer</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div class=
=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote class=3D"quote" sty=
le=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div d=
ir=3D"ltr"><div><br></div><div>I hope Scott will appreciate the low buffer =
comment, as the high performance data center switches today have significan=
tly less buffer than other networking devices.=C2=A0</div><div><br></div><d=
iv><div>Currently, typical data Center networking devices are characterized=
</div><div>by:</div><div><br></div><div>-High port density (48 ports of mor=
e)</div><div><br></div><div>-High speed (up to 100 Gb/s currently per port)=
</div><div><br></div><div>-High throughput (line rate on all ports for Laye=
r 2 and/or Layer 3)</div><div><br></div><div>-Low latency (in the microseco=
nd or nanosecond range)</div><div><br></div><div>-Low amount of buffer (in =
the Mb range)</div><div><br></div><div>-Layer 2 and Layer 3 forwarding capa=
bility (Layer 3 not mandatory)</div></div></div><div class=3D"elided-text">=
<div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, Jun 20, 2=
017 at 1:42 PM, Lucien Avramov <span dir=3D"ltr">&lt;<a href=3D"mailto:luci=
en.avramov@gmail.com" target=3D"_blank">lucien.avramov@gmail.com</a>&gt;</s=
pan> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex=
;border-left:1px #ccc solid;padding-left:1ex">Definitely agreed, will place=
 a statement in both documents today to explain what datacenter switches ar=
e at time of writing.<br>
<br>
Thank you,<br>
Lucien<br>
<div class=3D"m_2193317762784539422HOEnZb"><div class=3D"m_2193317762784539=
422h5"><br>
&gt; On Jun 20, 2017, at 13:28, Spencer Dawkins &lt;<a href=3D"mailto:spenc=
erdawkins.ietf@gmail.com" target=3D"_blank">spencerdawkins.ietf@gmail.com</=
a><wbr>&gt; wrote:<br>
&gt;<br>
&gt; Spencer Dawkins has entered the following ballot position for<br>
&gt; draft-ietf-bmwg-dcbench-method<wbr>ology-13: No Objection<br>
&gt;<br>
&gt; When responding, please keep the subject line intact and reply to all<=
br>
&gt; email addresses included in the To and CC lines. (Feel free to cut thi=
s<br>
&gt; introductory paragraph, however.)<br>
&gt;<br>
&gt;<br>
&gt; Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss=
-criteria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/i=
esg/stat<wbr>ement/discuss-criteria.html</a><br>
&gt; for more information about IESG DISCUSS and COMMENT positions.<br>
&gt;<br>
&gt;<br>
&gt; The document, along with other ballot positions, can be found here:<br=
>
&gt; <a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-me=
thodology/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.o=
rg/d<wbr>oc/draft-ietf-bmwg-dcbench-met<wbr>hodology/</a><br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt; COMMENT:<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt;<br>
&gt; (I&#39;m making the same comment on draft-ietf-bmwg-dcbench-termin<wbr=
>ology and<br>
&gt; draft-ietf-bmwg-dcbench-method<wbr>ology)<br>
&gt;<br>
&gt; I&#39;m looking at the ballot positions on draft-ietf-bmwg-dcbench-ter=
min<wbr>ology and<br>
&gt; draft-ietf-bmwg-dcbench-method<wbr>ology that assert these documents a=
ren&#39;t specific<br>
&gt; to &quot;data centers&quot;. That wouldn&#39;t surprise me, but I&#39;=
m not seeing a definition<br>
&gt; of &quot;data center&quot; in either document - did I miss it?<br>
&gt;<br>
&gt; I suspect that the authors have specific technical characteristics in =
mind,<br>
&gt; that happen to map onto what data centers look like today, but may not=
 in the<br>
&gt; future (&quot;RFCs last forever&quot;). Is it possible to tease those =
characteristics out?<br>
&gt;<br>
&gt; (Full disclosure: my first working group in the IETF could have been c=
alled<br>
&gt; &quot;TCP over cellular links&quot;, but it turned out that when we sa=
id &quot;cellular<br>
&gt; links&quot;, we meant &quot;low-speed links with high loss rates and a=
symmetric<br>
&gt; bandwidth&quot;. &quot;Cellular links&quot; in the late 1990s didn&#39=
;t have the same<br>
&gt; characteristics that &quot;cellular links&quot; have in 2017, but ther=
e are other link<br>
&gt; types with those characteristics, so the documents ended up being usef=
ul in<br>
&gt; places like CORE. I&#39;m not suggesting anything like a restructure o=
f the<br>
&gt; document(s), only that they be clear about how future readers would kn=
ow<br>
&gt; whether they should be reading them in 2027)<br>
&gt;<br>
&gt;<br>
</div></div></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div>

--94eb2c0afb0ad625a30552781151--


From nobody Wed Jun 21 08:50:00 2017
Return-Path: <bclaise@cisco.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E193912EB29; Wed, 21 Jun 2017 08:49:53 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: Benoit Claise <bclaise@cisco.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org, tim.chown@jisc.ac.uk
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149806019382.15579.8882065223122169054.idtracker@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 08:49:53 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/bC19g76g2ccmLMszSJzBsgEm3Lk>
Subject: [bmwg] Benoit Claise's No Objection on draft-ietf-bmwg-dcbench-methodology-15: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 15:49:54 -0000

Benoit Claise has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-15: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

At some point in time, I was wondering if this document was about
(benchmarking) terminology or about (benchmarking) performance metric
definition? And I was wondering what the connection was with the RFC 6390 PMOL
template and with
https://tools.ietf.org/html/draft-ietf-ippm-metric-registry-11? When I look at
the section 3 "jitter", there are obvious elements from the PMOL template.
Thinking about it, whether a performance metric is used for benchmark or not,
we need a formal definition. Is it time for: 1. IPPM to finish up
draft-ietf-ippm-metric-registry 2. start populating the registry 3. BMWG to
start using/specifying those performance metrics

Thoughts?

Below is Tim Chown's OPS DIR review:
verall, the document is well written, and I believe it to be Ready with minor
nits.

General comment:

It would be interesting to see an Appendix with an example of a recorded test
using the language defined here.

Specific comments:

Section 1:

“- Low amount of buffer (in the Mb range)”
Change to MB?  (given later you refer to KB/MB/GB as the measurement unit for
buffers)

Section 2.1

Expand DUT on first use.

Section 3.1

Perhaps clarify relationship of Delay and Latency, since you focus on Latency
in the document and not Delay?

Last para, you say “If” here, but for Latency the FILO timestamp was a MUST in
Section 2?  This doesn’t seem consistent?

Expand PDV on first use.

Section 6.1.1

“1518 bytes frames” -> “1518 byte frames”

Section 7.1,

Why is ‘and’ in quotes here?  Not sure you can say the balance is defined by
goodput?  Do you mean that goodput is an indication of the balance?  For
standard TCP, a very small loss can have a dramatic effect on application
throughput.

The second para should follow the first, change “[RFC2647].  Goodput…” to
“[RFC2647], i.e., goodput…”

Section 7.3

I don’t understand how the example given correlates to G = S/Ft ?

There’s a few typos in this section; please re-check.



From nobody Wed Jun 21 09:49:12 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 85E2D12EC00; Wed, 21 Jun 2017 09:48:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cooperw.in header.b=vTe2TaGn; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=OlJlA7Xd
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qwmMmJAREUJk; Wed, 21 Jun 2017 09:48:49 -0700 (PDT)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DECC012EBCD; Wed, 21 Jun 2017 09:48:46 -0700 (PDT)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 33FA820C95; Wed, 21 Jun 2017 12:48:46 -0400 (EDT)
Received: from frontend1 ([10.202.2.160]) by compute7.internal (MEProxy); Wed, 21 Jun 2017 12:48:46 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cooperw.in; h=cc :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=fm1; bh=Z5Ou/fe3Xcme7EuzGObKmjbChPadzP84E+mvw9Z+Y 5o=; b=vTe2TaGnfshaOd8n0jq8chvCxRUrPjMsQuzEDvD1/MFja+0fXalM1/tgZ G8zLEmjziHNVSUl/pQzkX2shvcZBP6KyJRyeYqGe4iKy2ttWpgNyN1/a37JseVkb fWoyd1vunHc9V1uzeAL0iCjlJA9e17Dwdbxvxza/b8E7caZajLAmnHzT3Ft6ARRM ID3+GxFGO01jT7YFUrMCctvM2DFLZrXeXfAi4Je5HgAJP3CYWLf5iVsjfvGSlK7F A4mxZcBnfwosqvYdW2p6k0tewaqT4UVsSis+Wi1uIxqolKBn5CA+DY5EDqtfH1l5 u36eG+yJmCrvL8/yxXg8ktrHsFpow==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=Z5Ou/fe3Xcme7EuzGO bKmjbChPadzP84E+mvw9Z+Y5o=; b=OlJlA7Xd+hbUV/itGXx0pszdqPeR+OC3by lGw1kF2/LdBDRr2nmGl8/cQDZngMPokvRYaBmibsuE1pUxJnIRQwvSHSjsa7C2nf fAxTjcH42J3cszFSd9BAOFYWRaPvBEQp+9lKqzRqu1MFt1ixf7eapuum1agQCago WINnFQ1YiLhOIC162/CLQVjRexywD1O1eN9RE1By94EQtgXIMrLbtIve9GTmDkZS ef/7VL9bUa2jdw85MJsTc9Dgz8wfyIRlW230hP2VshtJ2c42VNb8dRBilQXhapJJ KDxuhZyP1PAk7CqO3tq3n/M2LQSCditfpTkjH1eA+rGZ7jTXXm9w==
X-ME-Sender: <xms:bqNKWRM62QLV1EEvrzTxd-wcgTz2b1AKzoiK4JUqvjaDxUwPCX_s9g>
X-Sasl-enc: g80yF0khpRd9o1W7AeoYjslAqKc4MPrMpxHaAHFQZb/T 1498063725
Received: from sjc-alcoop-8812.cisco.com (unknown [128.107.241.165]) by mail.messagingengine.com (Postfix) with ESMTPA id 5B6027E72B; Wed, 21 Jun 2017 12:48:44 -0400 (EDT)
Content-Type: multipart/alternative; boundary="Apple-Mail=_AFC8A1E5-5097-4FF6-8B0E-5E79144E9572"
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Alissa Cooper <alissa@cooperw.in>
In-Reply-To: <CALTEt=DWK5n8xVJ_ZD3j5BZmBT=_id1ZERCHPY3SGEqYL6N8sg@mail.gmail.com>
Date: Wed, 21 Jun 2017 12:48:42 -0400
Cc: Matthew Miller <linuxwolf+ietf@outer-planes.net>, gen-art <gen-art@ietf.org>, ietf <ietf@ietf.org>, bmwg@ietf.org, draft-ietf-bmwg-dcbench-terminology.all@ietf.org
Message-Id: <4776794B-BB06-451C-8B6D-A58051638306@cooperw.in>
References: <149741280522.7481.5692213354540706614@ietfa.amsl.com> <CALTEt=DWK5n8xVJ_ZD3j5BZmBT=_id1ZERCHPY3SGEqYL6N8sg@mail.gmail.com>
To: Lucien Avramov <lucienav@google.com>
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Y11_B8ngwuzM3pEBJRX3ioRZV0U>
Subject: Re: [bmwg] [Gen-art] Genart last call review of draft-ietf-bmwg-dcbench-terminology-10
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 16:48:54 -0000

--Apple-Mail=_AFC8A1E5-5097-4FF6-8B0E-5E79144E9572
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Mtt, thanks for your review. Lucien, than you for your responses. I have =
entered a No Objection ballot position on this document.

Alissa


> On Jun 16, 2017, at 1:09 AM, Lucien Avramov <lucienav@google.com> =
wrote:
>=20
> Hi Matthew!
>=20
> Thank you for the review!
> We incorporated your feedback and now it's in -12: of the draft =
<https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/>
>=20
> Please see inline more details to each of your feedback.
>=20
> Thank you for such a detailed write up.
>=20
> Best,
> Lucien
>=20
>=20
>=20
>>=20
>> Subject: Genart last call review of =
draft-ietf-bmwg-dcbench-terminology-10
>>=20
>> Reviewer: Matthew Miller
>> Review result: Almost Ready
>>=20
>> I am the assigned Gen-ART reviewer for this draft. The General Area
>> Review Team (Gen-ART) reviews all IETF documents being processed
>> by the IESG for the IETF Chair.  Please treat these comments just
>> like any other last call comments.
>>=20
>> For more information, please see the FAQ at
>>=20
>> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq =
<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>>.
>>=20
>> Document: draft-ietf-bmwg-dcbench-terminology-10
>> Reviewer: Matthew A. Miller
>> Review Date: 2017-06-13
>> IETF LC End Date: 2017-06-13
>> IESG Telechat date: N/A
>>=20
>> Summary:
>>=20
>> This document is almost ready; this document is on the path to its
>> goals set out in the abstract and introduction, but the execution
>> needs more work.
>>=20
>> I think the authors would benefit from another round of proofreading
>> and submit a new revision.  Below are the most obvious or egregious
>> issues I have.
>>=20
>> Major issues:
>>=20
>> * In section 6.2 "Incast", it is unclear what is being measured,
>> and what the units (if any) for that measurement are.  Is it the
>> percentage of synchronization?  Is it the synchronous arrival time
>> (which presumably is measured as a delta over some magnitude of
>> seconds)? Is it the number of ingress and egress ports?  If it is
>> the number of ingress and egress ports, then it should define what
>> "egress" means for the purpose of this measurement
>=20
>=20
> fixed!=20
>>=20
>>=20
>> * In Section 8. "Security Considerations", what is the reason for
>> the "SHOULD" and "SHOULD NOT" in the last paragraph (discussing
>> special capabilities)?  At the least it seems safest for the
>> exceptions that rationalize the "SHOULD" and "SHOULD NOT" be
>> explicitly stated here, or changed to "MUST" and "MUST NOT" if
>> that rationale cannot be stated.
>=20
>=20
> These are the standard security considerations all BMWG uses for the =
drafts. We want to stay with the considerations we chose.  =20
>>=20
>>=20
>> Minor issues:
>>=20
>> * RFC 2119 should be a normative reference, as the reader MUST
>> understand what the terms as-written mean.
>=20
> done
> =20
>>=20
>>=20
>> * RFC 5841 needs to be a listed reference; It is explicitly pointed
>> to in Sections 3.1. and 3.2. =46rom the text is seems a Normative
>> reference is warranted but it at least needs to be documented in
>> References.
>=20
> done=20
>>=20
>>=20
>> * RFC 2647 needs to be a listed reference. It is explicitly pointed
>> to in Section 7.1.; from the text it seems appropriate to be an
>> Informative reference.
>=20
> done=20
>>=20
>>=20
>> * The definition in Section 1.2. of what the "Measurement Units"
>> subsections definitions format doesn't seem to be followed in
>> this document.  Specifically, the "Measurement Units" subsections
>> rarely ever mention a unit of measure; instead the unit of measure
>> is almost always specified as part of the "Definition" subsection.
>> It may be worth revisiting the name for the "Measurement Units"
>> subsections, or to move the unit of measure out of the
>> "Definitions" subsections and into the "Measurement Units"
>> subsections.
>=20
> Section 1.2 has the purpose to explain how the document is formatted.=20=

> it has 3 subsections each time moving forward as described in 1.2. We =
use that third subsection in this document to explain how to measure the =
topic discussed.=20
>=20
> Definition: The specific definition for the term.
> Discussion: A brief discussion about the term, its application
> Measurement Units
>>=20
>>=20
>> * In section 2.3., the use of MUST, MAY, and MUST NOT seems
>> a little contradictory.  The MUST for point 1 would seem to
>> already negate point 3, and prohibit point 2.  Would changing
>> that MUST to SHOULD be acceptable?
>=20
> We wanted to keep MUST , MAY and MUST NOT to explain that : 1) is the =
way to go ALWAYS. 2) can be done but it very specific to particular =
corner use case. 3) should never be done, although 1) negates 3), we =
want to call out 3, because people often time do 1) and don't follow 3. =
(they would do a LIFO test although they must not do it).=20
>=20
> =20
>=20
> These are the standard security considerations all BMWG uses for the =
drafts. We want to stay with the considerations we chose. =20
>>=20
>>=20
>> Nits/editorial comments:
>>=20
>> * It would be helpful to include a reference to what "north-south"
>> and "east-west" mean, if possible.
>=20
> done!=20
>>=20
>>=20
>> * the acronyms DUT and SUT -- at the least -- need to be
>> expanded on first use, or in Section 1.1. "Terminology".
>=20
>=20
> done!=20
>>=20
>>=20
>> * In a number of subsections (notably under Section 4. and Section
>> 6.), square brackets ("[" and "]") are used instead of parentheses
>> ("(" and ")"), without any clear reason for the difference.  They
>> should be switched to parentheses.
>=20
> done!=20
>>=20
>>=20
>> * In Section 2.2., the word "for" or "at" should be removed in the
>> sentence "The change of behavior happens for at specific larger
>> packet sizes."
>=20
> done!=20
>>=20
>>=20
>> * Section 2.3., the phrase "the application commonly need to" should
>> be changed to "the application commonly needs to".
>>=20
>=20
> done!=20
>> * In Section 5.2., there is a comma missing between "crystals" and
>> "or" in "The crystals or clock modules, usually have a specific".
>=20
> done!=20
>>=20
>>=20
>> * Also in Section 5.2., the term "devices under test" is used
>> instead of DUTs the rest of the document seems to use.
>=20
> done!=20
>>=20
>>=20
>> * Throughout Section 6.1., the acronyms "cos" and "dcsp" should be
>> expanded on first use.
>=20
> done!=20
>>=20
>>=20
>> * In Section 6.1.1, the definition of "Buffer Size" calls out using
>> some magnitude of bytes (B, KB, MB, GB), then the example denotes
>> "Mb" -- which commonly is used for megabits.  =46rom the rest of the
>> definition, it seems this should be MB, but I have not calculated
>> if the numerical value preceding the unit needs to be revisited.
>=20
> done!=20
>>=20
>>=20
>> * In Section 6.2.1, the Stateful and Stateless terms both use the
>> phrase "such as for example", which is redundant.  Either "such as"
>> or "for example" is sufficient, and the other should be removed.
>>=20
>> * The formatting in Section 10. "References" is very inconsistent.
>> There is odd indentation and inconsistent anchor usage.
>=20
>=20
> fixed as much as i could, there is a problem with nits in submission =
tool when =20
>>=20
>>=20
>> * The affiliation and email address for Lucien Avramov is Google,
>> but the street address is mostly that for Cisco Systems, Inc.
>=20
> done=20
> _______________________________________________
> Gen-art mailing list
> Gen-art@ietf.org
> https://www.ietf.org/mailman/listinfo/gen-art


--Apple-Mail=_AFC8A1E5-5097-4FF6-8B0E-5E79144E9572
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">Mtt, thanks for your review. Lucien, than you =
for your responses. I have entered a No Objection ballot position on =
this document.</div><div class=3D""><br class=3D""></div><div =
class=3D"">Alissa</div><div class=3D""><br class=3D""></div><br =
class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On =
Jun 16, 2017, at 1:09 AM, Lucien Avramov &lt;<a =
href=3D"mailto:lucienav@google.com" class=3D"">lucienav@google.com</a>&gt;=
 wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div =
dir=3D"ltr" class=3D"">Hi Matthew!<div class=3D""><br =
class=3D""></div><div class=3D"">Thank you for the review!</div><div =
class=3D"">We incorporated your feedback and now it's in -12: of the <a =
href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminolo=
gy/" class=3D"">draft</a></div><div class=3D""><br class=3D""></div><div =
class=3D"">Please see inline more details to each of your =
feedback.</div><div class=3D""><br class=3D""></div><div class=3D"">Thank =
you for such a detailed write up.</div><div class=3D""><br =
class=3D""></div><div class=3D"">Best,</div><div =
class=3D"">Lucien</div><div class=3D""><br class=3D""></div><div =
class=3D""><br class=3D""><div class=3D"gmail_extra"><br class=3D""><div =
class=3D"gmail_quote"><div style=3D"min-height:100%" class=3D""><div =
class=3D"gmail-nH" style=3D"width:1188px"><div class=3D"gmail-nH" =
style=3D""><div class=3D"gmail-nH gmail-bkL"><div class=3D"gmail-no"><div =
class=3D"gmail-nH gmail-bkK gmail-nn" style=3D"width:986px"><div =
class=3D"gmail-nH"><div class=3D"gmail-nH"><div class=3D"gmail-nH =
gmail-ar4 gmail-z"><div class=3D"gmail-aeI"><div class=3D"gmail-AO"><div =
id=3D"gmail-:4" class=3D"gmail-Tm gmail-aeJ" style=3D"height:551px"><div =
id=3D"gmail-:2" class=3D"gmail-aeF" style=3D"min-height:121px"><div =
class=3D"gmail-nH"><div class=3D"gmail-nH"><div class=3D"gmail-nH =
gmail-g"><table class=3D"gmail-Bs gmail-nH gmail-iY" cellpadding=3D"0" =
style=3D"width:956px"><tbody class=3D""><tr class=3D""><td =
class=3D"gmail-Bu"><div class=3D"gmail-nH gmail-if"><div class=3D"gmail-nH=
 gmail-aHU"><div class=3D"gmail-nH gmail-hx"><div class=3D"gmail-nH"><div =
class=3D"gmail-hn gmail-ie gmail-nH gmail-oy8Mbf gmail-h7" =
tabindex=3D"-1"><div class=3D"gmail-Bk" style=3D"width:705px"><div =
class=3D"gmail-G3 gmail-G2 gmail-afm"><div id=3D"gmail-:1ph" =
class=3D""><div class=3D"gmail-adn gmail-ads"><div class=3D"gmail-gs"><div=
 id=3D"gmail-:1qg" class=3D"gmail-gt gmail-ii gmail-adO" =
style=3D"font-size:12.8px"><div id=3D"gmail-:1z2" class=3D"gmail-a3s =
gmail-aXjCH gmail-m15caf4bc03a5f652"><div dir=3D"ltr" class=3D""><div =
class=3D"gmail_extra"><div class=3D"gmail_quote"><div =
class=3D"gmail-adM"><div class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" =
class=3D""><blockquote type=3D"cite" style=3D"font-size:12.8px" =
class=3D""><br class=3D""><b class=3D"">Subject:</b>&nbsp;<b =
class=3D"">Genart last call review of draft-ietf-bmwg-dcbench-termin<wbr =
class=3D"">ology-10</b><br class=3D""><br class=3D""></blockquote><div =
style=3D"font-size:12.8px" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D"">Reviewer: Matthew Miller<br =
class=3D"">Review result: Almost Ready<br class=3D""><br class=3D"">I am =
the assigned Gen-ART reviewer for this draft. The General Area<br =
class=3D"">Review Team (Gen-ART) reviews all IETF documents being =
processed<br class=3D"">by the IESG for the IETF Chair.&nbsp; Please =
treat these comments just<br class=3D"">like any other last call =
comments.<br class=3D""><br class=3D"">For more information, please see =
the FAQ at<br class=3D""><br class=3D"">&lt;<a =
href=3D"https://trac.ietf.org/trac/gen/wiki/GenArtfaq" target=3D"_blank" =
class=3D"">https://trac.ietf.org/trac/ge<wbr =
class=3D"">n/wiki/GenArtfaq</a>&gt;.<br class=3D""><br =
class=3D"">Document: draft-ietf-bmwg-dcbench-termin<wbr =
class=3D"">ology-10<br class=3D"">Reviewer: Matthew A. Miller<br =
class=3D"">Review Date: 2017-06-13<br class=3D"">IETF LC End Date: =
2017-06-13<br class=3D"">IESG Telechat date: N/A<br class=3D""><br =
class=3D"">Summary:<br class=3D""><br class=3D"">This document is almost =
ready; this document is on the path to its<br class=3D"">goals set out =
in the abstract and introduction, but the execution<br class=3D"">needs =
more work.<br class=3D""><br class=3D"">I think the authors would =
benefit from another round of proofreading<br class=3D"">and submit a =
new revision.&nbsp; Below are the most obvious or egregious<br =
class=3D"">issues I have.<br class=3D""><br class=3D"">Major issues:<br =
class=3D""><br class=3D"">* In section 6.2 "Incast", it is unclear what =
is being measured,<br class=3D"">and what the units (if any) for that =
measurement are.&nbsp; Is it the<br class=3D"">percentage of =
synchronization?&nbsp; Is it the synchronous arrival time<br =
class=3D"">(which presumably is measured as a delta over some magnitude =
of<br class=3D"">seconds)? Is it the number of ingress and egress =
ports?&nbsp; If it is<br class=3D"">the number of ingress and egress =
ports, then it should define what<br class=3D"">"egress" means for the =
purpose of this =
measurement</blockquote></div></div></div></div></div></div></div></blockq=
uote><div class=3D""><br class=3D""></div></div></div><div =
class=3D"">fixed!&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* In Section =
8. "Security Considerations", what is the reason for<br class=3D"">the =
"SHOULD" and "SHOULD NOT" in the last paragraph (discussing<br =
class=3D"">special capabilities)?&nbsp; At the least it seems safest for =
the<br class=3D"">exceptions that rationalize the "SHOULD" and "SHOULD =
NOT" be<br class=3D"">explicitly stated here, or changed to "MUST" and =
"MUST NOT" if<br class=3D"">that rationale cannot be =
stated.</blockquote></div></div></div></div></div></div></blockquote><div =
class=3D""><br class=3D""></div></span><div class=3D"">These are the =
standard security considerations all BMWG uses for the drafts. We want =
to stay with the considerations we chose. &nbsp;&nbsp;</div><span =
class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">Minor =
issues:<br class=3D""><br class=3D"">* RFC 2119 should be a normative =
reference, as the reader MUST<br class=3D"">understand what the terms =
as-written =
mean.</blockquote></div></div></div></div></div></div></blockquote></span>=
<div class=3D"">done</div><span class=3D"gmail-im"><div =
class=3D"">&nbsp;</div><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* RFC 5841 =
needs to be a listed reference; It is explicitly pointed<br class=3D"">to =
in Sections 3.1. and 3.2. =46rom the text is seems a Normative<br =
class=3D"">reference is warranted but it at least needs to be documented =
in<br =
class=3D"">References.</blockquote></div></div></div></div></div></div></b=
lockquote></span><div class=3D"">done&nbsp;</div><span =
class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* RFC 2647 =
needs to be a listed reference. It is explicitly pointed<br class=3D"">to =
in Section 7.1.; from the text it seems appropriate to be an<br =
class=3D"">Informative =
reference.</blockquote></div></div></div></div></div></div></blockquote></=
span><div class=3D"">done&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* The =
definition in Section 1.2. of what the "Measurement Units"<br =
class=3D"">subsections definitions format doesn't seem to be followed =
in<br class=3D"">this document.&nbsp; Specifically, the "Measurement =
Units" subsections<br class=3D"">rarely ever mention a unit of measure; =
instead the unit of measure<br class=3D"">is almost always specified as =
part of the "Definition" subsection.<br class=3D"">It may be worth =
revisiting the name for the "Measurement Units"<br class=3D"">subsections,=
 or to move the unit of measure out of the<br class=3D"">"Definitions" =
subsections and into the "Measurement Units"<br =
class=3D"">subsections.</blockquote></div></div></div></div></div></div></=
blockquote></span><div class=3D"">Section 1.2 has the purpose to explain =
how the document is formatted.&nbsp;</div><div class=3D"">it has 3 =
subsections each time moving forward as described in 1.2. We use that =
third subsection in this document to explain how to measure the topic =
discussed.&nbsp;</div><div class=3D""><br class=3D""></div><div =
class=3D""><div class=3D""><ul class=3D""><li class=3D"">Definition: The =
specific definition for the term.</li></ul></div><div class=3D""><ul =
class=3D""><li class=3D"">Discussion: A brief discussion about the term, =
its application</li></ul></div><div class=3D""><ul class=3D""><li =
class=3D"">Measurement Units<br class=3D""></li></ul></div></div><span =
class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* In section =
2.3., the use of MUST, MAY, and MUST NOT seems<br class=3D"">a little =
contradictory.&nbsp; The MUST for point 1 would seem to<br =
class=3D"">already negate point 3, and prohibit point 2.&nbsp; Would =
changing<br class=3D"">that MUST to SHOULD be =
acceptable?</blockquote></div></div></div></div></div></div></blockquote><=
/span><div class=3D"">We wanted to keep MUST , MAY and MUST NOT to =
explain that : 1) is the way to go ALWAYS. 2) can be done but it very =
specific to particular corner use case. 3) should never be done, =
although 1) negates 3), we want to call out 3, because people often time =
do 1) and don't follow 3. (they would do a LIFO test although they must =
not do it).&nbsp;</div><div class=3D""><br class=3D""></div><div =
class=3D"">&nbsp;</div><div class=3D""><br class=3D""></div><div =
class=3D"">These are the standard security considerations all BMWG uses =
for the drafts. We want to stay with the considerations we chose. =
&nbsp;</div><span class=3D"gmail-im"><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">Nits/editorial=
 comments:<br class=3D""><br class=3D"">* It would be helpful to include =
a reference to what "north-south"<br class=3D"">and "east-west" mean, if =
possible.</blockquote></div></div></div></div></div></div></blockquote></s=
pan><div class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* the =
acronyms DUT and SUT -- at the least -- need to be<br class=3D"">expanded =
on first use, or in Section 1.1. =
"Terminology".</blockquote></div></div></div></div></div></div></blockquot=
e><div class=3D""><br class=3D""></div></span><div =
class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* In a =
number of subsections (notably under Section 4. and Section<br =
class=3D"">6.), square brackets ("[" and "]") are used instead of =
parentheses<br class=3D"">("(" and ")"), without any clear reason for =
the difference.&nbsp; They<br class=3D"">should be switched to =
parentheses.</blockquote></div></div></div></div></div></div></blockquote>=
</span><div class=3D"">done!&nbsp;</div><span =
class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* In Section =
2.2., the word "for" or "at" should be removed in the<br =
class=3D"">sentence "The change of behavior happens for at specific =
larger<br class=3D"">packet =
sizes."</blockquote></div></div></div></div></div></div></blockquote></spa=
n><div class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* Section =
2.3., the phrase "the application commonly need to" should<br =
class=3D"">be changed to "the application commonly needs to".<br =
class=3D""><br =
class=3D""></blockquote></div></div></div></div></div></div></blockquote><=
/span><div class=3D"">done!&nbsp;</div><span =
class=3D"gmail-im"><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D"">* In Section 5.2., there is a comma =
missing between "crystals" and<br class=3D"">"or" in "The crystals or =
clock modules, usually have a =
specific".</blockquote></div></div></div></div></div></div></blockquote></=
span><div class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote=
 class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* Also in =
Section 5.2., the term "devices under test" is used<br class=3D"">instead =
of DUTs the rest of the document seems to =
use.</blockquote></div></div></div></div></div></div></blockquote></span><=
div class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* Throughout =
Section 6.1., the acronyms "cos" and "dcsp" should be<br =
class=3D"">expanded on first =
use.</blockquote></div></div></div></div></div></div></blockquote></span><=
div class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* In Section =
6.1.1, the definition of "Buffer Size" calls out using<br class=3D"">some =
magnitude of bytes (B, KB, MB, GB), then the example denotes<br =
class=3D"">"Mb" -- which commonly is used for megabits.&nbsp; =46rom the =
rest of the<br class=3D"">definition, it seems this should be MB, but I =
have not calculated<br class=3D"">if the numerical value preceding the =
unit needs to be =
revisited.</blockquote></div></div></div></div></div></div></blockquote></=
span><div class=3D"">done!&nbsp;</div><span class=3D"gmail-im"><blockquote=
 class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* In Section =
6.2.1, the Stateful and Stateless terms both use the<br class=3D"">phrase =
"such as for example", which is redundant.&nbsp; Either "such as"<br =
class=3D"">or "for example" is sufficient, and the other should be =
removed.<br class=3D""><br class=3D"">* The formatting in Section 10. =
"References" is very inconsistent.<br class=3D"">There is odd =
indentation and inconsistent anchor =
usage.</blockquote></div></div></div></div></div></div></blockquote><div =
class=3D""><br class=3D""></div></span><div class=3D"">fixed as much as =
i could, there is a problem with nits in submission tool when =
&nbsp;</div><span class=3D"gmail-im"><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div =
class=3D"gmail-m_-566981276781100764gmail-HOEnZb"><div =
class=3D"gmail-m_-566981276781100764gmail-h5"><div dir=3D"ltr" =
class=3D""><div class=3D"gmail_quote"><div dir=3D"auto" class=3D""><div =
class=3D"gmail-m_-566981276781100764gmail-m_8608744719394608261h5"><blockq=
uote type=3D"cite" class=3D""><br class=3D""><br class=3D"">* The =
affiliation and email address for Lucien Avramov is Google,<br =
class=3D"">but the street address is mostly that for Cisco Systems, =
Inc.</blockquote></div></div></div></div></div></div></blockquote></span><=
div =
class=3D"">done&nbsp;</div></div></div></div></div></div></div></div></div=
></div></div></div></div></div></div></div></td></tr></tbody></table></div=
></div></div></div></div></div></div></div></div></div></div></div></div><=
/div></div></div></div></div></div></div>
_______________________________________________<br class=3D"">Gen-art =
mailing list<br class=3D""><a href=3D"mailto:Gen-art@ietf.org" =
class=3D"">Gen-art@ietf.org</a><br =
class=3D"">https://www.ietf.org/mailman/listinfo/gen-art<br =
class=3D""></div></blockquote></div><br class=3D""></body></html>=

--Apple-Mail=_AFC8A1E5-5097-4FF6-8B0E-5E79144E9572--


From nobody Wed Jun 21 10:08:23 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 97428127876; Wed, 21 Jun 2017 10:08:13 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149806489346.15702.4378487506430727152@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 10:08:13 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/JEcxrgE9j2Nu_r_-F2ivjFGOivA>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-17.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 17:08:14 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-17.txt
	Pages           : 17
	Date            : 2017-06-21

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to performance
evaluations of data center network equipment. This document establishes
the important concepts for benchmarking network switches and routers in
the data center and, is a pre-requisite to the test methodology
publication [1]. Many of these terms and methods may be applicable to
network equipment beyond this publication's scope as the technologies
originally applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-17
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-17

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-17


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Wed Jun 21 10:17:31 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 71275127876; Wed, 21 Jun 2017 10:17:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level: 
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4ociLgxpFIhm; Wed, 21 Jun 2017 10:17:26 -0700 (PDT)
Received: from mail-yb0-x232.google.com (mail-yb0-x232.google.com [IPv6:2607:f8b0:4002:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1140A128B38; Wed, 21 Jun 2017 10:17:25 -0700 (PDT)
Received: by mail-yb0-x232.google.com with SMTP id 84so48520492ybe.0; Wed, 21 Jun 2017 10:17:25 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=jhcG3KxRPjUuDIde11J8il63q5i27XX6vGVTu7kT2iw=; b=SIhOc6MC+yWGJI8uoS7V7TtTqH7dQHSj2qYuYD5icpia0OH0J+mAiN6GYFhHcI8yYq K80etZVWTS35220ix9RLSh7FypN0E0mNyASBwG79na8lQQml6IvRWPUn2NgzOMZmKf2P Ug0MkYb11XZwfWl44nVpwORXbBjOsHLJFR5oRpVfNuT95SXoMrczGs7MIBEYExeh43nZ G+WMSsDzOhalqqgo3J+h5Tpm5mkGe/ByRm4gDAnGmECl41GbGauxttrhrjRgOEPoNzCP dgQze1H7DLKHDwHl7bcNfMecSu9ul1mX6MaHMIIWnM6KOA+GHUgwWpQsvsF1xWZQmzhB Hi0g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=jhcG3KxRPjUuDIde11J8il63q5i27XX6vGVTu7kT2iw=; b=Km7EBCLFCKQGwGmdAs5FxyL94XWHHDEAPX+qgkRcRIWWO0fABHwTmFABjSFVQgdUgd jUBK66R/uAH5nO5N7Y6t+7lB/4oWxPm7cjou+dfJhre8xfFZPD3n6L9sSxXtX7ahNRK9 uZcydnbeGvZyTH/vcMVQsT3UtoUGvyYF2TZ572dZX9z7pxV6FBeQKM2y49iS294z3RiU 7t2YR9iWpUFrb4x5esBKFmUUll3AZAlJIFN/Y6b49xrlLv/jZrSJjQG29xK4kGY+2I0t rMlpIgtp12evqaavdIRRBOVUeZmtztmi3aQ3DNlaCqyq+g3n9/df/T3a663XvDhwhLHc JtMw==
X-Gm-Message-State: AKS2vOx6cFE445zvR/ybgBdRteiwKuMS4tHpIkNBDD7mZ6h20YTOwynL nWimjvvdCyBLKMltcTBUBpPwMb6V/Q==
X-Received: by 10.37.51.67 with SMTP id z64mr27961890ybz.145.1498065444157; Wed, 21 Jun 2017 10:17:24 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 10:17:23 -0700 (PDT)
In-Reply-To: <149806019382.15579.8882065223122169054.idtracker@ietfa.amsl.com>
References: <149806019382.15579.8882065223122169054.idtracker@ietfa.amsl.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 10:17:23 -0700
Message-ID: <CAArZqeU-kLzgLWoSsrMAjgZ8rmmAFB=N0MXW_wFNFV2ta2Dnag@mail.gmail.com>
To: Benoit Claise <bclaise@cisco.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org,  Tim Chown <tim.chown@jisc.ac.uk>
Content-Type: multipart/alternative; boundary="001a1148af24a6575405527b8acb"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/SjlKO7-mpVoW6fdnFMcgNUDHeic>
Subject: Re: [bmwg] Benoit Claise's No Objection on draft-ietf-bmwg-dcbench-methodology-15: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 17:17:29 -0000

--001a1148af24a6575405527b8acb
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Benoit,

Thank you for the comment! ( I did address also Tim's comment by updating
the draft and answering his email)

Please see inline for your comments:

On Wed, Jun 21, 2017 at 8:49 AM, Benoit Claise <bclaise@cisco.com> wrote:

> Benoit Claise has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-15: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> At some point in time, I was wondering if this document was about
> (benchmarking) terminology or about (benchmarking) performance metric
> definition? And I was wondering what the connection was with the RFC 6390
> PMOL
> template and with
> https://tools.ietf.org/html/draft-ietf-ippm-metric-registry-11? When I
> look at
> the section 3 "jitter", there are obvious elements from the PMOL template=
.
> Thinking about it, whether a performance metric is used for benchmark or
> not,
> we need a formal definition. Is it time for: 1. IPPM to finish up
> draft-ietf-ippm-metric-registry 2. start populating the registry 3. BMWG
> to
> start using/specifying those performance metrics
>
> Thoughts?
>

Initially when we started this over 4 years ago i wanted to do 1 document
for both terminology and method of benchmarking. It became a very long and
intense publication, and I believe back then when you were also our AD, the
feedback we kept getting was to make this easier and break it down in two
publications wich we did. Now looking back, I am glad we did because it
provides a much better structure and it's more digestable and succinct.

The document terminology is about defining the terms specific for data
center benchmarking. It's not about the performance per say. But it's
related to the implication of performance testing.

The document methodology (on the same last call), is about how and what to
benchmark (using the terminologies defined in the other document).

I think it would be a great idea to harmonize all performance metrics
accross WGs, but that's really beyond the scope of what our goal is in
those two publications. I am happy to help participate in such incentive /
project if there is critical mass willing to do it.





>
> Below is Tim Chown's OPS DIR review:
> verall, the document is well written, and I believe it to be Ready with
> minor
> nits.
>
> General comment:
>
> It would be interesting to see an Appendix with an example of a recorded
> test
> using the language defined here.
>
> Specific comments:
>
> Section 1:
>
> =E2=80=9C- Low amount of buffer (in the Mb range)=E2=80=9D
> Change to MB?  (given later you refer to KB/MB/GB as the measurement unit
> for
> buffers)
>
> Section 2.1
>
> Expand DUT on first use.
>
> Section 3.1
>
> Perhaps clarify relationship of Delay and Latency, since you focus on
> Latency
> in the document and not Delay?
>
> Last para, you say =E2=80=9CIf=E2=80=9D here, but for Latency the FILO ti=
mestamp was a
> MUST in
> Section 2?  This doesn=E2=80=99t seem consistent?
>
> Expand PDV on first use.
>
> Section 6.1.1
>
> =E2=80=9C1518 bytes frames=E2=80=9D -> =E2=80=9C1518 byte frames=E2=80=9D
>
> Section 7.1,
>
> Why is =E2=80=98and=E2=80=99 in quotes here?  Not sure you can say the ba=
lance is defined
> by
> goodput?  Do you mean that goodput is an indication of the balance?  For
> standard TCP, a very small loss can have a dramatic effect on application
> throughput.
>
> The second para should follow the first, change =E2=80=9C[RFC2647].  Good=
put=E2=80=A6=E2=80=9D to
> =E2=80=9C[RFC2647], i.e., goodput=E2=80=A6=E2=80=9D
>
> Section 7.3
>
> I don=E2=80=99t understand how the example given correlates to G =3D S/Ft=
 ?
>
> There=E2=80=99s a few typos in this section; please re-check.
>
>
>

--001a1148af24a6575405527b8acb
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Benoit,=C2=A0<div><br></div><div>Thank you for the comm=
ent! ( I did address also Tim&#39;s comment by updating the draft and answe=
ring his email)</div><div><br></div><div>Please see inline for your comment=
s:<br><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Jun=
 21, 2017 at 8:49 AM, Benoit Claise <span dir=3D"ltr">&lt;<a href=3D"mailto=
:bclaise@cisco.com" target=3D"_blank">bclaise@cisco.com</a>&gt;</span> wrot=
e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l=
eft:1px #ccc solid;padding-left:1ex">Benoit Claise has entered the followin=
g ballot position for<br>
draft-ietf-bmwg-dcbench-<wbr>methodology-15: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodo=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>methodology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
At some point in time, I was wondering if this document was about<br>
(benchmarking) terminology or about (benchmarking) performance metric<br>
definition? And I was wondering what the connection was with the RFC 6390 P=
MOL<br>
template and with<br>
<a href=3D"https://tools.ietf.org/html/draft-ietf-ippm-metric-registry-11" =
rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<wbr>draft=
-ietf-ippm-metric-<wbr>registry-11</a>? When I look at<br>
the section 3 &quot;jitter&quot;, there are obvious elements from the PMOL =
template.<br>
Thinking about it, whether a performance metric is used for benchmark or no=
t,<br>
we need a formal definition. Is it time for: 1. IPPM to finish up<br>
draft-ietf-ippm-metric-<wbr>registry 2. start populating the registry 3. BM=
WG to<br>
start using/specifying those performance metrics<br>
<br>
Thoughts?<br></blockquote><div><br></div><div>Initially when we started thi=
s over 4 years ago i wanted to do 1 document for both terminology and metho=
d of benchmarking. It became a very long and intense publication, and I bel=
ieve back then when you were also our AD, the feedback we kept getting was =
to make this easier and break it down in two publications wich we did. Now =
looking back, I am glad we did because it provides a much better structure =
and it&#39;s more digestable and succinct.=C2=A0</div><div><br></div><div>T=
he document terminology is about defining the terms specific for data cente=
r benchmarking. It&#39;s not about the performance per say. But it&#39;s re=
lated to the implication of performance testing.=C2=A0</div><div><br></div>=
<div>The document methodology (on the same last call), is about how and wha=
t to benchmark (using the terminologies defined in the other document).=C2=
=A0</div><div><br></div><div>I think it would be a great idea to harmonize =
all performance metrics accross WGs, but that&#39;s really beyond the scope=
 of what our goal is in those two publications. I am happy to help particip=
ate in such incentive / project if there is critical mass willing to do it.=
=C2=A0</div><div><br></div><div><br></div><div><br></div><div>=C2=A0</div><=
blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px=
 #ccc solid;padding-left:1ex">
<br>
Below is Tim Chown&#39;s OPS DIR review:<br>
verall, the document is well written, and I believe it to be Ready with min=
or<br>
nits.<br>
<br>
General comment:<br>
<br>
It would be interesting to see an Appendix with an example of a recorded te=
st<br>
using the language defined here.<br>
<br>
Specific comments:<br>
<br>
Section 1:<br>
<br>
=E2=80=9C- Low amount of buffer (in the Mb range)=E2=80=9D<br>
Change to MB?=C2=A0 (given later you refer to KB/MB/GB as the measurement u=
nit for<br>
buffers)<br>
<br>
Section 2.1<br>
<br>
Expand DUT on first use.<br>
<br>
Section 3.1<br>
<br>
Perhaps clarify relationship of Delay and Latency, since you focus on Laten=
cy<br>
in the document and not Delay?<br>
<br>
Last para, you say =E2=80=9CIf=E2=80=9D here, but for Latency the FILO time=
stamp was a MUST in<br>
Section 2?=C2=A0 This doesn=E2=80=99t seem consistent?<br>
<br>
Expand PDV on first use.<br>
<br>
Section 6.1.1<br>
<br>
=E2=80=9C1518 bytes frames=E2=80=9D -&gt; =E2=80=9C1518 byte frames=E2=80=
=9D<br>
<br>
Section 7.1,<br>
<br>
Why is =E2=80=98and=E2=80=99 in quotes here?=C2=A0 Not sure you can say the=
 balance is defined by<br>
goodput?=C2=A0 Do you mean that goodput is an indication of the balance?=C2=
=A0 For<br>
standard TCP, a very small loss can have a dramatic effect on application<b=
r>
throughput.<br>
<br>
The second para should follow the first, change =E2=80=9C[RFC2647].=C2=A0 G=
oodput=E2=80=A6=E2=80=9D to<br>
=E2=80=9C[RFC2647], i.e., goodput=E2=80=A6=E2=80=9D<br>
<br>
Section 7.3<br>
<br>
I don=E2=80=99t understand how the example given correlates to G =3D S/Ft ?=
<br>
<br>
There=E2=80=99s a few typos in this section; please re-check.<br>
<br>
<br>
</blockquote></div><br></div></div></div>

--001a1148af24a6575405527b8acb--


From nobody Wed Jun 21 10:18:35 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A1781128656; Wed, 21 Jun 2017 10:18:33 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149806551364.15863.13119660386518766466@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 10:18:33 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/-5FQya1KXZS6X8W7TGsc_U30rb0>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-16.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 17:18:33 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-16.txt
	Pages           : 20
	Date            : 2017-06-21

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. A pre-requisite to this
   publication is the terminology document [1]. Many of these terms and
   methods may be applicable beyond this publication's scope as the
   technologies originally applied in the data center are deployed
   elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-16
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-16

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-16


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Wed Jun 21 10:53:27 2017
Return-Path: <alissa@cooperw.in>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5A3CA128D44; Wed, 21 Jun 2017 10:53:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Level: 
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cooperw.in header.b=TApRVUuj; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=fmmlIOmC
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id upvzYG269ure; Wed, 21 Jun 2017 10:53:23 -0700 (PDT)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 62E3A129431; Wed, 21 Jun 2017 10:53:23 -0700 (PDT)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 9C736209A2; Wed, 21 Jun 2017 13:53:22 -0400 (EDT)
Received: from frontend1 ([10.202.2.160]) by compute7.internal (MEProxy); Wed, 21 Jun 2017 13:53:22 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cooperw.in; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=bqeAzQcE4MMwhfoFE4 bG+oNer6POLRSYbduC3tESnNM=; b=TApRVUujbqCGdX9vJpMHSJpN5Tw7O8v6oy 2w7p35m3+E8WnxExCDYfjrGD2lECfKYFxHGD65N+IbnOjl3w00u+7frzD1tseh3F 5N1y9W4XxypMy+xg5qBoc1qSN2dQCb416qwZqtaJ4POBVVdJNPZwj4ZiX2YChoIK rg3HoKdXJEqh3JrqdU0+sYByEbH0dCZ4FH++bxqGifMKHnxU2iriNUSXE3tvlSJX fwl0LRNtfjZV7QGlQ3HBnlyi6kWnEYhajXIMLLtjuopxbbN8gRX46AhBd05Kfv1p 8m6DK7yahUXvoA6ycu6BdujOl5bnyKVqXLMzWcbbrvGd/J/YjGRg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=bqeAzQcE4MMwhfoFE4bG+oNer6POLRSYbduC3tESnNM=; b=fmmlIOmC x0m9xPyh6jqGZmWeqz/RtLV9s7b/QulFX4R+yKQf4p/+xJwvEFwMgwS0SX17BaEw HyBdQDu07nPbn41y8u8HWuNsG46JazuQ4hF8ZwAdrt3qCzshonBoIN8tn4A71kF+ FB58sQl+SgI2ew1Rr/yLdfRp3yaqjhAQeLjfRTqKrQl5/67aiD00lGtBGWxOO8yo yn3FXvn2vIiKt9UQVDHtIR1PNPOf5eIjkQqUPajrr3YPA2Wm/By4mERe00yQlc9n gjsOrqzgehAGw5UJOsmcNiHQ/m/cNssk7mNNMNp6Hw2baBH+M0/J29qK5/FCLKwd hqBGJtRzRkMjdw==
X-ME-Sender: <xms:krJKWeSjgZoqlAWjL0CPSPcBmUImgXt7emeZzLYdxnd_NHQuJetJjg>
X-Sasl-enc: 4JWMCIbaKu5cQ1YQ8v5TNjrjKvHtzWIBHoqdt+IA5vlR 1498067602
Received: from sjc-alcoop-8812.cisco.com (unknown [128.107.241.165]) by mail.messagingengine.com (Postfix) with ESMTPA id 9A4317E76A; Wed, 21 Jun 2017 13:53:21 -0400 (EDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Alissa Cooper <alissa@cooperw.in>
In-Reply-To: <149786475813.10715.10157714153398648057@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 13:53:19 -0400
Cc: gen-art <gen-art@ietf.org>, draft-ietf-bmwg-dcbench-methodology.all@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <FDFE11DF-D90D-447E-B75D-14660537E36E@cooperw.in>
References: <149786475813.10715.10157714153398648057@ietfa.amsl.com>
To: Dan Romascanu <dromasca@gmail.com>
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/aASki_e33zXfc7gLimNqiSLfXrM>
Subject: Re: [bmwg] [Gen-art] Genart telechat review of draft-ietf-bmwg-dcbench-methodology-12
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 17:53:25 -0000

Dan, thanks for your review. I have entered a No Objection ballot =
position.

Alissa

> On Jun 19, 2017, at 5:32 AM, Dan Romascanu <dromasca@gmail.com> wrote:
>=20
> Reviewer: Dan Romascanu
> Review result: Ready
>=20
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair. Please wait for direction from your
> document shepherd or AD before posting a new version of the draft.
>=20
> For more information, please see the FAQ at
>=20
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>=20
> Document: draft-ietf-bmwg-dcbench-methodology-??
> Reviewer: Dan Romascanu
> Review Date: 2017-06-19
> IETF LC End Date: 2017-06-13
> IESG Telechat date: 2017-06-22
>=20
> Summary:
>=20
> This Informational I-D describes test and evaluation methodology and
> measurement techniques for physical network equipment in the data =
center. It's
> an useful and clearly written document and it is ready for =
publication. A few
> minor issues and nits that were brought up in my LC review were =
addressed by
> the authors - thanks!
>=20
> Major issues:
>=20
> Minor issues:
>=20
> Nits/editorial comments:
>=20
>=20
> _______________________________________________
> Gen-art mailing list
> Gen-art@ietf.org
> https://www.ietf.org/mailman/listinfo/gen-art


From nobody Wed Jun 21 11:13:37 2017
Return-Path: <suresh.krishnan@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 4ABDD126E3A; Wed, 21 Jun 2017 11:13:29 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Suresh Krishnan <suresh.krishnan@gmail.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 11:13:29 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/P-V_PQObu7dgKJdVhDunfWhHiUQ>
Subject: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 18:13:29 -0000

Suresh Krishnan has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-16: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Not sure if the WG discussed this, but it would have been interesting to have
some energy efficiency related metrics as the draft is datacenter focused and
energy efficiency is an important consideration in datacenters.



From nobody Wed Jun 21 11:41:54 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 96F22129496; Wed, 21 Jun 2017 11:41:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.998
X-Spam-Level: 
X-Spam-Status: No, score=-0.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rsfaNX9sro-m; Wed, 21 Jun 2017 11:41:45 -0700 (PDT)
Received: from mail-yb0-x22e.google.com (mail-yb0-x22e.google.com [IPv6:2607:f8b0:4002:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BB6F8129488; Wed, 21 Jun 2017 11:41:45 -0700 (PDT)
Received: by mail-yb0-x22e.google.com with SMTP id e201so45264250ybb.1; Wed, 21 Jun 2017 11:41:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fGMjeB3jD/uIzV30L+c8jmpFXyYWrwfCeG2Oq12r3H8=; b=WEujzvNZ2PhEQmWC4+Lhw6jtD8Djiu7Op96WnGRML6enTXLlnIXv1t88pIO5bqVJaW C1HjpaIeQVEzdvoFq2VL0+e6PJTUfNqt5pvheMW6ZUG2ZFuCIGD0S8DTskIAI9C8StUi HWrNuGF76WiiX6NLWTGsSAfcfbKU+hE/DDnmFcsXNan2sJdm/3iZuhIfr4GKOtKU0QDU Co1js4cicT3wye6gkapo70yRinGENcYitNpjJBg1EL6vbI9iBNadvlNapyFxXNzctOrG Iyc6dtHvlxNvDjDQHmW1DoB28n0QldjTDll5LlXgHAah9i2UV3VYdNd5toVYT8ikDU9w 4xSw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fGMjeB3jD/uIzV30L+c8jmpFXyYWrwfCeG2Oq12r3H8=; b=XoGr1/d8lOQsv7XbALNEMhsvC7rOZEpj0qc4pJapw56holSXZwuYTd7/NJEh28clmd DEgA7+6NmENYIniO04gosvB81vVDfAAZRAdyz4Y0glJFgi+mi8AKIsdrQ32neLgweKm3 PHr9kuFuVmOL41DAUr46b958Yjv6wy/zljRjda+n/lcUlkR8Z1M3V/IWnrIeVdzcWEZg Qxx2D0LPhWcuaPrb7SCCSYZ0/MBKWmEr8TjW9F//8JXlfIar3aN1xF7G0v0KS63seTgq Hc2MYiZZHSoHeGX9RCHeJqK81bczZie3GB0Ep1JsI4JuitmlxAGEZj7+ZrO/FRtPL7+r nOtQ==
X-Gm-Message-State: AKS2vOxLdDxMuipGICsNm9oETNnmzGP6H1ln7EgPVXpzd+EvBF1q6Shq I/f5PPpXOwqBs3yzH5TSUZR1qHRnoNC9
X-Received: by 10.37.194.198 with SMTP id s189mr29708052ybf.185.1498070504985;  Wed, 21 Jun 2017 11:41:44 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 11:41:44 -0700 (PDT)
In-Reply-To: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com>
References: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 11:41:44 -0700
Message-ID: <CAArZqeX9vVNS0iawMeV9-U5ZD5c1OA-eLjHCzPz0kKrRDsBRUg@mail.gmail.com>
To: Suresh Krishnan <suresh.krishnan@gmail.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c0546b44c728405527cb8ab"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/lbE_ycwjAOdIKm8FUu4kg13H5nU>
Subject: Re: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 18:41:48 -0000

--94eb2c0546b44c728405527cb8ab
Content-Type: text/plain; charset="UTF-8"

Thanks Suresh,

We actually have talked about this in the beginning, and this is not in the
scope of the networking metrics we are after for performance. The amount of
power a data center switch uses at no use or at 100% load is marginal.

Cheers,
Lucien

On Wed, Jun 21, 2017 at 11:13 AM, Suresh Krishnan <suresh.krishnan@gmail.com
> wrote:

> Suresh Krishnan has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-16: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> Not sure if the WG discussed this, but it would have been interesting to
> have
> some energy efficiency related metrics as the draft is datacenter focused
> and
> energy efficiency is an important consideration in datacenters.
>
>
>

--94eb2c0546b44c728405527cb8ab
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks Suresh,=C2=A0<div><br></div><div>We actually have t=
alked about this in the beginning, and this is not in the scope of the netw=
orking metrics we are after for performance. The amount of power a data cen=
ter switch uses at no use or at 100% load is marginal.=C2=A0</div><div><br>=
</div><div>Cheers,</div><div>Lucien</div></div><div class=3D"gmail_extra"><=
br><div class=3D"gmail_quote">On Wed, Jun 21, 2017 at 11:13 AM, Suresh Kris=
hnan <span dir=3D"ltr">&lt;<a href=3D"mailto:suresh.krishnan@gmail.com" tar=
get=3D"_blank">suresh.krishnan@gmail.com</a>&gt;</span> wrote:<br><blockquo=
te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so=
lid;padding-left:1ex">Suresh Krishnan has entered the following ballot posi=
tion for<br>
draft-ietf-bmwg-dcbench-<wbr>methodology-16: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodo=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>methodology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
Not sure if the WG discussed this, but it would have been interesting to ha=
ve<br>
some energy efficiency related metrics as the draft is datacenter focused a=
nd<br>
energy efficiency is an important consideration in datacenters.<br>
<br>
<br>
</blockquote></div><br></div>

--94eb2c0546b44c728405527cb8ab--


From nobody Wed Jun 21 11:42:30 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 75B23129488; Wed, 21 Jun 2017 11:42:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level: 
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kkfMN-9QYGKJ; Wed, 21 Jun 2017 11:42:26 -0700 (PDT)
Received: from mail-yw0-x22a.google.com (mail-yw0-x22a.google.com [IPv6:2607:f8b0:4002:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 46F1E129480; Wed, 21 Jun 2017 11:42:26 -0700 (PDT)
Received: by mail-yw0-x22a.google.com with SMTP id 63so67845057ywr.0; Wed, 21 Jun 2017 11:42:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vWEB0BHSmhOhdTwnpXllfvKaHPMNydEga6fNsIJBoyw=; b=TsEkBmR6Ihfd92Hoo8cTR/kZvZQ/lX3/9ZRdMwXgTNyzVlUsZUI+Fdau9+OD3AmEzW yMzekJ5IxuWl2MPWSpkplmJzpWQX54dw33F62lJwyXwFLKnyiAJzRV6jChDov2QiqXKt gQICa9Y70Uc00679gT4LkvnMJMbba9kbmxUa69luguwzMmvFT5MCxxwtM4ploVmHEY0Z n3CrHTtWxOIpainXmA5LJOX3CFFAat1CQHXrdPLsXJV/1HNKTaZk6zvfeXeoEyVQKVLJ QWwS9GnTCmAaXs78qfWcVbNUU9OttpDdzzI1hDy3GHfOKMF5HccswvCO5sztJOYqBd9l xsrA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vWEB0BHSmhOhdTwnpXllfvKaHPMNydEga6fNsIJBoyw=; b=bau9LENjp9hWsZeZj4Auw1PrrEVxpar41qiVrGN3Mdb8J0LTvJLDGXGt99X/yGqUOR cQJA1IzwoR25gYOUM5ZAKsNDO+DJzbm0HUCAK8IoTiOXuLlUSo04rgPnGllw8BW5fGtT gIEPTqkggngvuvh6MyKijH8YHW/LZpQqnm9irdRKr5MDy3UPkiQ/mynecCekI23+IGTE /8rAy5KYSfcndKDAA8E2ej1u8RANiKwxKYa1Fhdk7i+kKyYhSCzcColSCXJtLOamdxQx 1n+6ohif6h0BRz3TG8w8/kyYRAGL43QUbNSV49AgwY1pSn2X+a/zIDypvVMWvc07miJv b5QA==
X-Gm-Message-State: AKS2vOwzJgImFfvvynleeFdA0Yp032nVMmr3Y1ScM42KkySouISH1NXy rRntIZ7MLJin/CgW6fkKdYSoLMEnww==
X-Received: by 10.129.80.193 with SMTP id e184mr29166326ywb.44.1498070545497;  Wed, 21 Jun 2017 11:42:25 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 11:42:25 -0700 (PDT)
In-Reply-To: <CAArZqeX9vVNS0iawMeV9-U5ZD5c1OA-eLjHCzPz0kKrRDsBRUg@mail.gmail.com>
References: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com> <CAArZqeX9vVNS0iawMeV9-U5ZD5c1OA-eLjHCzPz0kKrRDsBRUg@mail.gmail.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 11:42:25 -0700
Message-ID: <CAArZqeVNE9pnva8xf9XoOEUHAWBn60+xvgCDbzHvQD55EmN+FQ@mail.gmail.com>
To: Suresh Krishnan <suresh.krishnan@gmail.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a1147e4e2b690dd05527cba2d"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/WAE68A_4uKzGJdEfXn1WQCrOpf8>
Subject: Re: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 18:42:28 -0000

--001a1147e4e2b690dd05527cba2d
Content-Type: text/plain; charset="UTF-8"

I meant to say the delta of power when the data center switch is iddle and
at 100% of capacity is marginal.

On Wed, Jun 21, 2017 at 11:41 AM, Lucien <lucien.avramov@gmail.com> wrote:

> Thanks Suresh,
>
> We actually have talked about this in the beginning, and this is not in
> the scope of the networking metrics we are after for performance. The
> amount of power a data center switch uses at no use or at 100% load is
> marginal.
>
> Cheers,
> Lucien
>
> On Wed, Jun 21, 2017 at 11:13 AM, Suresh Krishnan <
> suresh.krishnan@gmail.com> wrote:
>
>> Suresh Krishnan has entered the following ballot position for
>> draft-ietf-bmwg-dcbench-methodology-16: No Objection
>>
>> When responding, please keep the subject line intact and reply to all
>> email addresses included in the To and CC lines. (Feel free to cut this
>> introductory paragraph, however.)
>>
>>
>> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
>> for more information about IESG DISCUSS and COMMENT positions.
>>
>>
>> The document, along with other ballot positions, can be found here:
>> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>>
>>
>>
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>
>> Not sure if the WG discussed this, but it would have been interesting to
>> have
>> some energy efficiency related metrics as the draft is datacenter focused
>> and
>> energy efficiency is an important consideration in datacenters.
>>
>>
>>
>

--001a1147e4e2b690dd05527cba2d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I meant to say the delta of power when the data center swi=
tch is iddle and at 100% of capacity is marginal.=C2=A0</div><div class=3D"=
gmail_extra"><br><div class=3D"gmail_quote">On Wed, Jun 21, 2017 at 11:41 A=
M, Lucien <span dir=3D"ltr">&lt;<a href=3D"mailto:lucien.avramov@gmail.com"=
 target=3D"_blank">lucien.avramov@gmail.com</a>&gt;</span> wrote:<br><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex"><div dir=3D"ltr">Thanks Suresh,=C2=A0<div><br></di=
v><div>We actually have talked about this in the beginning, and this is not=
 in the scope of the networking metrics we are after for performance. The a=
mount of power a data center switch uses at no use or at 100% load is margi=
nal.=C2=A0</div><div><br></div><div>Cheers,</div><div>Lucien</div></div><di=
v class=3D"HOEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><div cl=
ass=3D"gmail_quote">On Wed, Jun 21, 2017 at 11:13 AM, Suresh Krishnan <span=
 dir=3D"ltr">&lt;<a href=3D"mailto:suresh.krishnan@gmail.com" target=3D"_bl=
ank">suresh.krishnan@gmail.com</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">Suresh Krishnan has entered the following ballot position for=
<br>
draft-ietf-bmwg-dcbench-method<wbr>ology-16: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/s=
tat<wbr>ement/discuss-criteria.html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodo=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/d<=
wbr>oc/draft-ietf-bmwg-dcbench-met<wbr>hodology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
Not sure if the WG discussed this, but it would have been interesting to ha=
ve<br>
some energy efficiency related metrics as the draft is datacenter focused a=
nd<br>
energy efficiency is an important consideration in datacenters.<br>
<br>
<br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a1147e4e2b690dd05527cba2d--


From nobody Wed Jun 21 12:06:16 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 0701612946E; Wed, 21 Jun 2017 12:06:06 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Ben Campbell <ben@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-terminology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149807196602.15838.8048520689148705215.idtracker@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 12:06:06 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/msRPP-t61Z50xYdfQPz_hfLoYSU>
Subject: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 19:06:06 -0000

Ben Campbell has entered the following ballot position for
draft-ietf-bmwg-dcbench-terminology-17: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I find the naming of the draft fairly confusing. It goes way beyond
"terminology"; it makes a number of normative (using 2119 language) statements
about benchmarking procedures. I wonder why the sections about procedure did
not go into the methodology draft instead. In general, I don't think putting
normative language in an informational terminology draft is a good idea. (This
would have been a DISCUSS, except that I am aware the bmwg has decided to make
all its drafts informational and to still use 2119 language. For the record, I
think that policy falls down with this draft.)

I agree with the comment from others that this does not seem to be specific to
datacenters.

- 2.2: Definitions of "store-and-forward" and "cut-through" when used in this
context would be helpful. The first may be obvious, but the best I can do with
"cut-through" is assume it means the opposite of "store-and-forward".

- 6.2: After reading the definition of "Incast" several times, I'm still not
sure what it means or what is being measured.



From nobody Wed Jun 21 12:08:56 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5AD4A1296CD; Wed, 21 Jun 2017 12:08:49 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Ben Campbell <ben@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149807212936.15488.7309772374304994572.idtracker@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 12:08:49 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/McLL8pWr1tmHO5kybQxrEiS6Di8>
Subject: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 19:08:49 -0000

Ben Campbell has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-16: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

- I support Alvaro's DISCUSS

- I agree with the questions about why this is specific to datacenters.

- Please expand DUT on first use.



From nobody Wed Jun 21 12:18:30 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 235EE12946E; Wed, 21 Jun 2017 12:18:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level: 
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NPCziw3HFMYI; Wed, 21 Jun 2017 12:17:56 -0700 (PDT)
Received: from mail-yw0-x233.google.com (mail-yw0-x233.google.com [IPv6:2607:f8b0:4002:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5881212943A; Wed, 21 Jun 2017 12:17:56 -0700 (PDT)
Received: by mail-yw0-x233.google.com with SMTP id e142so68256028ywa.1; Wed, 21 Jun 2017 12:17:56 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=BAymDwym53Y0kIFsg7H0G1M6k8Abqs6YRt7Qy8ccq+A=; b=gyEroCGd2juq5+wq0yjH7/nxE9gwrqV0mn2bFU3SVyccO1Vitz2YNkHRI/FVwOzL2y ijjaGIRXmLcgaTYldvKvOFskI2ywIJCpFTmamUIcZ5fYIFPkJPWWNSTKQ/bzIwbwDNOH sc6N0YpTbK0yxTUDCnveh0efL7SMWifcXt+OKcEYcaLzXiuk1xfo0bE/u4jlG2wrhA6P rI//quUMJ3ktrwVGASkjIhHnmI1xmtWLnHAisNRseZGf0F6hqaBxaC4BWo1GTSmv+88M tgKTuoltdXKbiA6V3WoX9dF5WWODEp5XgAwl4z3SrPXm6zOO8sFjg80iNNduKOCyofBj erBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=BAymDwym53Y0kIFsg7H0G1M6k8Abqs6YRt7Qy8ccq+A=; b=M6aoEtXr7HhLAIdD1oxIemDBiELtXtKaxswd+L4LpupmnPZhDWjIhRDTRdUVuqmoXS jzVJtAbRtVG18iZ56GsSm1RUQHGdgYwmuNVEiUfQXzDGZDC+J3/rZkwRExwIGJdvbIfL +fr9ZtAh5hzFyDY0utdGS4/Zar7xZzFln43H1thgczFZ13dniNf8MVIbCO6KcYXjDqcg kE0atOm7yued33imp3YDut8u0Azf4REWIz8QrEzMqqQVq5IEb3PfHvxiFT+k+aEaXzHk Boyz7EsdHq6cLZLcMypHhK3WhKJuW7HFypvoiddySs+JhRMDwQv6OOmbgDVYVIbNqMt1 qw4g==
X-Gm-Message-State: AKS2vOyiBxoFtO4M2JfCcVGcYUaOMJYSaaiwOyQTQCCOauNm1Lh2E85z cAiCyj2sEcgWpL9UWT8nwDLywxVXzQ==
X-Received: by 10.129.89.9 with SMTP id n9mr3554900ywb.302.1498072675595; Wed, 21 Jun 2017 12:17:55 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 12:17:55 -0700 (PDT)
In-Reply-To: <149807196602.15838.8048520689148705215.idtracker@ietfa.amsl.com>
References: <149807196602.15838.8048520689148705215.idtracker@ietfa.amsl.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 12:17:55 -0700
Message-ID: <CAArZqeX2_1XJhuqhMdHYRELrERgMzu+AV9zQAFLQUdpMbh7PcA@mail.gmail.com>
To: Ben Campbell <ben@nostrum.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a11471a1ead3c1005527d3972"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/NYPVGpgWGva8q4vXLdDdWU4OuzU>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 19:18:29 -0000

--001a11471a1ead3c1005527d3972
Content-Type: text/plain; charset="UTF-8"

Hi Ben,

Thanks for the comment! Please find my answers inline, and please
acknowledge back!

Appreciate you taking the time to look at our work.

Lucien

On Wed, Jun 21, 2017 at 12:06 PM, Ben Campbell <ben@nostrum.com> wrote:

> Ben Campbell has entered the following ballot position for
> draft-ietf-bmwg-dcbench-terminology-17: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> I find the naming of the draft fairly confusing. It goes way beyond
> "terminology"; it makes a number of normative (using 2119 language)
> statements
> about benchmarking procedures. I wonder why the sections about procedure
> did
> not go into the methodology draft instead. In general, I don't think
> putting
> normative language in an informational terminology draft is a good idea.
> (This
> would have been a DISCUSS, except that I am aware the bmwg has decided to
> make
> all its drafts informational and to still use 2119 language. For the
> record, I
> think that policy falls down with this draft.)
>

That's how we have decided it makes sense at BMWG to proceed with these two
drafts. We have been hashing this out for 4+ years now and the current
state is the consensus.  As author, I have no intention nor desire to
change this

>
> I agree with the comment from others that this does not seem to be
> specific to
> datacenters.
>

Great, so did we, this is why we already worked on addressing this by:

   -  calling out specifically that this specifically applies to data
   center switches (defining what those are today)
   - stating clearly that it can be applied to switches out of the data
   center, but that's not the specific scope of this


> - 2.2: Definitions of "store-and-forward" and "cut-through" when used in
> this
> context would be helpful. The first may be obvious, but the best I can do
> with
> "cut-through" is assume it means the opposite of "store-and-forward".
>

Those are cleary defined in RFC 1242, which we reference. We don't want to
duplicate definitions here.



>
> - 6.2: After reading the definition of "Incast" several times, I'm still
> not
> sure what it means or what is being measured.
>

It's many to one type of network traffic patterns. Its very commonly found
in cloud data centers implementing distributed storage/compute frameworks
(big data for example).

--001a11471a1ead3c1005527d3972
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Ben,=C2=A0<div><br></div><div>Thanks for the comment! P=
lease find my answers inline, and please acknowledge back!</div><div><br></=
div><div>Appreciate you taking the time to look at our work.</div><div><br>=
</div><div>Lucien</div><div class=3D"gmail_extra"><br><div class=3D"gmail_q=
uote">On Wed, Jun 21, 2017 at 12:06 PM, Ben Campbell <span dir=3D"ltr">&lt;=
<a href=3D"mailto:ben@nostrum.com" target=3D"_blank">ben@nostrum.com</a>&gt=
;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color=
:rgb(204,204,204);padding-left:1ex">Ben Campbell has entered the following =
ballot position for<br>
draft-ietf-bmwg-dcbench-termin<wbr>ology-17: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/s=
tat<wbr>ement/discuss-criteria.html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-termino=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/d<=
wbr>oc/draft-ietf-bmwg-dcbench-ter<wbr>minology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
I find the naming of the draft fairly confusing. It goes way beyond<br>
&quot;terminology&quot;; it makes a number of normative (using 2119 languag=
e) statements<br>
about benchmarking procedures. I wonder why the sections about procedure di=
d<br>
not go into the methodology draft instead. In general, I don&#39;t think pu=
tting<br>
normative language in an informational terminology draft is a good idea. (T=
his<br>
would have been a DISCUSS, except that I am aware the bmwg has decided to m=
ake<br>
all its drafts informational and to still use 2119 language. For the record=
, I<br>
think that policy falls down with this draft.)<br></blockquote><div><br></d=
iv><div>That&#39;s how we have decided it makes sense at BMWG to proceed wi=
th these two drafts. We have been hashing this out for 4+ years now and the=
 current state is the consensus.=C2=A0 As author, I have no intention nor d=
esire to change this</div><blockquote class=3D"gmail_quote" style=3D"margin=
:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-lef=
t-color:rgb(204,204,204);padding-left:1ex">
<br>
I agree with the comment from others that this does not seem to be specific=
 to<br>
datacenters.<br></blockquote><div><br></div><div>Great, so did we, this is =
why we already worked on addressing this by:</div><div><ul><li>=C2=A0callin=
g out specifically that this specifically applies to data center switches (=
defining what those are today)<br></li><li>stating clearly that it can be a=
pplied to switches out of the data center, but that&#39;s not the specific =
scope of this</li></ul></div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-=
left-color:rgb(204,204,204);padding-left:1ex">
<br>
- 2.2: Definitions of &quot;store-and-forward&quot; and &quot;cut-through&q=
uot; when used in this<br>
context would be helpful. The first may be obvious, but the best I can do w=
ith<br>
&quot;cut-through&quot; is assume it means the opposite of &quot;store-and-=
forward&quot;.<br></blockquote><div><br></div><div>Those are cleary defined=
 in=C2=A0<span style=3D"color:rgb(0,0,0);font-size:13.333333015441895px">RF=
C 1242, which we reference. We don&#39;t want to duplicate definitions here=
.=C2=A0</span></div><div><span style=3D"color:rgb(0,0,0);font-size:13.33333=
3015441895px">=C2=A0=C2=A0</span></div><div>=C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;bo=
rder-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
- 6.2: After reading the definition of &quot;Incast&quot; several times, I&=
#39;m still not<br>
sure what it means or what is being measured.<br></blockquote><div><br></di=
v><div>It&#39;s many to one type of network traffic patterns. Its very comm=
only found in cloud data centers implementing distributed storage/compute f=
rameworks (big data for example). =C2=A0=C2=A0</div><div><br></div><div><br=
></div></div></div></div>

--001a11471a1ead3c1005527d3972--


From nobody Wed Jun 21 12:22:41 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4F96812943A; Wed, 21 Jun 2017 12:22:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level: 
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 92kPxHXVANZp; Wed, 21 Jun 2017 12:22:31 -0700 (PDT)
Received: from mail-yw0-x236.google.com (mail-yw0-x236.google.com [IPv6:2607:f8b0:4002:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 17B1F1241FC; Wed, 21 Jun 2017 12:22:31 -0700 (PDT)
Received: by mail-yw0-x236.google.com with SMTP id v7so68256761ywc.2; Wed, 21 Jun 2017 12:22:31 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=M+G9Ez8Erbh3xcGXm14cAkEK56Y2apq3A5ITHqnwyaM=; b=LsAYBtNa9D1TITeUGOph4YVC0qlKG6PoLtCK67ecC91mc/3rHWQbvkxUl8UYoXzzOJ tdMQAoDH3Znel7ohLDmwdn+xqr5MdxxOyDJsgGT58OleyfXMbo7nvQH6mXgY0xsGxofr UjvvdF0EalUeizfCkqKDIfHkvhepxIOTRszYKIVozh2Q5/zAApEdrKZ3CfzP6XSHGtZZ 1Hxb3/RYRhlau4SpC/UvIos9QMzTcyszK1FHHx2ctfR+dVhYzgNLMCmymCkRC1o2dx0T 8pH8C9lrEDDYHQEbCKBwh4Sd3SZ3QKWXcgKriwJeU0Ceb2/NMml+BnP/mvl2gRHKe//8 i8tA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=M+G9Ez8Erbh3xcGXm14cAkEK56Y2apq3A5ITHqnwyaM=; b=uQHDzGhtfhjteXy8zEb47rIbq3gilY+FBKuXcuabKfQxDPAMoxkRKEyn2+XLhYHty6 +DxOrsG8zsUfZL7HrR7xfV2H2rmQ7pRHOL++sLLlrZvbMolZuOcy6uFt6a4UtQEyQUQl M+UjR1pm7JmAHcgh8WSbo8VEjoTBrkl10nSjumNPaXRdmDjWZNcxyKpJsbfDv3nButVf cnflUdM8Cl4KkM+QW1VnG/MrLz1PLgXqT0bzGYxMDNlqHeYLShLYk4YQ0COhoSdhwgHA YcnPio/V4yJ9ngtupyAXWjdrp6sORUvJ9Vrt+Vv1T/gdaq5mp/Z5SK0+eCQOVwgauAmW FvDw==
X-Gm-Message-State: AKS2vOyYS2vnlCZ2dOI1r+xn/2u0kAPRsP4zfQDYP1/jQIFkbsLLfHxG ootI5kRf2oyPTwv76SaSd+NcVG8Smscg
X-Received: by 10.129.124.215 with SMTP id x206mr27785498ywc.322.1498072950244;  Wed, 21 Jun 2017 12:22:30 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 12:22:29 -0700 (PDT)
In-Reply-To: <149807212936.15488.7309772374304994572.idtracker@ietfa.amsl.com>
References: <149807212936.15488.7309772374304994572.idtracker@ietfa.amsl.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 12:22:29 -0700
Message-ID: <CAArZqeV_LQ_SSvQif652aZfPrgF2sjwPKxn4uD8vC-bTc40Z0g@mail.gmail.com>
To: Ben Campbell <ben@nostrum.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a1149378c0c105d05527d4ae3"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/uDp5cMVYkVG-hyQfMEdFjdmXM34>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 19:22:33 -0000

--001a1149378c0c105d05527d4ae3
Content-Type: text/plain; charset="UTF-8"

Hi Ben!

Please find inline my comments!

Thanks for reviewing this draft!

On Wed, Jun 21, 2017 at 12:08 PM, Ben Campbell <ben@nostrum.com> wrote:

> Ben Campbell has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-16: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> - I support Alvaro's DISCUSS
>

As author, the meaning of the words chosen (MUST..) are important to me and
I don't agree with make it more loose.


>
> - I agree with the questions about why this is specific to datacenters.
>

Since this comment was on your other draft review, same answer applies here:

Great, so did we, this is why we already worked on addressing this by:

   -  calling out specifically that this specifically applies to data
   center switches (defining what those are today)
   - stating clearly that it can be applied to switches out of the data
   center, but that's not the specific scope of this
   -




> - Please expand DUT on first use.
>

Thanks, this was expanded already on first use in the introduction of -16
document (the Device Under Test (DUT)). I just doubled checked. Please let
me know if I have missed anything, and I will fix it right away.

--001a1149378c0c105d05527d4ae3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Ben!<div><br></div><div>Please find inline my comments!=
</div><div><br></div><div>Thanks for reviewing this draft!</div><div class=
=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Jun 21, 2017 at 12:=
08 PM, Ben Campbell <span dir=3D"ltr">&lt;<a href=3D"mailto:ben@nostrum.com=
" target=3D"_blank">ben@nostrum.com</a>&gt;</span> wrote:<br><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px=
;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1e=
x">Ben Campbell has entered the following ballot position for<br>
draft-ietf-bmwg-dcbench-method<wbr>ology-16: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/s=
tat<wbr>ement/discuss-criteria.html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodo=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/d<=
wbr>oc/draft-ietf-bmwg-dcbench-met<wbr>hodology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
- I support Alvaro&#39;s DISCUSS<br></blockquote><div><br></div><div>As aut=
hor, the meaning of the words chosen (MUST..) are important to me and I don=
&#39;t agree with make it more loose.=C2=A0</div><div>=C2=A0</div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-widt=
h:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-le=
ft:1ex">
<br>
- I agree with the questions about why this is specific to datacenters.<br>=
</blockquote><div><br></div><div>Since this comment was on your other draft=
 review, same answer applies here:</div><div><br></div><div><div style=3D"f=
ont-size:12.800000190734863px">Great, so did we, this is why we already wor=
ked on addressing this by:</div><div style=3D"font-size:12.800000190734863p=
x"><ul><li style=3D"margin-left:15px">=C2=A0calling out specifically that t=
his specifically applies to data center switches (defining what those are t=
oday)<br></li><li style=3D"margin-left:15px">stating clearly that it can be=
 applied to switches out of the data center, but that&#39;s not the specifi=
c scope of this</li><li></li></ul></div><span class=3D"m_717169729082355276=
gmail-im" style=3D"font-size:12.800000190734863px"></span></div><div><br></=
div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color=
:rgb(204,204,204);padding-left:1ex">
<br>
- Please expand DUT on first use.<br></blockquote><div><br></div><div>Thank=
s, this was expanded already on first use in the introduction of -16 docume=
nt (the Device Under Test (DUT)). I just doubled checked. Please let me kno=
w if I have missed anything, and I will fix it right away.</div><div><br></=
div></div><br></div></div>

--001a1149378c0c105d05527d4ae3--


From nobody Wed Jun 21 12:37:00 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CBB261241FC; Wed, 21 Jun 2017 12:36:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.88
X-Spam-Level: 
X-Spam-Status: No, score=-1.88 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GLBmPXVY1jDW; Wed, 21 Jun 2017 12:36:56 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5122D12714F; Wed, 21 Jun 2017 12:36:56 -0700 (PDT)
Received: from [10.0.1.63] (cpe-66-25-7-22.tx.res.rr.com [66.25.7.22]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v5LJapfP050986 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 21 Jun 2017 14:36:52 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host cpe-66-25-7-22.tx.res.rr.com [66.25.7.22] claimed to be [10.0.1.63]
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Ben Campbell <ben@nostrum.com>
In-Reply-To: <CAArZqeX2_1XJhuqhMdHYRELrERgMzu+AV9zQAFLQUdpMbh7PcA@mail.gmail.com>
Date: Wed, 21 Jun 2017 14:36:51 -0500
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <CA7A6183-2EEA-475F-B33B-9B77F075D5FD@nostrum.com>
References: <149807196602.15838.8048520689148705215.idtracker@ietfa.amsl.com> <CAArZqeX2_1XJhuqhMdHYRELrERgMzu+AV9zQAFLQUdpMbh7PcA@mail.gmail.com>
To: Lucien <lucien.avramov@gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/_D6Uv5-wb9qkSf4ok1IByKjZYZo>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 19:36:59 -0000

Thanks for the response. Comments inline:

Ben.

> On Jun 21, 2017, at 2:17 PM, Lucien <lucien.avramov@gmail.com> wrote:
>=20
> Hi Ben,=20
>=20
> Thanks for the comment! Please find my answers inline, and please =
acknowledge back!
>=20
> Appreciate you taking the time to look at our work.
>=20
> Lucien
>=20
> On Wed, Jun 21, 2017 at 12:06 PM, Ben Campbell <ben@nostrum.com> =
wrote:
> Ben Campbell has entered the following ballot position for
> draft-ietf-bmwg-dcbench-terminology-17: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> I find the naming of the draft fairly confusing. It goes way beyond
> "terminology"; it makes a number of normative (using 2119 language) =
statements
> about benchmarking procedures. I wonder why the sections about =
procedure did
> not go into the methodology draft instead. In general, I don't think =
putting
> normative language in an informational terminology draft is a good =
idea. (This
> would have been a DISCUSS, except that I am aware the bmwg has decided =
to make
> all its drafts informational and to still use 2119 language. For the =
record, I
> think that policy falls down with this draft.)
>=20
> That's how we have decided it makes sense at BMWG to proceed with =
these two drafts. We have been hashing this out for 4+ years now and the =
current state is the consensus.  As author, I have no intention nor =
desire to change this

For the record, I think BMWG has adopted interpretations of both =
informational RFC and 2119 keywords that are at best unconventional, and =
at worst tortured. But I also recognize the precedent has been set, and =
don=E2=80=99t mean to hold this draft hostage to it. Thus it was a =
non-blocking comment which you can feel free to ignore :-)


>=20
> I agree with the comment from others that this does not seem to be =
specific to
> datacenters.
>=20
> Great, so did we, this is why we already worked on addressing this by:
> 	=E2=80=A2  calling out specifically that this specifically =
applies to data center switches (defining what those are today)
> 	=E2=80=A2 stating clearly that it can be applied to switches out =
of the data center, but that's not the specific scope of this

Works for me, thanks.


>=20
> - 2.2: Definitions of "store-and-forward" and "cut-through" when used =
in this
> context would be helpful. The first may be obvious, but the best I can =
do with
> "cut-through" is assume it means the opposite of "store-and-forward".
>=20
> Those are cleary defined in RFC 1242, which we reference. We don't =
want to duplicate definitions here.=20

Apologies, I missed the earlier citation to 1242, which was specifically =
in regard to these terms.

>  =20
> =20
>=20
> - 6.2: After reading the definition of "Incast" several times, I'm =
still not
> sure what it means or what is being measured.
>=20
> It's many to one type of network traffic patterns. Its very commonly =
found in cloud data centers implementing distributed storage/compute =
frameworks (big data for example).  =20
>=20

As written, it=E2=80=99s not clear to me if =E2=80=9CIncast=E2=80=9D =
means =E2=80=9Cone to many/many-to-many=E2=80=9D communication in =
general, or the patterns of synchronization that result from that. The =
first paragraph seems to say the former but the second paragraph seems =
to say the latter.



From nobody Wed Jun 21 12:52:40 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A7D5126C89; Wed, 21 Jun 2017 12:52:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.88
X-Spam-Level: 
X-Spam-Status: No, score=-1.88 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vxIUwJnFPgsj; Wed, 21 Jun 2017 12:52:37 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5CB0D1252BA; Wed, 21 Jun 2017 12:52:37 -0700 (PDT)
Received: from [10.0.1.63] (cpe-66-25-7-22.tx.res.rr.com [66.25.7.22]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v5LJqWY8053428 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 21 Jun 2017 14:52:33 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host cpe-66-25-7-22.tx.res.rr.com [66.25.7.22] claimed to be [10.0.1.63]
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Ben Campbell <ben@nostrum.com>
In-Reply-To: <CAArZqeV_LQ_SSvQif652aZfPrgF2sjwPKxn4uD8vC-bTc40Z0g@mail.gmail.com>
Date: Wed, 21 Jun 2017 14:52:32 -0500
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <9D1D7116-BBE9-4BCF-A82C-6D6752C373AA@nostrum.com>
References: <149807212936.15488.7309772374304994572.idtracker@ietfa.amsl.com> <CAArZqeV_LQ_SSvQif652aZfPrgF2sjwPKxn4uD8vC-bTc40Z0g@mail.gmail.com>
To: Lucien <lucien.avramov@gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/iU6E-k-kFWHbHKXeb3_92g-4m_E>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 19:52:39 -0000

Thanks for the response; comments inline:


> On Jun 21, 2017, at 2:22 PM, Lucien <lucien.avramov@gmail.com> wrote:
>=20
> Hi Ben!
>=20
> Please find inline my comments!
>=20
> Thanks for reviewing this draft!
>=20
> On Wed, Jun 21, 2017 at 12:08 PM, Ben Campbell <ben@nostrum.com> =
wrote:
> Ben Campbell has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-16: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut =
this
> introductory paragraph, however.)
>=20
>=20
> Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> - I support Alvaro's DISCUSS
>=20
> As author, the meaning of the words chosen (MUST..) are important to =
me and I don't agree with make it more loose.=20

The issue is that the text (even in 16) has non-matching definitions for =
MUST, SHOULD, MAY, etc. It=E2=80=99s not a matter of =E2=80=9Clooseness=E2=
=80=9D. The 2119 definitions are about interoperability. The definitions =
in 1.2 do not seem to be about interoperability.
> =20
>=20
> - I agree with the questions about why this is specific to =
datacenters.
>=20
> Since this comment was on your other draft review, same answer applies =
here:
>=20
> Great, so did we, this is why we already worked on addressing this by:
> 	=E2=80=A2  calling out specifically that this specifically =
applies to data center switches (defining what those are today)
> 	=E2=80=A2 stating clearly that it can be applied to switches out =
of the data center, but that's not the specific scope of this
>=20
>=20

WFM

>=20
> - Please expand DUT on first use.
>=20
> Thanks, this was expanded already on first use in the introduction of =
-16 document (the Device Under Test (DUT)). I just doubled checked. =
Please let me know if I have missed anything, and I will fix it right =
away.

Ah, I reviewed version 14, but it progressed to version 16 before I =
wrote up my notes and entered them into the tracker. I didn=E2=80=99t =
notice the new revisions until you mentioned them here. I agree DUT is =
properly expanded in 16.
\=


From nobody Wed Jun 21 14:33:47 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 31F801289B0; Wed, 21 Jun 2017 14:33:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ih_-dXgE1XZ3; Wed, 21 Jun 2017 14:33:44 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9BB5B124B0A; Wed, 21 Jun 2017 14:33:44 -0700 (PDT)
Received: from pps.filterd (m0049287.ppops.net [127.0.0.1]) by m0049287.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v5LIZTEF026055; Wed, 21 Jun 2017 14:42:51 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049287.ppops.net-00191d01. with ESMTP id 2b7wxu8yg2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 21 Jun 2017 14:42:51 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5LIgobq018162; Wed, 21 Jun 2017 13:42:50 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5LIgjAZ018147 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 21 Jun 2017 13:42:45 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor); Wed, 21 Jun 2017 18:42:33 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5LIgXq4005783; Wed, 21 Jun 2017 13:42:33 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5LIgQll005366; Wed, 21 Jun 2017 13:42:26 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 338F1E01DE; Wed, 21 Jun 2017 14:42:26 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Wed, 21 Jun 2017 14:42:26 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Suresh Krishnan <suresh.krishnan@gmail.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
Thread-Index: AQHS6roa7rkq6OigoU+mYJNWsuclrKIvpRCA
Date: Wed, 21 Jun 2017 18:42:25 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FE2F16@njmtexg5.research.att.com>
References: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com>
In-Reply-To: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-21_04:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706210311
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/m45latf0RQphg7dzVJVn8yI71gk>
Subject: Re: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 21:33:46 -0000

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdXJlc2ggS3Jpc2huYW4gW21h
aWx0bzpzdXJlc2gua3Jpc2huYW5AZ21haWwuY29tXQ0KPiBTZW50OiBXZWRuZXNkYXksIEp1bmUg
MjEsIDIwMTcgMjoxMyBQTQ0KLi4uDQo+IA0KPiBTdXJlc2ggS3Jpc2huYW4gaGFzIGVudGVyZWQg
dGhlIGZvbGxvd2luZyBiYWxsb3QgcG9zaXRpb24gZm9yDQo+IGRyYWZ0LWlldGYtYm13Zy1kY2Jl
bmNoLW1ldGhvZG9sb2d5LTE2OiBObyBPYmplY3Rpb24NCj4gDQouLi4NCj4gLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LQ0KPiBDT01NRU5UOg0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IA0KPiBOb3Qgc3VyZSBpZiB0aGUgV0cg
ZGlzY3Vzc2VkIHRoaXMsIGJ1dCBpdCB3b3VsZCBoYXZlIGJlZW4gaW50ZXJlc3RpbmcgdG8NCj4g
aGF2ZQ0KPiBzb21lIGVuZXJneSBlZmZpY2llbmN5IHJlbGF0ZWQgbWV0cmljcyBhcyB0aGUgZHJh
ZnQgaXMgZGF0YWNlbnRlcg0KPiBmb2N1c2VkIGFuZA0KPiBlbmVyZ3kgZWZmaWNpZW5jeSBpcyBh
biBpbXBvcnRhbnQgY29uc2lkZXJhdGlvbiBpbiBkYXRhY2VudGVycy4NCj4gDQpbQUNNXSANCkhp
IFN1cmVzaCwNCg0KV2UgaGFkIGEgZHJhZnQvcHJvcG9zZWQgV0kgb24gdGhpcyB0b3BpYywgYnV0
IHdlIGNvdWxkbid0IGZpbmQgc3VmZmljaWVudA0Kc3VwcG9ydCBhbmQgZXhwZXJ0aXNlIHRvIHJl
dmlldyBhbmQgY29tcGxldGUgYSBoaWdoIHF1YWxpdHkNCmRvY3VtZW50LCBzbyB0aGUgcHJvcG9u
ZW50cyBtb3ZlZC1vbi4NCg0KSSBhZ3JlZSwgaXQncyBhbiBpbXBvcnRhbnQgYXJlYSB0byBjb25z
aWRlciBpZiB0aGUgZXhwZXJ0aXNlDQpjYW4gYmUgZ2F0aGVyZWQgaW4gQk1XRy4NCg0KcmVnYXJk
cywNCkFsDQoNCg0K


From nobody Wed Jun 21 18:52:31 2017
Return-Path: <adam@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 8455A1241FC; Wed, 21 Jun 2017 18:52:28 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Adam Roach <adam@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-terminology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 18:52:28 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/alfyNyb5Gi5t4txnXxMHySHekzw>
Subject: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 01:52:28 -0000

Adam Roach has entered the following ballot position for
draft-ietf-bmwg-dcbench-terminology-17: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I am surprised to find normative statements in a terminology document. It might
be appropriate -- if awkward -- to say things like "this term MUST mean x," but
this document includes statements pertaining specifically to what "MUST be
measured," which seems well beyond its purported scope. I would suggest either
removing all such statements, or clearly expanding the scope of the document
(including, and quite importantly, revising its title).

Editorial:
- The first paragraph of Section 6.1 uses plural forms for B, kB, and MB, but
singular for GB. Please make these consistent. - Typically, data units are
capitalized per SI-system prefix rules, which would make "kB" the correct
abbreviation for kilobytes, rather than "KB." - Something has gone well and
truly bonkers with the references section formatting. - Please fix reference
[1] so that it correctly points to draft-ietf-bmwg-dcbench-methodology. This
will ensure that it is updated to the correct RFC value at publication.

Please expand the following acronyms upon first use;
see https://www.rfc-editor.org/materials/abbrev.expansion.txt for guidance.

 - FPGA - Field-Programmable Gate Array
 - LLDP - Link Layer Discovery Protocol

Nits:

  ** The abstract seems to contain references ([1]), which it shouldn't.
     Please replace those with straight textual mentions of the documents in
     question.

  == Missing Reference: 'RFC5481' is mentioned on line 285, but not defined

  == Unused Reference: 'RFC5841' is defined on line 732, but no explicit
     reference was found in the text

  ** Obsolete normative reference: RFC 2554 (ref. 'RFC2544') (Obsoleted by
     RFC 4954)



From nobody Wed Jun 21 20:01:33 2017
Return-Path: <adam@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6F8811293D6; Wed, 21 Jun 2017 20:01:23 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Adam Roach <adam@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149810048336.30434.8858369887731037073.idtracker@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 20:01:23 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Us4hgfm5arQTbUHVEmb_ZuO2JZo>
Subject: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 03:01:23 -0000

Adam Roach has entered the following ballot position for
draft-ietf-bmwg-dcbench-methodology-16: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I agree with Alvaro's discuss -- you can't cite 2119 and then override it. It
may well be that what 2119 does not make sense for this document; if so, don't
cite it, and be clear up front that you're using a *different* set of meanings
for these specific terms.

I *think* I can suss out the nature of "east-west" versus "north-south" in the
introduction, but I'm really not sure. Can you please define these terms or
point to a document that does so?

Editorial:

- Something has gone well and truly bonkers with the references section
formatting.

- Please fix reference [1] so that it correctly points to
draft-ietf-bmwg-dcbench-terminology. This will ensure that it is updated to the
correct RFC value at publication.

Nits:

Please expand "SUT" on first use.

  ** The abstract seems to contain references ([1]), which it shouldn't.
     Please replace those with straight textual mentions of the documents in
     question.



From nobody Wed Jun 21 20:48:02 2017
Return-Path: <kishoret@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DDA37127A90 for <bmwg@ietfa.amsl.com>; Wed, 21 Jun 2017 20:48:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.022
X-Spam-Level: 
X-Spam-Status: No, score=-2.022 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yXzc5COwjwQz for <bmwg@ietfa.amsl.com>; Wed, 21 Jun 2017 20:47:59 -0700 (PDT)
Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0091.outbound.protection.outlook.com [104.47.33.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 53447126CD6 for <bmwg@ietf.org>; Wed, 21 Jun 2017 20:47:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hfjcfgOOLSTuHSE+s7f3keHiBrgO6sq6209y09l+uG8=; b=KpL7f0u96nH/PbpaA0EhQDsnKXK5w899WqeaPEe3fTMx3dFkfAvoFiDEkf7/sGGehZ9lY9Cmdvse3UTIhpzzBzSx8LxWvX+p4tC4luAL8ZqqcMLTGWkjnPgIVPobBZtPRLvS65jsDdytiRx0v0kRy8KLq3fj6OTqeMpSXKe4rOs=
Received: from BN3PR05MB2481.namprd05.prod.outlook.com (10.167.3.22) by BN3PR05MB2674.namprd05.prod.outlook.com (10.166.72.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Thu, 22 Jun 2017 03:47:58 +0000
Received: from BN3PR05MB2481.namprd05.prod.outlook.com ([10.167.3.22]) by BN3PR05MB2481.namprd05.prod.outlook.com ([10.167.3.22]) with mapi id 15.01.1178.023; Thu, 22 Jun 2017 03:47:57 +0000
From: Kishore Tiruveedhula <kishoret@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AQHS6wpWE677kQ0g70yOFJ97nj9MgQ==
Date: Thu, 22 Jun 2017 03:47:57 +0000
Message-ID: <D570B57C.867F2%kishoret@juniper.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.7.4.170508
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [66.129.241.10]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BN3PR05MB2674; 7:v1axb7GE2ScnFkytEBlcQBizTH4sioTFsHVVrecTTUIIW3vVjfzMzZ2m350+5iogqlfHOtGi/XOEj2kSusZpba4GcFGlgQxzIO/nkiokAz/qrAU2zdHVIQX8463jbtno9eVU4GYtb/h2FA3dm+c/goozENH0VGmPRTTsQCRdHx6iiWEzDBqOiXt98jcuXTraDYGAC7qba2gvBujCUrICj8bT7gyLuZzVr4fnA7w5LVAYxdkfm+sYo1UkMrLnCUkI5D98BMEJCHzyyJIN5oKl3uO+no8OTCvLCDCJJKI96yUU8HHsVvJ+qDjIk3+DCmHrY5oZZwFFruXEbcxFYbngKHdV+HHXUfze96HWaOD8hXNQw6pu1aXltRRVYOdKrqYiV+h4G6IvOj1NTKlrAcX/K4d8G3BOnxcH8eCclyhrnRQMTGMd80WUIZqKgZptetci+lb4T79P22GXOYeL0wbcDkk9uhEQiowQfgJadMD8E+gBvwY8aXrEJky/lf2GWa1a8BbHJja2/cYBVrgdU/XxMcrYahM67hZLarPZU6MmIOurvFAXLftfShmyY/Nr7lXXsVGUOYgkQhm6S23uXlSCpsv3bhkX1DxHbby3XDly7bGMTltMiGmxeLSQKKoxHEBZ4uSqraoGksKY73jH4JvMJf2To2VhCRkNCoSa5AfM16KYK1xOABl/QfFMdLjms6dg9wncVKdQ0INpe87nMczbMjUog6kxYJkftUAs4ydfFZoBLpTfnz/l0tXW6M9UyprfGvfD3Y9OzNfSwzcRRpBge2MBZlHYuXA4XbkQQ+9SBYI=
x-ms-office365-filtering-correlation-id: f40fc285-d229-4905-7492-08d4b92178e6
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500055)(300135000095)(300000501055)(300135300095)(300000502055)(300135100095)(22001)(2017030254075)(300000503055)(300135400095)(48565401081)(201703131423075)(201703031133081)(201702281549075)(300000504055)(300135200095)(300000505055)(300135600095)(300000506048)(300135500095); SRVR:BN3PR05MB2674; 
x-ms-traffictypediagnostic: BN3PR05MB2674:
x-microsoft-antispam-prvs: <BN3PR05MB267454AFA7091AB90D71D301C1DB0@BN3PR05MB2674.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(192374486261705)(97927398514766); 
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123558100)(20161123560025)(20161123562025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN3PR05MB2674; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN3PR05MB2674; 
x-forefront-prvs: 03468CBA43
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39400400002)(39410400002)(39860400002)(39850400002)(39840400002)(39450400003)(377454003)(13464003)(3846002)(6506006)(2900100001)(66066001)(2501003)(189998001)(6916009)(5660300001)(36756003)(2906002)(110136004)(3660700001)(3280700002)(102836003)(38730400002)(83506001)(229853002)(305945005)(6116002)(6246003)(53936002)(230783001)(54356999)(6306002)(478600001)(6512007)(25786009)(2351001)(99286003)(4001350100001)(50986999)(122556002)(966005)(77096006)(575784001)(8676002)(86362001)(5640700003)(1730700003)(6436002)(81166006)(14454004)(6486002)(53546010)(8936002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN3PR05MB2674; H:BN3PR05MB2481.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-ID: <504E051B3828604CBED36411FD49E7A5@namprd05.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jun 2017 03:47:57.8620 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR05MB2674
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/FwsV1w8_SfODzdvzuluCOO6Ap_E>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 03:48:02 -0000

 As a co-author, I support the adoption of this draft.


Thanks,
Kishore

>
>> -----Original Message-----
>> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED
>> C
>> (AL)
>> Sent: Saturday, June 03, 2017 6:34 PM
>> To: bmwg@ietf.org
>> Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
>>=20
>> *** Security Advisory: This Message Originated Outside of AT&T ***.
>> Reference http://cso.att.com/EmailSecurity/IDSP.html for more
>> information.
>>=20
>> BMWG,
>>=20
>> At our Chicago meeting, the BMWG reviewed a version of the Internet
>> Draft on
>>=20
>> Benchmarking Methodology for EVPN and PBB-EVPN
>> draft-kishjac-bmwg-evpntest-05
>>=20
>> There was sufficient readership present at the meeting and
>> positive/constructive comments on the work, so the chairs asked
>> whether support existed to adopt the draft as a WG item. There was
>> support and no objections. As always, such decisions are officially
>> taken on the mailing list.
>>=20
>> This message*, therefore, starts a call for adoption on:
>>=20
>> Benchmarking Methodology for EVPN and PBB-EVPN
>> draft-kishjac-bmwg-evpntest-07
>>=20
>> to run until Friday 23 June 2017.
>>=20
>> Please reply to bmwg@ietf.org indicating:
>>=20
>> (1) whether you support addition of the following milestone to
>>    the BMWG charter:
>>=20
>> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
>>=20
>> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>>    as the basis document for this milestone
>>=20
>> (3) whether you commit to reviewing the WG document if adopted
>>=20
>> We are aware that BMWG has nearly completed all its existing chartered
>> items.  This usually begins discussion of re-chartering, this time
>> with Warren (our new AD Advisor). The final decision/schedule for
>> adoption will depend on these discussions.
>>=20
>> thanks for your consideration of the draft, and regards,
>>=20
>> Al
>> bmwg co-chair
>>=20
>> *using Brian Trammell's recent message to IPPM as a template (thanks!)
>>=20
>> _______________________________________________
>> bmwg mailing list
>> bmwg@ietf.org
>> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
>> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
>> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D3tXKQNBR8Q4HC7BBx6pc5me=
8u
>> 9z Ne9QVo_owOF5-ciM&s=3DxIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=3D
>
>
>
>------------------------------
>
>Subject: Digest Footer
>
>_______________________________________________
>bmwg mailing list
>bmwg@ietf.org
>https://www.ietf.org/mailman/listinfo/bmwg
>
>
>------------------------------
>
>End of bmwg Digest, Vol 153, Issue 25
>*************************************


From nobody Wed Jun 21 21:57:04 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id D11E91200C5; Wed, 21 Jun 2017 21:56:54 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149810741481.30434.16274266305696682157@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 21:56:54 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/PccMGuaeo9AnOXcRQ-LQvBkyShI>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-18.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 04:56:55 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-18.txt
	Pages           : 17
	Date            : 2017-06-21

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to performance
evaluations of data center network equipment. This document establishes
the important concepts for benchmarking network switches and routers in
the data center and, is a pre-requisite to the test methodology
publication [draft-ietf-bmwg-dcbench-methodology]. Many of these terms
and methods may be applicable to network equipment beyond this
publication's scope as the technologies originally applied in the data
center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-18
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-18

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-18


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Wed Jun 21 21:57:23 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id EA0FA129481; Wed, 21 Jun 2017 21:57:06 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149810742691.30699.6729766127868840450@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 21:57:06 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/sbCxggrAQNCUihfEMv-dWk_Lvks>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-17.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 04:57:07 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-17.txt
	Pages           : 20
	Date            : 2017-06-21

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. A pre-requisite to this
   publication is the terminology document [draft-ietf-bmwg-dcbench-
   terminology]. Many of these terms and methods may be applicable
   beyond this publication's scope as the technologies originally
   applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-17
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-17

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-17


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Wed Jun 21 21:58:13 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6BD6B127B52; Wed, 21 Jun 2017 21:58:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HHWZ13qVWxuK; Wed, 21 Jun 2017 21:58:05 -0700 (PDT)
Received: from mail-yw0-x231.google.com (mail-yw0-x231.google.com [IPv6:2607:f8b0:4002:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8C31B1200C5; Wed, 21 Jun 2017 21:58:05 -0700 (PDT)
Received: by mail-yw0-x231.google.com with SMTP id 63so1903277ywr.0; Wed, 21 Jun 2017 21:58:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=f/TQY/6VgWQaI+Y7zeTFIZ3sWtMUaMUFBjH7DVPQ6Fk=; b=LC2jOR3V2Qxw/bFeN+6OY8683EG7kJWSQvfUZpJMHf0d3wYK9+7caiYDNNJJ7Zr+I4 NVD45EeJrsBXz0/Ykqj5j7OLH+5mw6Jxm6fhUlxcTj0mxXsk6tJEOL0bqijtliXuTw26 Y6IL3zA8DN7OxsbETGAR2k344a3cws/uNL2ZoEtqJG8wVqidV+xuTYUaUAiG+/HDoNiE 3c8iWMENcCcwr7GdnZwRJvU4LB0GWfftxrOTOi4KfvWaG13OhXv9NAHbMmGB27KBnpPR isDI+1IKQqJCGpHsFJv4emcZ2WqlaEBGGPdkl3oR6P72vJL0+OWC+Lo+DZeAO3GK3Tmn CAbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=f/TQY/6VgWQaI+Y7zeTFIZ3sWtMUaMUFBjH7DVPQ6Fk=; b=PA6qfRLlGM4KezyjAtXqsOcnDGz78dZfVq3F6XxE/Pf6uM6JGlcBN2PcUIxznkAlBa 03xpIjLetcLB15CDM6PlDVZeZHE6tFXAASpMsPe3Y30qN5foEnDliWI7d5EMT45ttyc9 T1KdEfG1lUgBtIjBO7QhXUpEZmwgTfKjPA2+jCMTDbJYKBqFCQG9oOwxbIC7nEDEoW2m sm76f13TlRVgBb2+tdoNFCCEekutIjFoKPVaj4E/TsYXe5D+nErcmrMTNPyhNr9r7Srt ZpordJ39JYaJPxJkiyoSzYclpxFBcuHE1eoqghO8mDElcWB0V1SL6vDjFOgFNvGXEWwC mPCA==
X-Gm-Message-State: AKS2vOxsGZ12Y/joLQrvbiLAXBpeFdpQnFubU/WLUhGUV3vdUb3uOW7Z 77hYCavvCiHZWm+RcAQIHkTMg3fyBQ==
X-Received: by 10.129.154.17 with SMTP id r17mr514738ywg.254.1498107484773; Wed, 21 Jun 2017 21:58:04 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 21:58:04 -0700 (PDT)
In-Reply-To: <CAArZqeWN2suTLp5DUNsqEK5uCP7pa7-rtu7510c508jQMwYTeg@mail.gmail.com>
References: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com> <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com> <D78D2206-5A27-40E3-9E29-8C4A8D1ABD01@cisco.com> <CAArZqeWN2suTLp5DUNsqEK5uCP7pa7-rtu7510c508jQMwYTeg@mail.gmail.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 21:58:04 -0700
Message-ID: <CAArZqeXLEnN0XQBvC2L5WrLx33NLg2nH9Jqs6PbPcycb9yRqhQ@mail.gmail.com>
To: "Alvaro Retana (aretana)" <aretana@cisco.com>
Cc: Lucien Avramov <lucienav@google.com>, The IESG <iesg@ietf.org>,  "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>,  Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>,  "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c0b8eee7722810552855439"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Uc7buXwduCe_Yjp8kvoBw7XgAwM>
Subject: Re: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 04:58:07 -0000

--94eb2c0b8eee7722810552855439
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Alvaro,

I removed 1.2 per your suggestion. It's now all consistent.

Please confirm it's good for you?

Thanks for insisting on this change.

Lucien

On Tue, Jun 20, 2017 at 9:40 AM, Lucien <lucien.avramov@gmail.com> wrote:

> Thanks for clarifying Alvaro!! Much appreciate it.
>
> Got it, i'd like to keep 1.2 add-on as it makes it more clear for the
> reader to understand what I want to convey when I use this language in th=
e
> doc, instead of having them go read another RFC.
>
> On Tue, Jun 20, 2017 at 9:35 AM, Alvaro Retana (aretana) <
> aretana@cisco.com> wrote:
>
>> On 6/20/17, 3:17 AM, "Lucien Avramov" <lucienav@google.com> wrote:
>>
>>
>>
>> Lucien:
>>
>>
>>
>> Hi!
>>
>>
>>
>> > In the context of the tests we describe in this publication, it's key
>> to keep
>>
>> > the words we have chosen=E2=80=A6
>>
>>
>>
>> I don=E2=80=99t have a problem with the words themselves =E2=80=93 the p=
oint of my
>> DISCUSS is the fact that each word (=E2=80=9CMUST=E2=80=9D, for example)=
 has two different
>> definitions.  Yes, they are close, but different.  In fact, they are so
>> close that I think you could take out the definitions in 1.2 and not los=
e
>> anything.
>>
>>
>>
>> Regards,
>>
>>
>>
>> Alvaro.
>>
>
>

--94eb2c0b8eee7722810552855439
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Alvaro,<div><br></div><div>I removed 1.2 per your sugge=
stion. It&#39;s now all consistent.=C2=A0</div><div><br></div><div>Please c=
onfirm it&#39;s good for you?</div><div><br></div><div>Thanks for insisting=
 on this change.</div><div><br></div><div>Lucien</div></div><div class=3D"g=
mail_extra"><br><div class=3D"gmail_quote">On Tue, Jun 20, 2017 at 9:40 AM,=
 Lucien <span dir=3D"ltr">&lt;<a href=3D"mailto:lucien.avramov@gmail.com" t=
arget=3D"_blank">lucien.avramov@gmail.com</a>&gt;</span> wrote:<br><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc s=
olid;padding-left:1ex"><div dir=3D"ltr">Thanks for clarifying Alvaro!! Much=
 appreciate it.=C2=A0<div><br></div><div><div>Got it, i&#39;d like to keep =
1.2 add-on as it makes it more clear for the reader to understand what I wa=
nt to convey when I use this language in the doc, instead of having them go=
 read another RFC.=C2=A0</div></div></div><div class=3D"HOEnZb"><div class=
=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, J=
un 20, 2017 at 9:35 AM, Alvaro Retana (aretana) <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:aretana@cisco.com" target=3D"_blank">aretana@cisco.com</a>&gt;=
</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .=
8ex;border-left:1px #ccc solid;padding-left:1ex">







<div bgcolor=3D"white" lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"m_-1345388604861953658m_-7067659852252104260WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">On 6/20/17, 3:17=
 AM, &quot;Lucien Avramov&quot; &lt;<a href=3D"mailto:lucienav@google.com" =
target=3D"_blank">lucienav@google.com</a>&gt; wrote:<u></u><u></u></span></=
p>
<div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Lucien:<u></u><u=
></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Hi!<u></u><u></u=
></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
</div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
 In the context of the tests we describe in this publication, it&#39;s key =
to keep
<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
 the words we have chosen=E2=80=A6<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black"><u><=
/u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">I do=
n=E2=80=99t have a problem with the words themselves =E2=80=93 the point of=
 my DISCUSS is the fact that each word (=E2=80=9CMUST=E2=80=9D, for example=
) has two different definitions.=C2=A0 Yes, they are close, but different.=
=C2=A0
 In fact, they are so close that I think you could take out the definitions=
 in 1.2 and not lose anything.
<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black"><u><=
/u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">Rega=
rds,<span class=3D"m_-1345388604861953658HOEnZb"><font color=3D"#888888"><u=
></u><u></u></font></span></span></p><span class=3D"m_-1345388604861953658H=
OEnZb"><font color=3D"#888888">
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black"><u><=
/u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">Alva=
ro.</span><span style=3D"font-family:Calibri"><u></u><u></u></span></p>
</font></span></div>
</div>

</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--94eb2c0b8eee7722810552855439--


From nobody Wed Jun 21 21:59:51 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 098CA127B52; Wed, 21 Jun 2017 21:59:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0Nhs-a7thYJO; Wed, 21 Jun 2017 21:59:48 -0700 (PDT)
Received: from mail-yb0-x22a.google.com (mail-yb0-x22a.google.com [IPv6:2607:f8b0:4002:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D2DE6127A90; Wed, 21 Jun 2017 21:59:47 -0700 (PDT)
Received: by mail-yb0-x22a.google.com with SMTP id e201so1442343ybb.1; Wed, 21 Jun 2017 21:59:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=igszqE3866gVFha/w3GT63XLlyxWUpVgYdFuVTPuKtg=; b=Ptmk90Xg5/xJyu4n5/WjTV8TzGjZJ/IA7rERYoTnh/sdpVLITjq9cW+p3Tbxzg/zmt +1ioZC+Et0A3XFN08fiTDzVtg+98/00mEXNISJe9uTjZZbiokllPh4JGpWtp6tIuruYO xnC0qBqz0hKDktpoWtnziNyAfdC94I4Bh519NPWk+85WSzEnJoBuF1oE55LTQUxsNhk9 +zsFxXh/nz1TMH9hv5+aMHT7/zAYMn9t6oaoUBI2gknQ8dJiFUdmf6QQTcGN2ua3CLAK HW1lz+5ncDrc5/MPZBuNqVrxlz0K4EO08wRzafx/qCm65lharUGNv0YkJFaNf1RZcJfJ 2O4A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=igszqE3866gVFha/w3GT63XLlyxWUpVgYdFuVTPuKtg=; b=QV7i+7yg3tmwAlEcQhDaMsj6pfEa5kofR5fotx0xHcqB5ChEfOFWpPASLcOe98opby pWX2Gc5Bv0PsacTNgsHBicnjcrqhB/+movKSzN9n4UPdyf+cRPTHkjUFy351DVvcmi1x isM53SvlkdCAHsP/DImzdDAqEtxDrexPUCcyYaCPvrGTjvmgJbhHtUHG+Rht/jCtjdX5 FKpV+ySrPkaaww5wxRQn8uW9xbdp10kaFkoGLHU6lmjEvAgJqI7Sm7WIhM/oX2t184yD Sp2MnaiOm9+4Mx8nug9e1VHK6YKnkAJldKVu61GP+ZZTeHffnU8JNB951MuWsHvt+t1T qkZg==
X-Gm-Message-State: AKS2vOzivJkvgEwjfAWga1kKbrs5FQTPtNt6YUKFKgBqgZ0tdI/vmsQ+ z66z6tN0kgBWwl1HQkHSgrPD+XqiDlwY
X-Received: by 10.37.38.17 with SMTP id m17mr341404ybm.191.1498107586995; Wed, 21 Jun 2017 21:59:46 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 21:59:46 -0700 (PDT)
In-Reply-To: <CA7A6183-2EEA-475F-B33B-9B77F075D5FD@nostrum.com>
References: <149807196602.15838.8048520689148705215.idtracker@ietfa.amsl.com> <CAArZqeX2_1XJhuqhMdHYRELrERgMzu+AV9zQAFLQUdpMbh7PcA@mail.gmail.com> <CA7A6183-2EEA-475F-B33B-9B77F075D5FD@nostrum.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 21:59:46 -0700
Message-ID: <CAArZqeUL7Zoc72SR5CRDxm005Ys=d3xSnAHLPYPHX6N1wJMcww@mail.gmail.com>
To: Ben Campbell <ben@nostrum.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c18fbda8eedd00552855a05"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/PMR43rZ-hQFpwddu1ZXTIOkBahg>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 04:59:50 -0000

--94eb2c18fbda8eedd00552855a05
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Ben!

Please see inline!

Also fixed the Alvaro comment as well.

Please let us know if you are okay with the change approve?

Thanks!
Lucien

On Wed, Jun 21, 2017 at 12:36 PM, Ben Campbell <ben@nostrum.com> wrote:

> Thanks for the response. Comments inline:
>
> Ben.
>
> > On Jun 21, 2017, at 2:17 PM, Lucien <lucien.avramov@gmail.com> wrote:
> >
> > Hi Ben,
> >
> > Thanks for the comment! Please find my answers inline, and please
> acknowledge back!
> >
> > Appreciate you taking the time to look at our work.
> >
> > Lucien
> >
> > On Wed, Jun 21, 2017 at 12:06 PM, Ben Campbell <ben@nostrum.com> wrote:
> > Ben Campbell has entered the following ballot position for
> > draft-ietf-bmwg-dcbench-terminology-17: No Objection
> >
> > When responding, please keep the subject line intact and reply to all
> > email addresses included in the To and CC lines. (Feel free to cut this
> > introductory paragraph, however.)
> >
> >
> > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.
> html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
> >
> >
> >
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> >
> > I find the naming of the draft fairly confusing. It goes way beyond
> > "terminology"; it makes a number of normative (using 2119 language)
> statements
> > about benchmarking procedures. I wonder why the sections about procedur=
e
> did
> > not go into the methodology draft instead. In general, I don't think
> putting
> > normative language in an informational terminology draft is a good idea=
.
> (This
> > would have been a DISCUSS, except that I am aware the bmwg has decided
> to make
> > all its drafts informational and to still use 2119 language. For the
> record, I
> > think that policy falls down with this draft.)
> >
> > That's how we have decided it makes sense at BMWG to proceed with these
> two drafts. We have been hashing this out for 4+ years now and the curren=
t
> state is the consensus.  As author, I have no intention nor desire to
> change this
>
> For the record, I think BMWG has adopted interpretations of both
> informational RFC and 2119 keywords that are at best unconventional, and =
at
> worst tortured. But I also recognize the precedent has been set, and don=
=E2=80=99t
> mean to hold this draft hostage to it. Thus it was a non-blocking comment
> which you can feel free to ignore :-)
>
>
> >
> > I agree with the comment from others that this does not seem to be
> specific to
> > datacenters.
> >
> > Great, so did we, this is why we already worked on addressing this by:
> >       =E2=80=A2  calling out specifically that this specifically applie=
s to data
> center switches (defining what those are today)
> >       =E2=80=A2 stating clearly that it can be applied to switches out =
of the
> data center, but that's not the specific scope of this
>
> Works for me, thanks.
>
>
> >
> > - 2.2: Definitions of "store-and-forward" and "cut-through" when used i=
n
> this
> > context would be helpful. The first may be obvious, but the best I can
> do with
> > "cut-through" is assume it means the opposite of "store-and-forward".
> >
> > Those are cleary defined in RFC 1242, which we reference. We don't want
> to duplicate definitions here.
>
> Apologies, I missed the earlier citation to 1242, which was specifically
> in regard to these terms.
>
> >
> >
> >
> > - 6.2: After reading the definition of "Incast" several times, I'm stil=
l
> not
> > sure what it means or what is being measured.
> >
> > It's many to one type of network traffic patterns. Its very commonly
> found in cloud data centers implementing distributed storage/compute
> frameworks (big data for example).
> >
>
> As written, it=E2=80=99s not clear to me if =E2=80=9CIncast=E2=80=9D mean=
s =E2=80=9Cone to
> many/many-to-many=E2=80=9D communication in general, or the patterns of
> synchronization that result from that. The first paragraph seems to say t=
he
> former but the second paragraph seems to say the latter.
>
>
Clarified the paragraph and re-wrote it, take a look. incast is either
many-to-one (simpler scenario), or many-to-many.

--94eb2c18fbda8eedd00552855a05
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Ben!<div><br></div><div>Please see inline!</div><div><b=
r></div><div>Also fixed the Alvaro comment as well.</div><div><br></div><di=
v>Please let us know if you are okay with the change approve?</div><div><br=
></div><div>Thanks!</div><div>Lucien</div><div class=3D"gmail_extra"><br><d=
iv class=3D"gmail_quote">On Wed, Jun 21, 2017 at 12:36 PM, Ben Campbell <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:ben@nostrum.com" target=3D"_blank">ben=
@nostrum.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" sty=
le=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks=
 for the response. Comments inline:<br>
<br>
Ben.<br>
<div><div class=3D"h5"><br>
&gt; On Jun 21, 2017, at 2:17 PM, Lucien &lt;<a href=3D"mailto:lucien.avram=
ov@gmail.com">lucien.avramov@gmail.com</a>&gt; wrote:<br>
&gt;<br>
&gt; Hi Ben,<br>
&gt;<br>
&gt; Thanks for the comment! Please find my answers inline, and please ackn=
owledge back!<br>
&gt;<br>
&gt; Appreciate you taking the time to look at our work.<br>
&gt;<br>
&gt; Lucien<br>
&gt;<br>
&gt; On Wed, Jun 21, 2017 at 12:06 PM, Ben Campbell &lt;<a href=3D"mailto:b=
en@nostrum.com">ben@nostrum.com</a>&gt; wrote:<br>
&gt; Ben Campbell has entered the following ballot position for<br>
&gt; draft-ietf-bmwg-dcbench-<wbr>terminology-17: No Objection<br>
&gt;<br>
&gt; When responding, please keep the subject line intact and reply to all<=
br>
&gt; email addresses included in the To and CC lines. (Feel free to cut thi=
s<br>
&gt; introductory paragraph, however.)<br>
&gt;<br>
&gt;<br>
&gt; Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss=
-criteria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/i=
esg/<wbr>statement/discuss-criteria.<wbr>html</a><br>
&gt; for more information about IESG DISCUSS and COMMENT positions.<br>
&gt;<br>
&gt;<br>
&gt; The document, along with other ballot positions, can be found here:<br=
>
&gt; <a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-te=
rminology/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.o=
rg/<wbr>doc/draft-ietf-bmwg-dcbench-<wbr>terminology/</a><br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt; COMMENT:<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt;<br>
&gt; I find the naming of the draft fairly confusing. It goes way beyond<br=
>
&gt; &quot;terminology&quot;; it makes a number of normative (using 2119 la=
nguage) statements<br>
&gt; about benchmarking procedures. I wonder why the sections about procedu=
re did<br>
&gt; not go into the methodology draft instead. In general, I don&#39;t thi=
nk putting<br>
&gt; normative language in an informational terminology draft is a good ide=
a. (This<br>
&gt; would have been a DISCUSS, except that I am aware the bmwg has decided=
 to make<br>
&gt; all its drafts informational and to still use 2119 language. For the r=
ecord, I<br>
&gt; think that policy falls down with this draft.)<br>
&gt;<br>
&gt; That&#39;s how we have decided it makes sense at BMWG to proceed with =
these two drafts. We have been hashing this out for 4+ years now and the cu=
rrent state is the consensus.=C2=A0 As author, I have no intention nor desi=
re to change this<br>
<br>
</div></div>For the record, I think BMWG has adopted interpretations of bot=
h informational RFC and 2119 keywords that are at best unconventional, and =
at worst tortured. But I also recognize the precedent has been set, and don=
=E2=80=99t mean to hold this draft hostage to it. Thus it was a non-blockin=
g comment which you can feel free to ignore :-)<br>
<span class=3D""><br>
<br>
&gt;<br>
&gt; I agree with the comment from others that this does not seem to be spe=
cific to<br>
&gt; datacenters.<br>
&gt;<br>
&gt; Great, so did we, this is why we already worked on addressing this by:=
<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2=C2=A0 calling out specifically tha=
t this specifically applies to data center switches (defining what those ar=
e today)<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2 stating clearly that it can be app=
lied to switches out of the data center, but that&#39;s not the specific sc=
ope of this<br>
<br>
</span>Works for me, thanks.<br>
<span class=3D""><br>
<br>
&gt;<br>
&gt; - 2.2: Definitions of &quot;store-and-forward&quot; and &quot;cut-thro=
ugh&quot; when used in this<br>
&gt; context would be helpful. The first may be obvious, but the best I can=
 do with<br>
&gt; &quot;cut-through&quot; is assume it means the opposite of &quot;store=
-and-forward&quot;.<br>
&gt;<br>
&gt; Those are cleary defined in RFC 1242, which we reference. We don&#39;t=
 want to duplicate definitions here.<br>
<br>
</span>Apologies, I missed the earlier citation to 1242, which was specific=
ally in regard to these terms.<br>
<span class=3D""><br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; - 6.2: After reading the definition of &quot;Incast&quot; several time=
s, I&#39;m still not<br>
&gt; sure what it means or what is being measured.<br>
&gt;<br>
&gt; It&#39;s many to one type of network traffic patterns. Its very common=
ly found in cloud data centers implementing distributed storage/compute fra=
meworks (big data for example).<br>
&gt;<br>
<br>
</span>As written, it=E2=80=99s not clear to me if =E2=80=9CIncast=E2=80=9D=
 means =E2=80=9Cone to many/many-to-many=E2=80=9D communication in general,=
 or the patterns of synchronization that result from that. The first paragr=
aph seems to say the former but the second paragraph seems to say the latte=
r.<br>
<br></blockquote><div><br></div><div>Clarified the paragraph and re-wrote i=
t, take a look. incast is either many-to-one (simpler scenario), or many-to=
-many. =C2=A0</div></div><br></div></div>

--94eb2c18fbda8eedd00552855a05--


From nobody Wed Jun 21 22:00:55 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B5A15127A90; Wed, 21 Jun 2017 22:00:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4OSluqPb56np; Wed, 21 Jun 2017 22:00:51 -0700 (PDT)
Received: from mail-yw0-x22e.google.com (mail-yw0-x22e.google.com [IPv6:2607:f8b0:4002:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A5B5F1200CF; Wed, 21 Jun 2017 22:00:51 -0700 (PDT)
Received: by mail-yw0-x22e.google.com with SMTP id v7so1893690ywc.2; Wed, 21 Jun 2017 22:00:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uxV0PduxBn9rHFqieHImhmynPaTLyP6oKCkB23UTpuQ=; b=D9D3czyEKcCFTxZFUbVhTiGKy+XSSmvkVXl12ywv1ZMaRdqPho0s2YiwvlmoSyMj6P 0SK91FEiJNvQ+/2IbWCQVr9ooVCVc03MAiMm6dhx3Un9yLfbApoBIYx91WUkfzm/gDt5 10lONPnZNdAJ3AwOZ5VrJGZyRoOYmOO4XvvF//vE5K5GXYcd93ndXR1dSNXY2NewqUyN OEh6qpZDWb4g6GY4nfFgtB1ZXvh4MCDhTR46NkVaCv+oHvdInZVK4JVKL+kVjtvWYYzn v7ANeVnUn2s34Wvh5ydn71x4TKfN4SI5jbIH3zOcxh04ZOFpTtQthAn7HFtZmu3hnLGi Wq/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uxV0PduxBn9rHFqieHImhmynPaTLyP6oKCkB23UTpuQ=; b=SKNIqQMuL/zh7yG6qVWpkJyyp4fFysGhKTqCd/KppBC+S9LLhaju2pjSjHfaAaaWah uT9zaBLVtooYtH17npcGUfL9wvSPv+cAW3/wpMnACRwSjqrEyTG4lRqMdqjOD7/2f2qn OWSi7K8JlFK9qSzgZt0ptlH3nD+Urm/kjceB1RGtdLDpgfLxlIFmrDte7KGKOw9zQ2Rw t+lgt0caMreKkC9s8jpiFQYAk55WgG5ZwoJzA78IREKvu+VFIw+V/tjb3R/3LwAFzsNj BsEt0M30IjREKvTfBaJ/5oijgU11y7ZKAm0vsvcvUY48+BCny98XrLdRyY6gJb2zuHpM btyA==
X-Gm-Message-State: AKS2vOx74cPt11M+F4WNTZLEyhem4YL2WFQrymPN7ZSGHQi9iR3fPYRK ksxKgXMmcca1kRvb4M7y4ZVmI19vjQ==
X-Received: by 10.129.161.15 with SMTP id y15mr489054ywg.116.1498107649412; Wed, 21 Jun 2017 22:00:49 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 22:00:49 -0700 (PDT)
In-Reply-To: <9D1D7116-BBE9-4BCF-A82C-6D6752C373AA@nostrum.com>
References: <149807212936.15488.7309772374304994572.idtracker@ietfa.amsl.com> <CAArZqeV_LQ_SSvQif652aZfPrgF2sjwPKxn4uD8vC-bTc40Z0g@mail.gmail.com> <9D1D7116-BBE9-4BCF-A82C-6D6752C373AA@nostrum.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 22:00:49 -0700
Message-ID: <CAArZqeVYYuGiSXe2Zt3a4mA=RWOPgPOHXh+fxRvTiHSLMsu+Hg@mail.gmail.com>
To: Ben Campbell <ben@nostrum.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a114f087a4756070552855e95"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/kr8LZWkmdbKKUjwqbk8b7YWGxSY>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 05:00:54 -0000

--001a114f087a4756070552855e95
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Ben!

Just for the archive, answering on this thread also, please see inline and
ackowledge back:

Many thanks for the time spent on our drafts.

Lucien

On Wed, Jun 21, 2017 at 12:52 PM, Ben Campbell <ben@nostrum.com> wrote:

> Thanks for the response; comments inline:
>
>
> > On Jun 21, 2017, at 2:22 PM, Lucien <lucien.avramov@gmail.com> wrote:
> >
> > Hi Ben!
> >
> > Please find inline my comments!
> >
> > Thanks for reviewing this draft!
> >
> > On Wed, Jun 21, 2017 at 12:08 PM, Ben Campbell <ben@nostrum.com> wrote:
> > Ben Campbell has entered the following ballot position for
> > draft-ietf-bmwg-dcbench-methodology-16: No Objection
> >
> > When responding, please keep the subject line intact and reply to all
> > email addresses included in the To and CC lines. (Feel free to cut this
> > introductory paragraph, however.)
> >
> >
> > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.
> html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
> >
> >
> >
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> >
> > - I support Alvaro's DISCUSS
> >
> > As author, the meaning of the words chosen (MUST..) are important to me
> and I don't agree with make it more loose.
>
> The issue is that the text (even in 16) has non-matching definitions for
> MUST, SHOULD, MAY, etc. It=E2=80=99s not a matter of =E2=80=9Clooseness=
=E2=80=9D. The 2119
> definitions are about interoperability. The definitions in 1.2 do not see=
m
> to be about interoperability.
> >
>

Fixed by removing this section 1.2 part.


> >
> > - I agree with the questions about why this is specific to datacenters.
> >
> > Since this comment was on your other draft review, same answer applies
> here:
> >
> > Great, so did we, this is why we already worked on addressing this by:
> >       =E2=80=A2  calling out specifically that this specifically applie=
s to data
> center switches (defining what those are today)
> >       =E2=80=A2 stating clearly that it can be applied to switches out =
of the
> data center, but that's not the specific scope of this
> >
> >
>
> WFM
>
> >
> > - Please expand DUT on first use.
> >
> > Thanks, this was expanded already on first use in the introduction of
> -16 document (the Device Under Test (DUT)). I just doubled checked. Pleas=
e
> let me know if I have missed anything, and I will fix it right away.
>
> Ah, I reviewed version 14, but it progressed to version 16 before I wrote
> up my notes and entered them into the tracker. I didn=E2=80=99t notice th=
e new
> revisions until you mentioned them here. I agree DUT is properly expanded
> in 16.
> \

--001a114f087a4756070552855e95
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Ben!<div><br></div><div>Just for the archive, answering=
 on this thread also, please see inline and ackowledge back:</div><div><br>=
</div><div>Many thanks for the time spent on our drafts.</div><div><br></di=
v><div>Lucien</div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote=
">On Wed, Jun 21, 2017 at 12:52 PM, Ben Campbell <span dir=3D"ltr">&lt;<a h=
ref=3D"mailto:ben@nostrum.com" target=3D"_blank">ben@nostrum.com</a>&gt;</s=
pan> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex=
;border-left:1px #ccc solid;padding-left:1ex">Thanks for the response; comm=
ents inline:<br>
<span class=3D""><br>
<br>
&gt; On Jun 21, 2017, at 2:22 PM, Lucien &lt;<a href=3D"mailto:lucien.avram=
ov@gmail.com">lucien.avramov@gmail.com</a>&gt; wrote:<br>
&gt;<br>
&gt; Hi Ben!<br>
&gt;<br>
&gt; Please find inline my comments!<br>
&gt;<br>
&gt; Thanks for reviewing this draft!<br>
&gt;<br>
&gt; On Wed, Jun 21, 2017 at 12:08 PM, Ben Campbell &lt;<a href=3D"mailto:b=
en@nostrum.com">ben@nostrum.com</a>&gt; wrote:<br>
&gt; Ben Campbell has entered the following ballot position for<br>
&gt; draft-ietf-bmwg-dcbench-<wbr>methodology-16: No Objection<br>
&gt;<br>
&gt; When responding, please keep the subject line intact and reply to all<=
br>
&gt; email addresses included in the To and CC lines. (Feel free to cut thi=
s<br>
&gt; introductory paragraph, however.)<br>
&gt;<br>
&gt;<br>
&gt; Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss=
-criteria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/i=
esg/<wbr>statement/discuss-criteria.<wbr>html</a><br>
&gt; for more information about IESG DISCUSS and COMMENT positions.<br>
&gt;<br>
&gt;<br>
&gt; The document, along with other ballot positions, can be found here:<br=
>
&gt; <a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-me=
thodology/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.o=
rg/<wbr>doc/draft-ietf-bmwg-dcbench-<wbr>methodology/</a><br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt; COMMENT:<br>
&gt; ------------------------------<wbr>------------------------------<wbr>=
----------<br>
&gt;<br>
&gt; - I support Alvaro&#39;s DISCUSS<br>
&gt;<br>
&gt; As author, the meaning of the words chosen (MUST..) are important to m=
e and I don&#39;t agree with make it more loose.<br>
<br>
</span>The issue is that the text (even in 16) has non-matching definitions=
 for MUST, SHOULD, MAY, etc. It=E2=80=99s not a matter of =E2=80=9Cloosenes=
s=E2=80=9D. The 2119 definitions are about interoperability. The definition=
s in 1.2 do not seem to be about interoperability.<br>
<span class=3D"">&gt;<br></span></blockquote><div><br></div><div>Fixed by r=
emoving this section 1.2 part.=C2=A0</div><div>=C2=A0</div><blockquote clas=
s=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pad=
ding-left:1ex"><span class=3D"">
&gt;<br>
&gt; - I agree with the questions about why this is specific to datacenters=
.<br>
&gt;<br>
&gt; Since this comment was on your other draft review, same answer applies=
 here:<br>
&gt;<br>
&gt; Great, so did we, this is why we already worked on addressing this by:=
<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2=C2=A0 calling out specifically tha=
t this specifically applies to data center switches (defining what those ar=
e today)<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=A2 stating clearly that it can be app=
lied to switches out of the data center, but that&#39;s not the specific sc=
ope of this<br>
&gt;<br>
&gt;<br>
<br>
</span>WFM<br>
<span class=3D""><br>
&gt;<br>
&gt; - Please expand DUT on first use.<br>
&gt;<br>
&gt; Thanks, this was expanded already on first use in the introduction of =
-16 document (the Device Under Test (DUT)). I just doubled checked. Please =
let me know if I have missed anything, and I will fix it right away.<br>
<br>
</span>Ah, I reviewed version 14, but it progressed to version 16 before I =
wrote up my notes and entered them into the tracker. I didn=E2=80=99t notic=
e the new revisions until you mentioned them here. I agree DUT is properly =
expanded in 16.<br>
\</blockquote></div><br></div></div>

--001a114f087a4756070552855e95--


From nobody Wed Jun 21 22:02:22 2017
Return-Path: <jrapp@vmware.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 04C1C1200CF; Wed, 21 Jun 2017 22:02:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.922
X-Spam-Level: 
X-Spam-Status: No, score=-1.922 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=onevmw.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DdO7EKajLfqC; Wed, 21 Jun 2017 22:02:18 -0700 (PDT)
Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0067.outbound.protection.outlook.com [104.47.37.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3D425127A90; Wed, 21 Jun 2017 22:02:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=s134+dy4SP38uOVx209t+gUG6UFhh5r1e524by0hTQc=; b=QRPLzVwPJigju1aJvv9gtCzCrriUbDsgXwrTb55BMfNI/TIbAebkh1TbroujiAdFflJxs9cezDOckTmJWQf0G5cjwOcIoj4d1u3Kn7c0vrKrUjTPgWNq5seJ/wT8x/PFe7bqUAW9sa1fyVBMTqGRI8GHrcNJaxuNSxEgUR0k+Fo=
Received: from CY1PR0501MB1673.namprd05.prod.outlook.com (10.163.138.158) by CY1PR0501MB1130.namprd05.prod.outlook.com (10.160.144.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6; Thu, 22 Jun 2017 05:02:16 +0000
Received: from CY1PR0501MB1673.namprd05.prod.outlook.com ([10.163.138.158]) by CY1PR0501MB1673.namprd05.prod.outlook.com ([10.163.138.158]) with mapi id 15.01.1199.016; Thu, 22 Jun 2017 05:02:16 +0000
From: Jacob Rapp <jrapp@vmware.com>
To: Adam Roach <adam@nostrum.com>
CC: The IESG <iesg@ietf.org>, "draft-ietf-bmwg-dcbench-terminology@ietf.org" <draft-ietf-bmwg-dcbench-terminology@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
Thread-Index: AQHS6vo3hJZEKl8nkESh2AD+ujkPHaIwUtCA
Date: Thu, 22 Jun 2017 05:02:15 +0000
Message-ID: <88C9C25C-E2E8-4466-98A0-341CD30243B7@vmware.com>
References: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com>
In-Reply-To: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=vmware.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-originating-ip: [108.198.96.247]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; CY1PR0501MB1130; 20:RsNAMMKmiwfaGNkZ1uWxUfNESWuxybBSEPYeFRineSYKKUcofWuMdPmVH0iBGaqu7Ij/thG+FrNguLcd69ort/FeSfSydCsyInT9t00I+XXwLs0C8GGR0HmIkSMWQ+FlE1GifHFdWx0V7GkSQDaFhykozrxoCwRA81jySrTG/6Q=
x-ms-office365-filtering-correlation-id: f03b25a8-6c8d-4268-85ec-08d4b92bda1f
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(201703131423075)(201703031133081); SRVR:CY1PR0501MB1130; 
x-ms-traffictypediagnostic: CY1PR0501MB1130:
x-microsoft-antispam-prvs: <CY1PR0501MB1130CDAC74155857F0D63094D6DB0@CY1PR0501MB1130.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(788757137089);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(93006095)(93001095)(3002001)(6041248)(20161123558100)(20161123564025)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0501MB1130; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0501MB1130; 
x-forefront-prvs: 03468CBA43
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(39860400002)(39410400002)(39400400002)(39450400003)(39850400002)(39840400002)(377454003)(24454002)(2906002)(122556002)(14454004)(5660300001)(66066001)(966005)(33656002)(189998001)(6506006)(305945005)(6916009)(82746002)(229853002)(83716003)(53546010)(575784001)(478600001)(86362001)(5003630100001)(2950100002)(6246003)(53936002)(50986999)(3660700001)(54356999)(76176999)(4326008)(38730400002)(110136004)(6486002)(77096006)(6512007)(9886003)(99286003)(6436002)(6306002)(7736002)(54906002)(25786009)(230783001)(6116002)(102836003)(3846002)(2900100001)(36756003)(3280700002)(81166006)(8676002)(8936002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0501MB1130; H:CY1PR0501MB1673.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-ID: <56E0D20F99C78E498B9ACE34E7FB0C5A@onevmw.onmicrosoft.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: vmware.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jun 2017 05:02:15.8126 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0501MB1130
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2K_we7qQjzOGO6UU0jzg0LrK7z8>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 05:02:21 -0000

Thanks Adam for the editorial and nits.  We'll updated to address. Lucien w=
ill send out detail.

--
Jacob



Sent from my iPhone
> On Jun 21, 2017, at 6:52 PM, Adam Roach <@nostrum.com> wrote:
>=20
> Adam Roach has entered the following ballot position for
> draft-ietf-bmwg-dcbench-terminology-17: No Objection
>=20
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>=20
>=20
> Please refer to https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__ww=
w.ietf.org_iesg_statement_discuss-2Dcriteria.html&d=3DDwICaQ&c=3DuilaK90D4T=
OVoH58JNXRgQ&r=3D9ugCzbhur3I51kAqkWT0Pw&m=3DFCMkZeiWgN0iNPvQedG3FxtnLyS8QBO=
Pkt5rBP_GmKM&s=3DT07RQYIFZSuVEz038A90IqBXtFZYgAd3hzJJxYr1OaM&e=3D=20
> for more information about IESG DISCUSS and COMMENT positions.
>=20
>=20
> The document, along with other ballot positions, can be found here:
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.o=
rg_doc_draft-2Dietf-2Dbmwg-2Ddcbench-2Dterminology_&d=3DDwICaQ&c=3DuilaK90D=
4TOVoH58JNXRgQ&r=3D9ugCzbhur3I51kAqkWT0Pw&m=3DFCMkZeiWgN0iNPvQedG3FxtnLyS8Q=
BOPkt5rBP_GmKM&s=3DXlC1lB7b_LeDDPpmYoSWSeQ3ry3MTiSbVFTh48uXh18&e=3D=20
>=20
>=20
>=20
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>=20
> I am surprised to find normative statements in a terminology document. It=
 might
> be appropriate -- if awkward -- to say things like "this term MUST mean x=
," but
> this document includes statements pertaining specifically to what "MUST b=
e
> measured," which seems well beyond its purported scope. I would suggest e=
ither
> removing all such statements, or clearly expanding the scope of the docum=
ent
> (including, and quite importantly, revising its title).
>=20
> Editorial:
> - The first paragraph of Section 6.1 uses plural forms for B, kB, and MB,=
 but
> singular for GB. Please make these consistent. - Typically, data units ar=
e
> capitalized per SI-system prefix rules, which would make "kB" the correct
> abbreviation for kilobytes, rather than "KB." - Something has gone well a=
nd
> truly bonkers with the references section formatting. - Please fix refere=
nce
> [1] so that it correctly points to draft-ietf-bmwg-dcbench-methodology. T=
his
> will ensure that it is updated to the correct RFC value at publication.
>=20
> Please expand the following acronyms upon first use;
> see https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.rfc-2Dedit=
or.org_materials_abbrev.expansion.txt&d=3DDwICaQ&c=3DuilaK90D4TOVoH58JNXRgQ=
&r=3D9ugCzbhur3I51kAqkWT0Pw&m=3DFCMkZeiWgN0iNPvQedG3FxtnLyS8QBOPkt5rBP_GmKM=
&s=3DjPBkPkCSaxDZ5qrjYs77kcdBKvrXy-7ndq6o2iXm0n8&e=3D  for guidance.
>=20
> - FPGA - Field-Programmable Gate Array
> - LLDP - Link Layer Discovery Protocol
>=20
> Nits:
>=20
>  ** The abstract seems to contain references ([1]), which it shouldn't.
>     Please replace those with straight textual mentions of the documents =
in
>     question.
>=20
>  =3D=3D Missing Reference: 'RFC5481' is mentioned on line 285, but not de=
fined
>=20
>  =3D=3D Unused Reference: 'RFC5841' is defined on line 732, but no explic=
it
>     reference was found in the text
>=20
>  ** Obsolete normative reference: RFC 2554 (ref. 'RFC2544') (Obsoleted by
>     RFC 4954)
>=20
>=20


From nobody Wed Jun 21 22:05:04 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7E1B127A90; Wed, 21 Jun 2017 22:05:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ME5vxLPzuXCx; Wed, 21 Jun 2017 22:05:00 -0700 (PDT)
Received: from mail-yb0-x22f.google.com (mail-yb0-x22f.google.com [IPv6:2607:f8b0:4002:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4FC661200CF; Wed, 21 Jun 2017 22:05:00 -0700 (PDT)
Received: by mail-yb0-x22f.google.com with SMTP id 84so1471225ybe.0; Wed, 21 Jun 2017 22:05:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=/KRFVMSrL5+vriW+vfzDv0kJu0cJ1B/R77cgRpl6uQ0=; b=jZyXRzLmhgejJ5pZMDH5kWvgL85VNrHxy/Po/ZRmPEi/J7FmK91Y8qvBQMdRwILWW/ OiGb10bPAHpFt9UPkk+m7dK54a+8dM2EwG2DkftHc3vFGmUvCb6gGadPKborM3UOsP6A a7x4CiKgdT1lrqQigoGL+ZewE2A/TH8ua/V/i3Wf3uWOnc1Lx9F6DJb5BpR6sC1v34nn SgxxsrVxy1FDwChojpBCw2mLSElVx+4s69Udcw3jOCEV1f8OEgt3o7NJYRbIbsrxGKLz 7jPylhR/ceplOj6pl6tYTSA5tsRa+JOSwZvWQMmhWomOMmSLeenyPvuUBgXqQoTbr4PQ WNBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=/KRFVMSrL5+vriW+vfzDv0kJu0cJ1B/R77cgRpl6uQ0=; b=orFGfO8FVR1ijHAzqJIiN0lvSJRo1cSCWveXlW1v/Pk5kI/ikydou7z9ZMxQzIhddh 7j/N95238BvhlVHnlVHA7itOKNJnc6FRe1oMFjReZdQ19pY95xNPR1biZJXmt2jm06Kp uW1WZNighS6PMm9cynKqoukMiJ/HdNAgvqwmnXxY7r1sY5YMIutdAeggxVmk/dXlRVHp oOmLCFq0IL1TnYSQ3FwE9PvXARG5sGHjMwOC8iAskcqrGM2X4rUrokr2Q2zmIohkF8Ak e1nWKF87z2bxaQ7h/LxsP0raJUBFvpfkTFfxZSR6mgGdKjtr1il0lVyrEytmQlmrcCiN cf0w==
X-Gm-Message-State: AKS2vOzkGyidd7tjqdWO/O5pCiF3+4+B5fOYxmSYxVOnGieAEsc9fAwj 3tZIgZneU1eyBysowtUwljJrYHf8Sg==
X-Received: by 10.37.180.18 with SMTP id n18mr381853ybj.258.1498107899553; Wed, 21 Jun 2017 22:04:59 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 22:04:59 -0700 (PDT)
In-Reply-To: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com>
References: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 22:04:59 -0700
Message-ID: <CAArZqeUssxdEZrauVB=yn7i+8uBopmUvwyDu-jhHNOOvWqKM5w@mail.gmail.com>
To: Adam Roach <adam@nostrum.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="f403045e6aa23030290552856dd1"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/tVZZUKO5rJBMyCLnAVjjengtk5s>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 05:05:03 -0000

--f403045e6aa23030290552856dd1
Content-Type: text/plain; charset="UTF-8"

Hi Adam,

thanks for spending time to review and provide more comments to the list.
I addressed both of your emails with the latest publication i just did.

Please see inline for details:



On Wed, Jun 21, 2017 at 6:52 PM, Adam Roach <adam@nostrum.com> wrote:

> Adam Roach has entered the following ballot position for
> draft-ietf-bmwg-dcbench-terminology-17: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> I am surprised to find normative statements in a terminology document. It
> might
> be appropriate -- if awkward -- to say things like "this term MUST mean
> x," but
> this document includes statements pertaining specifically to what "MUST be
> measured," which seems well beyond its purported scope. I would suggest
> either
> removing all such statements, or clearly expanding the scope of the
> document
> (including, and quite importantly, revising its title).
>
> Editorial:
> - The first paragraph of Section 6.1 uses plural forms for B, kB, and MB,
> but
> singular for GB. Please make these consistent. - Typically, data units are
>

many thanks, made it consistent with singular as per the prefix rules


> capitalized per SI-system prefix rules, which would make "kB" the correct
> abbreviation for kilobytes, rather than "KB." - Something has gone well and
> truly bonkers with the references section formatting. - Please fix
> reference
> [1] so that it correctly points to draft-ietf-bmwg-dcbench-methodology.
> This
> will ensure that it is updated to the correct RFC value at publication.
>
>
I will disagree a bit here. in the context of 1024 bytes, its very common
and accepted to use KB (with capital K instead of kB) This is the case in
many networking publication.

https://en.wikipedia.org/wiki/Kilobyte

"The binary representation of 1024 bytes typically uses the symbol KB
(uppercase *K*). The *B* is often omitted in informal use. For example, a
processor with 65,536 bytes of cache might be said to have "64K" of cache."

I want to keep KB, to be consistent with the other capital letters for my
publication.



> Please expand the following acronyms upon first use;
> see https://www.rfc-editor.org/materials/abbrev.expansion.txt for
> guidance.
>
>  - FPGA - Field-Programmable Gate Array
>  - LLDP - Link Layer Discovery Protocol
>
>
Thanks! Fixed!!


> Nits:
>
>   ** The abstract seems to contain references ([1]), which it shouldn't.
>      Please replace those with straight textual mentions of the documents
> in
>      question.
>

Thanks Fixed!

>
>   == Missing Reference: 'RFC5481' is mentioned on line 285, but not defined
>

Removed this RFC as we no longer reference it

>
>   == Unused Reference: 'RFC5841' is defined on line 732, but no explicit
>      reference was found in the text
>
Removed this RFC as we no longer reference it

>
>   ** Obsolete normative reference: RFC 2554 (ref. 'RFC2544') (Obsoleted by
>      RFC 4954)
>
> I am sorry we use RFC 2544, not 2554 so we are good actually, its not
obsoleted at all.

--f403045e6aa23030290552856dd1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Adam,<div><br></div><div>thanks for spending time to re=
view and provide more comments to the list.</div><div>I addressed both of y=
our emails with the latest publication i just did.=C2=A0</div><div><br></di=
v><div>Please see inline for details:</div><div><br></div><div><br></div><d=
iv class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Jun 21, 201=
7 at 6:52 PM, Adam Roach <span dir=3D"ltr">&lt;<a href=3D"mailto:adam@nostr=
um.com" target=3D"_blank">adam@nostrum.com</a>&gt;</span> wrote:<br><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-wi=
dth:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-=
left:1ex">Adam Roach has entered the following ballot position for<br>
draft-ietf-bmwg-dcbench-<wbr>terminology-17: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-termino=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>terminology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
I am surprised to find normative statements in a terminology document. It m=
ight<br>
be appropriate -- if awkward -- to say things like &quot;this term MUST mea=
n x,&quot; but<br>
this document includes statements pertaining specifically to what &quot;MUS=
T be<br>
measured,&quot; which seems well beyond its purported scope. I would sugges=
t either<br>
removing all such statements, or clearly expanding the scope of the documen=
t<br>
(including, and quite importantly, revising its title).<br>
<br>
Editorial:<br>
- The first paragraph of Section 6.1 uses plural forms for B, kB, and MB, b=
ut<br>
singular for GB. Please make these consistent. - Typically, data units are<=
br>
</blockquote><div><br></div><div>many thanks, made it consistent with singu=
lar as per the prefix rules</div><div>=C2=A0</div><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-lef=
t-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">capitali=
zed per SI-system prefix rules, which would make &quot;kB&quot; the correct=
<br>
abbreviation for kilobytes, rather than &quot;KB.&quot; - Something has gon=
e well and<br>
truly bonkers with the references section formatting. - Please fix referenc=
e<br>
[1] so that it correctly points to draft-ietf-bmwg-dcbench-<wbr>methodology=
. This<br>
will ensure that it is updated to the correct RFC value at publication.<br>
<br></blockquote><div><br></div><div>I will disagree a bit here. in the con=
text of 1024 bytes, its very common and accepted to use KB (with capital K =
instead of kB) This is the case in many networking publication.=C2=A0</div>=
<div><br></div><div><a href=3D"https://en.wikipedia.org/wiki/Kilobyte">http=
s://en.wikipedia.org/wiki/Kilobyte</a></div><div><br></div><div><span style=
=3D"font-family:sans-serif;font-size:14px">&quot;The binary representation =
of 1024 bytes typically uses the symbol KB (uppercase=C2=A0</span><i style=
=3D"font-family:sans-serif;font-size:14px">K</i><span style=3D"font-family:=
sans-serif;font-size:14px">). The=C2=A0</span><i style=3D"font-family:sans-=
serif;font-size:14px">B</i><span style=3D"font-family:sans-serif;font-size:=
14px">=C2=A0is often omitted in informal use. For example, a processor with=
 65,536 bytes of cache might be said to have &quot;64K&quot; of cache.&quot=
;</span><br></div><div><br></div><div><font face=3D"sans-serif"><span style=
=3D"font-size:14px">I want to keep KB, to be consistent with the other capi=
tal letters for my publication.=C2=A0</span></font></div><div>=C2=A0</div><=
div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rg=
b(204,204,204);padding-left:1ex">
Please expand the following acronyms upon first use;<br>
see <a href=3D"https://www.rfc-editor.org/materials/abbrev.expansion.txt" r=
el=3D"noreferrer" target=3D"_blank">https://www.rfc-editor.org/<wbr>materia=
ls/abbrev.expansion.txt</a> for guidance.<br>
<br>
=C2=A0- FPGA - Field-Programmable Gate Array<br>
=C2=A0- LLDP - Link Layer Discovery Protocol<br>
<br></blockquote><div><br></div><div>Thanks! Fixed!!</div><div>=C2=A0</div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);p=
adding-left:1ex">
Nits:<br>
<br>
=C2=A0 ** The abstract seems to contain references ([1]), which it shouldn&=
#39;t.<br>
=C2=A0 =C2=A0 =C2=A0Please replace those with straight textual mentions of =
the documents in<br>
=C2=A0 =C2=A0 =C2=A0question.<br></blockquote><div><br></div><div>Thanks Fi=
xed!=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rg=
b(204,204,204);padding-left:1ex">
<br>
=C2=A0 =3D=3D Missing Reference: &#39;RFC5481&#39; is mentioned on line 285=
, but not defined<br></blockquote><div><br></div><div>Removed this RFC as w=
e no longer reference it=C2=A0</div><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;=
border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
=C2=A0 =3D=3D Unused Reference: &#39;RFC5841&#39; is defined on line 732, b=
ut no explicit<br>
=C2=A0 =C2=A0 =C2=A0reference was found in the text<br></blockquote><div>Re=
moved this RFC as we no longer reference it =C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;bo=
rder-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
=C2=A0 ** Obsolete normative reference: RFC 2554 (ref. &#39;RFC2544&#39;) (=
Obsoleted by<br>
=C2=A0 =C2=A0 =C2=A0RFC 4954)<br>
<br></blockquote><div>I am sorry we use RFC 2544, not 2554 so we are good a=
ctually, its not obsoleted at all.=C2=A0</div><div>=C2=A0</div></div><br></=
div></div>

--f403045e6aa23030290552856dd1--


From nobody Wed Jun 21 22:07:08 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0251126FDC; Wed, 21 Jun 2017 22:07:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L78lJKHMYOsZ; Wed, 21 Jun 2017 22:06:59 -0700 (PDT)
Received: from mail-yw0-x234.google.com (mail-yw0-x234.google.com [IPv6:2607:f8b0:4002:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 93D80127A90; Wed, 21 Jun 2017 22:06:59 -0700 (PDT)
Received: by mail-yw0-x234.google.com with SMTP id 63so1952650ywr.0; Wed, 21 Jun 2017 22:06:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=oGjx5zpwPx6Jtel9QAPLeB796VLy2YEpX0ZJ5shvOCI=; b=kXOXmU5vk8tjo5rxcAi5Bce1QQm85ZATsL+b8ZUdxAtmUE37/fbf1WDGqSwrx1rlhS 6aGsnzkuMPZ6X5Mew9DKxAstq4kAWwLEHfEIVYWqP2I81K/KBz1LwXy5jlarINwvW+J5 8DlbvFGuwJinURv+Gt2M6veBACcAAmo58Dj7C8bNzYaKCHieBoTiZzIRy8mHsorpQHfR QqnQ6DQc2UjkFdslKyGvMspM1CvO3t54Pz6pbpL9ffM/suZXdUoKoB78Qftric3JVgiQ z3T8id46GkT7GOOvWqquUxMPw16YClSEGgCMEJp9sn7i9sDQr5JBnUMs0e4Ka97/v4eM Fn6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=oGjx5zpwPx6Jtel9QAPLeB796VLy2YEpX0ZJ5shvOCI=; b=M5mjr2acAHyUd1YkL2vWD/HwN6h7CKk3LrE+1LzTWL2arUhRglQ/DzEPpgDuy5JAKC fMUgsfMV4VI2W5fGrhMKIqE85jAivJcpZ51EywhFLcXjXLwDoUkUdxnn0UoTCKkI4KH+ VmeMjsgRdp28D7jYrX+RGnDL+lO4JFJxWwOPHh27TEU6z4GZlHVpUbFk6ZR/Z2wP4qP+ NZSgw1GsOVDxqq95xC2omQjtNPB91iEtnFs0DXZecICLSHvmSpdNqQnqSiE/ZURA84E+ Uqdtosdi4l6SyGxNIwrlRToIpwbIJ4M7hnDRhF85BzrK6l424Q5mUgLRFIK5XGxdkdoh 32JA==
X-Gm-Message-State: AKS2vOwznsCYsKoemwWrTklONKbAhOMECuNMnNH2maXeTbaDWQIicu/y ueHJkFDKCMvpYiCdkItS0gs0daAbyQ==
X-Received: by 10.129.154.17 with SMTP id r17mr535289ywg.254.1498108018878; Wed, 21 Jun 2017 22:06:58 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 22:06:58 -0700 (PDT)
In-Reply-To: <149810048336.30434.8858369887731037073.idtracker@ietfa.amsl.com>
References: <149810048336.30434.8858369887731037073.idtracker@ietfa.amsl.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 22:06:58 -0700
Message-ID: <CAArZqeXGUHiKsxs21s96ktEJhAAetZUBy5Qem+a3MyQ7x6jPpg@mail.gmail.com>
To: Adam Roach <adam@nostrum.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c0b8eee4cf3e305528574c3"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/aYm3aTEE0zDxD1dP1lEr_AdkZbU>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 05:07:02 -0000

--94eb2c0b8eee4cf3e305528574c3
Content-Type: text/plain; charset="UTF-8"

Hi Adam,

Thanks for the review!! Fixed that one too with the last version :-)

Please see inline:

On Wed, Jun 21, 2017 at 8:01 PM, Adam Roach <adam@nostrum.com> wrote:

> Adam Roach has entered the following ballot position for
> draft-ietf-bmwg-dcbench-methodology-16: No Objection
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
>
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> I agree with Alvaro's discuss -- you can't cite 2119 and then override it.
> It
> may well be that what 2119 does not make sense for this document; if so,
> don't
> cite it, and be clear up front that you're using a *different* set of
> meanings
> for these specific terms.
>

Thanks I fixed it, by taking care of Alvaro's discuss and i removed 1.2.


>
> I *think* I can suss out the nature of "east-west" versus "north-south" in
> the
> introduction, but I'm really not sure. Can you please define these terms or
> point to a document that does so?
>

Thank you, I had documented in the other document but omitted this one,
it's fixed now!

>
> Editorial:
>
> - Something has gone well and truly bonkers with the references section
> formatting.
>
> - Please fix reference [1] so that it correctly points to
> draft-ietf-bmwg-dcbench-terminology. This will ensure that it is updated
> to the
> correct RFC value at publication.
>

fixed!

>
> Nits:
>
> Please expand "SUT" on first use.
>

Fixed!

>
>   ** The abstract seems to contain references ([1]), which it shouldn't.
>      Please replace those with straight textual mentions of the documents
> in
>      question.
>

Fixed!

--94eb2c0b8eee4cf3e305528574c3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Adam,<div><br></div><div>Thanks for the review!! Fixed =
that one too with the last version :-)</div><div><br></div><div>Please see =
inline:</div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On W=
ed, Jun 21, 2017 at 8:01 PM, Adam Roach <span dir=3D"ltr">&lt;<a href=3D"ma=
ilto:adam@nostrum.com" target=3D"_blank">adam@nostrum.com</a>&gt;</span> wr=
ote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border=
-left:1px #ccc solid;padding-left:1ex">Adam Roach has entered the following=
 ballot position for<br>
draft-ietf-bmwg-dcbench-<wbr>methodology-16: No Objection<br>
<br>
When responding, please keep the subject line intact and reply to all<br>
email addresses included in the To and CC lines. (Feel free to cut this<br>
introductory paragraph, however.)<br>
<br>
<br>
Please refer to <a href=3D"https://www.ietf.org/iesg/statement/discuss-crit=
eria.html" rel=3D"noreferrer" target=3D"_blank">https://www.ietf.org/iesg/<=
wbr>statement/discuss-criteria.<wbr>html</a><br>
for more information about IESG DISCUSS and COMMENT positions.<br>
<br>
<br>
The document, along with other ballot positions, can be found here:<br>
<a href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodo=
logy/" rel=3D"noreferrer" target=3D"_blank">https://datatracker.ietf.org/<w=
br>doc/draft-ietf-bmwg-dcbench-<wbr>methodology/</a><br>
<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
COMMENT:<br>
------------------------------<wbr>------------------------------<wbr>-----=
-----<br>
<br>
I agree with Alvaro&#39;s discuss -- you can&#39;t cite 2119 and then overr=
ide it. It<br>
may well be that what 2119 does not make sense for this document; if so, do=
n&#39;t<br>
cite it, and be clear up front that you&#39;re using a *different* set of m=
eanings<br>
for these specific terms.<br></blockquote><div><br></div><div>Thanks I fixe=
d it, by taking care of Alvaro&#39;s discuss and i removed 1.2.</div><div>=
=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bo=
rder-left:1px #ccc solid;padding-left:1ex">
<br>
I *think* I can suss out the nature of &quot;east-west&quot; versus &quot;n=
orth-south&quot; in the<br>
introduction, but I&#39;m really not sure. Can you please define these term=
s or<br>
point to a document that does so?<br></blockquote><div><br></div><div>Thank=
 you, I had documented in the other document but omitted this one, it&#39;s=
 fixed now!=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 =
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Editorial:<br>
<br>
- Something has gone well and truly bonkers with the references section<br>
formatting.<br>
<br>
- Please fix reference [1] so that it correctly points to<br>
draft-ietf-bmwg-dcbench-<wbr>terminology. This will ensure that it is updat=
ed to the<br>
correct RFC value at publication.<br></blockquote><div><br></div><div>fixed=
!=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;b=
order-left:1px #ccc solid;padding-left:1ex">
<br>
Nits:<br>
<br>
Please expand &quot;SUT&quot; on first use.<br></blockquote><div><br></div>=
<div>Fixed!=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 =
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
=C2=A0 ** The abstract seems to contain references ([1]), which it shouldn&=
#39;t.<br>
=C2=A0 =C2=A0 =C2=A0Please replace those with straight textual mentions of =
the documents in<br>
=C2=A0 =C2=A0 =C2=A0question.<br></blockquote><div><br></div><div>Fixed!=C2=
=A0</div></div><br></div></div>

--94eb2c0b8eee4cf3e305528574c3--


From nobody Wed Jun 21 22:41:37 2017
Return-Path: <rukeshd@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 073E21242F5 for <bmwg@ietfa.amsl.com>; Wed, 21 Jun 2017 22:41:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.802
X-Spam-Level: 
X-Spam-Status: No, score=-4.802 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iDCd_0-FzamH for <bmwg@ietfa.amsl.com>; Wed, 21 Jun 2017 22:41:32 -0700 (PDT)
Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0135.outbound.protection.outlook.com [104.47.38.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 739301200CF for <bmwg@ietf.org>; Wed, 21 Jun 2017 22:41:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=JZJdTd+/GYnFdKCNbcL19rHq7SFHvsoP98qIwH/WK98=; b=EAr8ZjyU6zLu2EFo6fK1pseV6GUT5ks20pEn79Eheod+XDJ59RSWvtXN8i6dUxlyfjpvvM7nM3VQXE2GpRgjhE0yr60wHOxEGjX3jAAosTxIQ8A8/XyoQdyW1CAii2j0aEr0JzJGWvk1neSPSOjZowswc6q0x5X+FhejIyzOD68=
Received: from MWHPR05MB2815.namprd05.prod.outlook.com (10.168.245.9) by MWHPR05MB3392.namprd05.prod.outlook.com (10.174.175.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Thu, 22 Jun 2017 05:41:30 +0000
Received: from MWHPR05MB2815.namprd05.prod.outlook.com ([10.168.245.9]) by MWHPR05MB2815.namprd05.prod.outlook.com ([10.168.245.9]) with mapi id 15.01.1199.016; Thu, 22 Jun 2017 05:41:30 +0000
From: Rukesh D <rukeshd@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Thread-Topic: bmwg Digest, Vol 153, Issue 25
Thread-Index: AQHS6fePT+vpxS4fPUa2Gn6Fr+FK+aIupZnQgAG5BAA=
Date: Thu, 22 Jun 2017 05:41:30 +0000
Message-ID: <MWHPR05MB281530EE9D52627CF0F4EBB9A2DB0@MWHPR05MB2815.namprd05.prod.outlook.com>
References: <mailman.89.1497985213.8807.bmwg@ietf.org> <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
In-Reply-To: <BY1PR0501MB13525D09B8D08518E20C9D6EC2DA0@BY1PR0501MB1352.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [66.129.239.10]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MWHPR05MB3392; 7:wYUhOGNlktTP1MiKn/G1CZh9v9S7S6NE9cLuBumXYTpYZGpK3U3muAudIZTuMG9gpavwtt+GfBnQSjyVQQ3Vigiy69gAn0ixsyoeKGPoNslu/kfsOh6ZDd6m0BbwPo1UipWFpUSZRRU1dUnAU70k4ZrWYwccYLvCI0Y1f3FMluOqgW22FGS7m06s0xo+P1lkvPby5pg5O8+u/ZzRPw4g6aNXHvu0CcpUXNnTHazHcsknplHKjDLiMsTzMZSGq4pZDA7loL/280iQ6mZatnc8dSuix6GJFuTPok7c/c9SodsuyF50bH6K1AS8w4HndC8fD56CCdiZCSMfoVCCEWnr5lelJNd0/JT+8Nf/ahZ3UbgzxhKoPDxsalbLd+NP6Iqf8h6Q2oEJFXLkrcsImAV++1SW0+xszjjfb1m0MfruZI0JY101gAAlPOGCFVoTcPvyLFFTNg4nVeLnVkk6rbdep1Lra7djAZi8Or23YUZ4R/ZWHH+fkfSH/TkhsmxVIZe+gk5Y0rtZPBTXnMfgharUHfOE7Pmx9Jz7PJIQMP5XTJnM8/H5fpb75Dv+vkRWPUshPjsucm/gjvfehqIvWEVjNMGTQ4ggbPhCrDTHpB4HWztVAsAvGCzNaBbpGOXkuXX6o4g89GQhFYYUR40R3eJ2QYD28LTxGvvIb4oWPDz0tZZ86+mNlz9l/Hib/UPUi8xhyPWO2g2iJeaMhwYQ0Nw3kkwO6qOYZwxSdY14CYDbCFt1w/FgkTbN3zklhGr9QzCJbk39yPvm9y2Oqtt7KZMDlq0C2z3M5B9nKl00vpJ8J6Q=
x-ms-traffictypediagnostic: MWHPR05MB3392:
x-ms-office365-filtering-correlation-id: 2ef7e609-6d43-436b-b2a2-08d4b9315574
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081)(201702281549075); SRVR:MWHPR05MB3392; 
x-microsoft-antispam-prvs: <MWHPR05MB3392C7FF4ED95D0530A770E3A2DB0@MWHPR05MB3392.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(192374486261705)(97927398514766); 
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(100000703101)(100105400095)(10201501046)(6055026)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123560025)(20161123562025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR05MB3392; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR05MB3392; 
x-forefront-prvs: 03468CBA43
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39850400002)(39400400002)(39410400002)(39840400002)(39860400002)(39450400003)(5403001)(27574002)(18543002)(377454003)(13464003)(10533003)(53936002)(6246003)(25786009)(76176999)(6436002)(54356999)(81166006)(6506006)(1730700003)(77096006)(8676002)(3660700001)(3280700002)(4326008)(189998001)(5660300001)(2906002)(74316002)(2501003)(5640700003)(2950100002)(8936002)(6916009)(575784001)(229853002)(966005)(3846002)(86362001)(33656002)(50986999)(66066001)(6116002)(102836003)(53546010)(110136004)(6306002)(9686003)(122556002)(38730400002)(99286003)(305945005)(7696004)(55016002)(478600001)(512954002)(14454004)(2900100001)(2351001)(7736002); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR05MB3392; H:MWHPR05MB2815.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jun 2017 05:41:30.4009 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB3392
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Y6tspFSz3TFeLRYdIhSj3jMwBxM>
Subject: Re: [bmwg] bmwg Digest, Vol 153, Issue 25
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 05:41:36 -0000

> (3) whether you commit to reviewing the WG document if adopted

Am committing to review this draft if adopted. This covers needed test meth=
odologies for EVPN deployments.

Thanks,
Rukesh

-----Original Message-----
From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmwg-request@ietf.or=
g
Sent: Wednesday, June 21, 2017 12:30 AM
To: bmwg@ietf.org
Subject: bmwg Digest, Vol 153, Issue 25

Send bmwg mailing list submissions to
	bmwg@ietf.org

To subscribe or unsubscribe via the World Wide Web, visit
	https://www.ietf.org/mailman/listinfo/bmwg
or, via email, send a message with subject or body 'help' to
	bmwg-request@ietf.org

You can reach the person managing the list at
	bmwg-owner@ietf.org

When replying, please edit your Subject line so it is more specific than "R=
e: Contents of bmwg digest..."


Today's Topics:

   1. Re: Call For Adoption: draft-kishjac-bmwg-evpntest-07
      (MORTON, ALFRED C (AL))


----------------------------------------------------------------------

Message: 1
Date: Tue, 20 Jun 2017 18:43:18 +0000
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
Message-ID:
	<4D7F4AD313D3FC43A053B309F97543CF25FE116A@njmtexg5.research.att.com>
Content-Type: text/plain; charset=3D"us-ascii"

BMWG,

So far, I count about eight people who offered support for the draft, and o=
ne person who made a technical comment along with their message.

*No one* has answered this question:
>=20
> (3) whether you commit to reviewing the WG document if adopted
>

With no commitments to review the draft and provide the necessary expertise=
 to develop appropriate benchmarks for our industry, it will not be appropr=
iate to add a milestone to our charter (or re-charter, if necessary).

There is still time for everyone to reconsider their position, and provide =
an answer to the question above.  The WGLC closes on Friday 23 June 2017.

thanks for your consideration,
Al
bmwg co-chair


> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED=20
> C
> (AL)
> Sent: Saturday, June 03, 2017 6:34 PM
> To: bmwg@ietf.org
> Subject: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
>=20
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more=20
> information.
>=20
> BMWG,
>=20
> At our Chicago meeting, the BMWG reviewed a version of the Internet=20
> Draft on
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-05
>=20
> There was sufficient readership present at the meeting and=20
> positive/constructive comments on the work, so the chairs asked=20
> whether support existed to adopt the draft as a WG item. There was=20
> support and no objections. As always, such decisions are officially=20
> taken on the mailing list.
>=20
> This message*, therefore, starts a call for adoption on:
>=20
> Benchmarking Methodology for EVPN and PBB-EVPN
> draft-kishjac-bmwg-evpntest-07
>=20
> to run until Friday 23 June 2017.
>=20
> Please reply to bmwg@ietf.org indicating:
>=20
> (1) whether you support addition of the following milestone to
>    the BMWG charter:
>=20
> date TBD: Benchmarking Methodology for EVPN and PBB-EVPN to the IESG
>=20
> (2) whether you support the adoption of draft-kishjac-bmwg-evpntest-07
>    as the basis document for this milestone
>=20
> (3) whether you commit to reviewing the WG document if adopted
>=20
> We are aware that BMWG has nearly completed all its existing chartered=20
> items.  This usually begins discussion of re-chartering, this time=20
> with Warren (our new AD Advisor). The final decision/schedule for=20
> adoption will depend on these discussions.
>=20
> thanks for your consideration of the draft, and regards,
>=20
> Al
> bmwg co-chair
>=20
> *using Brian Trammell's recent message to IPPM as a template (thanks!)
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D3tXKQNBR8Q4HC7BBx6pc5me8=
u
> 9z Ne9QVo_owOF5-ciM&s=3DxIKgDP2zI9F0IvfisGlEBjIij3I_H95zkVWWJGP8zlg&e=3D



------------------------------

Subject: Digest Footer

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


------------------------------

End of bmwg Digest, Vol 153, Issue 25
*************************************

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


From nobody Wed Jun 21 23:49:00 2017
Return-Path: <aretana@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 39945120227; Wed, 21 Jun 2017 23:48:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.522
X-Spam-Level: 
X-Spam-Status: No, score=-14.522 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 557dklTrSdfy; Wed, 21 Jun 2017 23:48:49 -0700 (PDT)
Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6FF521205F0; Wed, 21 Jun 2017 23:48:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6054; q=dns/txt; s=iport; t=1498114129; x=1499323729; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=1GR+CmlGjE+Lzfe2VpkMZ6F5XOgYH26WFEpDjAfdU60=; b=kqUKpBMe7pO3KSM8ioystDRZtQ51qJT8bWkKIlJ4Czrcpm5OY1pI/GHd 96SqoOg7nMQ31WjLbM+MwRDJO96G/z1Yadpl9pe0kzE+8nhjQju6m8XRm oseTC+d4kT4IsgiNoEUnQOkOVSGKfuFLZElEIsSF6EtqHE849CDxX+w+0 A=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0BwAQAPZ0tZ/4cNJK1dGQEBAQEBAQEBA?= =?us-ascii?q?QEBBwEBAQEBgm9pYoENB4NlihmRPCKILIgihSqCEYYkAhqCXz8YAQIBAQEBAQE?= =?us-ascii?q?BayiFGQEEASNWBQsCAQg/AwICAh8RFBECBA4FiUhMAw0Iq0iCJiqHCQ2EJgEBA?= =?us-ascii?q?QEBAQEBAQEBAQEBAQEBAQEBAR2GbYILC4JugleFJjCCMQWXHYcLOwKOeYRnggm?= =?us-ascii?q?FSIo+i1+JNgEfOIEKdBVbAYUvgU12iFCBDQEBAQ?=
X-IronPort-AV: E=Sophos;i="5.39,372,1493683200";  d="scan'208,217";a="439044514"
Received: from alln-core-2.cisco.com ([173.36.13.135]) by alln-iport-2.cisco.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 22 Jun 2017 06:48:48 +0000
Received: from XCH-ALN-005.cisco.com (xch-aln-005.cisco.com [173.36.7.15]) by alln-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id v5M6mmfN003655 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 22 Jun 2017 06:48:48 GMT
Received: from xch-aln-002.cisco.com (173.36.7.12) by XCH-ALN-005.cisco.com (173.36.7.15) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Thu, 22 Jun 2017 01:48:47 -0500
Received: from xch-aln-002.cisco.com ([173.36.7.12]) by XCH-ALN-002.cisco.com ([173.36.7.12]) with mapi id 15.00.1210.000; Thu, 22 Jun 2017 01:48:47 -0500
From: "Alvaro Retana (aretana)" <aretana@cisco.com>
To: Lucien <lucien.avramov@gmail.com>
CC: "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, Lucien Avramov <lucienav@google.com>, The IESG <iesg@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>, "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>, Sarah Banks <sbanks@encrypted.net>
Thread-Topic: Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
Thread-Index: AQHS6UORi83ftabEaky9N6fUtwLWHqItR1qAgAEh4wD//9//gIACYFEAgABAdQA=
Date: Thu, 22 Jun 2017 06:48:47 +0000
Message-ID: <D20E70F6-3E3C-4929-B7DB-CA718BF376A2@cisco.com>
References: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com> <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com> <D78D2206-5A27-40E3-9E29-8C4A8D1ABD01@cisco.com> <CAArZqeWN2suTLp5DUNsqEK5uCP7pa7-rtu7510c508jQMwYTeg@mail.gmail.com> <CAArZqeXLEnN0XQBvC2L5WrLx33NLg2nH9Jqs6PbPcycb9yRqhQ@mail.gmail.com>
In-Reply-To: <CAArZqeXLEnN0XQBvC2L5WrLx33NLg2nH9Jqs6PbPcycb9yRqhQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/f.20.0.170309
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.61.208.170]
Content-Type: multipart/alternative; boundary="_000_D20E70F63E3C4929B7DBCA718BF376A2ciscocom_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/15Bn96xJ8wauk8AMgU25gi8pjnk>
Subject: Re: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 06:48:51 -0000

--_000_D20E70F63E3C4929B7DBCA718BF376A2ciscocom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

T24gNi8yMi8xNywgNjo1OCBBTSwgImllc2cgb24gYmVoYWxmIG9mIEx1Y2llbiIgPGllc2ctYm91
bmNlc0BpZXRmLm9yZzxtYWlsdG86aWVzZy1ib3VuY2VzQGlldGYub3JnPiBvbiBiZWhhbGYgb2Yg
bHVjaWVuLmF2cmFtb3ZAZ21haWwuY29tPG1haWx0bzpsdWNpZW4uYXZyYW1vdkBnbWFpbC5jb20+
PiB3cm90ZToNCg0KTHVjaWVuOg0KDQpIaSENCg0KPiBJIHJlbW92ZWQgMS4yIHBlciB5b3VyIHN1
Z2dlc3Rpb24uIEl0J3Mgbm93IGFsbCBjb25zaXN0ZW50Lg0KPg0KPiBQbGVhc2UgY29uZmlybSBp
dCdzIGdvb2QgZm9yIHlvdT8NCg0KWWVzLCB0aGUgY2hhbmdlIGlzIGZpbmUgd2l0aCBtZS4gIEni
gJlsbCBjbGVhciB0aGUgRElTQ1VTUy4NCg0KSnVzdCBvbmUgbml0OiAgVGhlIHBocmFzZSDigJxB
ZGRpdGlvbmFsIGludGVycHJldGF0aW9uIG9mIFJGQzIxMTkgdGVybXM64oCdIHNlZW1zIG91dCBv
ZiBwbGFjZSBiZWNhdXNlIHRoZXJlIGlzIG5vIGFkZGl0aW9uYWwgY2xhcmlmaWNhdGlvbiBhYm91
dCB0aGUga2V5d29yZHMgaW4gdGhlIHBhcmFncmFwaCBhZnRlciBpdC4NCg0KVGhhbmtzIQ0KDQpB
bHZhcm8uDQo=

--_000_D20E70F63E3C4929B7DBCA718BF376A2ciscocom_
Content-Type: text/html; charset="utf-8"
Content-ID: <017EE56121EBE446B758C0EA1ADF9C72@emea.cisco.com>
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4
bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo
dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo
dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp
dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l
dGEgbmFtZT0iVGl0bGUiIGNvbnRlbnQ9IiI+DQo8bWV0YSBuYW1lPSJLZXl3b3JkcyIgY29udGVu
dD0iIj4NCjxtZXRhIG5hbWU9IkdlbmVyYXRvciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUg
KGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxlPjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8N
CkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0
IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJ
cGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8N
CnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsN
CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWls
eToiVGltZXMgTmV3IFJvbWFuIjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1z
dHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxp
bmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1w
cmlvcml0eTo5OTsNCgljb2xvcjpwdXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9
DQpzcGFuLkVtYWlsU3R5bGUxNw0KCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbC1yZXBseTsNCglm
b250LWZhbWlseTpDYWxpYnJpOw0KCWNvbG9yOndpbmRvd3RleHQ7DQoJZm9udC13ZWlnaHQ6bm9y
bWFsOw0KCWZvbnQtc3R5bGU6bm9ybWFsO30NCnNwYW4ubXNvSW5zDQoJe21zby1zdHlsZS10eXBl
OmV4cG9ydC1vbmx5Ow0KCW1zby1zdHlsZS1uYW1lOiIiOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRl
cmxpbmU7DQoJY29sb3I6dGVhbDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlwZTpl
eHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtz
aXplOjguNWluIDExLjBpbjsNCgltYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW47fQ0KZGl2
LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQotLT48L3N0eWxlPg0KPC9oZWFk
Pg0KPGJvZHkgYmdjb2xvcj0id2hpdGUiIGxhbmc9IkVOLVVTIiBsaW5rPSJibHVlIiB2bGluaz0i
cHVycGxlIj4NCjxkaXYgY2xhc3M9IldvcmRTZWN0aW9uMSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSI+T24gNi8yMi8xNywgNjo1OCBBTSwg
JnF1b3Q7aWVzZyBvbiBiZWhhbGYgb2YgTHVjaWVuJnF1b3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86
aWVzZy1ib3VuY2VzQGlldGYub3JnIj5pZXNnLWJvdW5jZXNAaWV0Zi5vcmc8L2E+IG9uIGJlaGFs
ZiBvZg0KPGEgaHJlZj0ibWFpbHRvOmx1Y2llbi5hdnJhbW92QGdtYWlsLmNvbSI+bHVjaWVuLmF2
cmFtb3ZAZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48L3NwYW4+PC9wPg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJp
Ij48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh
biBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSI+THVjaWVuOjxvOnA+PC9vOnA+PC9zcGFuPjwv
cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJp
Ij48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh
biBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSI+SGkhPG86cD48L286cD48L3NwYW4+PC9wPg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGlicmkiPjxv
OnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJpO2NvbG9yOmJsYWNrIj4mZ3Q7
IEkgcmVtb3ZlZCAxLjIgcGVyIHlvdXIgc3VnZ2VzdGlvbi4gSXQncyBub3cgYWxsIGNvbnNpc3Rl
bnQuJm5ic3A7PG86cD48L286cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGlicmk7Y29sb3I6YmxhY2si
PiZndDs8bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaTtjb2xvcjpibGFj
ayI+Jmd0OyBQbGVhc2UgY29uZmlybSBpdCdzIGdvb2QgZm9yIHlvdT88bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LWZh
bWlseTpDYWxpYnJpIj48YnI+DQpZZXMsIHRoZSBjaGFuZ2UgaXMgZmluZSB3aXRoIG1lLiZuYnNw
OyBJ4oCZbGwgY2xlYXIgdGhlIERJU0NVU1MuPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGlicmkiPjxvOnA+Jm5i
c3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm
b250LWZhbWlseTpDYWxpYnJpIj5KdXN0IG9uZSBuaXQ6Jm5ic3A7IFRoZSBwaHJhc2Ug4oCcQWRk
aXRpb25hbCBpbnRlcnByZXRhdGlvbiBvZiBSRkMyMTE5IHRlcm1zOuKAnSBzZWVtcyBvdXQgb2Yg
cGxhY2UgYmVjYXVzZSB0aGVyZSBpcyBubyBhZGRpdGlvbmFsIGNsYXJpZmljYXRpb24gYWJvdXQg
dGhlIGtleXdvcmRzIGluIHRoZSBwYXJhZ3JhcGggYWZ0ZXIgaXQuPG86cD48L286cD48L3NwYW4+
PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGli
cmkiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz
cGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJpIj5UaGFua3MhPG86cD48L286cD48L3NwYW4+
PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGli
cmkiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz
cGFuIHN0eWxlPSJmb250LWZhbWlseTpDYWxpYnJpIj5BbHZhcm8uPG86cD48L286cD48L3NwYW4+
PC9wPg0KPC9kaXY+DQo8L2JvZHk+DQo8L2h0bWw+DQo=

--_000_D20E70F63E3C4929B7DBCA718BF376A2ciscocom_--


From nobody Wed Jun 21 23:55:48 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 45C1D1205F1; Wed, 21 Jun 2017 23:55:39 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149811453925.30703.12953068060298600371@ietfa.amsl.com>
Date: Wed, 21 Jun 2017 23:55:39 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/37dPcBc0uET7YMLyuCxU2TiFlEw>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-18.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 06:55:39 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-18.txt
	Pages           : 20
	Date            : 2017-06-21

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center. A pre-requisite to this
   publication is the terminology document [draft-ietf-bmwg-dcbench-
   terminology]. Many of these terms and methods may be applicable
   beyond this publication's scope as the technologies originally
   applied in the data center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-18
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-18

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-18


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Wed Jun 21 23:56:18 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6EC5128D16; Wed, 21 Jun 2017 23:56:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XbJbyzx2KIIW; Wed, 21 Jun 2017 23:56:10 -0700 (PDT)
Received: from mail-yb0-x231.google.com (mail-yb0-x231.google.com [IPv6:2607:f8b0:4002:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 80E4712946B; Wed, 21 Jun 2017 23:56:04 -0700 (PDT)
Received: by mail-yb0-x231.google.com with SMTP id 84so2006853ybe.0; Wed, 21 Jun 2017 23:56:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=FDMUt2dx0YYQGUbCiet5beJUp9Z8Za3Fi5Im7y+YSyI=; b=mq0evb1JIAeKInbP6pvSM2zgTtwgKkRE4ldTsnKxMPraTIkjFvQROxTXPIbRBQMfOA sP3x6SCDBye79rVxl9Ox3KP6UiQNeX6UVtb/e4+kBM85ZQBv8ES2LGBtSCUWC/F4DG0H XD413oK0khrerAAflwiii1p4kRsVAfjUK7Tt8MLn/5rHfv6iEf89D7qC4xZriLMlLg+n dz9GId2TDH/zKF8nv/0JGNCKQW535P7UGSHu2CBbvHBGEsiKm7Ps54NwnvYFWSh8+316 wC+YXILI4nk/E+ZO5hiA03WPO/FydHspNZlmMAxK34XRYR0qayJSFsto3AO8yQPn5GGd qOLw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=FDMUt2dx0YYQGUbCiet5beJUp9Z8Za3Fi5Im7y+YSyI=; b=PdYmWcNR8Yn4doeBihVXtY88KWesBqZESyIjGfx6YAyD+rtsKI8QJtPXRJ9d62wGHZ J2JTP2NaH5tGzXopARkU5OQgVM+mrlcY7Tg1Kat6TC2dyKhyH7MZfr/LX1OqwAX4yw8h ugqZy/QjFYbyx+Z96nSAlNVg1C0YVtqH61F8cBBfN7FItD2dgfpZ95xqlNFsPi4rYvo8 sXyN/GZxVUMllAm2js/4I5ZVYBze3z/r6I/BjhiHGY+S1yIbqEZRCBdOpvGV6F9sYdzp NOwl1Ox35GOAPIGpDGmn5PfMv2mRtpZ/xcBHfhCAa/curGjgDJz+f+7nmMqkiG6XOuwp G/tQ==
X-Gm-Message-State: AKS2vOy2r5MyID4vI5DR1nqapQzx0D5/TnKBVKghufvBwTT4K3Bsyono 6ON5VU2WffJqIGpaoqT8SZwOQ4vzZw==
X-Received: by 10.37.180.18 with SMTP id n18mr605104ybj.258.1498114563680; Wed, 21 Jun 2017 23:56:03 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Wed, 21 Jun 2017 23:56:03 -0700 (PDT)
In-Reply-To: <D20E70F6-3E3C-4929-B7DB-CA718BF376A2@cisco.com>
References: <149790794238.10693.2532866777748124406.idtracker@ietfa.amsl.com> <CALTEt=C9pzd4w9HjstXjzbj6awbdhTR66CF=H75O_39q0NK_iA@mail.gmail.com> <D78D2206-5A27-40E3-9E29-8C4A8D1ABD01@cisco.com> <CAArZqeWN2suTLp5DUNsqEK5uCP7pa7-rtu7510c508jQMwYTeg@mail.gmail.com> <CAArZqeXLEnN0XQBvC2L5WrLx33NLg2nH9Jqs6PbPcycb9yRqhQ@mail.gmail.com> <D20E70F6-3E3C-4929-B7DB-CA718BF376A2@cisco.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Wed, 21 Jun 2017 23:56:03 -0700
Message-ID: <CAArZqeVbBUjOBf1BUV+y9DXa1kM9RKuNW6_DE3=VjMDN7AvkOg@mail.gmail.com>
To: "Alvaro Retana (aretana)" <aretana@cisco.com>
Cc: "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, Lucien Avramov <lucienav@google.com>, The IESG <iesg@ietf.org>,  "bmwg@ietf.org" <bmwg@ietf.org>,  "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>,  Sarah Banks <sbanks@encrypted.net>
Content-Type: multipart/alternative; boundary="f403045e6aa266b3ee055286fa04"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/zbrTzVLy8QsqCmkWjt2dpicdAFE>
Subject: Re: [bmwg] Alvaro Retana's Discuss on draft-ietf-bmwg-dcbench-methodology-12: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 06:56:12 -0000

--f403045e6aa266b3ee055286fa04
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks Alvaro! Fixed the Nit also now...


On Wed, Jun 21, 2017 at 11:48 PM, Alvaro Retana (aretana) <aretana@cisco.co=
m
> wrote:

> On 6/22/17, 6:58 AM, "iesg on behalf of Lucien" <iesg-bounces@ietf.org on
> behalf of lucien.avramov@gmail.com> wrote:
>
>
>
> Lucien:
>
>
>
> Hi!
>
>
>
> > I removed 1.2 per your suggestion. It's now all consistent.
>
> >
>
> > Please confirm it's good for you?
>
>
> Yes, the change is fine with me.  I=E2=80=99ll clear the DISCUSS.
>
>
>
> Just one nit:  The phrase =E2=80=9CAdditional interpretation of RFC2119 t=
erms:=E2=80=9D
> seems out of place because there is no additional clarification about the
> keywords in the paragraph after it.
>
>
>
> Thanks!
>
>
>
> Alvaro.
>

--f403045e6aa266b3ee055286fa04
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks Alvaro! Fixed the Nit also now...<div><br></div></d=
iv><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Jun 21=
, 2017 at 11:48 PM, Alvaro Retana (aretana) <span dir=3D"ltr">&lt;<a href=
=3D"mailto:aretana@cisco.com" target=3D"_blank">aretana@cisco.com</a>&gt;</=
span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex">







<div bgcolor=3D"white" lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"m_3730175654895504355WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">On 6/22/17, 6:58=
 AM, &quot;iesg on behalf of Lucien&quot; &lt;<a href=3D"mailto:iesg-bounce=
s@ietf.org" target=3D"_blank">iesg-bounces@ietf.org</a> on behalf of
<a href=3D"mailto:lucien.avramov@gmail.com" target=3D"_blank">lucien.avramo=
v@gmail.com</a>&gt; wrote:<u></u><u></u></span></p>
<div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Lucien:<u></u><u=
></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Hi!<u></u><u></u=
></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
</div><span class=3D"">
<div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
 I removed 1.2 per your suggestion. It&#39;s now all consistent.=C2=A0<u></=
u><u></u></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
<u></u>=C2=A0<u></u></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri;color:black">&gt;=
 Please confirm it&#39;s good for you?<u></u><u></u></span></p>
</div>
</span><p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><br>
Yes, the change is fine with me.=C2=A0 I=E2=80=99ll clear the DISCUSS.<u></=
u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Just one nit:=C2=
=A0 The phrase =E2=80=9CAdditional interpretation of RFC2119 terms:=E2=80=
=9D seems out of place because there is no additional clarification about t=
he keywords in the paragraph after it.<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Thanks!<span cla=
ss=3D"HOEnZb"><font color=3D"#888888"><u></u><u></u></font></span></span></=
p><span class=3D"HOEnZb"><font color=3D"#888888">
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri"><u></u>=C2=A0<u>=
</u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-family:Calibri">Alvaro.<u></u><u=
></u></span></p>
</font></span></div>
</div>

</blockquote></div><br></div>

--f403045e6aa266b3ee055286fa04--


From nobody Thu Jun 22 00:04:16 2017
Return-Path: <IHussain@infinera.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 796A8127011 for <bmwg@ietfa.amsl.com>; Thu, 22 Jun 2017 00:04:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.922
X-Spam-Level: 
X-Spam-Status: No, score=-1.922 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=infinera.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HMEVHluO2Msj for <bmwg@ietfa.amsl.com>; Thu, 22 Jun 2017 00:04:12 -0700 (PDT)
Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0062.outbound.protection.outlook.com [104.47.37.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB99A1205F1 for <bmwg@ietf.org>; Thu, 22 Jun 2017 00:04:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infinera.onmicrosoft.com; s=selector1-infinera-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=7ItYZ+h2TC0XFTj1TiM+WQnB7ROge6WWZcvkSzy4xDI=; b=KOgUCSWuozLHtaXIx+GZN3LHoW3fiax0os5B090RFmF3eS4IkdGcwvbYf7v/pvVt1ztEBB1dcSRpJ9QYnSQHFMjWslhrKn2RmZnUL29KHDEZf4sCUwiiNnsisukD1X7ALyxkj57+D5F2HICRBYE7vmqO1hoHFHz/2cmCSnoNWf8=
Received: from DM3PR10CA0035.namprd10.prod.outlook.com (10.164.12.45) by BN3PR10MB0243.namprd10.prod.outlook.com (10.160.120.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.15; Thu, 22 Jun 2017 07:04:10 +0000
Received: from DM3NAM03FT045.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::207) by DM3PR10CA0035.outlook.office365.com (2a01:111:e400:598a::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.15 via Frontend Transport; Thu, 22 Jun 2017 07:04:10 +0000
Authentication-Results: spf=pass (sender IP is 204.128.141.23) smtp.mailfrom=infinera.com; ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=bestguesspass action=none header.from=infinera.com;
Received-SPF: Pass (protection.outlook.com: domain of infinera.com designates 204.128.141.23 as permitted sender) receiver=protection.outlook.com;  client-ip=204.128.141.23; helo=owa.infinera.com;
Received: from owa.infinera.com (204.128.141.23) by DM3NAM03FT045.mail.protection.outlook.com (10.152.82.208) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1199.9 via Frontend Transport; Thu, 22 Jun 2017 07:04:09 +0000
Received: from SV-EX13-PRD1.infinera.com (10.100.103.228) by sv-ex13-prd1.infinera.com (10.100.103.228) with Microsoft SMTP Server (TLS) id 15.0.1178.4; Thu, 22 Jun 2017 00:04:09 -0700
Received: from SV-EX13-PRD1.infinera.com ([10.100.97.11]) by sv-ex13-prd1.infinera.com ([10.100.97.11]) with mapi id 15.00.1178.000; Thu, 22 Jun 2017 00:04:09 -0700
From: Iftekhar Hussain <IHussain@infinera.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
Thread-Index: AdLrJVB1lqbQmMJqRU27WulOfrrp/A==
Date: Thu, 22 Jun 2017 07:04:08 +0000
Message-ID: <245cc2c686e44af38b4eeb5964d12929@sv-ex13-prd1.infinera.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.100.99.93]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-EOPAttributedMessage: 0
X-Forefront-Antispam-Report: CIP:204.128.141.23; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39410400002)(39850400002)(39400400002)(39450400003)(39840400002)(2980300002)(438002)(199003)(189002)(8676002)(77096006)(50466002)(189998001)(588024002)(2351001)(33646002)(47776003)(24736003)(3846002)(102836003)(86362001)(6116002)(19618925003)(8746002)(80792005)(8936002)(46406003)(1730700003)(72206003)(14454004)(97756001)(2501003)(5660300001)(4270600005)(7636002)(38730400002)(6246003)(110136004)(7736002)(305945005)(2906002)(5640700003)(478600001)(2900100001)(356003)(558084003)(230783001)(106466001)(53416004)(229853002)(7696004)(108616004)(54356999)(50986999)(6916009); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR10MB0243; H:owa.infinera.com; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; 
X-Microsoft-Exchange-Diagnostics: 1; DM3NAM03FT045; 1:65H6B46FqSOkv6Qo+mU60TJY1pwEr1tgw49Y+OoTbFOfZW3+NPBWB5SN+uZMD+ObLu1G07VaLedRVAeaQ87G/jxmJ/qzVz+VRIkvAHZCoPMBjnjV3SBVbi+T47NfQrWaIBLmwsqC30lU+oIHg3APR6PTcZIT+gogaEQ34BtwePimV1x1FQWEADLcsZ2k5rXngXtXRjLLDWYZ23DF99k5D39t2Gj50EPqeNDwOSOF/0xrkCS5vtgZEXKSqMji5WflRjmiGiM054zp/YAMnTSpYbQFa2ZN6KvUNpWvmS1C/LaDEnM7ZLNYJuQMXr7uNcdAJxiWIhCPMYkOPtcQ687E6Eo89aQbdbs9TsgDdQi1SCo7WtEX9ny5Mh9uDcqWLTZFPKOfmCANoOykXbZ4OsC9oJTG4hsIrGKO9QZL/kM3kb2zRkUIVI97peaLAtpKq2cuvwVKALH7wDLDbStuWw9Hyi0HDWM2Y9AMzJbdZZuXvm2j62+YGC88stN8i9zHC0I291UNwsHBSHsrcYd0xGHuZoF6Ye0i4oS5+stQRoRU5k6wpGW5nJzM7p+rKeNPzeFbRxR6Qd8uZ4m+tAdDqVID32rBqdc5dX0b0SoqR3q38zTI5Qnol/QUSIUG1e7rCAe9/D+c+mha42EjAK+HfsRuryjXEXJdskURwCKgpNli5AUBX3fr/GkrSgbKdfQVcx+mi/3UuaM2l8iJzAq+Xm1FjpHPRP5x9ZYLeYlZT8laQ3rjuhWblYn4IBS1mW1h/R3JLu1AB9TPZp23N2IzoQa/hTEpW8d53SUg25IFmAj450vID60leBzRAN03AtTl20CzDX/BypL0IuSUYBmYzoGPXm13jWf6mL+ISPhkMMKnHLDWfSKtCRCOn0rgeD1UFWz+
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0a3dd395-75f3-4559-7a15-08d4b93ce1ae
X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500055)(300135000095)(300000501055)(300135300095)(22001)(300000502055)(300135100095)(8251501002)(2017030254075)(300000503055)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504055)(300135200095)(300000505055)(300135600095); SRVR:BN3PR10MB0243; 
X-Microsoft-Exchange-Diagnostics: 1; BN3PR10MB0243; 3:TrErEmSH2SkSQCMLfvKW2EDjbgyBlHbL5YTyG2yCDX0JaHm5IWudL4Uq7kbzWHfyCPA6DESVylBIs5OaxhjOZQMM6TnvAz+jyfDA8or2WQLg2eboEwcX7tqyaEWsbejHPwcqyrYd9Zru36ehg6n6Ang+GC5zoyiHY9VpOYEPmUPhDbWLx+1sxar5VoXdDjBbdu2+dxUaIucn+QhqIUZmTTLk8alXg7UqymUATtbHF258mc1YO2AZ2SwqYULGm/QwkEyLCGOb5Dt0CO5tu3JAyDpwgUccCT6OiXt3fOEmDvAwhwADZHLMTqosDAWf7VraEspoM0kEBKgZtz7mwAc8giHeE2dRAUoZJMkcWn2SDzDilIR3xllp0RXgIfvooel5TfMX0Qg4VtHV+cinn36QZ5LeY2eGWVng0Up0g/GJYGXHFWFu33/ksbSL8BUihRp2gmKwZVeqnC5TqTLCX33jZZtsMrR2x7nYAngivv91ulkeNS6tzDffMEfHrHpU5FB+nU/V+YYxFrI1fle9n5us2cwBr8bEF29IjkFyCbVNRzAWGH68G2RpV5CXZd3DDVQCnfFwqcDSGd2zPpyNUUZKqncFmYmO40FU5Ox8dCW/cpvPnUD2idgL6ZT6crjQhdalwD8TC6Jn4uL5Vf5QHl9qPE722JkI0Lz0zF2WBxH/++tk7aL7QCuR5tsUcIaVa9Suuh7sFAXUQCytiVqdSQpGRyPuB3GhK45IYH2IRGpGRkeEAuFtPhme7R4494an5yIERLD7eKVxUNowOZ88QUEP/T6G+UupdFuL1bZFYxHpfjIqmSMIwcBHMX5Tii57cwzNVQnzq4Aq68gJVBNuBtUe68hB6d/J7Zzfp7r91rGGEdntq5ZkRV0ZIHAeU9XTelxF
X-MS-TrafficTypeDiagnostic: BN3PR10MB0243:
X-Microsoft-Exchange-Diagnostics: 1; BN3PR10MB0243; 25:RSYuVoYkMnWFNzzXwRWxkuzyjRW7+o4xjUU5imuKmUpJqmJD5wj3VtOiJZbIgLAdW96eD7BK9rNC0KiXrFPAUiJHLWvPg0VRwztWyaVYKRqA24m85fO0ZTh/i3HfyIc5SrA0Q0Lp0pB514iXiwW75GuN83DbzkpizKT6eTxfU+jZJ4IMhxsI7VK+cwUN1NoULbtM9OjfKxWkA4gcUp8IV8LyIasXs6IsBXyYjCGXUIQPeVdkAeti3u2QY5Sii3mVGU13+ONdO2Yu0NRS7zDgEaOLSugX7pF8vqcs4GirfomJyOmMR1ZbEbUNTslRxUlPiSbMuws4u1Unwh5W6M9ykyrQUjINTw+7Y+/z2lCuKkMeXG2dxtNX5HH7NN3s7Utv5R8FJ5PK+yAqdcbwlym12TD0acEWE6XGq2Z8Usq1uAolb4IYsAZEQe13A/XsaOEMfQGj0KVfHX5fpul6SmS39joEeSLA2ZpaLCMoqs+71OyQeNKloH8c8T4ELIxKvax+hVHR4t70GmLyjWoHody0yeoHk1F1Nfk9hRh5Z5aWeTYofcyrAs4ahvLEw/zg+dQ3hLWrsUwln01dG5AcuVCEM8Fxrm9kDQStNEfaSxdGe266GmPGbbPidpqWyUJ38rj87TgN7i/PtnDESODjixj0xlj0YtBGOIMWs1mYl1fJ4spmfFXAfWG75icbWUKoVydIs18g/MphSg/itpJuxQ2Y+SMA2rluEjJYk9TOjXS7FfuKFp7oiv41NL4FuU/PrJ4KQnUG9ZEdXPd9Qrw5reFTXNwiZF5qBKbeb1pQRAPfxPsdrr5nI8bO/THUzDr+QCd8krSdkE2rw2gUKcsoT7XBjFO9BafzCY03CVYnHX1NbN/WpIatK+bPcgam10OK9lY8fm8wh8o/dOsByd7Cdzr+sAr4TIwnpAoEzqBrCtzHeXM=
X-Microsoft-Exchange-Diagnostics: 1; BN3PR10MB0243; 31:zIERIpDcsGvhYeNLCmQpVQfj47hh4TEWd1i1PQKmRCa53dOVt54vqu6zKC3g4WrSXeC09aRAWxITJladtuYPu0xAi7uyFXc2+Dml+8QG775ONe4fryF2EjTUJmR8eebKNVFJZgxR/1xQ/FY4RXdzFni8d6hatp/lJ6KAXcyLNWBhQz8fiiLR5YUY287bYUkK5tQ3fUhko0Usdi65RYB/2aomoC0mzepPx09Shwhs+V+bdbzCmioeEz56di1Cyd3pBCR3EG45BBcVLKndpHKvgIRdBe0/NSCQ8aannCQmhUEOXW2QQavtO9z6WSa4l8K0+ui6eh9e0SRKUljaFonXkhSf9AE3AmwkvA2T9C5fFhKLdHLCfxCoxCgPSoaYlZdWuCc+KeAFXeLvE7cxkKy6U0Mp3+IuTuzAe131QV2XCZF022QDZLEHd3v5pmZffEKNUNEygykCb2iD2e44wBllVEb8xG1TIpWysq/jJVuGFW9ij7rNcgn2UpnHej1ms4QfjbiWKp9TMmvETyGekvzsBjb5Zqnjw2Mgw2hxav2RC5iumBTzh5TcWfsuQI659EZI/K99FWCxb6xaNG/w4cC/Oh1uANUPIp/KnTFc+D/o/1+20+1ODk6FimF/B9zgpQkHsT7OaeW2oG2gJaVcKyFpmWm9PeBqbsnGt267RCOY/vWpjHuwAJggkZC8bXHF+E5v
X-Microsoft-Antispam-PRVS: <BN3PR10MB0243C6F447979342F8310454B7DB0@BN3PR10MB0243.namprd10.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93004095)(6041248)(20161123562025)(20161123564025)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN3PR10MB0243; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN3PR10MB0243; 
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR10MB0243; 4:kuXhTO6MBsGEsbsTn2uKYRiE9ty3QQDZkcUocE516e?= =?us-ascii?Q?mhk/uzsTq0z+58rS1QJRGCz3/9zZJ+rvHGV+CLGef57OwU8UPOYVTsSOH5AO?= =?us-ascii?Q?Y989wL7/MGMHUltnrry9Ahho7jSZ1Wa2cMAKUwsRSKYgKi+iA9Xt3EwEcBt4?= =?us-ascii?Q?3ywS0BywKRtEtnBCX8XGrGXfybvpUXqRYsGzyuZOu4qix6Z7vsrytGQ+xmQy?= =?us-ascii?Q?WbfV+AnzaIEX4CwgemkhhW9iDzaAIyZMdXLM4/59DZ3ul1xHMxT3ZcaOmXsJ?= =?us-ascii?Q?pJs/ej5yNHeYmZiHjTpjFB/PBVuQNdQ4R+sjGSgEYmpS9bfv2OGMKHXqkT2M?= =?us-ascii?Q?Q+zMlaVLlHaEtpqXdObG/Mq/NZ+uq6BaoAMbTx6RmhU+5UNfLUeakVGW9jvP?= =?us-ascii?Q?fQtFf/rSTWEdzjff9JxgIqC8VxAE+xb1tnhkxoExnmWlGFUnEOYkKBXKcMvi?= =?us-ascii?Q?9SeZ4Rg91yWQMi6k0uc5gfAHX+cv4rIt20aPBNOpQ0mNwLVaBWGTWy6KEJk1?= =?us-ascii?Q?d3AnnwNE+VjLzEKBAiEhrjVOnJUXoRcyKqid2qojifhUj/taB1239jWreA+U?= =?us-ascii?Q?fFID/tF+o84nQ7iZTQXY2Nr+XH6HkENQqbOs94DEPDYUfPVJWD9yXF/p1Dvr?= =?us-ascii?Q?wI+3fKMqqO9gDILnfBX11pw0nfmIBKGW9+KHU8hC+CMk9mvxegm0uIGsQdPI?= =?us-ascii?Q?Xokxi7cvuEFplHYR5OdkP2eXQSO4rKwYdGJAPOQz4t3qkz8J/0THyh5y/g97?= =?us-ascii?Q?wwRBan2DQK2MOsLMNomre1TZ9ccrm/Ss/69wj5b3Fwut32rUhVcC8h4/DRfe?= =?us-ascii?Q?Ay4ILD+0lRsoc4931BCKyy0tRuM/lLt0Y6zxscNrr3UbIqy2LZYDJIT63Jzr?= =?us-ascii?Q?RPHfmtpXnv9XTWKDzDUiIocb5xgm58jyACOopunGnMX0mHzs7aigUiZYIX8o?= =?us-ascii?Q?AF9+DNw2UGKcL2HDSZhQTm6TCkhFSzGVp+rn10jO5IjJc77gboAhxkm1GgRR?= =?us-ascii?Q?TkUEFs2YMrZRtbyX7YXNCboOOspZssCNSpHAnK9WG10oxtl4n+aj9K701lR/?= =?us-ascii?Q?NNRimMh41JoVkKGJ9BrmPpJA2WH3An91X0gQy+EPnyq31L5gqcEogv6P03xE?= =?us-ascii?Q?C8Sw3Hng66y23gYFdMt+hPAS/oryRMNQ1QMrRagqwtVcirXRCDAim2Q4SWBE?= =?us-ascii?Q?yA+/hlFzZNbwI=3D?=
X-Forefront-PRVS: 03468CBA43
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR10MB0243; 23:O5Lfb0bCj5xYo9mE7NjoFfaVJ5KXyoXBC5KPj+XN2?= =?us-ascii?Q?OL5VZfXZ1VCmSAJh62c6+Tr7BETB8Y0PTyBbBxgWxRiKP3BRvawQ7Iog4wTl?= =?us-ascii?Q?Pj4U6DCXyLUlU+lJaAjVj3+k8SRXi1fJWMuf82ce1H9YW5DQw6E5qXJA9Jj3?= =?us-ascii?Q?TWqCPLa0WbtvRBEylSKRubZWvlCk2LaZype1JXnbp+onmOpnZcMc+P24Wo4f?= =?us-ascii?Q?/SohvFHDAbeWuJootRuFIBg/VWoAqmQCh38ib+a1mpsVSeTevKOX7OpzwTCf?= =?us-ascii?Q?RBUULT26EZoRyq/zbIXw7CB7OkSFoIai1L+WkqXuKnSvKV5MXRJsx9IFwhhi?= =?us-ascii?Q?fNRmLcxRBv+8ebGZ6+JtSTKJtp0DxD+HDCT/2FhBxHJ0r056GM5xwUZveJlT?= =?us-ascii?Q?rrtAF/vmJ0UlP4bj2b+oT+3nqSt4w9ptnlh/S/YWabhxXmqMRSTgGM6AW/e6?= =?us-ascii?Q?wZ4geCjDOxVDwMooKDhFRQKe5rz5S2ZkOI/fP2+5I6eoDv05g9i7UezWaB9v?= =?us-ascii?Q?GESwnrNPxM3JGmc1UA9V544wgRRsHpKJomaHRG908pdsk8ZyT/s9VvLHR48b?= =?us-ascii?Q?DZOFQMXzZ2L/yF/Ryz7NYaTvDCf7JhPT9vsHVthtQAPMbPTTwC0NQp4dLXWm?= =?us-ascii?Q?D5AAf9BUdXmbtrbnRwfkXLUcJRqf5T5Fv3m6sDscljfxdSJv2NF5J+FtPDwv?= =?us-ascii?Q?0j6VckSca9A10I3c10FpkxLRiryRSxRZc33O5g+IBq6Z8C7mgVCTe4QrZIdR?= =?us-ascii?Q?efqnL4mCqU8uUhs67FzClvrfNCuVDypwMAQ7Cf1coiMPM8cveXb1K7bWL9S2?= =?us-ascii?Q?J0Bpzby2NpGDmfPAnZLb5FXbW4chyxGnOeQMIZNZCSvE/tIGjGSzrMWQgOUb?= =?us-ascii?Q?TgxlR+rthei8EXzGEsxoyRo32pSMq5gn+d4cB0ek50FA+gM1KYnt43XvaWkz?= =?us-ascii?Q?9ZKeDUONH5JFoFdxJZyTUuhl9/g8lAikf6TsPU7bpOho/UJMHZ9LDxyiYZGF?= =?us-ascii?Q?zBM5CFDZYdrZccg63hI9Z/vXy78qotN11aM4WjTSp+heapXPqHd/oQjfQH7C?= =?us-ascii?Q?Jk7E6CPKA+zZYtuNhP1siwhpj9//tHeSD67+soYQRymRSrnLlSyDcjt/4p1/?= =?us-ascii?Q?/9SsSJsiwt0J38ZW1/h7WD8lnbEN0jiIzulHlDU7yaGo86ziM369H0F37HAR?= =?us-ascii?Q?Cwe9HCL84ybN3nprL2mhMRanL8zTAAhiFuKvaY5W6HuZ7d46c/0Onadzdzvp?= =?us-ascii?Q?AR1jJ6yboqbHWV9Mg0LvImI3toN8niPA75hoJpmlDj7UvIR182n4dFtXK59A?= =?us-ascii?Q?8WdlfadoXXEiprbo3xedRjg91880eN7+PpCi3bM17TqRRxN3Ut/TDO5WjJF8?= =?us-ascii?Q?UAxhw=3D=3D?=
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR10MB0243; 6:zswJvWhozz8P7QeGn8PgaQHVpFTMiNk+1Fhc5fcQHe?= =?us-ascii?Q?0xQkvxOjPF1OhoYtZE8fyEhIUWAex/hPS+w5qoZKtzEu+gmgH/CBGLvnw8FX?= =?us-ascii?Q?bqGAZMN57RKzyVvMeRX4Re+fZMyPyfXZ1KJx5RBOXUNI+M9668/YhzdAUiYH?= =?us-ascii?Q?DYIHf++ZKP1zPBuE63iHJTpwMHf8WIFM8CHJvuITvzDo8F8YcaHQNSlCyCYB?= =?us-ascii?Q?RosG+nBA+0kmcXelZJSWM3OEdWQbhBenDBsukcq8ZrNBs0tP999/h98jCZrH?= =?us-ascii?Q?Dxdvab9TIqwltlLIHPwOSVKaBFa+DsQdmptNzz110gaE7Jjt0/9ZR+y7JxtP?= =?us-ascii?Q?mGa5Gc1fNDqX252F32nxc8YCco0Mls545P1/BVNry42y8WwTPcQud6GYJrOR?= =?us-ascii?Q?SqM6AY4NaKwKBM//IY1PYz7+ywkbgxLVh8FSpMI0DhmsTPY6/fwR/IGG0gMf?= =?us-ascii?Q?MeyDpJb6OyWURrAytEtQRfKzHbNjJBY1tmDn9KViOb07WaRHvZ07B3ywoZhe?= =?us-ascii?Q?c/RGvPApM0zJ26GVAK49tCo3DNgH9BG/O1RHA5QP0IicIwFxoPbSe1taIw10?= =?us-ascii?Q?qUySSZTjMx6PKsfCJAItZ+LkHjWlMGPQ/1ZKJPwRspdCssshvqsH76rDMqwJ?= =?us-ascii?Q?xiTeKqQyzknXZZbXkGTG3BSEVvTJv2VIJ/CpO3JCiBlY8IfR/AVsel2pCjl2?= =?us-ascii?Q?DFTHnodEKjQeaJsjPaWu04Kzi1ZrVI5lmM8vS6yLN+9BcJG0kPCECtBpS4Gd?= =?us-ascii?Q?qHxt5ZvC8E8Dg/dEj/DrP7dR7EZzSjB4KOYEFOXr3ILpRu/cKLiGZ6eHpzP9?= =?us-ascii?Q?6FPLcNeVK6JZC2JmIxvWEJCoiGEK3EX1DG6TmNo1ku1dHK2Us68KFtMAdTAF?= =?us-ascii?Q?mcHzb1IgfJAAMpMWguwXruvJXup1X9erjoxsSgND2E6X4OhRcjGgfa39C+dV?= =?us-ascii?Q?5WtwLcI+WddphVUF7q/9MqNWEbbxxFj27QM++VWTcZ1+8e1RoB/LmbKSfZcc?= =?us-ascii?Q?o=3D?=
X-Microsoft-Exchange-Diagnostics: 1; BN3PR10MB0243; 5:7wEfw51KE0Fr9leAJJIUadKkgZwG+KyiM+tYkvmL6z8mAjYZLevHNM2h6SXG37P9TNiGAao4bEA+NODBKgM0kE3vxi2pz0alk3XGtbFlpRXUJC4YRwh4xxgpw8b2iRZfv/dl7SPxCBeSt5M79ZagR/AXRlcMW9ecbbj7s7yn3xHdRl9Jwz/xhQUb0Zfy2oXnNaSQNgSH3Sft68XWUHpivs4DDY6oD1oRgtrFXFf8SLaIe4At0F2sXw1dXE86bwjtXTfuOU8S/J7S8K8m4wNrgu+nrHqEUXA4up5RqXfjhPEZyBCblTuffHSsjbL2k/ryvgj4I6DyIeFp0UnWLy/kmed7b7v8QrdESZ+U6eqSaCsB0F1PIL5UEA/MkFfB87EPgYZwonq8UgvQMbrmKLh8PKU7Zf2+VYBqp/lV9HoflmR2g38BTWy/g7hRc9iOnI97c26zsLfeD4PQOi+BoqX8zdqeL2XSRtMwzqF4QqnjKvgfx1ROYBLPmxvg1VwYN4ng; 24:Dv7/WB5LSbEcPk+SNVyUXRfcX9l+W+giMQAAt8Lhbt5wbNSpvBCBQUY3xqWbh/vq2i+dqb4ygkRyfrvOSZlO/QFc08YQeznPNuh9Tk7U/nQ=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1; BN3PR10MB0243; 7:Lj6fbyhouCMmNosVyZdF2xPZbwrURt9Yp64KCzgKB/e8GXfrxhysMvxCOIFQncSL8X4dqv+tmlrlfNg62soIhiozBcB/mocAmDB2qKySg+1NABx4w/+BHkODU6msyw36RNGUkn9yjIsCRa7YBowolh2ZL2fa91QPOb25u1O5LrbdIALnW+FC2yTuVb+TOxrp0SiRqsRuZ0rAXqwVM3hB4MIDY0+8gEVQIsavo6XlBsE9duPHJac75PRL9DCtVW160uQtQlZqqjmWPO3w+w7k0zikvkEQbdgIBfAoLtdZcw5vLDnqaoamNudm9qQXY8R35dDcBuHtOdgDrO1EaC4AVPiIuecop4DdeLzShiM4USLqFeyOmhbZlcpM4qts5pzxir8WT09kS/Z1LAEQBgU8iKIfSqS5SrfRDiijGBuqy8T2qcMBlBSBWuvDjT+HBWw8IMROSFqG1NwJJeCA3sWENd1094qj5S8kh4ilgDlhsTyMUxEy1ZMJa6bRQLR9dlKS3RIJZ84FCG3EJ0Gg1VMQhqYt0NGuCIzuqqG4CcBxmcNq1PecdTePCrME3hKoWSdexT+zALjqLRtXKpqNpaoxSqD/HT1LfBYPE+0VSX1PFA7aCxZ9zZftJnujcbN65C10EUKR3w3gXJQvE2fMahTRY2lKjlrcAA0WDIiKS/9BDhKFbN27EXsG1lBg3FAMPrQEaPaSgQV7wEq84ImB4lbepn7IbhNKsfHUFAQpzRPepk5CjBvQabUcCeY21MQr94QxLVybHbE2GtSC0c2RtXwtA19Yv+XZUfPhNX9Y7lx71iY=
X-OriginatorOrg: infinera.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2017 07:04:09.9507 (UTC)
X-MS-Exchange-CrossTenant-Id: 285643de-5f5b-4b03-a153-0ae2dc8aaf77
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=285643de-5f5b-4b03-a153-0ae2dc8aaf77; Ip=[204.128.141.23];  Helo=[owa.infinera.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR10MB0243
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/HKNyzvErzxhHMnBlD2R2U7Ncnzk>
Subject: Re: [bmwg] Call For Adoption: draft-kishjac-bmwg-evpntest-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 07:04:14 -0000

Support.

Regards,
Iftekhar


From nobody Thu Jun 22 05:12:20 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BCE5128BA2; Thu, 22 Jun 2017 05:12:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Level: 
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PxysHnCr5mC0; Thu, 22 Jun 2017 05:12:16 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A4B8F126D45; Thu, 22 Jun 2017 05:12:16 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v5MC5nB7030931; Thu, 22 Jun 2017 08:12:11 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by mx0a-00191d01.pphosted.com with ESMTP id 2b82mjp577-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jun 2017 08:12:11 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5MCCAd9115301; Thu, 22 Jun 2017 07:12:10 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5MCC2TF115268 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 22 Jun 2017 07:12:03 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor); Thu, 22 Jun 2017 12:11:58 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5MCBwXV032571; Thu, 22 Jun 2017 07:11:58 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5MCBmal031567; Thu, 22 Jun 2017 07:11:48 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 33AE5E05FF; Thu, 22 Jun 2017 08:11:47 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 22 Jun 2017 08:11:47 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Lucien <lucien.avramov@gmail.com>, Adam Roach <adam@nostrum.com>
CC: The IESG <iesg@ietf.org>, "draft-ietf-bmwg-dcbench-terminology@ietf.org" <draft-ietf-bmwg-dcbench-terminology@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
Thread-Index: AQHS6xUgHssWT8+KCUWtOTUYp7fMs6IwyZbg
Date: Thu, 22 Jun 2017 12:11:46 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FE33B0@njmtexg5.research.att.com>
References: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com> <CAArZqeUssxdEZrauVB=yn7i+8uBopmUvwyDu-jhHNOOvWqKM5w@mail.gmail.com>
In-Reply-To: <CAArZqeUssxdEZrauVB=yn7i+8uBopmUvwyDu-jhHNOOvWqKM5w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF25FE33B0njmtexg5researc_"
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-22_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706220209
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/DORC9sZNrpOakjmzBG-HnAotrbw>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 12:12:18 -0000

--_000_4D7F4AD313D3FC43A053B309F97543CF25FE33B0njmtexg5researc_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SGkgTHVjaWVuLA0KSSBjYXVnaHQgYSBzbWFsbCBtaXgtdXAgaW4geW91ciBsYXRlLW5pZ2h0IHJl
cGx5IHRvIEFkYW0sIGJlbG93Lg0KVGhhbmtzIGZvciB5b3VyIG1hc3NpdmUgZWRpdGluZyBlZmZv
cnRzIQ0KQWwNCg0KRnJvbTogTHVjaWVuIFttYWlsdG86bHVjaWVuLmF2cmFtb3ZAZ21haWwuY29t
XQ0KU2VudDogVGh1cnNkYXksIEp1bmUgMjIsIDIwMTcgMTowNSBBTQ0KVG86IEFkYW0gUm9hY2gN
CkNjOiBUaGUgSUVTRzsgZHJhZnQtaWV0Zi1ibXdnLWRjYmVuY2gtdGVybWlub2xvZ3lAaWV0Zi5v
cmc7IFNhcmFoIEJhbmtzOyBibXdnLWNoYWlyc0BpZXRmLm9yZzsgYm13Z0BpZXRmLm9yZw0KU3Vi
amVjdDogUmU6IEFkYW0gUm9hY2gncyBObyBPYmplY3Rpb24gb24gZHJhZnQtaWV0Zi1ibXdnLWRj
YmVuY2gtdGVybWlub2xvZ3ktMTc6ICh3aXRoIENPTU1FTlQpDQoNCkhpIEFkYW0sDQoNCnRoYW5r
cyBmb3Igc3BlbmRpbmcgdGltZSB0byByZXZpZXcgYW5kIHByb3ZpZGUgbW9yZSBjb21tZW50cyB0
byB0aGUgbGlzdC4NCkkgYWRkcmVzc2VkIGJvdGggb2YgeW91ciBlbWFpbHMgd2l0aCB0aGUgbGF0
ZXN0IHB1YmxpY2F0aW9uIGkganVzdCBkaWQuDQoNClBsZWFzZSBzZWUgaW5saW5lIGZvciBkZXRh
aWxzOg0K4oCmc3R1ZmYgZGVsZXRlZOKApg0KDQogID09IE1pc3NpbmcgUmVmZXJlbmNlOiAnUkZD
NTQ4MScgaXMgbWVudGlvbmVkIG9uIGxpbmUgMjg1LCBidXQgbm90IGRlZmluZWQNCg0KUmVtb3Zl
ZCB0aGlzIFJGQyBhcyB3ZSBubyBsb25nZXIgcmVmZXJlbmNlIGl0DQoNCiAgPT0gVW51c2VkIFJl
ZmVyZW5jZTogJ1JGQzU4NDEnIGlzIGRlZmluZWQgb24gbGluZSA3MzIsIGJ1dCBubyBleHBsaWNp
dA0KICAgICByZWZlcmVuY2Ugd2FzIGZvdW5kIGluIHRoZSB0ZXh0DQpSZW1vdmVkIHRoaXMgUkZD
IGFzIHdlIG5vIGxvbmdlciByZWZlcmVuY2UgaXQNCg0KW0FDTV0NClRoaXMgaXMgYW5vdGhlciBj
YXNlIG9mIHRyYW5zcG9zZWQgUkZDIG51bWJlcnMuDQpUaGUgZHJhZnQgcmVmZXJlbmNlcyBSRkMg
NTQ4MSBpbiBzZWN0aW9uIDMuMSAodGVybXMpIGFuZCB0aGlzIGlzIGNvcnJlY3QhDQpCdXQgdGhl
IG9yaWdpbmFsIHJlZmVyZW5jZSB3YXMgZm9yIOKAnDU4NDHigJ0sIHdoaWNoIHdhc27igJl0DQp1
c2VkIGluIHRoZSB0ZXh0LiAgWW91IG5lZWQgdG8gYWRkIGEgcmVmZXJlbmNlIHRvDQpSRkMgNTQ4
MSBpbiB0aGUgbmV4dCB1cGRhdGUuDQpUSEFOS1MhDQoNCiAgKiogT2Jzb2xldGUgbm9ybWF0aXZl
IHJlZmVyZW5jZTogUkZDIDI1NTQgKHJlZi4gJ1JGQzI1NDQnKSAoT2Jzb2xldGVkIGJ5DQogICAg
IFJGQyA0OTU0KQ0KSSBhbSBzb3JyeSB3ZSB1c2UgUkZDIDI1NDQsIG5vdCAyNTU0IHNvIHdlIGFy
ZSBnb29kIGFjdHVhbGx5LCBpdHMgbm90IG9ic29sZXRlZCBhdCBhbGwuDQoNCg0K

--_000_4D7F4AD313D3FC43A053B309F97543CF25FE33B0njmtexg5researc_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTQgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl
PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6
Q2FsaWJyaTsNCglwYW5vc2UtMToyIDE1IDUgMiAyIDIgNCAzIDIgNDt9DQpAZm9udC1mYWNlDQoJ
e2ZvbnQtZmFtaWx5OlRhaG9tYTsNCglwYW5vc2UtMToyIDExIDYgNCAzIDUgNCA0IDIgNDt9DQov
KiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KcC5Nc29Ob3JtYWwsIGxpLk1zb05vcm1hbCwgZGl2Lk1z
b05vcm1hbA0KCXttYXJnaW46MGluOw0KCW1hcmdpbi1ib3R0b206LjAwMDFwdDsNCglmb250LXNp
emU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLCJzZXJpZiI7fQ0KYTps
aW5rLCBzcGFuLk1zb0h5cGVybGluaw0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJY29sb3I6
Ymx1ZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30NCmE6dmlzaXRlZCwgc3Bhbi5Nc29I
eXBlcmxpbmtGb2xsb3dlZA0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJY29sb3I6cHVycGxl
Ow0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0Kc3Bhbi5FbWFpbFN0eWxlMTcNCgl7bXNv
LXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3IjsN
Cgljb2xvcjpibGFjazt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQt
b25seTsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO30NCkBwYWdlIFdvcmRT
ZWN0aW9uMQ0KCXtzaXplOjguNWluIDExLjBpbjsNCgltYXJnaW46MS4waW4gMS4waW4gMS4waW4g
MS4waW47fQ0KZGl2LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQotLT48L3N0
eWxlPjwhLS1baWYgZ3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVkZWZhdWx0cyB2OmV4dD0iZWRp
dCIgc3BpZG1heD0iMTAyNiIgLz4NCjwveG1sPjwhW2VuZGlmXS0tPjwhLS1baWYgZ3RlIG1zbyA5
XT48eG1sPg0KPG86c2hhcGVsYXlvdXQgdjpleHQ9ImVkaXQiPg0KPG86aWRtYXAgdjpleHQ9ImVk
aXQiIGRhdGE9IjEiIC8+DQo8L286c2hhcGVsYXlvdXQ+PC94bWw+PCFbZW5kaWZdLS0+DQo8L2hl
YWQ+DQo8Ym9keSBsYW5nPSJFTi1VUyIgbGluaz0iYmx1ZSIgdmxpbms9InB1cnBsZSI+DQo8ZGl2
IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9
ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29s
b3I6YmxhY2siPkhpIEx1Y2llbiw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD
b3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+SSBjYXVnaHQgYSBzbWFsbCBtaXgtdXAgaW4g
eW91ciBsYXRlLW5pZ2h0IHJlcGx5IHRvIEFkYW0sIGJlbG93LjxvOnA+PC9vOnA+PC9zcGFuPjwv
cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv
bnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJsYWNrIj5UaGFua3MgZm9y
IHlvdXIgbWFzc2l2ZSBlZGl0aW5nIGVmZm9ydHMhPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p
bHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPkFsPG86cD48L286cD48L3Nw
YW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4w
cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPjxvOnA+
Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxkaXYgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1s
ZWZ0OnNvbGlkIGJsdWUgMS41cHQ7cGFkZGluZzowaW4gMGluIDBpbiA0LjBwdCI+DQo8ZGl2Pg0K
PGRpdiBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLXRvcDpzb2xpZCAjQjVDNERGIDEuMHB0O3Bh
ZGRpbmc6My4wcHQgMGluIDBpbiAwaW4iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGI+PHNwYW4g
c3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGFob21hJnF1b3Q7LCZx
dW90O3NhbnMtc2VyaWYmcXVvdDsiPkZyb206PC9zcGFuPjwvYj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1zZXJp
ZiZxdW90OyI+IEx1Y2llbiBbbWFpbHRvOmx1Y2llbi5hdnJhbW92QGdtYWlsLmNvbV0NCjxicj4N
CjxiPlNlbnQ6PC9iPiBUaHVyc2RheSwgSnVuZSAyMiwgMjAxNyAxOjA1IEFNPGJyPg0KPGI+VG86
PC9iPiBBZGFtIFJvYWNoPGJyPg0KPGI+Q2M6PC9iPiBUaGUgSUVTRzsgZHJhZnQtaWV0Zi1ibXdn
LWRjYmVuY2gtdGVybWlub2xvZ3lAaWV0Zi5vcmc7IFNhcmFoIEJhbmtzOyBibXdnLWNoYWlyc0Bp
ZXRmLm9yZzsgYm13Z0BpZXRmLm9yZzxicj4NCjxiPlN1YmplY3Q6PC9iPiBSZTogQWRhbSBSb2Fj
aCdzIE5vIE9iamVjdGlvbiBvbiBkcmFmdC1pZXRmLWJtd2ctZGNiZW5jaC10ZXJtaW5vbG9neS0x
NzogKHdpdGggQ09NTUVOVCk8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+SGkgQWRhbSw8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiPnRoYW5rcyBmb3Igc3BlbmRpbmcgdGltZSB0byByZXZpZXcgYW5kIHByb3Zp
ZGUgbW9yZSBjb21tZW50cyB0byB0aGUgbGlzdC48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkkgYWRkcmVzc2VkIGJvdGggb2YgeW91ciBlbWFpbHMg
d2l0aCB0aGUgbGF0ZXN0IHB1YmxpY2F0aW9uIGkganVzdCBkaWQuJm5ic3A7PG86cD48L286cD48
L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpw
PjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPlBsZWFzZSBzZWUgaW5s
aW5lIGZvciBkZXRhaWxzOjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxk
aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48aT48c3BhbiBzdHlsZT0iY29sb3I6YmxhY2si
PuKApnN0dWZmIGRlbGV0ZWTigKY8L3NwYW4+PG86cD48L286cD48L2k+PC9iPjwvcD4NCjwvZGl2
Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0ND
Q0MgMS4wcHQ7cGFkZGluZzowaW4gMGluIDBpbiA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJn
aW4tcmlnaHQ6MGluIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCiZuYnNwOyA9PSBNaXNz
aW5nIFJlZmVyZW5jZTogJ1JGQzU0ODEnIGlzIG1lbnRpb25lZCBvbiBsaW5lIDI4NSwgYnV0IG5v
dCBkZWZpbmVkPG86cD48L286cD48L3A+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj5SZW1vdmVkIHRoaXMgUkZDIGFzIHdlIG5vIGxvbmdlciByZWZlcmVuY2Ug
aXQmbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRl
cjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowaW4gMGluIDBp
biA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGluIj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxicj4NCiZuYnNwOyA9PSBVbnVzZWQgUmVmZXJlbmNlOiAnUkZDNTg0MScgaXMg
ZGVmaW5lZCBvbiBsaW5lIDczMiwgYnV0IG5vIGV4cGxpY2l0PGJyPg0KJm5ic3A7ICZuYnNwOyAm
bmJzcDtyZWZlcmVuY2Ugd2FzIGZvdW5kIGluIHRoZSB0ZXh0PG86cD48L286cD48L3A+DQo8L2Js
b2NrcXVvdGU+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+UmVtb3ZlZCB0aGlzIFJGQyBh
cyB3ZSBubyBsb25nZXIgcmVmZXJlbmNlIGl0ICZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+
DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0ND
QyAxLjBwdDtwYWRkaW5nOjBpbiAwaW4gMGluIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdp
bi1yaWdodDowaW4iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0b206
MTIuMHB0Ij48c3BhbiBzdHlsZT0iY29sb3I6YmxhY2siPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFu
PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+
PGI+PGk+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291
cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPltBQ01dDQo8bzpwPjwvbzpwPjwvc3Bhbj48L2k+
PC9iPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBw
dCI+PGI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291
cmllciBOZXcmcXVvdDs7Y29sb3I6YmxhY2siPlRoaXMgaXMgYW5vdGhlciBjYXNlIG9mIHRyYW5z
cG9zZWQgUkZDIG51bWJlcnMuPG86cD48L286cD48L3NwYW4+PC9iPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+PGI+PHNwYW4gc3R5bGU9ImZv
bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29sb3I6
YmxhY2siPlRoZSBkcmFmdCByZWZlcmVuY2VzIFJGQyA1NDgxIGluIHNlY3Rpb24gMy4xICh0ZXJt
cykgYW5kIHRoaXMgaXMgY29ycmVjdCE8bzpwPjwvbzpwPjwvc3Bhbj48L2I+PC9wPg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0b206MTIuMHB0Ij48Yj48c3BhbiBzdHls
ZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztj
b2xvcjpibGFjayI+QnV0IHRoZSBvcmlnaW5hbCByZWZlcmVuY2Ugd2FzIGZvciDigJw1ODQx4oCd
LCB3aGljaCB3YXNu4oCZdDxvOnA+PC9vOnA+PC9zcGFuPjwvYj48L3A+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIiBzdHlsZT0ibWFyZ2luLWJvdHRvbToxMi4wcHQiPjxiPjxzcGFuIHN0eWxlPSJmb250
LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NvdXJpZXIgTmV3JnF1b3Q7O2NvbG9yOmJs
YWNrIj51c2VkIGluIHRoZSB0ZXh0LiZuYnNwOyBZb3UgbmVlZCB0byBhZGQgYSByZWZlcmVuY2Ug
dG88bzpwPjwvbzpwPjwvc3Bhbj48L2I+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9
Im1hcmdpbi1ib3R0b206MTIuMHB0Ij48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm
b250LWZhbWlseTomcXVvdDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjpibGFjayI+UkZDIDU0ODEg
aW4gdGhlIG5leHQgdXBkYXRlLg0KPG86cD48L286cD48L3NwYW4+PC9iPjwvcD4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+PGI+PHNwYW4gc3R5bGU9
ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q291cmllciBOZXcmcXVvdDs7Y29s
b3I6YmxhY2siPlRIQU5LUyE8bzpwPjwvbzpwPjwvc3Bhbj48L2I+PC9wPg0KPHAgY2xhc3M9Ik1z
b05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0b206MTIuMHB0Ij48YnI+DQombmJzcDsgKiogT2Jz
b2xldGUgbm9ybWF0aXZlIHJlZmVyZW5jZTogUkZDIDI1NTQgKHJlZi4gJ1JGQzI1NDQnKSAoT2Jz
b2xldGVkIGJ5PGJyPg0KJm5ic3A7ICZuYnNwOyAmbmJzcDtSRkMgNDk1NCk8bzpwPjwvbzpwPjwv
cD4NCjwvYmxvY2txdW90ZT4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5JIGFtIHNvcnJ5
IHdlIHVzZSBSRkMgMjU0NCwgbm90IDI1NTQgc28gd2UgYXJlIGdvb2QgYWN0dWFsbHksIGl0cyBu
b3Qgb2Jzb2xldGVkIGF0IGFsbC4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8
L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2JvZHk+DQo8L2h0bWw+DQo=

--_000_4D7F4AD313D3FC43A053B309F97543CF25FE33B0njmtexg5researc_--


From nobody Thu Jun 22 05:13:50 2017
Return-Path: <suresh.krishnan@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 928261293EB; Thu, 22 Jun 2017 05:13:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b941dY_KVmUv; Thu, 22 Jun 2017 05:13:46 -0700 (PDT)
Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 11C8D1293D8; Thu, 22 Jun 2017 05:13:46 -0700 (PDT)
Received: by mail-qt0-x22a.google.com with SMTP id v20so9953079qtg.1; Thu, 22 Jun 2017 05:13:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=86RmWRaJMq9l8u0iHH6TR6YZ4UBXmoD4eUzuLq5mfEQ=; b=NRBhCtUtkR6LwDfsNHT2fe+0q3+1neUw71j4ZV/eNYh69FZ+Meo+RKlE0AfdxkQpzT FGHo8L5p9S9583Tfi23xn5nO4eUJG9AFvcJv4b4E1yn0C6JSKdpoXA+mDM3/puzvAVSL OWLZJQ13nqv1Ek/vSbGELaW0o7rjaOr/81Z+RMhjbACZp/xgqmRDRCsiiWE6fagv2bpr fxpK95H/ejLEl7IyKRUhfMfrktPGabc1OmWTSdSn/F0UZYJI2EVScgWn7qDcdhMu/euk i9tum1JhThWpAtGuh4wzGc8p9xz4qzEnd6lKecrMvijVDIz79eP413mA0dUVjMOzLibc IhDg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=86RmWRaJMq9l8u0iHH6TR6YZ4UBXmoD4eUzuLq5mfEQ=; b=X2Tze8L3EPnoAIvJTGY3q+zm4GGO0+Q8i2Om9uyXgYRc48/hWQp0jDWB0RBJmZ95vq dNDaQhznMgMMN8KVZIg1QgfFhparFS8UqilvYIPRIMx7hGx2Axr7yEKqCZy0wTqsm7Ec FOk08QT6zlRwOvqZgkZlc/ByysUguhGw3sDkXVkFNX/+ZkK9kSQ9GNySV2ZItP1mpSAy qgFt6jC6UPSjwfP7+QjPAiakbTw/YQ92sSHmc4Y+CmMrnAq6DUmqF+sgEL8zGhV3qO/3 VI+GDH+g6/Vl/CtS1b4TJA+ePDaRC5ENXnR+5sw2xRiSsK7wZINGESrFW1ACmBVoom1C WTNw==
X-Gm-Message-State: AKS2vOyb5hQL3UZ8adYobQ7ZwJA/ojaba/ay7JPUfTOwMOSYcpzNSK4+ naeAZRlSOKfx6wBn4jpnZywpRdY0rg==
X-Received: by 10.237.45.196 with SMTP id i62mr2478045qtd.194.1498133625200; Thu, 22 Jun 2017 05:13:45 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.55.159.211 with HTTP; Thu, 22 Jun 2017 05:13:44 -0700 (PDT)
In-Reply-To: <CAArZqeVNE9pnva8xf9XoOEUHAWBn60+xvgCDbzHvQD55EmN+FQ@mail.gmail.com>
References: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com> <CAArZqeX9vVNS0iawMeV9-U5ZD5c1OA-eLjHCzPz0kKrRDsBRUg@mail.gmail.com> <CAArZqeVNE9pnva8xf9XoOEUHAWBn60+xvgCDbzHvQD55EmN+FQ@mail.gmail.com>
From: Suresh Krishnan <suresh.krishnan@gmail.com>
Date: Thu, 22 Jun 2017 17:43:44 +0530
Message-ID: <CA+MHpBpeHvw02VgmBVN+5rXvTQpLZdN_F5f6a8Ez_xcaTgrq2g@mail.gmail.com>
To: Lucien <lucien.avramov@gmail.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-methodology@ietf.org,  Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/6d5yQ2zMzt3rWtQsmEJIVZIEKak>
Subject: Re: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 12:13:48 -0000

Hi Lucien,

On Thu, Jun 22, 2017 at 12:12 AM, Lucien <lucien.avramov@gmail.com> wrote:
> I meant to say the delta of power when the data center switch is iddle and
> at 100% of capacity is marginal.

Right, but that was not exactly what I was looking for. I was looking
for performance benchmarking of the switches in based on throughput
per unit of energy (e.g. Gbps/W). Some switches might be more
efficient in this metric than others. e.g. an Ethernet switch ASIC
will most likely be more efficient in this metric than an x86 based
softswitch (at the cost of flexibility).

Thanks
Suresh


From nobody Thu Jun 22 05:15:32 2017
Return-Path: <suresh.krishnan@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 14B01126D45; Thu, 22 Jun 2017 05:15:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5CKRIme8-Xh7; Thu, 22 Jun 2017 05:15:29 -0700 (PDT)
Received: from mail-qt0-x232.google.com (mail-qt0-x232.google.com [IPv6:2607:f8b0:400d:c0d::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 27D09128BA2; Thu, 22 Jun 2017 05:15:29 -0700 (PDT)
Received: by mail-qt0-x232.google.com with SMTP id v20so9987804qtg.1; Thu, 22 Jun 2017 05:15:29 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=TbspT24DZs2S42RwAkakXRmUT4Ggix77a2CP8TbK/wg=; b=LjuQB1xuxI8EKRR5P4su5ORnR2/poNODUicIUbxwbQGPx1rGwwNtNEVurkyGVah4uX n/hODpML2r8Diw9n3zV4ptFaZVb3Dcq3Hx07abtGmgKJFo6GQquywWG2ilum6LIf73Xr NmDEb9Lbj2Q61eE6OZja4Ct+zO/FjXFLVKDZEdqzdY0OVoYOoc3hIKfT0JMtcRltthTU 9KGTzA16dY8tsO5lI4XiQHQPX5JiCyl56wLJ/UY7m8m0lJa3qZis3f2tnrYSISAXL8a3 3zZBGOv7KYXUtjNi9ilo4WlmuiZTGqZXMalmYgtLEgzmBCefHQlvA6U+DxlXlgv6Xt39 n+Ag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=TbspT24DZs2S42RwAkakXRmUT4Ggix77a2CP8TbK/wg=; b=Ss06KlL6Suc9q7lYdLpJ1k/y8VxV78Rl9ei1Qd5VSIuQgfZMUug1DOJZhjOP5zXKnn MTfpW14+163RLDBkXPLspTjerQh7g4BLDlfyO0K8GE/s7y4kiRLceSE2pPyDxC+dfdXo Baw9hlEdHeCb4DSDNVrefDaDI3TJDBCEzXkGHFX2GSD0HKZ1ycZCG++49EguczDQ9PyA TbvSTgN+se1LqulBVzbe/BL1pNH1WAz8GI0KKfHC81acLovTq9aLeVF0VPZaXlW1mlj6 n7x0jGdePMHWogID91Y5pFW2S6nCC8fhzzPFLEst2G+IoCWMYQKw3vnTRIGT9h/bq7vG pBWg==
X-Gm-Message-State: AKS2vOz/HvWbhIcr/8NDw9u7uhx95b8NhaWx3hX/IUpgWgxRoBIKaBn+ CLFVWMmPk3Ge+SOtM5z7ulOKQp8Hfw==
X-Received: by 10.237.35.178 with SMTP id j47mr2566502qtc.215.1498133728223; Thu, 22 Jun 2017 05:15:28 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.55.159.211 with HTTP; Thu, 22 Jun 2017 05:15:27 -0700 (PDT)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FE2F16@njmtexg5.research.att.com>
References: <149806880921.15906.12225670606905404415.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FE2F16@njmtexg5.research.att.com>
From: Suresh Krishnan <suresh.krishnan@gmail.com>
Date: Thu, 22 Jun 2017 17:45:27 +0530
Message-ID: <CA+MHpBpajfNwpAsRbY4aoBpTY9OYO5dmYmuqsOV7d1EexG55nQ@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Cc: The IESG <iesg@ietf.org>, "draft-ietf-bmwg-dcbench-methodology@ietf.org" <draft-ietf-bmwg-dcbench-methodology@ietf.org>,  "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>,  "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/pv-R9YZJ0GSIHn5Uuu4J2jKtXQo>
Subject: Re: [bmwg] Suresh Krishnan's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 12:15:31 -0000

Hi Al,

On Thu, Jun 22, 2017 at 12:12 AM, MORTON, ALFRED C (AL)
<acmorton@att.com> wrote:
>> -----Original Message-----
>> From: Suresh Krishnan [mailto:suresh.krishnan@gmail.com]
>> Sent: Wednesday, June 21, 2017 2:13 PM
> ...
>>
>> Suresh Krishnan has entered the following ballot position for
>> draft-ietf-bmwg-dcbench-methodology-16: No Objection
>>
> ...
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>
>> Not sure if the WG discussed this, but it would have been interesting to
>> have
>> some energy efficiency related metrics as the draft is datacenter
>> focused and
>> energy efficiency is an important consideration in datacenters.
>>
> [ACM]
> Hi Suresh,
>
> We had a draft/proposed WI on this topic, but we couldn't find sufficient
> support and expertise to review and complete a high quality
> document, so the proponents moved-on.

Thanks. It is unfortunate, but it certainly clarifies things for me.

>
> I agree, it's an important area to consider if the expertise
> can be gathered in BMWG.

Great.

Regards
Suresh


From nobody Thu Jun 22 05:55:02 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 95B8E12941D; Thu, 22 Jun 2017 05:54:49 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149813608958.30641.13960506019388458099@ietfa.amsl.com>
Date: Thu, 22 Jun 2017 05:54:49 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/zT1DzuBn_MiYgK1OKWSiZhnLFRY>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-19.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 12:54:50 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-19.txt
	Pages           : 18
	Date            : 2017-06-22

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to performance
evaluations of data center network equipment. This document establishes
the important concepts for benchmarking network switches and routers in
the data center and, is a pre-requisite to the test methodology
publication [draft-ietf-bmwg-dcbench-methodology]. Many of these terms
and methods may be applicable to network equipment beyond this
publication's scope as the technologies originally applied in the data
center are deployed elsewhere.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-19
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-19

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-19


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun 22 05:56:29 2017
Return-Path: <lucien.avramov@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 346E11287A0; Thu, 22 Jun 2017 05:56:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CNLFWHO_E8BX; Thu, 22 Jun 2017 05:56:20 -0700 (PDT)
Received: from mail-yw0-x22f.google.com (mail-yw0-x22f.google.com [IPv6:2607:f8b0:4002:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B3DC1275C5; Thu, 22 Jun 2017 05:56:20 -0700 (PDT)
Received: by mail-yw0-x22f.google.com with SMTP id 63so5562033ywr.0; Thu, 22 Jun 2017 05:56:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=kY8tO2F5iqZksHab91yEqM/N94XfKYxqLeFbakvsTa0=; b=Al4vE/VXqkjlzQ0gRN4/MUQBvWC2vlu68wAi0ekMlPrj9cJa7edk3VfbGe9XUFEHkM Fs4rigvZyxA/kx6+ikB1cEJ6FusxZdRJj4R6NOiZwRMz2OfYC7Do2efpNKXRueI0bJj0 1O6BZSwSk+4czqTDRqt48e2VtcHs1IbxETJ6dUfFXxRdBRxkRaJmJvJkQ5Wp9Ym0oCgM IOh71pELSACxqi+fw41VCE7+HzysekI1q9Wce7f/Q8fgHSpPzDIoFjluzRZ2OdCY0rvX nOu/R107pML7TyxQb2HoHVb3J5j6nPI6j6L5TMdH7eN1NHnKPQEsKcqkE+8DHCe7sfsI VbSQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=kY8tO2F5iqZksHab91yEqM/N94XfKYxqLeFbakvsTa0=; b=Q+HEdQSIdswWlNsWT5IpIpyO3VirpeRgfJ047k7iRrHtTTl3tJ5TJ5FN7qSEc1GtNg iikkvzLZDIpoSBmCZOlKJDowIuZYSvnuQ73lbncQlRLIf0pgDryYqjkHow7ZtN0Xxe++ 8CazCy9qFujHKKHlm6PQUwyNMO7M9cnLG/GjJK+90LSIXYEePuh68TOsqX1SQrB9rXbv GDJDOSUqOLsYcy0SQANRwaevmQytrHw6zwhR6HcJmM6+o/hDg1PISG2OzqQDqgt8Pehv jWHeVD9hnpnnx4i17i8c2LiGOGO/66qbNO9MAV9p8MRfqy3/3JfCSqdsgyH3jk5OIWvy de8A==
X-Gm-Message-State: AKS2vOwIR8i4x/vnrDHjO+7fndkOEPLD2d9JokgJzqHqdA/rDReq3Z5P IPwYL6qifLE4kazUZwv2iR+Oq8fUbA==
X-Received: by 10.129.80.193 with SMTP id e184mr1739598ywb.44.1498136179355; Thu, 22 Jun 2017 05:56:19 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.145.2 with HTTP; Thu, 22 Jun 2017 05:56:18 -0700 (PDT)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25FE33B0@njmtexg5.research.att.com>
References: <149809634844.30729.7905314498887048672.idtracker@ietfa.amsl.com> <CAArZqeUssxdEZrauVB=yn7i+8uBopmUvwyDu-jhHNOOvWqKM5w@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF25FE33B0@njmtexg5.research.att.com>
From: Lucien <lucien.avramov@gmail.com>
Date: Thu, 22 Jun 2017 05:56:18 -0700
Message-ID: <CAArZqeVvdTsTxmZsR2nNt-c_-29ayX+R+4irGkju4uYsAad_uA@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Cc: Adam Roach <adam@nostrum.com>, The IESG <iesg@ietf.org>,  "draft-ietf-bmwg-dcbench-terminology@ietf.org" <draft-ietf-bmwg-dcbench-terminology@ietf.org>,  Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>,  "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="001a1147e4e2cbcf3d05528c022d"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/js3OCmCBDmGeiVx59V7NdFBQ9ig>
Subject: Re: [bmwg] Adam Roach's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 12:56:22 -0000

--001a1147e4e2cbcf3d05528c022d
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Al!

Great catch thank you, I just fixed it and published -19 referencing to the
proper RFC number!

Thanks,
Lucien

On Thu, Jun 22, 2017 at 5:11 AM, MORTON, ALFRED C (AL) <acmorton@att.com>
wrote:

> Hi Lucien,
>
> I caught a small mix-up in your late-night reply to Adam, below.
>
> Thanks for your massive editing efforts!
>
> Al
>
>
>
> *From:* Lucien [mailto:lucien.avramov@gmail.com]
> *Sent:* Thursday, June 22, 2017 1:05 AM
> *To:* Adam Roach
> *Cc:* The IESG; draft-ietf-bmwg-dcbench-terminology@ietf.org; Sarah
> Banks; bmwg-chairs@ietf.org; bmwg@ietf.org
> *Subject:* Re: Adam Roach's No Objection on draft-ietf-bmwg-dcbench-termi=
nology-17:
> (with COMMENT)
>
>
>
> Hi Adam,
>
>
>
> thanks for spending time to review and provide more comments to the list.
>
> I addressed both of your emails with the latest publication i just did.
>
>
>
> Please see inline for details:
>
> *=E2=80=A6stuff deleted=E2=80=A6*
>
>
>   =3D=3D Missing Reference: 'RFC5481' is mentioned on line 285, but not d=
efined
>
>
>
> Removed this RFC as we no longer reference it
>
>
>   =3D=3D Unused Reference: 'RFC5841' is defined on line 732, but no expli=
cit
>      reference was found in the text
>
> Removed this RFC as we no longer reference it
>
>
>
> *[ACM] *
>
> *This is another case of transposed RFC numbers.*
>
> *The draft references RFC 5481 in section 3.1 (terms) and this is correct=
!*
>
> *But the original reference was for =E2=80=9C5841=E2=80=9D, which wasn=E2=
=80=99t*
>
> *used in the text.  You need to add a reference to*
>
> *RFC 5481 in the next update. *
>
> *THANKS!*
>
>
>   ** Obsolete normative reference: RFC 2554 (ref. 'RFC2544') (Obsoleted b=
y
>      RFC 4954)
>
> I am sorry we use RFC 2544, not 2554 so we are good actually, its not
> obsoleted at all.
>
>
>
>
>

--001a1147e4e2cbcf3d05528c022d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Al!<div><br></div><div>Great catch thank you, I just fi=
xed it and published -19 referencing to the proper RFC number!</div><div><b=
r></div><div>Thanks,</div><div>Lucien</div></div><div class=3D"gmail_extra"=
><br><div class=3D"gmail_quote">On Thu, Jun 22, 2017 at 5:11 AM, MORTON, AL=
FRED C (AL) <span dir=3D"ltr">&lt;<a href=3D"mailto:acmorton@att.com" targe=
t=3D"_blank">acmorton@att.com</a>&gt;</span> wrote:<br><blockquote class=3D=
"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding=
-left:1ex">





<div lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"m_1768213891910008188WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Hi Lucien,<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">I caught a small mix-up in your late-night rep=
ly to Adam, below.<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Thanks for your massive editing efforts!<u></u=
><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Al<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><u></u>=C2=A0<u></u></span></p>
<div style=3D"border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in =
4.0pt">
<div>
<div style=3D"border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b><span style=3D"font-size:10.0pt;font-family:&quot=
;Tahoma&quot;,&quot;sans-serif&quot;">From:</span></b><span style=3D"font-s=
ize:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> Lucien [=
mailto:<a href=3D"mailto:lucien.avramov@gmail.com" target=3D"_blank">lucien=
.avramov@gmail.<wbr>com</a>]
<br>
<b>Sent:</b> Thursday, June 22, 2017 1:05 AM<br>
<b>To:</b> Adam Roach<br>
<b>Cc:</b> The IESG; <a href=3D"mailto:draft-ietf-bmwg-dcbench-terminology@=
ietf.org" target=3D"_blank">draft-ietf-bmwg-dcbench-<wbr>terminology@ietf.o=
rg</a>; Sarah Banks; <a href=3D"mailto:bmwg-chairs@ietf.org" target=3D"_bla=
nk">bmwg-chairs@ietf.org</a>; <a href=3D"mailto:bmwg@ietf.org" target=3D"_b=
lank">bmwg@ietf.org</a><br>
<b>Subject:</b> Re: Adam Roach&#39;s No Objection on draft-ietf-bmwg-dcbenc=
h-<wbr>terminology-17: (with COMMENT)<u></u><u></u></span></p>
</div>
</div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<div><span class=3D"">
<p class=3D"MsoNormal">Hi Adam,<u></u><u></u></p>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">thanks for spending time to review and provide more =
comments to the list.<u></u><u></u></p>
</div>
<div>
<p class=3D"MsoNormal">I addressed both of your emails with the latest publ=
ication i just did.=C2=A0<u></u><u></u></p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">Please see inline for details:<u></u><u></u></p>
</div>
</span><div>
<div>
<div>
<p class=3D"MsoNormal"><b><i><span style=3D"color:black">=E2=80=A6stuff del=
eted=E2=80=A6</span><u></u><u></u></i></b></p>
</div><span class=3D"">
<blockquote style=3D"border:none;border-left:solid #cccccc 1.0pt;padding:0i=
n 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class=3D"MsoNormal"><br>
=C2=A0 =3D=3D Missing Reference: &#39;RFC5481&#39; is mentioned on line 285=
, but not defined<u></u><u></u></p>
</blockquote>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">Removed this RFC as we no longer reference it=C2=A0<=
u></u><u></u></p>
</div>
<blockquote style=3D"border:none;border-left:solid #cccccc 1.0pt;padding:0i=
n 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class=3D"MsoNormal"><br>
=C2=A0 =3D=3D Unused Reference: &#39;RFC5841&#39; is defined on line 732, b=
ut no explicit<br>
=C2=A0 =C2=A0 =C2=A0reference was found in the text<u></u><u></u></p>
</blockquote>
<div>
<p class=3D"MsoNormal">Removed this RFC as we no longer reference it =C2=A0=
<u></u><u></u></p>
</div>
</span><blockquote style=3D"border:none;border-left:solid #cccccc 1.0pt;pad=
ding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><span style=3D"color:=
black"><u></u>=C2=A0<u></u></span></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><i><span style=3D"=
font-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">[ACM]
<u></u><u></u></span></i></b></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"fon=
t-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">This is anot=
her case of transposed RFC numbers.<u></u><u></u></span></b></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"fon=
t-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">The draft re=
ferences RFC 5481 in section 3.1 (terms) and this is correct!<u></u><u></u>=
</span></b></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"fon=
t-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">But the orig=
inal reference was for =E2=80=9C5841=E2=80=9D, which wasn=E2=80=99t<u></u><=
u></u></span></b></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"fon=
t-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">used in the =
text.=C2=A0 You need to add a reference to<u></u><u></u></span></b></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"fon=
t-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">RFC 5481 in =
the next update.
<u></u><u></u></span></b></p>
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"fon=
t-size:11.0pt;font-family:&quot;Courier New&quot;;color:black">THANKS!<u></=
u><u></u></span></b></p><span class=3D"">
<p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><br>
=C2=A0 ** Obsolete normative reference: RFC 2554 (ref. &#39;RFC2544&#39;) (=
Obsoleted by<br>
=C2=A0 =C2=A0 =C2=A0RFC 4954)<u></u><u></u></p>
</span></blockquote><span class=3D"">
<div>
<p class=3D"MsoNormal">I am sorry we use RFC 2544, not 2554 so we are good =
actually, its not obsoleted at all.=C2=A0<u></u><u></u></p>
</div>
<div>
<p class=3D"MsoNormal">=C2=A0<u></u><u></u></p>
</div>
</span></div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
</div>
</div>
</div>
</div>

</blockquote></div><br></div>

--001a1147e4e2cbcf3d05528c022d--


From nobody Thu Jun 22 14:56:22 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8833212949D; Thu, 22 Jun 2017 14:56:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.881
X-Spam-Level: 
X-Spam-Status: No, score=-1.881 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ggPUYq37Vt2G; Thu, 22 Jun 2017 14:56:10 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8D60C126C83; Thu, 22 Jun 2017 14:56:10 -0700 (PDT)
Received: from [10.0.1.63] (cpe-66-25-7-22.tx.res.rr.com [66.25.7.22]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v5MLu50h007143 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 22 Jun 2017 16:56:06 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host cpe-66-25-7-22.tx.res.rr.com [66.25.7.22] claimed to be [10.0.1.63]
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Ben Campbell <ben@nostrum.com>
In-Reply-To: <CAArZqeUL7Zoc72SR5CRDxm005Ys=d3xSnAHLPYPHX6N1wJMcww@mail.gmail.com>
Date: Thu, 22 Jun 2017 16:56:05 -0500
Cc: bmwg-chairs@ietf.org, Sarah Banks <sbanks@encrypted.net>, draft-ietf-bmwg-dcbench-terminology@ietf.org, The IESG <iesg@ietf.org>, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <1B8139B5-B630-4197-B4EF-9BE3993085FA@nostrum.com>
References: <149807196602.15838.8048520689148705215.idtracker@ietfa.amsl.com> <CAArZqeX2_1XJhuqhMdHYRELrERgMzu+AV9zQAFLQUdpMbh7PcA@mail.gmail.com> <CA7A6183-2EEA-475F-B33B-9B77F075D5FD@nostrum.com> <CAArZqeUL7Zoc72SR5CRDxm005Ys=d3xSnAHLPYPHX6N1wJMcww@mail.gmail.com>
To: Lucien <lucien.avramov@gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dl5q-TzHHVfe1SWte6Vz7HE_T_g>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-terminology-17: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 21:56:15 -0000

The latest version resolves all of my comments, except for the comment =
that it was confusing to find normative procedures in a =
=E2=80=9Cterminology=E2=80=9D draft. I understand the authors have =
chosen not to address that. It was a non-blocking comment, so I will not =
pursue it further.

Thanks!

Ben.

> On Jun 21, 2017, at 11:59 PM, Lucien <lucien.avramov@gmail.com> wrote:
>=20
> Hi Ben!
>=20
> Please see inline!
>=20
> Also fixed the Alvaro comment as well.
>=20
> Please let us know if you are okay with the change approve?
>=20
> Thanks!
> Lucien
>=20
> On Wed, Jun 21, 2017 at 12:36 PM, Ben Campbell <ben@nostrum.com> =
wrote:
> Thanks for the response. Comments inline:
>=20
> Ben.
>=20
> > On Jun 21, 2017, at 2:17 PM, Lucien <lucien.avramov@gmail.com> =
wrote:
> >
> > Hi Ben,
> >
> > Thanks for the comment! Please find my answers inline, and please =
acknowledge back!
> >
> > Appreciate you taking the time to look at our work.
> >
> > Lucien
> >
> > On Wed, Jun 21, 2017 at 12:06 PM, Ben Campbell <ben@nostrum.com> =
wrote:
> > Ben Campbell has entered the following ballot position for
> > draft-ietf-bmwg-dcbench-terminology-17: No Objection
> >
> > When responding, please keep the subject line intact and reply to =
all
> > email addresses included in the To and CC lines. (Feel free to cut =
this
> > introductory paragraph, however.)
> >
> >
> > Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/
> >
> >
> >
> > =
----------------------------------------------------------------------
> > COMMENT:
> > =
----------------------------------------------------------------------
> >
> > I find the naming of the draft fairly confusing. It goes way beyond
> > "terminology"; it makes a number of normative (using 2119 language) =
statements
> > about benchmarking procedures. I wonder why the sections about =
procedure did
> > not go into the methodology draft instead. In general, I don't think =
putting
> > normative language in an informational terminology draft is a good =
idea. (This
> > would have been a DISCUSS, except that I am aware the bmwg has =
decided to make
> > all its drafts informational and to still use 2119 language. For the =
record, I
> > think that policy falls down with this draft.)
> >
> > That's how we have decided it makes sense at BMWG to proceed with =
these two drafts. We have been hashing this out for 4+ years now and the =
current state is the consensus.  As author, I have no intention nor =
desire to change this
>=20
> For the record, I think BMWG has adopted interpretations of both =
informational RFC and 2119 keywords that are at best unconventional, and =
at worst tortured. But I also recognize the precedent has been set, and =
don=E2=80=99t mean to hold this draft hostage to it. Thus it was a =
non-blocking comment which you can feel free to ignore :-)
>=20
>=20
> >
> > I agree with the comment from others that this does not seem to be =
specific to
> > datacenters.
> >
> > Great, so did we, this is why we already worked on addressing this =
by:
> >       =E2=80=A2  calling out specifically that this specifically =
applies to data center switches (defining what those are today)
> >       =E2=80=A2 stating clearly that it can be applied to switches =
out of the data center, but that's not the specific scope of this
>=20
> Works for me, thanks.
>=20
>=20
> >
> > - 2.2: Definitions of "store-and-forward" and "cut-through" when =
used in this
> > context would be helpful. The first may be obvious, but the best I =
can do with
> > "cut-through" is assume it means the opposite of =
"store-and-forward".
> >
> > Those are cleary defined in RFC 1242, which we reference. We don't =
want to duplicate definitions here.
>=20
> Apologies, I missed the earlier citation to 1242, which was =
specifically in regard to these terms.
>=20
> >
> >
> >
> > - 6.2: After reading the definition of "Incast" several times, I'm =
still not
> > sure what it means or what is being measured.
> >
> > It's many to one type of network traffic patterns. Its very commonly =
found in cloud data centers implementing distributed storage/compute =
frameworks (big data for example).
> >
>=20
> As written, it=E2=80=99s not clear to me if =E2=80=9CIncast=E2=80=9D =
means =E2=80=9Cone to many/many-to-many=E2=80=9D communication in =
general, or the patterns of synchronization that result from that. The =
first paragraph seems to say the former but the second paragraph seems =
to say the latter.
>=20
>=20
> Clarified the paragraph and re-wrote it, take a look. incast is either =
many-to-one (simpler scenario), or many-to-many. =20
>=20


From nobody Thu Jun 22 14:57:58 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 90A6212949D; Thu, 22 Jun 2017 14:57:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.881
X-Spam-Level: 
X-Spam-Status: No, score=-1.881 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iGuAAz7LaDc8; Thu, 22 Jun 2017 14:57:49 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 66DCE129470; Thu, 22 Jun 2017 14:57:49 -0700 (PDT)
Received: from [10.0.1.63] (cpe-66-25-7-22.tx.res.rr.com [66.25.7.22]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v5MLvkK8007284 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 22 Jun 2017 16:57:46 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host cpe-66-25-7-22.tx.res.rr.com [66.25.7.22] claimed to be [10.0.1.63]
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
From: Ben Campbell <ben@nostrum.com>
In-Reply-To: <CAArZqeVYYuGiSXe2Zt3a4mA=RWOPgPOHXh+fxRvTiHSLMsu+Hg@mail.gmail.com>
Date: Thu, 22 Jun 2017 16:57:46 -0500
Cc: draft-ietf-bmwg-dcbench-methodology@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, The IESG <iesg@ietf.org>, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <C301F046-DCAE-48F1-8B7D-E802F6AAF0CA@nostrum.com>
References: <149807212936.15488.7309772374304994572.idtracker@ietfa.amsl.com> <CAArZqeV_LQ_SSvQif652aZfPrgF2sjwPKxn4uD8vC-bTc40Z0g@mail.gmail.com> <9D1D7116-BBE9-4BCF-A82C-6D6752C373AA@nostrum.com> <CAArZqeVYYuGiSXe2Zt3a4mA=RWOPgPOHXh+fxRvTiHSLMsu+Hg@mail.gmail.com>
To: Lucien <lucien.avramov@gmail.com>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/5miFIwZ3guFXdEdH94MZXKem64Y>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-dcbench-methodology-16: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jun 2017 21:57:52 -0000

Hi,

Version 18 resolves my comments.

Thanks!

Ben.

> On Jun 22, 2017, at 12:00 AM, Lucien <lucien.avramov@gmail.com> wrote:
>=20
> Hi Ben!
>=20
> Just for the archive, answering on this thread also, please see inline =
and ackowledge back:
>=20
> Many thanks for the time spent on our drafts.
>=20
> Lucien
>=20
> On Wed, Jun 21, 2017 at 12:52 PM, Ben Campbell <ben@nostrum.com> =
wrote:
> Thanks for the response; comments inline:
>=20
>=20
> > On Jun 21, 2017, at 2:22 PM, Lucien <lucien.avramov@gmail.com> =
wrote:
> >
> > Hi Ben!
> >
> > Please find inline my comments!
> >
> > Thanks for reviewing this draft!
> >
> > On Wed, Jun 21, 2017 at 12:08 PM, Ben Campbell <ben@nostrum.com> =
wrote:
> > Ben Campbell has entered the following ballot position for
> > draft-ietf-bmwg-dcbench-methodology-16: No Objection
> >
> > When responding, please keep the subject line intact and reply to =
all
> > email addresses included in the To and CC lines. (Feel free to cut =
this
> > introductory paragraph, however.)
> >
> >
> > Please refer to =
https://www.ietf.org/iesg/statement/discuss-criteria.html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/
> >
> >
> >
> > =
----------------------------------------------------------------------
> > COMMENT:
> > =
----------------------------------------------------------------------
> >
> > - I support Alvaro's DISCUSS
> >
> > As author, the meaning of the words chosen (MUST..) are important to =
me and I don't agree with make it more loose.
>=20
> The issue is that the text (even in 16) has non-matching definitions =
for MUST, SHOULD, MAY, etc. It=E2=80=99s not a matter of =
=E2=80=9Clooseness=E2=80=9D. The 2119 definitions are about =
interoperability. The definitions in 1.2 do not seem to be about =
interoperability.
> >
>=20
> Fixed by removing this section 1.2 part.=20
> =20
> >
> > - I agree with the questions about why this is specific to =
datacenters.
> >
> > Since this comment was on your other draft review, same answer =
applies here:
> >
> > Great, so did we, this is why we already worked on addressing this =
by:
> >       =E2=80=A2  calling out specifically that this specifically =
applies to data center switches (defining what those are today)
> >       =E2=80=A2 stating clearly that it can be applied to switches =
out of the data center, but that's not the specific scope of this
> >
> >
>=20
> WFM
>=20
> >
> > - Please expand DUT on first use.
> >
> > Thanks, this was expanded already on first use in the introduction =
of -16 document (the Device Under Test (DUT)). I just doubled checked. =
Please let me know if I have missed anything, and I will fix it right =
away.
>=20
> Ah, I reviewed version 14, but it progressed to version 16 before I =
wrote up my notes and entered them into the tracker. I didn=E2=80=99t =
notice the new revisions until you mentioned them here. I agree DUT is =
properly expanded in 16.
> \


From nobody Fri Jun 23 17:07:23 2017
Return-Path: <agenda@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id B6706129B26; Fri, 23 Jun 2017 17:06:57 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: "\"IETF Secretariat\"" <agenda@ietf.org>
To: <bmwg-chairs@ietf.org>, <acmorton@att.com>
Cc: warren@kumari.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149826281774.7840.9977457577206101978.idtracker@ietfa.amsl.com>
Date: Fri, 23 Jun 2017 17:06:57 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Sf1Sk2LP9ly9ZU80Uy9_jVdhZyQ>
Subject: [bmwg] bmwg - Requested session has been scheduled for IETF 99
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 24 Jun 2017 00:06:58 -0000

Dear Al Morton,

The session(s) that you have requested have been scheduled.
Below is the scheduled session information followed by
the original request. 

bmwg Session 1 (2:30:00)
    Monday, Morning Session I 0930-1200
    Room Name: Karlin III size: 60
    ---------------------------------------------
    


Request Information:


---------------------------------------------------------
Working Group Name: Benchmarking Methodology
Area Name: Operations and Management Area
Session Requester: Al Morton

Number of Sessions: 1
Length of Session(s):  2.5 Hours
Number of Attendees: 30
Conflicts to Avoid: 
 First Priority: ntp quic ippm nfvrg idr tsvwg tictoc sfc
 Second Priority: irtfopen xrblock tsvarea rmcat saag



People who must be present:
  Al Morton
  Sarah Banks
  Warren Kumari

Resources Requested:

Special Requests:
  Much better attendance and participation when session NOT on Friday
---------------------------------------------------------


From nobody Mon Jun 26 09:35:39 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 58A5512EB22; Mon, 26 Jun 2017 09:35:21 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Cc: Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, The IESG <iesg@ietf.org>, draft-ietf-bmwg-dcbench-terminology@ietf.org, bmwg@ietf.org, sbanks@encrypted.net, warren@kumari.net, rfc-editor@rfc-editor.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149849492126.18355.11951876536298682921.idtracker@ietfa.amsl.com>
Date: Mon, 26 Jun 2017 09:35:21 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/C-obasRBPfJoaN1jnm7BoifJEcs>
Subject: [bmwg] Document Action: 'Data Center Benchmarking Terminology' to Informational RFC (draft-ietf-bmwg-dcbench-terminology-19.txt)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Jun 2017 16:35:21 -0000

The IESG has approved the following document:
- 'Data Center Benchmarking Terminology'
  (draft-ietf-bmwg-dcbench-terminology-19.txt) as Informational RFC

This document is the product of the Benchmarking Methodology Working Group.

The IESG contact persons are Warren Kumari and Benoit Claise.

A URL of this Internet Draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/




Technical Summary

 This document establishes definitions and
 describes measurement techniques for data center benchmarking, as
well as introduces new terminologies applicable to data center
performance evaluations. 

Working Group Summary
  
  Note to IESG: draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology are closely related.
  There has been an extensive amount of work done on these drafts, and progress made on revisions, feedback, and comments. There have been extensive conversations on, for example, "goodput". This are some of the most reviewed drafts in BMWG in quite some time, and the documents  have undergone 2 WGLCs.

Document Quality

  Document is well written and easy to read.   

Personnel

   Sarah Banks is the Shepherd.
  Warren Kumari is RAD!


From nobody Tue Jun 27 09:06:34 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A1E51126D05 for <bmwg@ietfa.amsl.com>; Tue, 27 Jun 2017 09:06:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I46lLdsshV7x for <bmwg@ietfa.amsl.com>; Tue, 27 Jun 2017 09:06:31 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 74DAD126BF3 for <bmwg@ietf.org>; Tue, 27 Jun 2017 09:06:31 -0700 (PDT)
Received: from pps.filterd (m0049295.ppops.net [127.0.0.1]) by m0049295.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v5REaAfr017724 for <bmwg@ietf.org>; Tue, 27 Jun 2017 10:44:33 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049295.ppops.net-00191d01. with ESMTP id 2bbnpb6191-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Tue, 27 Jun 2017 10:44:32 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5REiV6H098608 for <bmwg@ietf.org>; Tue, 27 Jun 2017 09:44:31 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v5REiPL1098535 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Tue, 27 Jun 2017 09:44:25 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Tue, 27 Jun 2017 14:44:03 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5REi22Y008441 for <bmwg@ietf.org>; Tue, 27 Jun 2017 09:44:02 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v5REhtVd008047 for <bmwg@ietf.org>; Tue, 27 Jun 2017 09:43:55 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id 668E2E4DBD for <bmwg@ietf.org>; Tue, 27 Jun 2017 10:43:42 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 27 Jun 2017 10:43:55 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: IETF-99 Agenda requests
Thread-Index: AdLvUyzxZgsS9YPaQemrJoCQMoeGHA==
Date: Tue, 27 Jun 2017 14:43:54 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FE6D51@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-27_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706270237
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/3AIef9bOPg7CZdSiPhExqx6ivYA>
Subject: [bmwg] IETF-99 Agenda requests
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 27 Jun 2017 16:06:33 -0000

BMWG,

While we all work on drafts to meet the deadline (July 3),
please take a minute and let us know about the drafts where
you have made some progress, especially work proposals,
for discussion and further development at IETF-99.

Requests for remote presentation should also be
communicated at this time.

thanks and regards,
Al
for the co-chairs


From nobody Thu Jun 29 10:19:57 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6269F1200C1; Thu, 29 Jun 2017 10:19:49 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149875678928.27950.6356441277264642880@ietfa.amsl.com>
Date: Thu, 29 Jun 2017 10:19:49 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dg4Eq9-7a9Kz6-dmLNmlKWo0O14>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-sdn-controller-benchmark-meth-04.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Jun 2017 17:19:49 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for SDN Controller Performance
        Authors         : Bhuvaneswaran Vengainathan
                          Anton Basil
                          Mark Tassinari
                          Vishwas Manral
                          Sarah Banks
	Filename        : draft-ietf-bmwg-sdn-controller-benchmark-meth-04.txt
	Pages           : 53
	Date            : 2017-06-28

Abstract:
   This document defines the methodologies for benchmarking control
   plane performance of SDN controllers. Terminology related to
   benchmarking SDN controllers is described in the companion
   terminology document. SDN controllers have been implemented with
   many varying designs in order to achieve their intended network
   functionality. Hence, the authors have taken the approach of
   considering an SDN controller as a black box, defining the
   methodology in a manner that is agnostic to protocols and network
   services supported by controllers. The intent of this document is to
   provide a standard mechanism to measure the performance of all
   controller implementations.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-sdn-controller-benchmark-meth/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-sdn-controller-benchmark-meth-04
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-sdn-controller-benchmark-meth-04

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-sdn-controller-benchmark-meth-04


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Jun 29 21:30:49 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E78312762F; Thu, 29 Jun 2017 21:30:40 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.55.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149879704040.4650.8501745429929931157@ietfa.amsl.com>
Date: Thu, 29 Jun 2017 21:30:40 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/ljrl85iq5pFD9-5x8rtFn32VpmI>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-sdn-controller-benchmark-term-04.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Jun 2017 04:30:40 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Terminology for Benchmarking SDN Controller Performance
        Authors         : Bhuvaneswaran Vengainathan
                          Anton Basil
                          Mark Tassinari
                          Vishwas Manral
                          Sarah Banks
	Filename        : draft-ietf-bmwg-sdn-controller-benchmark-term-04.txt
	Pages           : 23
	Date            : 2017-06-29

Abstract:
   This document defines terminology for benchmarking an SDN
   controller's control plane performance. It extends the terminology
   already defined in RFC 7426 for the purpose of benchmarking SDN
   controllers. The terms provided in this document help to benchmark
   SDN controller's performance independent of the controller's
   supported protocols and/or network services. A mechanism for
   benchmarking the performance of SDN controllers is defined in the
   companion methodology document. These two documents provide a
   standard mechanism to measure and evaluate the performance of
   various controller implementations.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-sdn-controller-benchmark-term/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-sdn-controller-benchmark-term-04
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-sdn-controller-benchmark-term-04

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-sdn-controller-benchmark-term-04


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

