
From nobody Wed Mar  1 02:42:05 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id D16611294A1; Wed,  1 Mar 2017 02:41:59 -0800 (PST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.46.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <148836491985.6179.75132381597155091.idtracker@ietfa.amsl.com>
Date: Wed, 01 Mar 2017 02:41:59 -0800
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/hZld0qOWKNjlcuR0KgZBuencwCo>
Cc: bmwg@ietf.org
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-04.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 01 Mar 2017 10:42:00 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition Technologies
        Authors         : Marius Georgescu
                          Liviu Pislaru
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-04.txt
	Pages           : 28
	Date            : 2017-03-01

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a tentative metric for benchmarking
   load scalability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-04

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-tran-tech-benchmarking-04


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu Mar  2 08:20:03 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id BB0ED12952F; Thu,  2 Mar 2017 08:20:02 -0800 (PST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.46.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <148847160275.19488.7040820338953098750.idtracker@ietfa.amsl.com>
Date: Thu, 02 Mar 2017 08:20:02 -0800
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/7GlwG4_syO0mk9uDHVNBkl3iKFw>
Cc: bmwg@ietf.org
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-nd-06.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Mar 2017 16:20:02 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking The Neighbor Discovery Protocol
        Authors         : Bill Cerveny
                          Ron Bonica
                          Reji Thomas
	Filename        : draft-ietf-bmwg-ipv6-nd-06.txt
	Pages           : 16
	Date            : 2017-03-02

Abstract:
   This document provides benchmarking procedures for Neighbor Discovery
   Protocol (NDP).  It also proposes metrics by which an NDP
   implementation's scaling capabilities can be measured.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-nd/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-nd-06

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-nd-06


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Fri Mar  3 16:03:08 2017
Return-Path: <agenda@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A9E9E129A68; Fri,  3 Mar 2017 15:55:38 -0800 (PST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: "\"IETF Secretariat\"" <agenda@ietf.org>
To: <sbanks@encrypted.net>, <bmwg-chairs@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.46.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <148858533869.15846.7882677973981887130.idtracker@ietfa.amsl.com>
Date: Fri, 03 Mar 2017 15:55:38 -0800
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/d5US4g_Orv-5I0FMjfyfpZtuFLk>
Cc: bmwg@ietf.org
Subject: [bmwg] bmwg - Requested session has been scheduled for IETF 98
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 03 Mar 2017 23:55:39 -0000

Dear Sarah Banks,

The session(s) that you have requested have been scheduled.
Below is the scheduled session information followed by
the original request. 

bmwg Session 1 (2:30:00)
    Thursday, Morning Session I 0900-1130
    Room Name: Zurich B size: 100
    ---------------------------------------------
    


Request Information:


---------------------------------------------------------
Working Group Name: Benchmarking Methodology
Area Name: Operations and Management Area
Session Requester: Sarah Banks

Number of Sessions: 1
Length of Session(s):  2.5 Hours
Number of Attendees: 30
Conflicts to Avoid: 
 First Priority: nfvrg ippm lmap ntp idr tsvwg tictoc sfc sdnrg
 Second Priority: irtfopen xrblock tsvarea



People who must be present:
  Joel Jaeggli
  Al Morton
  Sarah Banks

Resources Requested:
  Meetecho support in room

Special Requests:
  Much better attendance and participation when session NOT on Friday
---------------------------------------------------------


From nobody Sun Mar  5 19:47:33 2017
Return-Path: <sjacob@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F147712940E for <bmwg@ietfa.amsl.com>; Sun,  5 Mar 2017 19:47:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.921
X-Spam-Level: 
X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=junipernetworks.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fZ6XxzAJ9N4n for <bmwg@ietfa.amsl.com>; Sun,  5 Mar 2017 19:47:30 -0800 (PST)
Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0136.outbound.protection.outlook.com [104.47.38.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D2989126B6D for <bmwg@ietf.org>; Sun,  5 Mar 2017 19:47:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=junipernetworks.onmicrosoft.com; s=selector1-juniper-net; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Y3LKs0hKO72FGSaBulcPk4G0pP2SzezwDY2Lpc2TEPA=; b=VarOad4PZmdrKFe1JY06/nbw17dRVKgG99NYZSC+/rGdNx4ScWyvzZ0WJQuxn4QmNqtmKG6jPdslFdtjwvUYIPiBO9snB4iOI7T4kXYOFo/WiPv2Mc6XrHIHx8zZqMnnFufDO5dgcSDO9gUC5gu3Y4FRpqE9od9DjQALpaGBheA=
Received: from CY1PR0501MB2075.namprd05.prod.outlook.com (10.164.3.25) by CY1PR0501MB2076.namprd05.prod.outlook.com (10.164.3.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.961.8; Mon, 6 Mar 2017 03:47:27 +0000
Received: from CY1PR0501MB2075.namprd05.prod.outlook.com ([10.164.3.25]) by CY1PR0501MB2075.namprd05.prod.outlook.com ([10.164.3.25]) with mapi id 15.01.0961.012; Mon, 6 Mar 2017 03:47:27 +0000
From: Sudhin Jacob <sjacob@juniper.net>
To: "Saurabh Chattopadhyay - ERS, HCL Tech" <saurabhchattopadhya@hcl.com>
Thread-Topic: Heads up / comments on draft "Benchmarking Methodology for EVPN and PBB-EVPN"
Thread-Index: AdJAfhVyncpaMRqCTSe32LlrVEHwSgABKbUwFWpLT2A=
Date: Mon, 6 Mar 2017 03:47:27 +0000
Message-ID: <CY1PR0501MB20750B72DF170D9BC143D7EAC22C0@CY1PR0501MB2075.namprd05.prod.outlook.com>
References: <PS1PR0401MB1594356D983773DD9A62C8F4ACB10@PS1PR0401MB1594.apcprd04.prod.outlook.com> <BN6PR05MB2963D9C08DB98395000970A9C2B10@BN6PR05MB2963.namprd05.prod.outlook.com>
In-Reply-To: <BN6PR05MB2963D9C08DB98395000970A9C2B10@BN6PR05MB2963.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: hcl.com; dkim=none (message not signed) header.d=none;hcl.com; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.12]
x-microsoft-exchange-diagnostics: 1; CY1PR0501MB2076; 7:ovDFZj2MvbA5FUow4EsXyuxMqOOF1k8aPmBkHF8VmaABJFzj6vlhJX/idGZEgRZCp5gJHEvJj5ZnMwkCgdPV23IUdVGC3nIn2owH0COcqxIEtFOUItZ8+ePe+ntucBy23rVcIJ9iq06NxlmM0YfHEv/FQHjV/7RrF9WnOYW6PedovLQRiQAR/qDMYd8AId3ThqKR6LQgOWn8dR00Nq8g4QROSgJRJa5zDLy4Bl8yUMiP/9YbibP5JW6+kIhXeww9VrfJBPD3CunwyWoHHr9FP+FvibMsMugFgbl9tE4sPgBO1I9OWOm2+EOlZxLS1nFBvQtAiSN9ENA5jvLKjmuzYQ==
x-ms-office365-filtering-correlation-id: 3a8a2bde-9810-4416-b5e1-08d464438229
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081); SRVR:CY1PR0501MB2076; 
x-microsoft-antispam-prvs: <CY1PR0501MB207651F7B414F60CDF7F9180C22C0@CY1PR0501MB2076.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(138986009662008)(21748063052155)(3501301888747); 
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6055026)(6041248)(20161123558025)(20161123555025)(20161123562025)(20161123564025)(20161123560025)(6072148); SRVR:CY1PR0501MB2076; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0501MB2076; 
x-forefront-prvs: 0238AEEDB0
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(7916002)(39450400003)(39840400002)(39860400002)(39850400002)(39410400002)(377454003)(38564003)(606005)(5890100001)(77096006)(3846002)(6116002)(54356999)(790700001)(76176999)(33656002)(6506006)(110136004)(102836003)(6436002)(8936002)(345774005)(50986999)(189998001)(3280700002)(2906002)(122556002)(4326008)(3660700001)(53936002)(9326002)(38730400002)(66066001)(7906003)(74316002)(53546006)(8676002)(2950100002)(6916009)(92566002)(6306002)(2900100001)(54896002)(7736002)(9686003)(86362001)(81166006)(25786008)(236005)(99286003)(5660300001)(7696004)(55016002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0501MB2076; H:CY1PR0501MB2075.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_CY1PR0501MB20750B72DF170D9BC143D7EAC22C0CY1PR0501MB2075_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Mar 2017 03:47:27.4346 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0501MB2076
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/xvhnXDMTHPZ0tn-3AFv6QgsaLik>
Cc: "bmwg@ietf.org" <bmwg@ietf.org>
Subject: Re: [bmwg] Heads up / comments on draft "Benchmarking Methodology for EVPN and PBB-EVPN"
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Mar 2017 03:47:33 -0000

--_000_CY1PR0501MB20750B72DF170D9BC143D7EAC22C0CY1PR0501MB2075_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Saurabh,

Apologies for the late reply. Based on your inputs we have modified the dra=
ft. Kindly see and let us know the comments.

https://www.ietf.org/id/draft-kishjac-bmwg-evpntest-05.txt

Regards,
Sudhin

From: Sudhin Jacob
Sent: Thursday, November 17, 2016 9:03 AM
To: Saurabh Chattopadhyay - ERS, HCL Tech <saurabhchattopadhya@hcl.com>
Subject: RE: Heads up / comments on draft "Benchmarking Methodology for EVP=
N and PBB-EVPN"

Hi Saurabh,

Let me thank you for the feedback. Much appreciate, this is a good point, I=
 will come back to you next week. After that I humbly request you to post t=
he queries in list so that it will be captured there.

Regards,
Sudhin

From: Saurabh Chattopadhyay - ERS, HCL Tech [mailto:saurabhchattopadhya@hcl=
.com]
Sent: Thursday, November 17, 2016 8:45 AM
To: Sudhin Jacob <sjacob@juniper.net<mailto:sjacob@juniper.net>>
Subject: Heads up / comments on draft "Benchmarking Methodology for EVPN an=
d PBB-EVPN"

Hi Sudhin,

As requested by you and WG, I was starting off the review of the draft ment=
ioned in the subject - but have some initial observations and thought it wo=
uld be better to provide you a heads up on this and understand your point o=
f view.

As we discussed in person on Tuesday at Conrad Seoul, it appears to me that=
 measuring the time taken for MAC Learning can be very complex, given the l=
evel of timing error which can get introduced during the process of measure=
ment itself.  Measuring the rate of learning or un-learning appears to be a=
 more feasible benchmark-able data point, since sampling the result against=
 delta time of 1 sec will help reducing the relative impact of the timing e=
rror. Hope you agree with it.

As next step, I started scanning through the other BMWG drafts, and realize=
d that MAC Learning Rate and some other relevant L2/L3 measurable parameter=
s are already captured in an RFC, 2889. Can you please take a look at these=
 parameters, and also the test procedure, topology and setup configurations=
 mentioned there?

Typically, while we submit a new draft that works somewhat on similar set o=
f measurement parameters as defined in a previous draft, we make attempt to=
 create the new draft as an incremental one. However, the new draft should =
extend and define the newer parts based on the need of the new type of fabr=
ic and underlying technologies. So, as much I understand, the test topology=
 and configurations will vary significantly for your draft on EVPN & PBB-EV=
PN, and even some test procedures will also vary since your devices rely on=
 both data plane as well as control plane learning.

But, before I suggest this approach of making your draft as extension of th=
e relevant earlier RFCs in the WG, I thought let me understand your point o=
f view first on this suggestion. Let me know.

Regards,
Saurabh



::DISCLAIMER::
---------------------------------------------------------------------------=
-------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and inte=
nded for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as informa=
tion could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in trans=
mission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability =
on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the =
author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, disse=
mination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written=
 consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please=
 delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses =
and other defects.
---------------------------------------------------------------------------=
-------------------------------------------------------------------------

--_000_CY1PR0501MB20750B72DF170D9BC143D7EAC22C0CY1PR0501MB2075_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:#0563C1;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:#954F72;
	text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
	{mso-style-name:msonormal;
	mso-margin-top-alt:auto;
	margin-right:0in;
	mso-margin-bottom-alt:auto;
	margin-left:0in;
	font-size:12.0pt;
	font-family:"Times New Roman",serif;}
span.EmailStyle18
	{mso-style-type:personal;
	font-family:"Calibri",sans-serif;
	color:windowtext;}
span.EmailStyle19
	{mso-style-type:personal;
	font-family:"Calibri",sans-serif;
	color:#1F497D;}
span.EmailStyle20
	{mso-style-type:personal-reply;
	font-family:"Calibri",sans-serif;
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Hi Saurabh,<o:p></o:p>=
</span></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><o:p>&nbsp;</o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Apologies for the late=
 reply. Based on your inputs we have modified the draft. Kindly see and let=
 us know the comments.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><o:p>&nbsp;</o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><a href=3D"https://www=
.ietf.org/id/draft-kishjac-bmwg-evpntest-05.txt">https://www.ietf.org/id/dr=
aft-kishjac-bmwg-evpntest-05.txt</a><o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><o:p>&nbsp;</o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Regards,<o:p></o:p></s=
pan></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Sudhin<o:p></o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><o:p>&nbsp;</o:p></spa=
n></p>
<div>
<div style=3D"border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b>From:</b> Sudhin Jacob <br>
<b>Sent:</b> Thursday, November 17, 2016 9:03 AM<br>
<b>To:</b> Saurabh Chattopadhyay - ERS, HCL Tech &lt;saurabhchattopadhya@hc=
l.com&gt;<br>
<b>Subject:</b> RE: Heads up / comments on draft &quot;Benchmarking Methodo=
logy for EVPN and PBB-EVPN&quot;<o:p></o:p></p>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Hi Saurabh,<o:p></o:p>=
</span></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><o:p>&nbsp;</o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Let me thank you for t=
he feedback. Much appreciate, this is a good point, I will come back to you=
 next week. After that I humbly request you to post the queries in list so =
that it will be captured there.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D"><o:p>&nbsp;</o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Regards,<o:p></o:p></s=
pan></p>
<p class=3D"MsoNormal"><span style=3D"color:#1F497D">Sudhin<o:p></o:p></spa=
n></p>
<p class=3D"MsoNormal"><a name=3D"_MailEndCompose"><span style=3D"color:#1F=
497D"><o:p>&nbsp;</o:p></span></a></p>
<span style=3D"mso-bookmark:_MailEndCompose"></span>
<div>
<div style=3D"border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b>From:</b> Saurabh Chattopadhyay - ERS, HCL Tech [=
<a href=3D"mailto:saurabhchattopadhya@hcl.com">mailto:saurabhchattopadhya@h=
cl.com</a>]
<br>
<b>Sent:</b> Thursday, November 17, 2016 8:45 AM<br>
<b>To:</b> Sudhin Jacob &lt;<a href=3D"mailto:sjacob@juniper.net">sjacob@ju=
niper.net</a>&gt;<br>
<b>Subject:</b> Heads up / comments on draft &quot;Benchmarking Methodology=
 for EVPN and PBB-EVPN&quot;<o:p></o:p></p>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Hi Sudhin,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">As requested by you and WG, I was starting off the r=
eview of the draft mentioned in the subject &#8211; but have some initial o=
bservations and thought it would be better to provide you a heads up on thi=
s and understand your point of view.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">As we discussed in person on Tuesday at Conrad Seoul=
, it appears to me that measuring the time taken for MAC Learning can be ve=
ry complex, given the level of timing error which can get introduced during=
 the process of measurement itself.
 &nbsp;Measuring the <b><i>rate</i></b> of learning or un-learning appears =
to be a more feasible benchmark-able data point, since sampling the result =
against delta time of 1 sec will help reducing the relative impact of the t=
iming error. Hope you agree with it.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">As next step, I started scanning through the other B=
MWG drafts, and realized that MAC Learning Rate and some other relevant L2/=
L3 measurable parameters are already captured in an RFC, 2889. Can you plea=
se take a look at these parameters,
 and also the test procedure, topology and setup configurations mentioned t=
here?<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Typically, while we submit a new draft that works so=
mewhat on similar set of measurement parameters as defined in a previous dr=
aft, we make attempt to create the new draft as an incremental one. However=
, the new draft should extend and
 define the newer parts based on the need of the new type of fabric and und=
erlying technologies. So, as much I understand, the test topology and confi=
gurations will vary significantly for your draft on EVPN &amp; PBB-EVPN, an=
d even some test procedures will also
 vary since your devices rely on both data plane as well as control plane l=
earning.
<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">But, before I suggest this approach of making your d=
raft as extension of the relevant earlier RFCs in the WG, I thought let me =
understand your point of view first on this suggestion. Let me know.<o:p></=
o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Regards,<o:p></o:p></p>
<p class=3D"MsoNormal">Saurabh<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal"><span style=3D"color:gray"><br>
<br>
</span><span style=3D"font-size:7.0pt;font-family:&quot;Arial&quot;,sans-se=
rif;color:gray">::DISCLAIMER::<br>
---------------------------------------------------------------------------=
-------------------------------------------------------------------------</=
span><span style=3D"color:gray"><o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:&quot;Ari=
al&quot;,sans-serif;color:gray">The contents of this e-mail and any attachm=
ent(s) are confidential and intended for the named recipient(s) only.<br>
E-mail transmission is not guaranteed to be secure or error-free as informa=
tion could be intercepted, corrupted,
<br>
lost, destroyed, arrive late or incomplete, or may contain viruses in trans=
mission. The e mail and its contents
<br>
(with or without referred errors) shall therefore not attach any liability =
on the originator or HCL or its affiliates.
<br>
Views or opinions, if any, presented in this email are solely those of the =
author and may not necessarily reflect the
<br>
views or opinions of HCL or its affiliates. Any form of reproduction, disse=
mination, copying, disclosure, modification,
<br>
distribution and / or publication of this message without the prior written=
 consent of authorized representative of
<br>
HCL is strictly prohibited. If you have received this email in error please=
 delete it and notify the sender immediately.
<br>
Before opening any email and/or attachments, please check them for viruses =
and other defects.</span><span style=3D"color:gray"><o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:&quot;Ari=
al&quot;,sans-serif;color:gray">-------------------------------------------=
---------------------------------------------------------------------------=
------------------------------</span><o:p></o:p></p>
</div>
</body>
</html>

--_000_CY1PR0501MB20750B72DF170D9BC143D7EAC22C0CY1PR0501MB2075_--


From nobody Sat Mar 11 12:09:03 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 064591295A3 for <bmwg@ietfa.amsl.com>; Sat, 11 Mar 2017 12:09:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.101
X-Spam-Level: 
X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5SyW6a_KCAb7 for <bmwg@ietfa.amsl.com>; Sat, 11 Mar 2017 12:09:00 -0800 (PST)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C1B3112959C for <bmwg@ietf.org>; Sat, 11 Mar 2017 12:09:00 -0800 (PST)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v2BJsq0V022648 for <bmwg@ietf.org>; Sat, 11 Mar 2017 15:08:58 -0500
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0083689.ppops.net-00191d01. with ESMTP id 294n2dax45-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Sat, 11 Mar 2017 15:08:57 -0500
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v2BK8vLx009258 for <bmwg@ietf.org>; Sat, 11 Mar 2017 15:08:57 -0500
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v2BK8peo009220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Sat, 11 Mar 2017 15:08:51 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor) for <bmwg@ietf.org>; Sat, 11 Mar 2017 20:08:34 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v2BK8YW6009232 for <bmwg@ietf.org>; Sat, 11 Mar 2017 14:08:34 -0600
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v2BK8Wvl009174 for <bmwg@ietf.org>; Sat, 11 Mar 2017 14:08:32 -0600
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id C9C35F0411 for <bmwg@ietf.org>; Sat, 11 Mar 2017 15:08:31 -0500 (EST)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Sat, 11 Mar 2017 15:08:31 -0500
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Draft Agenda for IETF-98
Thread-Index: AdKaogdqA8GQbxZ8RzGhre0gCMvROQ==
Date: Sat, 11 Mar 2017 20:08:31 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F2B2A2@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.237.204]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-11_12:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703110171
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/JkE7oSpfYfES3ipyrI6n7_gXV5o>
Subject: [bmwg] Draft Agenda for IETF-98
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 11 Mar 2017 20:09:02 -0000

BMWG,

The draft agenda for our session at IETF-98 in Chicago
is available with the other meeting materials:
https://datatracker.ietf.org/meeting/98/agenda/bmwg/

Please let me know of any:=20

- requests for Remote Presentation (before March 24)
- changes to Presenter Names
- revised drafts (after today)
- additional requests for agenda time
- other Agenda Bashing...

thanks, and see you in Chicago (UTC-5 when we get there).
Al
bmwg co-chair




From nobody Mon Mar 13 08:15:34 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id AA2071294A4; Mon, 13 Mar 2017 08:15:29 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.47.0
Auto-Submitted: auto-generated
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <148941812969.16934.17942767817200629806.idtracker@ietfa.amsl.com>
Date: Mon, 13 Mar 2017 08:15:29 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/UTua6fZPqBISABgGXkbCF3VE6Yg>
Cc: bmwg-chairs@ietf.org, acmorton@att.com, joelja@gmail.com, draft-ietf-bmwg-ipv6-nd@ietf.org, The IESG <iesg@ietf.org>, bmwg@ietf.org, rfc-editor@rfc-editor.org
Subject: [bmwg] Document Action: 'Benchmarking The Neighbor Discovery Protocol' to Informational RFC (draft-ietf-bmwg-ipv6-nd-06.txt)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Mar 2017 15:15:29 -0000

The IESG has approved the following document:
- 'Benchmarking The Neighbor Discovery Protocol'
  (draft-ietf-bmwg-ipv6-nd-06.txt) as Informational RFC

This document is the product of the Benchmarking Methodology Working
Group.

The IESG contact persons are Benoit Claise and Joel Jaeggli.

A URL of this Internet Draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-nd/





Technical Summary

   IPv6 nodes use the Neighbor Discovery Protocol (NDP) [RFC4861] to
   maintain the Neighbor Cache.  Operational experience [RFC6583] shows 
   that when an implementation cannot maintain a sufficiently complete 
   Neighbor Cache, its ability to forward packets is impaired. 
   This document provides benchmarking procedures for the Neighbor
   Discovery Protocol (NDP). Benchmarking procedures include a Baseline Test 
   and an NDP Scaling Test. This document defines an NDP scaling metric, 
   called NDP-MAX-NEIGHBORS. 

Working Group Summary

   Once the Working Group agreed on the benchmarking problem to solve,
   revisions to the draft were straightforward and consensus was 
   smooth.

Document Quality

  INT Area Directorate review was requested. Draft was discussed 
in v6ops and is an offshoot of work done there for RFC 6583.

Personnel

  Al Morton is the Document Shepherd.
  Joel Jaeggli is the Responsible AD.


From skommu@vmware.com  Mon Mar 13 16:50:55 2017
Return-Path: <skommu@vmware.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED158129C20 for <bmwg@ietfa.amsl.com>; Mon, 13 Mar 2017 16:50:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level: 
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=onevmw.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ylbQHPTg5W4a for <bmwg@ietfa.amsl.com>; Mon, 13 Mar 2017 16:50:54 -0700 (PDT)
Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0085.outbound.protection.outlook.com [104.47.34.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E33D7129C14 for <bmwg@ietf.org>; Mon, 13 Mar 2017 16:50:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=qiwkJ1H6wdLSPiLkI3L8CgZgfUoePCcwbsubqwVrLAk=; b=nyVa/mKnGRI7fxCrs/SysuwK2uTc2hO1FCeGi3UVzkfNX41WdRHgaJp/wvpVBd8JV9JU0yGC1uWiycF2ZnyEqBcbmosS6xAeQtuwUGEeX+AYca0AgKZOB1wRh/UUsqHecIbTHR3Cw1n4V8PtuOaE70BvwfO4kDyOfWRGax9/tJ0=
Received: from SN2PR05MB2750.namprd05.prod.outlook.com (10.167.19.25) by SN2PR05MB2752.namprd05.prod.outlook.com (10.167.19.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.947.2; Mon, 13 Mar 2017 23:50:52 +0000
Received: from SN2PR05MB2750.namprd05.prod.outlook.com ([10.167.19.25]) by SN2PR05MB2750.namprd05.prod.outlook.com ([10.167.19.25]) with mapi id 15.01.0947.012; Mon, 13 Mar 2017 23:50:52 +0000
From: Samuel Kommu <skommu@vmware.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Benchmarking Considerations for NVP
Thread-Index: AQHSnFSlHZAnbhTEBU+ydKJJEhXLpw==
Date: Mon, 13 Mar 2017 23:50:52 +0000
Message-ID: <D4EC7E68.68DBF%skommu@vmware.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.7.1.161129
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=vmware.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-originating-ip: [71.198.210.15]
x-ms-office365-filtering-correlation-id: 56bf9f90-d2b9-4e26-fc7e-08d46a6bc8af
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:SN2PR05MB2752;
x-microsoft-exchange-diagnostics: 1; SN2PR05MB2752; 7:kwd68zGg1IrQjFpHIjd+lu5D4DKgZJZmBR92S4VKfktBGBqv3ZvGjN0puSkuKSSwmi8oECsrz88Ou4v+bXDEptRdnV5IYQHqSPriVIVHriMj+Qpu/p3QUtkMbnfQclT5EuIF+xRfeKGMxbNwdPQwCNwjtur4yedAh+GiNHYiz+Rinq6I8DIAaGOoax+JZdrRjeMdanlwaPouw6bDDN9gKk+XLLyZQ/b7mv3akrSNY0TRioQOkhnIdcYijVgRcG4MwoTLalRUPVWlJl6zZPwxqLE+Y9VZ2Y3RF2hl41DFqh1cSkNJg+wwNYgPS3oKYV5mJg8yKaV50yxbmLoBf0865w==; 20:MYoMRjgDpi5yIYQqRpgz7hVmlTR57MdoQquRzrrMuYEdkMJ3emjEEGXP842bhUfdcjj9iAY95NxC6SmTMzG+nt43RQmpV6hrPNOSwdsIteLFXNYTSVpYJ25Z2qvPf63klStB05EFQTiv2NcAsgH/8fx/QdlCq3iWCVGSntTK+L8=
x-microsoft-antispam-prvs: <SN2PR05MB275282B1D25DCBA38D44B803AD250@SN2PR05MB2752.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(120809045254105);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123562025)(20161123558025)(20161123564025)(20161123560025)(20161123555025)(6072148); SRVR:SN2PR05MB2752; BCL:0; PCL:0; RULEID:; SRVR:SN2PR05MB2752; 
x-forefront-prvs: 0245702D7B
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39410400002)(39830400002)(39450400003)(122556002)(6916009)(558084003)(3846002)(6116002)(102836003)(15650500001)(2501003)(66066001)(7906003)(7736002)(2906002)(110136004)(3280700002)(86362001)(38730400002)(36756003)(2900100001)(3660700001)(106116001)(1730700003)(53936002)(236005)(8936002)(54896002)(6306002)(6512007)(81166006)(8676002)(50986999)(54356999)(4001350100001)(99286003)(5660300001)(25786008)(2351001)(5640700003)(6506006)(6436002)(606005)(189998001)(83506001)(6486002)(77096006); DIR:OUT; SFP:1101; SCL:1; SRVR:SN2PR05MB2752; H:SN2PR05MB2750.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_D4EC7E6868DBFskommuvmwarecom_"
MIME-Version: 1.0
X-OriginatorOrg: vmware.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Mar 2017 23:50:52.5682 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR05MB2752
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/q-cxfGalm_maf3mnzHozljdJfIQ>
Subject: [bmwg] Updated: Benchmarking Considerations for NVP
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Mar 2017 23:53:43 -0000

--_000_D4EC7E6868DBFskommuvmwarecom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

BMWG Team,

Have updated the draft based on comments from last discussion:  https://dat=
atracker.ietf.org/doc/draft-skommu-bmwg-nvp/  Please review?

Thank you!

Regards,
Samuel.

--_000_D4EC7E6868DBFskommuvmwarecom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <37C423C5B42BD440B5C96D4650E01440@namprd05.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 15px; font-fami=
ly: Calibri, sans-serif;">
<div>BMWG Team,</div>
<div><br>
</div>
<div>Have updated the draft based on comments from last discussion: &nbsp;<=
a href=3D"https://datatracker.ietf.org/doc/draft-skommu-bmwg-nvp/">https://=
datatracker.ietf.org/doc/draft-skommu-bmwg-nvp/</a>&nbsp; Please review?</d=
iv>
<div><br>
</div>
<div>Thank you!</div>
<div><br>
</div>
<div>Regards,</div>
<div>Samuel.</div>
</body>
</html>

--_000_D4EC7E6868DBFskommuvmwarecom_--


From nobody Tue Mar 14 09:00:37 2017
Return-Path: <ietf@kuehlewind.net>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C252213292C; Tue, 14 Mar 2017 09:00:31 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: =?utf-8?q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-virtual-net@ietf.org, Sarah Banks <sbanks@encrypted.net>,  bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.47.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <148950723176.30221.8570553212215298771.idtracker@ietfa.amsl.com>
Date: Tue, 14 Mar 2017 09:00:31 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/7EMlA_yjOXY9KOlyJDhIKRzjuEI>
Subject: [bmwg] =?utf-8?q?Mirja_K=C3=BChlewind=27s_No_Objection_on_draft-i?= =?utf-8?q?etf-bmwg-virtual-net-04=3A_=28with_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.21
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Mar 2017 16:00:32 -0000

Mirja Kühlewind has entered the following ballot position for
draft-ietf-bmwg-virtual-net-04: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks for the well written doc!



From nobody Wed Mar 15 19:04:08 2017
Return-Path: <ben@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5422212F265; Wed, 15 Mar 2017 19:04:07 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Ben Campbell <ben@nostrum.com>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-virtual-net@ietf.org, Sarah Banks <sbanks@encrypted.net>,  bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.47.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <148962984725.14221.15348302407623063224.idtracker@ietfa.amsl.com>
Date: Wed, 15 Mar 2017 19:04:07 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/6wlqwdMTA5rltgwOzQ-c7APQwbI>
Subject: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-virtual-net-04: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Mar 2017 02:04:07 -0000

Ben Campbell has entered the following ballot position for
draft-ietf-bmwg-virtual-net-04: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I have a few mostly editorial comments:

- Abstract and Introduction: Missing "the" before "Benchmarking..."

-Abstract: Will the paragraph about new version history stay in the
RFC?

-1: Much of this section, especially the 2nd paragraph, reads like a
commercial, or a marketing white paper. I'm not going to put this in the
way of publication, but an IETF RFC should generally take a more neutral
tone. It's enough to acknowledge that people are doing (or plan to do)
NFV.

-2: Language of the form of "BMG will consider" will quickly become
dated. Consider something to the effect of "At the time of this writing,
BMG is considering/plans to consider..."

Can you offer a definition or citation for "bare metal"?

"Also,  benchmarking combinations of physical and virtual devices and
functions in a System Under Test.": Sentence fragment.

- 4.2, first paragraph, last sentence: Can you offer a citation for the
4x3 matrix?
:



From nobody Wed Mar 15 21:02:02 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D1BAF130141; Wed, 15 Mar 2017 21:01:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.397
X-Spam-Level: 
X-Spam-Status: No, score=-5.397 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.796, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8UJTdv0ZMxsX; Wed, 15 Mar 2017 21:01:58 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF29712F257; Wed, 15 Mar 2017 21:01:54 -0700 (PDT)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v2G3tNw1015859; Thu, 16 Mar 2017 00:01:53 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0083689.ppops.net-00191d01. with ESMTP id 297k3y20vf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 16 Mar 2017 00:01:52 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v2G41pfa008750; Thu, 16 Mar 2017 00:01:52 -0400
Received: from mlpi409.sfdc.sbc.com (mlpi409.sfdc.sbc.com [130.9.128.241]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v2G41gZx008583 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 16 Mar 2017 00:01:44 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi409.sfdc.sbc.com (RSA Interceptor); Thu, 16 Mar 2017 04:01:31 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v2G41VGx026839; Wed, 15 Mar 2017 23:01:31 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v2G41QqG025961; Wed, 15 Mar 2017 23:01:27 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 49744E0A8A; Thu, 16 Mar 2017 00:01:26 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 16 Mar 2017 00:01:26 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Ben Campbell <ben@nostrum.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-virtual-net@ietf.org" <draft-ietf-bmwg-virtual-net@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Ben Campbell's No Objection on draft-ietf-bmwg-virtual-net-04: (with COMMENT)
Thread-Index: AQHSnfmc3+w3SgXPzEOj9GDQir8IgqGW0SsA
Date: Thu, 16 Mar 2017 04:01:24 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F32FD9@njmtexg5.research.att.com>
References: <148962984725.14221.15348302407623063224.idtracker@ietfa.amsl.com>
In-Reply-To: <148962984725.14221.15348302407623063224.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.251.147]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-16_03:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703160029
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/ijnwfYhKMc3z5RiqVpOkpUKBcOo>
Subject: Re: [bmwg] Ben Campbell's No Objection on draft-ietf-bmwg-virtual-net-04: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Mar 2017 04:02:00 -0000

VGhhbmtzIGZvciB5b3VyIGNvbW1lbnRzLCBCZW4uDQpSZXBsaWVzIGJlbG93LA0KQWwNCg0KPiAt
LS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBCZW4gQ2FtcGJlbGwgW21haWx0bzpi
ZW5Abm9zdHJ1bS5jb21dDQo+IFNlbnQ6IFdlZG5lc2RheSwgTWFyY2ggMTUsIDIwMTcgMTA6MDQg
UE0NCj4gVG86IFRoZSBJRVNHDQo+IENjOiBkcmFmdC1pZXRmLWJtd2ctdmlydHVhbC1uZXRAaWV0
Zi5vcmc7IFNhcmFoIEJhbmtzOyBibXdnLQ0KPiBjaGFpcnNAaWV0Zi5vcmc7IHNiYW5rc0BlbmNy
eXB0ZWQubmV0OyBibXdnQGlldGYub3JnDQo+IFN1YmplY3Q6IEJlbiBDYW1wYmVsbCdzIE5vIE9i
amVjdGlvbiBvbiBkcmFmdC1pZXRmLWJtd2ctdmlydHVhbC1uZXQtMDQ6DQo+ICh3aXRoIENPTU1F
TlQpDQo+IA0KPiBCZW4gQ2FtcGJlbGwgaGFzIGVudGVyZWQgdGhlIGZvbGxvd2luZyBiYWxsb3Qg
cG9zaXRpb24gZm9yDQo+IGRyYWZ0LWlldGYtYm13Zy12aXJ0dWFsLW5ldC0wNDogTm8gT2JqZWN0
aW9uDQo+IA0KPiBXaGVuIHJlc3BvbmRpbmcsIHBsZWFzZSBrZWVwIHRoZSBzdWJqZWN0IGxpbmUg
aW50YWN0IGFuZCByZXBseSB0byBhbGwNCj4gZW1haWwgYWRkcmVzc2VzIGluY2x1ZGVkIGluIHRo
ZSBUbyBhbmQgQ0MgbGluZXMuIChGZWVsIGZyZWUgdG8gY3V0IHRoaXMNCj4gaW50cm9kdWN0b3J5
IHBhcmFncmFwaCwgaG93ZXZlci4pDQo+IA0KPiANCj4gUGxlYXNlIHJlZmVyIHRvIGh0dHBzOi8v
dXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0FfX3d3dy5pZXRm
Lm9yZ19pZXNnX3N0YXRlbWVudF9kaXNjdXNzLTJEY3JpdGVyaWEuaHRtbCZkPUR3SUNhUSZjPUxG
WVotDQo+IG85X0hVTWVNVFNRaWN2aklnJnI9T2ZzU3U4a1RJbHRWeUQxb0w3MmNCdyZtPXk2d2Uy
Y3lpeHhvOFQwZ3ctDQo+IGlkQ2RrVWN4UkNWSXFOUUQ4WEFDWWF3ek5FJnM9TTNwRmpKZ3lTUDND
WVR1VEtrZ1F0a3NJa0M5M0w0MzBlc1ZkSXVzdUp5MA0KPiAmZT0NCj4gZm9yIG1vcmUgaW5mb3Jt
YXRpb24gYWJvdXQgSUVTRyBESVNDVVNTIGFuZCBDT01NRU5UIHBvc2l0aW9ucy4NCj4gDQo+IA0K
PiBUaGUgZG9jdW1lbnQsIGFsb25nIHdpdGggb3RoZXIgYmFsbG90IHBvc2l0aW9ucywgY2FuIGJl
IGZvdW5kIGhlcmU6DQo+IGh0dHBzOi8vdXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/
dT1odHRwcy0NCj4gM0FfX2RhdGF0cmFja2VyLmlldGYub3JnX2RvY19kcmFmdC0yRGlldGYtMkRi
bXdnLTJEdmlydHVhbC0NCj4gMkRuZXRfJmQ9RHdJQ2FRJmM9TEZZWi0NCj4gbzlfSFVNZU1UU1Fp
Y3ZqSWcmcj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3Jm09eTZ3ZTJjeWl4eG84VDBndy0NCj4gaWRD
ZGtVY3hSQ1ZJcU5RRDhYQUNZYXd6TkUmcz1TcTcwdzdDRFVqUkEzUGh0YllXU2JYbDRjek9rVU4x
TU5qdkprU0RJdFJBDQo+ICZlPQ0KPiANCj4gDQo+IA0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IENPTU1F
TlQ6DQo+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gDQo+IEkgaGF2ZSBhIGZldyBtb3N0bHkgZWRpdG9yaWFs
IGNvbW1lbnRzOg0KPiANCj4gLSBBYnN0cmFjdCBhbmQgSW50cm9kdWN0aW9uOiBNaXNzaW5nICJ0
aGUiIGJlZm9yZSAiQmVuY2htYXJraW5nLi4uIg0KPiANCj4gLUFic3RyYWN0OiBXaWxsIHRoZSBw
YXJhZ3JhcGggYWJvdXQgbmV3IHZlcnNpb24gaGlzdG9yeSBzdGF5IGluIHRoZQ0KPiBSRkM/DQpb
QUNNXSANCm5vLCB0aGUgdmVyc2lvbiBoaXN0b3J5IHVzdWFsbHkgZ29lcywgdG9vLg0KDQo+IA0K
PiAtMTogTXVjaCBvZiB0aGlzIHNlY3Rpb24sIGVzcGVjaWFsbHkgdGhlIDJuZCBwYXJhZ3JhcGgs
IHJlYWRzIGxpa2UgYQ0KPiBjb21tZXJjaWFsLCBvciBhIG1hcmtldGluZyB3aGl0ZSBwYXBlci4g
SSdtIG5vdCBnb2luZyB0byBwdXQgdGhpcyBpbiB0aGUNCj4gd2F5IG9mIHB1YmxpY2F0aW9uLCBi
dXQgYW4gSUVURiBSRkMgc2hvdWxkIGdlbmVyYWxseSB0YWtlIGEgbW9yZSBuZXV0cmFsDQo+IHRv
bmUuIEl0J3MgZW5vdWdoIHRvIGFja25vd2xlZGdlIHRoYXQgcGVvcGxlIGFyZSBkb2luZyAob3Ig
cGxhbiB0byBkbykNCj4gTkZWLg0KPiANCj4gLTI6IExhbmd1YWdlIG9mIHRoZSBmb3JtIG9mICJC
TUcgd2lsbCBjb25zaWRlciIgd2lsbCBxdWlja2x5IGJlY29tZQ0KPiBkYXRlZC4gQ29uc2lkZXIg
c29tZXRoaW5nIHRvIHRoZSBlZmZlY3Qgb2YgIkF0IHRoZSB0aW1lIG9mIHRoaXMgd3JpdGluZywN
Cj4gQk1HIGlzIGNvbnNpZGVyaW5nL3BsYW5zIHRvIGNvbnNpZGVyLi4uIg0KW0FDTV0gDQpPSywg
dGhhdCdzIGZpbmUuDQo+IA0KPiBDYW4geW91IG9mZmVyIGEgZGVmaW5pdGlvbiBvciBjaXRhdGlv
biBmb3IgImJhcmUgbWV0YWwiPw0KW0FDTV0gDQpJIGZpcnN0IHJlYWQgdGhpcyB0ZXJtIGluIHRo
ZSBjb250ZXh0IG9mIGh5cGVydmlzb3IgY2F0ZWdvcmllcywNCndoZXJlIFR5cGUgMSBydW5zIGRp
cmVjdGx5IG9uIHRoZSBob3N0IGhhcmR3YXJlLCBjYWxsZWQgbmF0aXZlIG9yIGJhcmUgbWV0YWwu
DQpUeXBlIDIgaXMgYSBob3N0ZWQgSHlwZXJ2aXNvciwgcnVubmluZyBvbiB0aGUgaG9zdCBPUyAo
ZS5nLiwgVkJveCkuDQoNCldpa2lwZWRpYSBwcm92aWRlcyB0aGlzIGVhcmx5IGNpdGF0aW9uOg0K
UG9wZWssIEdlcmFsZCBKLjsgR29sZGJlcmcsIFJvYmVydCBQLiAoMTk3NCkuICJGb3JtYWwgcmVx
dWlyZW1lbnRzIGZvciB2aXJ0dWFsaXphYmxlIHRoaXJkIGdlbmVyYXRpb24gYXJjaGl0ZWN0dXJl
cyIuIENvbW11bmljYXRpb25zIG9mIHRoZSBBQ00uIDE3ICg3KTogNDEy4oCTNDIxLiBkb2k6MTAu
MTE0NS8zNjEwMTEuMzYxMDczLiBSZXRyaWV2ZWQgMjAxNS0wMy0wMS4NCg0KImJhcmUgbWV0YWwi
IGhhcyBjb21lIHRvIG1lYW4gdGhlIGNvbXB1dGUgaGFyZHdhcmUsIA0KYSBub24tdmlydHVhbGl6
ZWQgY29tcHV0ZSBlbnZpcm9ubWVudCwNCmZvbGxvd2luZyBhY2NlcHRhbmNlIG9mIHRoZXNlIGh5
cGVydmlzb3IgY2F0ZWdvcmllcy4NCj4gDQo+ICJBbHNvLCAgYmVuY2htYXJraW5nIGNvbWJpbmF0
aW9ucyBvZiBwaHlzaWNhbCBhbmQgdmlydHVhbCBkZXZpY2VzIGFuZA0KPiBmdW5jdGlvbnMgaW4g
YSBTeXN0ZW0gVW5kZXIgVGVzdC4iOiBTZW50ZW5jZSBmcmFnbWVudC4NCltBQ01dIA0Kd2lsbCBm
aXgNCj4gDQo+IC0gNC4yLCBmaXJzdCBwYXJhZ3JhcGgsIGxhc3Qgc2VudGVuY2U6IENhbiB5b3Ug
b2ZmZXIgYSBjaXRhdGlvbiBmb3IgdGhlDQo+IDR4MyBtYXRyaXg/DQo+IDoNCiBbQUNNXSANClRo
ZSAzeDQgbWF0cml4IHdhcyBmaXJzdCBkZXZlbG9wZWQgaW4gdGhpcyBkb2N1bWVudCwgYnkgdGhl
IEJNV0cuDQoNClRoZSBvcmlnaW5hbCAzeDMgbWF0cml4IGdvZXMgYmFjayB0byBBTlNJIFgzLjEw
MiBTdGFuZGFyZCwNCiJVc2VyLU9yaWVudGVkIERhdGEgQ29tbXVuaWNhdGlvbnMgRnJhbWV3b3Jr
IiwgaW4gdGhlIDE5ODBzLg0KDQpTb21lb25lIGZyb20gTlRJQS9JVFMgZ2F2ZSBhIHRhbGsgaW5j
bHVkaW5nIHRoaXMgbWF0cml4IGF0IElFVEYtMzgsDQppbiB0aGUgUlRGTSBXRyAocG9zc2libHkg
TmVhbCBTZWl0eik6DQpodHRwczovL3d3dy5pZXRmLm9yZy9wcm9jZWVkaW5ncy8zOC9vcHMvcnRm
bS0yL3NsZDAxNC5odG0NCg0KSSdsbCBhZGQgYSBjaXRhdGlvbiBmb3IgdGhlIDN4MyBNYXRyaXgu
DQoNCg0K


From Brooks.Hickman@spirent.com  Thu Mar 16 21:11:06 2017
Return-Path: <Brooks.Hickman@spirent.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 95E37129BE9 for <bmwg@ietfa.amsl.com>; Thu, 16 Mar 2017 21:11:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.921
X-Spam-Level: 
X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=spirent1.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vWa8b1v2n4TT for <bmwg@ietfa.amsl.com>; Thu, 16 Mar 2017 21:11:03 -0700 (PDT)
Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0137.outbound.protection.outlook.com [104.47.38.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6708E129BDB for <bmwg@ietf.org>; Thu, 16 Mar 2017 21:11:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spirent1.onmicrosoft.com; s=selector1-spirent-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ZSzB3hpt84ZVptZVVEdXsWFYBhL1Ri1FMoKH0rz3pWI=; b=m9Ubw7xwpOl8pcJgL52/RVehi8nkTGVWtkBVbcLVSHj+LQb1Un2GwrGGCycWlWA9F85+nLO4BpgNeYAGfFoOuNJ7kPrKBxMa4Jov55iGNLyuTXbSiaKkunoG+cIcRJwr5ocH2uicQyKLf9yquiTA3G+gExzCIp30AbhP3Mpz+v0=
Received: from DM5PR10MB1610.namprd10.prod.outlook.com (10.172.34.143) by DM5PR10MB1611.namprd10.prod.outlook.com (10.172.34.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.977.11; Fri, 17 Mar 2017 04:11:01 +0000
Received: from DM5PR10MB1610.namprd10.prod.outlook.com ([10.172.34.143]) by DM5PR10MB1610.namprd10.prod.outlook.com ([10.172.34.143]) with mapi id 15.01.0947.020; Fri, 17 Mar 2017 04:11:01 +0000
From: "Hickman, Brooks" <Brooks.Hickman@spirent.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Comments on draft- huang-bmwg-virtual-network-performance-02
Thread-Index: AdKezzXDxqxQIjZASLG0u/Tj67MSUw==
Date: Fri, 17 Mar 2017 04:11:01 +0000
Message-ID: <DM5PR10MB161005B77BD2E308253A08B49F390@DM5PR10MB1610.namprd10.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=spirent.com;
x-originating-ip: [108.192.130.1]
x-ms-office365-filtering-correlation-id: 5f628e81-9605-4f24-eabb-08d46ceb9fa2
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:DM5PR10MB1611;
x-microsoft-exchange-diagnostics: 1; DM5PR10MB1611; 7:e8CP+udM5JGT3F6yw6EVRhzH6TWTujutwJY7OLVQk5oCBmA7+Aqt/VscltjVDMtSIoeaY8grNefT6nIDJQS/Dgm5o08QHLE82HH7Cv7e1qxofHx+RwFFue7bhq/UfplNUVwRd7+QPrcqMAu9AmWrD5fg3zMuhdx7PytezRZgUbTEKEDM3FqwafbefSUuV5d4/v7uV92ShghJx5j+55c1Dhp66kXvWg/cz/vMS8KXPb29S0ftU26yfpbMfc73X0r8eEU/WcXN39pO1rMiNXJfvLHloDz6cUzBKH/3u9BmV24zS30G1hfF8lL3vsKfSFbXH1vLu3FUwtqIRY4b3Hq06A==
x-disclaimer: Yes
x-microsoft-antispam-prvs: <DM5PR10MB16117B6433200F631C7F45A19F390@DM5PR10MB1611.namprd10.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(158342451672863)(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6041248)(20161123560025)(20161123564025)(20161123555025)(20161123562025)(20161123558025)(6072148); SRVR:DM5PR10MB1611; BCL:0; PCL:0; RULEID:; SRVR:DM5PR10MB1611; 
x-forefront-prvs: 0249EFCB0B
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(979002)(6009001)(39840400002)(39450400003)(39410400002)(85714005)(9686003)(5660300001)(53936002)(2900100001)(105586002)(230783001)(6306002)(81166006)(54896002)(8676002)(1730700003)(66066001)(54356999)(2501003)(33656002)(38730400002)(7736002)(2351001)(110136004)(8936002)(6916009)(50986999)(3280700002)(2906002)(25786008)(3660700001)(189998001)(790700001)(86362001)(7696004)(122556002)(6506006)(6116002)(5640700003)(6436002)(102836003)(99286003)(3846002)(74316002)(55016002)(77096006)(5630700001)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR10MB1611; H:DM5PR10MB1610.namprd10.prod.outlook.com; FPR:; SPF:None; MLV:ovrnspm; PTR:InfoNoRecords; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_DM5PR10MB161005B77BD2E308253A08B49F390DM5PR10MB1610namp_"
MIME-Version: 1.0
X-OriginatorOrg: spirent.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Mar 2017 04:11:01.4368 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: eb68cad0-6bd5-483f-a802-0f72f974373f
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR10MB1611
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CxIo6jJbGJKQofxSRDc6uOCOK1g>
Subject: [bmwg] Comments on draft- huang-bmwg-virtual-network-performance-02
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Mar 2017 13:28:52 -0000

--_000_DM5PR10MB161005B77BD2E308253A08B49F390DM5PR10MB1610namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Given that there have been some issues raised on the draft- huang-bmwg-virt=
ual-network-performance-02, I have reviewed the draft and have identified t=
he following issues/problems that I believe need to be addressed:

Most importantly, I have a concern with the "virtual tester" sharing the sa=
me resources as the DUT(vSwitch) that is being tested and its effect on tes=
ting results. Specifically, by allocating host server resources(i.e. - vCPU=
's/Memory) to the virtual tester, one virtual tester using test model A and=
 two virtual testers using test model B, those resources cannot by assigned=
 to the DUT. This might reduce the DUT performance results, such as lower t=
hroughput or higher latencies that might otherwise be attained if the afore=
mentioned  resources were assigned to the DUT.

Throughput Test(Section 6.1)



In the throughput test process described in section 6.1.4(step 2) it notes =
to:



"Increase the number of vCPU's in the tester until the traffic has no packe=
t loss"



If the packet loss is a performance limitation of the DUT(vSwitch), then it=
 seems to me that assigning more vCPU's to the virtual tester will have no =
effect on the packet loss and that the additional vCPU resources would need=
 to be increased on the DUT(vSwitch) side, if available. If, in fact, incre=
asing  the vCPU's on the virtual tester reduces or eliminates the packet lo=
ss, it seems to me to be an indication that there is a performance issue wi=
th the virtual tester in its ability to process incoming test traffic.


Latency Test(Section 6.5)



In section 6.5, it discusses use of model A setup and substituting the virt=
ual tester with an echo server. Then, assuming the "one-way delay" is half =
of the round trip. I don't believe it's a safe assumption that the one-way =
delay is half of the round trip time. There could be significant delta's be=
tween the two directions.

In addition, the setup for performing the latency test is not the same as t=
he setup for the throughput test. Specifically, replacing the virtual teste=
r with an echo server. While I understand the intent is to provide for a mo=
re accurate latency measurement, my concern is that the performance of the =
echo server is not known. Traffic is offered at the throughput determined i=
n section 6.1, but will the echo server be able to handle the throughput ra=
te? If not, then the latency measurement would include buffering by the ech=
o server or worse, the packet(s) that is tagged for the latency measurement=
 gets dropped altogether.



Also, is the reported latency the round trip measurement or the delta betwe=
en with and without the DUT?



Test Consideration(Section 3)


Not saying there are issues here, but clarification may be helpful as I am =
unclear on the calibration requirements described in this section. Specific=
ally, the section notes:


"Just like the DUT, tester running as software will have its own performanc=
e peak under various conditions.

And


"Tester's calibration is essential in benchmarking testing in a virtual env=
ironment. Furthermore, to reduce the enormous combination

of various conditions, tester must be calibrated with the exact same combin=
ation and parameter settings the user wants to measure against the DUT."

This, to me, infers that baseline testing of the virtual testers may be req=
uired, prior to testing the DUT(vSwitch) . In addition, those same paramete=
rs used in the baseline testing must remain in force once the DUT(vSwitch) =
is introduced. Is this correct?





Spirent Communications e-mail confidentiality.
------------------------------------------------------------------------
This e-mail contains confidential and / or privileged information belonging=
 to Spirent Communications plc, its affiliates and / or subsidiaries. If yo=
u are not the intended recipient, you are hereby notified that any disclosu=
re, copying, distribution and / or the taking of any action based upon reli=
ance on the contents of this transmission is strictly forbidden. If you hav=
e received this message in error please notify the sender by return e-mail =
and delete it from your system.

Spirent Communications plc
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdo=
m.
Tel No. +44 (0) 1293 767676
Fax No. +44 (0) 1293 767677

Registered in England Number 470893
Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN,=
 United Kingdom.

Or if within the US,

Spirent Communications,
27349 Agoura Road, Calabasas, CA, 91301, USA.
Tel No. 1-818-676- 2300

--_000_DM5PR10MB161005B77BD2E308253A08B49F390DM5PR10MB1610namp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
	{mso-style-priority:99;
	mso-style-link:"Plain Text Char";
	margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
pre
	{mso-style-priority:99;
	mso-style-link:"HTML Preformatted Char";
	margin:0in;
	margin-bottom:.0001pt;
	font-size:10.0pt;
	font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
	{mso-style-priority:34;
	margin-top:0in;
	margin-right:0in;
	margin-bottom:0in;
	margin-left:.5in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
span.PlainTextChar
	{mso-style-name:"Plain Text Char";
	mso-style-priority:99;
	mso-style-link:"Plain Text";
	font-family:"Calibri","sans-serif";}
span.HTMLPreformattedChar
	{mso-style-name:"HTML Preformatted Char";
	mso-style-priority:99;
	mso-style-link:"HTML Preformatted";
	font-family:"Courier New";}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
/* List Definitions */
@list l0
	{mso-list-id:2126266548;
	mso-list-type:hybrid;
	mso-list-template-ids:-1375449894 67698705 67698713 67698715 67698703 6769=
8713 67698715 67698703 67698713 67698715;}
@list l0:level1
	{mso-level-text:"%1\)";
	mso-level-tab-stop:none;
	mso-level-number-position:left;
	text-indent:-.25in;}
@list l0:level2
	{mso-level-number-format:alpha-lower;
	mso-level-tab-stop:none;
	mso-level-number-position:left;
	text-indent:-.25in;}
@list l0:level3
	{mso-level-number-format:roman-lower;
	mso-level-tab-stop:none;
	mso-level-number-position:right;
	text-indent:-9.0pt;}
@list l0:level4
	{mso-level-tab-stop:none;
	mso-level-number-position:left;
	text-indent:-.25in;}
@list l0:level5
	{mso-level-number-format:alpha-lower;
	mso-level-tab-stop:none;
	mso-level-number-position:left;
	text-indent:-.25in;}
@list l0:level6
	{mso-level-number-format:roman-lower;
	mso-level-tab-stop:none;
	mso-level-number-position:right;
	text-indent:-9.0pt;}
@list l0:level7
	{mso-level-tab-stop:none;
	mso-level-number-position:left;
	text-indent:-.25in;}
@list l0:level8
	{mso-level-number-format:alpha-lower;
	mso-level-tab-stop:none;
	mso-level-number-position:left;
	text-indent:-.25in;}
@list l0:level9
	{mso-level-number-format:roman-lower;
	mso-level-tab-stop:none;
	mso-level-number-position:right;
	text-indent:-9.0pt;}
ol
	{margin-bottom:0in;}
ul
	{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Given that there have been some issues raised on the=
 draft- huang-bmwg-virtual-network-performance-02, I have reviewed the draf=
t and have identified the following issues/problems that I believe need to =
be addressed:<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Most importantly, I have a concern with the &#8220;v=
irtual tester&#8221; sharing the same resources as the DUT(vSwitch) that is=
 being tested and its effect on testing results. Specifically, by allocatin=
g host server resources(i.e. &#8211; vCPU&#8217;s/Memory)
 to the virtual tester, one virtual tester using test model A and two virtu=
al testers using test model B, those resources cannot by assigned to the DU=
T. This might reduce the DUT performance results, such as lower throughput =
or higher latencies that might otherwise
 be attained if the aforementioned &nbsp;resources were assigned to the DUT=
.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal"><b>Throughput Test(Section 6.1)<o:p></o:p></b></p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoPlainText">In the throughput test process described in secti=
on 6.1.4(step 2) it notes to:<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoPlainText" style=3D"text-indent:.5in">&quot;Increase the num=
ber of vCPU's in the tester until the traffic has no packet loss&quot;<o:p>=
</o:p></p>
<p class=3D"MsoPlainText">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<o:p></o:p></p>
<p class=3D"MsoPlainText">If the packet loss is a performance limitation of=
 the DUT(vSwitch), then it seems to me that assigning more vCPU's to the vi=
rtual tester will have no effect on the packet loss and that the additional=
 vCPU resources would need to be increased
 on the DUT(vSwitch) side, if available. If, in fact, increasing &nbsp;the =
vCPU&#8217;s on the virtual tester reduces or eliminates the packet loss, i=
t seems to me to be an indication that there is a performance issue with th=
e virtual tester in its ability to process
 incoming test traffic.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal"><b>Latency Test(Section 6.5)<o:p></o:p></b></p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoPlainText">In section 6.5, it discusses use of model A setup=
 and substituting the virtual tester with an echo server. Then, assuming th=
e &quot;one-way delay&quot; is half of the round trip. I don&#8217;t believ=
e it&#8217;s a safe assumption that the one-way delay is
 half of the round trip time. There could be significant delta's between th=
e two directions.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p></o:p></p>
<p class=3D"MsoPlainText">In addition, the setup for performing the latency=
 test is not the same as the setup for the throughput test. Specifically, r=
eplacing the virtual tester with an echo server. While I understand the int=
ent is to provide for a more accurate
 latency measurement, my concern is that the performance of the echo server=
 is not known. Traffic is offered at the throughput determined in section 6=
.1, but will the echo server be able to handle the throughput rate? If not,=
 then the latency measurement would
 include buffering by the echo server or worse, the packet(s) that is tagge=
d for the latency measurement gets dropped altogether.
<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoPlainText">Also, is the reported latency the round trip meas=
urement or the delta between with and without the DUT?<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoPlainText"><b>Test Consideration(Section 3)<o:p></o:p></b></=
p>
<p class=3D"MsoPlainText"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Not saying there are issues here, but clarification =
may be helpful as I am unclear on the calibration requirements described in=
 this section. Specifically, the section notes:<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<pre>&#8220;Just like the DUT, tester running as software will have its own=
 performance peak under various conditions.<o:p></o:p></pre>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">And<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<pre>&#8220;Tester's calibration is essential in benchmarking testing in a =
virtual environment. Furthermore, to reduce the enormous combination<o:p></=
o:p></pre>
<pre>of various conditions, tester must be calibrated with the exact same c=
ombination and parameter settings the user wants to measure against the DUT=
.&#8221;<o:p></o:p></pre>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">This, to me, infers that baseline testing of the vir=
tual testers may be required, prior to testing the DUT(vSwitch) . In additi=
on, those same parameters used in the baseline testing must remain in force=
 once the DUT(vSwitch) is introduced.
 Is this correct?<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<br>
<br>
<br>
<br>
<div><font size=3D"1">Spirent Communications e-mail confidentiality.<br>
------------------------------------------------------------------------<br=
>
This e-mail contains confidential and / or privileged information belonging=
 to Spirent Communications plc, its affiliates and / or subsidiaries. If yo=
u are not the intended recipient, you are hereby notified that any disclosu=
re, copying, distribution and /
 or the taking of any action based upon reliance on the contents of this tr=
ansmission is strictly forbidden. If you have received this message in erro=
r please notify the sender by return e-mail and delete it from your system.=
<br>
<br>
Spirent Communications plc<br>
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdo=
m.<br>
Tel No. &#43;44 (0) 1293 767676<br>
Fax No. &#43;44 (0) 1293 767677<br>
<br>
Registered in England Number 470893<br>
Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN,=
 United Kingdom.<br>
<br>
Or if within the US, <br>
<br>
Spirent Communications,<br>
27349 Agoura Road, Calabasas, CA, 91301, USA.<br>
Tel No. 1-818-676- 2300 <br>
</font></div>
</body>
</html>

--_000_DM5PR10MB161005B77BD2E308253A08B49F390DM5PR10MB1610namp_--


From nobody Fri Mar 17 08:08:37 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 095BA12947A; Fri, 17 Mar 2017 08:08:35 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.47.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <148976331490.14193.16357702669003644658@ietfa.amsl.com>
Date: Fri, 17 Mar 2017 08:08:34 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CeiC1x_ucES9W6raLLrkgit978c>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-virtual-net-05.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Mar 2017 15:08:35 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
        Author          : Al Morton
	Filename        : draft-ietf-bmwg-virtual-net-05.txt
	Pages           : 15
	Date            : 2017-03-17

Abstract:
   The Benchmarking Methodology Working Group has traditionally
   conducted laboratory characterization of dedicated physical
   implementations of internetworking functions.  This memo investigates
   additional considerations when network functions are virtualized and
   performed in general purpose hardware.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-virtual-net-05


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Mon Mar 20 14:44:01 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E6CE128BE1; Mon, 20 Mar 2017 14:43:47 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.47.2
Auto-Submitted: auto-generated
Precedence: bulk
Cc: Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, The IESG <iesg@ietf.org>, joelja@gmail.com, draft-ietf-bmwg-virtual-net@ietf.org, bmwg@ietf.org, sbanks@encrypted.net, rfc-editor@rfc-editor.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149004622746.25009.8249778727650865098.idtracker@ietfa.amsl.com>
Date: Mon, 20 Mar 2017 14:43:47 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/RKGjCItmepuKrQRXSNk3L19AIyg>
Subject: [bmwg] Document Action: 'Considerations for Benchmarking Virtual Network Functions and Their Infrastructure' to Informational RFC (draft-ietf-bmwg-virtual-net-05.txt)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Mar 2017 21:43:48 -0000

The IESG has approved the following document:
- 'Considerations for Benchmarking Virtual Network Functions and Their
   Infrastructure'
  (draft-ietf-bmwg-virtual-net-05.txt) as Informational RFC

This document is the product of the Benchmarking Methodology Working
Group.

The IESG contact persons are Benoit Claise and Joel Jaeggli.

A URL of this Internet Draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/





Technical Summary

This draft examines considerations for benchmarking virtualized network
 functions (VNFs) and their supported infrastructure, with consideration 
for general-purpose hardware, and considers benchmarking both physical 
and virtual network functions in the same fashion, allowing for comparison 
between the two, where possible.

Working Group Summary

There has been a fair amount of work done on this draft, and progress 
made on revisions, feedback, and comments. Several presentations have 
been made in the room during IETF meetings, and followup and discussion 
taken to the BMWG list. This draft is particularly useful, given the popularity 
of VNF's within the industry.

Personnel

Sarah Banks is Shepherd, Joel Jaeggli is AD.E AD>.'


From nobody Thu Mar 23 09:55:56 2017
Return-Path: <hlisname@yahoo.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 13D30129A4E for <bmwg@ietfa.amsl.com>; Thu, 23 Mar 2017 09:55:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yahoo.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8zCpmzMcb3PX for <bmwg@ietfa.amsl.com>; Thu, 23 Mar 2017 09:55:52 -0700 (PDT)
Received: from sonic322-2.consmr.mail.ne1.yahoo.com (sonic322-2.consmr.mail.ne1.yahoo.com [66.163.189.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 83EC3129A26 for <bmwg@ietf.org>; Thu, 23 Mar 2017 09:55:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1490288150; bh=RSstFfVyHyOOzo4jVUdrlV9jTFSbcQEsylJbxI84Yns=; h=Date:From:Reply-To:To:In-Reply-To:References:Subject:From:Subject; b=ibnEc1XBqn92rKNsR4L7ZQbOPIwukfP1vQXreiBc3WuUz71wZUBfIFJRXtwBKgdFi39ZUzFN9A0nYGPtBiw1L3h6HE8BzN/gTXZU3J7gRSr/QF3QDvvT+fnu7vaLBHy2W7tM3c5jo0qh0eFNfEgtMbkBgTxILADe/m8eJOC7ANSyx0RQtYKD+eNjCjNGVqjcM0DLKUUekw7nP7iZR1POs64Z0M+8TpxBD/5f6aqZKLklLhSPGt5Useq7ywGmHSYJoErif9escp4jtMyj5MPph8+FpcA8mydSDzHQuXXeS4TbL/CEbz5mXcLehd16GCjycTI8R8BiUxLMXATpt7TE1Q==
X-YMail-OSG: AuhL91MVM1nD_gts4ZhazAQn68HYKplGKCFvO6WwbDNAguTy0tkud_NFCGPsb9f DBO27.k1CLhEcgQkQPsXYJHZ5r3bUXIsWWiv_YOVQ54XrpoOWIdTc_e2WPM_pMnTS8_A_i.hMVIl 6SJFQdfNiEsCeeKNEni2x7D9ZZU3.avCjV8unGF1p0zgEKrYgaiWcNI_Z16nY23thVwTwwLVfxHd 9WeP8TU63YL.RLoU8h1NiV25QOxuljiZdV2YGOL4RkLyhxx5A9W5i18zjz7rpQdvDTCJGYjmsvIX WXK7S.Z8UR4CrS1yCyHynj4vDkRw.OzWMUCAWtni7OuhR7vOOEaRRgnBJLXeEwoCa9c95gXnTw9Z YCe9YZQ2RPfkCMtwx8o8EImfA89EGgTnnniHkbB_TeGuUMP9Xmo7g_LS4waObBaJS5Tarfz3Js5P j3lnMTZbFaBr7M6ONHmByUz94qShbHQSH6K2Tlb6Gail6eVaHLZhHgnjBwZv3V1IzeSRgjLYqK1k zP_XZMuA-
Received: from sonic.gate.mail.ne1.yahoo.com by sonic322.consmr.mail.ne1.yahoo.com with HTTP; Thu, 23 Mar 2017 16:55:50 +0000
Date: Thu, 23 Mar 2017 16:51:50 +0000 (UTC)
From: LuHuang <hlisname@yahoo.com>
Reply-To: LuHuang <hlisname@yahoo.com>
To: "Hickman, Brooks" <Brooks.Hickman@spirent.com>,  "bmwg@ietf.org" <bmwg@ietf.org>
Message-ID: <1627434345.2299847.1490287910166@mail.yahoo.com>
In-Reply-To: <DM5PR10MB161005B77BD2E308253A08B49F390@DM5PR10MB1610.namprd10.prod.outlook.com>
References: <DM5PR10MB161005B77BD2E308253A08B49F390@DM5PR10MB1610.namprd10.prod.outlook.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;  boundary="----=_Part_2299846_926477953.1490287910162"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/gH9yasTqSKeMgdivbbTHepOese0>
Subject: [bmwg] =?utf-8?b?5Zue5aSN77yaICBDb21tZW50cyBvbiBkcmFmdC0gaHVhbmct?= =?utf-8?q?bmwg-virtual-network-performance-02?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Mar 2017 16:55:55 -0000

------=_Part_2299846_926477953.1490287910162
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi, Hickman
Wish the following could be helpful:
1. In the real network, DUT must be running with VMs for different applicat=
ions. Virtual tester is a kind of application, so sharing the same resource=
 seems make sense. Of course virtual tester will somehow impact the perform=
ance of DUT. That's why we should calibrate virtual tester beforehand to fi=
nd its own performance under different CPU/MEM allocations. Then we can ret=
ain enough resources for DUT.
2. Throughput Test: I agree with you. When packets loss happens, we should =
decrease the traffic instead of increasing vCPU for tester.
3. Latency Test: Yes, we can use Model A or B if all testers could become t=
ime synchronized.
4. Test consideration: You are right. Firstly find the baseline of virtual =
tester, then use it in a holiday.=C2=A0Thanks a lot!
--------------------LuHuang
China Mobile Research Institute
Mobile: +86 13810820540
=20

    "Hickman, Brooks" <Brooks.Hickman@spirent.com> =E4=BA=8E 2017=E5=B9=B43=
=E6=9C=8817=E6=97=A5, =E6=98=9F=E6=9C=9F=E4=BA=94, =E4=B8=8B=E5=8D=88 9:29 =
=E5=86=99=E9=81=93=EF=BC=9A
=20

  <!--#yiv3122636935 _filtered #yiv3122636935 {font-family:Calibri;panose-1=
:2 15 5 2 2 2 4 3 2 4;}#yiv3122636935 #yiv3122636935 p.yiv3122636935MsoNorm=
al, #yiv3122636935 li.yiv3122636935MsoNormal, #yiv3122636935 div.yiv3122636=
935MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family=
:"Calibri", "sans-serif";}#yiv3122636935 a:link, #yiv3122636935 span.yiv312=
2636935MsoHyperlink {color:blue;text-decoration:underline;}#yiv3122636935 a=
:visited, #yiv3122636935 span.yiv3122636935MsoHyperlinkFollowed {color:purp=
le;text-decoration:underline;}#yiv3122636935 p.yiv3122636935MsoPlainText, #=
yiv3122636935 li.yiv3122636935MsoPlainText, #yiv3122636935 div.yiv312263693=
5MsoPlainText {margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-famil=
y:"Calibri", "sans-serif";}#yiv3122636935 pre {margin:0in;margin-bottom:.00=
01pt;font-size:10.0pt;font-family:"Courier New";}#yiv3122636935 p.yiv312263=
6935MsoListParagraph, #yiv3122636935 li.yiv3122636935MsoListParagraph, #yiv=
3122636935 div.yiv3122636935MsoListParagraph {margin-top:0in;margin-right:0=
in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-size:11.0p=
t;font-family:"Calibri", "sans-serif";}#yiv3122636935 span.yiv3122636935Ema=
ilStyle17 {font-family:"Calibri", "sans-serif";color:windowtext;}#yiv312263=
6935 span.yiv3122636935PlainTextChar {font-family:"Calibri", "sans-serif";}=
#yiv3122636935 span.yiv3122636935HTMLPreformattedChar {font-family:"Courier=
 New";}#yiv3122636935 .yiv3122636935MsoChpDefault {font-family:"Calibri", "=
sans-serif";} _filtered #yiv3122636935 {margin:1.0in 1.0in 1.0in 1.0in;}#yi=
v3122636935 div.yiv3122636935WordSection1 {}#yiv3122636935 _filtered #yiv31=
22636935 {} _filtered #yiv3122636935 {} _filtered #yiv3122636935 {} _filter=
ed #yiv3122636935 {} _filtered #yiv3122636935 {} _filtered #yiv3122636935 {=
} _filtered #yiv3122636935 {} _filtered #yiv3122636935 {} _filtered #yiv312=
2636935 {} _filtered #yiv3122636935 {}#yiv3122636935 ol {margin-bottom:0in;=
}#yiv3122636935 ul {margin-bottom:0in;}-->Given that there have been some i=
ssues raised on the draft- huang-bmwg-virtual-network-performance-02, I hav=
e reviewed the draft and have identified the following issues/problems that=
 I believe need to be addressed:  =C2=A0 Most importantly, I have a concern=
 with the =E2=80=9Cvirtual tester=E2=80=9D sharing the same resources as th=
e DUT(vSwitch) that is being tested and its effect on testing results. Spec=
ifically, by allocating host server resources(i.e. =E2=80=93 vCPU=E2=80=99s=
/Memory) to the virtual tester, one virtual tester using test model A and t=
wo virtual testers using test model B, those resources cannot by assigned t=
o the DUT. This might reduce the DUT performance results, such as lower thr=
oughput or higher latencies that might otherwise be attained if the aforeme=
ntioned =C2=A0resources were assigned to the DUT.  =C2=A0 Throughput Test(S=
ection 6.1)  =C2=A0 In the throughput test process described in section 6.1=
.4(step 2) it notes to:  =C2=A0 "Increase the number of vCPU's in the teste=
r until the traffic has no packet loss" =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 If the p=
acket loss is a performance limitation of the DUT(vSwitch), then it seems t=
o me that assigning more vCPU's to the virtual tester will have no effect o=
n the packet loss and that the additional vCPU resources would need to be i=
ncreased on the DUT(vSwitch) side, if available. If, in fact, increasing =
=C2=A0the vCPU=E2=80=99s on the virtual tester reduces or eliminates the pa=
cket loss, it seems to me to be an indication that there is a performance i=
ssue with the virtual tester in its ability to process incoming test traffi=
c.  =C2=A0 Latency Test(Section 6.5)  =C2=A0 In section 6.5, it discusses u=
se of model A setup and substituting the virtual tester with an echo server=
. Then, assuming the "one-way delay" is half of the round trip. I don=E2=80=
=99t believe it=E2=80=99s a safe assumption that the one-way delay is half =
of the round trip time. There could be significant delta's between the two =
directions.  In addition, the setup for performing the latency test is not =
the same as the setup for the throughput test. Specifically, replacing the =
virtual tester with an echo server. While I understand the intent is to pro=
vide for a more accurate latency measurement, my concern is that the perfor=
mance of the echo server is not known. Traffic is offered at the throughput=
 determined in section 6.1, but will the echo server be able to handle the =
throughput rate? If not, then the latency measurement would include bufferi=
ng by the echo server or worse, the packet(s) that is tagged for the latenc=
y measurement gets dropped altogether.  =C2=A0 Also, is the reported latenc=
y the round trip measurement or the delta between with and without the DUT?=
  =C2=A0 Test Consideration(Section 3)  =C2=A0 Not saying there are issues =
here, but clarification may be helpful as I am unclear on the calibration r=
equirements described in this section. Specifically, the section notes:  =
=C2=A0 =E2=80=9CJust like the DUT, tester running as software will have its=
 own performance peak under various conditions.  =C2=A0 And  =C2=A0 =E2=80=
=9CTester's calibration is essential in benchmarking testing in a virtual e=
nvironment. Furthermore, to reduce the enormous combination of various cond=
itions, tester must be calibrated with the exact same combination and param=
eter settings the user wants to measure against the DUT.=E2=80=9D  =C2=A0 T=
his, to me, infers that baseline testing of the virtual testers may be requ=
ired, prior to testing the DUT(vSwitch) . In addition, those same parameter=
s used in the baseline testing must remain in force once the DUT(vSwitch) i=
s introduced. Is this correct?  =C2=A0=20



Spirent Communications e-mail confidentiality.
------------------------------------------------------------------------
This e-mail contains confidential and / or privileged information belonging=
 to Spirent Communications plc, its affiliates and / or subsidiaries. If yo=
u are not the intended recipient, you are hereby notified that any disclosu=
re, copying, distribution and / or the taking of any action based upon reli=
ance on the contents of this transmission is strictly forbidden. If you hav=
e received this message in error please notify the sender by return e-mail =
and delete it from your system.

Spirent Communications plc
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdo=
m.
Tel No. +44 (0) 1293 767676
Fax No. +44 (0) 1293 767677

Registered in England Number 470893
Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN,=
 United Kingdom.

Or if within the US,=20

Spirent Communications,
27349 Agoura Road, Calabasas, CA, 91301, USA.
Tel No. 1-818-676- 2300=20
_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


  =20
------=_Part_2299846_926477953.1490287910162
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<html><head></head><body><div style=3D"color:#000; background-color:#fff; f=
ont-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font=
-size:16px"><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1490280864998_11296"><=
span>Hi, Hickman</span></div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_14902=
80864998_11061"><span><br></span></div><div dir=3D"ltr" id=3D"yui_3_16_0_ym=
19_1_1490280864998_11062"><span id=3D"yui_3_16_0_ym19_1_1490280864998_11199=
">Wish the following could be helpful:</span></div><div dir=3D"ltr" id=3D"y=
ui_3_16_0_ym19_1_1490280864998_11062"><span><br></span></div><div dir=3D"lt=
r" id=3D"yui_3_16_0_ym19_1_1490280864998_11156">1. In the real network, DUT=
 must be running with VMs for different applications. Virtual tester is a k=
ind of application, so sharing the same resource seems make sense. Of cours=
e virtual tester will somehow impact the performance of DUT. That's why we =
should calibrate virtual tester beforehand to find its own performance unde=
r different CPU/MEM allocations. Then we can retain enough resources for DU=
T.</div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1490280864998_11157"><br i=
d=3D"yui_3_16_0_ym19_1_1490280864998_11158"></div><div dir=3D"ltr" id=3D"yu=
i_3_16_0_ym19_1_1490280864998_11159">2. Throughput Test: I agree with you. =
When packets loss happens, we should decrease the traffic instead of increa=
sing vCPU for tester.</div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1490280=
864998_11160"><br id=3D"yui_3_16_0_ym19_1_1490280864998_11161"></div><div d=
ir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1490280864998_11162">3. Latency Test: Ye=
s, we can use Model A or B if all testers could become time synchronized.</=
div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1490280864998_11163"><br id=3D=
"yui_3_16_0_ym19_1_1490280864998_11164"></div><div dir=3D"ltr" id=3D"yui_3_=
16_0_ym19_1_1490280864998_11062"></div><div dir=3D"ltr" id=3D"yui_3_16_0_ym=
19_1_1490280864998_11165">4. Test consideration: You are right. Firstly fin=
d the baseline of virtual tester, then use it in a holiday.</div><div></div=
><div id=3D"yui_3_16_0_ym19_1_1490280864998_11060">&nbsp;</div><div id=3D"y=
ui_3_16_0_ym19_1_1490280864998_11060">Thanks a lot!</div><div id=3D"yui_3_1=
6_0_ym19_1_1490280864998_11060"><br></div><div class=3D"signature" id=3D"yu=
i_3_16_0_ym19_1_1490280864998_11057">--------------------<div id=3D"yui_3_1=
6_0_ym19_1_1490280864998_11096"><div class=3D"MsoNormal" style=3D"text-alig=
n:justify;" id=3D"yui_3_16_0_ym19_1_1490280864998_11095"><span lang=3D"EN-U=
S" style=3D"font-size:10.0pt;color:#1F497D;" id=3D"yui_3_16_0_ym19_1_149028=
0864998_11166">LuHuang<br>China Mobile Research Institute<br>
Mobile: +86 13810820540<br></span></div></div></div> <div class=3D"qtdSepar=
ateBR"><br><br></div><div class=3D"yahoo_quoted" style=3D"display: block;">=
 <div style=3D"font-family: Helvetica Neue, Helvetica, Arial, Lucida Grande=
, sans-serif; font-size: 16px;"> <div style=3D"font-family: HelveticaNeue, =
Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16p=
x;"> <div dir=3D"ltr"><font size=3D"2" face=3D"Arial"> "Hickman, Brooks" &l=
t;Brooks.Hickman@spirent.com&gt; =E4=BA=8E 2017=E5=B9=B43=E6=9C=8817=E6=97=
=A5, =E6=98=9F=E6=9C=9F=E4=BA=94, =E4=B8=8B=E5=8D=88 9:29 =E5=86=99=E9=81=
=93=EF=BC=9A<br></font></div>  <br><br> <div class=3D"y_msg_container"><div=
 id=3D"yiv3122636935">

=20
=20
<style><!--
#yiv3122636935 =20
 _filtered #yiv3122636935 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 =
4;}
#yiv3122636935 =20
#yiv3122636935 p.yiv3122636935MsoNormal, #yiv3122636935 li.yiv3122636935Mso=
Normal, #yiv3122636935 div.yiv3122636935MsoNormal
=09{margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri"=
, "sans-serif";}
#yiv3122636935 a:link, #yiv3122636935 span.yiv3122636935MsoHyperlink
=09{color:blue;text-decoration:underline;}
#yiv3122636935 a:visited, #yiv3122636935 span.yiv3122636935MsoHyperlinkFoll=
owed
=09{color:purple;text-decoration:underline;}
#yiv3122636935 p.yiv3122636935MsoPlainText, #yiv3122636935 li.yiv3122636935=
MsoPlainText, #yiv3122636935 div.yiv3122636935MsoPlainText
=09{margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri"=
, "sans-serif";}
#yiv3122636935 pre
=09{margin:0in;margin-bottom:.0001pt;font-size:10.0pt;font-family:"Courier =
New";}
#yiv3122636935 p.yiv3122636935MsoListParagraph, #yiv3122636935 li.yiv312263=
6935MsoListParagraph, #yiv3122636935 div.yiv3122636935MsoListParagraph
=09{margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;marg=
in-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", "sans-serif";}
#yiv3122636935 span.yiv3122636935EmailStyle17
=09{font-family:"Calibri", "sans-serif";color:windowtext;}
#yiv3122636935 span.yiv3122636935PlainTextChar
=09{font-family:"Calibri", "sans-serif";}
#yiv3122636935 span.yiv3122636935HTMLPreformattedChar
=09{font-family:"Courier New";}
#yiv3122636935 .yiv3122636935MsoChpDefault
=09{font-family:"Calibri", "sans-serif";}
 _filtered #yiv3122636935 {margin:1.0in 1.0in 1.0in 1.0in;}
#yiv3122636935 div.yiv3122636935WordSection1
=09{}
#yiv3122636935 =20
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
 _filtered #yiv3122636935 {}
#yiv3122636935 ol
=09{margin-bottom:0in;}
#yiv3122636935 ul
=09{margin-bottom:0in;}
--></style>

<div>
<div class=3D"yiv3122636935WordSection1">
<div class=3D"yiv3122636935MsoNormal">Given that there have been some issue=
s raised on the draft- huang-bmwg-virtual-network-performance-02, I have re=
viewed the draft and have identified the following issues/problems that I b=
elieve need to be addressed:</div>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoNormal">Most importantly, I have a concern wi=
th the =E2=80=9Cvirtual tester=E2=80=9D sharing the same resources as the D=
UT(vSwitch) that is being tested and its effect on testing results. Specifi=
cally, by allocating host server resources(i.e. =E2=80=93 vCPU=E2=80=99s/Me=
mory)
 to the virtual tester, one virtual tester using test model A and two virtu=
al testers using test model B, those resources cannot by assigned to the DU=
T. This might reduce the DUT performance results, such as lower throughput =
or higher latencies that might otherwise
 be attained if the aforementioned &nbsp;resources were assigned to the DUT=
.</div>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoNormal"><b>Throughput Test(Section 6.1)</b></=
div>=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoPlainText">In the throughput test process des=
cribed in section 6.1.4(step 2) it notes to:</div>=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoPlainText" style=3D"text-indent:.5in;">"Incre=
ase the number of vCPU's in the tester until the traffic has no packet loss=
"</div>=20
<div class=3D"yiv3122636935MsoPlainText">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</div>=20
<div class=3D"yiv3122636935MsoPlainText">If the packet loss is a performanc=
e limitation of the DUT(vSwitch), then it seems to me that assigning more v=
CPU's to the virtual tester will have no effect on the packet loss and that=
 the additional vCPU resources would need to be increased
 on the DUT(vSwitch) side, if available. If, in fact, increasing &nbsp;the =
vCPU=E2=80=99s on the virtual tester reduces or eliminates the packet loss,=
 it seems to me to be an indication that there is a performance issue with =
the virtual tester in its ability to process
 incoming test traffic.</div>=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoNormal"><b>Latency Test(Section 6.5)</b></div=
>=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoPlainText">In section 6.5, it discusses use o=
f model A setup and substituting the virtual tester with an echo server. Th=
en, assuming the "one-way delay" is half of the round trip. I don=E2=80=99t=
 believe it=E2=80=99s a safe assumption that the one-way delay is
 half of the round trip time. There could be significant delta's between th=
e two directions.</div>=20
<div class=3D"yiv3122636935MsoPlainText"></div>=20
<div class=3D"yiv3122636935MsoPlainText">In addition, the setup for perform=
ing the latency test is not the same as the setup for the throughput test. =
Specifically, replacing the virtual tester with an echo server. While I und=
erstand the intent is to provide for a more accurate
 latency measurement, my concern is that the performance of the echo server=
 is not known. Traffic is offered at the throughput determined in section 6=
.1, but will the echo server be able to handle the throughput rate? If not,=
 then the latency measurement would
 include buffering by the echo server or worse, the packet(s) that is tagge=
d for the latency measurement gets dropped altogether.
</div>=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoPlainText">Also, is the reported latency the =
round trip measurement or the delta between with and without the DUT?</div>=
=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoPlainText"><b>Test Consideration(Section 3)</=
b></div>=20
<div class=3D"yiv3122636935MsoPlainText"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoNormal">Not saying there are issues here, but=
 clarification may be helpful as I am unclear on the calibration requiremen=
ts described in this section. Specifically, the section notes:</div>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
<pre>=E2=80=9CJust like the DUT, tester running as software will have its o=
wn performance peak under various conditions.</pre>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoNormal">And</div>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
<pre>=E2=80=9CTester's calibration is essential in benchmarking testing in =
a virtual environment. Furthermore, to reduce the enormous combination</pre=
>=20
<pre>of various conditions, tester must be calibrated with the exact same c=
ombination and parameter settings the user wants to measure against the DUT=
.=E2=80=9D</pre>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
<div class=3D"yiv3122636935MsoNormal">This, to me, infers that baseline tes=
ting of the virtual testers may be required, prior to testing the DUT(vSwit=
ch) . In addition, those same parameters used in the baseline testing must =
remain in force once the DUT(vSwitch) is introduced.
 Is this correct?</div>=20
<div class=3D"yiv3122636935MsoNormal"> &nbsp;</div>=20
</div>
<br>
<br>
<br>
<br>
<div><font size=3D"1">Spirent Communications e-mail confidentiality.<br>
------------------------------------------------------------------------<br=
>
This e-mail contains confidential and / or privileged information belonging=
 to Spirent Communications plc, its affiliates and / or subsidiaries. If yo=
u are not the intended recipient, you are hereby notified that any disclosu=
re, copying, distribution and /
 or the taking of any action based upon reliance on the contents of this tr=
ansmission is strictly forbidden. If you have received this message in erro=
r please notify the sender by return e-mail and delete it from your system.=
<br>
<br>
Spirent Communications plc<br>
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdo=
m.<br>
Tel No. +44 (0) 1293 767676<br>
Fax No. +44 (0) 1293 767677<br>
<br>
Registered in England Number 470893<br>
Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN,=
 United Kingdom.<br>
<br>
Or if within the US, <br>
<br>
Spirent Communications,<br>
27349 Agoura Road, Calabasas, CA, 91301, USA.<br>
Tel No. 1-818-676- 2300 <br>
</font></div>
</div>
</div>_______________________________________________<br>bmwg mailing list<=
br><a ymailto=3D"mailto:bmwg@ietf.org" href=3D"mailto:bmwg@ietf.org">bmwg@i=
etf.org</a><br><a href=3D"https://www.ietf.org/mailman/listinfo/bmwg" targe=
t=3D"_blank">https://www.ietf.org/mailman/listinfo/bmwg</a><br><br><br></di=
v>  </div> </div>  </div></div></body></html>
------=_Part_2299846_926477953.1490287910162--


From nobody Tue Mar 28 17:31:39 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DEB61129479 for <bmwg@ietfa.amsl.com>; Tue, 28 Mar 2017 17:31:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.396
X-Spam-Level: 
X-Spam-Status: No, score=-5.396 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.796, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TafTSSwu61mr for <bmwg@ietfa.amsl.com>; Tue, 28 Mar 2017 17:31:37 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C597A12704A for <bmwg@ietf.org>; Tue, 28 Mar 2017 17:31:36 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v2T0OvMG039342 for <bmwg@ietf.org>; Tue, 28 Mar 2017 20:31:34 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049463.ppops.net-00191d01. with ESMTP id 29g0aubajp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Tue, 28 Mar 2017 20:31:33 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v2T0VX9l003673 for <bmwg@ietf.org>; Tue, 28 Mar 2017 20:31:33 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v2T0VSHK003609 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Tue, 28 Mar 2017 20:31:29 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor) for <bmwg@ietf.org>; Wed, 29 Mar 2017 00:31:16 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v2T0VFJx015759 for <bmwg@ietf.org>; Tue, 28 Mar 2017 19:31:15 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v2T0V5KA015220 for <bmwg@ietf.org>; Tue, 28 Mar 2017 19:31:08 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 90555F0531 for <bmwg@ietf.org>; Tue, 28 Mar 2017 20:31:05 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 28 Mar 2017 20:31:05 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Draft Agenda for IETF-98 (Slides!)
Thread-Index: AQHSqCO/EqX1F2ZRT0CqXC+Q5eflFQ==
Date: Wed, 29 Mar 2017 00:31:04 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F3C521@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [31.133.146.107]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-28_20:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703290001
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/uU523d-moVd0LU0-dQRYT1COqNE>
Subject: Re: [bmwg] Draft Agenda for IETF-98 (Slides!)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 29 Mar 2017 00:31:39 -0000

Presenters,

If you haven't sent your slides for our Thursday morning meeting, please do=
!=20

Al
________________________________________
From: bmwg [bmwg-bounces@ietf.org] on behalf of MORTON, ALFRED C (AL)
Sent: Saturday, March 11, 2017 3:08 PM
To: bmwg@ietf.org
Subject: [bmwg] Draft Agenda for IETF-98

*** Security Advisory: This Message Originated Outside of AT&T ***.
Reference http://cso.att.com/EmailSecurity/IDSP.html for more information.

BMWG,

The draft agenda for our session at IETF-98 in Chicago
is available with the other meeting materials:
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.org=
_meeting_98_agenda_bmwg_&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kT=
IltVyD1oL72cBw&m=3DCmiLBGBsDQpsyFiColhl8DgKOAjw6UDOM7ZvR4uO8f8&s=3DMB6sKDxD=
vSqqR8wmWLxXmIzxdG4bYlULcTSrzDB_Hjk&e=3D

Please let me know of any:

- requests for Remote Presentation (before March 24)
- changes to Presenter Names
- revised drafts (after today)
- additional requests for agenda time
- other Agenda Bashing...

thanks, and see you in Chicago (UTC-5 when we get there).
Al
bmwg co-chair



_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.ietf.org_mailman=
_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL7=
2cBw&m=3DCmiLBGBsDQpsyFiColhl8DgKOAjw6UDOM7ZvR4uO8f8&s=3DcywZqwp8FeifSSIc8W=
opFxpHLSgfm5UOhvMtpqYeU70&e=3D


From nobody Wed Mar 29 01:14:22 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id B790A129677; Wed, 29 Mar 2017 01:14:20 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.48.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149077526071.1166.15432669773600214880@ietfa.amsl.com>
Date: Wed, 29 Mar 2017 01:14:20 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/-ajhfA30pV-K1koPl-F12mvWVQs>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 29 Mar 2017 08:14:21 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition Technologies
        Authors         : Marius Georgescu
                          Liviu Pislaru
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05.txt
	Pages           : 28
	Date            : 2017-03-29

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a tentative metric for benchmarking
   load scalability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

