
From nobody Mon May  1 10:02:32 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 5DC04129C14; Mon,  1 May 2017 10:02:25 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149365814526.9820.5663311508985097873@ietfa.amsl.com>
Date: Mon, 01 May 2017 10:02:25 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/1HJPRbc27PrlqzBWaynvZ0VF3Cg>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-vswitch-opnfv-03.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 May 2017 17:02:25 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Virtual Switches in OPNFV
        Authors         : Maryam Tahhan
                          Billy O'Mahony
                          Al Morton
	Filename        : draft-ietf-bmwg-vswitch-opnfv-03.txt
	Pages           : 22
	Date            : 2017-05-01

Abstract:
   This memo describes the progress of the Open Platform for NFV (OPNFV)
   project on virtual switch performance "VSPERF".  This project intends
   to build on the current and completed work of the Benchmarking
   Methodology Working Group in IETF, by referencing existing
   literature.  The Benchmarking Methodology Working Group has
   traditionally conducted laboratory characterization of dedicated
   physical implementations of internetworking functions.  Therefore,
   this memo begins to describe the additional considerations when
   virtual switches are implemented in general-purpose hardware.  The
   expanded tests and benchmarks are also influenced by the OPNFV
   mission to support virtualization of the "telco" infrastructure.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-vswitch-opnfv-03
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-vswitch-opnfv-03

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-vswitch-opnfv-03


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Mon May  1 10:21:05 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4B1B512EAA9 for <bmwg@ietfa.amsl.com>; Mon,  1 May 2017 10:21:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yaSoT72Vs0k1 for <bmwg@ietfa.amsl.com>; Mon,  1 May 2017 10:21:00 -0700 (PDT)
Received: from mail-qt0-x231.google.com (mail-qt0-x231.google.com [IPv6:2607:f8b0:400d:c0d::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B163512EAAE for <bmwg@ietf.org>; Mon,  1 May 2017 10:17:59 -0700 (PDT)
Received: by mail-qt0-x231.google.com with SMTP id g60so93917533qtd.3 for <bmwg@ietf.org>; Mon, 01 May 2017 10:17:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=AmrWISn9y2tacShYMj1koKbabsr5ORk5/26FeOgA38A=; b=ZkZBncKF3Q6TWgv7+yk0eeEkXhmMi/IKfYjNC+xYkhXP0EsszcHHEMnASOlcMgRAcz UjsWPjYFrYuaTmDqA8XKphJo/uwN6Ros/4S35PlOMJ08e2utcVcTV5aXLAxpH5fgr0fz 6sOnw2ZhLnAaJorSpbcASHol+C6YmA6kOim6QwZdjrD+TJZaiQlW2VaCquT3ca8FFdyI AywlaKiaTtSCb0GIfbQpHCPsvvly6FFqwppnGmT1NqdRfMvqO+0Yo+wE6iKR503hx+pQ u0Ahqi6uNXC0qTaedUBh14gQGIuiW3r6ll4uMJIwnklvZASi+WAmMh89mElM7lox6xII +8tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=AmrWISn9y2tacShYMj1koKbabsr5ORk5/26FeOgA38A=; b=YFIrhPTFAs6zKV+bX8GpEQt0hYsWDFtT+QjnjP6HOIIJP3PkTZLAP5etny8vcWAW/G y1h7sLmrX8L0+4nSsnQXThzXukL+bHCDij5TuqyX8gcjQJExMoKQoxrKmv6VtObgsyTI Ss0Ak4pm5IxOJN3/ERP/CA80kT4zsm9Lsc87uKj8JnD9T0d97WPK4Yki4AoPHPDVMBMl Dg66Nb0Ju0xJeQMOgm+pEbqaYynjgITW9Xhd6mZ0vunD5/tNbHa199S9EN7rF7lbFZDK n8LegUoFA1KE40YKowqeD639oPV1EuANBcAmUS/He/db1q745e+MWIDNBdnutEnFZhww NPKw==
X-Gm-Message-State: AN3rC/592Ur6aaPoo+0Z2ylOVMgpRX4up0sfShCbzgs2jS+gNyrloZEp eetsaR90UTCe3rvIR4JOmq7Q1dIg8RdW
X-Received: by 10.200.38.132 with SMTP id 4mr21510656qto.99.1493659078715; Mon, 01 May 2017 10:17:58 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.12.189.164 with HTTP; Mon, 1 May 2017 10:17:18 -0700 (PDT)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25F9080A@njmtexg4.research.att.com>
References: <CAHw9_iLhzXyV42a9cMVLMCcxNFDgFUODqXLyFnsRrq1XpXiQvA@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF25F9080A@njmtexg4.research.att.com>
From: Warren Kumari <warren@kumari.net>
Date: Mon, 1 May 2017 13:17:18 -0400
Message-ID: <CAHw9_iJYuEkJUPByHBxVt6rH4zLkKcbf4ik1t8UWzZtPLQbrjw@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Cc: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>,  "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: text/plain; charset=UTF-8
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/T9sSdRlifnliGhHghnvl8UelhaI>
Subject: Re: [bmwg] AD Eval of draft-ietf-bmwg-vswitch-opnfv
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 May 2017 17:21:03 -0000

Thank you.

IETL LC started.

W

On Mon, May 1, 2017 at 1:08 PM, MORTON, ALFRED C (AL) <acmorton@att.com> wrote:
> Hi Warren,
>
> The latest version of this draft:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/
>
> addresses your comments plus some other editorial
> clean-up, and includes an abbreviations section
> (which should help with further reviews).
>
> thanks again for your review,
> Al (for the co-authors)
>
>> -----Original Message-----
>> From: MORTON, ALFRED C (AL)
>> Sent: Tuesday, April 25, 2017 7:43 AM
>> To: 'Warren Kumari'; draft-ietf-bmwg-vswitch-opnfv@ietf.org;
>> sbanks@encrypted.net; bmwg@ietf.org
>> Subject: RE: AD Eval of draft-ietf-bmwg-vswitch-opnfv
>>
>> Thanks for your review & comments, Warren; the edits will
>> take a few days to reach the top of the ToDo list.
>> Al
>>
>> > -----Original Message-----
>> > From: Warren Kumari [mailto:warren@kumari.net]
>> > Sent: Sunday, April 23, 2017 10:40 AM
>> > To: draft-ietf-bmwg-vswitch-opnfv@ietf.org; sbanks@encrypted.net;
>> > bmwg@ietf.org
>> > Subject: AD Eval of draft-ietf-bmwg-vswitch-opnfv
>> >
>> > Hi there,
>> >
>> > I've just done the AD eval of draft-ietf-bmwg-vswitch-opnfv - thank
>> > you for writing this, I think that it is a useful document.
>> >
>> > I had a few questions and comments which I think  it would be useful
>> > to address before starting IETF LC. I'm not a benchmarking (or NFV /
>> > vswitch person), so some of these questions / comments may be very
>> > basic.
>> >
>> > I've noted my questions with [WK: ...]. My questions / comments look
>> > long, because I kept much of the text to make readability easier.
>> >
>> > ---------
>> >
>> >
>> > 2.  Scope
>> >
>> >    The primary purpose and scope of the memo is to inform the industry
>> >    of work-in-progress that builds on the body of extensive BMWG
>> >    literature and experience, and describe the extensions needed for
>> >    benchmarking virtual switches.  Inital feedback indicates that many
>> >
>> > [WK: s/Inital/Initial/]
>> >
>> >    of these extensions may be applicable beyond the current scope (to
>> >    hardware switches in the NFV Infrastructure and to virtual routers,
>> >    for example).  Additionally, this memo serves as a vehicle to
>> include
>> >    more detail and commentary from BMWG and other Open Source
>> >    communities, under BMWG's chartered work to characterize the NFV
>> >
>> >    Infrastructure (a virtual switch is an important aspect of that
>> >    infrastructure).
>> >
>> >    The benchmarking covered in this memo should be applicable to many
>> >    types of vswitches, and remain vswitch-agnostic to great degree.
>> >
>> > [WK: Please expand / define vswitch - perhaps in the parenthesis
>> above.]
>> >
>> >    There has been no attempt to track and test all features of any
>> >    specific vswitch implementation.
>> >
>> > ...
>> >
>> > 3.1.  Comparison with Physical Network Functions
>> >
>> >    To compare the performance of virtual designs and implementations
>> >    with their physical counterparts, identical benchmarks are needed.
>> >    BMWG has developed specifications for many network functions this
>> >    memo re-uses existing benchmarks through references, and expands
>> them
>> >    during development of new methods.
>> >
>> > [ WK: This sentence makes no sense / is missing some words / should be
>> > 2 sentences / something....]
>> >
>> >    A key configuration aspect is the
>> >    number of parallel cores required to achieve comparable performance
>> >    with a given physical device, or whether some limit of scale was
>> >    reached before the cores could achieve the comparable level.
>> >
>> >    It's unlikely that the virtual switch will be the only application
>> >    running on the SUT, so CPU utilization, Cache utilization, and
>> Memory
>> >    footprint should also be recorded for the virtual implementations
>> of
>> >    internetworking functions.
>> >
>> > ...
>> >
>> > 3.3.  New Configuration Parameters
>> >
>> >    A key consideration when conducting any sort of benchmark is trying
>> >    to ensure the consistency and repeatability of test results.  When
>> >    benchmarking the performance of a vSwitch
>> >
>> > [ WK: Please choose a single term (vswitch / vSwitch) ]
>> >
>> >    there are many factors that
>> >    can affect the consistency of results, one key factor is matching
>> the
>> >    various hardware and software details of the SUT.  This section
>> lists
>> >    some of the many new parameters which this project believes are
>> >    critical to report in order to achieve repeatability.
>> >
>> >    Hardware details including:
>> >
>> >    o  Platform details
>> >
>> >    o  Processor details
>> >
>> >    o  Memory information (type and size)
>> >
>> > [ WK: I would also think that the memory layout is important - if not,
>> > ignore. Oh, it is mentioned below ("Memory DIMM configurations"). This
>> > would be easier to read if you grouped memory together, processor /
>> > cores together, etc.]
>> >
>> >
>> >    o  Number of enabled cores
>> >
>> >    o  Number of cores used for the test
>> >
>> >    o  Number of physical NICs, as well as their details (manufacturer,
>> >       versions, type and the PCI slot they are plugged into)
>> >
>> >    o  NIC interrupt configuration
>> >
>> > [WK: What about other NIC features (DDP, checksum magic,
>> > scatter-gather, etc)? Or are these not applicable? (if not, ignore) ]
>> >
>> >    o  BIOS version, release date and any configurations that were
>> >       modified
>> >
>> >    o  CPU microcode level
>> >
>> >    o  Memory DIMM configurations (quad rank performance may not be the
>> >       same as dual rank) in size, freq and slot locations
>> >
>> >    o  PCI configuration parameters (payload size, early ack option...)
>> >
>> >    o  Power management at all levels (ACPI sleep states, processor
>> >       package, OS...)
>> >
>> >    Software details including:
>> >
>> >    o  OS parameters and behavior (text vs graphical no one typing at
>> the
>> >       console on one system)
>> >
>> >    o  OS version (for host and VNF)
>> >
>> >    o  Kernel version (for host and VNF)
>> >
>> >    o  GRUB boot parameters (for host and VNF)
>> >
>> >    o  Hypervisor details (Type and version)
>> >
>> >    o  Selected vSwitch, version number or commit id used
>> >
>> >    o  vSwitch launch command line if it has been parameterised
>> >
>> >    o  Memory allocation to the vSwitch
>> >
>> >    o  which NUMA node it is using, and how many memory channels
>> >
>> >    o  DPDK or any other SW dependency version number or commit id used
>> >
>> >    o  Memory allocation to a VM - if it's from Hugpages/elsewhere
>> >
>> > [WK: Hugpages? It this the emo version of hugepages? :-) ]
>> >
>> >
>> >    o  VM storage type: snapshot/independent persistent/independent
>> non-
>> >       persistent
>> >
>> >    o  Number of VMs
>> >
>> >    o  Number of Virtual NICs (vNICs), versions, type and driver
>> >
>> >
>> > [ WK: What about IO virtualization stuff like SR-IOV and similar. Or
>> > not applicable?]
>> >
>> >    o  Number of virtual CPUs and their core affinity on the host
>> >
>> >    o  Number vNIC interrupt configuration
>> >
>> >    o  Thread affinitization for the applications (including the
>> vSwitch
>> >       itself) on the host
>> >
>> >    o  Details of Resource isolation, such as CPUs designated for Host/
>> >       Kernel (isolcpu) and CPUs designated for specific processes
>> >       (taskset). - Test duration. - Number of flows.
>> >
>> >    Test Traffic Information:
>> >
>> >    o  Traffic type - UDP, TCP, IMIX / Other
>> >
>> >
>> > [ WK: I don't think that IMIX is a well known acronym. I think either
>> > define it (or ref: RFC6985?), or drop it (the section is "some of the
>> > many new parameters"...)]
>> >
>> >    o  Packet Sizes
>> >
>> >    o  Deployment Scenario
>> >
>> > 3.4.  Flow classification
>> >
>> >    Virtual switches group packets into flows by processing and
>> matching
>> >    particular packet or frame header information, or by matching
>> packets
>> >    based on the input ports.  Thus a flow can be thought of a sequence
>> >    of packets that have the same set of header field values (5-tuple)
>> or
>> >    have arrived on the same port.  Performance results can vary based
>> on
>> >    the parameters the vSwitch uses to match for a flow.  The
>> recommended
>> >    flow classification parameters for any vSwitch performance tests
>> are:
>> >    the input port, the source IP address, the destination IP address
>> and
>> >    the Ethernet protocol type field.
>> >
>> > [ WK: Is it useful to say "5-tuple" above? To me 5-tuple is src ip,
>> > src port, dst ip, dst port, ip protocol. This says for vswitch it is
>> > input port, src ip, dst ip and eth protocol. The two close together
>> > may confuse readers - I think just drop "(5-tuple)". Also, this says
>> > "input port" - is that actually input interface / should it be called
>> > "physical port"? (when hearing IP and then port, people think L3
>> > port)]
>> >
>> >
>> >    o  Scalability Tests to understand how the virtual switch performs
>> as
>> >       the number of flows, active ports, complexity of the forwarding
>> >       logic's configuration... it has to deal with increases.
>> >
>> > [ WK: Looks like got sidetracked in this sentence. ]
>> >
>> >
>> > ---------
>> >
>> > Thanks again,
>> > W
>> >
>> > --
>> > I don't think the execution is relevant when it was obviously a bad
>> > idea in the first place.
>> > This is like putting rabid weasels in your pants, and later expressing
>> > regret at having chosen those particular rabid weasels and that pair
>> > of pants.
>> >    ---maf



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Mon May  1 10:45:11 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 54248129C14; Mon,  1 May 2017 10:45:09 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
CC: Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, draft-ietf-bmwg-vswitch-opnfv@ietf.org, bmwg@ietf.org, sbanks@encrypted.net, warren@kumari.net
Reply-To: ietf@ietf.org
Sender: <iesg-secretary@ietf.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149366070925.9919.14066094110781904701.idtracker@ietfa.amsl.com>
Date: Mon, 01 May 2017 10:45:09 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/XH99KTROBNBawigOK6LfIBB9MW0>
Subject: [bmwg] Last Call: <draft-ietf-bmwg-vswitch-opnfv-03.txt> (Benchmarking Virtual Switches in OPNFV) to Informational RFC
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 May 2017 17:45:09 -0000

The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Benchmarking Virtual Switches in OPNFV'
  <draft-ietf-bmwg-vswitch-opnfv-03.txt> as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-05-15. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


   This memo describes the progress of the Open Platform for NFV (OPNFV)
   project on virtual switch performance "VSPERF".  This project intends
   to build on the current and completed work of the Benchmarking
   Methodology Working Group in IETF, by referencing existing
   literature.  The Benchmarking Methodology Working Group has
   traditionally conducted laboratory characterization of dedicated
   physical implementations of internetworking functions.  Therefore,
   this memo begins to describe the additional considerations when
   virtual switches are implemented in general-purpose hardware.  The
   expanded tests and benchmarks are also influenced by the OPNFV
   mission to support virtualization of the "telco" infrastructure.





The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/ballot/


No IPR declarations have been submitted directly on this I-D.





From nobody Mon May  1 12:00:34 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 271CE12EB27; Mon,  1 May 2017 12:00:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tTIYVlo8LrMx; Mon,  1 May 2017 12:00:30 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1611F129C60; Mon,  1 May 2017 11:57:26 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net [127.0.0.1]) by m0048589.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v41GjdOG020929; Mon, 1 May 2017 13:09:06 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0048589.ppops.net-00191d01. with ESMTP id 2a64yuy569-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 May 2017 13:09:06 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v41H94Rf024179; Mon, 1 May 2017 13:09:05 -0400
Received: from mlpi407.sfdc.sbc.com (mlpi407.sfdc.sbc.com [130.9.128.239]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v41H8w4I024007 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 May 2017 13:09:00 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi407.sfdc.sbc.com (RSA Interceptor); Mon, 1 May 2017 17:08:33 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v41H8XLO020240; Mon, 1 May 2017 12:08:33 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v41H8R9W019462; Mon, 1 May 2017 12:08:28 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 133DEE03D5; Mon,  1 May 2017 13:08:27 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Mon, 1 May 2017 13:08:26 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Warren Kumari <warren@kumari.net>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: AD Eval of draft-ietf-bmwg-vswitch-opnfv
Thread-Index: AQHSvD+m1+lMKFsOy0SwDArzS5XAjKHV+UWwgAnHf1A=
Date: Mon, 1 May 2017 17:08:25 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F9080A@njmtexg4.research.att.com>
References: <CAHw9_iLhzXyV42a9cMVLMCcxNFDgFUODqXLyFnsRrq1XpXiQvA@mail.gmail.com> 
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.70.214.123]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-01_12:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705010109
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/V0t7diyXa5HrD48KlwSHrRdR_sA>
Subject: Re: [bmwg] AD Eval of draft-ietf-bmwg-vswitch-opnfv
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 May 2017 19:00:33 -0000

SGkgV2FycmVuLA0KDQpUaGUgbGF0ZXN0IHZlcnNpb24gb2YgdGhpcyBkcmFmdDoNCmh0dHBzOi8v
ZGF0YXRyYWNrZXIuaWV0Zi5vcmcvZG9jL2RyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2Lw0K
DQphZGRyZXNzZXMgeW91ciBjb21tZW50cyBwbHVzIHNvbWUgb3RoZXIgZWRpdG9yaWFsDQpjbGVh
bi11cCwgYW5kIGluY2x1ZGVzIGFuIGFiYnJldmlhdGlvbnMgc2VjdGlvbiANCih3aGljaCBzaG91
bGQgaGVscCB3aXRoIGZ1cnRoZXIgcmV2aWV3cykuDQoNCnRoYW5rcyBhZ2FpbiBmb3IgeW91ciBy
ZXZpZXcsDQpBbCAoZm9yIHRoZSBjby1hdXRob3JzKQ0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn
ZS0tLS0tDQo+IEZyb206IE1PUlRPTiwgQUxGUkVEIEMgKEFMKQ0KPiBTZW50OiBUdWVzZGF5LCBB
cHJpbCAyNSwgMjAxNyA3OjQzIEFNDQo+IFRvOiAnV2FycmVuIEt1bWFyaSc7IGRyYWZ0LWlldGYt
Ym13Zy12c3dpdGNoLW9wbmZ2QGlldGYub3JnOw0KPiBzYmFua3NAZW5jcnlwdGVkLm5ldDsgYm13
Z0BpZXRmLm9yZw0KPiBTdWJqZWN0OiBSRTogQUQgRXZhbCBvZiBkcmFmdC1pZXRmLWJtd2ctdnN3
aXRjaC1vcG5mdg0KPiANCj4gVGhhbmtzIGZvciB5b3VyIHJldmlldyAmIGNvbW1lbnRzLCBXYXJy
ZW47IHRoZSBlZGl0cyB3aWxsDQo+IHRha2UgYSBmZXcgZGF5cyB0byByZWFjaCB0aGUgdG9wIG9m
IHRoZSBUb0RvIGxpc3QuDQo+IEFsDQo+IA0KPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0t
DQo+ID4gRnJvbTogV2FycmVuIEt1bWFyaSBbbWFpbHRvOndhcnJlbkBrdW1hcmkubmV0XQ0KPiA+
IFNlbnQ6IFN1bmRheSwgQXByaWwgMjMsIDIwMTcgMTA6NDAgQU0NCj4gPiBUbzogZHJhZnQtaWV0
Zi1ibXdnLXZzd2l0Y2gtb3BuZnZAaWV0Zi5vcmc7IHNiYW5rc0BlbmNyeXB0ZWQubmV0Ow0KPiA+
IGJtd2dAaWV0Zi5vcmcNCj4gPiBTdWJqZWN0OiBBRCBFdmFsIG9mIGRyYWZ0LWlldGYtYm13Zy12
c3dpdGNoLW9wbmZ2DQo+ID4NCj4gPiBIaSB0aGVyZSwNCj4gPg0KPiA+IEkndmUganVzdCBkb25l
IHRoZSBBRCBldmFsIG9mIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2IC0gdGhhbmsNCj4g
PiB5b3UgZm9yIHdyaXRpbmcgdGhpcywgSSB0aGluayB0aGF0IGl0IGlzIGEgdXNlZnVsIGRvY3Vt
ZW50Lg0KPiA+DQo+ID4gSSBoYWQgYSBmZXcgcXVlc3Rpb25zIGFuZCBjb21tZW50cyB3aGljaCBJ
IHRoaW5rICBpdCB3b3VsZCBiZSB1c2VmdWwNCj4gPiB0byBhZGRyZXNzIGJlZm9yZSBzdGFydGlu
ZyBJRVRGIExDLiBJJ20gbm90IGEgYmVuY2htYXJraW5nIChvciBORlYgLw0KPiA+IHZzd2l0Y2gg
cGVyc29uKSwgc28gc29tZSBvZiB0aGVzZSBxdWVzdGlvbnMgLyBjb21tZW50cyBtYXkgYmUgdmVy
eQ0KPiA+IGJhc2ljLg0KPiA+DQo+ID4gSSd2ZSBub3RlZCBteSBxdWVzdGlvbnMgd2l0aCBbV0s6
IC4uLl0uIE15IHF1ZXN0aW9ucyAvIGNvbW1lbnRzIGxvb2sNCj4gPiBsb25nLCBiZWNhdXNlIEkg
a2VwdCBtdWNoIG9mIHRoZSB0ZXh0IHRvIG1ha2UgcmVhZGFiaWxpdHkgZWFzaWVyLg0KPiA+DQo+
ID4gLS0tLS0tLS0tDQo+ID4NCj4gPg0KPiA+IDIuICBTY29wZQ0KPiA+DQo+ID4gICAgVGhlIHBy
aW1hcnkgcHVycG9zZSBhbmQgc2NvcGUgb2YgdGhlIG1lbW8gaXMgdG8gaW5mb3JtIHRoZSBpbmR1
c3RyeQ0KPiA+ICAgIG9mIHdvcmstaW4tcHJvZ3Jlc3MgdGhhdCBidWlsZHMgb24gdGhlIGJvZHkg
b2YgZXh0ZW5zaXZlIEJNV0cNCj4gPiAgICBsaXRlcmF0dXJlIGFuZCBleHBlcmllbmNlLCBhbmQg
ZGVzY3JpYmUgdGhlIGV4dGVuc2lvbnMgbmVlZGVkIGZvcg0KPiA+ICAgIGJlbmNobWFya2luZyB2
aXJ0dWFsIHN3aXRjaGVzLiAgSW5pdGFsIGZlZWRiYWNrIGluZGljYXRlcyB0aGF0IG1hbnkNCj4g
Pg0KPiA+IFtXSzogcy9Jbml0YWwvSW5pdGlhbC9dDQo+ID4NCj4gPiAgICBvZiB0aGVzZSBleHRl
bnNpb25zIG1heSBiZSBhcHBsaWNhYmxlIGJleW9uZCB0aGUgY3VycmVudCBzY29wZSAodG8NCj4g
PiAgICBoYXJkd2FyZSBzd2l0Y2hlcyBpbiB0aGUgTkZWIEluZnJhc3RydWN0dXJlIGFuZCB0byB2
aXJ0dWFsIHJvdXRlcnMsDQo+ID4gICAgZm9yIGV4YW1wbGUpLiAgQWRkaXRpb25hbGx5LCB0aGlz
IG1lbW8gc2VydmVzIGFzIGEgdmVoaWNsZSB0bw0KPiBpbmNsdWRlDQo+ID4gICAgbW9yZSBkZXRh
aWwgYW5kIGNvbW1lbnRhcnkgZnJvbSBCTVdHIGFuZCBvdGhlciBPcGVuIFNvdXJjZQ0KPiA+ICAg
IGNvbW11bml0aWVzLCB1bmRlciBCTVdHJ3MgY2hhcnRlcmVkIHdvcmsgdG8gY2hhcmFjdGVyaXpl
IHRoZSBORlYNCj4gPg0KPiA+ICAgIEluZnJhc3RydWN0dXJlIChhIHZpcnR1YWwgc3dpdGNoIGlz
IGFuIGltcG9ydGFudCBhc3BlY3Qgb2YgdGhhdA0KPiA+ICAgIGluZnJhc3RydWN0dXJlKS4NCj4g
Pg0KPiA+ICAgIFRoZSBiZW5jaG1hcmtpbmcgY292ZXJlZCBpbiB0aGlzIG1lbW8gc2hvdWxkIGJl
IGFwcGxpY2FibGUgdG8gbWFueQ0KPiA+ICAgIHR5cGVzIG9mIHZzd2l0Y2hlcywgYW5kIHJlbWFp
biB2c3dpdGNoLWFnbm9zdGljIHRvIGdyZWF0IGRlZ3JlZS4NCj4gPg0KPiA+IFtXSzogUGxlYXNl
IGV4cGFuZCAvIGRlZmluZSB2c3dpdGNoIC0gcGVyaGFwcyBpbiB0aGUgcGFyZW50aGVzaXMNCj4g
YWJvdmUuXQ0KPiA+DQo+ID4gICAgVGhlcmUgaGFzIGJlZW4gbm8gYXR0ZW1wdCB0byB0cmFjayBh
bmQgdGVzdCBhbGwgZmVhdHVyZXMgb2YgYW55DQo+ID4gICAgc3BlY2lmaWMgdnN3aXRjaCBpbXBs
ZW1lbnRhdGlvbi4NCj4gPg0KPiA+IC4uLg0KPiA+DQo+ID4gMy4xLiAgQ29tcGFyaXNvbiB3aXRo
IFBoeXNpY2FsIE5ldHdvcmsgRnVuY3Rpb25zDQo+ID4NCj4gPiAgICBUbyBjb21wYXJlIHRoZSBw
ZXJmb3JtYW5jZSBvZiB2aXJ0dWFsIGRlc2lnbnMgYW5kIGltcGxlbWVudGF0aW9ucw0KPiA+ICAg
IHdpdGggdGhlaXIgcGh5c2ljYWwgY291bnRlcnBhcnRzLCBpZGVudGljYWwgYmVuY2htYXJrcyBh
cmUgbmVlZGVkLg0KPiA+ICAgIEJNV0cgaGFzIGRldmVsb3BlZCBzcGVjaWZpY2F0aW9ucyBmb3Ig
bWFueSBuZXR3b3JrIGZ1bmN0aW9ucyB0aGlzDQo+ID4gICAgbWVtbyByZS11c2VzIGV4aXN0aW5n
IGJlbmNobWFya3MgdGhyb3VnaCByZWZlcmVuY2VzLCBhbmQgZXhwYW5kcw0KPiB0aGVtDQo+ID4g
ICAgZHVyaW5nIGRldmVsb3BtZW50IG9mIG5ldyBtZXRob2RzLg0KPiA+DQo+ID4gWyBXSzogVGhp
cyBzZW50ZW5jZSBtYWtlcyBubyBzZW5zZSAvIGlzIG1pc3Npbmcgc29tZSB3b3JkcyAvIHNob3Vs
ZCBiZQ0KPiA+IDIgc2VudGVuY2VzIC8gc29tZXRoaW5nLi4uLl0NCj4gPg0KPiA+ICAgIEEga2V5
IGNvbmZpZ3VyYXRpb24gYXNwZWN0IGlzIHRoZQ0KPiA+ICAgIG51bWJlciBvZiBwYXJhbGxlbCBj
b3JlcyByZXF1aXJlZCB0byBhY2hpZXZlIGNvbXBhcmFibGUgcGVyZm9ybWFuY2UNCj4gPiAgICB3
aXRoIGEgZ2l2ZW4gcGh5c2ljYWwgZGV2aWNlLCBvciB3aGV0aGVyIHNvbWUgbGltaXQgb2Ygc2Nh
bGUgd2FzDQo+ID4gICAgcmVhY2hlZCBiZWZvcmUgdGhlIGNvcmVzIGNvdWxkIGFjaGlldmUgdGhl
IGNvbXBhcmFibGUgbGV2ZWwuDQo+ID4NCj4gPiAgICBJdCdzIHVubGlrZWx5IHRoYXQgdGhlIHZp
cnR1YWwgc3dpdGNoIHdpbGwgYmUgdGhlIG9ubHkgYXBwbGljYXRpb24NCj4gPiAgICBydW5uaW5n
IG9uIHRoZSBTVVQsIHNvIENQVSB1dGlsaXphdGlvbiwgQ2FjaGUgdXRpbGl6YXRpb24sIGFuZA0K
PiBNZW1vcnkNCj4gPiAgICBmb290cHJpbnQgc2hvdWxkIGFsc28gYmUgcmVjb3JkZWQgZm9yIHRo
ZSB2aXJ0dWFsIGltcGxlbWVudGF0aW9ucw0KPiBvZg0KPiA+ICAgIGludGVybmV0d29ya2luZyBm
dW5jdGlvbnMuDQo+ID4NCj4gPiAuLi4NCj4gPg0KPiA+IDMuMy4gIE5ldyBDb25maWd1cmF0aW9u
IFBhcmFtZXRlcnMNCj4gPg0KPiA+ICAgIEEga2V5IGNvbnNpZGVyYXRpb24gd2hlbiBjb25kdWN0
aW5nIGFueSBzb3J0IG9mIGJlbmNobWFyayBpcyB0cnlpbmcNCj4gPiAgICB0byBlbnN1cmUgdGhl
IGNvbnNpc3RlbmN5IGFuZCByZXBlYXRhYmlsaXR5IG9mIHRlc3QgcmVzdWx0cy4gIFdoZW4NCj4g
PiAgICBiZW5jaG1hcmtpbmcgdGhlIHBlcmZvcm1hbmNlIG9mIGEgdlN3aXRjaA0KPiA+DQo+ID4g
WyBXSzogUGxlYXNlIGNob29zZSBhIHNpbmdsZSB0ZXJtICh2c3dpdGNoIC8gdlN3aXRjaCkgXQ0K
PiA+DQo+ID4gICAgdGhlcmUgYXJlIG1hbnkgZmFjdG9ycyB0aGF0DQo+ID4gICAgY2FuIGFmZmVj
dCB0aGUgY29uc2lzdGVuY3kgb2YgcmVzdWx0cywgb25lIGtleSBmYWN0b3IgaXMgbWF0Y2hpbmcN
Cj4gdGhlDQo+ID4gICAgdmFyaW91cyBoYXJkd2FyZSBhbmQgc29mdHdhcmUgZGV0YWlscyBvZiB0
aGUgU1VULiAgVGhpcyBzZWN0aW9uDQo+IGxpc3RzDQo+ID4gICAgc29tZSBvZiB0aGUgbWFueSBu
ZXcgcGFyYW1ldGVycyB3aGljaCB0aGlzIHByb2plY3QgYmVsaWV2ZXMgYXJlDQo+ID4gICAgY3Jp
dGljYWwgdG8gcmVwb3J0IGluIG9yZGVyIHRvIGFjaGlldmUgcmVwZWF0YWJpbGl0eS4NCj4gPg0K
PiA+ICAgIEhhcmR3YXJlIGRldGFpbHMgaW5jbHVkaW5nOg0KPiA+DQo+ID4gICAgbyAgUGxhdGZv
cm0gZGV0YWlscw0KPiA+DQo+ID4gICAgbyAgUHJvY2Vzc29yIGRldGFpbHMNCj4gPg0KPiA+ICAg
IG8gIE1lbW9yeSBpbmZvcm1hdGlvbiAodHlwZSBhbmQgc2l6ZSkNCj4gPg0KPiA+IFsgV0s6IEkg
d291bGQgYWxzbyB0aGluayB0aGF0IHRoZSBtZW1vcnkgbGF5b3V0IGlzIGltcG9ydGFudCAtIGlm
IG5vdCwNCj4gPiBpZ25vcmUuIE9oLCBpdCBpcyBtZW50aW9uZWQgYmVsb3cgKCJNZW1vcnkgRElN
TSBjb25maWd1cmF0aW9ucyIpLiBUaGlzDQo+ID4gd291bGQgYmUgZWFzaWVyIHRvIHJlYWQgaWYg
eW91IGdyb3VwZWQgbWVtb3J5IHRvZ2V0aGVyLCBwcm9jZXNzb3IgLw0KPiA+IGNvcmVzIHRvZ2V0
aGVyLCBldGMuXQ0KPiA+DQo+ID4NCj4gPiAgICBvICBOdW1iZXIgb2YgZW5hYmxlZCBjb3Jlcw0K
PiA+DQo+ID4gICAgbyAgTnVtYmVyIG9mIGNvcmVzIHVzZWQgZm9yIHRoZSB0ZXN0DQo+ID4NCj4g
PiAgICBvICBOdW1iZXIgb2YgcGh5c2ljYWwgTklDcywgYXMgd2VsbCBhcyB0aGVpciBkZXRhaWxz
IChtYW51ZmFjdHVyZXIsDQo+ID4gICAgICAgdmVyc2lvbnMsIHR5cGUgYW5kIHRoZSBQQ0kgc2xv
dCB0aGV5IGFyZSBwbHVnZ2VkIGludG8pDQo+ID4NCj4gPiAgICBvICBOSUMgaW50ZXJydXB0IGNv
bmZpZ3VyYXRpb24NCj4gPg0KPiA+IFtXSzogV2hhdCBhYm91dCBvdGhlciBOSUMgZmVhdHVyZXMg
KEREUCwgY2hlY2tzdW0gbWFnaWMsDQo+ID4gc2NhdHRlci1nYXRoZXIsIGV0Yyk/IE9yIGFyZSB0
aGVzZSBub3QgYXBwbGljYWJsZT8gKGlmIG5vdCwgaWdub3JlKSBdDQo+ID4NCj4gPiAgICBvICBC
SU9TIHZlcnNpb24sIHJlbGVhc2UgZGF0ZSBhbmQgYW55IGNvbmZpZ3VyYXRpb25zIHRoYXQgd2Vy
ZQ0KPiA+ICAgICAgIG1vZGlmaWVkDQo+ID4NCj4gPiAgICBvICBDUFUgbWljcm9jb2RlIGxldmVs
DQo+ID4NCj4gPiAgICBvICBNZW1vcnkgRElNTSBjb25maWd1cmF0aW9ucyAocXVhZCByYW5rIHBl
cmZvcm1hbmNlIG1heSBub3QgYmUgdGhlDQo+ID4gICAgICAgc2FtZSBhcyBkdWFsIHJhbmspIGlu
IHNpemUsIGZyZXEgYW5kIHNsb3QgbG9jYXRpb25zDQo+ID4NCj4gPiAgICBvICBQQ0kgY29uZmln
dXJhdGlvbiBwYXJhbWV0ZXJzIChwYXlsb2FkIHNpemUsIGVhcmx5IGFjayBvcHRpb24uLi4pDQo+
ID4NCj4gPiAgICBvICBQb3dlciBtYW5hZ2VtZW50IGF0IGFsbCBsZXZlbHMgKEFDUEkgc2xlZXAg
c3RhdGVzLCBwcm9jZXNzb3INCj4gPiAgICAgICBwYWNrYWdlLCBPUy4uLikNCj4gPg0KPiA+ICAg
IFNvZnR3YXJlIGRldGFpbHMgaW5jbHVkaW5nOg0KPiA+DQo+ID4gICAgbyAgT1MgcGFyYW1ldGVy
cyBhbmQgYmVoYXZpb3IgKHRleHQgdnMgZ3JhcGhpY2FsIG5vIG9uZSB0eXBpbmcgYXQNCj4gdGhl
DQo+ID4gICAgICAgY29uc29sZSBvbiBvbmUgc3lzdGVtKQ0KPiA+DQo+ID4gICAgbyAgT1MgdmVy
c2lvbiAoZm9yIGhvc3QgYW5kIFZORikNCj4gPg0KPiA+ICAgIG8gIEtlcm5lbCB2ZXJzaW9uIChm
b3IgaG9zdCBhbmQgVk5GKQ0KPiA+DQo+ID4gICAgbyAgR1JVQiBib290IHBhcmFtZXRlcnMgKGZv
ciBob3N0IGFuZCBWTkYpDQo+ID4NCj4gPiAgICBvICBIeXBlcnZpc29yIGRldGFpbHMgKFR5cGUg
YW5kIHZlcnNpb24pDQo+ID4NCj4gPiAgICBvICBTZWxlY3RlZCB2U3dpdGNoLCB2ZXJzaW9uIG51
bWJlciBvciBjb21taXQgaWQgdXNlZA0KPiA+DQo+ID4gICAgbyAgdlN3aXRjaCBsYXVuY2ggY29t
bWFuZCBsaW5lIGlmIGl0IGhhcyBiZWVuIHBhcmFtZXRlcmlzZWQNCj4gPg0KPiA+ICAgIG8gIE1l
bW9yeSBhbGxvY2F0aW9uIHRvIHRoZSB2U3dpdGNoDQo+ID4NCj4gPiAgICBvICB3aGljaCBOVU1B
IG5vZGUgaXQgaXMgdXNpbmcsIGFuZCBob3cgbWFueSBtZW1vcnkgY2hhbm5lbHMNCj4gPg0KPiA+
ICAgIG8gIERQREsgb3IgYW55IG90aGVyIFNXIGRlcGVuZGVuY3kgdmVyc2lvbiBudW1iZXIgb3Ig
Y29tbWl0IGlkIHVzZWQNCj4gPg0KPiA+ICAgIG8gIE1lbW9yeSBhbGxvY2F0aW9uIHRvIGEgVk0g
LSBpZiBpdCdzIGZyb20gSHVncGFnZXMvZWxzZXdoZXJlDQo+ID4NCj4gPiBbV0s6IEh1Z3BhZ2Vz
PyBJdCB0aGlzIHRoZSBlbW8gdmVyc2lvbiBvZiBodWdlcGFnZXM/IDotKSBdDQo+ID4NCj4gPg0K
PiA+ICAgIG8gIFZNIHN0b3JhZ2UgdHlwZTogc25hcHNob3QvaW5kZXBlbmRlbnQgcGVyc2lzdGVu
dC9pbmRlcGVuZGVudA0KPiBub24tDQo+ID4gICAgICAgcGVyc2lzdGVudA0KPiA+DQo+ID4gICAg
byAgTnVtYmVyIG9mIFZNcw0KPiA+DQo+ID4gICAgbyAgTnVtYmVyIG9mIFZpcnR1YWwgTklDcyAo
dk5JQ3MpLCB2ZXJzaW9ucywgdHlwZSBhbmQgZHJpdmVyDQo+ID4NCj4gPg0KPiA+IFsgV0s6IFdo
YXQgYWJvdXQgSU8gdmlydHVhbGl6YXRpb24gc3R1ZmYgbGlrZSBTUi1JT1YgYW5kIHNpbWlsYXIu
IE9yDQo+ID4gbm90IGFwcGxpY2FibGU/XQ0KPiA+DQo+ID4gICAgbyAgTnVtYmVyIG9mIHZpcnR1
YWwgQ1BVcyBhbmQgdGhlaXIgY29yZSBhZmZpbml0eSBvbiB0aGUgaG9zdA0KPiA+DQo+ID4gICAg
byAgTnVtYmVyIHZOSUMgaW50ZXJydXB0IGNvbmZpZ3VyYXRpb24NCj4gPg0KPiA+ICAgIG8gIFRo
cmVhZCBhZmZpbml0aXphdGlvbiBmb3IgdGhlIGFwcGxpY2F0aW9ucyAoaW5jbHVkaW5nIHRoZQ0K
PiB2U3dpdGNoDQo+ID4gICAgICAgaXRzZWxmKSBvbiB0aGUgaG9zdA0KPiA+DQo+ID4gICAgbyAg
RGV0YWlscyBvZiBSZXNvdXJjZSBpc29sYXRpb24sIHN1Y2ggYXMgQ1BVcyBkZXNpZ25hdGVkIGZv
ciBIb3N0Lw0KPiA+ICAgICAgIEtlcm5lbCAoaXNvbGNwdSkgYW5kIENQVXMgZGVzaWduYXRlZCBm
b3Igc3BlY2lmaWMgcHJvY2Vzc2VzDQo+ID4gICAgICAgKHRhc2tzZXQpLiAtIFRlc3QgZHVyYXRp
b24uIC0gTnVtYmVyIG9mIGZsb3dzLg0KPiA+DQo+ID4gICAgVGVzdCBUcmFmZmljIEluZm9ybWF0
aW9uOg0KPiA+DQo+ID4gICAgbyAgVHJhZmZpYyB0eXBlIC0gVURQLCBUQ1AsIElNSVggLyBPdGhl
cg0KPiA+DQo+ID4NCj4gPiBbIFdLOiBJIGRvbid0IHRoaW5rIHRoYXQgSU1JWCBpcyBhIHdlbGwg
a25vd24gYWNyb255bS4gSSB0aGluayBlaXRoZXINCj4gPiBkZWZpbmUgaXQgKG9yIHJlZjogUkZD
Njk4NT8pLCBvciBkcm9wIGl0ICh0aGUgc2VjdGlvbiBpcyAic29tZSBvZiB0aGUNCj4gPiBtYW55
IG5ldyBwYXJhbWV0ZXJzIi4uLildDQo+ID4NCj4gPiAgICBvICBQYWNrZXQgU2l6ZXMNCj4gPg0K
PiA+ICAgIG8gIERlcGxveW1lbnQgU2NlbmFyaW8NCj4gPg0KPiA+IDMuNC4gIEZsb3cgY2xhc3Np
ZmljYXRpb24NCj4gPg0KPiA+ICAgIFZpcnR1YWwgc3dpdGNoZXMgZ3JvdXAgcGFja2V0cyBpbnRv
IGZsb3dzIGJ5IHByb2Nlc3NpbmcgYW5kDQo+IG1hdGNoaW5nDQo+ID4gICAgcGFydGljdWxhciBw
YWNrZXQgb3IgZnJhbWUgaGVhZGVyIGluZm9ybWF0aW9uLCBvciBieSBtYXRjaGluZw0KPiBwYWNr
ZXRzDQo+ID4gICAgYmFzZWQgb24gdGhlIGlucHV0IHBvcnRzLiAgVGh1cyBhIGZsb3cgY2FuIGJl
IHRob3VnaHQgb2YgYSBzZXF1ZW5jZQ0KPiA+ICAgIG9mIHBhY2tldHMgdGhhdCBoYXZlIHRoZSBz
YW1lIHNldCBvZiBoZWFkZXIgZmllbGQgdmFsdWVzICg1LXR1cGxlKQ0KPiBvcg0KPiA+ICAgIGhh
dmUgYXJyaXZlZCBvbiB0aGUgc2FtZSBwb3J0LiAgUGVyZm9ybWFuY2UgcmVzdWx0cyBjYW4gdmFy
eSBiYXNlZA0KPiBvbg0KPiA+ICAgIHRoZSBwYXJhbWV0ZXJzIHRoZSB2U3dpdGNoIHVzZXMgdG8g
bWF0Y2ggZm9yIGEgZmxvdy4gIFRoZQ0KPiByZWNvbW1lbmRlZA0KPiA+ICAgIGZsb3cgY2xhc3Np
ZmljYXRpb24gcGFyYW1ldGVycyBmb3IgYW55IHZTd2l0Y2ggcGVyZm9ybWFuY2UgdGVzdHMNCj4g
YXJlOg0KPiA+ICAgIHRoZSBpbnB1dCBwb3J0LCB0aGUgc291cmNlIElQIGFkZHJlc3MsIHRoZSBk
ZXN0aW5hdGlvbiBJUCBhZGRyZXNzDQo+IGFuZA0KPiA+ICAgIHRoZSBFdGhlcm5ldCBwcm90b2Nv
bCB0eXBlIGZpZWxkLg0KPiA+DQo+ID4gWyBXSzogSXMgaXQgdXNlZnVsIHRvIHNheSAiNS10dXBs
ZSIgYWJvdmU/IFRvIG1lIDUtdHVwbGUgaXMgc3JjIGlwLA0KPiA+IHNyYyBwb3J0LCBkc3QgaXAs
IGRzdCBwb3J0LCBpcCBwcm90b2NvbC4gVGhpcyBzYXlzIGZvciB2c3dpdGNoIGl0IGlzDQo+ID4g
aW5wdXQgcG9ydCwgc3JjIGlwLCBkc3QgaXAgYW5kIGV0aCBwcm90b2NvbC4gVGhlIHR3byBjbG9z
ZSB0b2dldGhlcg0KPiA+IG1heSBjb25mdXNlIHJlYWRlcnMgLSBJIHRoaW5rIGp1c3QgZHJvcCAi
KDUtdHVwbGUpIi4gQWxzbywgdGhpcyBzYXlzDQo+ID4gImlucHV0IHBvcnQiIC0gaXMgdGhhdCBh
Y3R1YWxseSBpbnB1dCBpbnRlcmZhY2UgLyBzaG91bGQgaXQgYmUgY2FsbGVkDQo+ID4gInBoeXNp
Y2FsIHBvcnQiPyAod2hlbiBoZWFyaW5nIElQIGFuZCB0aGVuIHBvcnQsIHBlb3BsZSB0aGluayBM
Mw0KPiA+IHBvcnQpXQ0KPiA+DQo+ID4NCj4gPiAgICBvICBTY2FsYWJpbGl0eSBUZXN0cyB0byB1
bmRlcnN0YW5kIGhvdyB0aGUgdmlydHVhbCBzd2l0Y2ggcGVyZm9ybXMNCj4gYXMNCj4gPiAgICAg
ICB0aGUgbnVtYmVyIG9mIGZsb3dzLCBhY3RpdmUgcG9ydHMsIGNvbXBsZXhpdHkgb2YgdGhlIGZv
cndhcmRpbmcNCj4gPiAgICAgICBsb2dpYydzIGNvbmZpZ3VyYXRpb24uLi4gaXQgaGFzIHRvIGRl
YWwgd2l0aCBpbmNyZWFzZXMuDQo+ID4NCj4gPiBbIFdLOiBMb29rcyBsaWtlIGdvdCBzaWRldHJh
Y2tlZCBpbiB0aGlzIHNlbnRlbmNlLiBdDQo+ID4NCj4gPg0KPiA+IC0tLS0tLS0tLQ0KPiA+DQo+
ID4gVGhhbmtzIGFnYWluLA0KPiA+IFcNCj4gPg0KPiA+IC0tDQo+ID4gSSBkb24ndCB0aGluayB0
aGUgZXhlY3V0aW9uIGlzIHJlbGV2YW50IHdoZW4gaXQgd2FzIG9idmlvdXNseSBhIGJhZA0KPiA+
IGlkZWEgaW4gdGhlIGZpcnN0IHBsYWNlLg0KPiA+IFRoaXMgaXMgbGlrZSBwdXR0aW5nIHJhYmlk
IHdlYXNlbHMgaW4geW91ciBwYW50cywgYW5kIGxhdGVyIGV4cHJlc3NpbmcNCj4gPiByZWdyZXQg
YXQgaGF2aW5nIGNob3NlbiB0aG9zZSBwYXJ0aWN1bGFyIHJhYmlkIHdlYXNlbHMgYW5kIHRoYXQg
cGFpcg0KPiA+IG9mIHBhbnRzLg0KPiA+ICAgIC0tLW1hZg0K


From nobody Tue May  2 05:32:01 2017
Return-Path: <sjacob@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6188D129B07 for <bmwg@ietfa.amsl.com>; Tue,  2 May 2017 05:31:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.021
X-Spam-Level: 
X-Spam-Status: No, score=-2.021 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XeIm8DTEiOdt for <bmwg@ietfa.amsl.com>; Tue,  2 May 2017 05:31:57 -0700 (PDT)
Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0108.outbound.protection.outlook.com [104.47.34.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0145C1316A3 for <bmwg@ietf.org>; Tue,  2 May 2017 05:28:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=6bXmdstx/wQ0tiXw9/rYYrJlgUlTgjyIxT4vuB5kcuY=; b=VTQ/l69DIfkG38h821DNV3URlDjSRNU6/1v2RWxVQMhTrpvVkCXE4Sc5N2JMCknGRiXSCvTylCZ0KSCNNvbS3/8xplDbgI90JpmbezkNDiR6SE+8ven4uqBQKhk/zWOvnBLy/UTzOBsButKfwcPOEjt9hvxYlvPKfzeBvQu0mWY=
Received: from DM2PR0501MB1360.namprd05.prod.outlook.com (10.160.130.24) by CY1PR05MB2492.namprd05.prod.outlook.com (10.167.10.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1075.1; Tue, 2 May 2017 12:28:22 +0000
Received: from DM2PR0501MB1360.namprd05.prod.outlook.com ([10.160.130.24]) by DM2PR0501MB1360.namprd05.prod.outlook.com ([10.160.130.24]) with mapi id 15.01.1075.010; Tue, 2 May 2017 12:28:19 +0000
From: Sudhin Jacob <sjacob@juniper.net>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: "LINGALA, AVINASH" <ar977m@att.com>, Kishore Tiruveedhula <kishoret@juniper.net>
Thread-Topic: REG:draft-kishjac-bmwg-evpntest
Thread-Index: AdLDPwh+QhQahxB0QYifcGnKUrvt6A==
Date: Tue, 2 May 2017 12:28:19 +0000
Message-ID: <DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170@DM2PR0501MB1360.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.10]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; CY1PR05MB2492; 7:a9rPfcinOZUsfc+hZXsHPemcGI/Mfp8zDx7QBUlnWtQKFBFgc6nbzcp6X2GshgQQhTzhnKrT7JHPMjPQwO1LI+/5eO3J298zwBiloTz8YNv1lH6OX9DZNznjhuB7z9V8kd4So8kX51hkDAgzrydf+ESFN0Du+JvKO2ju5OJrdtHRrIpW9FwDfKGk1jfDBCfBYiSXdOzo9C8sT2ktptTlWB//KH8XnsSF1KKxJgF6bXfM5XIasABMXmql9++r/Tn2QIYZv4PRKKpxuOtbblolhqwSShfw1wEwe7Hw6F0HGRZ5iejRnFfgxczd8LEp/7E4ydWN/pbI8SGGUZy+gwJOFQ==
x-forefront-antispam-report: SFV:SKI; SCL:-1SFV:NSPM; SFS:(10019020)(6009001)(39850400002)(39400400002)(39450400003)(39860400002)(39840400002)(39410400002)(53754006)(74316002)(966004)(86362001)(122556002)(189998001)(53936002)(305945005)(4326008)(2501003)(6916009)(38730400002)(110136004)(33656002)(107886003)(2900100001)(77096006)(8936002)(3280700002)(81166006)(2906002)(6116002)(102836003)(6436002)(3846002)(1730700003)(66066001)(2351001)(478600001)(8676002)(3660700001)(5640700003)(50986999)(54356999)(230783001)(25786009)(7696004)(9686003)(6306002)(7736002)(54906002)(5660300001)(6506006)(99286003)(55016002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR05MB2492; H:DM2PR0501MB1360.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
x-ms-office365-filtering-correlation-id: 77bed941-2660-4378-dcb2-08d49156b78f
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081);  SRVR:CY1PR05MB2492; 
x-microsoft-antispam-prvs: <CY1PR05MB24922D24FE61E7895F5E8EB2C2170@CY1PR05MB2492.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(120809045254105)(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(20161123562025)(20161123555025)(20161123560025)(6072148); SRVR:CY1PR05MB2492; BCL:0; PCL:0; RULEID:; SRVR:CY1PR05MB2492; 
x-forefront-prvs: 02951C14DC
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170DM2PR0501MB1360_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 May 2017 12:28:19.8018 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR05MB2492
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/9iqlWzPTflVk1QjG-Aoryt7E0XM>
Subject: [bmwg] REG:draft-kishjac-bmwg-evpntest
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 02 May 2017 12:31:59 -0000

--_000_DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170DM2PR0501MB1360_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi All,

Kindly find the latest revision of our draft. This current revision of the =
 draft has incorporated inputs given by Avinash and Jay. New section is add=
ed for type 5 route scaling.
Kindly requesting the workgroup chairs for the adoption of this draft as th=
is was discussed in IETF 98. Since this draft has undergrown many revisions=
 and it has reached a maturity stage.


https://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntest/

Regards,
Sudhin


--_000_DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170DM2PR0501MB1360_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left:=
 #800000 2px solid; } --></style>
</head>
<body>
<font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;">
<div>Hi All,</div>
<div>&nbsp;</div>
<div>Kindly find the latest revision of our draft. This current revision of=
 the&nbsp; draft has incorporated inputs given by Avinash and Jay. New sect=
ion is added for type 5 route scaling.</div>
<div>Kindly requesting the workgroup chairs for the adoption of this draft =
as this was discussed in IETF 98. Since this draft has undergrown many revi=
sions and it has reached a maturity stage.</div>
<div>&nbsp;</div>
<div>&nbsp;</div>
<div><a href=3D"https://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntes=
t/"><font color=3D"#0563C1"><u>https://datatracker.ietf.org/doc/draft-kishj=
ac-bmwg-evpntest/</u></font></a></div>
<div>&nbsp;</div>
<div>Regards,</div>
<div>Sudhin</div>
<div>&nbsp;</div>
</span></font>
</body>
</html>

--_000_DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170DM2PR0501MB1360_--


From nobody Tue May  2 06:17:47 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83072131697 for <bmwg@ietfa.amsl.com>; Tue,  2 May 2017 06:17:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.411
X-Spam-Level: 
X-Spam-Status: No, score=-3.411 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lHCF7J6k3pXm for <bmwg@ietfa.amsl.com>; Tue,  2 May 2017 06:17:44 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AB0A813170C for <bmwg@ietf.org>; Tue,  2 May 2017 06:13:29 -0700 (PDT)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v42D5HXX046892; Tue, 2 May 2017 09:13:28 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0083689.ppops.net-00191d01. with ESMTP id 2a6hekkdbq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 02 May 2017 09:13:28 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v42DDQYL003834; Tue, 2 May 2017 09:13:27 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v42DDGdQ003521 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 2 May 2017 09:13:20 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor); Tue, 2 May 2017 13:13:12 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v42DDCPe007534; Tue, 2 May 2017 08:13:12 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v42DD2CC006788; Tue, 2 May 2017 08:13:04 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id C1990E070A; Tue,  2 May 2017 09:12:38 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 2 May 2017 09:13:01 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Sudhin Jacob <sjacob@juniper.net>, "bmwg@ietf.org" <bmwg@ietf.org>
CC: Kishore Tiruveedhula <kishoret@juniper.net>, "LINGALA, AVINASH" <ar977m@att.com>
Thread-Topic: REG:draft-kishjac-bmwg-evpntest
Thread-Index: AdLDPwh+QhQahxB0QYifcGnKUrvt6AABoDlw
Date: Tue, 2 May 2017 13:13:01 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F91035@njmtexg4.research.att.com>
References: <DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170@DM2PR0501MB1360.namprd05.prod.outlook.com>
In-Reply-To: <DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170@DM2PR0501MB1360.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF25F91035njmtexg4researc_"
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-02_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705020074
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/bZZl7ya9Kw9zS-nO2lnZDJpj4Yo>
Subject: Re: [bmwg] REG:draft-kishjac-bmwg-evpntest
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 02 May 2017 13:17:46 -0000

--_000_4D7F4AD313D3FC43A053B309F97543CF25F91035njmtexg4researc_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Thanks for your updated draft, Sudhin.

There was good readership and support
of the draft at the meeting, and we'll conduct
a call for adoption on the mailing list shortly.

regards,
Al
bmwg co-chair

From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Sudhin Jacob
Sent: Tuesday, May 02, 2017 8:28 AM
To: bmwg@ietf.org
Cc: Kishore Tiruveedhula; LINGALA, AVINASH
Subject: [bmwg] REG:draft-kishjac-bmwg-evpntest

Hi All,

Kindly find the latest revision of our draft. This current revision of the =
 draft has incorporated inputs given by Avinash and Jay. New section is add=
ed for type 5 route scaling.
Kindly requesting the workgroup chairs for the adoption of this draft as th=
is was discussed in IETF 98. Since this draft has undergrown many revisions=
 and it has reached a maturity stage.


https://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntest/<https://urlde=
fense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.org_doc_draft-2D=
kishjac-2Dbmwg-2Devpntest_&d=3DDwMFAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8=
kTIltVyD1oL72cBw&m=3DbOk8cy70zHgOaT298ICyXSydjcmirM9FZa2HACAPEvs&s=3DJ6spbj=
gYyRBwGZZm3avjmruTL9h65BsA7NXHYMHK6Lw&e=3D>

Regards,
Sudhin


--_000_4D7F4AD313D3FC43A053B309F97543CF25F91035njmtexg4researc_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
p.emailquote, li.emailquote, div.emailquote
	{mso-style-name:emailquote;
	mso-margin-top-alt:auto;
	margin-right:0in;
	mso-margin-bottom-alt:auto;
	margin-left:1.0pt;
	border:none;
	padding:0in;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
span.EmailStyle18
	{mso-style-type:personal-reply;
	font-family:"Courier New";
	color:black;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Thanks for your updated draft, Sudhin.<o:p></o=
:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">There was good readership and support<o:p></o:=
p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">of the draft at the meeting, and we&#8217;ll c=
onduct
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">a call for adoption on the mailing list shortl=
y.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">regards,<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Al<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">bmwg co-chair<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<div style=3D"border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in =
4.0pt">
<div>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b><span style=3D"font-size:10.0pt;font-family:&quot=
;Tahoma&quot;,&quot;sans-serif&quot;">From:</span></b><span style=3D"font-s=
ize:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> bmwg [ma=
ilto:bmwg-bounces@ietf.org]
<b>On Behalf Of </b>Sudhin Jacob<br>
<b>Sent:</b> Tuesday, May 02, 2017 8:28 AM<br>
<b>To:</b> bmwg@ietf.org<br>
<b>Cc:</b> Kishore Tiruveedhula; LINGALA, AVINASH<br>
<b>Subject:</b> [bmwg] REG:draft-kishjac-bmwg-evpntest<o:p></o:p></span></p=
>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">Hi All,<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">Kindly find the latest revision of our =
draft. This current revision of the&nbsp; draft has incorporated inputs giv=
en by Avinash and Jay. New section is added for type 5 route
 scaling.<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">Kindly requesting the workgroup chairs =
for the adoption of this draft as this was discussed in IETF 98. Since this=
 draft has undergrown many revisions and it has reached
 a maturity stage.<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;"><a href=3D"https://urldefense.proofpoin=
t.com/v2/url?u=3Dhttps-3A__datatracker.ietf.org_doc_draft-2Dkishjac-2Dbmwg-=
2Devpntest_&amp;d=3DDwMFAg&amp;c=3DLFYZ-o9_HUMeMTSQicvjIg&amp;r=3DOfsSu8kTI=
ltVyD1oL72cBw&amp;m=3DbOk8cy70zHgOaT298ICyXSydjcmirM9FZa2HACAPEvs&amp;s=3DJ=
6spbjgYyRBwGZZm3avjmruTL9h65BsA7NXHYMHK6Lw&amp;e=3D"><span style=3D"color:#=
0563C1">https://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntest/</span=
></a><o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">Regards,<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">Sudhin<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,&quot;sans-serif&quot;">&nbsp;<o:p></o:p></span></p>
</div>
</div>
</div>
</body>
</html>

--_000_4D7F4AD313D3FC43A053B309F97543CF25F91035njmtexg4researc_--


From nobody Tue May  2 06:19:14 2017
Return-Path: <sjacob@juniper.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 376D2129B5A for <bmwg@ietfa.amsl.com>; Tue,  2 May 2017 06:19:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.032
X-Spam-Level: 
X-Spam-Status: No, score=-0.032 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7LYD79RxOf3r for <bmwg@ietfa.amsl.com>; Tue,  2 May 2017 06:19:05 -0700 (PDT)
Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0102.outbound.protection.outlook.com [104.47.32.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 740A51316C4 for <bmwg@ietf.org>; Tue,  2 May 2017 06:15:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=2hgQEWQZm/y6hVzjiRapNFRRd20nNcHhrbOQJvvB3JY=; b=cO6uusADK6M+216E01NVziy0Dyp0KdSDWNXemQg9VW9wl//6XGxWywOGvJ1bwDTHPfYf8AepZCoaWIirQA7W31qG4tKBCUmICzAgJXX60RMiLEaVGp0pqQnZwTIsv21F7I1bv8sL1dfMKRk16Rufo3wjvMEbrwnXUChD44sl7dQ=
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com (10.160.200.24) by CO2PR05MB2487.namprd05.prod.outlook.com (10.166.95.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1075.1; Tue, 2 May 2017 13:15:00 +0000
Received: from BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) by BY1PR0501MB1352.namprd05.prod.outlook.com ([10.160.200.24]) with mapi id 15.01.1075.010; Tue, 2 May 2017 13:14:58 +0000
From: Sudhin Jacob <sjacob@juniper.net>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>, "bmwg@ietf.org" <bmwg@ietf.org>
CC: Kishore Tiruveedhula <kishoret@juniper.net>, "LINGALA, AVINASH" <ar977m@att.com>
Thread-Topic: REG:draft-kishjac-bmwg-evpntest
Thread-Index: AdLDPwh+QhQahxB0QYifcGnKUrvt6AABoDlwAAAdybA=
Date: Tue, 2 May 2017 13:14:58 +0000
Message-ID: <BY1PR0501MB135239B8EAE02F8BF6BB97B1C2170@BY1PR0501MB1352.namprd05.prod.outlook.com>
References: <DM2PR0501MB1360BE0A4A68EE0DA6699E81C2170@DM2PR0501MB1360.namprd05.prod.outlook.com> <4D7F4AD313D3FC43A053B309F97543CF25F91035@njmtexg4.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25F91035@njmtexg4.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: att.com; dkim=none (message not signed) header.d=none;att.com; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [116.197.184.10]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; CO2PR05MB2487; 7:ww35PO577Iqbx36FQd/tawec7OnCZMijMbQNvZN6YMXoirVvqJuTJptq8FUjH+dDHMX4cJ60DWwCpr1vxUPUuqqWnEqDIPNz3ISLLIDXYURBge8HNKp3WCSS2dNayQFX4BdKMjew3+eHnjRisa0KM7KMQzqYoBDmyKpP4sNoMzZagCRsQQqUtz8ZTUQFf4eHp2ieImd+hmcpPklNRL7zenAzD5ESSnNH46D82chSMMX87dBtS5ephRGBNFPGeAZLe8f31MHfi2dUFjBZ1EfdXiX6msMK96zJBWBmgiYYZDtzRc0gBB/Ooxl1HVS7Af9x58MqX9PGHdquh0sAPxhNjA==
x-forefront-antispam-report: SFV:SKI; SCL:-1SFV:NSPM; SFS:(10019020)(39410400002)(39850400002)(39860400002)(39840400002)(39400400002)(39450400003)(377454003)(53754006)(42274003)(7696004)(2906002)(2501003)(2950100002)(6246003)(53936002)(8936002)(345774005)(6506006)(54906002)(77096006)(236005)(6306002)(55016002)(3846002)(790700001)(99286003)(9686003)(54896002)(8676002)(102836003)(3280700002)(3660700001)(6116002)(38730400002)(606005)(33656002)(6436002)(478600001)(66066001)(53546009)(4326008)(81166006)(122556002)(230783001)(189998001)(25786009)(2900100001)(76176999)(50986999)(54356999)(5660300001)(7736002)(19609705001)(86362001)(7906003)(74316002)(229853002); DIR:OUT; SFP:1102; SCL:1; SRVR:CO2PR05MB2487; H:BY1PR0501MB1352.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; 
x-ms-office365-filtering-correlation-id: e9d3ca1b-700c-420a-97a4-08d4915d3bb8
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081)(201702281549075); SRVR:CO2PR05MB2487; 
x-microsoft-antispam-prvs: <CO2PR05MB248703A115283578B50D50C8C2170@CO2PR05MB2487.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(10436049006162)(120809045254105)(138986009662008)(97927398514766)(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123562025)(20161123555025)(20161123560025)(6072148); SRVR:CO2PR05MB2487; BCL:0; PCL:0; RULEID:; SRVR:CO2PR05MB2487; 
x-forefront-prvs: 02951C14DC
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_BY1PR0501MB135239B8EAE02F8BF6BB97B1C2170BY1PR0501MB1352_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 May 2017 13:14:58.3990 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR05MB2487
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/LcI9Aqh81hXjP4Daf3KoMjKJR0c>
Subject: Re: [bmwg] REG:draft-kishjac-bmwg-evpntest
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 02 May 2017 13:19:12 -0000

--_000_BY1PR0501MB135239B8EAE02F8BF6BB97B1C2170BY1PR0501MB1352_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Al/Sarah,

Thank you for your support and help.

Regards,
Sudhin

From: MORTON, ALFRED C (AL) [mailto:acmorton@att.com]
Sent: Tuesday, May 2, 2017 6:43 PM
To: Sudhin Jacob <sjacob@juniper.net>; bmwg@ietf.org
Cc: Kishore Tiruveedhula <kishoret@juniper.net>; LINGALA, AVINASH <ar977m@a=
tt.com>
Subject: RE: REG:draft-kishjac-bmwg-evpntest

Thanks for your updated draft, Sudhin.

There was good readership and support
of the draft at the meeting, and we'll conduct
a call for adoption on the mailing list shortly.

regards,
Al
bmwg co-chair

From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Sudhin Jacob
Sent: Tuesday, May 02, 2017 8:28 AM
To: bmwg@ietf.org<mailto:bmwg@ietf.org>
Cc: Kishore Tiruveedhula; LINGALA, AVINASH
Subject: [bmwg] REG:draft-kishjac-bmwg-evpntest

Hi All,

Kindly find the latest revision of our draft. This current revision of the =
 draft has incorporated inputs given by Avinash and Jay. New section is add=
ed for type 5 route scaling.
Kindly requesting the workgroup chairs for the adoption of this draft as th=
is was discussed in IETF 98. Since this draft has undergrown many revisions=
 and it has reached a maturity stage.


https://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntest/<https://urlde=
fense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.org_doc_draft-2D=
kishjac-2Dbmwg-2Devpntest_&d=3DDwMFAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8=
kTIltVyD1oL72cBw&m=3DbOk8cy70zHgOaT298ICyXSydjcmirM9FZa2HACAPEvs&s=3DJ6spbj=
gYyRBwGZZm3avjmruTL9h65BsA7NXHYMHK6Lw&e=3D>

Regards,
Sudhin


--_000_BY1PR0501MB135239B8EAE02F8BF6BB97B1C2170BY1PR0501MB1352_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
	{mso-style-name:msonormal;
	mso-margin-top-alt:auto;
	margin-right:0in;
	mso-margin-bottom-alt:auto;
	margin-left:0in;
	font-size:12.0pt;
	font-family:"Times New Roman",serif;}
p.emailquote, li.emailquote, div.emailquote
	{mso-style-name:emailquote;
	mso-margin-top-alt:auto;
	margin-right:0in;
	mso-margin-bottom-alt:auto;
	margin-left:1.0pt;
	font-size:12.0pt;
	font-family:"Times New Roman",serif;}
span.EmailStyle19
	{mso-style-type:personal;
	font-family:"Courier New";
	color:black;}
span.EmailStyle20
	{mso-style-type:personal-reply;
	font-family:"Calibri",sans-serif;
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D">Hi Al/Sarah,<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D">Thank you for your support and help.<=
o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D">Regards,<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D">Sudhin<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif;color:#1F497D"><o:p>&nbsp;</o:p></span></p>
<div>
<div style=3D"border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b><span style=3D"font-size:11.0pt;font-family:&quot=
;Calibri&quot;,sans-serif">From:</span></b><span style=3D"font-size:11.0pt;=
font-family:&quot;Calibri&quot;,sans-serif"> MORTON, ALFRED C (AL) [mailto:=
acmorton@att.com]
<br>
<b>Sent:</b> Tuesday, May 2, 2017 6:43 PM<br>
<b>To:</b> Sudhin Jacob &lt;sjacob@juniper.net&gt;; bmwg@ietf.org<br>
<b>Cc:</b> Kishore Tiruveedhula &lt;kishoret@juniper.net&gt;; LINGALA, AVIN=
ASH &lt;ar977m@att.com&gt;<br>
<b>Subject:</b> RE: REG:draft-kishjac-bmwg-evpntest<o:p></o:p></span></p>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Thanks for your updated draft, Sudhin.<o:p></o=
:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">There was good readership and support<o:p></o:=
p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">of the draft at the meeting, and we&#8217;ll c=
onduct
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">a call for adoption on the mailing list shortl=
y.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">regards,<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Al<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">bmwg co-chair<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<div style=3D"border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in =
4.0pt">
<div>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b><span style=3D"font-size:10.0pt;font-family:&quot=
;Tahoma&quot;,sans-serif">From:</span></b><span style=3D"font-size:10.0pt;f=
ont-family:&quot;Tahoma&quot;,sans-serif"> bmwg [<a href=3D"mailto:bmwg-bou=
nces@ietf.org">mailto:bmwg-bounces@ietf.org</a>]
<b>On Behalf Of </b>Sudhin Jacob<br>
<b>Sent:</b> Tuesday, May 02, 2017 8:28 AM<br>
<b>To:</b> <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
<b>Cc:</b> Kishore Tiruveedhula; LINGALA, AVINASH<br>
<b>Subject:</b> [bmwg] REG:draft-kishjac-bmwg-evpntest<o:p></o:p></span></p=
>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">Hi All,<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">Kindly find the latest revision of our draft. This =
current revision of the&nbsp; draft has incorporated inputs given by Avinas=
h and Jay. New section is added for type 5 route scaling.<o:p></o:p></span>=
</p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">Kindly requesting the workgroup chairs for the adop=
tion of this draft as this was discussed in IETF 98. Since this draft has u=
ndergrown many revisions and it has reached a
 maturity stage.<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif"><a href=3D"https://urldefense.proofpoint.com/v2/url=
?u=3Dhttps-3A__datatracker.ietf.org_doc_draft-2Dkishjac-2Dbmwg-2Devpntest_&=
amp;d=3DDwMFAg&amp;c=3DLFYZ-o9_HUMeMTSQicvjIg&amp;r=3DOfsSu8kTIltVyD1oL72cB=
w&amp;m=3DbOk8cy70zHgOaT298ICyXSydjcmirM9FZa2HACAPEvs&amp;s=3DJ6spbjgYyRBwG=
ZZm3avjmruTL9h65BsA7NXHYMHK6Lw&amp;e=3D"><span style=3D"color:#0563C1">http=
s://datatracker.ietf.org/doc/draft-kishjac-bmwg-evpntest/</span></a><o:p></=
o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">Regards,<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">Sudhin<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Ca=
libri&quot;,sans-serif">&nbsp;<o:p></o:p></span></p>
</div>
</div>
</div>
</body>
</html>

--_000_BY1PR0501MB135239B8EAE02F8BF6BB97B1C2170BY1PR0501MB1352_--


From nobody Thu May 11 04:05:59 2017
Return-Path: <dromasca@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id D696212EBC0; Thu, 11 May 2017 04:05:50 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Dan Romascanu <dromasca@gmail.com>
To: <gen-art@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv.all@ietf.org, ietf@ietf.org, bmwg@ietf.org,  dromasca@gmail.com
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149450075072.16690.14546662616864459158@ietfa.amsl.com>
Date: Thu, 11 May 2017 04:05:50 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dPPvMYty2hTja-odYZx0FgwuUuQ>
Subject: [bmwg] Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 May 2017 11:05:51 -0000

Reviewer: Dan Romascanu
Review result: Almost Ready

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-vswitch-opnfv-??
Reviewer: Dan Romascanu
Review Date: 2017-05-11
IETF LC End Date: 2017-05-15
IESG Telechat date: Not scheduled for a telechat

Summary:

Almost Ready. 

This document describes describes the progress of the Open Platform
for NFV (OPNFV) project on virtual switch performance "VSPERF". That
project reuses the BMWG framework and specifications to benchmark
virtual switches implemented in general-purpose hardware. Some
differences with the benchmarking of specialized HW platforms are
identified and they may become work items for BMWG in the future. It's
a well written and clear document, but I have reservations about it
being published as an RFC, and I cannot find coverage for it in the WG
charter. I also have concerns that parts of the methodology used by
OPNFV break the BMWG principles, especially repeatability and
'black-box', and this is not clear enough articulated in the document.
 

Major issues:

1. It is not clear to me why this document needs to be published as an
RFC. The introduction says: 'This memo describes the progress of the
Open Platform for NFV (OPNFV) project on virtual switch performance
"VSPERF".  This project intends to build on the current and completed
work of the Benchmarking Methodology Working Group in IETF, by
referencing existing literature.' Why should the WG and the IESG
invest resources in publishing this, why an I-D or an Independent
Stream RFC is not sufficient? The WG charter says something about:
'VNF and Related Infrastructure Benchmarking: Benchmarking
Methodologies have reliably characterized many physical devices. This
work item extends and enhances the methods to virtual network
functions (VNF) and their unique supporting infrastructure. A first
deliverable from this activity will be a document that considers the
new benchmarking space to ensure that common issues are recognized
from the start, using background materials from industry and SDOs
(e.g., IETF, ETSI NFV).'. I do not believe that this document covers
the intent of the charter, as it focused on one organization only. 

2. In section 3 there 'repeatability' is mentioned, while
acknowledging that in a virtual environment there is no guarantee and
actually no way to know what other applications are being run.
Measuring parameters as the ones listed in 3.3 provides just part of
the answer, and they are internal parameters to the SUT. Also, the
different deployment scenarios in section 4 require different
configurations for the SUT, thus breaking the 'black-box' principle. I
believe that there is a need for a more clear explanation of why BMWG
specifications are appropriate and how comparison can be made while
repeatability cannot be ensured, and measurements are dependent upon
parameters internal to the SUT. 

Minor issues:

1. Some of the tests mentioned in Section 4 have no prior or in
progress work in the IETF: Control Path and Datapath Coupling Tests,
Noisy Neighbour Tests, characterization of acceleration technologies.
If new work is needed / proposed to be added for the BMWG scope and
framework it would be useful for BMWG to list these separately. 


Nits/editorial comments: 

1. What is called 'Deployment scenarios' from VS perspective in
Section 4 describe in fact different configurations of the SUT in BMWG
terms. It seems better to separate this second part of section 4 in a
separate section. If it belongs to an existing section it rather
belongs in 3 than in 4. 




From nobody Thu May 11 04:11:21 2017
Return-Path: <dromasca@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E66F612EC61; Thu, 11 May 2017 04:11:15 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Dan Romascanu <dromasca@gmail.com>
To: <ops-dir@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv.all@ietf.org, ietf@ietf.org, bmwg@ietf.org,  dromasca@gmail.com
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149450107581.16728.18142473358782662679@ietfa.amsl.com>
Date: Thu, 11 May 2017 04:11:15 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/XqSdh97_m2bBJXOkwbfthAkr8BY>
Subject: [bmwg] Opsdir last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 May 2017 11:11:16 -0000

Reviewer: Dan Romascanu
Review result: Has Issues

This document describes describes the progress of the Open Platform
for NFV (OPNFV) project on virtual switch performance "VSPERF". That
project reuses the BMWG framework and specifications to benchmark
virtual switches implemented in general-purpose hardware. Some
differences with the benchmarking of specialized HW platforms are
identified and they may become work items for BMWG in the future. It's
a well written and clear document, but I have reservations about it
being published as an RFC, and I cannot find coverage for it in the WG
charter. I also have concerns that parts of the methodology used by
OPNFV break the BMWG principles, especially repeatability and
'black-box', and this is not clear enough articulated in the document.
 As I was assigned both OPS-DIR and Gen-ART reviews for this document,
I detailed the concerns in my Gen-ART review, they seem to belong
better there. 

>From an OPS-DIR perspective this document has no issues. If the
concerns in section 6 are addressed and caution is taken to isolate
the SUTs and benchmarking environments from the Internet or
operational intranets, there is no operational impact. If approved it
would be a useful tool for operators to get some information about how
benchmarking of OPNFV project products are being designed. 




From nobody Thu May 11 10:06:11 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3D24D12948E; Thu, 11 May 2017 10:06:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J8J-RD_U1eNV; Thu, 11 May 2017 10:06:07 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF8CA12EBAE; Thu, 11 May 2017 10:01:10 -0700 (PDT)
Received: from pps.filterd (m0049295.ppops.net [127.0.0.1]) by m0049295.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v4BGuWhv004428; Thu, 11 May 2017 13:01:08 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049295.ppops.net-00191d01. with ESMTP id 2acrmw7df8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 May 2017 13:01:07 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v4BH12Ks024014; Thu, 11 May 2017 13:01:05 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v4BH0h80022033 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 11 May 2017 13:00:57 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor); Thu, 11 May 2017 17:00:29 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4BH0Ssq007091; Thu, 11 May 2017 12:00:29 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4BH0L1n006154; Thu, 11 May 2017 12:00:21 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id A3FE3E08B7; Thu, 11 May 2017 13:00:20 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 11 May 2017 13:00:20 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Dan Romascanu <dromasca@gmail.com>, "gen-art@ietf.org" <gen-art@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" <draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>, "ietf@ietf.org" <ietf@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
Thread-Index: AQHSykaQpvZtcVQqDEiJ+tr7bcWy3KHvK1EQ
Date: Thu, 11 May 2017 17:00:20 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F9B072@njmtexg4.research.att.com>
References: <149450075072.16690.14546662616864459158@ietfa.amsl.com>
In-Reply-To: <149450075072.16690.14546662616864459158@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.70.214.90]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-11_13:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705110087
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/drt81Cfg_5fDN_05XYO1pnXUZ3w>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 May 2017 17:06:10 -0000

SGkgRGFuLA0KcGxlYXNlIHNlZSByZXBsaWVzLCBbQUNNXSwgYmVsb3cuDQoNCj4gLS0tLS1Pcmln
aW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogRGFuIFJvbWFzY2FudSBbbWFpbHRvOmRyb21hc2Nh
QGdtYWlsLmNvbV0NCj4gU2VudDogVGh1cnNkYXksIE1heSAxMSwgMjAxNyA3OjA2IEFNDQo+IFRv
OiBnZW4tYXJ0QGlldGYub3JnDQo+IENjOiBkcmFmdC1pZXRmLWJtd2ctdnN3aXRjaC1vcG5mdi5h
bGxAaWV0Zi5vcmc7IGlldGZAaWV0Zi5vcmc7DQo+IGJtd2dAaWV0Zi5vcmc7IGRyb21hc2NhQGdt
YWlsLmNvbQ0KPiBTdWJqZWN0OiBHZW5hcnQgbGFzdCBjYWxsIHJldmlldyBvZiBkcmFmdC1pZXRm
LWJtd2ctdnN3aXRjaC1vcG5mdi0wMw0KPiANCj4gUmV2aWV3ZXI6IERhbiBSb21hc2NhbnUNCj4g
UmV2aWV3IHJlc3VsdDogQWxtb3N0IFJlYWR5DQo+IA0KPiBJIGFtIHRoZSBhc3NpZ25lZCBHZW4t
QVJUIHJldmlld2VyIGZvciB0aGlzIGRyYWZ0LiBUaGUgR2VuZXJhbCBBcmVhDQo+IFJldmlldyBU
ZWFtIChHZW4tQVJUKSByZXZpZXdzIGFsbCBJRVRGIGRvY3VtZW50cyBiZWluZyBwcm9jZXNzZWQN
Cj4gYnkgdGhlIElFU0cgZm9yIHRoZSBJRVRGIENoYWlyLiAgUGxlYXNlIHRyZWF0IHRoZXNlIGNv
bW1lbnRzIGp1c3QNCj4gbGlrZSBhbnkgb3RoZXIgbGFzdCBjYWxsIGNvbW1lbnRzLg0KPiANCj4g
Rm9yIG1vcmUgaW5mb3JtYXRpb24sIHBsZWFzZSBzZWUgdGhlIEZBUSBhdA0KPiANCj4gPGh0dHBz
Oi8vdXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0FfX3RyYWMu
aWV0Zi5vcmdfdHJhY19nZW5fd2lraV9HZW5BcnRmYXEmZD1Ed0lDYVEmYz1MRllaLQ0KPiBvOV9I
VU1lTVRTUWljdmpJZyZyPU9mc1N1OGtUSWx0VnlEMW9MNzJjQncmbT1OVFZsTEJOLQ0KPiBMM3U5
ekdQSG1fQ05WY1hXN19PR1g4XzE4Q3RhQWFsWmluMCZzPTJIci0NCj4gZGhLYURISWd1WTdXOTd6
MzNSbEtqcVBEdG1vWW1NMi1qV3JiUy1vJmU9ID4uDQo+IA0KPiBEb2N1bWVudDogZHJhZnQtaWV0
Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDMNCj4gUmV2aWV3ZXI6IERhbiBSb21hc2NhbnUNCj4gUmV2
aWV3IERhdGU6IDIwMTctMDUtMTENCj4gSUVURiBMQyBFbmQgRGF0ZTogMjAxNy0wNS0xNQ0KPiBJ
RVNHIFRlbGVjaGF0IGRhdGU6IE5vdCBzY2hlZHVsZWQgZm9yIGEgdGVsZWNoYXQNCj4gDQo+IFN1
bW1hcnk6DQo+IA0KPiBBbG1vc3QgUmVhZHkuDQo+IA0KPiBUaGlzIGRvY3VtZW50IGRlc2NyaWJl
cyBkZXNjcmliZXMgdGhlIHByb2dyZXNzIG9mIHRoZSBPcGVuIFBsYXRmb3JtDQo+IGZvciBORlYg
KE9QTkZWKSBwcm9qZWN0IG9uIHZpcnR1YWwgc3dpdGNoIHBlcmZvcm1hbmNlICJWU1BFUkYiLiBU
aGF0DQo+IHByb2plY3QgcmV1c2VzIHRoZSBCTVdHIGZyYW1ld29yayBhbmQgc3BlY2lmaWNhdGlv
bnMgdG8gYmVuY2htYXJrDQo+IHZpcnR1YWwgc3dpdGNoZXMgaW1wbGVtZW50ZWQgaW4gZ2VuZXJh
bC1wdXJwb3NlIGhhcmR3YXJlLiBTb21lDQo+IGRpZmZlcmVuY2VzIHdpdGggdGhlIGJlbmNobWFy
a2luZyBvZiBzcGVjaWFsaXplZCBIVyBwbGF0Zm9ybXMgYXJlDQo+IGlkZW50aWZpZWQgYW5kIHRo
ZXkgbWF5IGJlY29tZSB3b3JrIGl0ZW1zIGZvciBCTVdHIGluIHRoZSBmdXR1cmUuIEl0J3MNCj4g
YSB3ZWxsIHdyaXR0ZW4gYW5kIGNsZWFyIGRvY3VtZW50LCBidXQgSSBoYXZlIHJlc2VydmF0aW9u
cyBhYm91dCBpdA0KPiBiZWluZyBwdWJsaXNoZWQgYXMgYW4gUkZDLCBhbmQgSSBjYW5ub3QgZmlu
ZCBjb3ZlcmFnZSBmb3IgaXQgaW4gdGhlIFdHDQo+IGNoYXJ0ZXIuIEkgYWxzbyBoYXZlIGNvbmNl
cm5zIHRoYXQgcGFydHMgb2YgdGhlIG1ldGhvZG9sb2d5IHVzZWQgYnkNCj4gT1BORlYgYnJlYWsg
dGhlIEJNV0cgcHJpbmNpcGxlcywgZXNwZWNpYWxseSByZXBlYXRhYmlsaXR5IGFuZA0KPiAnYmxh
Y2stYm94JywgYW5kIHRoaXMgaXMgbm90IGNsZWFyIGVub3VnaCBhcnRpY3VsYXRlZCBpbiB0aGUg
ZG9jdW1lbnQuDQpbQUNNXSANCk9rLCBsZXQncyBhZGRyZXNzIHlvdXIgc3BlY2lmaWMgaXNzdWVz
LCBhbmQgY29tZSBiYWNrIHRvIHlvdXIgcmVzZXJ2YXRpb25zLg0KDQo+IA0KPiANCj4gTWFqb3Ig
aXNzdWVzOg0KPiANCj4gMS4gSXQgaXMgbm90IGNsZWFyIHRvIG1lIHdoeSB0aGlzIGRvY3VtZW50
IG5lZWRzIHRvIGJlIHB1Ymxpc2hlZCBhcyBhbg0KPiBSRkMuIFRoZSBpbnRyb2R1Y3Rpb24gc2F5
czogJ1RoaXMgbWVtbyBkZXNjcmliZXMgdGhlIHByb2dyZXNzIG9mIHRoZQ0KPiBPcGVuIFBsYXRm
b3JtIGZvciBORlYgKE9QTkZWKSBwcm9qZWN0IG9uIHZpcnR1YWwgc3dpdGNoIHBlcmZvcm1hbmNl
DQo+ICJWU1BFUkYiLiAgVGhpcyBwcm9qZWN0IGludGVuZHMgdG8gYnVpbGQgb24gdGhlIGN1cnJl
bnQgYW5kIGNvbXBsZXRlZA0KPiB3b3JrIG9mIHRoZSBCZW5jaG1hcmtpbmcgTWV0aG9kb2xvZ3kg
V29ya2luZyBHcm91cCBpbiBJRVRGLCBieQ0KPiByZWZlcmVuY2luZyBleGlzdGluZyBsaXRlcmF0
dXJlLicgV2h5IHNob3VsZCB0aGUgV0cgYW5kIHRoZSBJRVNHDQo+IGludmVzdCByZXNvdXJjZXMg
aW4gcHVibGlzaGluZyB0aGlzLCB3aHkgYW4gSS1EIG9yIGFuIEluZGVwZW5kZW50DQo+IFN0cmVh
bSBSRkMgaXMgbm90IHN1ZmZpY2llbnQ/IA0KW0FDTV0gDQpUaGUgV0cgY29uc2lkZXJlZCBhbmQg
ZGlzY3Vzc2VkIHRoaXMgZG9jdW1lbnQgb3ZlciAzIHJldmlzaW9ucw0KYW5kIGEgeWVhciBvZiB0
aW1lIGJlZm9yZSByZWFjaGluZyBjb25zZW5zdXMgdG8gZGV2ZWxvcCBpdCBmdXJ0aGVyDQphcyBh
IGNoYXJ0ZXJlZCBpdGVtLCBzbyB0aGlzIGRlY2lzaW9uIHdhcyBub3QgdGFrZW4gbGlnaHRseS4N
ClNlZSBtb3JlIGJlbG93Lg0KDQo+IFRoZSBXRyBjaGFydGVyIHNheXMgc29tZXRoaW5nIGFib3V0
Og0KPiAnVk5GIGFuZCBSZWxhdGVkIEluZnJhc3RydWN0dXJlIEJlbmNobWFya2luZzogQmVuY2ht
YXJraW5nDQo+IE1ldGhvZG9sb2dpZXMgaGF2ZSByZWxpYWJseSBjaGFyYWN0ZXJpemVkIG1hbnkg
cGh5c2ljYWwgZGV2aWNlcy4gVGhpcw0KPiB3b3JrIGl0ZW0gZXh0ZW5kcyBhbmQgZW5oYW5jZXMg
dGhlIG1ldGhvZHMgdG8gdmlydHVhbCBuZXR3b3JrDQo+IGZ1bmN0aW9ucyAoVk5GKSBhbmQgdGhl
aXIgdW5pcXVlIHN1cHBvcnRpbmcgaW5mcmFzdHJ1Y3R1cmUuIEEgZmlyc3QNCj4gZGVsaXZlcmFi
bGUgZnJvbSB0aGlzIGFjdGl2aXR5IHdpbGwgYmUgYSBkb2N1bWVudCB0aGF0IGNvbnNpZGVycyB0
aGUNCj4gbmV3IGJlbmNobWFya2luZyBzcGFjZSB0byBlbnN1cmUgdGhhdCBjb21tb24gaXNzdWVz
IGFyZSByZWNvZ25pemVkDQo+IGZyb20gdGhlIHN0YXJ0LCB1c2luZyBiYWNrZ3JvdW5kIG1hdGVy
aWFscyBmcm9tIGluZHVzdHJ5IGFuZCBTRE9zDQo+IChlLmcuLCBJRVRGLCBFVFNJIE5GVikuJy4g
SSBkbyBub3QgYmVsaWV2ZSB0aGF0IHRoaXMgZG9jdW1lbnQgY292ZXJzDQo+IHRoZSBpbnRlbnQg
b2YgdGhlIGNoYXJ0ZXIsIGFzIGl0IGZvY3VzZWQgb24gb25lIG9yZ2FuaXphdGlvbiBvbmx5Lg0K
W0FDTV0gDQpJJ20gc29ycnksIGJ1dCBoZXJlIHlvdSBhcmUgbWlzdGFrZW4uIFRoZSBkb2N1bWVu
dCB0aGF0IHNhdGlzZmllZA0KdGhlICJmaXJzdCBkZWxpdmVyYWJsZSAuLi4gZG9jdW1lbnQgdGhh
dCBjb25zaWRlcnMgdGhlIG5ldyBiZW5jaG1hcmtpbmcgc3BhY2UiDQppczogaHR0cHM6Ly90b29s
cy5pZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtYm13Zy12aXJ0dWFsLW5ldC0wNQ0KdGl0bGVkOiBD
b25zaWRlcmF0aW9ucyBmb3IgQmVuY2htYXJraW5nIFZpcnR1YWwgTmV0d29yayBGdW5jdGlvbnMg
YW5kIFRoZWlyIEluZnJhc3RydWN0dXJlDQp3aGljaCBoYXMgYmVlbiBzdWJtaXR0ZWQgdG8gSUVT
RyBhbmQgYXBwcm92ZWQgZm9yIHB1YmxpY2F0aW9uLg0KRnVydGhlciwgdGhlIGN1cnJlbnQgZHJh
ZnQgKGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LTAzKQ0KcmVmZXJlbmNlcyB0aGUgYXBw
cm92ZWQgIkNvbnNpZGVyYXRpb25zIiBkcmFmdCBpbiBTZWN0aW9uIDMNCihhcyBkb2VzIGFsbW9z
dCBldmVyeSByZWxhdGVkIEluZHVzdHJ5IHNwZWMgSSdtIGF3YXJlIG9mKS4NCg0KVGhlIEJNV0cg
Q2hhcnRlciBjb250aW51ZXM6DQogIEJlbmNobWFya3MgZm9yIHBsYXRmb3JtIGNhcGFjaXR5IGFu
ZCBwZXJmb3JtYW5jZSBjaGFyYWN0ZXJpc3RpY3Mgb2YNCiAgdmlydHVhbCByb3V0ZXJzLCBzd2l0
Y2hlcywgYW5kIHJlbGF0ZWQgY29tcG9uZW50cyB3aWxsIGZvbGxvdywgaW5jbHVkaW5nDQogIGNv
bXBhcmlzb25zIGJldHdlZW4gcGh5c2ljYWwgYW5kIHZpcnR1YWwgbmV0d29yayBmdW5jdGlvbnMu
IEluIG1hbnkgY2FzZXMsDQogIHRoZSB0cmFkaXRpb25hbCBiZW5jaG1hcmtzIHNob3VsZCBiZSBh
cHBsaWNhYmxlIHRvIFZORnMsIGJ1dCB0aGUgbGFiDQogIHNldC11cHMsIGNvbmZpZ3VyYXRpb25z
LCBhbmQgbWVhc3VyZW1lbnQgbWV0aG9kcyB3aWxsIGxpa2VseSBuZWVkIHRvDQogIGJlIHJldmlz
ZWQgb3IgZW5oYW5jZWQuDQoNClRoaXMgZHJhZnQgY29uc3RpdHV0ZXMgb25lIG9mIHNldmVyYWwg
Zm9sbG93LW9uIGVmZm9ydHMsIGFwcHJvYWNoaW5nIA0KdGhlIHByb2JsZW0gZXhhY3RseSBhcyB3
ZSBkZXNjcmliZWQgaW4gdGhlIGxhc3Qgc2VudGVuY2UgYWJvdmUuDQoNCkFuIGFzcGVjdCBvZiBJ
bmR1c3RyeSBjb2xsYWJvcmF0aW9uIHRoYXQgd2UgZGlkIG5vdCBhbnRpY2lwYXRlIGluIHRoZSAN
CkJNV0cgQ2hhcnRlciBpcyBvdXIgY3VycmVudCBpbnRlcmFjdGlvbnMgd2l0aCBPcGVuIFNvdXJj
ZSBDb21tdW5pdGllcy4NClRoZSBjdXJyZW50IENoYXJ0ZXIgd2FzIGFwcHJvdmVkIGluIEp1bmUg
MjAxNCwgdGhlbiBPUE5GViB3YXMgZm91bmRlZCANCm9uIFNlcHRlbWJlciAzMCwgMjAxNCBbMF0g
YW5kIHRoZSBWU1BFUkYgUHJvamVjdCB3YXMgY3JlYXRlZCBvbg0KRGVjIDE2LCAyMDE0LCBzbyB3
ZSBkaWQgbm90IGFudGljaXBhdGUgZXh0ZW5zaXZlIGNvbGxhYm9yYXRpb24gb24NCnRoaXMgYW5k
IG90aGVyIGJlbmNobWFya2luZyB0b3BpY3MuIA0KDQoNCg0KPiANCj4gMi4gSW4gc2VjdGlvbiAz
IHRoZXJlICdyZXBlYXRhYmlsaXR5JyBpcyBtZW50aW9uZWQsIHdoaWxlDQo+IGFja25vd2xlZGdp
bmcgdGhhdCBpbiBhIHZpcnR1YWwgZW52aXJvbm1lbnQgdGhlcmUgaXMgbm8gZ3VhcmFudGVlIGFu
ZA0KPiBhY3R1YWxseSBubyB3YXkgdG8ga25vdyB3aGF0IG90aGVyIGFwcGxpY2F0aW9ucyBhcmUg
YmVpbmcgcnVuLg0KW0FDTV0gDQpTZWU6DQpodHRwczovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJh
ZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1I3NlY3Rpb24tMy40DQoNClRoZXJlIGFyZSBjZXJ0
YWlubHkgd2F5cyB0byBhc3Nlc3MgdGhlIGN1cnJlbnQgc2V0IG9mIHByb2Nlc3Nlcw0KYXQgYSBw
YXJ0aWN1bGFyIHRpbWUuIFRoZSBTb2Z0d2FyZSBjb25maWd1cmF0aW9uIHBhcmFtZXRlcnMgaW4N
ClNlY3Rpb24gMy4zIGFyZSBpbnRlbmRlZCB0byBjYXB0dXJlIHRoaXMgYXNwZWN0IGFzIHBhcnQg
b2Ygc2V0LXVwLg0KQXQgdGhlIHNhbWUgdGltZSwgdGhlcmUgd2lsbCBiZSBjaGFsbGVuZ2VzIHRv
IGFzc2VzcyB0aGUgRFVUDQpwZXJmb3JtYW5jZSB3aGVuIHJlc291cmNlcyBhcmUgZnVsbHkgc2hh
cmVkLCBhbmQgbmV3IHRlc3RpbmcNCnN0cmF0ZWdpZXMgd2lsbCBiZSBuZWVkZWQ6DQpodHRwczov
L3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1I3NlY3Rp
b24tMy4zDQoNCg0KPiBNZWFzdXJpbmcgcGFyYW1ldGVycyBhcyB0aGUgb25lcyBsaXN0ZWQgaW4g
My4zIHByb3ZpZGVzIGp1c3QgcGFydCBvZg0KPiB0aGUgYW5zd2VyLCBhbmQgdGhleSBhcmUgaW50
ZXJuYWwgcGFyYW1ldGVycyB0byB0aGUgU1VULiANCltBQ01dIA0KWWVzLCBrbm93aW5nIHRoZSB0
ZXN0ZWQgY29uZmlndXJhdGlvbiBpcyBhIGNyaXRpY2FsIHBpbGxhciANCnN1cHBvcnRpbmcgcmVw
ZWF0YWJpbGl0eSAodGhlc2UgaXRlbXMgYXJlIG5vdCBtZWFzdXJlZCwgYnV0IGNvbmZpZ3VyZWQp
LA0KYW5kIHdoeSB3ZSBwcm92aWRlZCB0aGlzIHNlY3Rpb24uDQoNCj4gQWxzbywgdGhlDQo+IGRp
ZmZlcmVudCBkZXBsb3ltZW50IHNjZW5hcmlvcyBpbiBzZWN0aW9uIDQgcmVxdWlyZSBkaWZmZXJl
bnQNCj4gY29uZmlndXJhdGlvbnMgZm9yIHRoZSBTVVQsIHRodXMgYnJlYWtpbmcgdGhlICdibGFj
ay1ib3gnIHByaW5jaXBsZS4NCltBQ01dIA0KU3BlY2lmeWluZyBEVVQgY29uZmlndXJhdGlvbiBk
b2VzIG5vdCBicmVhayBhbnkgcGFydCBvZiB0aGUgDQpibGFjay1ib3ggcHJpbmNpcGxlLCB3aGlj
aCBlc3RhYmxpc2hlcyB0aGF0IGJlbmNobWFyayBtZWFzdXJlbWVudHMNCndpbGwgYmUgYmFzZWQg
b24gZXh0ZXJuYWxseSBvYnNlcnZhYmxlIHBoZW5vbWVuYS4gU2VlOg0KaHR0cHM6Ly90b29scy5p
ZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtYm13Zy12aXJ0dWFsLW5ldC0wNSNzZWN0aW9uLTQuMg0K
DQpQcmV2aW91cyBCTVdHIFJGQ3MgaGF2ZSBpZGVudGlmaWVkIHRoZSBjcml0aWNhbCBjb25maWd1
cmF0aW9uDQpwYXJhbWV0ZXJzIG9mIHRoZSBEVVQsIHN1Y2ggYXMgdGhlIG51bWJlciBhbmQgdHlw
ZSBvZiANCm5ldHdvcmsgaW50ZXJmYWNlcywgdGhlIGFycmFuZ2VtZW50IG9mIERVVHMgaW4gYSBT
VVQsIGV0Yy4NCg0KPiBJIGJlbGlldmUgdGhhdCB0aGVyZSBpcyBhIG5lZWQgZm9yIGEgbW9yZSBj
bGVhciBleHBsYW5hdGlvbiBvZiB3aHkgQk1XRw0KPiBzcGVjaWZpY2F0aW9ucyBhcmUgYXBwcm9w
cmlhdGUgYW5kIGhvdyBjb21wYXJpc29uIGNhbiBiZSBtYWRlIHdoaWxlDQo+IHJlcGVhdGFiaWxp
dHkgY2Fubm90IGJlIGVuc3VyZWQsIGFuZCBtZWFzdXJlbWVudHMgYXJlIGRlcGVuZGVudCB1cG9u
DQo+IHBhcmFtZXRlcnMgaW50ZXJuYWwgdG8gdGhlIFNVVC4NCltBQ01dIA0KSSBiZWxpZXZlIHRo
YXQgZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1IGFscmVhZHkgDQppbmRpY2F0ZXMgd2h5
IHRoZSBleGlzdGluZyBCTVdHIFJGQ3MgYXJlIGEgcmVhc29uYWJsZSANCnN0YXJ0aW5nIHBsYWNl
IGZvciBORlYgYmVuY2htYXJrcywgaW4gcGFydCBiZWNhdXNlIA0Kd2Ugd2FudCB0byBtZWFzdXJl
IHRoZSBzYW1lIGJlbmNobWFya3Mgb2YgcGh5c2ljYWwgDQpuZXR3b3JrIGZ1bmN0aW9ucyBpbiBt
YW55IGNhc2VzLiBTZWUNCmh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1pZXRmLWJt
d2ctdmlydHVhbC1uZXQtMDUjc2VjdGlvbi00LjENCg0KUmVwZWF0YWJpbGl0eSBpcyBhIGdvYWwg
b2YgYWxsIGV4cGVyaW1lbnRzLCBhbmQgd2UgdW5kZXJzdGFuZA0KdGhhdCB0aGVyZSBpcyBtb3Jl
IHdvcmsgdG8gZG8gaW4gdGhpcyByZWdhcmQsIGJ1dCB3aGF0DQp3ZSBrbm93IG5vdyAoZG9jdW1l
bnRlZCBpbiB0aGlzIGRyYWZ0KSBzaG91bGQNCmJlIGEgdmFsdWFibGUgY29udHJpYnV0aW9uIHRv
IHRoZSBJbmR1c3RyeS4NCg0KDQo+IA0KPiBNaW5vciBpc3N1ZXM6DQo+IA0KPiAxLiBTb21lIG9m
IHRoZSB0ZXN0cyBtZW50aW9uZWQgaW4gU2VjdGlvbiA0IGhhdmUgbm8gcHJpb3Igb3IgaW4NCj4g
cHJvZ3Jlc3Mgd29yayBpbiB0aGUgSUVURjogQ29udHJvbCBQYXRoIGFuZCBEYXRhcGF0aCBDb3Vw
bGluZyBUZXN0cywNCj4gTm9pc3kgTmVpZ2hib3VyIFRlc3RzLCBjaGFyYWN0ZXJpemF0aW9uIG9m
IGFjY2VsZXJhdGlvbiB0ZWNobm9sb2dpZXMuDQpbQUNNXSANCkknbSBzb3JyeSwgYnV0IHRoYXQn
cyBub3QgYW4gYWNjdXJhdGUgcG9ydHJheWFsIG9mIEJNV0cncyBsaXRlcmF0dXJlLg0KDQpodHRw
czovL3Rvb2xzLmlldGYub3JnL2h0bWwvcmZjNjQxMyBleGFtaW5lZCBDb250cm9sIFBsYW5lL0Rh
dGFwbGFuZSANCmludGVyYWN0aW9ucywgZm9yIGV4YW1wbGUuDQoNCmh0dHBzOi8vdG9vbHMuaWV0
Zi5vcmcvaHRtbC9kcmFmdC1pZXRmLWJtd2ctdmlydHVhbC1uZXQtMDUjc2VjdGlvbi0zLjMNCml0
ZW0gMiBzcGVjaWZpY2FsbHkgaW5jbHVkZWQgTm9pc3kgTmVpZ2hib3VyIGFtb25nIHRoZSBuZXcg
DQp0ZXN0aW5nIHN0cmF0ZWdpZXMuDQoNCkV2ZXJ5IG5ldHdvcmsgaW50ZXJmYWNlIHdpdGggYW4g
QVNJQyBpcyBhbiBleGFtcGxlIG9mIGFjY2VsZXJhdGlvbiwNCm9uZSB0aGF0IHdlJ3ZlIGNoYXJh
Y3Rlcml6ZWQgaW4gcGh5c2ljYWwgbmV0d29yayBkZXZpY2VzIGZvciB5ZWFycy4NCg0KPiBJZiBu
ZXcgd29yayBpcyBuZWVkZWQgLyBwcm9wb3NlZCB0byBiZSBhZGRlZCBmb3IgdGhlIEJNV0cgc2Nv
cGUgYW5kDQo+IGZyYW1ld29yayBpdCB3b3VsZCBiZSB1c2VmdWwgZm9yIEJNV0cgdG8gbGlzdCB0
aGVzZSBzZXBhcmF0ZWx5Lg0KPiANCj4gDQo+IE5pdHMvZWRpdG9yaWFsIGNvbW1lbnRzOg0KPiAN
Cj4gMS4gV2hhdCBpcyBjYWxsZWQgJ0RlcGxveW1lbnQgc2NlbmFyaW9zJyBmcm9tIFZTIHBlcnNw
ZWN0aXZlIGluDQo+IFNlY3Rpb24gNCBkZXNjcmliZSBpbiBmYWN0IGRpZmZlcmVudCBjb25maWd1
cmF0aW9ucyBvZiB0aGUgU1VUIGluIEJNV0cNCj4gdGVybXMuIEl0IHNlZW1zIGJldHRlciB0byBz
ZXBhcmF0ZSB0aGlzIHNlY29uZCBwYXJ0IG9mIHNlY3Rpb24gNCBpbiBhDQo+IHNlcGFyYXRlIHNl
Y3Rpb24uIElmIGl0IGJlbG9uZ3MgdG8gYW4gZXhpc3Rpbmcgc2VjdGlvbiBpdCByYXRoZXINCj4g
YmVsb25ncyBpbiAzIHRoYW4gaW4gNC4NCj4gDQpbQUNNXSANClNlY3Rpb24gMyBpcyBtb3JlIGFi
b3V0IGV4dGVuZGluZyB0aGUgY29uZmlndXJhdGlvbiBndWlkYW5jZQ0KZnJvbSBodHRwczovL3Rv
b2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1I3NlY3Rpb24t
My4yDQoNClNlY3Rpb24gNCBzdW1tYXJpemVzIHRoZSBWU1BFUkYgTGV2ZWwgVGVzdCBEZXNpZ24g
ZG9jdW1lbnQsDQpvZiB3aGljaCB0aGVzZSBkZXBsb3ltZW50IHNjZW5hcmlvcyBhcmUgYSBrZXkg
cGFydC4NCg0KdGhhbmtzIGZvciB5b3VyIGNvbW1lbnRzOyBob3BlZnVsbHkgdGhpcyBkZXRhaWxl
ZCByZXBseQ0Kd2lsbCByZWR1Y2UgeW91ciByZXNlcnZhdGlvbnMgYWJvdXQgcHVibGljYXRpb24u
DQoNCkFsDQooZm9yIHRoZSBjby1hdXRob3JzKQ0KDQpbMF0gaHR0cHM6Ly93d3cub3BuZnYub3Jn
L2Fubm91bmNlbWVudHMvMjAxNC8wOS8zMC90ZWxlY29tLWluZHVzdHJ5LWFuZC12ZW5kb3JzLXVu
aXRlLXRvLWJ1aWxkLWNvbW1vbi1vcGVuLXBsYXRmb3JtLXRvLWFjY2VsZXJhdGUtbmV0d29yay1m
dW5jdGlvbnMtdmlydHVhbGl6YXRpb24NCg0K


From nobody Thu May 11 10:07:32 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C6B012EC7D; Thu, 11 May 2017 10:07:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gy4xGUQsdAja; Thu, 11 May 2017 10:07:06 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C9DC412EB94; Thu, 11 May 2017 10:01:24 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net [127.0.0.1]) by m0048589.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v4BGu7H6005916; Thu, 11 May 2017 13:01:22 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0048589.ppops.net-00191d01. with ESMTP id 2act58m503-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 May 2017 13:01:21 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v4BH1EXJ025187; Thu, 11 May 2017 13:01:20 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v4BH0rmS022964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 11 May 2017 13:01:11 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor); Thu, 11 May 2017 17:00:37 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4BH0ZSM007719; Thu, 11 May 2017 12:00:37 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4BH0TN3007110; Thu, 11 May 2017 12:00:29 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id B82A8F0486; Thu, 11 May 2017 13:00:28 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 11 May 2017 13:00:28 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Dan Romascanu <dromasca@gmail.com>, "ops-dir@ietf.org" <ops-dir@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" <draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>, "ietf@ietf.org" <ietf@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Opsdir last call review of draft-ietf-bmwg-vswitch-opnfv-03
Thread-Index: AQHSykdUS/fJfaxnyEyPSoOu45aoKqHvWyeQ
Date: Thu, 11 May 2017 17:00:27 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F9B079@njmtexg4.research.att.com>
References: <149450107581.16728.18142473358782662679@ietfa.amsl.com>
In-Reply-To: <149450107581.16728.18142473358782662679@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.70.214.90]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-11_13:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705110087
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/LbkKfxyMv8q14d_f229BNZhEjpo>
Subject: Re: [bmwg] Opsdir last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 May 2017 17:07:09 -0000

RGFuLCBwcm92aWRpbmcgeW91ciBHZW4tQVJUIGNvbW1lbnRzIGFuZCByZXBseSBiZWxvdy4NCg0K
PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBEYW4gUm9tYXNjYW51IFttYWls
dG86ZHJvbWFzY2FAZ21haWwuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgTWF5IDExLCAyMDE3IDc6
MTEgQU0NCj4gVG86IG9wcy1kaXJAaWV0Zi5vcmcNCj4gQ2M6IGRyYWZ0LWlldGYtYm13Zy12c3dp
dGNoLW9wbmZ2LmFsbEBpZXRmLm9yZzsgaWV0ZkBpZXRmLm9yZzsNCj4gYm13Z0BpZXRmLm9yZzsg
ZHJvbWFzY2FAZ21haWwuY29tDQo+IFN1YmplY3Q6IE9wc2RpciBsYXN0IGNhbGwgcmV2aWV3IG9m
IGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LTAzDQo+IA0KPiBSZXZpZXdlcjogRGFuIFJv
bWFzY2FudQ0KPiBSZXZpZXcgcmVzdWx0OiBIYXMgSXNzdWVzDQo+IA0KPiBUaGlzIGRvY3VtZW50
IGRlc2NyaWJlcyBkZXNjcmliZXMgdGhlIHByb2dyZXNzIG9mIHRoZSBPcGVuIFBsYXRmb3JtDQo+
IGZvciBORlYgKE9QTkZWKSBwcm9qZWN0IG9uIHZpcnR1YWwgc3dpdGNoIHBlcmZvcm1hbmNlICJW
U1BFUkYiLiBUaGF0DQo+IHByb2plY3QgcmV1c2VzIHRoZSBCTVdHIGZyYW1ld29yayBhbmQgc3Bl
Y2lmaWNhdGlvbnMgdG8gYmVuY2htYXJrDQo+IHZpcnR1YWwgc3dpdGNoZXMgaW1wbGVtZW50ZWQg
aW4gZ2VuZXJhbC1wdXJwb3NlIGhhcmR3YXJlLiBTb21lDQo+IGRpZmZlcmVuY2VzIHdpdGggdGhl
IGJlbmNobWFya2luZyBvZiBzcGVjaWFsaXplZCBIVyBwbGF0Zm9ybXMgYXJlDQo+IGlkZW50aWZp
ZWQgYW5kIHRoZXkgbWF5IGJlY29tZSB3b3JrIGl0ZW1zIGZvciBCTVdHIGluIHRoZSBmdXR1cmUu
IEl0J3MNCj4gYSB3ZWxsIHdyaXR0ZW4gYW5kIGNsZWFyIGRvY3VtZW50LCBidXQgSSBoYXZlIHJl
c2VydmF0aW9ucyBhYm91dCBpdA0KPiBiZWluZyBwdWJsaXNoZWQgYXMgYW4gUkZDLCBhbmQgSSBj
YW5ub3QgZmluZCBjb3ZlcmFnZSBmb3IgaXQgaW4gdGhlIFdHDQo+IGNoYXJ0ZXIuIEkgYWxzbyBo
YXZlIGNvbmNlcm5zIHRoYXQgcGFydHMgb2YgdGhlIG1ldGhvZG9sb2d5IHVzZWQgYnkNCj4gT1BO
RlYgYnJlYWsgdGhlIEJNV0cgcHJpbmNpcGxlcywgZXNwZWNpYWxseSByZXBlYXRhYmlsaXR5IGFu
ZA0KPiAnYmxhY2stYm94JywgYW5kIHRoaXMgaXMgbm90IGNsZWFyIGVub3VnaCBhcnRpY3VsYXRl
ZCBpbiB0aGUgZG9jdW1lbnQuDQo+ICBBcyBJIHdhcyBhc3NpZ25lZCBib3RoIE9QUy1ESVIgYW5k
IEdlbi1BUlQgcmV2aWV3cyBmb3IgdGhpcyBkb2N1bWVudCwNCj4gSSBkZXRhaWxlZCB0aGUgY29u
Y2VybnMgaW4gbXkgR2VuLUFSVCByZXZpZXcsIHRoZXkgc2VlbSB0byBiZWxvbmcNCj4gYmV0dGVy
IHRoZXJlLg0KPiANCj4gPkZyb20gYW4gT1BTLURJUiBwZXJzcGVjdGl2ZSB0aGlzIGRvY3VtZW50
IGhhcyBubyBpc3N1ZXMuIElmIHRoZQ0KPiBjb25jZXJucyBpbiBzZWN0aW9uIDYgYXJlIGFkZHJl
c3NlZCBhbmQgY2F1dGlvbiBpcyB0YWtlbiB0byBpc29sYXRlDQo+IHRoZSBTVVRzIGFuZCBiZW5j
aG1hcmtpbmcgZW52aXJvbm1lbnRzIGZyb20gdGhlIEludGVybmV0IG9yDQo+IG9wZXJhdGlvbmFs
IGludHJhbmV0cywgdGhlcmUgaXMgbm8gb3BlcmF0aW9uYWwgaW1wYWN0LiBJZiBhcHByb3ZlZCBp
dA0KPiB3b3VsZCBiZSBhIHVzZWZ1bCB0b29sIGZvciBvcGVyYXRvcnMgdG8gZ2V0IHNvbWUgaW5m
b3JtYXRpb24gYWJvdXQgaG93DQo+IGJlbmNobWFya2luZyBvZiBPUE5GViBwcm9qZWN0IHByb2R1
Y3RzIGFyZSBiZWluZyBkZXNpZ25lZC4NCj4gDQo+IA0KLT0tPS09LT0tPS09LT0tPS09LT0tPS09
LQ0KIA0KSGkgRGFuLA0KcGxlYXNlIHNlZSByZXBsaWVzLCBbQUNNXSwgYmVsb3cuDQoNCj4gLS0t
LS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogRGFuIFJvbWFzY2FudSBbbWFpbHRvOmRy
b21hc2NhQGdtYWlsLmNvbV0NCj4gU2VudDogVGh1cnNkYXksIE1heSAxMSwgMjAxNyA3OjA2IEFN
DQo+IFRvOiBnZW4tYXJ0QGlldGYub3JnDQo+IENjOiBkcmFmdC1pZXRmLWJtd2ctdnN3aXRjaC1v
cG5mdi5hbGxAaWV0Zi5vcmc7IGlldGZAaWV0Zi5vcmc7DQo+IGJtd2dAaWV0Zi5vcmc7IGRyb21h
c2NhQGdtYWlsLmNvbQ0KPiBTdWJqZWN0OiBHZW5hcnQgbGFzdCBjYWxsIHJldmlldyBvZiBkcmFm
dC1pZXRmLWJtd2ctdnN3aXRjaC1vcG5mdi0wMw0KPiANCj4gUmV2aWV3ZXI6IERhbiBSb21hc2Nh
bnUNCj4gUmV2aWV3IHJlc3VsdDogQWxtb3N0IFJlYWR5DQo+IA0KPiBJIGFtIHRoZSBhc3NpZ25l
ZCBHZW4tQVJUIHJldmlld2VyIGZvciB0aGlzIGRyYWZ0LiBUaGUgR2VuZXJhbCBBcmVhDQo+IFJl
dmlldyBUZWFtIChHZW4tQVJUKSByZXZpZXdzIGFsbCBJRVRGIGRvY3VtZW50cyBiZWluZyBwcm9j
ZXNzZWQNCj4gYnkgdGhlIElFU0cgZm9yIHRoZSBJRVRGIENoYWlyLiAgUGxlYXNlIHRyZWF0IHRo
ZXNlIGNvbW1lbnRzIGp1c3QNCj4gbGlrZSBhbnkgb3RoZXIgbGFzdCBjYWxsIGNvbW1lbnRzLg0K
PiANCj4gRm9yIG1vcmUgaW5mb3JtYXRpb24sIHBsZWFzZSBzZWUgdGhlIEZBUSBhdA0KPiANCj4g
PGh0dHBzOi8vdXJsZGVmZW5zZS5wcm9vZnBvaW50LmNvbS92Mi91cmw/dT1odHRwcy0NCj4gM0Ff
X3RyYWMuaWV0Zi5vcmdfdHJhY19nZW5fd2lraV9HZW5BcnRmYXEmZD1Ed0lDYVEmYz1MRllaLQ0K
PiBvOV9IVU1lTVRTUWljdmpJZyZyPU9mc1N1OGtUSWx0VnlEMW9MNzJjQncmbT1OVFZsTEJOLQ0K
PiBMM3U5ekdQSG1fQ05WY1hXN19PR1g4XzE4Q3RhQWFsWmluMCZzPTJIci0NCj4gZGhLYURISWd1
WTdXOTd6MzNSbEtqcVBEdG1vWW1NMi1qV3JiUy1vJmU9ID4uDQo+IA0KPiBEb2N1bWVudDogZHJh
ZnQtaWV0Zi1ibXdnLXZzd2l0Y2gtb3BuZnYtMDMNCj4gUmV2aWV3ZXI6IERhbiBSb21hc2NhbnUN
Cj4gUmV2aWV3IERhdGU6IDIwMTctMDUtMTENCj4gSUVURiBMQyBFbmQgRGF0ZTogMjAxNy0wNS0x
NQ0KPiBJRVNHIFRlbGVjaGF0IGRhdGU6IE5vdCBzY2hlZHVsZWQgZm9yIGEgdGVsZWNoYXQNCj4g
DQo+IFN1bW1hcnk6DQo+IA0KPiBBbG1vc3QgUmVhZHkuDQo+IA0KPiBUaGlzIGRvY3VtZW50IGRl
c2NyaWJlcyBkZXNjcmliZXMgdGhlIHByb2dyZXNzIG9mIHRoZSBPcGVuIFBsYXRmb3JtDQo+IGZv
ciBORlYgKE9QTkZWKSBwcm9qZWN0IG9uIHZpcnR1YWwgc3dpdGNoIHBlcmZvcm1hbmNlICJWU1BF
UkYiLiBUaGF0DQo+IHByb2plY3QgcmV1c2VzIHRoZSBCTVdHIGZyYW1ld29yayBhbmQgc3BlY2lm
aWNhdGlvbnMgdG8gYmVuY2htYXJrDQo+IHZpcnR1YWwgc3dpdGNoZXMgaW1wbGVtZW50ZWQgaW4g
Z2VuZXJhbC1wdXJwb3NlIGhhcmR3YXJlLiBTb21lDQo+IGRpZmZlcmVuY2VzIHdpdGggdGhlIGJl
bmNobWFya2luZyBvZiBzcGVjaWFsaXplZCBIVyBwbGF0Zm9ybXMgYXJlDQo+IGlkZW50aWZpZWQg
YW5kIHRoZXkgbWF5IGJlY29tZSB3b3JrIGl0ZW1zIGZvciBCTVdHIGluIHRoZSBmdXR1cmUuIEl0
J3MNCj4gYSB3ZWxsIHdyaXR0ZW4gYW5kIGNsZWFyIGRvY3VtZW50LCBidXQgSSBoYXZlIHJlc2Vy
dmF0aW9ucyBhYm91dCBpdA0KPiBiZWluZyBwdWJsaXNoZWQgYXMgYW4gUkZDLCBhbmQgSSBjYW5u
b3QgZmluZCBjb3ZlcmFnZSBmb3IgaXQgaW4gdGhlIFdHDQo+IGNoYXJ0ZXIuIEkgYWxzbyBoYXZl
IGNvbmNlcm5zIHRoYXQgcGFydHMgb2YgdGhlIG1ldGhvZG9sb2d5IHVzZWQgYnkNCj4gT1BORlYg
YnJlYWsgdGhlIEJNV0cgcHJpbmNpcGxlcywgZXNwZWNpYWxseSByZXBlYXRhYmlsaXR5IGFuZA0K
PiAnYmxhY2stYm94JywgYW5kIHRoaXMgaXMgbm90IGNsZWFyIGVub3VnaCBhcnRpY3VsYXRlZCBp
biB0aGUgZG9jdW1lbnQuDQpbQUNNXSANCk9rLCBsZXQncyBhZGRyZXNzIHlvdXIgc3BlY2lmaWMg
aXNzdWVzLCBhbmQgY29tZSBiYWNrIHRvIHlvdXIgcmVzZXJ2YXRpb25zLg0KDQo+IA0KPiANCj4g
TWFqb3IgaXNzdWVzOg0KPiANCj4gMS4gSXQgaXMgbm90IGNsZWFyIHRvIG1lIHdoeSB0aGlzIGRv
Y3VtZW50IG5lZWRzIHRvIGJlIHB1Ymxpc2hlZCBhcyBhbg0KPiBSRkMuIFRoZSBpbnRyb2R1Y3Rp
b24gc2F5czogJ1RoaXMgbWVtbyBkZXNjcmliZXMgdGhlIHByb2dyZXNzIG9mIHRoZQ0KPiBPcGVu
IFBsYXRmb3JtIGZvciBORlYgKE9QTkZWKSBwcm9qZWN0IG9uIHZpcnR1YWwgc3dpdGNoIHBlcmZv
cm1hbmNlDQo+ICJWU1BFUkYiLiAgVGhpcyBwcm9qZWN0IGludGVuZHMgdG8gYnVpbGQgb24gdGhl
IGN1cnJlbnQgYW5kIGNvbXBsZXRlZA0KPiB3b3JrIG9mIHRoZSBCZW5jaG1hcmtpbmcgTWV0aG9k
b2xvZ3kgV29ya2luZyBHcm91cCBpbiBJRVRGLCBieQ0KPiByZWZlcmVuY2luZyBleGlzdGluZyBs
aXRlcmF0dXJlLicgV2h5IHNob3VsZCB0aGUgV0cgYW5kIHRoZSBJRVNHDQo+IGludmVzdCByZXNv
dXJjZXMgaW4gcHVibGlzaGluZyB0aGlzLCB3aHkgYW4gSS1EIG9yIGFuIEluZGVwZW5kZW50DQo+
IFN0cmVhbSBSRkMgaXMgbm90IHN1ZmZpY2llbnQ/IA0KW0FDTV0gDQpUaGUgV0cgY29uc2lkZXJl
ZCBhbmQgZGlzY3Vzc2VkIHRoaXMgZG9jdW1lbnQgb3ZlciAzIHJldmlzaW9ucw0KYW5kIGEgeWVh
ciBvZiB0aW1lIGJlZm9yZSByZWFjaGluZyBjb25zZW5zdXMgdG8gZGV2ZWxvcCBpdCBmdXJ0aGVy
DQphcyBhIGNoYXJ0ZXJlZCBpdGVtLCBzbyB0aGlzIGRlY2lzaW9uIHdhcyBub3QgdGFrZW4gbGln
aHRseS4NClNlZSBtb3JlIGJlbG93Lg0KDQo+IFRoZSBXRyBjaGFydGVyIHNheXMgc29tZXRoaW5n
IGFib3V0Og0KPiAnVk5GIGFuZCBSZWxhdGVkIEluZnJhc3RydWN0dXJlIEJlbmNobWFya2luZzog
QmVuY2htYXJraW5nDQo+IE1ldGhvZG9sb2dpZXMgaGF2ZSByZWxpYWJseSBjaGFyYWN0ZXJpemVk
IG1hbnkgcGh5c2ljYWwgZGV2aWNlcy4gVGhpcw0KPiB3b3JrIGl0ZW0gZXh0ZW5kcyBhbmQgZW5o
YW5jZXMgdGhlIG1ldGhvZHMgdG8gdmlydHVhbCBuZXR3b3JrDQo+IGZ1bmN0aW9ucyAoVk5GKSBh
bmQgdGhlaXIgdW5pcXVlIHN1cHBvcnRpbmcgaW5mcmFzdHJ1Y3R1cmUuIEEgZmlyc3QNCj4gZGVs
aXZlcmFibGUgZnJvbSB0aGlzIGFjdGl2aXR5IHdpbGwgYmUgYSBkb2N1bWVudCB0aGF0IGNvbnNp
ZGVycyB0aGUNCj4gbmV3IGJlbmNobWFya2luZyBzcGFjZSB0byBlbnN1cmUgdGhhdCBjb21tb24g
aXNzdWVzIGFyZSByZWNvZ25pemVkDQo+IGZyb20gdGhlIHN0YXJ0LCB1c2luZyBiYWNrZ3JvdW5k
IG1hdGVyaWFscyBmcm9tIGluZHVzdHJ5IGFuZCBTRE9zDQo+IChlLmcuLCBJRVRGLCBFVFNJIE5G
VikuJy4gSSBkbyBub3QgYmVsaWV2ZSB0aGF0IHRoaXMgZG9jdW1lbnQgY292ZXJzDQo+IHRoZSBp
bnRlbnQgb2YgdGhlIGNoYXJ0ZXIsIGFzIGl0IGZvY3VzZWQgb24gb25lIG9yZ2FuaXphdGlvbiBv
bmx5Lg0KW0FDTV0gDQpJJ20gc29ycnksIGJ1dCBoZXJlIHlvdSBhcmUgbWlzdGFrZW4uIFRoZSBk
b2N1bWVudCB0aGF0IHNhdGlzZmllZA0KdGhlICJmaXJzdCBkZWxpdmVyYWJsZSAuLi4gZG9jdW1l
bnQgdGhhdCBjb25zaWRlcnMgdGhlIG5ldyBiZW5jaG1hcmtpbmcgc3BhY2UiDQppczogaHR0cHM6
Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtYm13Zy12aXJ0dWFsLW5ldC0wNQ0KdGl0
bGVkOiBDb25zaWRlcmF0aW9ucyBmb3IgQmVuY2htYXJraW5nIFZpcnR1YWwgTmV0d29yayBGdW5j
dGlvbnMgYW5kIFRoZWlyIEluZnJhc3RydWN0dXJlDQp3aGljaCBoYXMgYmVlbiBzdWJtaXR0ZWQg
dG8gSUVTRyBhbmQgYXBwcm92ZWQgZm9yIHB1YmxpY2F0aW9uLg0KRnVydGhlciwgdGhlIGN1cnJl
bnQgZHJhZnQgKGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2LTAzKQ0KcmVmZXJlbmNlcyB0
aGUgYXBwcm92ZWQgIkNvbnNpZGVyYXRpb25zIiBkcmFmdCBpbiBTZWN0aW9uIDMNCihhcyBkb2Vz
IGFsbW9zdCBldmVyeSByZWxhdGVkIEluZHVzdHJ5IHNwZWMgSSdtIGF3YXJlIG9mKS4NCg0KVGhl
IEJNV0cgQ2hhcnRlciBjb250aW51ZXM6DQogIEJlbmNobWFya3MgZm9yIHBsYXRmb3JtIGNhcGFj
aXR5IGFuZCBwZXJmb3JtYW5jZSBjaGFyYWN0ZXJpc3RpY3Mgb2YNCiAgdmlydHVhbCByb3V0ZXJz
LCBzd2l0Y2hlcywgYW5kIHJlbGF0ZWQgY29tcG9uZW50cyB3aWxsIGZvbGxvdywgaW5jbHVkaW5n
DQogIGNvbXBhcmlzb25zIGJldHdlZW4gcGh5c2ljYWwgYW5kIHZpcnR1YWwgbmV0d29yayBmdW5j
dGlvbnMuIEluIG1hbnkgY2FzZXMsDQogIHRoZSB0cmFkaXRpb25hbCBiZW5jaG1hcmtzIHNob3Vs
ZCBiZSBhcHBsaWNhYmxlIHRvIFZORnMsIGJ1dCB0aGUgbGFiDQogIHNldC11cHMsIGNvbmZpZ3Vy
YXRpb25zLCBhbmQgbWVhc3VyZW1lbnQgbWV0aG9kcyB3aWxsIGxpa2VseSBuZWVkIHRvDQogIGJl
IHJldmlzZWQgb3IgZW5oYW5jZWQuDQoNClRoaXMgZHJhZnQgY29uc3RpdHV0ZXMgb25lIG9mIHNl
dmVyYWwgZm9sbG93LW9uIGVmZm9ydHMsIGFwcHJvYWNoaW5nIA0KdGhlIHByb2JsZW0gZXhhY3Rs
eSBhcyB3ZSBkZXNjcmliZWQgaW4gdGhlIGxhc3Qgc2VudGVuY2UgYWJvdmUuDQoNCkFuIGFzcGVj
dCBvZiBJbmR1c3RyeSBjb2xsYWJvcmF0aW9uIHRoYXQgd2UgZGlkIG5vdCBhbnRpY2lwYXRlIGlu
IHRoZSANCkJNV0cgQ2hhcnRlciBpcyBvdXIgY3VycmVudCBpbnRlcmFjdGlvbnMgd2l0aCBPcGVu
IFNvdXJjZSBDb21tdW5pdGllcy4NClRoZSBjdXJyZW50IENoYXJ0ZXIgd2FzIGFwcHJvdmVkIGlu
IEp1bmUgMjAxNCwgdGhlbiBPUE5GViB3YXMgZm91bmRlZCANCm9uIFNlcHRlbWJlciAzMCwgMjAx
NCBbMF0gYW5kIHRoZSBWU1BFUkYgUHJvamVjdCB3YXMgY3JlYXRlZCBvbg0KRGVjIDE2LCAyMDE0
LCBzbyB3ZSBkaWQgbm90IGFudGljaXBhdGUgZXh0ZW5zaXZlIGNvbGxhYm9yYXRpb24gb24NCnRo
aXMgYW5kIG90aGVyIGJlbmNobWFya2luZyB0b3BpY3MuIA0KDQoNCg0KPiANCj4gMi4gSW4gc2Vj
dGlvbiAzIHRoZXJlICdyZXBlYXRhYmlsaXR5JyBpcyBtZW50aW9uZWQsIHdoaWxlDQo+IGFja25v
d2xlZGdpbmcgdGhhdCBpbiBhIHZpcnR1YWwgZW52aXJvbm1lbnQgdGhlcmUgaXMgbm8gZ3VhcmFu
dGVlIGFuZA0KPiBhY3R1YWxseSBubyB3YXkgdG8ga25vdyB3aGF0IG90aGVyIGFwcGxpY2F0aW9u
cyBhcmUgYmVpbmcgcnVuLg0KW0FDTV0gDQpTZWU6DQpodHRwczovL3Rvb2xzLmlldGYub3JnL2h0
bWwvZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1I3NlY3Rpb24tMy40DQoNClRoZXJlIGFy
ZSBjZXJ0YWlubHkgd2F5cyB0byBhc3Nlc3MgdGhlIGN1cnJlbnQgc2V0IG9mIHByb2Nlc3Nlcw0K
YXQgYSBwYXJ0aWN1bGFyIHRpbWUuIFRoZSBTb2Z0d2FyZSBjb25maWd1cmF0aW9uIHBhcmFtZXRl
cnMgaW4NClNlY3Rpb24gMy4zIGFyZSBpbnRlbmRlZCB0byBjYXB0dXJlIHRoaXMgYXNwZWN0IGFz
IHBhcnQgb2Ygc2V0LXVwLg0KQXQgdGhlIHNhbWUgdGltZSwgdGhlcmUgd2lsbCBiZSBjaGFsbGVu
Z2VzIHRvIGFzc2VzcyB0aGUgRFVUDQpwZXJmb3JtYW5jZSB3aGVuIHJlc291cmNlcyBhcmUgZnVs
bHkgc2hhcmVkLCBhbmQgbmV3IHRlc3RpbmcNCnN0cmF0ZWdpZXMgd2lsbCBiZSBuZWVkZWQ6DQpo
dHRwczovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1
I3NlY3Rpb24tMy4zDQoNCg0KPiBNZWFzdXJpbmcgcGFyYW1ldGVycyBhcyB0aGUgb25lcyBsaXN0
ZWQgaW4gMy4zIHByb3ZpZGVzIGp1c3QgcGFydCBvZg0KPiB0aGUgYW5zd2VyLCBhbmQgdGhleSBh
cmUgaW50ZXJuYWwgcGFyYW1ldGVycyB0byB0aGUgU1VULiANCltBQ01dIA0KWWVzLCBrbm93aW5n
IHRoZSB0ZXN0ZWQgY29uZmlndXJhdGlvbiBpcyBhIGNyaXRpY2FsIHBpbGxhciANCnN1cHBvcnRp
bmcgcmVwZWF0YWJpbGl0eSAodGhlc2UgaXRlbXMgYXJlIG5vdCBtZWFzdXJlZCwgYnV0IGNvbmZp
Z3VyZWQpLA0KYW5kIHdoeSB3ZSBwcm92aWRlZCB0aGlzIHNlY3Rpb24uDQoNCj4gQWxzbywgdGhl
DQo+IGRpZmZlcmVudCBkZXBsb3ltZW50IHNjZW5hcmlvcyBpbiBzZWN0aW9uIDQgcmVxdWlyZSBk
aWZmZXJlbnQNCj4gY29uZmlndXJhdGlvbnMgZm9yIHRoZSBTVVQsIHRodXMgYnJlYWtpbmcgdGhl
ICdibGFjay1ib3gnIHByaW5jaXBsZS4NCltBQ01dIA0KU3BlY2lmeWluZyBEVVQgY29uZmlndXJh
dGlvbiBkb2VzIG5vdCBicmVhayBhbnkgcGFydCBvZiB0aGUgDQpibGFjay1ib3ggcHJpbmNpcGxl
LCB3aGljaCBlc3RhYmxpc2hlcyB0aGF0IGJlbmNobWFyayBtZWFzdXJlbWVudHMNCndpbGwgYmUg
YmFzZWQgb24gZXh0ZXJuYWxseSBvYnNlcnZhYmxlIHBoZW5vbWVuYS4gU2VlOg0KaHR0cHM6Ly90
b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtYm13Zy12aXJ0dWFsLW5ldC0wNSNzZWN0aW9u
LTQuMg0KDQpQcmV2aW91cyBCTVdHIFJGQ3MgaGF2ZSBpZGVudGlmaWVkIHRoZSBjcml0aWNhbCBj
b25maWd1cmF0aW9uDQpwYXJhbWV0ZXJzIG9mIHRoZSBEVVQsIHN1Y2ggYXMgdGhlIG51bWJlciBh
bmQgdHlwZSBvZiANCm5ldHdvcmsgaW50ZXJmYWNlcywgdGhlIGFycmFuZ2VtZW50IG9mIERVVHMg
aW4gYSBTVVQsIGV0Yy4NCg0KPiBJIGJlbGlldmUgdGhhdCB0aGVyZSBpcyBhIG5lZWQgZm9yIGEg
bW9yZSBjbGVhciBleHBsYW5hdGlvbiBvZiB3aHkgQk1XRw0KPiBzcGVjaWZpY2F0aW9ucyBhcmUg
YXBwcm9wcmlhdGUgYW5kIGhvdyBjb21wYXJpc29uIGNhbiBiZSBtYWRlIHdoaWxlDQo+IHJlcGVh
dGFiaWxpdHkgY2Fubm90IGJlIGVuc3VyZWQsIGFuZCBtZWFzdXJlbWVudHMgYXJlIGRlcGVuZGVu
dCB1cG9uDQo+IHBhcmFtZXRlcnMgaW50ZXJuYWwgdG8gdGhlIFNVVC4NCltBQ01dIA0KSSBiZWxp
ZXZlIHRoYXQgZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1IGFscmVhZHkgDQppbmRpY2F0
ZXMgd2h5IHRoZSBleGlzdGluZyBCTVdHIFJGQ3MgYXJlIGEgcmVhc29uYWJsZSANCnN0YXJ0aW5n
IHBsYWNlIGZvciBORlYgYmVuY2htYXJrcywgaW4gcGFydCBiZWNhdXNlIA0Kd2Ugd2FudCB0byBt
ZWFzdXJlIHRoZSBzYW1lIGJlbmNobWFya3Mgb2YgcGh5c2ljYWwgDQpuZXR3b3JrIGZ1bmN0aW9u
cyBpbiBtYW55IGNhc2VzLiBTZWUNCmh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1p
ZXRmLWJtd2ctdmlydHVhbC1uZXQtMDUjc2VjdGlvbi00LjENCg0KUmVwZWF0YWJpbGl0eSBpcyBh
IGdvYWwgb2YgYWxsIGV4cGVyaW1lbnRzLCBhbmQgd2UgdW5kZXJzdGFuZA0KdGhhdCB0aGVyZSBp
cyBtb3JlIHdvcmsgdG8gZG8gaW4gdGhpcyByZWdhcmQsIGJ1dCB3aGF0DQp3ZSBrbm93IG5vdyAo
ZG9jdW1lbnRlZCBpbiB0aGlzIGRyYWZ0KSBzaG91bGQNCmJlIGEgdmFsdWFibGUgY29udHJpYnV0
aW9uIHRvIHRoZSBJbmR1c3RyeS4NCg0KDQo+IA0KPiBNaW5vciBpc3N1ZXM6DQo+IA0KPiAxLiBT
b21lIG9mIHRoZSB0ZXN0cyBtZW50aW9uZWQgaW4gU2VjdGlvbiA0IGhhdmUgbm8gcHJpb3Igb3Ig
aW4NCj4gcHJvZ3Jlc3Mgd29yayBpbiB0aGUgSUVURjogQ29udHJvbCBQYXRoIGFuZCBEYXRhcGF0
aCBDb3VwbGluZyBUZXN0cywNCj4gTm9pc3kgTmVpZ2hib3VyIFRlc3RzLCBjaGFyYWN0ZXJpemF0
aW9uIG9mIGFjY2VsZXJhdGlvbiB0ZWNobm9sb2dpZXMuDQpbQUNNXSANCkknbSBzb3JyeSwgYnV0
IHRoYXQncyBub3QgYW4gYWNjdXJhdGUgcG9ydHJheWFsIG9mIEJNV0cncyBsaXRlcmF0dXJlLg0K
DQpodHRwczovL3Rvb2xzLmlldGYub3JnL2h0bWwvcmZjNjQxMyBleGFtaW5lZCBDb250cm9sIFBs
YW5lL0RhdGFwbGFuZSANCmludGVyYWN0aW9ucywgZm9yIGV4YW1wbGUuDQoNCmh0dHBzOi8vdG9v
bHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1pZXRmLWJtd2ctdmlydHVhbC1uZXQtMDUjc2VjdGlvbi0z
LjMNCml0ZW0gMiBzcGVjaWZpY2FsbHkgaW5jbHVkZWQgTm9pc3kgTmVpZ2hib3VyIGFtb25nIHRo
ZSBuZXcgDQp0ZXN0aW5nIHN0cmF0ZWdpZXMuDQoNCkV2ZXJ5IG5ldHdvcmsgaW50ZXJmYWNlIHdp
dGggYW4gQVNJQyBpcyBhbiBleGFtcGxlIG9mIGFjY2VsZXJhdGlvbiwNCm9uZSB0aGF0IHdlJ3Zl
IGNoYXJhY3Rlcml6ZWQgaW4gcGh5c2ljYWwgbmV0d29yayBkZXZpY2VzIGZvciB5ZWFycy4NCg0K
PiBJZiBuZXcgd29yayBpcyBuZWVkZWQgLyBwcm9wb3NlZCB0byBiZSBhZGRlZCBmb3IgdGhlIEJN
V0cgc2NvcGUgYW5kDQo+IGZyYW1ld29yayBpdCB3b3VsZCBiZSB1c2VmdWwgZm9yIEJNV0cgdG8g
bGlzdCB0aGVzZSBzZXBhcmF0ZWx5Lg0KPiANCj4gDQo+IE5pdHMvZWRpdG9yaWFsIGNvbW1lbnRz
Og0KPiANCj4gMS4gV2hhdCBpcyBjYWxsZWQgJ0RlcGxveW1lbnQgc2NlbmFyaW9zJyBmcm9tIFZT
IHBlcnNwZWN0aXZlIGluDQo+IFNlY3Rpb24gNCBkZXNjcmliZSBpbiBmYWN0IGRpZmZlcmVudCBj
b25maWd1cmF0aW9ucyBvZiB0aGUgU1VUIGluIEJNV0cNCj4gdGVybXMuIEl0IHNlZW1zIGJldHRl
ciB0byBzZXBhcmF0ZSB0aGlzIHNlY29uZCBwYXJ0IG9mIHNlY3Rpb24gNCBpbiBhDQo+IHNlcGFy
YXRlIHNlY3Rpb24uIElmIGl0IGJlbG9uZ3MgdG8gYW4gZXhpc3Rpbmcgc2VjdGlvbiBpdCByYXRo
ZXINCj4gYmVsb25ncyBpbiAzIHRoYW4gaW4gNC4NCj4gDQpbQUNNXSANClNlY3Rpb24gMyBpcyBt
b3JlIGFib3V0IGV4dGVuZGluZyB0aGUgY29uZmlndXJhdGlvbiBndWlkYW5jZQ0KZnJvbSBodHRw
czovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1ibXdnLXZpcnR1YWwtbmV0LTA1I3Nl
Y3Rpb24tMy4yDQoNClNlY3Rpb24gNCBzdW1tYXJpemVzIHRoZSBWU1BFUkYgTGV2ZWwgVGVzdCBE
ZXNpZ24gZG9jdW1lbnQsDQpvZiB3aGljaCB0aGVzZSBkZXBsb3ltZW50IHNjZW5hcmlvcyBhcmUg
YSBrZXkgcGFydC4NCg0KdGhhbmtzIGZvciB5b3VyIGNvbW1lbnRzOyBob3BlZnVsbHkgdGhpcyBk
ZXRhaWxlZCByZXBseQ0Kd2lsbCByZWR1Y2UgeW91ciByZXNlcnZhdGlvbnMgYWJvdXQgcHVibGlj
YXRpb24uDQoNCkFsDQooZm9yIHRoZSBjby1hdXRob3JzKQ0KDQpbMF0gaHR0cHM6Ly93d3cub3Bu
ZnYub3JnL2Fubm91bmNlbWVudHMvMjAxNC8wOS8zMC90ZWxlY29tLWluZHVzdHJ5LWFuZC12ZW5k
b3JzLXVuaXRlLXRvLWJ1aWxkLWNvbW1vbi1vcGVuLXBsYXRmb3JtLXRvLWFjY2VsZXJhdGUtbmV0
d29yay1mdW5jdGlvbnMtdmlydHVhbGl6YXRpb24NCg==


From nobody Thu May 11 11:57:57 2017
Return-Path: <dromasca@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE194129442; Thu, 11 May 2017 11:57:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level: 
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d_c3en6yA7-B; Thu, 11 May 2017 11:57:30 -0700 (PDT)
Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B0439129C06; Thu, 11 May 2017 11:52:19 -0700 (PDT)
Received: by mail-qk0-x236.google.com with SMTP id u75so30270897qka.3; Thu, 11 May 2017 11:52:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=a825Gq36ZloTLXauiOQKs0SQrIYAc8wiRvoC+9B5vwc=; b=f3JWHKenhTfBoQX7KkdsTgKZQl2u7XxiKLRBSiAhWJp148bpsIVueQMmV+kHioH0GJ dNV3pp5QXXTyMRAwQHBRfn7v8FxCOS5Ln9mqs187vRkPbgYVaGLs8x7xB37ye4B1JZ/M ByrNtJ0SwdyBfBjniWnKk4cMrhVJD57NRWWOdpEOemulX9Vs/NR1UJ+6aHfpxlXv6GS0 sky+8mAzHEitdfmEoX8ZTaapMxEV6YhO8Dg0N953DI8So8KRm3wCv5QYNAMh+EQwNY9d 5cr2j/AdVnEK5t0/VdXZSmF3NCZGc4qc7z2yCOVcr+8v38tjgGJ299NtfKQI61ow5dVP hzUQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=a825Gq36ZloTLXauiOQKs0SQrIYAc8wiRvoC+9B5vwc=; b=NJMry+pgJkDtfyFe8OI0dEIN/SJohvYwURUEbRTB28fTZs13veIkgGRhFLWTUCsumu FW0TxpT11IXEl0sw3xXW8H+a6ZbIHNIa5WR2lQLx4nM9th6lfHj3J5+0A55LWSRYkOvJ wULCdtg7h2SUnRIfNPSDJ8xVvA70T84IxXpZod1i3dNdJiIU5UUnJKomL7UPQ2MWk3Uf RIaZ86opwIxmbPoGXOMqZjukYoA0WYUcem0T+NFQokx3/eK+tEpvaHhwCmx1ZPVd1MgF wyfHXpyyjimC/zS99y+xEum92YJ7LLgFvIsXB8JL0Et9TA95LZwjUI+O76k5wgZSVfOV pQww==
X-Gm-Message-State: AODbwcBO24YIK4QOQbj2bmDU1Lnb/9P1zzARPZoIwznLLWAp4Jc3vd1n 3AZpyPBU+jozfosNfW+z0TBpugwjTEG0
X-Received: by 10.55.43.144 with SMTP id r16mr256684qkr.213.1494528738801; Thu, 11 May 2017 11:52:18 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.140.19.39 with HTTP; Thu, 11 May 2017 11:52:18 -0700 (PDT)
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25F9B072@njmtexg4.research.att.com>
References: <149450075072.16690.14546662616864459158@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25F9B072@njmtexg4.research.att.com>
From: Dan Romascanu <dromasca@gmail.com>
Date: Thu, 11 May 2017 21:52:18 +0300
Message-ID: <CAFgnS4UAfMe5L7k=zwCqfUjgz14ArBpTE_bz675dCf0_Jk21fg@mail.gmail.com>
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Cc: "gen-art@ietf.org" <gen-art@ietf.org>,  "draft-ietf-bmwg-vswitch-opnfv.all@ietf.org" <draft-ietf-bmwg-vswitch-opnfv.all@ietf.org>,  "ietf@ietf.org" <ietf@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary=001a11471648953f5e054f4416ef
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/odjvawgW-Ccdj-Vb1wzVHoOmno0>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 May 2017 18:57:34 -0000

--001a11471648953f5e054f4416ef
Content-Type: text/plain; charset=UTF-8

Hi,

Please see in-line.

Regards,

Dan


On Thu, May 11, 2017 at 8:00 PM, MORTON, ALFRED C (AL) <acmorton@att.com>
wrote:

> Hi Dan,
> please see replies, [ACM], below.
>
> > -----Original Message-----
> > From: Dan Romascanu [mailto:dromasca@gmail.com]
> > Sent: Thursday, May 11, 2017 7:06 AM
> > To: gen-art@ietf.org
> > Cc: draft-ietf-bmwg-vswitch-opnfv.all@ietf.org; ietf@ietf.org;
> > bmwg@ietf.org; dromasca@gmail.com
> > Subject: Genart last call review of draft-ietf-bmwg-vswitch-opnfv-03
> >
> > Reviewer: Dan Romascanu
> > Review result: Almost Ready
> >
> > I am the assigned Gen-ART reviewer for this draft. The General Area
> > Review Team (Gen-ART) reviews all IETF documents being processed
> > by the IESG for the IETF Chair.  Please treat these comments just
> > like any other last call comments.
> >
> > For more information, please see the FAQ at
> >
> > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__trac.ietf.org_trac_gen_wiki_GenArtfaq&d=DwICaQ&c=LFYZ-
> > o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=NTVlLBN-
> > L3u9zGPHm_CNVcXW7_OGX8_18CtaAalZin0&s=2Hr-
> > dhKaDHIguY7W97z33RlKjqPDtmoYmM2-jWrbS-o&e= >.
> >
> > Document: draft-ietf-bmwg-vswitch-opnfv-03
> > Reviewer: Dan Romascanu
> > Review Date: 2017-05-11
> > IETF LC End Date: 2017-05-15
> > IESG Telechat date: Not scheduled for a telechat
> >
> > Summary:
> >
> > Almost Ready.
> >
> > This document describes describes the progress of the Open Platform
> > for NFV (OPNFV) project on virtual switch performance "VSPERF". That
> > project reuses the BMWG framework and specifications to benchmark
> > virtual switches implemented in general-purpose hardware. Some
> > differences with the benchmarking of specialized HW platforms are
> > identified and they may become work items for BMWG in the future. It's
> > a well written and clear document, but I have reservations about it
> > being published as an RFC, and I cannot find coverage for it in the WG
> > charter. I also have concerns that parts of the methodology used by
> > OPNFV break the BMWG principles, especially repeatability and
> > 'black-box', and this is not clear enough articulated in the document.
> [ACM]
> Ok, let's address your specific issues, and come back to your reservations.
>
> >
> >
> > Major issues:
> >
> > 1. It is not clear to me why this document needs to be published as an
> > RFC. The introduction says: 'This memo describes the progress of the
> > Open Platform for NFV (OPNFV) project on virtual switch performance
> > "VSPERF".  This project intends to build on the current and completed
> > work of the Benchmarking Methodology Working Group in IETF, by
> > referencing existing literature.' Why should the WG and the IESG
> > invest resources in publishing this, why an I-D or an Independent
> > Stream RFC is not sufficient?
> [ACM]
> The WG considered and discussed this document over 3 revisions
> and a year of time before reaching consensus to develop it further
> as a chartered item, so this decision was not taken lightly.
> See more below.
>
> > The WG charter says something about:
> > 'VNF and Related Infrastructure Benchmarking: Benchmarking
> > Methodologies have reliably characterized many physical devices. This
> > work item extends and enhances the methods to virtual network
> > functions (VNF) and their unique supporting infrastructure. A first
> > deliverable from this activity will be a document that considers the
> > new benchmarking space to ensure that common issues are recognized
> > from the start, using background materials from industry and SDOs
> > (e.g., IETF, ETSI NFV).'. I do not believe that this document covers
> > the intent of the charter, as it focused on one organization only.
> [ACM]
> I'm sorry, but here you are mistaken. The document that satisfied
> the "first deliverable ... document that considers the new benchmarking
> space"
> is: https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05
> titled: Considerations for Benchmarking Virtual Network Functions and
> Their Infrastructure
> which has been submitted to IESG and approved for publication.
> Further, the current draft (draft-ietf-bmwg-vswitch-opnfv-03)
> references the approved "Considerations" draft in Section 3
> (as does almost every related Industry spec I'm aware of).
>
> The BMWG Charter continues:
>   Benchmarks for platform capacity and performance characteristics of
>   virtual routers, switches, and related components will follow, including
>   comparisons between physical and virtual network functions. In many
> cases,
>   the traditional benchmarks should be applicable to VNFs, but the lab
>   set-ups, configurations, and measurement methods will likely need to
>   be revised or enhanced.
>
> This draft constitutes one of several follow-on efforts, approaching
> the problem exactly as we described in the last sentence above.
>

How? What last sentence?

Is it:

'In many cases,
  the traditional benchmarks should be applicable to VNFs, but the lab
  set-ups, configurations, and measurement methods will likely need to
  be revised or enhanced.'

How does this document approach this problem? If there is a need to revise
or enhance existing BMWG work, what is needed is specific revisions of
documents. This informational document only documents work in one external
organization. I have reservations that this is a WG task to advance this
document, and of the IESG to approve it. Why can't it stand as an I-D until
the WG decides whatever work needs to be undertaken (if any) to meet the
OPNFV needs? Or if they with to have an RFC, why can't it be Independent
Stream?

Will the WG write similar documents for all (or several) other
organizations that implement VNFs one way or another? Should it?


> An aspect of Industry collaboration that we did not anticipate in the
> BMWG Charter is our current interactions with Open Source Communities.
> The current Charter was approved in June 2014, then OPNFV was founded
> on September 30, 2014 [0] and the VSPERF Project was created on
> Dec 16, 2014, so we did not anticipate extensive collaboration on
> this and other benchmarking topics.
>
>
>
> >
> > 2. In section 3 there 'repeatability' is mentioned, while
> > acknowledging that in a virtual environment there is no guarantee and
> > actually no way to know what other applications are being run.
> [ACM]
> See:
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.4
>
> There are certainly ways to assess the current set of processes
> at a particular time. The Software configuration parameters in
> Section 3.3 are intended to capture this aspect as part of set-up.
> At the same time, there will be challenges to assess the DUT
> performance when resources are fully shared, and new testing
> strategies will be needed:
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3
>
>
> > Measuring parameters as the ones listed in 3.3 provides just part of
> > the answer, and they are internal parameters to the SUT.
> [ACM]
> Yes, knowing the tested configuration is a critical pillar
> supporting repeatability (these items are not measured, but configured),
> and why we provided this section.
>
> > Also, the
> > different deployment scenarios in section 4 require different
> > configurations for the SUT, thus breaking the 'black-box' principle.
> [ACM]
> Specifying DUT configuration does not break any part of the
> black-box principle, which establishes that benchmark measurements
> will be based on externally observable phenomena. See:
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.2
>
> Previous BMWG RFCs have identified the critical configuration
> parameters of the DUT, such as the number and type of
> network interfaces, the arrangement of DUTs in a SUT, etc.
>

I may have not been clear enough. The document talks about repeatability
and about comparing benchmarks with the ones of specialized HW
implementations. Then it goes into a long but still partial list of factors
that can influence the benchmarking, the majority of them depend on the HW
and SW measurements and parameters of the internal systems. How can this be
compared with a number of small and indeed externally observable
configuration parameters like the number and type of network interfaces.
This is several degrees of magnitude apart in complexity.

>
> > I believe that there is a need for a more clear explanation of why BMWG
> > specifications are appropriate and how comparison can be made while
> > repeatability cannot be ensured, and measurements are dependent upon
> > parameters internal to the SUT.
> [ACM]
> I believe that draft-ietf-bmwg-virtual-net-05 already
> indicates why the existing BMWG RFCs are a reasonable
> starting place for NFV benchmarks, in part because
> we want to measure the same benchmarks of physical
> network functions in many cases. See
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-4.1
>
> Repeatability is a goal of all experiments, and we understand
> that there is more work to do in this regard, but what
> we know now (documented in this draft) should
> be a valuable contribution to the Industry.
>
>
Measuring the same benchmarks is a good goal. I believe that the claim of
repeatability needs to be better argued.


>
> >
> > Minor issues:
> >
> > 1. Some of the tests mentioned in Section 4 have no prior or in
> > progress work in the IETF: Control Path and Datapath Coupling Tests,
> > Noisy Neighbour Tests, characterization of acceleration technologies.
> [ACM]
> I'm sorry, but that's not an accurate portrayal of BMWG's literature.
>
> https://tools.ietf.org/html/rfc6413 examined Control Plane/Dataplane
> interactions, for example.
>
> https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#section-3.3
> item 2 specifically included Noisy Neighbour among the new
> testing strategies.
>
>
Please provide references for each in text.


> Every network interface with an ASIC is an example of acceleration,
> one that we've characterized in physical network devices for years.
>
>
Yes, but we do not deal here with externally observable interfaces only,
and if the characterization of the acceleration technologies matters than
you need a way to express it (where can we find it in existing BMWG work?
new work?)


> > If new work is needed / proposed to be added for the BMWG scope and
> > framework it would be useful for BMWG to list these separately.
> >
> >
> > Nits/editorial comments:
> >
> > 1. What is called 'Deployment scenarios' from VS perspective in
> > Section 4 describe in fact different configurations of the SUT in BMWG
> > terms. It seems better to separate this second part of section 4 in a
> > separate section. If it belongs to an existing section it rather
> > belongs in 3 than in 4.
> >
> [ACM]
> Section 3 is more about extending the configuration guidance
> from https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-
> 05#section-3.2
>
> Section 4 summarizes the VSPERF Level Test Design document,
> of which these deployment scenarios are a key part.
>

Yes, but this seams to belong to configurations of SUTs, even if they are
called 'Deployment scenarios' in OPNFV-speak, and they impact
repeatibility.


>
> thanks for your comments; hopefully this detailed reply
> will reduce your reservations about publication.
>
> Al
> (for the co-authors)
>
> [0] https://www.opnfv.org/announcements/2014/09/30/
> telecom-industry-and-vendors-unite-to-build-common-open-
> platform-to-accelerate-network-functions-virtualization
>
>

--001a11471648953f5e054f4416ef
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div>Hi, <br><br></div>Please see in-line. <br><=
br></div>Regards,<br><br></div>Dan<br><br><div><div><div><div class=3D"gmai=
l_extra"><br><div class=3D"gmail_quote">On Thu, May 11, 2017 at 8:00 PM, MO=
RTON, ALFRED C (AL) <span dir=3D"ltr">&lt;<a href=3D"mailto:acmorton@att.co=
m" target=3D"_blank">acmorton@att.com</a>&gt;</span> wrote:<br><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex">Hi Dan,<br>
please see replies, [ACM], below.<br>
<span class=3D"gmail-"><br>
&gt; -----Original Message-----<br>
&gt; From: Dan Romascanu [mailto:<a href=3D"mailto:dromasca@gmail.com">drom=
asca@gmail.com</a>]<br>
&gt; Sent: Thursday, May 11, 2017 7:06 AM<br>
&gt; To: <a href=3D"mailto:gen-art@ietf.org">gen-art@ietf.org</a><br>
&gt; Cc: <a href=3D"mailto:draft-ietf-bmwg-vswitch-opnfv.all@ietf.org">draf=
t-ietf-bmwg-vswitch-opnfv.<wbr>all@ietf.org</a>; <a href=3D"mailto:ietf@iet=
f.org">ietf@ietf.org</a>;<br>
&gt; <a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a>; <a href=3D"mailto:=
dromasca@gmail.com">dromasca@gmail.com</a><br>
&gt; Subject: Genart last call review of draft-ietf-bmwg-vswitch-opnfv-<wbr=
>03<br>
&gt;<br>
&gt; Reviewer: Dan Romascanu<br>
&gt; Review result: Almost Ready<br>
&gt;<br>
&gt; I am the assigned Gen-ART reviewer for this draft. The General Area<br=
>
&gt; Review Team (Gen-ART) reviews all IETF documents being processed<br>
&gt; by the IESG for the IETF Chair.=C2=A0 Please treat these comments just=
<br>
&gt; like any other last call comments.<br>
&gt;<br>
&gt; For more information, please see the FAQ at<br>
&gt;<br>
</span>&gt; &lt;<a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhtt=
ps-" rel=3D"noreferrer" target=3D"_blank">https://urldefense.<wbr>proofpoin=
t.com/v2/url?u=3Dhttps-</a><br>
&gt; 3A__trac.ietf.org_trac_gen_<wbr>wiki_GenArtfaq&amp;d=3DDwICaQ&amp;c=3D=
<wbr>LFYZ-<br>
&gt; o9_HUMeMTSQicvjIg&amp;r=3D<wbr>OfsSu8kTIltVyD1oL72cBw&amp;m=3D<wbr>NTV=
lLBN-<br>
&gt; L3u9zGPHm_CNVcXW7_OGX8_<wbr>18CtaAalZin0&amp;s=3D2Hr-<br>
&gt; dhKaDHIguY7W97z33RlKjqPDtmoYmM<wbr>2-jWrbS-o&amp;e=3D &gt;.<br>
&gt;<br>
&gt; Document: draft-ietf-bmwg-vswitch-opnfv-<wbr>03<br>
<span class=3D"gmail-">&gt; Reviewer: Dan Romascanu<br>
&gt; Review Date: 2017-05-11<br>
&gt; IETF LC End Date: 2017-05-15<br>
&gt; IESG Telechat date: Not scheduled for a telechat<br>
&gt;<br>
&gt; Summary:<br>
&gt;<br>
&gt; Almost Ready.<br>
&gt;<br>
&gt; This document describes describes the progress of the Open Platform<br=
>
&gt; for NFV (OPNFV) project on virtual switch performance &quot;VSPERF&quo=
t;. That<br>
&gt; project reuses the BMWG framework and specifications to benchmark<br>
&gt; virtual switches implemented in general-purpose hardware. Some<br>
&gt; differences with the benchmarking of specialized HW platforms are<br>
&gt; identified and they may become work items for BMWG in the future. It&#=
39;s<br>
&gt; a well written and clear document, but I have reservations about it<br=
>
&gt; being published as an RFC, and I cannot find coverage for it in the WG=
<br>
&gt; charter. I also have concerns that parts of the methodology used by<br=
>
&gt; OPNFV break the BMWG principles, especially repeatability and<br>
&gt; &#39;black-box&#39;, and this is not clear enough articulated in the d=
ocument.<br>
</span>[ACM]<br>
Ok, let&#39;s address your specific issues, and come back to your reservati=
ons.<br>
<span class=3D"gmail-"><br>
&gt;<br>
&gt;<br>
&gt; Major issues:<br>
&gt;<br>
&gt; 1. It is not clear to me why this document needs to be published as an=
<br>
&gt; RFC. The introduction says: &#39;This memo describes the progress of t=
he<br>
&gt; Open Platform for NFV (OPNFV) project on virtual switch performance<br=
>
&gt; &quot;VSPERF&quot;.=C2=A0 This project intends to build on the current=
 and completed<br>
&gt; work of the Benchmarking Methodology Working Group in IETF, by<br>
&gt; referencing existing literature.&#39; Why should the WG and the IESG<b=
r>
&gt; invest resources in publishing this, why an I-D or an Independent<br>
&gt; Stream RFC is not sufficient?<br>
</span>[ACM]<br>
The WG considered and discussed this document over 3 revisions<br>
and a year of time before reaching consensus to develop it further<br>
as a chartered item, so this decision was not taken lightly.<br>
See more below.<br>
<span class=3D"gmail-"><br>
&gt; The WG charter says something about:<br>
&gt; &#39;VNF and Related Infrastructure Benchmarking: Benchmarking<br>
&gt; Methodologies have reliably characterized many physical devices. This<=
br>
&gt; work item extends and enhances the methods to virtual network<br>
&gt; functions (VNF) and their unique supporting infrastructure. A first<br=
>
&gt; deliverable from this activity will be a document that considers the<b=
r>
&gt; new benchmarking space to ensure that common issues are recognized<br>
&gt; from the start, using background materials from industry and SDOs<br>
&gt; (e.g., IETF, ETSI NFV).&#39;. I do not believe that this document cove=
rs<br>
&gt; the intent of the charter, as it focused on one organization only.<br>
</span>[ACM]<br>
I&#39;m sorry, but here you are mistaken. The document that satisfied<br>
the &quot;first deliverable ... document that considers the new benchmarkin=
g space&quot;<br>
is: <a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05" =
rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<wbr>draft=
-ietf-bmwg-virtual-net-05</a><br>
titled: Considerations for Benchmarking Virtual Network Functions and Their=
 Infrastructure<br>
which has been submitted to IESG and approved for publication.<br>
Further, the current draft (draft-ietf-bmwg-vswitch-<wbr>opnfv-03)<br>
references the approved &quot;Considerations&quot; draft in Section 3<br>
(as does almost every related Industry spec I&#39;m aware of).<br>
<br>
The BMWG Charter continues:<br>
=C2=A0 Benchmarks for platform capacity and performance characteristics of<=
br>
=C2=A0 virtual routers, switches, and related components will follow, inclu=
ding<br>
=C2=A0 comparisons between physical and virtual network functions. In many =
cases,<br>
=C2=A0 the traditional benchmarks should be applicable to VNFs, but the lab=
<br>
=C2=A0 set-ups, configurations, and measurement methods will likely need to=
<br>
=C2=A0 be revised or enhanced.<br>
<br>
This draft constitutes one of several follow-on efforts, approaching<br>
the problem exactly as we described in the last sentence above.<br></blockq=
uote><div><br></div><div>How? What last sentence? <br><br></div><div>Is it:=
 <br></div><div><br>&#39;In many cases,<br>
=C2=A0 the traditional benchmarks should be applicable to VNFs, but the lab=
<br>
=C2=A0 set-ups, configurations, and measurement methods will likely need to=
<br>
=C2=A0 be revised or enhanced.&#39;<br><br></div><div>How does this documen=
t approach this problem? If there is a need to revise or enhance existing B=
MWG work, what is needed is specific revisions of documents. This informati=
onal document only documents work in one external organization. I have rese=
rvations that this is a WG task to advance this document, and of the IESG t=
o approve it. Why can&#39;t it stand as an I-D until the WG decides whateve=
r work needs to be undertaken (if any) to meet the OPNFV needs? Or if they =
with to have an RFC, why can&#39;t it be Independent Stream? <br><br></div>=
<div>Will the WG write similar documents for all (or several) other organiz=
ations that implement VNFs one way or another? Should it?<br></div><div><br=
> </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex=
;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
An aspect of Industry collaboration that we did not anticipate in the<br>
BMWG Charter is our current interactions with Open Source Communities.<br>
The current Charter was approved in June 2014, then OPNFV was founded<br>
on September 30, 2014 [0] and the VSPERF Project was created on<br>
Dec 16, 2014, so we did not anticipate extensive collaboration on<br>
this and other benchmarking topics.<br>
<span class=3D"gmail-"><br>
<br>
<br>
&gt;<br>
&gt; 2. In section 3 there &#39;repeatability&#39; is mentioned, while<br>
&gt; acknowledging that in a virtual environment there is no guarantee and<=
br>
&gt; actually no way to know what other applications are being run.<br>
</span>[ACM]<br>
See:<br>
<a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#secti=
on-3.4" rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<w=
br>draft-ietf-bmwg-virtual-net-<wbr>05#section-3.4</a><br>
<br>
There are certainly ways to assess the current set of processes<br>
at a particular time. The Software configuration parameters in<br>
Section 3.3 are intended to capture this aspect as part of set-up.<br>
At the same time, there will be challenges to assess the DUT<br>
performance when resources are fully shared, and new testing<br>
strategies will be needed:<br>
<a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#secti=
on-3.3" rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<w=
br>draft-ietf-bmwg-virtual-net-<wbr>05#section-3.3</a><br>
<span class=3D"gmail-"><br>
<br>
&gt; Measuring parameters as the ones listed in 3.3 provides just part of<b=
r>
&gt; the answer, and they are internal parameters to the SUT.<br>
</span>[ACM]<br>
Yes, knowing the tested configuration is a critical pillar<br>
supporting repeatability (these items are not measured, but configured),<br=
>
and why we provided this section.<br>
<span class=3D"gmail-"><br>
&gt; Also, the<br>
&gt; different deployment scenarios in section 4 require different<br>
&gt; configurations for the SUT, thus breaking the &#39;black-box&#39; prin=
ciple.<br>
</span>[ACM]<br>
Specifying DUT configuration does not break any part of the<br>
black-box principle, which establishes that benchmark measurements<br>
will be based on externally observable phenomena. See:<br>
<a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#secti=
on-4.2" rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<w=
br>draft-ietf-bmwg-virtual-net-<wbr>05#section-4.2</a><br>
<br>
Previous BMWG RFCs have identified the critical configuration<br>
parameters of the DUT, such as the number and type of<br>
network interfaces, the arrangement of DUTs in a SUT, etc.<br></blockquote>=
<div><br></div><div>I may have not been clear enough. The document talks ab=
out repeatability and about comparing benchmarks with the ones of specializ=
ed HW implementations. Then it goes into a long but still partial list of f=
actors that can influence the benchmarking, the majority of them depend on =
the HW and SW measurements and parameters of the internal systems. How can =
this be compared with a number of small and indeed externally observable co=
nfiguration parameters like the number and type of network interfaces. This=
 is several degrees of magnitude apart in complexity.=C2=A0 <br></div><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:=
1px solid rgb(204,204,204);padding-left:1ex">
<span class=3D"gmail-"><br>
&gt; I believe that there is a need for a more clear explanation of why BMW=
G<br>
&gt; specifications are appropriate and how comparison can be made while<br=
>
&gt; repeatability cannot be ensured, and measurements are dependent upon<b=
r>
&gt; parameters internal to the SUT.<br>
</span>[ACM]<br>
I believe that draft-ietf-bmwg-virtual-net-05 already<br>
indicates why the existing BMWG RFCs are a reasonable<br>
starting place for NFV benchmarks, in part because<br>
we want to measure the same benchmarks of physical<br>
network functions in many cases. See<br>
<a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#secti=
on-4.1" rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<w=
br>draft-ietf-bmwg-virtual-net-<wbr>05#section-4.1</a><br>
<br>
Repeatability is a goal of all experiments, and we understand<br>
that there is more work to do in this regard, but what<br>
we know now (documented in this draft) should<br>
be a valuable contribution to the Industry.<br>
<span class=3D"gmail-"><br></span></blockquote><div><br></div><div>Measurin=
g the same benchmarks is a good goal. I believe that the claim of repeatabi=
lity needs to be better argued. <br>=C2=A0<br></div><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,=
204,204);padding-left:1ex"><span class=3D"gmail-">
<br>
&gt;<br>
&gt; Minor issues:<br>
&gt;<br>
&gt; 1. Some of the tests mentioned in Section 4 have no prior or in<br>
&gt; progress work in the IETF: Control Path and Datapath Coupling Tests,<b=
r>
&gt; Noisy Neighbour Tests, characterization of acceleration technologies.<=
br>
</span>[ACM]<br>
I&#39;m sorry, but that&#39;s not an accurate portrayal of BMWG&#39;s liter=
ature.<br>
<br>
<a href=3D"https://tools.ietf.org/html/rfc6413" rel=3D"noreferrer" target=
=3D"_blank">https://tools.ietf.org/html/<wbr>rfc6413</a> examined Control P=
lane/Dataplane<br>
interactions, for example.<br>
<br>
<a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#secti=
on-3.3" rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/html/<w=
br>draft-ietf-bmwg-virtual-net-<wbr>05#section-3.3</a><br>
item 2 specifically included Noisy Neighbour among the new<br>
testing strategies.<br>
<br></blockquote><div><br></div><div>Please provide references for each in =
text. <br>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin=
:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"=
>
Every network interface with an ASIC is an example of acceleration,<br>
one that we&#39;ve characterized in physical network devices for years.<br>
<span class=3D"gmail-"><br></span></blockquote><div><br></div><div>Yes, but=
 we do not deal here with externally observable interfaces only, and if the=
 characterization of the acceleration technologies matters than you need a =
way to express it (where can we find it in existing BMWG work? new work?)<b=
r>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px=
 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span c=
lass=3D"gmail-">
&gt; If new work is needed / proposed to be added for the BMWG scope and<br=
>
&gt; framework it would be useful for BMWG to list these separately.<br>
&gt;<br>
&gt;<br>
&gt; Nits/editorial comments:<br>
&gt;<br>
&gt; 1. What is called &#39;Deployment scenarios&#39; from VS perspective i=
n<br>
&gt; Section 4 describe in fact different configurations of the SUT in BMWG=
<br>
&gt; terms. It seems better to separate this second part of section 4 in a<=
br>
&gt; separate section. If it belongs to an existing section it rather<br>
&gt; belongs in 3 than in 4.<br>
&gt;<br>
</span>[ACM]<br>
Section 3 is more about extending the configuration guidance<br>
from <a href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-05#=
section-3.2" rel=3D"noreferrer" target=3D"_blank">https://tools.ietf.org/ht=
ml/<wbr>draft-ietf-bmwg-virtual-net-<wbr>05#section-3.2</a><br>
<br>
Section 4 summarizes the VSPERF Level Test Design document,<br>
of which these deployment scenarios are a key part.<br></blockquote><div><b=
r></div><div>Yes, but this seams to belong to configurations of SUTs, even =
if they are called &#39;Deployment scenarios&#39; in OPNFV-speak, and they =
impact repeatibility. <br>=C2=A0 <br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex">
<br>
thanks for your comments; hopefully this detailed reply<br>
will reduce your reservations about publication.<br>
<br>
Al<br>
(for the co-authors)<br>
<br>
[0] <a href=3D"https://www.opnfv.org/announcements/2014/09/30/telecom-indus=
try-and-vendors-unite-to-build-common-open-platform-to-accelerate-network-f=
unctions-virtualization" rel=3D"noreferrer" target=3D"_blank">https://www.o=
pnfv.org/<wbr>announcements/2014/09/30/<wbr>telecom-industry-and-vendors-<w=
br>unite-to-build-common-open-<wbr>platform-to-accelerate-<wbr>network-func=
tions-<wbr>virtualization</a><br>
<br>
</blockquote></div><br></div></div></div></div></div>

--001a11471648953f5e054f4416ef--


From nobody Thu May 11 12:08:10 2017
Return-Path: <ietf-secretariat-reply@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A1D2713151E for <bmwg@ietf.org>; Thu, 11 May 2017 12:08:08 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
To: <bmwg@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149452968865.16583.1881676704353489402.idtracker@ietfa.amsl.com>
Date: Thu, 11 May 2017 12:08:08 -0700
From: IETF Secretariat <ietf-secretariat-reply@ietf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/RcleKE7QfCROrNW_P9RuMrZMo3Q>
Subject: [bmwg] Milestones changed for bmwg WG
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 May 2017 19:08:09 -0000

Changed milestone "Draft on vSwitch Benchmarking in OPNFV to IESG
Review", set state to active from review, accepting new milestone.

URL: https://datatracker.ietf.org/wg/bmwg/about/


From nobody Thu May 11 18:51:28 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 35328129C1B; Thu, 11 May 2017 18:51:20 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149455388008.16571.2931297823875256605@ietfa.amsl.com>
Date: Thu, 11 May 2017 18:51:20 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/12HrWFtlw0aLJS1IUoENbLE6law>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-05.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 May 2017 01:51:20 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-05.txt
	Pages           : 18
	Date            : 2017-05-11

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-05
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-05

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-05


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu May 11 19:17:01 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id B05841314C5; Thu, 11 May 2017 19:16:53 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149455541368.16612.16337373042059985436@ietfa.amsl.com>
Date: Thu, 11 May 2017 19:16:53 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CZM2szWSVJ-LExgpfz4nSLaebl8>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-08.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 May 2017 02:16:54 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-08.txt
	Pages           : 16
	Date            : 2017-05-11

Abstract:
The purpose of this informational document is to establish definitions,
discussion and measurement techniques for data center benchmarking.
Also, it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts when
one is interested in benchmarking network switches and routers in the
data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-08
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-08


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu May 11 19:36:44 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id AA3A012EB02; Thu, 11 May 2017 19:36:35 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149455659558.16612.11429979881810262287@ietfa.amsl.com>
Date: Thu, 11 May 2017 19:36:35 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/YWXBpV7ti_4fXcTFksiSbRgzY0k>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-06.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 May 2017 02:36:36 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-06.txt
	Pages           : 18
	Date            : 2017-05-11

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-06
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-06

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-06


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu May 11 20:41:07 2017
Return-Path: <wwwrun@rfc-editor.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0E30912708C; Thu, 11 May 2017 20:40:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.202
X-Spam-Level: 
X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 79PZ6PStW2Eq; Thu, 11 May 2017 20:40:57 -0700 (PDT)
Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5BBB81294F0; Thu, 11 May 2017 20:34:35 -0700 (PDT)
Received: by rfc-editor.org (Postfix, from userid 30) id C7AA2B819B0; Thu, 11 May 2017 20:34:24 -0700 (PDT)
To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
X-PHP-Originating-Script: 1005:ams_util_lib.php
From: rfc-editor@rfc-editor.org
Cc: rfc-editor@rfc-editor.org, drafts-update-ref@iana.org, bmwg@ietf.org
Message-Id: <20170512033424.C7AA2B819B0@rfc-editor.org>
Date: Thu, 11 May 2017 20:34:24 -0700 (PDT)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/BgCPyKdtks5Q91ABzmbrND0tg_8>
Subject: [bmwg] RFC 8161 on Benchmarking the Neighbor Discovery Protocol
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 May 2017 03:40:59 -0000

A new Request for Comments is now available in online RFC libraries.

        
        RFC 8161

        Title:      Benchmarking the Neighbor Discovery Protocol 
        Author:     W. Cerveny, 
                    R. Bonica,
                    R. Thomas
        Status:     Informational
        Stream:     IETF
        Date:       May 2017
        Mailbox:    wcerveny@arbor.net, 
                    rbonica@juniper.net, 
                    rejithomas@juniper.net
        Pages:      17
        Characters: 32658
        Updates/Obsoletes/SeeAlso:   None

        I-D Tag:    draft-ietf-bmwg-ipv6-nd-06.txt

        URL:        https://www.rfc-editor.org/info/rfc8161

        DOI:        10.17487/RFC8161

This document provides benchmarking procedures for the Neighbor
Discovery Protocol (NDP).  It also proposes metrics by which an NDP
implementation's scaling capabilities can be measured.

This document is a product of the Benchmarking Methodology Working Group of the IETF.


INFORMATIONAL: This memo provides information for the Internet community.
It does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

This announcement is sent to the IETF-Announce and rfc-dist lists.
To subscribe or unsubscribe, see
  https://www.ietf.org/mailman/listinfo/ietf-announce
  https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist

For searching the RFC series, see https://www.rfc-editor.org/search
For downloading RFCs, see https://www.rfc-editor.org/retrieve/bulk

Requests for special distribution should be addressed to either the
author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
specifically noted otherwise on the RFC itself, all RFCs are for
unlimited distribution.


The RFC Editor Team
Association Management Solutions, LLC



From nobody Fri May 12 04:38:40 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4677812EB81 for <bmwg@ietfa.amsl.com>; Fri, 12 May 2017 04:38:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id abypqr22yemw for <bmwg@ietfa.amsl.com>; Fri, 12 May 2017 04:38:37 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B57531293DB for <bmwg@ietf.org>; Fri, 12 May 2017 04:33:15 -0700 (PDT)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v4CBP41N034530; Fri, 12 May 2017 07:33:12 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0083689.ppops.net-00191d01. with ESMTP id 2ad6wheqag-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 12 May 2017 07:33:12 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v4CBXBi5022591; Fri, 12 May 2017 07:33:11 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v4CBX3dj022441 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 12 May 2017 07:33:08 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor); Fri, 12 May 2017 11:33:00 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4CBWxMj005608; Fri, 12 May 2017 06:32:59 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4CBWsRh005314; Fri, 12 May 2017 06:32:54 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 7BB66E08BA; Fri, 12 May 2017 07:32:53 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Fri, 12 May 2017 07:32:53 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: joel jaeggli <joelja@gmail.com>
Thread-Topic: RFC 8161 on Benchmarking the Neighbor Discovery Protocol
Thread-Index: AQHSytGetKMVhzUsm0Wsq73YB0nYNqHwkMdQ
Date: Fri, 12 May 2017 11:32:52 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F9B52A@njmtexg4.research.att.com>
References: <20170512033424.C7AA2B819B0@rfc-editor.org>
In-Reply-To: <20170512033424.C7AA2B819B0@rfc-editor.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-12_06:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705120215
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/21Xzc_CnLMBZPQPL4F9-5WyYBAI>
Subject: Re: [bmwg] RFC 8161 on Benchmarking the Neighbor Discovery Protocol
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 May 2017 11:38:39 -0000

Congratulations, and thanks for your
efforts to complete AUTH48 in a timely way!=20

Al
bmwg co-chair

> -----Original Message-----
> From: IETF-Announce [mailto:ietf-announce-bounces@ietf.org] On Behalf Of
> rfc-editor@rfc-editor.org
> Sent: Thursday, May 11, 2017 11:34 PM
> To: ietf-announce@ietf.org; rfc-dist@rfc-editor.org
> Cc: drafts-update-ref@iana.org; bmwg@ietf.org; rfc-editor@rfc-editor.org
> Subject: RFC 8161 on Benchmarking the Neighbor Discovery Protocol
>=20
> A new Request for Comments is now available in online RFC libraries.
>=20
>=20
>         RFC 8161
>=20
>         Title:      Benchmarking the Neighbor Discovery Protocol
>         Author:     W. Cerveny,
>                     R. Bonica,
>                     R. Thomas
>         Status:     Informational
>         Stream:     IETF
>         Date:       May 2017
>         Mailbox:    wcerveny@arbor.net,
>                     rbonica@juniper.net,
>                     rejithomas@juniper.net
>         Pages:      17
>         Characters: 32658
>         Updates/Obsoletes/SeeAlso:   None
>=20
>         I-D Tag:    draft-ietf-bmwg-ipv6-nd-06.txt
>=20
>         URL:        https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.rfc-2Deditor.org_info_rfc8161&d=3DDwIBAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DrOZUY2r8z6KydD2YQGzO2WaC=
zmm
> QWLZd4t0gYedj0JU&s=3Dkl3iqAi6BY4N4mTeLV18pUWBowlphT0MbRKrPq402ec&e=3D
>=20
>         DOI:        10.17487/RFC8161
>=20
> This document provides benchmarking procedures for the Neighbor
> Discovery Protocol (NDP).  It also proposes metrics by which an NDP
> implementation's scaling capabilities can be measured.
>=20
> This document is a product of the Benchmarking Methodology Working Group
> of the IETF.
>=20
>=20
> INFORMATIONAL: This memo provides information for the Internet
> community.
> It does not specify an Internet standard of any kind. Distribution of
> this memo is unlimited.
>=20
> This announcement is sent to the IETF-Announce and rfc-dist lists.
> To subscribe or unsubscribe, see
>   https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_ietf-2Dannounce&d=3DDwIBAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DrOZUY2r8z6KydD2YQGzO2WaC=
zmm
> QWLZd4t0gYedj0JU&s=3DgYUt2jWbzPAAYHlNNKoGtqYRidPYaH4GECIXjWkxDyk&e=3D
>   https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__mailman.rfc-
> 2Deditor.org_mailman_listinfo_rfc-2Ddist&d=3DDwIBAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DrOZUY2r8z6KydD2YQGzO2WaC=
zmm
> QWLZd4t0gYedj0JU&s=3DWAvq2eG23Lt-6-3sRu0gg58dRFAEdqWKYLzpD0NATEE&e=3D
>=20
> For searching the RFC series, see
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.rfc-
> 2Deditor.org_search&d=3DDwIBAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DrOZUY2r8z6KydD2YQGzO2WaC=
zmm
> QWLZd4t0gYedj0JU&s=3DH5iFi3XW1fOxjHHaOtggwaGJXSJ-q1X-c6drIsb1jTU&e=3D
> For downloading RFCs, see
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.rfc-
> 2Deditor.org_retrieve_bulk&d=3DDwIBAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DrOZUY2r8z6KydD2YQGzO2WaC=
zmm
> QWLZd4t0gYedj0JU&s=3DETMrVYRF4jWuASAiEorZ_KbG_Qa80G_VPtjBdTKC6ws&e=3D
>=20
> Requests for special distribution should be addressed to either the
> author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
> specifically noted otherwise on the RFC itself, all RFCs are for
> unlimited distribution.
>=20
>=20
> The RFC Editor Team
> Association Management Solutions, LLC
>=20


From nobody Tue May 23 11:55:54 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 214AE1293F5 for <bmwg@ietfa.amsl.com>; Tue, 23 May 2017 11:54:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W2vYjeeXo3oO for <bmwg@ietfa.amsl.com>; Tue, 23 May 2017 11:54:18 -0700 (PDT)
Received: from mail-vk0-x232.google.com (mail-vk0-x232.google.com [IPv6:2607:f8b0:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 810E41270AC for <bmwg@ietf.org>; Tue, 23 May 2017 11:54:18 -0700 (PDT)
Received: by mail-vk0-x232.google.com with SMTP id h16so63377124vkd.2 for <bmwg@ietf.org>; Tue, 23 May 2017 11:54:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc; bh=vn/Ly1eYU3AclgUrdHn0RAigbO5yViZax+8NdCb9wYs=; b=lhriQJszFZxXibGrvAdQDjOjhXUbZaSgf1/y+83XkiNUota4s3RWKB5PGS7z5qFy45 8C4B7iIE8X6NcmRzQ9u9yg0AQxVeF4d6tdgEcm6gsHiQLailmeCB42GmUVpDdlBrFRn5 q0sDwdGuIsfTMnUEzJuHEzUujQwQUSjwXT3unddBPI7H3NRYSwFyL2Oq8oAbkN5XZkAp 4/82Fk/iVodjU2y6G/PizjUone3OcN5vs72XQBmk30HnyzPld+PjuaD5NUxUKgfJxVQ2 L9fhuSVv1/f1RLzFr+EkvfB9ofSdJAo3xEux+QrGlnzG0uLye6SFdqab8v2ywub2MmU1 8YEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=vn/Ly1eYU3AclgUrdHn0RAigbO5yViZax+8NdCb9wYs=; b=D8sbspBQ/Spdb9jITjBWs20pyMiDO/4GW7MDOFAFAKFQBLIfa7nkVA+/+Qrtbb5St7 pJ0g3kV8MOmX+L3aX8Mf87TA1hZNa8y250t4jt+5lfXyWf+2OqVRWh/HsVK/6VSCYCyL mhlMfzxf8bxqwHouFgehSm00bjmGbSC3lwwbuRPCaJFq4jLDbVOxsxyKTBNTyNA5EZ29 ByxwtH0bUTa8RuqR+OHsuDn0lPqUPAFrrf9CwKByH/gaVx3MQ6b2ne3UhiIRYW/qyhqG +QifVJOgJ+Oy+Glewn299rPiSWcORNcnWpgPZetb/4dUDDzQK0oBqq4wzQOfriEO21cG 5sSA==
X-Gm-Message-State: AODbwcCxkVcFQsadxE4Rn3tEQFF7YQOfZUZp2YK/MmXk60m5zH/0VZgL jb2rU3y+pzuFPc1fOC6xSANoRNixW3Li
X-Received: by 10.31.173.142 with SMTP id w136mr11709168vke.125.1495565657113;  Tue, 23 May 2017 11:54:17 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.71.83 with HTTP; Tue, 23 May 2017 11:53:36 -0700 (PDT)
From: Warren Kumari <warren@kumari.net>
Date: Tue, 23 May 2017 14:53:36 -0400
Message-ID: <CAHw9_i+yE8a02bP=NQym6nxUhCJM6OOdhkRwv=2GamQjV+KS1A@mail.gmail.com>
To: draft-ietf-bmwg-dcbench-terminology@ietf.org
Cc: bmwg@ietf.org
Content-Type: multipart/alternative; boundary="001a1143881ebb19ff0550358302"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/ZaZpEFvGve-WbYUM6LRNjFkDnUg>
X-Mailman-Approved-At: Tue, 23 May 2017 11:55:53 -0700
Subject: [bmwg] AD review of draft-ietf-bmwg-dcbench-terminology.
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 23 May 2017 18:54:22 -0000

--001a1143881ebb19ff0550358302
Content-Type: text/plain; charset="UTF-8"

Hi there,

Apologies, I had forgotten to click the AD eval button... {DONE}

Thanks for a (generally) clear and useful document.

I have completed my AD review of draft-ietf-bmwg-dcbench-terminology and I
have a number of suggested edits to make the document even more readable /
clearer. In a few instances I wasn't quite sure what the document was
trying to say, so please let me know if any of the suggestions don't make
sense, or change the text meaning...

Also, please poke me once updated, so that I can progress it to IETF LC.

W
------

Abstract

The purpose of this informational document is to establish definitions,
discussion and measurement techniques for data center benchmarking.
[O] establish definitions, discussion and measurement techniques
[R] I'm not sure what is meant by "discussion" here. "Discussion and
measurement techniques" doesn't make much sense to me. Perhaps "to
establish definitions, foster discussion and describe measurement
techniques ..."?



Also, it is to introduce new terminologies applicable to data center
[O] benchmarking. Also, it is to introduce
[P] benchmarking, as well as introduce
[R] clarity/no need for a separate sentence.

performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts when
one is interested in benchmarking network switches and routers in the
[O] when one is interested in benchmarking
[P] for benchmarking
[R] readability

data center.



1.  Introduction

   Traffic patterns in the data center are not uniform and are contently
[O] contently
[P] constantly
[R] word choice/typo

   changing. They are dictated by the nature and variety of applications
   utilized in the data center. It can be largely east-west traffic
   flows in one data center and north-south in another, while some may
   combine both. Traffic patterns can be bursty in nature and contain
   many-to-one, many-to-many, or one-to-many flows. Each flow may also
   be small and latency sensitive or large and throughput sensitive
   while containing a mix of UDP and TCP traffic. All of which can
   coexist in a single cluster and flow through a single network device
   all at the same time. Benchmarking of network devices have long used
[O] All of which can
   coexist in a single cluster and flow through a single network device
   all at the same time.
[P] One or more of these may coexist in a single cluster and flow through a
single network device simultaneously.
[R] grammar/readability - unless this changes the intent?

   [RFC1242], [RFC2432], [RFC2544], [2] and [3]. These benchmarks have
   largely been focused around various latency attributes and max
   throughput of the Device Under Test being benchmarked. These
   standards are good at measuring theoretical max throughput,
   forwarding rates and latency under testing conditions, but to not
[O] , but to not
[P]; but they do not
[R] grammar

   represent real traffic patterns that may affect these networking
   devices. The data center networking devices covered are switches and
   routers.



1.2. Definition format

   Term to be defined. (e.g., Latency)

   Definition: The specific definition for the term.

   Discussion: A brief discussion about the term, it's application and
[O] it's
[P] its
[R] Grammar. Should be a possessive, not a contraction of "it is."
   any restrictions on measurement procedures.

   Measurement Units: Methodology for the measure and units used to
   report measurements of this term, if applicable.


2.  Latency


2.1. Definition

   Latency is a the amount of time it takes a frame to transit the DUT.
   Latency is measured in unit of time (seconds, milliseconds,
   microseconds and so on). The purpose of measuring latency is to
   understand what is the impact of adding a device in the communication
[O]  understand what is the impact
[P] understand the impact
[R] grammar

   path.

   The Latency interval can be assessed between different combinations
   of events, irrespectively of the type of switching device (bit
[O] irrespectively
[P] regardless
[R] grammar

   forwarding aka cut-through or store forward type of device)



2.2 Discussion

   FILO is the most important measuring definition. Any type of switches
[O] Any type of switches
[P] All switches
[R] readability

   MUST be measured with the FILO mechanism: FILO will include the
   latency of the switch and the latency of the frame as well as the
   serialization delay. It is a picture of the 'whole' latency going
   through the DUT. For applications, which are latency sensitive and
   can function with initial bytes of the frame, FIFO MAY be an
   additional type of measuring to supplement FILO.


3 Jitter
3.2 Discussion

   In addition to PDV Range and or a high percentile of PDV, Inter-
[O]  and or
[P] and/or

   Packet Delay Variation (IPDV) as defined in section 4.1 of RFC5481
   (differences between two consecutive packets) MAY be used for the
   purpose of determining how packet spacing has changed during
   transfer, for example to see if packet stream has become closely-
[O] for example to see
[P] for example, to see
[R] grammar

   spaced or "bursty". However, the Absolute Value of IPDV SHOULD NOT be
   used, as this collapses the "bursty" and "dispersed" sides of the
   IPDV distribution together.

4 Physical Layer Calibration
4.2 Discussion

   Physical layer calibration is part of the end to end latency, which
   should be taken into acknowledgment while evaluating the DUT. Small
   variations of the physical components of the test may impact the
   latency being measure so they MUST be described when presenting
[O] latency being measure so
[P] latency being measured, so
[R] grammar

   results.


5 Line rate

5.1 Definition

   The transmit timing, or maximum transmitted data rate is controlled
   by the "transmit clock" in the DUT.  The receive timing (maximum
   ingress data rate) is derived from the transmit clock of the
   connected interface.

   The line rate or physical layer frame rate is the maximum capacity to
   send frames of a specific size at the transmit clock frequency of the
   DUT.

   The term port capacity term defines the maximum speed capability for
[O] The term port capacity term
[P] The term "port capacity"
[R] readability

   the given port; for example 1GE, 10GE, 40GE, 100GE etc.

   The frequency ("clock rate") of the transmit clock in any two
   connected interfaces will never be precisely the same, therefore a
   tolerance is needed, this will be expressed by Parts Per Million
[O] same, therefore a tolerance is needed, this will
[P] same; therefore, a tolerance is needed. This will
[R] grammar


5.2 Discussion

   For a transmit clock source, most Ethernet switches use "clock
   modules" (also called "oscillator modules") that are sealed,
   internally temperature-compensated, and very accurate. The output
   frequency of these modules is not adjustable because it is not
   necessary.  Many test sets, however, offer a software-controlled
   adjustment of the transmit clock rate, which should be used to
   compensate the test equipment to not send more than line rate of the
   DUT.

[O] which should be used to
   compensate the test equipment to not send more than line rate of the
   DUT.
[R] please reword for clarity.

   To allow for the minor variations typically found in the clock rate
   of commercially-available clock modules and other crystal-based


   Test set equipment manufacturers are well-aware of the standards, and
   allows a software-controlled +/- 100 PPM "offset" (clock-rate
[O] allows
[P] allow
[R] grammar

   adjustment) to compensate for normal variations in the clock speed of
   "devices under test". This offset adjustment allows engineers to
   determine the approximate speed the connected device is operating,
   and verify that it is within parameters allowed by standards.



5.3 Measurement Units


   In a production network, it is very unlikely to see precise line rate
   over a very brief period. There is no observable difference between
   dropping packets at 99% of line rate and 100% of line rate. -Line
   rate CAN measured at 100% of line rate with a -100PPM adjustment. -
   Line rate SHOULD be measured at 99,98% with 0 PPM adjustment.-The PPM
[O] .-The
[P] . The
[R] Typo Or was this intended to be a list?

   adjustment SHOULD only be used for a line rate type of measurement


6  Buffering
6.1 Buffer
6.1.1 Definition

   Buffer Size: the term buffer size, represents the total amount of
[O] the term buffer size, represents
[P] the term buffer size represents
[R] grammar

   frame buffering memory available on a DUT. This size is expressed in
   Byte; KB (kilobytes), MB (megabytes) or GB (gigabyte). When the
   buffer size is expressed it SHOULD be defined by a size metric
   defined above. When the buffer size is expressed, an indication of
[O]  defined above
[P] listed above [OR] stated above
[R] the sizes are not "defined" above

   the frame MTU used for that measurement is also necessary as well as
   the cos or dscp value set; as often times the buffers are carved by
   quality of service implementation. (please refer to the buffer
[O] . (please
[P] . (Please
[R] grammar

   efficiency section for further details).

   Example: Buffer Size of DUT when sending 1518 bytes frames is 18 Mb.

   Port Buffer Size: the port buffer size is the amount of buffer a
   single ingress port, egress port or combination of ingress and egress
   buffering location for a single port. The reason of mentioning the
[O] reason of mentioning
[P] reason for mentioning
[R] grammar

   three locations for the port buffer is, that the DUT buffering scheme
[O] is, that
[P] is because
[R] grammar


   can be unknown or untested, and therefore the indication of where the
   buffer is located helps understand the buffer architecture and
[O]  and therefore the indication of where the
   buffer is located helps understand the buffer architecture
[P] and so knowing the buffer location helps clarify the buffer architecture
[R] readability

   therefore the total buffer size. The Port Buffer Size is an
   informational value that MAY be provided from the DUT vendor. It is
   not a value that is tested by benchmarking. Benchmarking will be done
   using the Maximum Port Buffer Size or Maximum Buffer Size
   methodology.

   Maximum Port Buffer Size: this is in most cases the same as the Port
[O] this is in most cases the same
[P] in most cases, this is the same
[R] readability

   Buffer Size. In certain switch architecture called SoC (switch on
   chip), there is a concept of port buffer and shared buffer pool
[O] there is a concept
[R] ? in certain switch architecture there is a concept? Or is there an
actual port buffer?

   available for all ports. Maximum Port Buffer, defines the scenario of
   a SoC buffer, where this amount in B (byte), KB (kilobyte), MB
   (megabyte) or GB (gigabyte) would represent the sum of the port
   buffer along with the maximum value of shared buffer this given port
   can take. The Maximum Port Buffer Size needs to be expressed along
[O] Maximum Port Buffer, defines the scenario of
   a SoC buffer, where this amount in B (byte), KB (kilobyte), MB
   (megabyte) or GB (gigabyte) would represent the sum of the port
   buffer along with the maximum value of shared buffer this given port
   can take.
[P] Maximum Port Buffer, in terms of an SoC buffer, represents the sum of
the port buffer and the maximum value of shared buffer allowed for this
port, defined in terms of B (byte), KB (kilobyte), MB (megabyte), or GB
(gigabyte).
[R] readability - I think. I has a hard time parsing the previous sentence.

   with the frame MTU used for the measurement and the cos or dscp bit
   value set for the test.

   Example: a DUT has been measured to have 3KB of port buffer for 1518
   frame size packets and a total of 4.7 MB of maximum port buffer for
   1518 frame size packets and a cos of 0.

   Maximum DUT Buffer Size: this is the total size of Buffer a DUT can
   be measured to have. It is most likely different than the Maximum

[O] It is most likely different than
[P] either: It is, most likely, different to [OR] It is probably different
to
[R] grammar/readability

   Port Buffer Size. It CAN also be different from the sum of Maximum
   Port Buffer Size. The Maximum Buffer Size needs to be expressed along
   with the frame MTU used for the measurement and along with the cos or
   dscp value set during the test.

   Example: a DUT has been measured to have 3KB of port buffer for 1518
   frame size packets and a total of 4.7 MB of maximum port buffer for
   1518 frame size packets. The DUT has a Maximum Buffer Size of 18 MB
   at 1500 bytes and a cos of 0.

   Burst: The burst is a fixed number of packets sent over a percentage
   of linerate of a defined port speed. The amount of frames sent are
   evenly distributed across the interval T. A constant C, can be
[O] interval T. A constant C,
[P] interval, T. A constant, C,
[R] clarity

   defined to provide the average time between two consecutive packets
   evenly spaced.


6.1.2 Discussion

   When measuring buffering on a DUT, it is important to understand what
   the behavior is for each port, and also for all ports as this will
   provide an evidence of the total amount of buffering available on the
   switch. The terms of buffer efficiency here helps one understand what
[O] When measuring buffering on a DUT, it is important to understand what
   the behavior is for each port, and also for all ports as this will
   provide an evidence of the total amount of buffering available on the
   switch.
[P] When measuring buffering on a DUT, it is important to understand the
behavior for each and all ports. This provides data for the total amount of
buffering available on the switch.

   is the optimum packet size for the buffer to be used, or what is the
   real volume of buffer available for a specific packet size. This
[O] what is the optimum packet size for the buffer to be used, or what is
the
   real volume of buffer available for a specific packet size.
[P] the optimum packet size for the buffer, or the real volume of the
buffer available for a specific packet size
[R] readability

   section does not discuss how to conduct the test methodology, it
   rather explains the buffer definitions and what metrics should be
[O] , it rather explains
[P] ; instead, it explains
   provided for a comprehensive data center device buffering
   benchmarking.

6.1.3 Measurement Units

   When Buffer is measured:-the buffer size MUST be measured-the port
   buffer size MAY be provided for each port-the maximum port buffer
   size MUST be measured-the maximum DUT buffer size MUST be measured-
   the intensity of microburst MAY be mentioned when a microburst test
   is performed-the cos or dscp value set during the test SHOULD be
   provided

[O] entire paragraph above
[R] this is very difficult to read with the hyphens. Guessing this was
intended to be a list, but the XML got munged?

6.2 Incast
6.2.1 Definition

   The term Incast, very commonly utilized in the data center, refers to
   the traffic pattern of many-to-one or many-to-many conversations.
   Typically in the data center it would refer to many different ingress
   server ports(many), sending traffic to a common uplink (one), or
[O] ports(many)
[P] ports (many)

   multiple uplinks (many). This pattern is generalized for any network
   as many incoming ports sending traffic to one or few uplinks. It can
   also be found in many-to-many traffic patterns.


6.2.2 Discussion


   In this scenario, buffers are solicited on the DUT. In a ingress
   buffering mechanism, the ingress port buffers would be solicited
   along with Virtual Output Queues, when available; whereas in an
   egress buffer mechanism, the egress buffer of the one outgoing port
   would be used.

   In either cases, regardless of where the buffer memory is located on
   the switch architecture; the Incast creates buffer utilization.
[O] In either cases, regardless of where the buffer memory is located on
   the switch architecture; the Incast creates buffer utilization.
[P] In either case, regardless of where the buffer memory is located in the
switch architecture, the Incast creates the buffer utilization.
[R] grammar


   When one or more frames having synchronous arrival times at the DUT
   they are considered forming an incast.
[O] incast
[P] Incast
[R] consistency


7 Application Throughput: Data Center Goodput

7.3. Measurement Units

   Example: a TCP file transfer over HTTP protocol on a 10Gb/s media.
   The file cannot be transferred over Ethernet as a single continuous
   stream. It must be broken down into individual frames of 1500 bytes
   when the standard MTU [Maximum Transmission Unit] is used. Each
   packet requires 20 bytes of IP header information and 20 bytes of TCP
   header information, therefore 1460 byte are available per packet for
[O] information, therefore
[P] information; therefore,
[R] grammar

   the file transfer. Linux based systems are further limited to 1448
   bytes as they also carry a 12 byte timestamp. Finally, the date is
   transmitted in this example over Ethernet which adds a 26 byte
   overhead per packet.



-- 
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

--001a1143881ebb19ff0550358302
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:verdana,=
sans-serif">Hi there,</div><div class=3D"gmail_default" style=3D"font-famil=
y:verdana,sans-serif"><br></div><div class=3D"gmail_default" style=3D"font-=
family:verdana,sans-serif">Apologies, I had forgotten to click the AD eval =
button... {DONE}=C2=A0</div><div class=3D"gmail_default" style=3D"font-fami=
ly:verdana,sans-serif"><br></div><div class=3D"gmail_default" style=3D"font=
-family:verdana,sans-serif">Thanks for a (generally) clear and useful docum=
ent.=C2=A0</div><div class=3D"gmail_default" style=3D"font-family:verdana,s=
ans-serif"><br></div><div class=3D"gmail_default" style=3D"font-family:verd=
ana,sans-serif">I have completed my AD review of=C2=A0draft-ietf-bmwg-dcben=
ch-terminology and I have a number of suggested edits to make the document =
even more readable / clearer. In a few instances I wasn&#39;t quite sure wh=
at the document was trying to say, so please let me know if any of the sugg=
estions don&#39;t make sense, or change the text meaning...</div><div class=
=3D"gmail_default" style=3D"font-family:verdana,sans-serif"><br></div><div =
class=3D"gmail_default" style=3D"font-family:verdana,sans-serif">Also, plea=
se poke me once updated, so that I can progress it to IETF LC.</div><div cl=
ass=3D"gmail_default" style=3D"font-family:verdana,sans-serif"><br></div><d=
iv class=3D"gmail_default" style=3D"font-family:verdana,sans-serif">W</div>=
<div class=3D"gmail_default" style=3D"font-family:verdana,sans-serif">-----=
-</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br>=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">Abstract</font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif">The purpose of this informational document is to establis=
h definitions,</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">discussion and measurement techniques for data center bench=
marking.</font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif">[O] establish definitions, discussion and measurement techniques<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>[R] I&#39;m not sure what is meant by &quot;discussion&quot; here. &quot;D=
iscussion and measurement techniques&quot; doesn&#39;t make much sense to m=
e. Perhaps &quot;to establish definitions, foster discussion and describe m=
easurement techniques ...&quot;?</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">Also, it is to introd=
uce new terminologies applicable to data center</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">[O] benchmarking. Also, it=
 is to introduce</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif">[P] benchmarking, as well as introduce</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] clarity/no ne=
ed for a separate sentence.</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">performance evaluations. The purpose of =
this document is not to define</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif">the test methodology, but rather establish =
the important concepts when</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">one is interested in benchmarking network swit=
ches and routers in the</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">[O] when one is interested in benchmarking</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] fo=
r benchmarking</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">[R] readability</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">data center.</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></di=
v><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">1.=
=C2=A0 Introduction</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">=C2=A0 =C2=A0Traffic patterns in the data center=
 are not uniform and are contently</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">[O] contently</font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif">[P] constantly</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] word=
 choice/typo</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif">=C2=A0 =C2=A0changing. They are dictated by the nature =
and variety of applications</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">=C2=A0 =C2=A0utilized in the data center. It c=
an be largely east-west traffic</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0flows in one data center and =
north-south in another, while some may</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0combine both. Traffic =
patterns can be bursty in nature and contain</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0many-to-one, man=
y-to-many, or one-to-many flows. Each flow may also</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0be small=
 and latency sensitive or large and throughput sensitive</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0whil=
e containing a mix of UDP and TCP traffic. All of which can</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0c=
oexist in a single cluster and flow through a single network device</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0all at the same time. Benchmarking of network devices have long used=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">[O] All of which can</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0coexist in a single cluster and flow =
through a single network device</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0all at the same time.</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] On=
e or more of these may coexist in a single cluster and flow through a singl=
e network device simultaneously.</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">[R] grammar/readability - unless this cha=
nges the intent?</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0[RFC1242], [RFC2432], [RFC2544], [2] =
and [3]. These benchmarks have</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif">=C2=A0 =C2=A0largely been focused around va=
rious latency attributes and max</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0throughput of the Device Und=
er Test being benchmarked. These</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0standards are good at measur=
ing theoretical max throughput,</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0forwarding rates and latency =
under testing conditions, but to not</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">[O] , but to not</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">[P]; but they do no=
t</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">[R] grammar</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif">=C2=A0 =C2=A0represent real traffic patterns that may=
 affect these networking</font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">=C2=A0 =C2=A0devices. The data center networking =
devices covered are switches and</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0routers.</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
1.2. Definition format</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0Term to be defined. (e.g., Late=
ncy)</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">=C2=A0 =C2=A0Definition: The specific definition for the term.<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-=
serif">=C2=A0 =C2=A0Discussion: A brief discussion about the term, it&#39;s=
 application and</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif">[O] it&#39;s</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif">[P] its</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">[R] Grammar. Should be a possessiv=
e, not a contraction of &quot;it is.&quot;</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0any restrictions o=
n measurement procedures.</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0Measurement Units: Methodolog=
y for the measure and units used to</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0report measurements of th=
is term, if applicable.</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">2.=C2=A0 Latency</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">2.1. Def=
inition</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">=C2=A0 =C2=A0Latency is a the amount of time it takes a fram=
e to transit the DUT.</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0Latency is measured in unit of time (=
seconds, milliseconds,</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0microseconds and so on). The purpose =
of measuring latency is to</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">=C2=A0 =C2=A0understand what is the impact of a=
dding a device in the communication</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">[O] =C2=A0understand what is the impac=
t</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">[P] understand the impact</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif">[R] grammar</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0path.</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0The Latency interval can be assessed between different combina=
tions</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-=
serif">=C2=A0 =C2=A0of events, irrespectively of the type of switching devi=
ce (bit</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">[O] irrespectively</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[P] regardless</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0forwarding aka cut-through or store forward type of device)</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">2.2 Discussion</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0FILO is the most important me=
asuring definition. Any type of switches</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">[O] Any type of switches</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] All=
 switches</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[R] readability</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0MUST be measured with the FI=
LO mechanism: FILO will include the</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0latency of the switch and=
 the latency of the frame as well as the</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0serialization delay.=
 It is a picture of the &#39;whole&#39; latency going</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0through=
 the DUT. For applications, which are latency sensitive and</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0c=
an function with initial bytes of the frame, FIFO MAY be an</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0a=
dditional type of measuring to supplement FILO.=C2=A0</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">3 Jitter=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">3.2 Discussion</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0In addition to PDV Range and or a hig=
h percentile of PDV, Inter-</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[O] =C2=A0and or</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">[P] and/or</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0Packet Delay Variation (IPDV) as defined in section 4.1 of RFC5481</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0(differences between two consecutive packets) MAY be used for the=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0purpose of determining how packet spacing has changed during=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0transfer, for example to see if packet stream has become clo=
sely-</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-=
serif">[O] for example to see</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">[P] for example, to see</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] grammar</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0spaced or &quot;bursty&quot;. However, the Absolute Value of I=
PDV SHOULD NOT be</font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif">=C2=A0 =C2=A0used, as this collapses the &quot;bursty&qu=
ot; and &quot;dispersed&quot; sides of the</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0IPDV distribution =
together.</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">4 Physical Layer Calibration</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">4.2 Discussion</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0Physical layer calibration is part of the end to end latency, which<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0should be taken into acknowledgment while evaluating the DUT.=
 Small</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0variations of the physical components of the test may =
impact the</font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">=C2=A0 =C2=A0latency being measure so they MUST be described wh=
en presenting</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">[O] latency being measure so</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">[P] latency being measured, so=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">[R] grammar</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0results.</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">5 Line rate=C2=A0</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
<br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">5.1 Definition</font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">=C2=A0 =C2=A0The transmit timing, or maximum tr=
ansmitted data rate is controlled</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">=C2=A0 =C2=A0by the &quot;transmit clock=
&quot; in the DUT.=C2=A0 The receive timing (maximum</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0ingress =
data rate) is derived from the transmit clock of the</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0connecte=
d interface.</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif">=C2=A0 =C2=A0The line rate or physical layer frame rate=
 is the maximum capacity to</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">=C2=A0 =C2=A0send frames of a specific size at=
 the transmit clock frequency of the</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0DUT.</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0The term port capacity term defines the maximum speed capability for</fo=
nt></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O=
] The term port capacity term</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">[P] The term &quot;port capacity&quot;</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] =
readability</font></div><div class=3D"gmail_default"><font face=3D"verdana,=
 sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif">=C2=A0 =C2=A0the given port; for example 1GE, 10GE, 40GE=
, 100GE etc.=C2=A0</font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">=C2=A0 =C2=A0The frequency (&quot;clock rate&quot=
;) of the transmit clock in any two</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0connected interfaces will=
 never be precisely the same, therefore a</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0tolerance is needed=
, this will be expressed by Parts Per Million</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">[O] same, therefore a tolera=
nce is needed, this will</font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">[P] same; therefore, a tolerance is needed. This =
will</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">[R] grammar</font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">5.2 Discussion</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0For a tr=
ansmit clock source, most Ethernet switches use &quot;clock</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0m=
odules&quot; (also called &quot;oscillator modules&quot;) that are sealed,<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0internally temperature-compensated, and very accurate. The ou=
tput</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0frequency of these modules is not adjustable because it =
is not</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0necessary.=C2=A0 Many test sets, however, offer a soft=
ware-controlled</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">=C2=A0 =C2=A0adjustment of the transmit clock rate, which =
should be used to</font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif">=C2=A0 =C2=A0compensate the test equipment to not send m=
ore than line rate of the</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">=C2=A0 =C2=A0DUT.</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] which should be u=
sed to</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0compensate the test equipment to not send more than li=
ne rate of the</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0DUT.</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">[R] please reword for clarity.</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></fo=
nt></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0To allow for the minor variations typically found in the clock=
 rate</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-=
serif">=C2=A0 =C2=A0of commercially-available clock modules and other cryst=
al-based</font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif">=C2=A0</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0Test set equipment manufacturers are =
well-aware of the standards, and</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0allows a software-controlled=
 +/- 100 PPM &quot;offset&quot; (clock-rate</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">[O] allows</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] allow</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] gra=
mmar</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">=C2=A0 =C2=A0adjustment) to compensate for normal variations in=
 the clock speed of</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0&quot;devices under test&quot;. This offs=
et adjustment allows engineers to</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">=C2=A0 =C2=A0determine the approximate s=
peed the connected device is operating,</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0and verify that it is=
 within parameters allowed by standards.</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">5.3 Measurement=
 Units</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif">=C2=A0 =C2=A0In a production network, it is very unlike=
ly to see precise line rate</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">=C2=A0 =C2=A0over a very brief period. There i=
s no observable difference between</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0dropping packets at 99% of=
 line rate and 100% of line rate. -Line</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0rate CAN measured at =
100% of line rate with a -100PPM adjustment. -</font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Line rate SHOU=
LD be measured at 99,98% with 0 PPM adjustment.-The PPM</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] .-The</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] . T=
he</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">[R] Typo Or was this intended to be a list?</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif"><br></font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0adjustm=
ent SHOULD only be used for a line rate type of measurement</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></fo=
nt></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">6 =
=C2=A0Buffering</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">6.1 Buffer</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">6.1.1 Definition</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Buffer S=
ize: the term buffer size, represents the total amount of</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] the term buf=
fer size, represents=C2=A0</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">[P] the term buffer size represents</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] gramma=
r</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">=C2=A0 =C2=A0frame buffering memory available on a DUT. This size =
is expressed in</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">=C2=A0 =C2=A0Byte; KB (kilobytes), MB (megabytes) or GB (g=
igabyte). When the</font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif">=C2=A0 =C2=A0buffer size is expressed it SHOULD be defi=
ned by a size metric</font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif">=C2=A0 =C2=A0defined above. When the buffer size is e=
xpressed, an indication of</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">[O] =C2=A0defined above</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] listed above [OR]=
 stated above</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">[R] the sizes are not &quot;defined&quot; above</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0the frame MTU used for that measurement is also necessary as well=
 as</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">=C2=A0 =C2=A0the cos or dscp value set; as often times the buffers are=
 carved by</font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">=C2=A0 =C2=A0quality of service implementation. (please refer t=
o the buffer</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">[O] . (please</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[P] . (Please</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0efficiency section for further details).</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Example: Bu=
ffer Size of DUT when sending 1518 bytes frames is 18 Mb.</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0Port Buffer Size: the port buffer size is the amount of buffer a</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0single ingress port, egress port or combination of ingress and egres=
s</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">=C2=A0 =C2=A0buffering location for a single port. The reason of mention=
ing the</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">[O] reason of mentioning</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">[P] reason for mentioning</font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] grammar</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
<br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0three locations for the port buffer is, that the DUT buf=
fering scheme</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">[O] is, that</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[P] is because</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0can be unknown or untested, and therefore the indication of where=
 the</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0buffer is located helps understand the buffer architectu=
re and</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">[O] =C2=A0and therefore the indication of where the</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0b=
uffer is located helps understand the buffer architecture</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] and so knowi=
ng the buffer location helps clarify the buffer architecture</font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] readabili=
ty=C2=A0</font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0therefore the total buffer size. The Port Buff=
er Size is an</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">=C2=A0 =C2=A0informational value that MAY be provided from t=
he DUT vendor. It is</font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif">=C2=A0 =C2=A0not a value that is tested by benchmarki=
ng. Benchmarking will be done</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0using the Maximum Port Buffer S=
ize or Maximum Buffer Size</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">=C2=A0 =C2=A0methodology.</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0M=
aximum Port Buffer Size: this is in most cases the same as the Port</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] th=
is is in most cases the same=C2=A0</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">[P] in most cases, this is the same</fo=
nt></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R=
] readability</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0Buffer Size. In certain switch architectu=
re called SoC (switch on</font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">=C2=A0 =C2=A0chip), there is a concept of port bu=
ffer and shared buffer pool</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[O] there is a concept</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] ? in certain swit=
ch architecture there is a concept? Or is there an actual port buffer?</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br=
></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">=C2=A0 =C2=A0available for all ports. Maximum Port Buffer, defines the s=
cenario of</font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">=C2=A0 =C2=A0a SoC buffer, where this amount in B (byte), KB (k=
ilobyte), MB</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">=C2=A0 =C2=A0(megabyte) or GB (gigabyte) would represent the =
sum of the port</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">=C2=A0 =C2=A0buffer along with the maximum value of shared=
 buffer this given port</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0can take. The Maximum Port Buffer Siz=
e needs to be expressed along</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">[O] Maximum Port Buffer, defines the scenari=
o of</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0a SoC buffer, where this amount in B (byte), KB (kilobyt=
e), MB</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0(megabyte) or GB (gigabyte) would represent the sum of=
 the port</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">=C2=A0 =C2=A0buffer along with the maximum value of shared buffe=
r this given port</font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif">=C2=A0 =C2=A0can take.</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">[P] Maximum Port Buffer, in term=
s of an SoC buffer, represents the sum of the port buffer and the maximum v=
alue of shared buffer allowed for this port, defined in terms of B (byte), =
KB (kilobyte), MB (megabyte), or GB (gigabyte).</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">[R] readability - I think.=
 I has a hard time parsing the previous sentence.</font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0with =
the frame MTU used for the measurement and the cos or dscp bit</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0value set for the test.</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Example: a DUT has been meas=
ured to have 3KB of port buffer for 1518</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0frame size packets a=
nd a total of 4.7 MB of maximum port buffer for</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A01518 frame si=
ze packets and a cos of 0.</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Maximum DUT Buffer Size: thi=
s is the total size of Buffer a DUT can</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0be measured to have. =
It is most likely different than the Maximum</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] It is most likely=
 different than</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">[P] either: It is, most likely, different to [OR] It is pr=
obably different to</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">[R] grammar/readability</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Port Buf=
fer Size. It CAN also be different from the sum of Maximum</font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Po=
rt Buffer Size. The Maximum Buffer Size needs to be expressed along</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0with the frame MTU used for the measurement and along with the cos o=
r</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">=C2=A0 =C2=A0dscp value set during the test.</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif"><br></font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Example=
: a DUT has been measured to have 3KB of port buffer for 1518</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0frame size packets and a total of 4.7 MB of maximum port buffer for</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A01518 frame size packets. The DUT has a Maximum Buffer Size of 18 =
MB</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">=C2=A0 =C2=A0at 1500 bytes and a cos of 0.</font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Burst: T=
he burst is a fixed number of packets sent over a percentage</font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0=
of linerate of a defined port speed. The amount of frames sent are</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0evenly distributed across the interval T. A constant C, can be</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] i=
nterval T. A constant C,</font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">[P] interval, T. A constant, C,=C2=A0</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] clarit=
y</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">=C2=A0 =C2=A0defined to provide the average time between two conse=
cutive packets</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0evenly spaced.</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">6.1.2 Discussion</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
<br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0When measuring buffering on a DUT, it is important to un=
derstand what</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">=C2=A0 =C2=A0the behavior is for each port, and also for all=
 ports as this will</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0provide an evidence of the total amount o=
f buffering available on the</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif">=C2=A0 =C2=A0switch. The terms of buffer effi=
ciency here helps one understand what</font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">[O] When measuring buffering on a DU=
T, it is important to understand what</font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the behavior is for eac=
h port, and also for all ports as this will</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0provide an eviden=
ce of the total amount of buffering available on the</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0switch.<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>[P] When measuring buffering on a DUT, it is important to understand the b=
ehavior for each and all ports. This provides data for the total amount of =
buffering available on the switch.</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0is the optimum packe=
t size for the buffer to be used, or what is the</font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0real volume =
of buffer available for a specific packet size. This</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] what is the optim=
um packet size for the buffer to be used, or what is the</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0real=
 volume of buffer available for a specific packet size.=C2=A0</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] the opti=
mum packet size for the buffer, or the real volume of the buffer available =
for a specific packet size</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">[R] readability</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0section doe=
s not discuss how to conduct the test methodology, it</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0rather =
explains the buffer definitions and what metrics should be</font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] , it rather=
 explains</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[P] ; instead, it explains</font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0provided for a comprehe=
nsive data center device buffering</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0benchmarking.</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">6.1.3=
 Measurement Units</font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">=C2=A0 =C2=A0When Buffer is measured:-the buffer =
size MUST be measured-the port</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif">=C2=A0 =C2=A0buffer size MAY be provided fo=
r each port-the maximum port buffer</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0size MUST be measured-the=
 maximum DUT buffer size MUST be measured-</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the intensity of m=
icroburst MAY be mentioned when a microburst test</font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0is performe=
d-the cos or dscp value set during the test SHOULD be</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0provide=
d</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">[O] entire paragraph above</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">[R] this is very difficult to read wit=
h the hyphens. Guessing this was intended to be a list, but the XML got mun=
ged?</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">6.2 Incast</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">6.2.1 Definition=C2=A0</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif"><br></font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0The ter=
m Incast, very commonly utilized in the data center, refers to</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0the traffic pattern of many-to-one or many-to-many conversations.</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0Typically in the data center it would refer to many different ing=
ress</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0server ports(many), sending traffic to a common uplink (=
one), or</font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif">[O] ports(many)</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">[P] ports (many)</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0multiple up=
links (many). This pattern is generalized for any network</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0as =
many incoming ports sending traffic to one or few uplinks. It can</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0also be found in many-to-many traffic patterns.</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">6.2.2 Di=
scussion</font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif">=C2=A0 =C2=A0In this scenario, buffers are solicited =
on the DUT. In a ingress</font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">=C2=A0 =C2=A0buffering mechanism, the ingress por=
t buffers would be solicited</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif">=C2=A0 =C2=A0along with Virtual Output Queues=
, when available; whereas in an</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0egress buffer mechanism, the =
egress buffer of the one outgoing port</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0would be used.</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0In either cases, regardless of where the buffer memory is loca=
ted on</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0the switch architecture; the Incast creates buffer uti=
lization.</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[O] In either cases, regardless of where the buffer memory is lo=
cated on</font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif">=C2=A0 =C2=A0the switch architecture; the Incast creates buffer u=
tilization.</font></div><div class=3D"gmail_default"><font face=3D"verdana,=
 sans-serif">[P] In either case, regardless of where the buffer memory is l=
ocated in the switch architecture, the Incast creates the buffer utilizatio=
n.</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">[R] grammar</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0When one or more frames having =
synchronous arrival times at the DUT</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0they are considered form=
ing an incast.</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">[O] incast</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">[P] Incast</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">[R] consistency</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">7 Appl=
ication Throughput: Data Center Goodput</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">7.3. Measurement Units</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br>=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0Example: a TCP file transfer over HTTP protocol on a 10Gb/s =
media.</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0The file cannot be transferred over Ethernet as a sing=
le continuous</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">=C2=A0 =C2=A0stream. It must be broken down into individual =
frames of 1500 bytes</font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif">=C2=A0 =C2=A0when the standard MTU [Maximum Transmiss=
ion Unit] is used. Each</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0packet requires 20 bytes of IP header=
 information and 20 bytes of TCP</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0header information, therefor=
e 1460 byte are available per packet for</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">[O] information, therefore=C2=A0<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>[P] information; therefore,</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif">[R] grammar</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the file tran=
sfer. Linux based systems are further limited to 1448</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0bytes a=
s they also carry a 12 byte timestamp. Finally, the date is</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0t=
ransmitted in this example over Ethernet which adds a 26 byte</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0overhead per packet.</font></div><div class=3D"gmail_default" style=3D"f=
ont-family:verdana,sans-serif"><br></div><div class=3D"gmail_default" style=
=3D"font-family:verdana,sans-serif"><br></div><div><br></div>-- <br><div cl=
ass=3D"gmail_signature">I don&#39;t think the execution is relevant when it=
 was obviously a bad idea in the first place.<br>This is like putting rabid=
 weasels in your pants, and later expressing regret at having chosen those =
particular rabid weasels and that pair of pants.<br>=C2=A0 =C2=A0---maf</di=
v>
</div>

--001a1143881ebb19ff0550358302--


From nobody Wed May 24 22:20:44 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 899451271FD; Wed, 24 May 2017 22:20:43 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.51.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149568964344.8641.4937633232683690534@ietfa.amsl.com>
Date: Wed, 24 May 2017 22:20:43 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2YAvXN6Q6DtGPcTLNy8E7GBIgj8>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-09.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 25 May 2017 05:20:43 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-09.txt
	Pages           : 17
	Date            : 2017-05-24

Abstract:
The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-09
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-09

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-09


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Thu May 25 04:27:23 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8A0671271FD for <bmwg@ietfa.amsl.com>; Wed, 24 May 2017 22:21:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level: 
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fYOq0XZ6libq for <bmwg@ietfa.amsl.com>; Wed, 24 May 2017 22:21:47 -0700 (PDT)
Received: from mail-pf0-x230.google.com (mail-pf0-x230.google.com [IPv6:2607:f8b0:400e:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC907129AF9 for <bmwg@ietf.org>; Wed, 24 May 2017 22:21:45 -0700 (PDT)
Received: by mail-pf0-x230.google.com with SMTP id 9so156273164pfj.1 for <bmwg@ietf.org>; Wed, 24 May 2017 22:21:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=O7Vz4ANQZEpoXUd7FwYuO50tVU4dF69/WqGHps4JZwU=; b=WTYvgkKW9Q6nmnKrBwSJZcp9s3KTM6Dl42ogWBp8ps7y6zDUEFK/LpXZQfi532j5B4 QP3q+hjTsMk4gdcowZEFnJJfWSibN1HlsWmf5hxCHC/PVS1r2tQxN1y9rEeShAXZYtEA s5SUqh+9/r1HhpaGRwsLHkmf7xayRuQxDgM0QMDKX+nMyjKSmAS1yCF+TS4449BKnD0r FGLRJ8bXAcLQaMGu2kyG9Bop2X18UxtyPU7+mFlcvOU2YHgcPGtqZtFXOC4kjpsvbOlM sDYS6hG1OzDdMRoEQaAH/KoRdpEu276uTjJK4u3EA1v+bvwIKD3njSXJa+SJ8ICv9/bB a/9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=O7Vz4ANQZEpoXUd7FwYuO50tVU4dF69/WqGHps4JZwU=; b=S8euqNv0X3JExYGcnxkUjdYCWrcs20Z2N76X2gV8po0dSYYo4fXlPWoa3i2pN4lp20 bY0QYqfmTBydJB8ROWtfIitlok64kc8BqZA9pBsmDHsFkeWm4OD3fH/o+3rDw6BrF9og IlMfaIU76RgQqlotBp66tjJSHd89kfntedD8tAopPuvJonPgSiuLRK6L8YoS829KnNu4 8ZN8umOYFlEIQmFaqkh/s0WxrF99877TAZBs5Std0GBjbYcaTMsH6CjmT3D88G1+4zF2 EnZR9VOKIdwU5+1P72a0n6HxgFVYL5lK39uLawBBrpvwss8mH5YsHU2txQCh2nYH+3RL sDkg==
X-Gm-Message-State: AODbwcD8J1pVXRokwbQ2CxWrmnqT0rWT2CJFWqAdJJJ0IucKRHU3P3WD uYi/nLsDDnV/1Y2swUupLnbPZwmn1An/
X-Received: by 10.98.194.132 with SMTP id w4mr42354139pfk.176.1495689705146; Wed, 24 May 2017 22:21:45 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.100.138.38 with HTTP; Wed, 24 May 2017 22:21:24 -0700 (PDT)
In-Reply-To: <CAHw9_i+yE8a02bP=NQym6nxUhCJM6OOdhkRwv=2GamQjV+KS1A@mail.gmail.com>
References: <CAHw9_i+yE8a02bP=NQym6nxUhCJM6OOdhkRwv=2GamQjV+KS1A@mail.gmail.com>
From: Lucien Avramov <lucienav@google.com>
Date: Wed, 24 May 2017 22:21:24 -0700
Message-ID: <CALTEt=B0LSj_zAzBH5PUEdkMQ1DeU0w4sTv9xpWqSp3WDCdB_Q@mail.gmail.com>
To: Warren Kumari <warren@kumari.net>
Cc: draft-ietf-bmwg-dcbench-terminology@ietf.org, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c18e3b69231c505505265c1"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/b2DWb2CuG8_WCfetwRIeFDQkcL8>
X-Mailman-Approved-At: Thu, 25 May 2017 04:27:21 -0700
Subject: Re: [bmwg] AD review of draft-ietf-bmwg-dcbench-terminology.
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 25 May 2017 05:21:50 -0000

--94eb2c18e3b69231c505505265c1
Content-Type: text/plain; charset="UTF-8"

Thanks Warren!!

Submited -09 with all the modifications!

Cheers,
Lucien

On Tue, May 23, 2017 at 11:53 AM, Warren Kumari <warren@kumari.net> wrote:

> Hi there,
>
> Apologies, I had forgotten to click the AD eval button... {DONE}
>
> Thanks for a (generally) clear and useful document.
>
> I have completed my AD review of draft-ietf-bmwg-dcbench-terminology and
> I have a number of suggested edits to make the document even more readable
> / clearer. In a few instances I wasn't quite sure what the document was
> trying to say, so please let me know if any of the suggestions don't make
> sense, or change the text meaning...
>
> Also, please poke me once updated, so that I can progress it to IETF LC.
>
> W
> ------
>
> Abstract
>
> The purpose of this informational document is to establish definitions,
> discussion and measurement techniques for data center benchmarking.
> [O] establish definitions, discussion and measurement techniques
> [R] I'm not sure what is meant by "discussion" here. "Discussion and
> measurement techniques" doesn't make much sense to me. Perhaps "to
> establish definitions, foster discussion and describe measurement
> techniques ..."?
>
>
>
> Also, it is to introduce new terminologies applicable to data center
> [O] benchmarking. Also, it is to introduce
> [P] benchmarking, as well as introduce
> [R] clarity/no need for a separate sentence.
>
> performance evaluations. The purpose of this document is not to define
> the test methodology, but rather establish the important concepts when
> one is interested in benchmarking network switches and routers in the
> [O] when one is interested in benchmarking
> [P] for benchmarking
> [R] readability
>
> data center.
>
>
>
> 1.  Introduction
>
>    Traffic patterns in the data center are not uniform and are contently
> [O] contently
> [P] constantly
> [R] word choice/typo
>
>    changing. They are dictated by the nature and variety of applications
>    utilized in the data center. It can be largely east-west traffic
>    flows in one data center and north-south in another, while some may
>    combine both. Traffic patterns can be bursty in nature and contain
>    many-to-one, many-to-many, or one-to-many flows. Each flow may also
>    be small and latency sensitive or large and throughput sensitive
>    while containing a mix of UDP and TCP traffic. All of which can
>    coexist in a single cluster and flow through a single network device
>    all at the same time. Benchmarking of network devices have long used
> [O] All of which can
>    coexist in a single cluster and flow through a single network device
>    all at the same time.
> [P] One or more of these may coexist in a single cluster and flow through
> a single network device simultaneously.
> [R] grammar/readability - unless this changes the intent?
>
>    [RFC1242], [RFC2432], [RFC2544], [2] and [3]. These benchmarks have
>    largely been focused around various latency attributes and max
>    throughput of the Device Under Test being benchmarked. These
>    standards are good at measuring theoretical max throughput,
>    forwarding rates and latency under testing conditions, but to not
> [O] , but to not
> [P]; but they do not
> [R] grammar
>
>    represent real traffic patterns that may affect these networking
>    devices. The data center networking devices covered are switches and
>    routers.
>
>
>
> 1.2. Definition format
>
>    Term to be defined. (e.g., Latency)
>
>    Definition: The specific definition for the term.
>
>    Discussion: A brief discussion about the term, it's application and
> [O] it's
> [P] its
> [R] Grammar. Should be a possessive, not a contraction of "it is."
>    any restrictions on measurement procedures.
>
>    Measurement Units: Methodology for the measure and units used to
>    report measurements of this term, if applicable.
>
>
> 2.  Latency
>
>
> 2.1. Definition
>
>    Latency is a the amount of time it takes a frame to transit the DUT.
>    Latency is measured in unit of time (seconds, milliseconds,
>    microseconds and so on). The purpose of measuring latency is to
>    understand what is the impact of adding a device in the communication
> [O]  understand what is the impact
> [P] understand the impact
> [R] grammar
>
>    path.
>
>    The Latency interval can be assessed between different combinations
>    of events, irrespectively of the type of switching device (bit
> [O] irrespectively
> [P] regardless
> [R] grammar
>
>    forwarding aka cut-through or store forward type of device)
>
>
>
> 2.2 Discussion
>
>    FILO is the most important measuring definition. Any type of switches
> [O] Any type of switches
> [P] All switches
> [R] readability
>
>    MUST be measured with the FILO mechanism: FILO will include the
>    latency of the switch and the latency of the frame as well as the
>    serialization delay. It is a picture of the 'whole' latency going
>    through the DUT. For applications, which are latency sensitive and
>    can function with initial bytes of the frame, FIFO MAY be an
>    additional type of measuring to supplement FILO.
>
>
> 3 Jitter
> 3.2 Discussion
>
>    In addition to PDV Range and or a high percentile of PDV, Inter-
> [O]  and or
> [P] and/or
>
>    Packet Delay Variation (IPDV) as defined in section 4.1 of RFC5481
>    (differences between two consecutive packets) MAY be used for the
>    purpose of determining how packet spacing has changed during
>    transfer, for example to see if packet stream has become closely-
> [O] for example to see
> [P] for example, to see
> [R] grammar
>
>    spaced or "bursty". However, the Absolute Value of IPDV SHOULD NOT be
>    used, as this collapses the "bursty" and "dispersed" sides of the
>    IPDV distribution together.
>
> 4 Physical Layer Calibration
> 4.2 Discussion
>
>    Physical layer calibration is part of the end to end latency, which
>    should be taken into acknowledgment while evaluating the DUT. Small
>    variations of the physical components of the test may impact the
>    latency being measure so they MUST be described when presenting
> [O] latency being measure so
> [P] latency being measured, so
> [R] grammar
>
>    results.
>
>
> 5 Line rate
>
> 5.1 Definition
>
>    The transmit timing, or maximum transmitted data rate is controlled
>    by the "transmit clock" in the DUT.  The receive timing (maximum
>    ingress data rate) is derived from the transmit clock of the
>    connected interface.
>
>    The line rate or physical layer frame rate is the maximum capacity to
>    send frames of a specific size at the transmit clock frequency of the
>    DUT.
>
>    The term port capacity term defines the maximum speed capability for
> [O] The term port capacity term
> [P] The term "port capacity"
> [R] readability
>
>    the given port; for example 1GE, 10GE, 40GE, 100GE etc.
>
>    The frequency ("clock rate") of the transmit clock in any two
>    connected interfaces will never be precisely the same, therefore a
>    tolerance is needed, this will be expressed by Parts Per Million
> [O] same, therefore a tolerance is needed, this will
> [P] same; therefore, a tolerance is needed. This will
> [R] grammar
>
>
> 5.2 Discussion
>
>    For a transmit clock source, most Ethernet switches use "clock
>    modules" (also called "oscillator modules") that are sealed,
>    internally temperature-compensated, and very accurate. The output
>    frequency of these modules is not adjustable because it is not
>    necessary.  Many test sets, however, offer a software-controlled
>    adjustment of the transmit clock rate, which should be used to
>    compensate the test equipment to not send more than line rate of the
>    DUT.
>
> [O] which should be used to
>    compensate the test equipment to not send more than line rate of the
>    DUT.
> [R] please reword for clarity.
>
>    To allow for the minor variations typically found in the clock rate
>    of commercially-available clock modules and other crystal-based
>
>
>    Test set equipment manufacturers are well-aware of the standards, and
>    allows a software-controlled +/- 100 PPM "offset" (clock-rate
> [O] allows
> [P] allow
> [R] grammar
>
>    adjustment) to compensate for normal variations in the clock speed of
>    "devices under test". This offset adjustment allows engineers to
>    determine the approximate speed the connected device is operating,
>    and verify that it is within parameters allowed by standards.
>
>
>
> 5.3 Measurement Units
>
>
>    In a production network, it is very unlikely to see precise line rate
>    over a very brief period. There is no observable difference between
>    dropping packets at 99% of line rate and 100% of line rate. -Line
>    rate CAN measured at 100% of line rate with a -100PPM adjustment. -
>    Line rate SHOULD be measured at 99,98% with 0 PPM adjustment.-The PPM
> [O] .-The
> [P] . The
> [R] Typo Or was this intended to be a list?
>
>    adjustment SHOULD only be used for a line rate type of measurement
>
>
> 6  Buffering
> 6.1 Buffer
> 6.1.1 Definition
>
>    Buffer Size: the term buffer size, represents the total amount of
> [O] the term buffer size, represents
> [P] the term buffer size represents
> [R] grammar
>
>    frame buffering memory available on a DUT. This size is expressed in
>    Byte; KB (kilobytes), MB (megabytes) or GB (gigabyte). When the
>    buffer size is expressed it SHOULD be defined by a size metric
>    defined above. When the buffer size is expressed, an indication of
> [O]  defined above
> [P] listed above [OR] stated above
> [R] the sizes are not "defined" above
>
>    the frame MTU used for that measurement is also necessary as well as
>    the cos or dscp value set; as often times the buffers are carved by
>    quality of service implementation. (please refer to the buffer
> [O] . (please
> [P] . (Please
> [R] grammar
>
>    efficiency section for further details).
>
>    Example: Buffer Size of DUT when sending 1518 bytes frames is 18 Mb.
>
>    Port Buffer Size: the port buffer size is the amount of buffer a
>    single ingress port, egress port or combination of ingress and egress
>    buffering location for a single port. The reason of mentioning the
> [O] reason of mentioning
> [P] reason for mentioning
> [R] grammar
>
>    three locations for the port buffer is, that the DUT buffering scheme
> [O] is, that
> [P] is because
> [R] grammar
>
>
>    can be unknown or untested, and therefore the indication of where the
>    buffer is located helps understand the buffer architecture and
> [O]  and therefore the indication of where the
>    buffer is located helps understand the buffer architecture
> [P] and so knowing the buffer location helps clarify the buffer
> architecture
> [R] readability
>
>    therefore the total buffer size. The Port Buffer Size is an
>    informational value that MAY be provided from the DUT vendor. It is
>    not a value that is tested by benchmarking. Benchmarking will be done
>    using the Maximum Port Buffer Size or Maximum Buffer Size
>    methodology.
>
>    Maximum Port Buffer Size: this is in most cases the same as the Port
> [O] this is in most cases the same
> [P] in most cases, this is the same
> [R] readability
>
>    Buffer Size. In certain switch architecture called SoC (switch on
>    chip), there is a concept of port buffer and shared buffer pool
> [O] there is a concept
> [R] ? in certain switch architecture there is a concept? Or is there an
> actual port buffer?
>
>    available for all ports. Maximum Port Buffer, defines the scenario of
>    a SoC buffer, where this amount in B (byte), KB (kilobyte), MB
>    (megabyte) or GB (gigabyte) would represent the sum of the port
>    buffer along with the maximum value of shared buffer this given port
>    can take. The Maximum Port Buffer Size needs to be expressed along
> [O] Maximum Port Buffer, defines the scenario of
>    a SoC buffer, where this amount in B (byte), KB (kilobyte), MB
>    (megabyte) or GB (gigabyte) would represent the sum of the port
>    buffer along with the maximum value of shared buffer this given port
>    can take.
> [P] Maximum Port Buffer, in terms of an SoC buffer, represents the sum of
> the port buffer and the maximum value of shared buffer allowed for this
> port, defined in terms of B (byte), KB (kilobyte), MB (megabyte), or GB
> (gigabyte).
> [R] readability - I think. I has a hard time parsing the previous sentence.
>
>    with the frame MTU used for the measurement and the cos or dscp bit
>    value set for the test.
>
>    Example: a DUT has been measured to have 3KB of port buffer for 1518
>    frame size packets and a total of 4.7 MB of maximum port buffer for
>    1518 frame size packets and a cos of 0.
>
>    Maximum DUT Buffer Size: this is the total size of Buffer a DUT can
>    be measured to have. It is most likely different than the Maximum
>
> [O] It is most likely different than
> [P] either: It is, most likely, different to [OR] It is probably different
> to
> [R] grammar/readability
>
>    Port Buffer Size. It CAN also be different from the sum of Maximum
>    Port Buffer Size. The Maximum Buffer Size needs to be expressed along
>    with the frame MTU used for the measurement and along with the cos or
>    dscp value set during the test.
>
>    Example: a DUT has been measured to have 3KB of port buffer for 1518
>    frame size packets and a total of 4.7 MB of maximum port buffer for
>    1518 frame size packets. The DUT has a Maximum Buffer Size of 18 MB
>    at 1500 bytes and a cos of 0.
>
>    Burst: The burst is a fixed number of packets sent over a percentage
>    of linerate of a defined port speed. The amount of frames sent are
>    evenly distributed across the interval T. A constant C, can be
> [O] interval T. A constant C,
> [P] interval, T. A constant, C,
> [R] clarity
>
>    defined to provide the average time between two consecutive packets
>    evenly spaced.
>
>
> 6.1.2 Discussion
>
>    When measuring buffering on a DUT, it is important to understand what
>    the behavior is for each port, and also for all ports as this will
>    provide an evidence of the total amount of buffering available on the
>    switch. The terms of buffer efficiency here helps one understand what
> [O] When measuring buffering on a DUT, it is important to understand what
>    the behavior is for each port, and also for all ports as this will
>    provide an evidence of the total amount of buffering available on the
>    switch.
> [P] When measuring buffering on a DUT, it is important to understand the
> behavior for each and all ports. This provides data for the total amount of
> buffering available on the switch.
>
>    is the optimum packet size for the buffer to be used, or what is the
>    real volume of buffer available for a specific packet size. This
> [O] what is the optimum packet size for the buffer to be used, or what is
> the
>    real volume of buffer available for a specific packet size.
> [P] the optimum packet size for the buffer, or the real volume of the
> buffer available for a specific packet size
> [R] readability
>
>    section does not discuss how to conduct the test methodology, it
>    rather explains the buffer definitions and what metrics should be
> [O] , it rather explains
> [P] ; instead, it explains
>    provided for a comprehensive data center device buffering
>    benchmarking.
>
> 6.1.3 Measurement Units
>
>    When Buffer is measured:-the buffer size MUST be measured-the port
>    buffer size MAY be provided for each port-the maximum port buffer
>    size MUST be measured-the maximum DUT buffer size MUST be measured-
>    the intensity of microburst MAY be mentioned when a microburst test
>    is performed-the cos or dscp value set during the test SHOULD be
>    provided
>
> [O] entire paragraph above
> [R] this is very difficult to read with the hyphens. Guessing this was
> intended to be a list, but the XML got munged?
>
> 6.2 Incast
> 6.2.1 Definition
>
>    The term Incast, very commonly utilized in the data center, refers to
>    the traffic pattern of many-to-one or many-to-many conversations.
>    Typically in the data center it would refer to many different ingress
>    server ports(many), sending traffic to a common uplink (one), or
> [O] ports(many)
> [P] ports (many)
>
>    multiple uplinks (many). This pattern is generalized for any network
>    as many incoming ports sending traffic to one or few uplinks. It can
>    also be found in many-to-many traffic patterns.
>
>
> 6.2.2 Discussion
>
>
>    In this scenario, buffers are solicited on the DUT. In a ingress
>    buffering mechanism, the ingress port buffers would be solicited
>    along with Virtual Output Queues, when available; whereas in an
>    egress buffer mechanism, the egress buffer of the one outgoing port
>    would be used.
>
>    In either cases, regardless of where the buffer memory is located on
>    the switch architecture; the Incast creates buffer utilization.
> [O] In either cases, regardless of where the buffer memory is located on
>    the switch architecture; the Incast creates buffer utilization.
> [P] In either case, regardless of where the buffer memory is located in
> the switch architecture, the Incast creates the buffer utilization.
> [R] grammar
>
>
>    When one or more frames having synchronous arrival times at the DUT
>    they are considered forming an incast.
> [O] incast
> [P] Incast
> [R] consistency
>
>
> 7 Application Throughput: Data Center Goodput
>
> 7.3. Measurement Units
>
>    Example: a TCP file transfer over HTTP protocol on a 10Gb/s media.
>    The file cannot be transferred over Ethernet as a single continuous
>    stream. It must be broken down into individual frames of 1500 bytes
>    when the standard MTU [Maximum Transmission Unit] is used. Each
>    packet requires 20 bytes of IP header information and 20 bytes of TCP
>    header information, therefore 1460 byte are available per packet for
> [O] information, therefore
> [P] information; therefore,
> [R] grammar
>
>    the file transfer. Linux based systems are further limited to 1448
>    bytes as they also carry a 12 byte timestamp. Finally, the date is
>    transmitted in this example over Ethernet which adds a 26 byte
>    overhead per packet.
>
>
>
> --
> I don't think the execution is relevant when it was obviously a bad idea
> in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair of
> pants.
>    ---maf
>

--94eb2c18e3b69231c505505265c1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks Warren!!<div><br></div><div>Submited -09 with all t=
he modifications!</div><div><br></div><div>Cheers,</div><div>Lucien</div></=
div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, May 2=
3, 2017 at 11:53 AM, Warren Kumari <span dir=3D"ltr">&lt;<a href=3D"mailto:=
warren@kumari.net" target=3D"_blank">warren@kumari.net</a>&gt;</span> wrote=
:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-le=
ft:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_de=
fault" style=3D"font-family:verdana,sans-serif">Hi there,</div><div class=
=3D"gmail_default" style=3D"font-family:verdana,sans-serif"><br></div><div =
class=3D"gmail_default" style=3D"font-family:verdana,sans-serif">Apologies,=
 I had forgotten to click the AD eval button... {DONE}=C2=A0</div><div clas=
s=3D"gmail_default" style=3D"font-family:verdana,sans-serif"><br></div><div=
 class=3D"gmail_default" style=3D"font-family:verdana,sans-serif">Thanks fo=
r a (generally) clear and useful document.=C2=A0</div><div class=3D"gmail_d=
efault" style=3D"font-family:verdana,sans-serif"><br></div><div class=3D"gm=
ail_default" style=3D"font-family:verdana,sans-serif">I have completed my A=
D review of=C2=A0draft-ietf-bmwg-dcbench-<wbr>terminology and I have a numb=
er of suggested edits to make the document even more readable / clearer. In=
 a few instances I wasn&#39;t quite sure what the document was trying to sa=
y, so please let me know if any of the suggestions don&#39;t make sense, or=
 change the text meaning...</div><div class=3D"gmail_default" style=3D"font=
-family:verdana,sans-serif"><br></div><div class=3D"gmail_default" style=3D=
"font-family:verdana,sans-serif">Also, please poke me once updated, so that=
 I can progress it to IETF LC.</div><div class=3D"gmail_default" style=3D"f=
ont-family:verdana,sans-serif"><br></div><div class=3D"gmail_default" style=
=3D"font-family:verdana,sans-serif">W</div><div class=3D"gmail_default" sty=
le=3D"font-family:verdana,sans-serif">------</div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">Abstract</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">The purpose of=
 this informational document is to establish definitions,</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif">discussion and m=
easurement techniques for data center benchmarking.</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] establish definit=
ions, discussion and measurement techniques</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">[R] I&#39;m not sure what is m=
eant by &quot;discussion&quot; here. &quot;Discussion and measurement techn=
iques&quot; doesn&#39;t make much sense to me. Perhaps &quot;to establish d=
efinitions, foster discussion and describe measurement techniques ...&quot;=
?</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">Also, it is to introduce new terminologies applicable =
to data center</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">[O] benchmarking. Also, it is to introduce</font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] benchmarkin=
g, as well as introduce</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">[R] clarity/no need for a separate sentence.</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br>=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">performance evaluations. The purpose of this document is not to define</f=
ont></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">t=
he test methodology, but rather establish the important concepts when</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">one =
is interested in benchmarking network switches and routers in the</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] when=
 one is interested in benchmarking</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">[P] for benchmarking</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] readability</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
<br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">data center.</font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">1.=C2=A0 Introduction</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0Traffic patterns in the data center are not uniform and are contently=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">[O] contently</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">[P] constantly</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">[R] word choice/typo</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0c=
hanging. They are dictated by the nature and variety of applications</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0utilized in the data center. It can be largely east-west traffic<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0flows in one data center and north-south in another, while so=
me may</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0combine both. Traffic patterns can be bursty in nature=
 and contain</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">=C2=A0 =C2=A0many-to-one, many-to-many, or one-to-many flows.=
 Each flow may also</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0be small and latency sensitive or large a=
nd throughput sensitive</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0while containing a mix of UDP and TCP=
 traffic. All of which can</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">=C2=A0 =C2=A0coexist in a single cluster and fl=
ow through a single network device</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0all at the same time. Benc=
hmarking of network devices have long used</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">[O] All of which can</font></di=
v><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0coexist in a single cluster and flow through a single network device<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0all at the same time.</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">[P] One or more of these may coexist i=
n a single cluster and flow through a single network device simultaneously.=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">[R] grammar/readability - unless this changes the intent?</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0[RFC1242], [RFC2432], [RFC2544], [2] and [3]. These benchmarks have</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0largely been focused around various latency attributes and max=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0throughput of the Device Under Test being benchmarked. These=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0standards are good at measuring theoretical max throughput,<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0forwarding rates and latency under testing conditions, but to=
 not</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">[O] , but to not</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">[P]; but they do not</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></di=
v><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0represent real traffic patterns that may affect these networking</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0devices. The data center networking devices covered are switches =
and</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">=C2=A0 =C2=A0routers.</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">1.2. Definition format</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br><=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0Term to be defined. (e.g., Latency)</font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Defin=
ition: The specific definition for the term.</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Discussi=
on: A brief discussion about the term, it&#39;s application and</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] it&#39=
;s</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">[P] its</font></div><div class=3D"gmail_default"><font face=3D"verdana,=
 sans-serif">[R] Grammar. Should be a possessive, not a contraction of &quo=
t;it is.&quot;</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0any restrictions on measurement procedures.</f=
ont></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><=
br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">=C2=A0 =C2=A0Measurement Units: Methodology for the measure and units =
used to</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">=C2=A0 =C2=A0report measurements of this term, if applicable.</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br=
></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">2.=C2=A0 Latency</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">2.1. Definition</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0=
Latency is a the amount of time it takes a frame to transit the DUT.</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0Latency is measured in unit of time (seconds, milliseconds,</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0microseconds and so on). The purpose of measuring latency is to</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0understand what is the impact of adding a device in the commun=
ication</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">[O] =C2=A0understand what is the impact</font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif">[P] understand the impact=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">[R] grammar</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0path.</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0The Latency int=
erval can be assessed between different combinations</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0of event=
s, irrespectively of the type of switching device (bit</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] irrespectively<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>[P] regardless</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">[R] grammar</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0forwarding aka cut-through=
 or store forward type of device)</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">=C2=A0</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif"><br></font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">2.2 Discussion</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br>=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0FILO is the most important measuring definition. Any type of=
 switches</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[O] Any type of switches</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">[P] All switches</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] readability</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br=
></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">=C2=A0 =C2=A0MUST be measured with the FILO mechanism: FILO will include=
 the</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0latency of the switch and the latency of the frame as we=
ll as the</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">=C2=A0 =C2=A0serialization delay. It is a picture of the &#39;wh=
ole&#39; latency going</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0through the DUT. For applications, wh=
ich are latency sensitive and</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0can function with initial bytes=
 of the frame, FIFO MAY be an</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0additional type of measuring to=
 supplement FILO.=C2=A0</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">3 Jitter</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">3.2 Discussion</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0In addition to PDV Range and or a high percentile of PDV, Inter-</fo=
nt></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O=
] =C2=A0and or</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">[P] and/or</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Packet Delay Variation (IPDV=
) as defined in section 4.1 of RFC5481</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0(differences between t=
wo consecutive packets) MAY be used for the</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0purpose of determ=
ining how packet spacing has changed during</font></div><div class=3D"gmail=
_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0transfer, for exa=
mple to see if packet stream has become closely-</font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif">[O] for example to see</f=
ont></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[=
P] for example, to see</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">[R] grammar</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0spaced or &quot;bu=
rsty&quot;. However, the Absolute Value of IPDV SHOULD NOT be</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0used, as this collapses the &quot;bursty&quot; and &quot;dispersed&quot;=
 sides of the</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">=C2=A0 =C2=A0IPDV distribution together.</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">4 Physical L=
ayer Calibration</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif">4.2 Discussion</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Physical layer calibra=
tion is part of the end to end latency, which</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0should be taken=
 into acknowledgment while evaluating the DUT. Small</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0variatio=
ns of the physical components of the test may impact the</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0late=
ncy being measure so they MUST be described when presenting</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] latency be=
ing measure so</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">[P] latency being measured, so</font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif">[R] grammar</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0results.</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">5 Line rate=C2=A0</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">5.1 Definition</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br><=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0The transmit timing, or maximum transmitted data rate is cont=
rolled</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0by the &quot;transmit clock&quot; in the DUT.=C2=A0 Th=
e receive timing (maximum</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">=C2=A0 =C2=A0ingress data rate) is derived from =
the transmit clock of the</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">=C2=A0 =C2=A0connected interface.</font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0The line rate or physical layer frame rate is the maximum capacity to=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0send frames of a specific size at the transmit clock frequen=
cy of the</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">=C2=A0 =C2=A0DUT.</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0The term port capacity ter=
m defines the maximum speed capability for</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">[O] The term port capacity term=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">[P] The term &quot;port capacity&quot;</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">[R] readability</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0the given port; for example 1GE, 10GE, 40GE, 100GE etc.=C2=A0</font></di=
v><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0The frequency (&quot;clock rate&quot;) of the transmit clock in a=
ny two</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0connected interfaces will never be precisely the same,=
 therefore a</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">=C2=A0 =C2=A0tolerance is needed, this will be expressed by P=
arts Per Million</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif">[O] same, therefore a tolerance is needed, this will</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P]=
 same; therefore, a tolerance is needed. This will</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] grammar</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></fo=
nt></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><b=
r></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">5.2 Discussion</font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">=C2=A0 =C2=A0For a transmit clock source, most Et=
hernet switches use &quot;clock</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0modules&quot; (also called &q=
uot;oscillator modules&quot;) that are sealed,</font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0internally tem=
perature-compensated, and very accurate. The output</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0frequenc=
y of these modules is not adjustable because it is not</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0necess=
ary.=C2=A0 Many test sets, however, offer a software-controlled</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0adjustment of the transmit clock rate, which should be used to</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0compensate the test equipment to not send more than line rate of the<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0DUT.</font></div><div class=3D"gmail_default"><font face=3D"v=
erdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif">[O] which should be used to</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0compens=
ate the test equipment to not send more than line rate of the</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0DUT.</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">[R] please reword for clarity.</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0To allow for th=
e minor variations typically found in the clock rate</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0of comme=
rcially-available clock modules and other crystal-based</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0Test set equipment manufacturers are well-aware of the standards,=
 and</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0allows a software-controlled +/- 100 PPM &quot;offset&qu=
ot; (clock-rate</font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">[O] allows</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[P] allow</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0ad=
justment) to compensate for normal variations in the clock speed of</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0&quot;devices under test&quot;. This offset adjustment allows engine=
ers to</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">=C2=A0 =C2=A0determine the approximate speed the connected device i=
s operating,</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">=C2=A0 =C2=A0and verify that it is within parameters allowed =
by standards.</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">5.3 Measurement Units</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0In a production network, it is very unlikely to see precise line rate=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0over a very brief period. There is no observable difference =
between</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">=C2=A0 =C2=A0dropping packets at 99% of line rate and 100% of line=
 rate. -Line</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">=C2=A0 =C2=A0rate CAN measured at 100% of line rate with a -1=
00PPM adjustment. -</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0Line rate SHOULD be measured at 99,98% wi=
th 0 PPM adjustment.-The PPM</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif">[O] .-The</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">[P] . The</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] Typo Or was this =
intended to be a list?</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0adjustment SHOULD only be used =
for a line rate type of measurement</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">6 =C2=A0Buffering</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">6.1 Buf=
fer</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">6.1.1 Definition</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">=C2=A0 =C2=A0Buffer Size: the term buffer si=
ze, represents the total amount of</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">[O] the term buffer size, represents=C2=
=A0</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">[P] the term buffer size represents</font></div><div class=3D"gmail_de=
fault"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0f=
rame buffering memory available on a DUT. This size is expressed in</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0Byte; KB (kilobytes), MB (megabytes) or GB (gigabyte). When the</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0buffer size is expressed it SHOULD be defined by a size metric</f=
ont></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
=C2=A0 =C2=A0defined above. When the buffer size is expressed, an indicatio=
n of</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">[O] =C2=A0defined above</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">[P] listed above [OR] stated above</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] the =
sizes are not &quot;defined&quot; above</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the frame MTU u=
sed for that measurement is also necessary as well as</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the cos=
 or dscp value set; as often times the buffers are carved by</font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0=
quality of service implementation. (please refer to the buffer</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] . (plea=
se</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">[P] . (Please</font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif">[R] grammar</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0efficiency section for f=
urther details).</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0Example: Buffer Size of DUT when send=
ing 1518 bytes frames is 18 Mb.</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Port Buffer Size: the p=
ort buffer size is the amount of buffer a</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0single ingress port=
, egress port or combination of ingress and egress</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0bufferin=
g location for a single port. The reason of mentioning the</font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] reason of m=
entioning</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[P] reason for mentioning</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">[R] grammar</font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0three=
 locations for the port buffer is, that the DUT buffering scheme</font></di=
v><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] is, t=
hat</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">[P] is because</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">[R] grammar</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0can be unknown o=
r untested, and therefore the indication of where the</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0buffer =
is located helps understand the buffer architecture and</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] =C2=A0and ther=
efore the indication of where the</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">=C2=A0 =C2=A0buffer is located helps und=
erstand the buffer architecture</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">[P] and so knowing the buffer location hel=
ps clarify the buffer architecture</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">[R] readability=C2=A0</font></div><div =
class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0therefore the total buffer size. The Port Buffer Size is an</font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0informational value that MAY be provided from the DUT vendor. It is</fon=
t></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0not a value that is tested by benchmarking. Benchmarking will be =
done</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-s=
erif">=C2=A0 =C2=A0using the Maximum Port Buffer Size or Maximum Buffer Siz=
e</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">=C2=A0 =C2=A0methodology.</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Maximum Port Buffer Size: =
this is in most cases the same as the Port</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif">[O] this is in most cases the s=
ame=C2=A0</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[P] in most cases, this is the same</font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif">[R] readability</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0Buffer Size. In certain switch architecture called SoC (switch on=
</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif=
">=C2=A0 =C2=A0chip), there is a concept of port buffer and shared buffer p=
ool</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">[O] there is a concept</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif">[R] ? in certain switch architecture there is =
a concept? Or is there an actual port buffer?</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0availabl=
e for all ports. Maximum Port Buffer, defines the scenario of</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0a SoC buffer, where this amount in B (byte), KB (kilobyte), MB</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0(megabyte) or GB (gigabyte) would represent the sum of the port</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0buffer along with the maximum value of shared buffer this given p=
ort</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">=C2=A0 =C2=A0can take. The Maximum Port Buffer Size needs to be expres=
sed along</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[O] Maximum Port Buffer, defines the scenario of</font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0a=
 SoC buffer, where this amount in B (byte), KB (kilobyte), MB</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0(megabyte) or GB (gigabyte) would represent the sum of the port</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0buffer along with the maximum value of shared buffer this given port<=
/font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"=
>=C2=A0 =C2=A0can take.</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">[P] Maximum Port Buffer, in terms of an SoC buffer=
, represents the sum of the port buffer and the maximum value of shared buf=
fer allowed for this port, defined in terms of B (byte), KB (kilobyte), MB =
(megabyte), or GB (gigabyte).</font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif">[R] readability - I think. I has a hard time=
 parsing the previous sentence.</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0with the frame MTU used=
 for the measurement and the cos or dscp bit</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0value set for th=
e test.</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">=C2=A0 =C2=A0Example: a DUT has been measured to have 3KB of=
 port buffer for 1518</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0frame size packets and a total of 4.7=
 MB of maximum port buffer for</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif">=C2=A0 =C2=A01518 frame size packets and a =
cos of 0.</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verd=
ana, sans-serif">=C2=A0 =C2=A0Maximum DUT Buffer Size: this is the total si=
ze of Buffer a DUT can</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0be measured to have. It is most likel=
y different than the Maximum</font></div><div class=3D"gmail_default"><font=
 face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">[O] It is most likely different than</f=
ont></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[=
P] either: It is, most likely, different to [OR] It is probably different t=
o</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">[R] grammar/readability</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Port Buffer Size. It CAN als=
o be different from the sum of Maximum</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Port Buffer Size. The =
Maximum Buffer Size needs to be expressed along</font></div><div class=3D"g=
mail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0with the fram=
e MTU used for the measurement and along with the cos or</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0dscp=
 value set during the test.</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Example: a DUT has been mea=
sured to have 3KB of port buffer for 1518</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0frame size packets =
and a total of 4.7 MB of maximum port buffer for</font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A01518 frame s=
ize packets. The DUT has a Maximum Buffer Size of 18 MB</font></div><div cl=
ass=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0at 15=
00 bytes and a cos of 0.</font></div><div class=3D"gmail_default"><font fac=
e=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif">=C2=A0 =C2=A0Burst: The burst is a fixed nu=
mber of packets sent over a percentage</font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0of linerate of a defin=
ed port speed. The amount of frames sent are</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0evenly distribut=
ed across the interval T. A constant C, can be</font></div><div class=3D"gm=
ail_default"><font face=3D"verdana, sans-serif">[O] interval T. A constant =
C,</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">[P] interval, T. A constant, C,=C2=A0</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">[R] clarity</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0=
defined to provide the average time between two consecutive packets</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0=
 =C2=A0evenly spaced.</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><fon=
t face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">6.1.2 Discussion</font></div><div clas=
s=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><di=
v class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0W=
hen measuring buffering on a DUT, it is important to understand what</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0the behavior is for each port, and also for all ports as this wil=
l</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">=C2=A0 =C2=A0provide an evidence of the total amount of buffering availa=
ble on the</font></div><div class=3D"gmail_default"><font face=3D"verdana, =
sans-serif">=C2=A0 =C2=A0switch. The terms of buffer efficiency here helps =
one understand what</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">[O] When measuring buffering on a DUT, it is important=
 to understand what</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0the behavior is for each port, and also f=
or all ports as this will</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">=C2=A0 =C2=A0provide an evidence of the total am=
ount of buffering available on the</font></div><div class=3D"gmail_default"=
><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0switch.</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] When measurin=
g buffering on a DUT, it is important to understand the behavior for each a=
nd all ports. This provides data for the total amount of buffering availabl=
e on the switch.</font></div><div class=3D"gmail_default"><font face=3D"ver=
dana, sans-serif"><br></font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0is the optimum packet size for the bu=
ffer to be used, or what is the</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">=C2=A0 =C2=A0real volume of buffer availab=
le for a specific packet size. This</font></div><div class=3D"gmail_default=
"><font face=3D"verdana, sans-serif">[O] what is the optimum packet size fo=
r the buffer to be used, or what is the</font></div><div class=3D"gmail_def=
ault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0real volume of buffer=
 available for a specific packet size.=C2=A0</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">[P] the optimum packet size f=
or the buffer, or the real volume of the buffer available for a specific pa=
cket size</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[R] readability</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">=C2=A0 =C2=A0section does not discuss how=
 to conduct the test methodology, it</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0rather explains the buff=
er definitions and what metrics should be</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">[O] , it rather explains</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] ; =
instead, it explains</font></div><div class=3D"gmail_default"><font face=3D=
"verdana, sans-serif">=C2=A0 =C2=A0provided for a comprehensive data center=
 device buffering</font></div><div class=3D"gmail_default"><font face=3D"ve=
rdana, sans-serif">=C2=A0 =C2=A0benchmarking.</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif"><br></font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">6.1.3 Measurement Uni=
ts</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif">=C2=A0 =C2=A0When Buffer is measured:-the buffer size MUST be mea=
sured-the port</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0buffer size MAY be provided for each port-the =
maximum port buffer</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">=C2=A0 =C2=A0size MUST be measured-the maximum DUT buf=
fer size MUST be measured-</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif">=C2=A0 =C2=A0the intensity of microburst MAY be=
 mentioned when a microburst test</font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">=C2=A0 =C2=A0is performed-the cos or dsc=
p value set during the test SHOULD be</font></div><div class=3D"gmail_defau=
lt"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0provided</font></div><d=
iv class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] ent=
ire paragraph above</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">[R] this is very difficult to read with the hyphens. G=
uessing this was intended to be a list, but the XML got munged?</font></div=
><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">6.2 =
Incast</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans=
-serif">6.2.1 Definition=C2=A0</font></div><div class=3D"gmail_default"><fo=
nt face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0The term Incast, very co=
mmonly utilized in the data center, refers to</font></div><div class=3D"gma=
il_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the traffic pat=
tern of many-to-one or many-to-many conversations.</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0Typicall=
y in the data center it would refer to many different ingress</font></div><=
div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0server ports(many), sending traffic to a common uplink (one), or</font><=
/div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] po=
rts(many)</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">[P] ports (many)</font></div><div class=3D"gmail_default"><font =
face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default">=
<font face=3D"verdana, sans-serif">=C2=A0 =C2=A0multiple uplinks (many). Th=
is pattern is generalized for any network</font></div><div class=3D"gmail_d=
efault"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0as many incoming po=
rts sending traffic to one or few uplinks. It can</font></div><div class=3D=
"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0also be fou=
nd in many-to-many traffic patterns.</font></div><div class=3D"gmail_defaul=
t"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif">6.2.2 Discussion</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></f=
ont></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><=
br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-se=
rif">=C2=A0 =C2=A0In this scenario, buffers are solicited on the DUT. In a =
ingress</font></div><div class=3D"gmail_default"><font face=3D"verdana, san=
s-serif">=C2=A0 =C2=A0buffering mechanism, the ingress port buffers would b=
e solicited</font></div><div class=3D"gmail_default"><font face=3D"verdana,=
 sans-serif">=C2=A0 =C2=A0along with Virtual Output Queues, when available;=
 whereas in an</font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0egress buffer mechanism, the egress buffer of =
the one outgoing port</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0would be used.</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div=
 class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0In=
 either cases, regardless of where the buffer memory is located on</font></=
div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =
=C2=A0the switch architecture; the Incast creates buffer utilization.</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[O] =
In either cases, regardless of where the buffer memory is located on</font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0the switch architecture; the Incast creates buffer utilization.</=
font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=
[P] In either case, regardless of where the buffer memory is located in the=
 switch architecture, the Incast creates the buffer utilization.</font></di=
v><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[R] gramm=
ar</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if"><br></font></div><div class=3D"gmail_default"><font face=3D"verdana, sa=
ns-serif"><br></font></div><div class=3D"gmail_default"><font face=3D"verda=
na, sans-serif">=C2=A0 =C2=A0When one or more frames having synchronous arr=
ival times at the DUT</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0they are considered forming an incast=
.</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-seri=
f">[O] incast</font></div><div class=3D"gmail_default"><font face=3D"verdan=
a, sans-serif">[P] Incast</font></div><div class=3D"gmail_default"><font fa=
ce=3D"verdana, sans-serif">[R] consistency</font></div><div class=3D"gmail_=
default"><font face=3D"verdana, sans-serif"><br></font></div><div class=3D"=
gmail_default"><font face=3D"verdana, sans-serif"><br></font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">7 Application Throu=
ghput: Data Center Goodput</font></div><div class=3D"gmail_default"><font f=
ace=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif">7.3. Measurement Units</font></div><div c=
lass=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font></div>=
<div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=
=A0Example: a TCP file transfer over HTTP protocol on a 10Gb/s media.</font=
></div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=
=A0 =C2=A0The file cannot be transferred over Ethernet as a single continuo=
us</font></div><div class=3D"gmail_default"><font face=3D"verdana, sans-ser=
if">=C2=A0 =C2=A0stream. It must be broken down into individual frames of 1=
500 bytes</font></div><div class=3D"gmail_default"><font face=3D"verdana, s=
ans-serif">=C2=A0 =C2=A0when the standard MTU [Maximum Transmission Unit] i=
s used. Each</font></div><div class=3D"gmail_default"><font face=3D"verdana=
, sans-serif">=C2=A0 =C2=A0packet requires 20 bytes of IP header informatio=
n and 20 bytes of TCP</font></div><div class=3D"gmail_default"><font face=
=3D"verdana, sans-serif">=C2=A0 =C2=A0header information, therefore 1460 by=
te are available per packet for</font></div><div class=3D"gmail_default"><f=
ont face=3D"verdana, sans-serif">[O] information, therefore=C2=A0</font></d=
iv><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">[P] info=
rmation; therefore,</font></div><div class=3D"gmail_default"><font face=3D"=
verdana, sans-serif">[R] grammar</font></div><div class=3D"gmail_default"><=
font face=3D"verdana, sans-serif"><br></font></div><div class=3D"gmail_defa=
ult"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0the file transfer. Lin=
ux based systems are further limited to 1448</font></div><div class=3D"gmai=
l_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0bytes as they al=
so carry a 12 byte timestamp. Finally, the date is</font></div><div class=
=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0transmit=
ted in this example over Ethernet which adds a 26 byte</font></div><div cla=
ss=3D"gmail_default"><font face=3D"verdana, sans-serif">=C2=A0 =C2=A0overhe=
ad per packet.</font></div><span class=3D"HOEnZb"><font color=3D"#888888"><=
div class=3D"gmail_default" style=3D"font-family:verdana,sans-serif"><br></=
div><div class=3D"gmail_default" style=3D"font-family:verdana,sans-serif"><=
br></div><div><br></div>-- <br><div class=3D"m_-8167641820907207108gmail_si=
gnature">I don&#39;t think the execution is relevant when it was obviously =
a bad idea in the first place.<br>This is like putting rabid weasels in you=
r pants, and later expressing regret at having chosen those particular rabi=
d weasels and that pair of pants.<br>=C2=A0 =C2=A0---maf</div>
</font></span></div>
</blockquote></div><br></div>

--94eb2c18e3b69231c505505265c1--


From nobody Mon May 29 03:44:15 2017
Return-Path: <ietf@kuehlewind.net>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 75CB71242F7; Mon, 29 May 2017 03:44:06 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: =?utf-8?q?Mirja_K=C3=BChlewind?= <ietf@kuehlewind.net>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org, Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, sbanks@encrypted.net, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.51.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149605464638.26527.8494709607583368776.idtracker@ietfa.amsl.com>
Date: Mon, 29 May 2017 03:44:06 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/g8JsKo_Y-Y9AcBA6mk5tsvZ23O4>
Subject: [bmwg] =?utf-8?q?Mirja_K=C3=BChlewind=27s_No_Objection_on_draft-i?= =?utf-8?q?etf-bmwg-vswitch-opnfv-03=3A_=28with_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 29 May 2017 10:44:07 -0000

Mirja Kühlewind has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

One minor question:
sec 3.4: "The recommended flow classification parameters for any
vswitch
   performance tests are: the input port (physical or logical), the
   source IP address, the destination IP address and the Ethernet
   protocol type field."
Why does this not include the transport ports? Is it assumed that there
is always only one flow between two IP addresses during a test?

Minor editorial comment:
Not sure why it is so important to mention several times that this work
is done "by referencing existing literature" (without referencing any
additional work beside opnfv)...?



From nobody Mon May 29 07:15:15 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7CA2A1274D2; Mon, 29 May 2017 07:15:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gvB9aL9J_GIY; Mon, 29 May 2017 07:15:05 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 24B9A127B57; Mon, 29 May 2017 07:15:05 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net [127.0.0.1]) by m0048589.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v4TE4lkx011975; Mon, 29 May 2017 10:15:02 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0048589.ppops.net-00191d01. with ESMTP id 2aqj3xfwx8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 29 May 2017 10:15:02 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v4TEF1oG087532; Mon, 29 May 2017 09:15:01 -0500
Received: from dalint02.pst.cso.att.com (dalint02.pst.cso.att.com [135.31.133.160]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v4TEExPL087522 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 29 May 2017 09:14:59 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint02.pst.cso.att.com (RSA Interceptor); Mon, 29 May 2017 14:14:48 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4TEEmY7009759; Mon, 29 May 2017 09:14:48 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v4TEEevl009452; Mon, 29 May 2017 09:14:44 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 93375E0A0E; Mon, 29 May 2017 10:14:39 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Mon, 29 May 2017 10:14:39 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: =?utf-8?B?TWlyamEgS8O8aGxld2luZA==?= <ietf@kuehlewind.net>, The IESG <iesg@ietf.org>
CC: "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: =?utf-8?B?W2Jtd2ddIE1pcmphIEvDvGhsZXdpbmQncyBObyBPYmplY3Rpb24gb24gZHJh?= =?utf-8?Q?ft-ietf-bmwg-vswitch-opnfv-03:_(with_COMMENT)?=
Thread-Index: AQHS2GiDh+oBh8l4IU2m9hafttQ4/qILVhZQ
Date: Mon, 29 May 2017 14:14:39 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD0C97@njmtexg5.research.att.com>
References: <149605464638.26527.8494709607583368776.idtracker@ietfa.amsl.com>
In-Reply-To: <149605464638.26527.8494709607583368776.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-29_10:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705290263
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/mZPxYLhoHakRbURrHk4mq2Z6WX0>
Subject: Re: [bmwg]  =?utf-8?q?Mirja_K=C3=BChlewind=27s_No_Objection_on_draft-?= =?utf-8?q?ietf-bmwg-vswitch-opnfv-03=3A_=28with_COMMENT=29?=
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 29 May 2017 14:15:07 -0000

SGkgTWlyamEsDQp0aGFua3MgZm9yIHlvdXIgcmV2aWV3LCBwbGVhc2Ugc2VlIHJlcGxpZXMgYmVs
b3cuDQpBbCwgZm9yIHRoZSBjby1hdXRob3JzDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0t
LS0NCj4gRnJvbTogYm13ZyBbbWFpbHRvOmJtd2ctYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxm
IE9mIE1pcmphIEvDvGhsZXdpbmQNCj4gU2VudDogTW9uZGF5LCBNYXkgMjksIDIwMTcgNjo0NCBB
TQ0KPiBUbzogVGhlIElFU0cNCj4gQ2M6IGJtd2ctY2hhaXJzQGlldGYub3JnOyBkcmFmdC1pZXRm
LWJtd2ctdnN3aXRjaC1vcG5mdkBpZXRmLm9yZzsNCj4gYm13Z0BpZXRmLm9yZw0KPiBTdWJqZWN0
OiBbYm13Z10gTWlyamEgS8O8aGxld2luZCdzIE5vIE9iamVjdGlvbiBvbiBkcmFmdC1pZXRmLWJt
d2ctDQo+IHZzd2l0Y2gtb3BuZnYtMDM6ICh3aXRoIENPTU1FTlQpDQo+IA0KPiBNaXJqYSBLw7xo
bGV3aW5kIGhhcyBlbnRlcmVkIHRoZSBmb2xsb3dpbmcgYmFsbG90IHBvc2l0aW9uIGZvcg0KPiBk
cmFmdC1pZXRmLWJtd2ctdnN3aXRjaC1vcG5mdi0wMzogTm8gT2JqZWN0aW9uDQo+IA0KLi4uDQo+
IA0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tDQo+IENPTU1FTlQ6DQo+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gDQo+IE9u
ZSBtaW5vciBxdWVzdGlvbjoNCj4gc2VjIDMuNDogIlRoZSByZWNvbW1lbmRlZCBmbG93IGNsYXNz
aWZpY2F0aW9uIHBhcmFtZXRlcnMgZm9yIGFueQ0KPiB2c3dpdGNoDQo+ICAgIHBlcmZvcm1hbmNl
IHRlc3RzIGFyZTogdGhlIGlucHV0IHBvcnQgKHBoeXNpY2FsIG9yIGxvZ2ljYWwpLCB0aGUNCj4g
ICAgc291cmNlIElQIGFkZHJlc3MsIHRoZSBkZXN0aW5hdGlvbiBJUCBhZGRyZXNzIGFuZCB0aGUg
RXRoZXJuZXQNCj4gICAgcHJvdG9jb2wgdHlwZSBmaWVsZC4iDQo+IFdoeSBkb2VzIHRoaXMgbm90
IGluY2x1ZGUgdGhlIHRyYW5zcG9ydCBwb3J0cz8gSXMgaXQgYXNzdW1lZCB0aGF0IHRoZXJlDQo+
IGlzIGFsd2F5cyBvbmx5IG9uZSBmbG93IGJldHdlZW4gdHdvIElQIGFkZHJlc3NlcyBkdXJpbmcg
YSB0ZXN0Pw0KW0FDTV0gDQpJdCdzIGNlcnRhaW5seSBwb3NzaWJsZSB0byBjbGFzc2lmeSBvbiB0
cmFuc3BvcnQgcG9ydHMgYW5kIG90aGVyIGZpZWxkcy4NCldlJ2xsIGFkZCB0aGF0IGluIGEgcGFy
ZW50aGV0aWNhbCBub3RlOg0KDQouLi4gKGFsdGhvdWdoIGNsYXNzaWZpY2F0aW9uIG1heSB0YWtl
IHBsYWNlIG9uIG90aGVyIGZpZWxkcywgc3VjaCBhcyBzb3VyY2UgYW5kIGRlc3RpbmF0aW9uIHRy
YW5zcG9ydCBwb3J0IG51bWJlcnMpLg0KDQo+IA0KPiBNaW5vciBlZGl0b3JpYWwgY29tbWVudDoN
Cj4gTm90IHN1cmUgd2h5IGl0IGlzIHNvIGltcG9ydGFudCB0byBtZW50aW9uIHNldmVyYWwgdGlt
ZXMgdGhhdCB0aGlzIHdvcmsNCj4gaXMgZG9uZSAiYnkgcmVmZXJlbmNpbmcgZXhpc3RpbmcgbGl0
ZXJhdHVyZSIgKHdpdGhvdXQgcmVmZXJlbmNpbmcgYW55DQo+IGFkZGl0aW9uYWwgd29yayBiZXNp
ZGUgb3BuZnYpLi4uPw0KPiANCltBQ01dIA0KQmVzaWRlcyB0aGUgYWJzdHJhY3QsIHdlIG1lbnRp
b24gZXhpc3RpbmcgbGl0ZXJhdHVyZSBpbiB0aGU6DQoNCiAtIEludHJvLCB3aGVyZSB0aGUgbmV4
dCBzZW50ZW5jZSBnaXZlcyBCTVdHIHJlZmVyZW5jZSBleGFtcGxlcy4NCiANCiAtIFNlY3Rpb24g
My4xLCB3aXRoIG5vIHJlZmVyZW5jZXMgbmVhcmJ5DQoNCiAtIFNlY3Rpb24gNC4wLCB3aXRoIGEg
bGlzdCBvZiByZWZlcmVuY2VzIHByb3ZpZGVkIGluIHRoZSBuZXh0IHNlbnRlbmNlLg0KDQogLSBB
Z2FpbiBpbiBTZWN0aW9uIDQuMCwgd2l0aCBubyByZWZzIG5lYXJieSAod2hlbiBpbnRyb2R1Y2lu
ZyB0aGUgZmlndXJlcykNCg0KV2UnbGwgcmUtd29yZCB0aGUgc2VudGVuY2VzIHdpdGggbm8gcmVm
cyBuZWFyYnkgdG8gYXZvaWQgcmVwZXRpdGlvbi4NCj4gDQoNCg==


From nobody Tue May 30 07:51:51 2017
Return-Path: <rjsparks@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 385EB129564; Tue, 30 May 2017 07:51:44 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Robert Sparks <rjsparks@nostrum.com>
To: <gen-art@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking.all@ietf.org, ietf@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.52.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149615590409.14768.3879167079373046273@ietfa.amsl.com>
Date: Tue, 30 May 2017 07:51:44 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/9NQMvrD-rIpu2_oPXufXC47TdpI>
Subject: [bmwg] Genart telechat review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 14:51:44 -0000

Reviewer: Robert Sparks
Review result: Ready

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair. Please wait for direction from your
document shepherd or AD before posting a new version of the draft.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08
Reviewer: Robert Sparks
Review Date: 2017-05-30
IETF LC End Date: 2017-05-02
IESG Telechat date: 2017-06-08

Summary: Ready for publication as an Informational RFC

Thanks for addressing my comments on -06


From nobody Tue May 30 09:52:07 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 297A7129AC6 for <bmwg@ietfa.amsl.com>; Tue, 30 May 2017 09:52:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Level: 
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bOK-4w05mhLv for <bmwg@ietfa.amsl.com>; Tue, 30 May 2017 09:52:04 -0700 (PDT)
Received: from mail-vk0-x22d.google.com (mail-vk0-x22d.google.com [IPv6:2607:f8b0:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 65FD0129AC5 for <bmwg@ietf.org>; Tue, 30 May 2017 09:52:04 -0700 (PDT)
Received: by mail-vk0-x22d.google.com with SMTP id x71so48351991vkd.0 for <bmwg@ietf.org>; Tue, 30 May 2017 09:52:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=A3q74rOqGyrDbMF/OqiFQnEkW1ZnoCCJ8F5ZsmXLIX0=; b=1RNoy4hSWuuCdPmEYOo7eiEPYvUbNOZDhpQ2INim6gyHkY/VRBJMjdCjRqIpxRja7A YRQvgFcQ7SAILkgl8VM2GUDzn+TEz693j6WA3BXHUCprYEDPnLH/UpcYNKjGPG5kb1ky MnPalNx2yzrSERJsady59/bxyNMYxP3tf1A5aAdSzdrZMXC5JB73M3K9vn00+/J/Pnn2 Jg6HcZjJom5RYQSKNXjnvo9LnOcqlLx9rMfq3XmVw2N2OAxoluMwHvGi/fli2tJblKsQ /MjOrI7F3nge9dYLpMMek6aXi32VAOt5FcqECdssx7Cqf47kZLgqSlEUJnvLMnMmM11Z DGAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=A3q74rOqGyrDbMF/OqiFQnEkW1ZnoCCJ8F5ZsmXLIX0=; b=GluzqwOD18qkSo8BEmfIYycZ88B+hq/pEg/sFNMgeTz/xXq+FoTnZD05zSxYoRsfCi Qv4Bg3iIfLC0asPkvffUmBAF3x4MGqNkWzJLITuVANr/+fl47evF9SFbtitbT2ZDpoAe 9CoEJOx0PiRvsomtZs+TbGQ64nM7VZrjbqwYHgCtNzRb3fUO4Q67hgZMwX5x9ir13vAz wnBA598HBu00fj76M5dphYZ7BsBhnsheodN4en0QaZq6RrTe2sG8k7xJqLxcr00Rw80o T3ejBpOBNGlq/1i6Ua5afXPaMvkC+zv5cWxyrJkFqPIJqSdXGzBV/dSNa5G1EZ+FXttT /tww==
X-Gm-Message-State: AODbwcAJjMMtsbIARAUwqxeKbYMkn3JdGb2zL9uqFTp6g6nMh+gS7ZdW YO6/9E7ku8+lhbPjwd3mcEhknECNoyL4
X-Received: by 10.31.3.77 with SMTP id 74mr8774335vkd.80.1496163123394; Tue, 30 May 2017 09:52:03 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.71.83 with HTTP; Tue, 30 May 2017 09:51:22 -0700 (PDT)
From: Warren Kumari <warren@kumari.net>
Date: Tue, 30 May 2017 12:51:22 -0400
Message-ID: <CAHw9_i+JfGnLfs9xMgUax=VQGoOZ8hnADuztrQ0ZVdwuFEDGgQ@mail.gmail.com>
To: draft-ietf-bmwg-dcbench-methodology.all@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/4WUdKjTLFz7ovqxgGMuq4ZZL00I>
Subject: [bmwg] AD review of draft-ietf-bmwg-dcbench-methodology
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 16:52:06 -0000

Hi there,

Thanks for another clear document - this makes a nice companion to
draft-ietf-bmwg-dcbench-terminology, and I'd like to progress them
together....

I have a number of suggested edits to make the document even more
readable / clearer. In a few instances I wasn't quite sure what the
document was trying to say, so please let me know if any of the
suggestions don't make sense, or change the text meaning...

Also, please poke me once updated, so that I can progress it to IETF LC...

------

1.  Introduction

   Traffic patterns in the data center are not uniform and are
   constantly changing. They are dictated by the nature and variety of
   applications utilized in the data center. It can be largely east-west
   traffic flows in one data center and north-south in another, while
   some may combine both. Traffic patterns can be bursty in nature and

[O] some
[P] others
[R] clarity

   contain many-to-one, many-to-many, or one-to-many flows. Each flow
   may also be small and latency sensitive or large and throughput
   sensitive while containing a mix of UDP and TCP traffic. All of which

[O] which
[P] these
[R] grammar

   can coexist in a single cluster and flow through a single network
   device all at the same time. Benchmarking of network devices have

[O] all at the same time
[P] simultaneously
[R] clarity

   long used [RFC1242], [RFC2432], [RFC2544], [2] and [3] which have
   largely been focused around various latency attributes and Throughput
   [2] of the Device Under Test (DUT) being benchmarked. These standards
   are good at measuring theoretical Throughput, forwarding rates and
   latency under testing conditions however, they do not represent real

[O] conditions however,
[P] conditions; however,
[R] grammar

   traffic patterns that may affect these networking devices.

....

1.2. Methodology format and repeatability recommendation


   For each test methodology described, it is critical to obtain
   repeatability in the results. The recommendation is to perform enough
   iterations of the given test and to make sure the result is
   consistent, this is especially important for section 3, as the

[O]  consistent, this
[P] consistent. This
[R] grammar

   buffering testing has been historically the least reliable. The
   number of iterations SHOULD be explicitly reported. The relative
   standard deviation SHOULD be below 10%.


2. Line Rate Testing

2.1 Objective

   Provide a maximum rate test for the performance values for
   Throughput, latency and jitter. It is meant to provide the tests to
   perform and methodology to verify that a DUT is capable of forwarding

[O] perform and methodology to verify
[P] perform, and methodology to verify,
[R] grammar

   packets at line rate under non-congested conditions.

2.3 Reporting Format

   -reading for Throughput received in percentage of bandwidth, while

[C]: Perhaps something like: - reading for "Throughput  received in
percentage of bandwidth", while...
(to make it clear where the "term" is)>

   sending 99.98% of port capacity on each port, for each packet size
   from 64 bytes to 9216 bytes. As guidance, an increment of 64 byte
   packet size between each iteration being ideal, a 256 byte and 512
   bytes being also often time used, the most common packets sizes order

[O] being also often time used, the
[P] are also often used. The
[R] grammar

   for the report is: 64b,128b,256b,512b,1024b,1518b,4096,8000,9216b.


6. Incast Stateful and Stateless Traffic
6.2 Methodology

   In order to simulate the effects of stateless and stateful traffic on
   the DUT there MUST be multiple ingress ports receiving traffic

[O] the DUT there
[P] the DUT, there
[R] grammar

   destined for the same egress port. There also MAY be a mix of
   stateful and stateless traffic arriving on a single ingress port. The
   simplest setup would be 2 ingress ports receiving traffic destined to
   the same egress port.

   One ingress port MUST be maintaining a TCP connection trough the

[O] trough
[P] through
[R] typo

   ingress port to a receiver connected to an egress port. Traffic in
   the TCP stream MUST be sent at the maximum rate allowed by the
   traffic generator. At the same time the TCP traffic is flowing

[O] At the same time the
[P] At the same time, the
[R] grammar

   through the DUT the stateless traffic is sent destined to a receiver
   on the same egress port. The stateless traffic MUST be a microburst
   of 100% intensity.


   Stateless Traffic port variation:

   During Iterations number of Egress ports MAY vary as well. First

[O] During Iterations number
[P] During Iterations, the number
[R] grammar

   Iteration: 1 Ingress port receiving stateful TCP traffic and 1
   Ingress port receiving stateless traffic destined to 1 Egress Port
----

Thanks.
Please poke me when integrated and I'll start the IETF LC on both.

W


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Tue May 30 13:49:44 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 91E2C126CD8; Tue, 30 May 2017 13:49:36 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.52.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149617737655.19809.2945400495991967324@ietfa.amsl.com>
Date: Tue, 30 May 2017 13:49:36 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/MQr4rA41eDJH8IpJNICGJtO2Xlo>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-07.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 20:49:37 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-07.txt
	Pages           : 18
	Date            : 2017-05-30

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-07
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-07

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-07


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue May 30 13:50:33 2017
Return-Path: <lucienav@google.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B137D126CD8 for <bmwg@ietfa.amsl.com>; Tue, 30 May 2017 13:50:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LifYZ_8jK6Ee for <bmwg@ietfa.amsl.com>; Tue, 30 May 2017 13:50:30 -0700 (PDT)
Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1E00C12944B for <bmwg@ietf.org>; Tue, 30 May 2017 13:50:29 -0700 (PDT)
Received: by mail-pf0-x22b.google.com with SMTP id 9so82544599pfj.1 for <bmwg@ietf.org>; Tue, 30 May 2017 13:50:29 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=jYP4+2a7joMp7e2V65XL0itfiqJXiUWoU22OF64GXVQ=; b=nQTA6eMd5z9blBSD5NRpVx8SX45/Y/O1ZUCV4lmDDqeOJUomRanr/XQDIdCJCD3unj iea5gUGKO0aiA78mjZIL5W92C2rfda2XGe0OoYT/hxU2+CM6KYlDnddNPyljaHKkbLJ8 tojLz+hgBTZ9LYWj5SOyFLyEHFiPL1Z2strHlQWDBaczn7Om5ujclwseql97s6mAOK7P WLgisH1FBJGXeypuIWNPcotFLr3p9K0xidteyex0x/VzWV1h/C+auj9g8B/d4VqVLjYB UlGHRjnr7Wb8Zf5JmTR/yxnHic6w51B71hY4YuBLvgQ9QyC7jULZuEWGtUB44dS53B6F iBnQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=jYP4+2a7joMp7e2V65XL0itfiqJXiUWoU22OF64GXVQ=; b=Kr0UCcI3yR3AQZo+c6EFJx/035ZRCaQw+ztmjsxRlBhqj9jhiDo3BhW0GwcOVEu7/R aTGlaEiVJFwNTWCXEKKEX8NzxwnhovqELcHq4oNSlaNDHlgy6ZVvozd5rWowu2gpYEvC 4SBK3cEa5bg7hGhALyGRujZ/CmjdfbcpddLMVSjdpju4blB93EKVHt4OsHvuEV/FZDSs u2xvVtdnfJ65VZNB4cO+F6EMWD6AuUdcmFZun7khGn8MXMHg2pEUmascd8C981QL+wc0 BlMi+dvzZi/2LVNMtMditQ5pEkdMtRnEzF9OrwiYUrnYjxFpIK+mCdEO8wio6ORnutBO L4Rg==
X-Gm-Message-State: AODbwcDwQI5Z7XGTeZ8xR3pgAWI6h4GZ6OcJdhKHOes8sv41I9oYSbph FV6c3/yV5PE9BTL4a1Pb8Q==
X-Received: by 10.98.93.217 with SMTP id n86mr26076524pfj.113.1496177428270; Tue, 30 May 2017 13:50:28 -0700 (PDT)
Received: from [10.0.1.76] (c-98-248-223-72.hsd1.ca.comcast.net. [98.248.223.72]) by smtp.gmail.com with ESMTPSA id 67sm25614499pfn.84.2017.05.30.13.50.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 May 2017 13:50:26 -0700 (PDT)
Content-Type: multipart/alternative; boundary=Apple-Mail-CFBCC4BE-00E8-4856-A035-49BF0AC59828
Mime-Version: 1.0 (1.0)
From: Lucien Avramov <lucienav@google.com>
X-Mailer: iPhone Mail (14E304)
In-Reply-To: <CAHw9_i+JfGnLfs9xMgUax=VQGoOZ8hnADuztrQ0ZVdwuFEDGgQ@mail.gmail.com>
Date: Tue, 30 May 2017 13:50:26 -0700
Cc: draft-ietf-bmwg-dcbench-methodology.all@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: 7bit
Message-Id: <740A99B9-4436-435F-911D-96C6E8B3C07E@google.com>
References: <CAHw9_i+JfGnLfs9xMgUax=VQGoOZ8hnADuztrQ0ZVdwuFEDGgQ@mail.gmail.com>
To: Warren Kumari <warren@kumari.net>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CYLEpcIpXNgfqJSZeHfFOB1__u8>
Subject: Re: [bmwg] AD review of draft-ietf-bmwg-dcbench-methodology
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 20:50:33 -0000

--Apple-Mail-CFBCC4BE-00E8-4856-A035-49BF0AC59828
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: 7bit

Thanks Warren!
Updated with your comments and posted 
draft-ietf-bmwg-dcbench-methodology-07

Thank you,
Lucien

> On May 30, 2017, at 09:51, Warren Kumari <warren@kumari.net> wrote:
> 
> Hi there,
> 
> Thanks for another clear document - this makes a nice companion to
> draft-ietf-bmwg-dcbench-terminology, and I'd like to progress them
> together....
> 
> I have a number of suggested edits to make the document even more
> readable / clearer. In a few instances I wasn't quite sure what the
> document was trying to say, so please let me know if any of the
> suggestions don't make sense, or change the text meaning...
> 
> Also, please poke me once updated, so that I can progress it to IETF LC...
> 
> ------
> 
> 1.  Introduction
> 
>   Traffic patterns in the data center are not uniform and are
>   constantly changing. They are dictated by the nature and variety of
>   applications utilized in the data center. It can be largely east-west
>   traffic flows in one data center and north-south in another, while
>   some may combine both. Traffic patterns can be bursty in nature and
> 
> [O] some
> [P] others
> [R] clarity
> 
>   contain many-to-one, many-to-many, or one-to-many flows. Each flow
>   may also be small and latency sensitive or large and throughput
>   sensitive while containing a mix of UDP and TCP traffic. All of which
> 
> [O] which
> [P] these
> [R] grammar
> 
>   can coexist in a single cluster and flow through a single network
>   device all at the same time. Benchmarking of network devices have
> 
> [O] all at the same time
> [P] simultaneously
> [R] clarity
> 
>   long used [RFC1242], [RFC2432], [RFC2544], [2] and [3] which have
>   largely been focused around various latency attributes and Throughput
>   [2] of the Device Under Test (DUT) being benchmarked. These standards
>   are good at measuring theoretical Throughput, forwarding rates and
>   latency under testing conditions however, they do not represent real
> 
> [O] conditions however,
> [P] conditions; however,
> [R] grammar
> 
>   traffic patterns that may affect these networking devices.
> 
> ....
> 
> 1.2. Methodology format and repeatability recommendation
> 
> 
>   For each test methodology described, it is critical to obtain
>   repeatability in the results. The recommendation is to perform enough
>   iterations of the given test and to make sure the result is
>   consistent, this is especially important for section 3, as the
> 
> [O]  consistent, this
> [P] consistent. This
> [R] grammar
> 
>   buffering testing has been historically the least reliable. The
>   number of iterations SHOULD be explicitly reported. The relative
>   standard deviation SHOULD be below 10%.
> 
> 
> 2. Line Rate Testing
> 
> 2.1 Objective
> 
>   Provide a maximum rate test for the performance values for
>   Throughput, latency and jitter. It is meant to provide the tests to
>   perform and methodology to verify that a DUT is capable of forwarding
> 
> [O] perform and methodology to verify
> [P] perform, and methodology to verify,
> [R] grammar
> 
>   packets at line rate under non-congested conditions.
> 
> 2.3 Reporting Format
> 
>   -reading for Throughput received in percentage of bandwidth, while
> 
> [C]: Perhaps something like: - reading for "Throughput  received in
> percentage of bandwidth", while...
> (to make it clear where the "term" is)>
> 
>   sending 99.98% of port capacity on each port, for each packet size
>   from 64 bytes to 9216 bytes. As guidance, an increment of 64 byte
>   packet size between each iteration being ideal, a 256 byte and 512
>   bytes being also often time used, the most common packets sizes order
> 
> [O] being also often time used, the
> [P] are also often used. The
> [R] grammar
> 
>   for the report is: 64b,128b,256b,512b,1024b,1518b,4096,8000,9216b.
> 
> 
> 6. Incast Stateful and Stateless Traffic
> 6.2 Methodology
> 
>   In order to simulate the effects of stateless and stateful traffic on
>   the DUT there MUST be multiple ingress ports receiving traffic
> 
> [O] the DUT there
> [P] the DUT, there
> [R] grammar
> 
>   destined for the same egress port. There also MAY be a mix of
>   stateful and stateless traffic arriving on a single ingress port. The
>   simplest setup would be 2 ingress ports receiving traffic destined to
>   the same egress port.
> 
>   One ingress port MUST be maintaining a TCP connection trough the
> 
> [O] trough
> [P] through
> [R] typo
> 
>   ingress port to a receiver connected to an egress port. Traffic in
>   the TCP stream MUST be sent at the maximum rate allowed by the
>   traffic generator. At the same time the TCP traffic is flowing
> 
> [O] At the same time the
> [P] At the same time, the
> [R] grammar
> 
>   through the DUT the stateless traffic is sent destined to a receiver
>   on the same egress port. The stateless traffic MUST be a microburst
>   of 100% intensity.
> 
> 
>   Stateless Traffic port variation:
> 
>   During Iterations number of Egress ports MAY vary as well. First
> 
> [O] During Iterations number
> [P] During Iterations, the number
> [R] grammar
> 
>   Iteration: 1 Ingress port receiving stateful TCP traffic and 1
>   Ingress port receiving stateless traffic destined to 1 Egress Port
> ----
> 
> Thanks.
> Please poke me when integrated and I'll start the IETF LC on both.
> 
> W
> 
> 
> -- 
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>   ---maf

--Apple-Mail-CFBCC4BE-00E8-4856-A035-49BF0AC59828
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div>Thanks Warren!</div><div id=3D"AppleMa=
ilSignature">Updated with your comments and posted&nbsp;<h2 style=3D"box-siz=
ing: border-box; font-weight: 400; line-height: 1.1; margin-top: 21px; margi=
n-bottom: 10.5px;"><small style=3D"box-sizing: border-box; line-height: 1; f=
ont-size: 14px; background-color: rgba(255, 255, 255, 0);">draft-ietf-bmwg-d=
cbench-methodology-07</small></h2><br>Thank you,<div>Lucien</div></div><div>=
<br>On May 30, 2017, at 09:51, Warren Kumari &lt;<a href=3D"mailto:warren@ku=
mari.net">warren@kumari.net</a>&gt; wrote:<br><br></div><blockquote type=3D"=
cite"><div><span>Hi there,</span><br><span></span><br><span>Thanks for anoth=
er clear document - this makes a nice companion to</span><br><span>draft-iet=
f-bmwg-dcbench-terminology, and I'd like to progress them</span><br><span>to=
gether....</span><br><span></span><br><span>I have a number of suggested edi=
ts to make the document even more</span><br><span>readable / clearer. In a f=
ew instances I wasn't quite sure what the</span><br><span>document was tryin=
g to say, so please let me know if any of the</span><br><span>suggestions do=
n't make sense, or change the text meaning...</span><br><span></span><br><sp=
an>Also, please poke me once updated, so that I can progress it to IETF LC..=
.</span><br><span></span><br><span>------</span><br><span></span><br><span>1=
. &nbsp;Introduction</span><br><span></span><br><span> &nbsp;&nbsp;Traffic p=
atterns in the data center are not uniform and are</span><br><span> &nbsp;&n=
bsp;constantly changing. They are dictated by the nature and variety of</spa=
n><br><span> &nbsp;&nbsp;applications utilized in the data center. It can be=
 largely east-west</span><br><span> &nbsp;&nbsp;traffic flows in one data ce=
nter and north-south in another, while</span><br><span> &nbsp;&nbsp;some may=
 combine both. Traffic patterns can be bursty in nature and</span><br><span>=
</span><br><span>[O] some</span><br><span>[P] others</span><br><span>[R] cla=
rity</span><br><span></span><br><span> &nbsp;&nbsp;contain many-to-one, many=
-to-many, or one-to-many flows. Each flow</span><br><span> &nbsp;&nbsp;may a=
lso be small and latency sensitive or large and throughput</span><br><span> &=
nbsp;&nbsp;sensitive while containing a mix of UDP and TCP traffic. All of w=
hich</span><br><span></span><br><span>[O] which</span><br><span>[P] these</s=
pan><br><span>[R] grammar</span><br><span></span><br><span> &nbsp;&nbsp;can c=
oexist in a single cluster and flow through a single network</span><br><span=
> &nbsp;&nbsp;device all at the same time. Benchmarking of network devices h=
ave</span><br><span></span><br><span>[O] all at the same time</span><br><spa=
n>[P] simultaneously</span><br><span>[R] clarity</span><br><span></span><br>=
<span> &nbsp;&nbsp;long used [RFC1242], [RFC2432], [RFC2544], [2] and [3] wh=
ich have</span><br><span> &nbsp;&nbsp;largely been focused around various la=
tency attributes and Throughput</span><br><span> &nbsp;&nbsp;[2] of the Devi=
ce Under Test (DUT) being benchmarked. These standards</span><br><span> &nbs=
p;&nbsp;are good at measuring theoretical Throughput, forwarding rates and</=
span><br><span> &nbsp;&nbsp;latency under testing conditions however, they d=
o not represent real</span><br><span></span><br><span>[O] conditions however=
,</span><br><span>[P] conditions; however,</span><br><span>[R] grammar</span=
><br><span></span><br><span> &nbsp;&nbsp;traffic patterns that may affect th=
ese networking devices.</span><br><span></span><br><span>....</span><br><spa=
n></span><br><span>1.2. Methodology format and repeatability recommendation<=
/span><br><span></span><br><span></span><br><span> &nbsp;&nbsp;For each test=
 methodology described, it is critical to obtain</span><br><span> &nbsp;&nbs=
p;repeatability in the results. The recommendation is to perform enough</spa=
n><br><span> &nbsp;&nbsp;iterations of the given test and to make sure the r=
esult is</span><br><span> &nbsp;&nbsp;consistent, this is especially importa=
nt for section 3, as the</span><br><span></span><br><span>[O] &nbsp;consiste=
nt, this</span><br><span>[P] consistent. This</span><br><span>[R] grammar</s=
pan><br><span></span><br><span> &nbsp;&nbsp;buffering testing has been histo=
rically the least reliable. The</span><br><span> &nbsp;&nbsp;number of itera=
tions SHOULD be explicitly reported. The relative</span><br><span> &nbsp;&nb=
sp;standard deviation SHOULD be below 10%.</span><br><span></span><br><span>=
</span><br><span>2. Line Rate Testing</span><br><span></span><br><span>2.1 O=
bjective</span><br><span></span><br><span> &nbsp;&nbsp;Provide a maximum rat=
e test for the performance values for</span><br><span> &nbsp;&nbsp;Throughpu=
t, latency and jitter. It is meant to provide the tests to</span><br><span> &=
nbsp;&nbsp;perform and methodology to verify that a DUT is capable of forwar=
ding</span><br><span></span><br><span>[O] perform and methodology to verify<=
/span><br><span>[P] perform, and methodology to verify,</span><br><span>[R] g=
rammar</span><br><span></span><br><span> &nbsp;&nbsp;packets at line rate un=
der non-congested conditions.</span><br><span></span><br><span>2.3 Reporting=
 Format</span><br><span></span><br><span> &nbsp;&nbsp;-reading for Throughpu=
t received in percentage of bandwidth, while</span><br><span></span><br><spa=
n>[C]: Perhaps something like: - reading for "Throughput &nbsp;received in</=
span><br><span>percentage of bandwidth", while...</span><br><span>(to make i=
t clear where the "term" is)&gt;</span><br><span></span><br><span> &nbsp;&nb=
sp;sending 99.98% of port capacity on each port, for each packet size</span>=
<br><span> &nbsp;&nbsp;from 64 bytes to 9216 bytes. As guidance, an incremen=
t of 64 byte</span><br><span> &nbsp;&nbsp;packet size between each iteration=
 being ideal, a 256 byte and 512</span><br><span> &nbsp;&nbsp;bytes being al=
so often time used, the most common packets sizes order</span><br><span></sp=
an><br><span>[O] being also often time used, the</span><br><span>[P] are als=
o often used. The</span><br><span>[R] grammar</span><br><span></span><br><sp=
an> &nbsp;&nbsp;for the report is: 64b,128b,256b,512b,1024b,1518b,4096,8000,=
9216b.</span><br><span></span><br><span></span><br><span>6. Incast Stateful a=
nd Stateless Traffic</span><br><span>6.2 Methodology</span><br><span></span>=
<br><span> &nbsp;&nbsp;In order to simulate the effects of stateless and sta=
teful traffic on</span><br><span> &nbsp;&nbsp;the DUT there MUST be multiple=
 ingress ports receiving traffic</span><br><span></span><br><span>[O] the DU=
T there</span><br><span>[P] the DUT, there</span><br><span>[R] grammar</span=
><br><span></span><br><span> &nbsp;&nbsp;destined for the same egress port. T=
here also MAY be a mix of</span><br><span> &nbsp;&nbsp;stateful and stateles=
s traffic arriving on a single ingress port. The</span><br><span> &nbsp;&nbs=
p;simplest setup would be 2 ingress ports receiving traffic destined to</spa=
n><br><span> &nbsp;&nbsp;the same egress port.</span><br><span></span><br><s=
pan> &nbsp;&nbsp;One ingress port MUST be maintaining a TCP connection troug=
h the</span><br><span></span><br><span>[O] trough</span><br><span>[P] throug=
h</span><br><span>[R] typo</span><br><span></span><br><span> &nbsp;&nbsp;ing=
ress port to a receiver connected to an egress port. Traffic in</span><br><s=
pan> &nbsp;&nbsp;the TCP stream MUST be sent at the maximum rate allowed by t=
he</span><br><span> &nbsp;&nbsp;traffic generator. At the same time the TCP t=
raffic is flowing</span><br><span></span><br><span>[O] At the same time the<=
/span><br><span>[P] At the same time, the</span><br><span>[R] grammar</span>=
<br><span></span><br><span> &nbsp;&nbsp;through the DUT the stateless traffi=
c is sent destined to a receiver</span><br><span> &nbsp;&nbsp;on the same eg=
ress port. The stateless traffic MUST be a microburst</span><br><span> &nbsp=
;&nbsp;of 100% intensity.</span><br><span></span><br><span></span><br><span>=
 &nbsp;&nbsp;Stateless Traffic port variation:</span><br><span></span><br><s=
pan> &nbsp;&nbsp;During Iterations number of Egress ports MAY vary as well. =
First</span><br><span></span><br><span>[O] During Iterations number</span><b=
r><span>[P] During Iterations, the number</span><br><span>[R] grammar</span>=
<br><span></span><br><span> &nbsp;&nbsp;Iteration: 1 Ingress port receiving s=
tateful TCP traffic and 1</span><br><span> &nbsp;&nbsp;Ingress port receivin=
g stateless traffic destined to 1 Egress Port</span><br><span>----</span><br=
><span></span><br><span>Thanks.</span><br><span>Please poke me when integrat=
ed and I'll start the IETF LC on both.</span><br><span></span><br><span>W</s=
pan><br><span></span><br><span></span><br><span>-- </span><br><span>I don't t=
hink the execution is relevant when it was obviously a bad</span><br><span>i=
dea in the first place.</span><br><span>This is like putting rabid weasels i=
n your pants, and later expressing</span><br><span>regret at having chosen t=
hose particular rabid weasels and that pair</span><br><span>of pants.</span>=
<br><span> &nbsp;&nbsp;---maf</span><br></div></blockquote></body></html>=

--Apple-Mail-CFBCC4BE-00E8-4856-A035-49BF0AC59828--


From nobody Tue May 30 14:50:00 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BE1A127869; Tue, 30 May 2017 14:49:51 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.52.0
Auto-Submitted: auto-generated
Precedence: bulk
CC: Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, draft-ietf-bmwg-dcbench-terminology@ietf.org, bmwg@ietf.org, sbanks@encrypted.net, warren@kumari.net
Reply-To: ietf@ietf.org
Sender: <iesg-secretary@ietf.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149618099120.19840.5554966296276002947.idtracker@ietfa.amsl.com>
Date: Tue, 30 May 2017 14:49:51 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/AmV2yjHO9r7qoqBR0Zb2wlX2VNk>
Subject: [bmwg] Last Call: <draft-ietf-bmwg-dcbench-terminology-09.txt> (Data Center Benchmarking Terminology) to Informational RFC
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 21:49:51 -0000

The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Data Center Benchmarking Terminology'
  <draft-ietf-bmwg-dcbench-terminology-09.txt> as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-06-13. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/ballot/


No IPR declarations have been submitted directly on this I-D.





From nobody Tue May 30 14:50:20 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id F0EBA127F0E; Tue, 30 May 2017 14:50:18 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.52.0
Auto-Submitted: auto-generated
Precedence: bulk
CC: Sarah Banks <sbanks@encrypted.net>, bmwg-chairs@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-dcbench-methodology@ietf.org, sbanks@encrypted.net, warren@kumari.net
Reply-To: ietf@ietf.org
Sender: <iesg-secretary@ietf.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149618101897.19836.14562989572504289943.idtracker@ietfa.amsl.com>
Date: Tue, 30 May 2017 14:50:18 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/DEmP-Wdp29T7n8YKTOGxTC6_oj0>
Subject: [bmwg] Last Call: <draft-ietf-bmwg-dcbench-methodology-07.txt> (Data Center Benchmarking Methodology) to Informational RFC
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 21:50:19 -0000

The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Data Center Benchmarking Methodology'
  <draft-ietf-bmwg-dcbench-methodology-07.txt> as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-06-13. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/ballot/


No IPR declarations have been submitted directly on this I-D.




