
From nobody Sat Aug 12 16:56:31 2017
Return-Path: <wwwrun@rfc-editor.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D123B13240F; Sat, 12 Aug 2017 16:56:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Level: 
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s4jR1QIYfN9d; Sat, 12 Aug 2017 16:56:28 -0700 (PDT)
Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 04047132408; Sat, 12 Aug 2017 16:56:28 -0700 (PDT)
Received: by rfc-editor.org (Postfix, from userid 30) id 504BBB818BC; Sat, 12 Aug 2017 16:56:11 -0700 (PDT)
To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
X-PHP-Originating-Script: 1005:ams_util_lib.php
From: rfc-editor@rfc-editor.org
Cc: rfc-editor@rfc-editor.org, drafts-update-ref@iana.org, bmwg@ietf.org
Message-Id: <20170812235611.504BBB818BC@rfc-editor.org>
Date: Sat, 12 Aug 2017 16:56:11 -0700 (PDT)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/fIKYDu2A9aHvvM0O96Hb4E11WHI>
Subject: [bmwg] RFC 8219 on Benchmarking Methodology for IPv6 Transition Technologies
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 12 Aug 2017 23:56:30 -0000

A new Request for Comments is now available in online RFC libraries.

        
        RFC 8219

        Title:      Benchmarking Methodology for IPv6 Transition 
                    Technologies 
        Author:     M. Georgescu, 
                    L. Pislaru,
                    G. Lencse
        Status:     Informational
        Stream:     IETF
        Date:       August 2017
        Mailbox:    marius.georgescu@rcs-rds.ro, 
                    liviu.pislaru@rcs-rds.ro, 
                    lencse@sze.hu
        Pages:      30
        Characters: 66085
        Updates/Obsoletes/SeeAlso:   None

        I-D Tag:    draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08.txt

        URL:        https://www.rfc-editor.org/info/rfc8219

        DOI:        10.17487/RFC8219

Benchmarking methodologies that address the performance of
network interconnect devices that are IPv4- or IPv6-capable exist,
but the IPv6 transition technologies are outside of their scope.
This document provides complementary guidelines for evaluating the
performance of IPv6 transition technologies.  More specifically,
this document targets IPv6 transition technologies that employ
encapsulation or translation mechanisms, as dual-stack nodes can be
tested using the recommendations of RFCs 2544 and 5180.
The methodology also includes a metric for benchmarking load
scalability.

This document is a product of the Benchmarking Methodology Working Group of the IETF.


INFORMATIONAL: This memo provides information for the Internet community.
It does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

This announcement is sent to the IETF-Announce and rfc-dist lists.
To subscribe or unsubscribe, see
  https://www.ietf.org/mailman/listinfo/ietf-announce
  https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist

For searching the RFC series, see https://www.rfc-editor.org/search
For downloading RFCs, see https://www.rfc-editor.org/retrieve/bulk

Requests for special distribution should be addressed to either the
author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
specifically noted otherwise on the RFC itself, all RFCs are for
unlimited distribution.


The RFC Editor Team
Association Management Solutions, LLC



From nobody Sat Aug 12 17:49:27 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1C7211323A5 for <bmwg@ietfa.amsl.com>; Sat, 12 Aug 2017 17:49:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ClTB8wFL_U3t for <bmwg@ietfa.amsl.com>; Sat, 12 Aug 2017 17:49:22 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0507813239F for <bmwg@ietf.org>; Sat, 12 Aug 2017 17:49:22 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v7D0j1ox022611 for <bmwg@ietf.org>; Sat, 12 Aug 2017 20:49:19 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by mx0a-00191d01.pphosted.com with ESMTP id 2c9ur83at6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Sat, 12 Aug 2017 20:49:19 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v7D0nITU112691 for <bmwg@ietf.org>; Sat, 12 Aug 2017 19:49:18 -0500
Received: from dalint01.pst.cso.att.com (dalint01.pst.cso.att.com [135.31.133.159]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v7D0nGCr112651 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Sat, 12 Aug 2017 19:49:16 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint01.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Sun, 13 Aug 2017 00:48:59 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v7D0mxgC010673 for <bmwg@ietf.org>; Sat, 12 Aug 2017 19:48:59 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v7D0mrtF010400 for <bmwg@ietf.org>; Sat, 12 Aug 2017 19:48:57 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 450B1E01A2 for <bmwg@ietf.org>; Sat, 12 Aug 2017 20:48:52 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0361.001; Sat, 12 Aug 2017 20:48:52 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] RFC 8219 on Benchmarking Methodology for IPv6 Transition Technologies
Thread-Index: AQHTE8an8dqNHyNEIkSOAKLFdlKCRKKBc7eQ
Date: Sun, 13 Aug 2017 00:48:51 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF46BF0E5B@njmtexg5.research.att.com>
References: <20170812235611.504BBB818BC@rfc-editor.org>
In-Reply-To: <20170812235611.504BBB818BC@rfc-editor.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.204.224]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-12_13:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1706020000 definitions=main-1708130011
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/xWVl7e9ngbzEIK-vumP3kQQ_1xg>
Subject: Re: [bmwg] RFC 8219 on Benchmarking Methodology for IPv6 Transition Technologies
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 13 Aug 2017 00:49:25 -0000

Congratulations Marius, Liviu, and Gabor!
Your first RFC is an important one!

Al
bmwg co-chair

> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of rfc-editor@rfc-
> editor.org
> Sent: Saturday, August 12, 2017 7:56 PM
> To: ietf-announce@ietf.org; rfc-dist@rfc-editor.org
> Cc: drafts-update-ref@iana.org; bmwg@ietf.org; rfc-editor@rfc-editor.org
> Subject: [bmwg] RFC 8219 on Benchmarking Methodology for IPv6 Transition
> Technologies
>=20
> A new Request for Comments is now available in online RFC libraries.
>=20
>=20
>         RFC 8219
>=20
>         Title:      Benchmarking Methodology for IPv6 Transition
>                     Technologies
>         Author:     M. Georgescu,
>                     L. Pislaru,
>                     G. Lencse
>         Status:     Informational
>         Stream:     IETF
>         Date:       August 2017
>         Mailbox:    marius.georgescu@rcs-rds.ro,
>                     liviu.pislaru@rcs-rds.ro,
>                     lencse@sze.hu
>         Pages:      30
>         Characters: 66085
>         Updates/Obsoletes/SeeAlso:   None
>=20
>         I-D Tag:    draft-ietf-bmwg-ipv6-tran-tech-benchmarking-08.txt
>=20
>         URL:        https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.rfc-2Deditor.org_info_rfc8219&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D_svgBZxzMZyTcJfYNALDyIxM=
_Qf
> Ds2XhUS4dWxfyzmQ&s=3DU5dUoC20gfY-fywsnBKYBYz2V3ZnYUqwAbO9OnuDCfE&e=3D
>=20
>         DOI:        10.17487/RFC8219
>=20
> Benchmarking methodologies that address the performance of
> network interconnect devices that are IPv4- or IPv6-capable exist,
> but the IPv6 transition technologies are outside of their scope.
> This document provides complementary guidelines for evaluating the
> performance of IPv6 transition technologies.  More specifically,
> this document targets IPv6 transition technologies that employ
> encapsulation or translation mechanisms, as dual-stack nodes can be
> tested using the recommendations of RFCs 2544 and 5180.
> The methodology also includes a metric for benchmarking load
> scalability.
>=20
> This document is a product of the Benchmarking Methodology Working Group
> of the IETF.
>=20
>=20
> INFORMATIONAL: This memo provides information for the Internet
> community.
> It does not specify an Internet standard of any kind. Distribution of
> this memo is unlimited.
>=20
> This announcement is sent to the IETF-Announce and rfc-dist lists.
> To subscribe or unsubscribe, see
>   https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_ietf-2Dannounce&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D_svgBZxzMZyTcJfYNALDyIxM=
_Qf
> Ds2XhUS4dWxfyzmQ&s=3DkMvLrxjLnk7Gtm11u2yxS7vEVOShC10dOMdqXKkawY4&e=3D
>   https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__mailman.rfc-
> 2Deditor.org_mailman_listinfo_rfc-2Ddist&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D_svgBZxzMZyTcJfYNALDyIxM=
_Qf
> Ds2XhUS4dWxfyzmQ&s=3Dh4-C182lBhy0dzaVn_sJuXmlPAFHCtVfux-JBf6Mem0&e=3D
>=20
> For searching the RFC series, see
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.rfc-
> 2Deditor.org_search&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D_svgBZxzMZyTcJfYNALDyIxM=
_Qf
> Ds2XhUS4dWxfyzmQ&s=3DIk449VaP8TBCdHt6YcOdNi8zw06VrqUoeP7BaXi8KPg&e=3D
> For downloading RFCs, see
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.rfc-
> 2Deditor.org_retrieve_bulk&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D_svgBZxzMZyTcJfYNALDyIxM=
_Qf
> Ds2XhUS4dWxfyzmQ&s=3DkO3oH1LXFZ2Uh4M-A-1T_xgUAarLBXztUtWBx-JP63w&e=3D
>=20
> Requests for special distribution should be addressed to either the
> author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
> specifically noted otherwise on the RFC itself, all RFCs are for
> unlimited distribution.
>=20
>=20
> The RFC Editor Team
> Association Management Solutions, LLC
>=20
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D_svgBZxzMZyTcJfYNALDyIxM=
_Qf
> Ds2XhUS4dWxfyzmQ&s=3DcVkCdxFJTsxk_M6AeH2SkJitYUo2xIcCshOK4MzOrgk&e=3D


From nobody Mon Aug 14 10:51:56 2017
Return-Path: <session-request@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 48D551323B6; Mon, 14 Aug 2017 10:51:54 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: IETF Meeting Session Request Tool <session-request@ietf.org>
To: <session-request@ietf.org>
Cc: warren@kumari.net, bmwg-chairs@ietf.org, acmorton@att.com, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.58.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <150273311429.327.13922648626229459223.idtracker@ietfa.amsl.com>
Date: Mon, 14 Aug 2017 10:51:54 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Sm-1k2jt7eexBSrc7A7LhrGPBqQ>
Subject: [bmwg] bmwg - New Meeting Session Request for IETF 100
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 14 Aug 2017 17:51:54 -0000

A new meeting session request has just been submitted by Al Morton, a Chair of the bmwg working group.


---------------------------------------------------------
Working Group Name: Benchmarking Methodology
Area Name: Operations and Management Area
Session Requester: Al Morton

Number of Sessions: 1
Length of Session(s):  2 Hours
Number of Attendees: 30
Conflicts to Avoid: 
 First Priority: ntp quic ippm nfvrg idr tsvwg tictoc sfc
 Second Priority: irtfopen xrblock tsvarea rmcat saag



People who must be present:
  Al Morton
  Sarah Banks
  Warren Kumari

Resources Requested:

Special Requests:
  Much better attendance and participation when session NOT on Friday
---------------------------------------------------------


From nobody Sun Aug 20 13:24:58 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B1D9132193 for <bmwg@ietfa.amsl.com>; Sun, 20 Aug 2017 13:24:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.621
X-Spam-Level: 
X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zqhnkuELQSNz for <bmwg@ietfa.amsl.com>; Sun, 20 Aug 2017 13:24:53 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F050126CB6 for <bmwg@ietf.org>; Sun, 20 Aug 2017 13:24:53 -0700 (PDT)
Received: from pps.filterd (m0049287.ppops.net [127.0.0.1]) by m0049287.ppops.net-00191d01. (8.16.0.21/8.16.0.21) with SMTP id v7KKOWnl000853 for <bmwg@ietf.org>; Sun, 20 Aug 2017 16:24:52 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049287.ppops.net-00191d01. with ESMTP id 2cej6f7fng-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Sun, 20 Aug 2017 16:24:51 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v7KKOooD085966 for <bmwg@ietf.org>; Sun, 20 Aug 2017 15:24:50 -0500
Received: from dalint02.pst.cso.att.com (dalint02.pst.cso.att.com [135.31.133.160]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v7KKOkJ7085950 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Sun, 20 Aug 2017 15:24:47 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint02.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Sun, 20 Aug 2017 20:24:38 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v7KKOc5v027924 for <bmwg@ietf.org>; Sun, 20 Aug 2017 15:24:38 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v7KKOUpP027611 for <bmwg@ietf.org>; Sun, 20 Aug 2017 15:24:34 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id DFE00E018F for <bmwg@ietf.org>; Sun, 20 Aug 2017 16:24:29 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0361.001; Sun, 20 Aug 2017 16:24:29 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Comments on SDN Benchmarking Drafts
Thread-Index: AdMZ8QBUeRNnNZyxTOyyOkQnmLTd2g==
Date: Sun, 20 Aug 2017 20:24:28 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF46C07A0F@njmtexg4.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-20_08:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1708200335
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/wSPUcMkmi-EWTPypP6qT1Qp5s5Q>
Subject: [bmwg] Comments on SDN Benchmarking Drafts
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 20 Aug 2017 20:24:56 -0000

BMWG,

First, a public thank you to Sarah for preparing and
uploading our minutes from IETF-99, based on Marius'
extensive and "cross-checked with the audio" meeting notes.
If you missed the meeting or a trip to Prague, you can=20
catch-up on our progress here:
https://datatracker.ietf.org/meeting/99/materials/minutes-99-bmwg

Thanks to you both!

Now, as a participant, I'm responding to one of my=20
Action Items by following-up with comments on the=20
SDN Drafts.

I have proposed text for the Terminology and Methodology
drafts to cover one of my comments at the meeting, about
more clearly including benchmarks for both:
   1. Loss-free Asynchronous Message Processing Rate
   2. Maximum Asynchronous Message Processing Rate

I haven't suggested how the 3x3 matrix should be edited yet;
these comments should be discussed first.  The proposed
text follows.

thanks and regards,
Al
(participant)


-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Proposal Follows -=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-

Terminology:
2.3.1.3. Asynchronous Message Processing Rate
OLD
Definition:
   The maximum number of asynchronous messages (session aliveness check
   message, new flow arrival notification message etc.) that the
   controller(s) can process, defined as the iteration starting with
   sending asynchronous messages to the controller (s) at the maximum
   possible rate and ending with an iteration that the controller(s)
   processes the received asynchronous messages without dropping.
NEW
Definition:
   The number responses to asynchronous messages (such as=20
   new flow arrival notification message, etc.) for which the
   controller(s) performed processing and replied with a valid=20
   and productive (non-trivial) response message.=20


OLD
Discussion:
   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events that the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Devices at the rate that the
   controller processes without dropping. This test assumes that the
   controller will respond to all the received asynchronous messages.
NEW
Discussion:
   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events that the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Devices at the rate that the
   controller processes without dropping. This test assumes that the
   controller is respond to all the received asynchronous messages
   (and the messages can be designed to elicit individual responses).

   When sending asynchronous messages to the controller(s) at high
   rates, some messages or responses may be discarded or corrupted
   and require retransmission to to controller(s). Therefore, a useful
   qualification on Asynchronous Message Processing Rate is whether=20
   the in-coming message count equals the response count in each trial.
   This is called the Loss-free Asynchronous Message Processing Rate.

   Note that several of the early controller benchmarking tools did
   not consider lost messages, and instead report the maximum=20
   response rate. This is called the Maximum Asynchronous Message=20
   Processing Rate.

   To characterize both the Loss-free and Maximum Rates, a test=20
   could begin the first trial by sending asynchronous messages to=20
   the controller (s) at the maximum possible rate and record=20
   the message reply rate and the message loss rate.
   The message sending rate is then decreased by the step-size and
   and the message reply rate and the message loss rate are recorded.
   The test ends with a trial where the controller(s)
   processes the all asynchronous messages sent without loss.
   This is the Loss-free Asynchronous Message Processing Rate.

   The (intermediate) trial where the controller(s) produced the=20
   maximum response rate is the Maximum Asynchronous Message=20
   Processing Rate.

   Of course, the first trial could begin at a low sending rate
   with zero lost responses, and increase until the Loss-free and
   Maximum Rates are discovered.
  =20
Measurement Units:
   Messages processed per second.

See Also:
   None

-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-

Methodology=20
(more substantial changes here, so OLD/NEW are not used)
5.1.3. Asynchronous Message Processing Rate

Objective:

   This test will measure two benchmarks on Asynchronous Message=20
   Processing Rate using a single procedure. The two benchmarks
   are (see section 2.3.1.3 of [ref to Terminology]):

   1. Loss-free Asynchronous Message Processing Rate
   2. Maximum Asynchronous Message Processing Rate

   There two benchmarks are determined through a series of trials
   where the a number of messages are sent to the controller(s),
   and the responses from the controller(s) are counted over the
   trial duration. The message response rate and the message loss
   ratio are calculated for each trial.=20

Reference Test Setup:

   The test SHOULD use one of the test setups described in section 3.1
   or section 3.2 of this document in combination with Appendix A.

Prerequisite:

   1. The controller(s) MUST have successfully completed the network
     topology discovery for the connected Network Devices.
   2. Choose and record the Trial Duration (Td), the sending rate step-size
     (STEP), the tolerance on equality for two consecutive trials (P%),
     and the maximum possible message sending rate (Ntx1/Td).

Procedure:

   1. Generate asynchronous messages continuously at the maximum
     possible rate on the established connections from all the
     emulated/simulated Network Devices for the given Trial Duration
     (Td).
   2. Record the total number of responses received from the controller
     (Nrx1) as well as the number of messages sent (Ntx1) to the
     controller(s) within the trial duration(Td).
   3. Calculate the Asynchronous Message Processing Rate (Tr1) and the=20
     Message Loss Ratio (Lr1). Ensure that the controller(s) have returned
     to normal operation.
   4. Repeat the trial by reducing the asynchronous message sending
     rate used in last trial by the STEP size.=20
   5. Continue repeating the trials and reducing the sending rate until
     both the maximum value of Nrxn and the Nrxn corresponding to zero loss
     ratio have been found.
   4. The trials corresponding to the benchmark levels the MUST be repeated=
=20
     using the same asynchronous message rates until the responses received
     from the controller are equal (+/-P%) for two consecutive trials.
   5. Record the number of responses received from the controller (Nrxn)
     as well as the number of messages sent(Ntxn) to the controller in
     the last trial.

Calculations:

                                               Nrxn
   Asynchronous Message Processing Rate Trn =3D -----
                                               Td

   Maximum Asynchronous Message Processing Rate =3D MAX(Trn) for all n

                                              Nrxn
   Asynchronous Message Loss Ratio Lrn =3D 1 - -----
                                              Ntxn

   Loss-free Asynchronous Message Processing Rate =3D MAX(Trn) given Lrn=3D=
0


Reporting Format:

   The Asynchronous Message Processing Rate results MUST be reported in
   the format of a table with a row for each trial.=20

   The table should report the following information in addition to
   the configuration parameters captured in section 5, with columns:

   - Offered rate (Ntxn/Td)

   - Asynchronous Message Processing Rate (Trn)

   - Loss Ratio (Lr)

   - Benchmark at this iteration (blank for none, Maximum, Loss-Free)

   The results MAY be presented in the form of a graph.  The X
   axis SHOULD be the Offered rate, and dual Y axes would represent
   Asynchronous Message Processing Rate and Loss Ratio, respectively.

   If this test is repeated with varying number of nodes over same
   topology, the results SHOULD be reported in the form of a graph. The
   X axis SHOULD be the Number of nodes (N), the Y axis
   SHOULD be the Asynchronous Message Processing Rate. Both the=20
   Maximum and the Loss-Free Rates should be plotted for each N.

   If this test is repeated with same number of nodes over different
   topologies, the results SHOULD be reported in the form of a graph.
   The X axis SHOULD be the Topology Type, the Y axis
   SHOULD be the Asynchronous Message Processing Rate. Both the=20
   Maximum and the Loss-Free Rates should be plotted for each topology.


@@@@ Comment: I think we may not need:

                                                 Tr1 + Tr2 + Tr3..Trn
   Average Asynchronous Message Processing Rate=3D --------------------
                                                 Total Test Iterations
   The last row of
   the table indicates the average Asynchronous Message Processing
   Rate.


From nobody Tue Aug 22 09:34:27 2017
Return-Path: <bhuvaneswaran.vengainathan@veryxtech.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8916313235A for <bmwg@ietfa.amsl.com>; Tue, 22 Aug 2017 09:34:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rOAx3retiY1V for <bmwg@ietfa.amsl.com>; Tue, 22 Aug 2017 09:34:21 -0700 (PDT)
Received: from smtpout4.netcore.co.in (nm237108.nsmailserv.com [202.162.237.108]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2883A1326E8 for <bmwg@ietf.org>; Tue, 22 Aug 2017 09:34:16 -0700 (PDT)
Received: from smtpin5.netcore.co.in (unknown [192.168.2.96]) by cf3.netcore.co.in (Postfix) with ESMTP id 25F0B120048; Tue, 22 Aug 2017 22:04:05 +0530 (IST)
Received: from cloudmail14.netcore.co.in (cloudmail12.netcore.co.in [202.162.231.3]) by smtpin5.netcore.co.in (Postfix) with ESMTP id 03C464A544; Tue, 22 Aug 2017 22:04:08 +0530 (IST)
Mime-Version: 1.0
Date: Tue, 22 Aug 2017 16:34:08 +0000
Content-Type: multipart/alternative; boundary="----=_Part_270_100360405.1503419648"
Message-ID: <1b32258ddde9b548fadf203e5e00664a@cloudmail14.netcore.co.in>
X-Mailer: AfterLogic webmail client
From: bhuvaneswaran.vengainathan@veryxtech.com
To: "MORTON, ALFRED C (AL)" <acmorton@att.com>
Cc: "bmwg@ietf.org" <bmwg@ietf.org>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF46C07A0F@njmtexg4.research.att.com>
References: <4D7F4AD313D3FC43A053B309F97543CF46C07A0F@njmtexg4.research.att.com>
X-Priority: 3 (Normal)
X-SMTP30-MailScanner-Information: Please contact the ISP for more information
X-MailScanner-ID: 03C464A544.A7539
X-SMTP30-MailScanner: Found to be clean
X-MailScanner-From: bhuvaneswaran.vengainathan@veryxtech.com
X-Cloudmilter-Processed: 1
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/SzUL-1GGnMFVDBTRymZaZDPNdbE>
Subject: Re: [bmwg] Comments on SDN Benchmarking Drafts
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 22 Aug 2017 16:34:25 -0000

------=_Part_270_100360405.1503419648
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hi Al,
Thank you for sharing your comments on the terminology and methodology draf=
t.
All your comments are valid and we will revise the draft based on your feed=
back.
Best Regards,
Bhuvan
On Mon, Aug 21, 2017 at 01:55 AM, "MORTON, ALFRED C (AL)"  wrote:
BMWG,

First, a public thank you to Sarah for preparing and
uploading our minutes from IETF-99, based on Marius'
extensive and "cross-checked with the audio" meeting notes.
If you missed the meeting or a trip to Prague, you can=20
catch-up on our progress here:
https://datatracker.ietf.org/meeting/99/materials/minutes-99-bmwg (https://=
datatracker.ietf.org/meeting/99/materials/minutes-99-bmwg)

Thanks to you both!

Now, as a participant, I'm responding to one of my=20
Action Items by following-up with comments on the=20
SDN Drafts.

I have proposed text for the Terminology and Methodology
drafts to cover one of my comments at the meeting, about
more clearly including benchmarks for both:
1. Loss-free Asynchronous Message Processing Rate
2. Maximum Asynchronous Message Processing Rate

I haven't suggested how the 3x3 matrix should be edited yet;
these comments should be discussed first.  The proposed
text follows.

thanks and regards,
Al
(participant)

-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Proposal Follows -=3D-=3D-=3D-=3D-=3D-=3D-=3D-

Terminology:
2.3.1.3. Asynchronous Message Processing Rate
OLD
Definition:
The maximum number of asynchronous messages (session aliveness check
message, new flow arrival notification message etc.) that the
controller(s) can process, defined as the iteration starting with
sending asynchronous messages to the controller (s) at the maximum
possible rate and ending with an iteration that the controller(s)
processes the received asynchronous messages without dropping.
NEW
Definition:
The number responses to asynchronous messages (such as=20
new flow arrival notification message, etc.) for which the
controller(s) performed processing and replied with a valid=20
and productive (non-trivial) response message.=20

OLD
Discussion:
As SDN assures flexible network and agile provisioning, it is
important to measure how many network events that the controller can
handle at a time. This benchmark is obtained by sending asynchronous
messages from every connected Network Devices at the rate that the
controller processes without dropping. This test assumes that the
controller will respond to all the received asynchronous messages.
NEW
Discussion:
As SDN assures flexible network and agile provisioning, it is
important to measure how many network events that the controller can
handle at a time. This benchmark is obtained by sending asynchronous
messages from every connected Network Devices at the rate that the
controller processes without dropping. This test assumes that the
controller is respond to all the received asynchronous messages
(and the messages can be designed to elicit individual responses).

When sending asynchronous messages to the controller(s) at high
rates, some messages or responses may be discarded or corrupted
and require retransmission to to controller(s). Therefore, a useful
qualification on Asynchronous Message Processing Rate is whether=20
the in-coming message count equals the response count in each trial.
This is called the Loss-free Asynchronous Message Processing Rate.

Note that several of the early controller benchmarking tools did
not consider lost messages, and instead report the maximum=20
response rate. This is called the Maximum Asynchronous Message=20
Processing Rate.

To characterize both the Loss-free and Maximum Rates, a test=20
could begin the first trial by sending asynchronous messages to=20
the controller (s) at the maximum possible rate and record=20
the message reply rate and the message loss rate.
The message sending rate is then decreased by the step-size and
and the message reply rate and the message loss rate are recorded.
The test ends with a trial where the controller(s)
processes the all asynchronous messages sent without loss.
This is the Loss-free Asynchronous Message Processing Rate.

The (intermediate) trial where the controller(s) produced the=20
maximum response rate is the Maximum Asynchronous Message=20
Processing Rate.

Of course, the first trial could begin at a low sending rate
with zero lost responses, and increase until the Loss-free and
Maximum Rates are discovered.

Measurement Units:
Messages processed per second.

See Also:
None

-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-

Methodology=20
(more substantial changes here, so OLD/NEW are not used)
5.1.3. Asynchronous Message Processing Rate

Objective:

This test will measure two benchmarks on Asynchronous Message=20
Processing Rate using a single procedure. The two benchmarks
are (see section 2.3.1.3 of [ref to Terminology]):

1. Loss-free Asynchronous Message Processing Rate
2. Maximum Asynchronous Message Processing Rate

There two benchmarks are determined through a series of trials
where the a number of messages are sent to the controller(s),
and the responses from the controller(s) are counted over the
trial duration. The message response rate and the message loss
ratio are calculated for each trial.=20

Reference Test Setup:

The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A.

Prerequisite:

1. The controller(s) MUST have successfully completed the network
topology discovery for the connected Network Devices.
2. Choose and record the Trial Duration (Td), the sending rate step-size
(STEP), the tolerance on equality for two consecutive trials (P%),
and the maximum possible message sending rate (Ntx1/Td).

Procedure:

1. Generate asynchronous messages continuously at the maximum
possible rate on the established connections from all the
emulated/simulated Network Devices for the given Trial Duration
(Td).
2. Record the total number of responses received from the controller
(Nrx1) as well as the number of messages sent (Ntx1) to the
controller(s) within the trial duration(Td).
3. Calculate the Asynchronous Message Processing Rate (Tr1) and the=20
Message Loss Ratio (Lr1). Ensure that the controller(s) have returned
to normal operation.
4. Repeat the trial by reducing the asynchronous message sending
rate used in last trial by the STEP size.=20
5. Continue repeating the trials and reducing the sending rate until
both the maximum value of Nrxn and the Nrxn corresponding to zero loss
ratio have been found.
4. The trials corresponding to the benchmark levels the MUST be repeated=20
using the same asynchronous message rates until the responses received
from the controller are equal (+/-P%) for two consecutive trials.
5. Record the number of responses received from the controller (Nrxn)
as well as the number of messages sent(Ntxn) to the controller in
the last trial.

Calculations:

Nrxn
Asynchronous Message Processing Rate Trn =3D -----
Td

Maximum Asynchronous Message Processing Rate =3D MAX(Trn) for all n

Nrxn
Asynchronous Message Loss Ratio Lrn =3D 1 - -----
Ntxn

Loss-free Asynchronous Message Processing Rate =3D MAX(Trn) given Lrn=3D0

Reporting Format:

The Asynchronous Message Processing Rate results MUST be reported in
the format of a table with a row for each trial.=20

The table should report the following information in addition to
the configuration parameters captured in section 5, with columns:

- Offered rate (Ntxn/Td)

- Asynchronous Message Processing Rate (Trn)

- Loss Ratio (Lr)

- Benchmark at this iteration (blank for none, Maximum, Loss-Free)

The results MAY be presented in the form of a graph.  The X
axis SHOULD be the Offered rate, and dual Y axes would represent
Asynchronous Message Processing Rate and Loss Ratio, respectively.

If this test is repeated with varying number of nodes over same
topology, the results SHOULD be reported in the form of a graph. The
X axis SHOULD be the Number of nodes (N), the Y axis
SHOULD be the Asynchronous Message Processing Rate. Both the=20
Maximum and the Loss-Free Rates should be plotted for each N.

If this test is repeated with same number of nodes over different
topologies, the results SHOULD be reported in the form of a graph.
The X axis SHOULD be the Topology Type, the Y axis
SHOULD be the Asynchronous Message Processing Rate. Both the=20
Maximum and the Loss-Free Rates should be plotted for each topology.

@@@@ Comment: I think we may not need:

Tr1 + Tr2 + Tr3..Trn
Average Asynchronous Message Processing Rate=3D --------------------
Total Test Iterations
The last row of
the table indicates the average Asynchronous Message Processing
Rate.

_______________________________________________
bmwg mailing list
bmwg@ietf.org (mailto:bmwg@ietf.org)
https://www.ietf.org/mailman/listinfo/bmwg (https://www.ietf.org/mailman/li=
stinfo/bmwg)

DISCLAIMER: Privileged and/or Confidential information may be
contained in this message. If you are not the addressee of this message,
you may not copy, use or deliver this message to anyone. In such
event,you should destroy the message and kindly notify the sender by
reply e-mail.
It is understood that opinions or conclusions that do not relate to the
official business of the company are neither given nor endorsed by the
company.


------=_Part_270_100360405.1503419648
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><meta http-equiv=3D"Content-Type" content=3D"tex=
t/html; charset=3Dutf-8" /></head><body><div data-crea=3D"font-wrapper" sty=
le=3D"font-family: Calibri; font-size: 16px; direction: ltr"><div style=3D"=
font-family: Calibri; font-size: 16px"></div>Hi Al,<div><br></div><div>Than=
k you for sharing your comments on the terminology and methodology draft.</=
div><div>All your comments are valid and we will revise the draft based on =
your feedback.</div><div><br></div><div>Best Regards,</div><div>Bhuvan<br><=
div></div><br><br><div data-anchor=3D"reply-title">On Mon, Aug 21, 2017 at =
01:55 AM, "MORTON, ALFRED C (AL)" &lt;acmorton@att.com&gt; wrote:</div><blo=
ckquote><div>BMWG,<br><br>First, a public thank you to Sarah for preparing =
and<br>uploading our minutes from IETF-99, based on Marius'<br>extensive an=
d "cross-checked with the audio" meeting notes.<br>If you missed the meetin=
g or a trip to Prague, you can <br>catch-up on our progress here:<br><a tar=
get=3D"_blank" href=3D"https://datatracker.ietf.org/meeting/99/materials/mi=
nutes-99-bmwg">https://datatracker.ietf.org/meeting/99/materials/minutes-99=
-bmwg</a><br><br>Thanks to you both!<br><br>Now, as a participant, I'm resp=
onding to one of my <br>Action Items by following-up with comments on the <=
br>SDN Drafts.<br><br>I have proposed text for the Terminology and Methodol=
ogy<br>drafts to cover one of my comments at the meeting, about<br>more cle=
arly including benchmarks for both:<br>1. Loss-free Asynchronous Message Pr=
ocessing Rate<br>2. Maximum Asynchronous Message Processing Rate<br><br>I h=
aven't suggested how the 3x3 matrix should be edited yet;<br>these comments=
 should be discussed first.=C2=A0=C2=A0The proposed<br>text follows.<br><br=
>thanks and regards,<br>Al<br>(participant)<br><br>-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D- Proposal Follows -=3D-=3D-=3D-=3D-=3D-=3D-=3D-<br><br>Terminology:<br=
>2.3.1.3. Asynchronous Message Processing Rate<br>OLD<br>Definition:<br>The=
 maximum number of asynchronous messages (session aliveness check<br>messag=
e, new flow arrival notification message etc.) that the<br>controller(s) ca=
n process, defined as the iteration starting with<br>sending asynchronous m=
essages to the controller (s) at the maximum<br>possible rate and ending wi=
th an iteration that the controller(s)<br>processes the received asynchrono=
us messages without dropping.<br>NEW<br>Definition:<br>The number responses=
 to asynchronous messages (such as <br>new flow arrival notification messag=
e, etc.) for which the<br>controller(s) performed processing and replied wi=
th a valid <br>and productive (non-trivial) response message. <br><br>OLD<b=
r>Discussion:<br>As SDN assures flexible network and agile provisioning, it=
 is<br>important to measure how many network events that the controller can=
<br>handle at a time. This benchmark is obtained by sending asynchronous<br=
>messages from every connected Network Devices at the rate that the<br>cont=
roller processes without dropping. This test assumes that the<br>controller=
 will respond to all the received asynchronous messages.<br>NEW<br>Discussi=
on:<br>As SDN assures flexible network and agile provisioning, it is<br>imp=
ortant to measure how many network events that the controller can<br>handle=
 at a time. This benchmark is obtained by sending asynchronous<br>messages =
from every connected Network Devices at the rate that the<br>controller pro=
cesses without dropping. This test assumes that the<br>controller is respon=
d to all the received asynchronous messages<br>(and the messages can be des=
igned to elicit individual responses).<br><br>When sending asynchronous mes=
sages to the controller(s) at high<br>rates, some messages or responses may=
 be discarded or corrupted<br>and require retransmission to to controller(s=
). Therefore, a useful<br>qualification on Asynchronous Message Processing =
Rate is whether <br>the in-coming message count equals the response count i=
n each trial.<br>This is called the Loss-free Asynchronous Message Processi=
ng Rate.<br><br>Note that several of the early controller benchmarking tool=
s did<br>not consider lost messages, and instead report the maximum <br>res=
ponse rate. This is called the Maximum Asynchronous Message <br>Processing =
Rate.<br><br>To characterize both the Loss-free and Maximum Rates, a test <=
br>could begin the first trial by sending asynchronous messages to <br>the =
controller (s) at the maximum possible rate and record <br>the message repl=
y rate and the message loss rate.<br>The message sending rate is then decre=
ased by the step-size and<br>and the message reply rate and the message los=
s rate are recorded.<br>The test ends with a trial where the controller(s)<=
br>processes the all asynchronous messages sent without loss.<br>This is th=
e Loss-free Asynchronous Message Processing Rate.<br><br>The (intermediate)=
 trial where the controller(s) produced the <br>maximum response rate is th=
e Maximum Asynchronous Message <br>Processing Rate.<br><br>Of course, the f=
irst trial could begin at a low sending rate<br>with zero lost responses, a=
nd increase until the Loss-free and<br>Maximum Rates are discovered.<br><br=
>Measurement Units:<br>Messages processed per second.<br><br>See Also:<br>N=
one<br><br>-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-<br><br>Methodology <br>(more s=
ubstantial changes here, so OLD/NEW are not used)<br>5.1.3. Asynchronous Me=
ssage Processing Rate<br><br>Objective:<br><br>This test will measure two b=
enchmarks on Asynchronous Message <br>Processing Rate using a single proced=
ure. The two benchmarks<br>are (see section 2.3.1.3 of [ref to Terminology]=
):<br><br>1. Loss-free Asynchronous Message Processing Rate<br>2. Maximum A=
synchronous Message Processing Rate<br><br>There two benchmarks are determi=
ned through a series of trials<br>where the a number of messages are sent t=
o the controller(s),<br>and the responses from the controller(s) are counte=
d over the<br>trial duration. The message response rate and the message los=
s<br>ratio are calculated for each trial. <br><br>Reference Test Setup:<br>=
<br>The test SHOULD use one of the test setups described in section 3.1<br>=
or section 3.2 of this document in combination with Appendix A.<br><br>Prer=
equisite:<br><br>1. The controller(s) MUST have successfully completed the =
network<br>topology discovery for the connected Network Devices.<br>2. Choo=
se and record the Trial Duration (Td), the sending rate step-size<br>(STEP)=
, the tolerance on equality for two consecutive trials (P%),<br>and the max=
imum possible message sending rate (Ntx1/Td).<br><br>Procedure:<br><br>1. G=
enerate asynchronous messages continuously at the maximum<br>possible rate =
on the established connections from all the<br>emulated/simulated Network D=
evices for the given Trial Duration<br>(Td).<br>2. Record the total number =
of responses received from the controller<br>(Nrx1) as well as the number o=
f messages sent (Ntx1) to the<br>controller(s) within the trial duration(Td=
).<br>3. Calculate the Asynchronous Message Processing Rate (Tr1) and the <=
br>Message Loss Ratio (Lr1). Ensure that the controller(s) have returned<br=
>to normal operation.<br>4. Repeat the trial by reducing the asynchronous m=
essage sending<br>rate used in last trial by the STEP size. <br>5. Continue=
 repeating the trials and reducing the sending rate until<br>both the maxim=
um value of Nrxn and the Nrxn corresponding to zero loss<br>ratio have been=
 found.<br>4. The trials corresponding to the benchmark levels the MUST be =
repeated <br>using the same asynchronous message rates until the responses =
received<br>from the controller are equal (+/-P%) for two consecutive trial=
s.<br>5. Record the number of responses received from the controller (Nrxn)=
<br>as well as the number of messages sent(Ntxn) to the controller in<br>th=
e last trial.<br><br>Calculations:<br><br>Nrxn<br>Asynchronous Message Proc=
essing Rate Trn =3D -----<br>Td<br><br>Maximum Asynchronous Message Process=
ing Rate =3D MAX(Trn) for all n<br><br>Nrxn<br>Asynchronous Message Loss Ra=
tio Lrn =3D 1 - -----<br>Ntxn<br><br>Loss-free Asynchronous Message Process=
ing Rate =3D MAX(Trn) given Lrn=3D0<br><br>Reporting Format:<br><br>The Asy=
nchronous Message Processing Rate results MUST be reported in<br>the format=
 of a table with a row for each trial. <br><br>The table should report the =
following information in addition to<br>the configuration parameters captur=
ed in section 5, with columns:<br><br>- Offered rate (Ntxn/Td)<br><br>- Asy=
nchronous Message Processing Rate (Trn)<br><br>- Loss Ratio (Lr)<br><br>- B=
enchmark at this iteration (blank for none, Maximum, Loss-Free)<br><br>The =
results MAY be presented in the form of a graph.=C2=A0=C2=A0The X<br>axis S=
HOULD be the Offered rate, and dual Y axes would represent<br>Asynchronous =
Message Processing Rate and Loss Ratio, respectively.<br><br>If this test i=
s repeated with varying number of nodes over same<br>topology, the results =
SHOULD be reported in the form of a graph. The<br>X axis SHOULD be the Numb=
er of nodes (N), the Y axis<br>SHOULD be the Asynchronous Message Processin=
g Rate. Both the <br>Maximum and the Loss-Free Rates should be plotted for =
each N.<br><br>If this test is repeated with same number of nodes over diff=
erent<br>topologies, the results SHOULD be reported in the form of a graph.=
<br>The X axis SHOULD be the Topology Type, the Y axis<br>SHOULD be the Asy=
nchronous Message Processing Rate. Both the <br>Maximum and the Loss-Free R=
ates should be plotted for each topology.<br><br>@@@@ Comment: I think we m=
ay not need:<br><br>Tr1 + Tr2 + Tr3..Trn<br>Average Asynchronous Message Pr=
ocessing Rate=3D --------------------<br>Total Test Iterations<br>The last =
row of<br>the table indicates the average Asynchronous Message Processing<b=
r>Rate.<br><br>_______________________________________________<br>bmwg mail=
ing list<br><a target=3D"_blank" href=3D"mailto:bmwg@ietf.org">bmwg@ietf.or=
g</a><br><a target=3D"_blank" href=3D"https://www.ietf.org/mailman/listinfo=
/bmwg">https://www.ietf.org/mailman/listinfo/bmwg</a></div></blockquote></d=
iv></div>DISCLAIMER: Privileged and/or Confidential information may be
contained in this message. If you are not the addressee of this message,
you may not copy, use or deliver this message to anyone. In such
event,you should destroy the message and kindly notify the sender by
reply e-mail.
It is understood that opinions or conclusions that do not relate to the
official business of the company are neither given nor endorsed by the
company.
</body></html>

------=_Part_270_100360405.1503419648--


From nobody Wed Aug 30 15:17:48 2017
Return-Path: <wwwrun@rfc-editor.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7CEB132C2F; Wed, 30 Aug 2017 15:17:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.201
X-Spam-Level: 
X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JBSsjRnByf_X; Wed, 30 Aug 2017 15:17:45 -0700 (PDT)
Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1A636132A80; Wed, 30 Aug 2017 15:17:45 -0700 (PDT)
Received: by rfc-editor.org (Postfix, from userid 30) id BEDCFB81090; Wed, 30 Aug 2017 15:17:07 -0700 (PDT)
To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
X-PHP-Originating-Script: 1005:ams_util_lib.php
From: rfc-editor@rfc-editor.org
Cc: rfc-editor@rfc-editor.org, drafts-update-ref@iana.org, bmwg@ietf.org
Message-Id: <20170830221707.BEDCFB81090@rfc-editor.org>
Date: Wed, 30 Aug 2017 15:17:07 -0700 (PDT)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/bl1VuwpxRXBPqtJAnxpBJjW0_Y4>
Subject: [bmwg] RFC 8238 on Data Center Benchmarking Terminology
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Aug 2017 22:17:47 -0000

A new Request for Comments is now available in online RFC libraries.

        
        RFC 8238

        Title:      Data Center Benchmarking Terminology 
        Author:     L. Avramov,
                    J. Rapp
        Status:     Informational
        Stream:     IETF
        Date:       August 2017
        Mailbox:    lucien.avramov@gmail.com, 
                    jhrapp@gmail.com
        Pages:      20
        Characters: 39128
        Updates/Obsoletes/SeeAlso:   None

        I-D Tag:    draft-ietf-bmwg-dcbench-terminology-19.txt

        URL:        https://www.rfc-editor.org/info/rfc8238

        DOI:        10.17487/RFC8238

The purposes of this informational document are to establish
definitions and describe measurement techniques for data center
benchmarking, as well as to introduce new terminology applicable to
performance evaluations of data center network equipment.  This
document establishes the important concepts for benchmarking network
switches and routers in the data center and is a prerequisite for the
test methodology document (RFC 8239).  Many of these terms and
methods may be applicable to network equipment beyond the scope of
this document as the technologies originally applied in the data
center are deployed elsewhere.

This document is a product of the Benchmarking Methodology Working Group of the IETF.


INFORMATIONAL: This memo provides information for the Internet community.
It does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

This announcement is sent to the IETF-Announce and rfc-dist lists.
To subscribe or unsubscribe, see
  https://www.ietf.org/mailman/listinfo/ietf-announce
  https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist

For searching the RFC series, see https://www.rfc-editor.org/search
For downloading RFCs, see https://www.rfc-editor.org/retrieve/bulk

Requests for special distribution should be addressed to either the
author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
specifically noted otherwise on the RFC itself, all RFCs are for
unlimited distribution.


The RFC Editor Team
Association Management Solutions, LLC


From nobody Wed Aug 30 15:18:09 2017
Return-Path: <wwwrun@rfc-editor.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E1C88132C40; Wed, 30 Aug 2017 15:17:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.201
X-Spam-Level: 
X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 15kpeslJBmlb; Wed, 30 Aug 2017 15:17:52 -0700 (PDT)
Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DCCB0132C3B; Wed, 30 Aug 2017 15:17:51 -0700 (PDT)
Received: by rfc-editor.org (Postfix, from userid 30) id 83BE9B810A1; Wed, 30 Aug 2017 15:17:14 -0700 (PDT)
To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
X-PHP-Originating-Script: 1005:ams_util_lib.php
From: rfc-editor@rfc-editor.org
Cc: rfc-editor@rfc-editor.org, drafts-update-ref@iana.org, bmwg@ietf.org
Message-Id: <20170830221714.83BE9B810A1@rfc-editor.org>
Date: Wed, 30 Aug 2017 15:17:14 -0700 (PDT)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/3pgR0f_2WonMd9i1BxZLrUoWeKQ>
Subject: [bmwg] RFC 8239 on Data Center Benchmarking Methodology
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Aug 2017 22:17:54 -0000

A new Request for Comments is now available in online RFC libraries.

        
        RFC 8239

        Title:      Data Center Benchmarking Methodology 
        Author:     L. Avramov,
                    J. Rapp
        Status:     Informational
        Stream:     IETF
        Date:       August 2017
        Mailbox:    lucien.avramov@gmail.com, 
                    jhrapp@gmail.com
        Pages:      19
        Characters: 39066
        Updates/Obsoletes/SeeAlso:   None

        I-D Tag:    draft-ietf-bmwg-dcbench-methodology-18.txt

        URL:        https://www.rfc-editor.org/info/rfc8239

        DOI:        10.17487/RFC8239

The purpose of this informational document is to establish test and
evaluation methodology and measurement techniques for physical
network equipment in the data center.  RFC 8238 is a prerequisite for
this document, as it contains terminology that is considered
normative.  Many of these terms and methods may be applicable beyond
the scope of this document as the technologies originally applied in
the data center are deployed elsewhere.

This document is a product of the Benchmarking Methodology Working Group of the IETF.


INFORMATIONAL: This memo provides information for the Internet community.
It does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

This announcement is sent to the IETF-Announce and rfc-dist lists.
To subscribe or unsubscribe, see
  https://www.ietf.org/mailman/listinfo/ietf-announce
  https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist

For searching the RFC series, see https://www.rfc-editor.org/search
For downloading RFCs, see https://www.rfc-editor.org/retrieve/bulk

Requests for special distribution should be addressed to either the
author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
specifically noted otherwise on the RFC itself, all RFCs are for
unlimited distribution.


The RFC Editor Team
Association Management Solutions, LLC


From nobody Wed Aug 30 16:34:23 2017
Return-Path: <sbanks@encrypted.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 30379132703 for <bmwg@ietfa.amsl.com>; Wed, 30 Aug 2017 16:34:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4yOEM3Dv_T7R for <bmwg@ietfa.amsl.com>; Wed, 30 Aug 2017 16:34:18 -0700 (PDT)
Received: from aws.hosed.org (aws.hosed.org [50.16.104.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C7DB9132418 for <bmwg@ietf.org>; Wed, 30 Aug 2017 16:34:18 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by aws.hosed.org (Postfix) with ESMTP id 26C298007E; Wed, 30 Aug 2017 19:34:18 -0400 (EDT)
X-Virus-Scanned: Debian amavisd-new at aws.hosed.org
Received: from aws.hosed.org ([127.0.0.1]) by localhost (aws.hosed.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GGk7YD0uoNKH; Wed, 30 Aug 2017 19:34:18 -0400 (EDT)
Received: from [10.1.220.141] (unknown [63.194.187.130]) by aws.hosed.org (Postfix) with ESMTPSA id A507680079; Wed, 30 Aug 2017 19:34:17 -0400 (EDT)
From: Sarah B <sbanks@encrypted.net>
Content-Type: multipart/alternative; boundary="Apple-Mail=_7C4A47C3-6C3C-4876-94B2-9D1DABE93956"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Date: Wed, 30 Aug 2017 16:34:16 -0700
References: <20170830221714.83BE9B810A1@rfc-editor.org>
Cc: Lucien <lucien.avramov@gmail.com>, Jacob Rapp <jrapp@vmware.com>
To: bmwg@ietf.org
Message-Id: <4FF5E63A-A335-4E67-9B3B-2608073D86AF@encrypted.net>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/nRooSnwZsxQbaydDdbqPwLUn7E0>
Subject: [bmwg] Fwd:  RFC 8239 on Data Center Benchmarking Methodology
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Aug 2017 23:34:21 -0000

--Apple-Mail=_7C4A47C3-6C3C-4876-94B2-9D1DABE93956
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hello BMWG, Lucien, Jacob,
	Today RFCs 8238.8239 have been published, marking the end of a =
long journey towards Data Center Benchmarking. A huge thank you to the =
authors and BMWG alike; BMWG, thank you for your repeated reviews and =
consistent feedback, both at the meetings and on the list. Authors, =
thank you for the dedication to the work, particularly at the tail end. =
Again, congratulations on this exciting milestone!

Best,
Sarah


> Begin forwarded message:
>=20
> From: rfc-editor@rfc-editor.org
> Subject: [bmwg] RFC 8239 on Data Center Benchmarking Methodology
> Date: August 30, 2017 at 3:17:14 PM PDT
> To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
> Cc: drafts-update-ref@iana.org, bmwg@ietf.org, =
rfc-editor@rfc-editor.org
>=20
> A new Request for Comments is now available in online RFC libraries.
>=20
>=20
>        RFC 8239
>=20
>        Title:      Data Center Benchmarking Methodology=20
>        Author:     L. Avramov,
>                    J. Rapp
>        Status:     Informational
>        Stream:     IETF
>        Date:       August 2017
>        Mailbox:    lucien.avramov@gmail.com,=20
>                    jhrapp@gmail.com
>        Pages:      19
>        Characters: 39066
>        Updates/Obsoletes/SeeAlso:   None
>=20
>        I-D Tag:    draft-ietf-bmwg-dcbench-methodology-18.txt
>=20
>        URL:        https://www.rfc-editor.org/info/rfc8239
>=20
>        DOI:        10.17487/RFC8239
>=20
> The purpose of this informational document is to establish test and
> evaluation methodology and measurement techniques for physical
> network equipment in the data center.  RFC 8238 is a prerequisite for
> this document, as it contains terminology that is considered
> normative.  Many of these terms and methods may be applicable beyond
> the scope of this document as the technologies originally applied in
> the data center are deployed elsewhere.
>=20
> This document is a product of the Benchmarking Methodology Working =
Group of the IETF.
>=20
>=20
> INFORMATIONAL: This memo provides information for the Internet =
community.
> It does not specify an Internet standard of any kind. Distribution of
> this memo is unlimited.
>=20
> This announcement is sent to the IETF-Announce and rfc-dist lists.
> To subscribe or unsubscribe, see
>  https://www.ietf.org/mailman/listinfo/ietf-announce
>  https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist
>=20
> For searching the RFC series, see https://www.rfc-editor.org/search
> For downloading RFCs, see https://www.rfc-editor.org/retrieve/bulk
>=20
> Requests for special distribution should be addressed to either the
> author of the RFC in question, or to rfc-editor@rfc-editor.org.  =
Unless
> specifically noted otherwise on the RFC itself, all RFCs are for
> unlimited distribution.
>=20
>=20
> The RFC Editor Team
> Association Management Solutions, LLC
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


--Apple-Mail=_7C4A47C3-6C3C-4876-94B2-9D1DABE93956
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">Hello BMWG, Lucien, Jacob,<div class=3D""><span =
class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>Today =
RFCs 8238.8239 have been published, marking the end of a long journey =
towards Data Center Benchmarking. A huge thank you to the authors and =
BMWG alike; BMWG, thank you for your repeated reviews and consistent =
feedback, both at the meetings and on the list. Authors, thank you for =
the dedication to the work, particularly at the tail end. Again, =
congratulations on this exciting milestone!</div><div class=3D""><br =
class=3D""></div><div class=3D"">Best,</div><div =
class=3D"">Sarah</div><div class=3D""><br class=3D""><div><br =
class=3D""><blockquote type=3D"cite" class=3D""><div class=3D"">Begin =
forwarded message:</div><br class=3D"Apple-interchange-newline"><div =
style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
margin-left: 0px;" class=3D""><span style=3D"font-family: =
-webkit-system-font, Helvetica Neue, Helvetica, sans-serif; =
color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">From: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D""><a =
href=3D"mailto:rfc-editor@rfc-editor.org" =
class=3D"">rfc-editor@rfc-editor.org</a><br class=3D""></span></div><div =
style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
margin-left: 0px;" class=3D""><span style=3D"font-family: =
-webkit-system-font, Helvetica Neue, Helvetica, sans-serif; =
color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Subject: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D""><b class=3D"">[bmwg] RFC 8239 =
on Data Center Benchmarking Methodology</b><br =
class=3D""></span></div><div style=3D"margin-top: 0px; margin-right: =
0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span =
style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Date: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D"">August 30, 2017 at 3:17:14 PM =
PDT<br class=3D""></span></div><div style=3D"margin-top: 0px; =
margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span=
 style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">To: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D""><a =
href=3D"mailto:ietf-announce@ietf.org" =
class=3D"">ietf-announce@ietf.org</a>, <a =
href=3D"mailto:rfc-dist@rfc-editor.org" =
class=3D"">rfc-dist@rfc-editor.org</a><br class=3D""></span></div><div =
style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
margin-left: 0px;" class=3D""><span style=3D"font-family: =
-webkit-system-font, Helvetica Neue, Helvetica, sans-serif; =
color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Cc: </b></span><span =
style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif;" class=3D""><a href=3D"mailto:drafts-update-ref@iana.org" =
class=3D"">drafts-update-ref@iana.org</a>, <a =
href=3D"mailto:bmwg@ietf.org" class=3D"">bmwg@ietf.org</a>, <a =
href=3D"mailto:rfc-editor@rfc-editor.org" =
class=3D"">rfc-editor@rfc-editor.org</a><br class=3D""></span></div><br =
class=3D""><div class=3D""><div class=3D"">A new Request for Comments is =
now available in online RFC libraries.<br class=3D""><br class=3D""><br =
class=3D""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RFC 8239<br =
class=3D""><br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Title: =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data Center Benchmarking Methodology <br =
class=3D""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Author: =
&nbsp;&nbsp;&nbsp;&nbsp;L. Avramov,<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;J. Rapp<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Status: =
&nbsp;&nbsp;&nbsp;&nbsp;Informational<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Stream: =
&nbsp;&nbsp;&nbsp;&nbsp;IETF<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date: =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;August 2017<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Mailbox: &nbsp;&nbsp;&nbsp;<a =
href=3D"mailto:lucien.avramov@gmail.com" =
class=3D"">lucien.avramov@gmail.com</a>, <br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a =
href=3D"mailto:jhrapp@gmail.com" class=3D"">jhrapp@gmail.com</a><br =
class=3D""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pages: =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;19<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Characters: 39066<br class=3D"">=
 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Updates/Obsoletes/SeeAlso: =
&nbsp;&nbsp;None<br class=3D""><br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;I-D Tag: =
&nbsp;&nbsp;&nbsp;draft-ietf-bmwg-dcbench-methodology-18.txt<br =
class=3D""><br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;URL: =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a =
href=3D"https://www.rfc-editor.org/info/rfc8239" =
class=3D"">https://www.rfc-editor.org/info/rfc8239</a><br class=3D""><br =
class=3D""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;DOI: =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;10.17487/RFC8239<br =
class=3D""><br class=3D"">The purpose of this informational document is =
to establish test and<br class=3D"">evaluation methodology and =
measurement techniques for physical<br class=3D"">network equipment in =
the data center. &nbsp;RFC 8238 is a prerequisite for<br class=3D"">this =
document, as it contains terminology that is considered<br =
class=3D"">normative. &nbsp;Many of these terms and methods may be =
applicable beyond<br class=3D"">the scope of this document as the =
technologies originally applied in<br class=3D"">the data center are =
deployed elsewhere.<br class=3D""><br class=3D"">This document is a =
product of the Benchmarking Methodology Working Group of the IETF.<br =
class=3D""><br class=3D""><br class=3D"">INFORMATIONAL: This memo =
provides information for the Internet community.<br class=3D"">It does =
not specify an Internet standard of any kind. Distribution of<br =
class=3D"">this memo is unlimited.<br class=3D""><br class=3D"">This =
announcement is sent to the IETF-Announce and rfc-dist lists.<br =
class=3D"">To subscribe or unsubscribe, see<br class=3D""> &nbsp;<a =
href=3D"https://www.ietf.org/mailman/listinfo/ietf-announce" =
class=3D"">https://www.ietf.org/mailman/listinfo/ietf-announce</a><br =
class=3D""> &nbsp;<a =
href=3D"https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist" =
class=3D"">https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist</a><br=
 class=3D""><br class=3D"">For searching the RFC series, see <a =
href=3D"https://www.rfc-editor.org/search" =
class=3D"">https://www.rfc-editor.org/search</a><br class=3D"">For =
downloading RFCs, see <a href=3D"https://www.rfc-editor.org/retrieve/bulk"=
 class=3D"">https://www.rfc-editor.org/retrieve/bulk</a><br class=3D""><br=
 class=3D"">Requests for special distribution should be addressed to =
either the<br class=3D"">author of the RFC in question, or to <a =
href=3D"mailto:rfc-editor@rfc-editor.org" =
class=3D"">rfc-editor@rfc-editor.org</a>. &nbsp;Unless<br =
class=3D"">specifically noted otherwise on the RFC itself, all RFCs are =
for<br class=3D"">unlimited distribution.<br class=3D""><br class=3D""><br=
 class=3D"">The RFC Editor Team<br class=3D"">Association Management =
Solutions, LLC<br class=3D""><br =
class=3D"">_______________________________________________<br =
class=3D"">bmwg mailing list<br class=3D""><a =
href=3D"mailto:bmwg@ietf.org" class=3D"">bmwg@ietf.org</a><br =
class=3D"">https://www.ietf.org/mailman/listinfo/bmwg<br =
class=3D""></div></div></blockquote></div><br =
class=3D""></div></body></html>=

--Apple-Mail=_7C4A47C3-6C3C-4876-94B2-9D1DABE93956--

