
From nobody Mon Apr  3 13:31:42 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 896EE129443 for <bmwg@ietfa.amsl.com>; Mon,  3 Apr 2017 13:31:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level: 
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wV2HwnGDj91I for <bmwg@ietfa.amsl.com>; Mon,  3 Apr 2017 13:31:39 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C268312942F for <bmwg@ietf.org>; Mon,  3 Apr 2017 13:31:39 -0700 (PDT)
Received: from pps.filterd (m0049295.ppops.net [127.0.0.1]) by m0049295.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v33KOkxC018982 for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:31:37 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049295.ppops.net-00191d01. with ESMTP id 29kuar53b3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Mon, 03 Apr 2017 16:31:37 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v33KVahA001694 for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:31:36 -0400
Received: from mlpi409.sfdc.sbc.com (mlpi409.sfdc.sbc.com [130.9.128.241]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v33KVRkO001473 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:31:31 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi409.sfdc.sbc.com (RSA Interceptor) for <bmwg@ietf.org>; Mon, 3 Apr 2017 20:31:16 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KVGAf014933 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:16 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KV8uk014053 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:11 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 70E35F03E0 for <bmwg@ietf.org>; Mon,  3 Apr 2017 16:31:08 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Mon, 3 Apr 2017 16:31:08 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: IETF WG state changed for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
Thread-Index: AQHSrLe5AKTr3d4tWkO5Yg4GF0SymKG0GPZg
Date: Mon, 3 Apr 2017 20:31:08 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F68B85@njmtexg5.research.att.com>
References: <149125081514.30894.5558839292426681517.idtracker@ietfa.amsl.com>
In-Reply-To: <149125081514.30894.5558839292426681517.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.231.250]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-03_18:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704030181
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/cfS3LmrdvHYGabTcxO6QS3v-1wY>
Subject: [bmwg] FW: IETF WG state changed for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Apr 2017 20:31:41 -0000

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBJRVRGIFNlY3JldGFyaWF0IFtt
YWlsdG86aWV0Zi1zZWNyZXRhcmlhdC1yZXBseUBpZXRmLm9yZ10gDQpTZW50OiBNb25kYXksIEFw
cmlsIDAzLCAyMDE3IDQ6MjAgUE0NClRvOiBibXdnLWNoYWlyc0BpZXRmLm9yZzsgZHJhZnQtaWV0
Zi1ibXdnLWlwdjYtdHJhbi10ZWNoLWJlbmNobWFya2luZ0BpZXRmLm9yZzsgTU9SVE9OLCBBTEZS
RUQgQyAoQUwpDQpTdWJqZWN0OiBJRVRGIFdHIHN0YXRlIGNoYW5nZWQgZm9yIGRyYWZ0LWlldGYt
Ym13Zy1pcHY2LXRyYW4tdGVjaC1iZW5jaG1hcmtpbmcNCg0KDQpUaGUgSUVURiBXRyBzdGF0ZSBv
ZiBkcmFmdC1pZXRmLWJtd2ctaXB2Ni10cmFuLXRlY2gtYmVuY2htYXJraW5nIGhhcyBiZWVuDQpj
aGFuZ2VkIHRvICJXRyBDb25zZW5zdXM6IFdhaXRpbmcgZm9yIFdyaXRlLVVwIiBmcm9tICJXRyBE
b2N1bWVudCIgYnkgQWwNCk1vcnRvbjoNCg0KaHR0cHM6Ly91cmxkZWZlbnNlLnByb29mcG9pbnQu
Y29tL3YyL3VybD91PWh0dHBzLTNBX19kYXRhdHJhY2tlci5pZXRmLm9yZ19kb2NfZHJhZnQtMkRp
ZXRmLTJEYm13Zy0yRGlwdjYtMkR0cmFuLTJEdGVjaC0yRGJlbmNobWFya2luZ18mZD1Ed0lDYVEm
Yz1MRllaLW85X0hVTWVNVFNRaWN2aklnJnI9T2ZzU3U4a1RJbHRWeUQxb0w3MmNCdyZtPUVUZkxN
SEtFMTcwRkdzSUFCVUVKR1FnMUI1blRfQVhiWk90eldrc0VKSjgmcz1xM0NsWGJZVnZBeTBLYXE3
TnVkd0VQSm16dHNfYnJLb3M4QlY1QVgtd1R3JmU9IA0KDQo=


From nobody Mon Apr  3 13:31:57 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F19C0129406 for <bmwg@ietfa.amsl.com>; Mon,  3 Apr 2017 13:31:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.397
X-Spam-Level: 
X-Spam-Status: No, score=-5.397 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.796, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iWkexDfxvby5 for <bmwg@ietfa.amsl.com>; Mon,  3 Apr 2017 13:31:52 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9C5FB12942F for <bmwg@ietf.org>; Mon,  3 Apr 2017 13:31:50 -0700 (PDT)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v33KP0rS003483 for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:31:47 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0083689.ppops.net-00191d01. with ESMTP id 29ku67n5t9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Mon, 03 Apr 2017 16:31:47 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KVkaC114967 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:47 -0500
Received: from dalint02.pst.cso.att.com (dalint02.pst.cso.att.com [135.31.133.160]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KVg9M114884 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:42 -0500
Received: from tlpd252.dadc.sbc.com (tlpd252.dadc.sbc.com [135.31.184.157]) by dalint02.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Mon, 3 Apr 2017 20:31:27 GMT
Received: from dadc.sbc.com (localhost [127.0.0.1]) by tlpd252.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KVRhl004491 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:27 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by tlpd252.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KVM2Y004346 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:23 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id F3E16E070C for <bmwg@ietf.org>; Mon,  3 Apr 2017 16:31:06 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Mon, 3 Apr 2017 16:31:22 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Tags changed for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
Thread-Index: AQHSrLj62IYIz6+SbEGor0BJdP/P6qG0GQmw
Date: Mon, 3 Apr 2017 20:31:22 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F68B8C@njmtexg5.research.att.com>
References: <149125135279.13586.6946280485806053688.idtracker@ietfa.amsl.com>
In-Reply-To: <149125135279.13586.6946280485806053688.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.231.250]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-03_18:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704030181
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/tdra_h5KlHpYVGM4yQyyTra9TwY>
Subject: [bmwg] FW: Tags changed for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Apr 2017 20:31:56 -0000

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBJRVRGIFNlY3JldGFyaWF0IFtt
YWlsdG86aWV0Zi1zZWNyZXRhcmlhdC1yZXBseUBpZXRmLm9yZ10gDQpTZW50OiBNb25kYXksIEFw
cmlsIDAzLCAyMDE3IDQ6MjkgUE0NClRvOiBibXdnLWNoYWlyc0BpZXRmLm9yZzsgZHJhZnQtaWV0
Zi1ibXdnLWlwdjYtdHJhbi10ZWNoLWJlbmNobWFya2luZ0BpZXRmLm9yZzsgTU9SVE9OLCBBTEZS
RUQgQyAoQUwpDQpTdWJqZWN0OiBUYWdzIGNoYW5nZWQgZm9yIGRyYWZ0LWlldGYtYm13Zy1pcHY2
LXRyYW4tdGVjaC1iZW5jaG1hcmtpbmcNCg0KDQpUaGUgdGFncyBvbiBkcmFmdC1pZXRmLWJtd2ct
aXB2Ni10cmFuLXRlY2gtYmVuY2htYXJraW5nIGhhdmUgYmVlbiBjaGFuZ2VkDQpieSBBbCBNb3J0
b246DQpodHRwczovL3VybGRlZmVuc2UucHJvb2Zwb2ludC5jb20vdjIvdXJsP3U9aHR0cHMtM0Ff
X2RhdGF0cmFja2VyLmlldGYub3JnX2RvY19kcmFmdC0yRGlldGYtMkRibXdnLTJEaXB2Ni0yRHRy
YW4tMkR0ZWNoLTJEYmVuY2htYXJraW5nXyZkPUR3SUNhUSZjPUxGWVotbzlfSFVNZU1UU1FpY3Zq
SWcmcj1PZnNTdThrVElsdFZ5RDFvTDcyY0J3Jm09SU9kSzdlUE1jeUZyMl9fSDNEeTlmWktXWnUx
T0V4UXh0cERwYmx2RjRiZyZzPVhWTEJDaWFvZGJoWDItR1UwYUxleDNuV25HUEtWY2F3Y0VuRTJw
N2hDdEUmZT0gDQoNClRhZyAiRG9jIFNoZXBoZXJkIEZvbGxvdy11cCBVbmRlcndheSIgYWRkZWQu
DQoNCg0K


From nobody Mon Apr  3 15:01:49 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EEDD21293FF for <bmwg@ietfa.amsl.com>; Mon,  3 Apr 2017 15:01:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.397
X-Spam-Level: 
X-Spam-Status: No, score=-5.397 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.796, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J04s3FszZ9RY for <bmwg@ietfa.amsl.com>; Mon,  3 Apr 2017 15:01:47 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 22DE91289B0 for <bmwg@ietf.org>; Mon,  3 Apr 2017 15:01:47 -0700 (PDT)
Received: from pps.filterd (m0049459.ppops.net [127.0.0.1]) by m0049459.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v33KOspm043234 for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:32:13 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049459.ppops.net-00191d01. with ESMTP id 29ku04wpdp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Mon, 03 Apr 2017 16:32:12 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v33KWC0i002826 for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:32:12 -0400
Received: from mlpi407.sfdc.sbc.com (mlpi407.sfdc.sbc.com [130.9.128.239]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v33KW4Ms002594 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Mon, 3 Apr 2017 16:32:05 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi407.sfdc.sbc.com (RSA Interceptor) for <bmwg@ietf.org>; Mon, 3 Apr 2017 20:32:00 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KW0Av018246 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:32:00 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v33KVqlG017849 for <bmwg@ietf.org>; Mon, 3 Apr 2017 15:31:52 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id 0AD8BE0524 for <bmwg@ietf.org>; Mon,  3 Apr 2017 16:31:52 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Mon, 3 Apr 2017 16:31:51 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Publication has been requested for draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05
Thread-Index: AQHSrLkOtSS1h90Q7EiDUT9XyK020KG0GRrw
Date: Mon, 3 Apr 2017 20:31:51 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F68BA0@njmtexg5.research.att.com>
References: <149125139036.10907.7304536099712383444.idtracker@ietfa.amsl.com>
In-Reply-To: <149125139036.10907.7304536099712383444.idtracker@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.231.250]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-03_18:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704030181
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/aY3wrj_tPoLZNebMRuMDIVKC4Cs>
Subject: [bmwg] FW: Publication has been requested for draft-ietf-bmwg-ipv6-tran-tech-benchmarking-05
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Apr 2017 22:01:49 -0000

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBNT1JUT04sIEFMRlJFRCBDIChB
TCkgDQpTZW50OiBNb25kYXksIEFwcmlsIDAzLCAyMDE3IDQ6MzAgUE0NClRvOiB3YXJyZW5Aa3Vt
YXJpLm5ldA0KQ2M6IE1PUlRPTiwgQUxGUkVEIEMgKEFMKTsgaWVzZy1zZWNyZXRhcnlAaWV0Zi5v
cmc7IGJtd2ctY2hhaXJzQGlldGYub3JnOyBNT1JUT04sIEFMRlJFRCBDIChBTCkNClN1YmplY3Q6
IFB1YmxpY2F0aW9uIGhhcyBiZWVuIHJlcXVlc3RlZCBmb3IgZHJhZnQtaWV0Zi1ibXdnLWlwdjYt
dHJhbi10ZWNoLWJlbmNobWFya2luZy0wNQ0KDQpBbCBNb3J0b24gaGFzIHJlcXVlc3RlZCBwdWJs
aWNhdGlvbiBvZiBkcmFmdC1pZXRmLWJtd2ctaXB2Ni10cmFuLXRlY2gtYmVuY2htYXJraW5nLTA1
IGFzIEluZm9ybWF0aW9uYWwgb24gYmVoYWxmIG9mIHRoZSBCTVdHIHdvcmtpbmcgZ3JvdXAuDQoN
ClBsZWFzZSB2ZXJpZnkgdGhlIGRvY3VtZW50J3Mgc3RhdGUgYXQgaHR0cHM6Ly91cmxkZWZlbnNl
LnByb29mcG9pbnQuY29tL3YyL3VybD91PWh0dHBzLTNBX19kYXRhdHJhY2tlci5pZXRmLm9yZ19k
b2NfZHJhZnQtMkRpZXRmLTJEYm13Zy0yRGlwdjYtMkR0cmFuLTJEdGVjaC0yRGJlbmNobWFya2lu
Z18mZD1Ed0lDYVEmYz1MRllaLW85X0hVTWVNVFNRaWN2aklnJnI9T2ZzU3U4a1RJbHRWeUQxb0w3
MmNCdyZtPTJ6LWUzSG5iV1Jna2wyX3hVU2s0dmpDdDhuS3VlWHgtQlN6SHRwU3lmOHMmcz1Sd3Ji
WTRxSlljQVFNSWJIdTAtVzZfNEdNbHlRN3FCUjFnMXJDY3VNSHRFJmU9IA0KDQo=


From nobody Mon Apr 10 10:38:29 2017
Return-Path: <lsmt@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 07CA5129548; Mon, 10 Apr 2017 10:11:55 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: Liaison Statement Management Tool <lsmt@ietf.org>
To: "Sarah Banks" <sbanks@encrypted.net>, "Al Morton" <acmorton@att.com>
Cc: Sarah Banks <sbanks@encrypted.net>, Benchmarking Methodology Discussion List <bmwg@ietf.org>, Scott Mansfield <Scott.Mansfield@Ericsson.com>, Warren Kumari <warren@kumari.net>, John Drake <jdrake@juniper.net>, Al Morton <acmorton@att.com>, jessy.rouyer@nokia.com, Benoit Claise <bclaise@cisco.com>, itu-t-liaison@iab.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.49.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149184431502.3180.9156973568705866791.idtracker@ietfa.amsl.com>
Date: Mon, 10 Apr 2017 10:11:55 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Sp_CED84JMXcf_8jGAmmNyniSxQ>
X-Mailman-Approved-At: Mon, 10 Apr 2017 10:38:22 -0700
Subject: [bmwg] New Liaison Statement, "LS response to IETF BMWG on proposal to develop a benchmarking methodology for ITU-T Recommendation Y.1731 (reply to IETF BMWG-LS2 (TD12/WP3))"
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Apr 2017 17:11:55 -0000

Title: LS response to IETF BMWG on proposal to develop a benchmarking methodology for ITU-T Recommendation Y.1731 (reply to IETF BMWG-LS2 (TD12/WP3))
Submission Date: 2017-04-10
URL of the IETF Web page: https://datatracker.ietf.org/liaison/1513/
Please reply by 2017-06-07
From: Hiroshi Ota <hiroshi.ota@itu.int>
To: Sarah Banks <sbanks@encrypted.net>,Al Morton <acmorton@att.com>
Cc: Sarah Banks <sbanks@encrypted.net>,Benchmarking Methodology Discussion List <bmwg@ietf.org>,Scott Mansfield <Scott.Mansfield@Ericsson.com>,Warren Kumari <warren@kumari.net>,John Drake <jdrake@juniper.net>,Al Morton <acmorton@att.com>,Benoit Claise <bclaise@cisco.com>,itu-t-liaison@iab.org
Response Contacts: jessy.rouyer@nokia.com
Technical Contacts: 
Purpose: For action

Body: Thank you for your liaison titled “Proposal to Develop a Benchmarking Methodology for ITU-T Recommendation Y.1731 OAM Performance”.

Q10/15 took note of your charter strictly limiting your work to laboratory characterization.

We appreciate that IETF BMWG requested our comments prior to considering this work proposal further. Regarding your recognizing that “compliance and interoperability specifications are the mandate of SG 11”, we would like to clarify that while ITU-T SG11 leads ITU’s work on conformance and interoperability (C&I) testing and is responsible for coordinating ITU’s C&I programme, ITU-T SG11 and SG15 currently have no agreement regarding where in ITU-T the work you are interested in with respect to Recommendation ITU-T G.8013/Y.1731 would take place. Further, we would like to clarify, by reference to the question text for Q10/15 (https://www.itu.int/en/ITU-T/studygroups/2017-2020/15/Pages/q10.aspx), that Q10/15 is currently not chartered to work on tasks such as the laboratory characterization of the accuracy of performance monitoring sought by the Internet Draft at https://tools.ietf.org/html/draft-jacpra-bmwg-pmtest-02. However, Q10/15 believes that such tasks should be performed within the organization that has responsibility for the corresponding data plane protocols. As such, Q10/15 believes that the proposed work should not be further progressed in IETF.

The next meeting of ITU-T SG15 is in Geneva, Switzerland, 19-30 June 2017.
Attachments:

    SG15-LS018
    https://www.ietf.org/lib/dt/documents/LIAISON/liaison-2017-04-10-itu-t-sg-15-bmwg-ls-response-to-ietf-bmwg-on-proposal-to-develop-a-benchmarking-methodology-for-itu-t-recommendation-y1731-reply-t-attachment-1.pdf


From nobody Fri Apr 14 09:47:27 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 963AF1294CE for <bmwg@ietfa.amsl.com>; Fri, 14 Apr 2017 09:47:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jrJzsH9UT0of for <bmwg@ietfa.amsl.com>; Fri, 14 Apr 2017 09:47:24 -0700 (PDT)
Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 42B61129449 for <bmwg@ietf.org>; Fri, 14 Apr 2017 09:47:23 -0700 (PDT)
Received: by mail-qt0-x22d.google.com with SMTP id n46so68236698qta.2 for <bmwg@ietf.org>; Fri, 14 Apr 2017 09:47:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=OS0R/DXz1efixYM+R/JW9hfnT4dedpICVw4Iun0cQZ0=; b=fVielvB4oHNTovFbUGFQCe8bKyqE8KiP0/wEgq0XsL9ZcXVsHpGN79BafuKQCxifn0 B/ODHj3bTL4Pvnp8dNlGBVUIKcqjs/XRCcTiGk5zVWwJUhPblSm7u96evCo+z6HJlfu1 ymmcid1yvXB6yibZIPpdPUB/gblFBr79Cs98/nzz1gTtY8JIwGnX7NtHuFxc0alwPa8L Ia4ImkjZTCAJxzA6br0WWc8bwLRbnZISVAWSars4kfK4hpuBqM1rJRr34NMI9vWPaxTV GTLgwRvJfkCgciyvTz58rBrjBqk7H5ULFo2Satdp/peQponfMjCesmDzFhtT+HpAL4r8 mMZw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=OS0R/DXz1efixYM+R/JW9hfnT4dedpICVw4Iun0cQZ0=; b=TZdziNWUGzU+kipc5chWPkvwd8W/tK99ba3uLOK7HujizZqZZIOuwhI7HVhJRRRI6W 8wLECOygbnSBJjWUk7dgCxdxHp0d0GwMi7KVyJ1WurkEGlaUx03PIgEYgyn31AAKeFQk H6ErkLV0eGKH1z92tF+i2BU9lOxEHaDPiQjSOpWOuTYWOrnRvyDzicpHrSm93UcwlwWL V0+n/em25KC2ob1WFDty6Iny4sSyafQTToLLvgq5+0fcfXtj9gdJ8LRUnln4gh1sKqCk tp7kI/xw8JSJ8wpVTN3jubJDLSS18u8VmijYcgD2ZWMC1U5OPJgAlaYFkqYZjRSgC6BQ Ob2g==
X-Gm-Message-State: AN3rC/4d0cbM6R0R9d6lasfqaVN8q7CWy91glzfCjaKnm4Jb/Hn6fGFM PflHZ7RQP6SO3mVohV/cEsnrOU6tpqCodK7bYw==
X-Received: by 10.200.51.79 with SMTP id u15mr7828620qta.171.1492188442274; Fri, 14 Apr 2017 09:47:22 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.12.183.136 with HTTP; Fri, 14 Apr 2017 09:46:41 -0700 (PDT)
From: Warren Kumari <warren@kumari.net>
Date: Fri, 14 Apr 2017 12:46:41 -0400
Message-ID: <CAHw9_iJ+PcuJVS3ougbFUQkT0OwmEXbRVqy=44bORwUHar-FzQ@mail.gmail.com>
To: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset=UTF-8
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/qXOzBvTa1AmZjDCl1-LMelD5PRI>
Subject: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Apr 2017 16:47:26 -0000

Hi there,

First off, thanks for a really well written document -- I've done some
benchmarking of routers and switches and similar, but never and
translation devices. This document explains how I would benchmark this
nicely.

I had a few minor nits which I think would be nice if you could
address before I start IETF LC - this should help the document sail
though the process.
While addressing these, there are also 2 editorial notes in the
Shepherd writeup, both (IMO) easy to address.


W

Section 1:
"ability to gracefully accommodate greater numbers of flows
   than the maximum number of flows which the DUT can operate normally."

[O] DUT
[P] [spell out DUT]
[R] first use of the acronym


Section 1.1. IPv6 Transition Technologies
"4. Encapsulation: The production network is assumed to have all three
domains, Domains A and B are IPvX specific, while the core ..."

[O] three domains, Domains A and B
[P] three domains; Domains A and B
[R] grammar


Section 5.1. Frame Formats and Sizes

"The two documents can be referred for the dual-stack transition technologies."

[O] referred
[P] referenced
[R] not sure what is meant; referenced is just a guess here. Or
"referred to"? Nither sounds great.



"The calculation method for the Ethernet, as well as a calculation
example are detailed in Appendix A. "

[O] calculation example are
[P] calculation example, are
[R] grammar


Section 6. Modifiers

   The idea of testing under different operational conditions was first
   introduced in [RFC2544](Section 11) and represents an important
   aspect of benchmarking network elements, as it emulates to some
   extent the conditions of a production environment. Section 6 of

[O]  as it emulates to some extent the
[P] as it emulates, to some extent, the
[R] grammar


Section 8. Additional Benchmarking Tests for Stateful IPv6 Transition
Technologies

   This section describes additional tests dedicated to the stateful
   IPv6 transition technologies. For the tests described in this
   section the DUT devices SHOULD follow the test setup and test

[O] section the DUT device
[P] section, the DUT device
[R] grammar


- If the DNS64 server implements caching and there is a cache hit
        then step 1 is followed by step 6 (and steps 2 through 5 are
        omitted).
 - If the domain name has an AAAA record then it is returned in

[O] If the domain name has an AAAA record then it is returned
[P] If the domain name has an AAAA record, then it is returned



(When all the domain names are cached then the results do not depend
on what percentage of the ...

[O] are cached then the
[P] are cached, the
[R] readability



(When all the domain names
   are cached then the results do not depend on what percentage of the
   domain names have AAAA records, thus these combinations are not
   worth testing one by one.)
[O] are cached then the
[P] are cached, the
[R] readability


queries at the required frequency using up not more than the half of
the timeout time.

   Remark: a sample open-source test program, dns64perf++ is available

[O] dns64perf++ is
[P] dns64perf++, is
[R] grammar



  For encapsulation transition technologies a m:n setup can be

[O]  For encapsulation transition technologies
[P]  For encapsulation transition technologies,
[R] grammar

   created, where m is the number of flows applied to the same client
   device and n the number of client devices connected to the same
   server device.
   For the translation based transition technologies the client devices

[O] For the translation based transition technologies the client
[P] For the translation based transition technologies, the client
[R] grammar



 In other words, if flow I is started at time x, flow i+1

[O] flow I is started at time x, flow i+1
[R] I and i should be consistent; either use upper or lower case. Same
for other variables.



Section 11. NAT44 and NAT66

   Although these technologies are not the primarily scope of this

[O] the primarily scope
[P] the primary scope
[R] word choice


Section 12. Summarizing function and variation

For a fine grain analysis of the frequency distribution of the data,

[O] fine grain analysis
[P] fine grained analysis
[R] word choice


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Sat Apr 15 02:35:25 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F5F6124D6C; Sat, 15 Apr 2017 02:35:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level: 
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id spPhUFJL-vad; Sat, 15 Apr 2017 02:35:21 -0700 (PDT)
Received: from mailproxy2.rcs-rds.ro (mailproxy2.rcs-rds.ro [212.54.127.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E232912420B; Sat, 15 Apr 2017 02:35:20 -0700 (PDT)
Received: from [10.252.8.18] (unknown [10.252.8.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy2.rcs-rds.ro (Postfix) with ESMTPSA id E16ED10411E; Sat, 15 Apr 2017 12:35:17 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1492248918; bh=lshIfw9jQaiPe7gHmEI4lDKStPIkzU+IAwuWL9QxVYY=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=MeLhqe22lxweT6nmzAbT4VegxDmi7QS5dO3EGj/uQV4dub90qLp2UpJvFfGe7mqOZ 5mGCOcwZQQ7lY2i0WcwRhkf5eP60R+Y6vKDZhOTZDNkAhwOLuA8RUIzjdWRhOQCkp5 USKz3WmXyvz/DoCDk9Ul4XSzluWIUQqKkI5ihub4=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\))
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
In-Reply-To: <CAHw9_iJ+PcuJVS3ougbFUQkT0OwmEXbRVqy=44bORwUHar-FzQ@mail.gmail.com>
Date: Sat, 15 Apr 2017 12:35:11 +0300
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org, bmwg-chairs@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <ADC8E1B6-CEC2-42CE-9BB8-F4A8EE8630EF@rcs-rds.ro>
References: <CAHw9_iJ+PcuJVS3ougbFUQkT0OwmEXbRVqy=44bORwUHar-FzQ@mail.gmail.com>
To: Warren Kumari <warren@kumari.net>
X-Mailer: Apple Mail (2.3259)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/dbvFa2bvR-yIuC58t3woD8wFiuI>
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 15 Apr 2017 09:35:24 -0000

Hello Warren,

Thank you very much for your kind words and for the detailed review.
We will revise the draft considering the Shepherd writeup and your =
comments.=20
Is there a deadline we should be aware of?

Best regards,
Marius

> On Apr 14, 2017, at 7:46 PM, Warren Kumari <warren@kumari.net> wrote:
>=20
> Hi there,
>=20
> First off, thanks for a really well written document -- I've done some
> benchmarking of routers and switches and similar, but never and
> translation devices. This document explains how I would benchmark this
> nicely.
>=20
> I had a few minor nits which I think would be nice if you could
> address before I start IETF LC - this should help the document sail
> though the process.
> While addressing these, there are also 2 editorial notes in the
> Shepherd writeup, both (IMO) easy to address.
>=20
>=20
> W
>=20
> Section 1:
> "ability to gracefully accommodate greater numbers of flows
>   than the maximum number of flows which the DUT can operate =
normally."
>=20
> [O] DUT
> [P] [spell out DUT]
> [R] first use of the acronym
>=20
>=20
> Section 1.1. IPv6 Transition Technologies
> "4. Encapsulation: The production network is assumed to have all three
> domains, Domains A and B are IPvX specific, while the core ..."
>=20
> [O] three domains, Domains A and B
> [P] three domains; Domains A and B
> [R] grammar
>=20
>=20
> Section 5.1. Frame Formats and Sizes
>=20
> "The two documents can be referred for the dual-stack transition =
technologies."
>=20
> [O] referred
> [P] referenced
> [R] not sure what is meant; referenced is just a guess here. Or
> "referred to"? Nither sounds great.
>=20
>=20
>=20
> "The calculation method for the Ethernet, as well as a calculation
> example are detailed in Appendix A. "
>=20
> [O] calculation example are
> [P] calculation example, are
> [R] grammar
>=20
>=20
> Section 6. Modifiers
>=20
>   The idea of testing under different operational conditions was first
>   introduced in [RFC2544](Section 11) and represents an important
>   aspect of benchmarking network elements, as it emulates to some
>   extent the conditions of a production environment. Section 6 of
>=20
> [O]  as it emulates to some extent the
> [P] as it emulates, to some extent, the
> [R] grammar
>=20
>=20
> Section 8. Additional Benchmarking Tests for Stateful IPv6 Transition
> Technologies
>=20
>   This section describes additional tests dedicated to the stateful
>   IPv6 transition technologies. For the tests described in this
>   section the DUT devices SHOULD follow the test setup and test
>=20
> [O] section the DUT device
> [P] section, the DUT device
> [R] grammar
>=20
>=20
> - If the DNS64 server implements caching and there is a cache hit
>        then step 1 is followed by step 6 (and steps 2 through 5 are
>        omitted).
> - If the domain name has an AAAA record then it is returned in
>=20
> [O] If the domain name has an AAAA record then it is returned
> [P] If the domain name has an AAAA record, then it is returned
>=20
>=20
>=20
> (When all the domain names are cached then the results do not depend
> on what percentage of the ...
>=20
> [O] are cached then the
> [P] are cached, the
> [R] readability
>=20
>=20
>=20
> (When all the domain names
>   are cached then the results do not depend on what percentage of the
>   domain names have AAAA records, thus these combinations are not
>   worth testing one by one.)
> [O] are cached then the
> [P] are cached, the
> [R] readability
>=20
>=20
> queries at the required frequency using up not more than the half of
> the timeout time.
>=20
>   Remark: a sample open-source test program, dns64perf++ is available
>=20
> [O] dns64perf++ is
> [P] dns64perf++, is
> [R] grammar
>=20
>=20
>=20
>  For encapsulation transition technologies a m:n setup can be
>=20
> [O]  For encapsulation transition technologies
> [P]  For encapsulation transition technologies,
> [R] grammar
>=20
>   created, where m is the number of flows applied to the same client
>   device and n the number of client devices connected to the same
>   server device.
>   For the translation based transition technologies the client devices
>=20
> [O] For the translation based transition technologies the client
> [P] For the translation based transition technologies, the client
> [R] grammar
>=20
>=20
>=20
> In other words, if flow I is started at time x, flow i+1
>=20
> [O] flow I is started at time x, flow i+1
> [R] I and i should be consistent; either use upper or lower case. Same
> for other variables.
>=20
>=20
>=20
> Section 11. NAT44 and NAT66
>=20
>   Although these technologies are not the primarily scope of this
>=20
> [O] the primarily scope
> [P] the primary scope
> [R] word choice
>=20
>=20
> Section 12. Summarizing function and variation
>=20
> For a fine grain analysis of the frequency distribution of the data,
>=20
> [O] fine grain analysis
> [P] fine grained analysis
> [R] word choice
>=20
>=20
> --=20
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>   ---maf


From nobody Sat Apr 15 05:57:28 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55855126D85; Sat, 15 Apr 2017 05:57:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fkl4sB1QSP7b; Sat, 15 Apr 2017 05:57:26 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B628126B72; Sat, 15 Apr 2017 05:57:26 -0700 (PDT)
Received: from pps.filterd (m0049295.ppops.net [127.0.0.1]) by m0049295.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v3FCshGP042952; Sat, 15 Apr 2017 08:57:23 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049295.ppops.net-00191d01. with ESMTP id 29ufg6vc3r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 15 Apr 2017 08:57:23 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3FCvMw1025436; Sat, 15 Apr 2017 08:57:22 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3FCvIkg025420 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 15 Apr 2017 08:57:19 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor); Sat, 15 Apr 2017 12:57:02 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3FCv2SY007436; Sat, 15 Apr 2017 07:57:02 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3FCuuED007235; Sat, 15 Apr 2017 07:56:57 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-green.research.att.com (Postfix) with ESMTP id 90DC4E070C; Sat, 15 Apr 2017 08:56:37 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Sat, 15 Apr 2017 08:56:56 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Marius Georgescu <marius.georgescu@rcs-rds.ro>, Warren Kumari <warren@kumari.net>
CC: "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org" <draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
Thread-Index: AQHStT7P6GGywmc5/UW/liz/M3vuoqHGb7iA///xyCA=
Date: Sat, 15 Apr 2017 12:56:56 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F70C23@njmtexg5.research.att.com>
References: <CAHw9_iJ+PcuJVS3ougbFUQkT0OwmEXbRVqy=44bORwUHar-FzQ@mail.gmail.com> <ADC8E1B6-CEC2-42CE-9BB8-F4A8EE8630EF@rcs-rds.ro>
In-Reply-To: <ADC8E1B6-CEC2-42CE-9BB8-F4A8EE8630EF@rcs-rds.ro>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-15_11:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704150112
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/JOC8hw9iiIdMMnds5bfnyeuV5DU>
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 15 Apr 2017 12:57:28 -0000

Hi Marius,

Since all ADs read *many* documents, it is universally
preferred to respond quickly as you've done, and follow-up
with a revised version in this case (and when requested
in the steps that follow).=20

When the revised version is available, the next step will=20
likely be an IETF-wide Last Call. This is when the various=20
Directorates that help the Area Directors will review=20
the draft, and there will be some more comments as a result.

After Last Call, the draft will be scheduled for an IESG
telechat, and the other Area Directors will likely provide=20
even more comments. Prompt replies and resolving text will
again keep your draft moving.

If you have any questions as we proceed, don't hesitate to
get in touch.

regards,
Al
bmwg co-chair and doc shepherd



> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Marius Georgescu
> Sent: Saturday, April 15, 2017 5:35 AM
> To: Warren Kumari
> Cc: bmwg-chairs@ietf.org; draft-ietf-bmwg-ipv6-tran-tech-
> benchmarking@ietf.org; bmwg@ietf.org
> Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-
> benchmarking
>=20
> Hello Warren,
>=20
> Thank you very much for your kind words and for the detailed review.
> We will revise the draft considering the Shepherd writeup and your
> comments.
> Is there a deadline we should be aware of?
>=20
> Best regards,
> Marius
>=20
> > On Apr 14, 2017, at 7:46 PM, Warren Kumari <warren@kumari.net> wrote:
> >
> > Hi there,
> >
> > First off, thanks for a really well written document -- I've done some
> > benchmarking of routers and switches and similar, but never and
> > translation devices. This document explains how I would benchmark this
> > nicely.
> >
> > I had a few minor nits which I think would be nice if you could
> > address before I start IETF LC - this should help the document sail
> > though the process.
> > While addressing these, there are also 2 editorial notes in the
> > Shepherd writeup, both (IMO) easy to address.
> >
> >
> > W
> >
> > Section 1:
> > "ability to gracefully accommodate greater numbers of flows
> >   than the maximum number of flows which the DUT can operate
> normally."
> >
> > [O] DUT
> > [P] [spell out DUT]
> > [R] first use of the acronym
> >
> >
> > Section 1.1. IPv6 Transition Technologies
> > "4. Encapsulation: The production network is assumed to have all three
> > domains, Domains A and B are IPvX specific, while the core ..."
> >
> > [O] three domains, Domains A and B
> > [P] three domains; Domains A and B
> > [R] grammar
> >
> >
> > Section 5.1. Frame Formats and Sizes
> >
> > "The two documents can be referred for the dual-stack transition
> technologies."
> >
> > [O] referred
> > [P] referenced
> > [R] not sure what is meant; referenced is just a guess here. Or
> > "referred to"? Nither sounds great.
> >
> >
> >
> > "The calculation method for the Ethernet, as well as a calculation
> > example are detailed in Appendix A. "
> >
> > [O] calculation example are
> > [P] calculation example, are
> > [R] grammar
> >
> >
> > Section 6. Modifiers
> >
> >   The idea of testing under different operational conditions was first
> >   introduced in [RFC2544](Section 11) and represents an important
> >   aspect of benchmarking network elements, as it emulates to some
> >   extent the conditions of a production environment. Section 6 of
> >
> > [O]  as it emulates to some extent the
> > [P] as it emulates, to some extent, the
> > [R] grammar
> >
> >
> > Section 8. Additional Benchmarking Tests for Stateful IPv6 Transition
> > Technologies
> >
> >   This section describes additional tests dedicated to the stateful
> >   IPv6 transition technologies. For the tests described in this
> >   section the DUT devices SHOULD follow the test setup and test
> >
> > [O] section the DUT device
> > [P] section, the DUT device
> > [R] grammar
> >
> >
> > - If the DNS64 server implements caching and there is a cache hit
> >        then step 1 is followed by step 6 (and steps 2 through 5 are
> >        omitted).
> > - If the domain name has an AAAA record then it is returned in
> >
> > [O] If the domain name has an AAAA record then it is returned
> > [P] If the domain name has an AAAA record, then it is returned
> >
> >
> >
> > (When all the domain names are cached then the results do not depend
> > on what percentage of the ...
> >
> > [O] are cached then the
> > [P] are cached, the
> > [R] readability
> >
> >
> >
> > (When all the domain names
> >   are cached then the results do not depend on what percentage of the
> >   domain names have AAAA records, thus these combinations are not
> >   worth testing one by one.)
> > [O] are cached then the
> > [P] are cached, the
> > [R] readability
> >
> >
> > queries at the required frequency using up not more than the half of
> > the timeout time.
> >
> >   Remark: a sample open-source test program, dns64perf++ is available
> >
> > [O] dns64perf++ is
> > [P] dns64perf++, is
> > [R] grammar
> >
> >
> >
> >  For encapsulation transition technologies a m:n setup can be
> >
> > [O]  For encapsulation transition technologies
> > [P]  For encapsulation transition technologies,
> > [R] grammar
> >
> >   created, where m is the number of flows applied to the same client
> >   device and n the number of client devices connected to the same
> >   server device.
> >   For the translation based transition technologies the client devices
> >
> > [O] For the translation based transition technologies the client
> > [P] For the translation based transition technologies, the client
> > [R] grammar
> >
> >
> >
> > In other words, if flow I is started at time x, flow i+1
> >
> > [O] flow I is started at time x, flow i+1
> > [R] I and i should be consistent; either use upper or lower case. Same
> > for other variables.
> >
> >
> >
> > Section 11. NAT44 and NAT66
> >
> >   Although these technologies are not the primarily scope of this
> >
> > [O] the primarily scope
> > [P] the primary scope
> > [R] word choice
> >
> >
> > Section 12. Summarizing function and variation
> >
> > For a fine grain analysis of the frequency distribution of the data,
> >
> > [O] fine grain analysis
> > [P] fine grained analysis
> > [R] word choice
> >
> >
> > --
> > I don't think the execution is relevant when it was obviously a bad
> > idea in the first place.
> > This is like putting rabid weasels in your pants, and later expressing
> > regret at having chosen those particular rabid weasels and that pair
> > of pants.
> >   ---maf
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DAUvQF5fSs3nruuM6VEndHNo4=
IEW
> FISyixgNCDlimEFk&s=3DwgtEceg7IsJIzEfu1N0VBNJcuYUx4X7fqNHAJmsV2AE&e=3D


From nobody Mon Apr 17 08:44:56 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id BF6CD1314A1; Mon, 17 Apr 2017 08:44:45 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.49.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149244388572.17763.6511045661474480726@ietfa.amsl.com>
Date: Mon, 17 Apr 2017 08:44:45 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/TXK4ShdzwafFcDU3_n9Nn-OS93Y>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-vswitch-opnfv-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Apr 2017 15:44:46 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Virtual Switches in OPNFV
        Authors         : Maryam Tahhan
                          Billy O'Mahony
                          Al Morton
	Filename        : draft-ietf-bmwg-vswitch-opnfv-02.txt
	Pages           : 21
	Date            : 2017-04-17

Abstract:
   This memo describes the progress of the Open Platform for NFV (OPNFV)
   project on virtual switch performance "VSWITCHPERF".  This project
   intends to build on the current and completed work of the
   Benchmarking Methodology Working Group in IETF, by referencing
   existing literature.  The Benchmarking Methodology Working Group has
   traditionally conducted laboratory characterization of dedicated
   physical implementations of internetworking functions.  Therefore,
   this memo begins to describe the additional considerations when
   virtual switches are implemented in general-purpose hardware.  The
   expanded tests and benchmarks are also influenced by the OPNFV
   mission to support virtualization of the "telco" infrastructure.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-vswitch-opnfv-02
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-vswitch-opnfv-02

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-vswitch-opnfv-02


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Mon Apr 17 09:03:20 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B16DB131513; Mon, 17 Apr 2017 09:03:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xEHsbcGkfFFK; Mon, 17 Apr 2017 09:03:17 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E0EAB12F28A; Mon, 17 Apr 2017 09:03:13 -0700 (PDT)
Received: from pps.filterd (m0049462.ppops.net [127.0.0.1]) by m0049462.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v3HFt2TU000891; Mon, 17 Apr 2017 12:03:11 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049462.ppops.net-00191d01. with ESMTP id 29vyxv2m1p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Apr 2017 12:03:10 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3HG38ZL014901; Mon, 17 Apr 2017 12:03:10 -0400
Received: from mlpi409.sfdc.sbc.com (mlpi409.sfdc.sbc.com [130.9.128.241]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3HG2xUU014651 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 17 Apr 2017 12:03:03 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi409.sfdc.sbc.com (RSA Interceptor); Mon, 17 Apr 2017 16:02:48 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3HG2mkv009471; Mon, 17 Apr 2017 11:02:48 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3HG2gvf009137; Mon, 17 Apr 2017 11:02:42 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 1330DEF541; Mon, 17 Apr 2017 12:02:42 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Mon, 17 Apr 2017 12:02:41 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>
Thread-Topic: I-D Action: draft-ietf-bmwg-vswitch-opnfv-02.txt
Thread-Index: AQHSt5GTPcmqksYySUSbgxewFMyTf6HJt2jg
Date: Mon, 17 Apr 2017 16:02:41 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F710A7@njmtexg5.research.att.com>
References: <149244388572.17763.6511045661474480726@ietfa.amsl.com>
In-Reply-To: <149244388572.17763.6511045661474480726@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-17_14:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704170143
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/-u_Ug3rq4j6TtjNlGoghx9A3G4U>
Subject: [bmwg] FW: I-D Action: draft-ietf-bmwg-vswitch-opnfv-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Apr 2017 16:03:19 -0000

BMWG,
This draft expired long after WGLC,=20
so I've submitted a trivial update=20
to change the date and fix some URLs
that broke following OPNFV's Wiki Migration.
Al

-----Original Message-----
From: I-D-Announce [mailto:i-d-announce-bounces@ietf.org] On Behalf Of inte=
rnet-drafts@ietf.org
Sent: Monday, April 17, 2017 11:45 AM
To: i-d-announce@ietf.org
Cc: bmwg@ietf.org
Subject: I-D Action: draft-ietf-bmwg-vswitch-opnfv-02.txt


A New Internet-Draft is available from the on-line Internet-Drafts director=
ies.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Virtual Switches in OPNFV
        Authors         : Maryam Tahhan
                          Billy O'Mahony
                          Al Morton
	Filename        : draft-ietf-bmwg-vswitch-opnfv-02.txt
	Pages           : 21
	Date            : 2017-04-17

Abstract:
   This memo describes the progress of the Open Platform for NFV (OPNFV)
   project on virtual switch performance "VSWITCHPERF".  This project
   intends to build on the current and completed work of the
   Benchmarking Methodology Working Group in IETF, by referencing
   existing literature.  The Benchmarking Methodology Working Group has
   traditionally conducted laboratory characterization of dedicated
   physical implementations of internetworking functions.  Therefore,
   this memo begins to describe the additional considerations when
   virtual switches are implemented in general-purpose hardware.  The
   expanded tests and benchmarks are also influenced by the OPNFV
   mission to support virtualization of the "telco" infrastructure.



The IETF datatracker status page for this draft is:
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.org=
_doc_draft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv_&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQ=
icvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX=
_AlGVc&s=3D6vW7Rn1CLNS6IMn-CzUvahpfx1IRG-vLu6OyC6dxwac&e=3D=20

There are also htmlized versions available at:
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__tools.ietf.org_html_=
draft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv-2D02&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQi=
cvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_=
AlGVc&s=3DysSokYvIEmIcJi3xXgExYO8C0Lq4HZ7pdnYtrktbvNM&e=3D=20
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.org=
_doc_html_draft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv-2D02&d=3DDwICAg&c=3DLFYZ-o9=
_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHST=
feH8p9vIX_AlGVc&s=3DObVw1_C8LZ4Xm4hbr3dFH0y5nSEx22buynICB3jqe2k&e=3D=20

A diff from the previous version is available at:
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.ietf.org_rfcdiff=
-3Furl2-3Ddraft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv-2D02&d=3DDwICAg&c=3DLFYZ-o9=
_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHST=
feH8p9vIX_AlGVc&s=3DUNtcjwlxwySrNLcieT_MhCpq8LhnrLtcc-N3BiGCtHA&e=3D=20


Please note that it may take a couple of minutes from the time of submissio=
n
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
https://urldefense.proofpoint.com/v2/url?u=3Dftp-3A__ftp.ietf.org_internet-=
2Ddrafts_&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&=
m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3Dzq0CKZEiTbED5gL-XhNF7n9=
ZOq0aNI9slkN6R5s_MQA&e=3D=20

_______________________________________________
I-D-Announce mailing list
I-D-Announce@ietf.org
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.ietf.org_mailman=
_listinfo_i-2Dd-2Dannounce&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8=
kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3DWK_aU9=
ZL3BR_cvzJcffJymLc5Keu9XAOdOladHCo8Cw&e=3D=20
Internet-Draft directories: https://urldefense.proofpoint.com/v2/url?u=3Dht=
tp-3A__www.ietf.org_shadow.html&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DO=
fsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3DN=
Bq2Ejodc_989q2agGgcGXlHcyWp3j7ZkiSOPGOWLPU&e=3D=20
or https://urldefense.proofpoint.com/v2/url?u=3Dftp-3A__ftp.ietf.org_ietf_1=
shadow-2Dsites.txt&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD=
1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3DbxjbjC90oyZ74N=
saJKf-DoL3cwejnWbj-1jHA-OEMfY&e=3D=20


From nobody Mon Apr 17 15:06:10 2017
Return-Path: <session_request_developers@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B6C5131599; Mon, 17 Apr 2017 15:06:08 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: "\"IETF Meeting Session Request Tool\"" <session_request_developers@ietf.org>
To: <session-request@ietf.org>
Cc: warren@kumari.net, bmwg-chairs@ietf.org, acmorton@att.com, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.49.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149246676814.17807.15852647107915052477.idtracker@ietfa.amsl.com>
Date: Mon, 17 Apr 2017 15:06:08 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/BgnNdwDlt6FL7KXssCMjt-N_U8Y>
Subject: [bmwg] bmwg - New Meeting Session Request for IETF 99
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Apr 2017 22:06:08 -0000

A new meeting session request has just been submitted by Al Morton, a Chair of the bmwg working group.


---------------------------------------------------------
Working Group Name: Benchmarking Methodology
Area Name: Operations and Management Area
Session Requester: Al Morton

Number of Sessions: 1
Length of Session(s):  2.5 Hours
Number of Attendees: 30
Conflicts to Avoid: 
 First Priority: nfvrg ippm quic ntp idr tsvwg tictoc sfc
 Second Priority: irtfopen xrblock tsvarea rmcat saag



People who must be present:
  Al Morton
  Sarah Banks
  Warren Kumari

Resources Requested:

Special Requests:
  Much better attendance and participation when session NOT on Friday
---------------------------------------------------------


From nobody Tue Apr 18 00:50:01 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F6E21317F7; Tue, 18 Apr 2017 00:49:51 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.49.1
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149250179122.1682.17362390067340069920@ietfa.amsl.com>
Date: Tue, 18 Apr 2017 00:49:51 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/z7WzMEo4NpZlUvh9JvPnGfe7SOg>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Apr 2017 07:49:51 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition Technologies
        Authors         : Marius Georgescu
                          Liviu Pislaru
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06.txt
	Pages           : 28
	Date            : 2017-04-18

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a metric for benchmarking load
   scalability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Tue Apr 18 00:57:46 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3807C1243F6; Tue, 18 Apr 2017 00:57:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level: 
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MhCxuu_njETq; Tue, 18 Apr 2017 00:57:37 -0700 (PDT)
Received: from mailproxy3.rcs-rds.ro (mailproxy3.rcs-rds.ro [212.54.120.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EFB9C1317C7; Tue, 18 Apr 2017 00:57:36 -0700 (PDT)
Received: from [10.172.177.179] (unknown [10.172.177.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailproxy3.rcs-rds.ro (Postfix) with ESMTPSA id CD3CFC3E28; Tue, 18 Apr 2017 10:57:33 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1492502253; bh=XwJ0hv/ShLw0F8hvmVREf5rFcxzUqt/m4S2BG2TgO7E=; h=From:Subject:Date:In-Reply-To:Cc:To:References; b=lpz2LkpUUGPr4VGBJ+sVZ2j/P4FVXL0CMcW/Gi1y0HnT45LQCYXnNnk1Cl2REWUZe sT35KIXmsWZqAfP37EPTUwKIt9cYqVF6s6X9DGbIkw1f+TWbpZR+/+OoOSHyk4m7gm aaMSI0I2AQay/pjrnxi04t2M8YEVBoq8nstSMZro=
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
Message-Id: <C7D080C6-E54B-49F9-BB09-B7F6126D6237@rcs-rds.ro>
Content-Type: multipart/alternative; boundary="Apple-Mail=_1435E284-C185-44DE-A770-626C1B94F2ED"
Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\))
Date: Tue, 18 Apr 2017 10:57:24 +0300
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25F70C23@njmtexg5.research.att.com>
Cc: Warren Kumari <warren@kumari.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org" <draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org>,  "bmwg@ietf.org" <bmwg@ietf.org>
To: Alfred C Morton <acmorton@att.com>
References: <CAHw9_iJ+PcuJVS3ougbFUQkT0OwmEXbRVqy=44bORwUHar-FzQ@mail.gmail.com> <ADC8E1B6-CEC2-42CE-9BB8-F4A8EE8630EF@rcs-rds.ro> <4D7F4AD313D3FC43A053B309F97543CF25F70C23@njmtexg5.research.att.com>
X-Mailer: Apple Mail (2.3259)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/i5-Ctq4LbRQBzAUVYsossALSMHE>
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Apr 2017 07:57:40 -0000

--Apple-Mail=_1435E284-C185-44DE-A770-626C1B94F2ED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hello Al,

Thank you for your kind help with this draft.
The revised version covers the received comments.
=
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06=
 =
<https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-0=
6>
If there are any other issues, please let us know.

Best regards,
Marius

> On Apr 15, 2017, at 3:56 PM, MORTON, ALFRED C (AL) <acmorton@att.com> =
wrote:
>=20
> Hi Marius,
>=20
> Since all ADs read *many* documents, it is universally
> preferred to respond quickly as you've done, and follow-up
> with a revised version in this case (and when requested
> in the steps that follow).=20
>=20
> When the revised version is available, the next step will=20
> likely be an IETF-wide Last Call. This is when the various=20
> Directorates that help the Area Directors will review=20
> the draft, and there will be some more comments as a result.
>=20
> After Last Call, the draft will be scheduled for an IESG
> telechat, and the other Area Directors will likely provide=20
> even more comments. Prompt replies and resolving text will
> again keep your draft moving.
>=20
> If you have any questions as we proceed, don't hesitate to
> get in touch.
>=20
> regards,
> Al
> bmwg co-chair and doc shepherd
>=20
>=20
>=20
>> -----Original Message-----
>> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Marius =
Georgescu
>> Sent: Saturday, April 15, 2017 5:35 AM
>> To: Warren Kumari
>> Cc: bmwg-chairs@ietf.org; draft-ietf-bmwg-ipv6-tran-tech-
>> benchmarking@ietf.org; bmwg@ietf.org
>> Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-
>> benchmarking
>>=20
>> Hello Warren,
>>=20
>> Thank you very much for your kind words and for the detailed review.
>> We will revise the draft considering the Shepherd writeup and your
>> comments.
>> Is there a deadline we should be aware of?
>>=20
>> Best regards,
>> Marius
>>=20
>>> On Apr 14, 2017, at 7:46 PM, Warren Kumari <warren@kumari.net> =
wrote:
>>>=20
>>> Hi there,
>>>=20
>>> First off, thanks for a really well written document -- I've done =
some
>>> benchmarking of routers and switches and similar, but never and
>>> translation devices. This document explains how I would benchmark =
this
>>> nicely.
>>>=20
>>> I had a few minor nits which I think would be nice if you could
>>> address before I start IETF LC - this should help the document sail
>>> though the process.
>>> While addressing these, there are also 2 editorial notes in the
>>> Shepherd writeup, both (IMO) easy to address.
>>>=20
>>>=20
>>> W
>>>=20
>>> Section 1:
>>> "ability to gracefully accommodate greater numbers of flows
>>>  than the maximum number of flows which the DUT can operate
>> normally."
>>>=20
>>> [O] DUT
>>> [P] [spell out DUT]
>>> [R] first use of the acronym
>>>=20
>>>=20
>>> Section 1.1. IPv6 Transition Technologies
>>> "4. Encapsulation: The production network is assumed to have all =
three
>>> domains, Domains A and B are IPvX specific, while the core ..."
>>>=20
>>> [O] three domains, Domains A and B
>>> [P] three domains; Domains A and B
>>> [R] grammar
>>>=20
>>>=20
>>> Section 5.1. Frame Formats and Sizes
>>>=20
>>> "The two documents can be referred for the dual-stack transition
>> technologies."
>>>=20
>>> [O] referred
>>> [P] referenced
>>> [R] not sure what is meant; referenced is just a guess here. Or
>>> "referred to"? Nither sounds great.
>>>=20
>>>=20
>>>=20
>>> "The calculation method for the Ethernet, as well as a calculation
>>> example are detailed in Appendix A. "
>>>=20
>>> [O] calculation example are
>>> [P] calculation example, are
>>> [R] grammar
>>>=20
>>>=20
>>> Section 6. Modifiers
>>>=20
>>>  The idea of testing under different operational conditions was =
first
>>>  introduced in [RFC2544](Section 11) and represents an important
>>>  aspect of benchmarking network elements, as it emulates to some
>>>  extent the conditions of a production environment. Section 6 of
>>>=20
>>> [O]  as it emulates to some extent the
>>> [P] as it emulates, to some extent, the
>>> [R] grammar
>>>=20
>>>=20
>>> Section 8. Additional Benchmarking Tests for Stateful IPv6 =
Transition
>>> Technologies
>>>=20
>>>  This section describes additional tests dedicated to the stateful
>>>  IPv6 transition technologies. For the tests described in this
>>>  section the DUT devices SHOULD follow the test setup and test
>>>=20
>>> [O] section the DUT device
>>> [P] section, the DUT device
>>> [R] grammar
>>>=20
>>>=20
>>> - If the DNS64 server implements caching and there is a cache hit
>>>       then step 1 is followed by step 6 (and steps 2 through 5 are
>>>       omitted).
>>> - If the domain name has an AAAA record then it is returned in
>>>=20
>>> [O] If the domain name has an AAAA record then it is returned
>>> [P] If the domain name has an AAAA record, then it is returned
>>>=20
>>>=20
>>>=20
>>> (When all the domain names are cached then the results do not depend
>>> on what percentage of the ...
>>>=20
>>> [O] are cached then the
>>> [P] are cached, the
>>> [R] readability
>>>=20
>>>=20
>>>=20
>>> (When all the domain names
>>>  are cached then the results do not depend on what percentage of the
>>>  domain names have AAAA records, thus these combinations are not
>>>  worth testing one by one.)
>>> [O] are cached then the
>>> [P] are cached, the
>>> [R] readability
>>>=20
>>>=20
>>> queries at the required frequency using up not more than the half of
>>> the timeout time.
>>>=20
>>>  Remark: a sample open-source test program, dns64perf++ is available
>>>=20
>>> [O] dns64perf++ is
>>> [P] dns64perf++, is
>>> [R] grammar
>>>=20
>>>=20
>>>=20
>>> For encapsulation transition technologies a m:n setup can be
>>>=20
>>> [O]  For encapsulation transition technologies
>>> [P]  For encapsulation transition technologies,
>>> [R] grammar
>>>=20
>>>  created, where m is the number of flows applied to the same client
>>>  device and n the number of client devices connected to the same
>>>  server device.
>>>  For the translation based transition technologies the client =
devices
>>>=20
>>> [O] For the translation based transition technologies the client
>>> [P] For the translation based transition technologies, the client
>>> [R] grammar
>>>=20
>>>=20
>>>=20
>>> In other words, if flow I is started at time x, flow i+1
>>>=20
>>> [O] flow I is started at time x, flow i+1
>>> [R] I and i should be consistent; either use upper or lower case. =
Same
>>> for other variables.
>>>=20
>>>=20
>>>=20
>>> Section 11. NAT44 and NAT66
>>>=20
>>>  Although these technologies are not the primarily scope of this
>>>=20
>>> [O] the primarily scope
>>> [P] the primary scope
>>> [R] word choice
>>>=20
>>>=20
>>> Section 12. Summarizing function and variation
>>>=20
>>> For a fine grain analysis of the frequency distribution of the data,
>>>=20
>>> [O] fine grain analysis
>>> [P] fine grained analysis
>>> [R] word choice
>>>=20
>>>=20
>>> --
>>> I don't think the execution is relevant when it was obviously a bad
>>> idea in the first place.
>>> This is like putting rabid weasels in your pants, and later =
expressing
>>> regret at having chosen those particular rabid weasels and that pair
>>> of pants.
>>>  ---maf
>>=20
>> _______________________________________________
>> bmwg mailing list
>> bmwg@ietf.org
>> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
>> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
>> =
o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DAUvQF5fSs3nruuM6VEndHNo4I=
EW
>> FISyixgNCDlimEFk&s=3DwgtEceg7IsJIzEfu1N0VBNJcuYUx4X7fqNHAJmsV2AE&e=3D


--Apple-Mail=_1435E284-C185-44DE-A770-626C1B94F2ED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">Hello Al,<div class=3D""><br class=3D""></div><div =
class=3D"">Thank you for your kind help with this draft.</div><div =
class=3D"">The revised version covers the received comments.</div><div =
class=3D""><a =
href=3D"https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchma=
rking-06" =
class=3D"">https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benc=
hmarking-06</a></div><div class=3D"">If there are any other issues, =
please let us know.</div><div class=3D""><br class=3D""></div><div =
class=3D"">Best regards,</div><div class=3D"">Marius</div><div =
class=3D""><br class=3D""></div><div class=3D""><div><blockquote =
type=3D"cite" class=3D""><div class=3D"">On Apr 15, 2017, at 3:56 PM, =
MORTON, ALFRED C (AL) &lt;<a href=3D"mailto:acmorton@att.com" =
class=3D"">acmorton@att.com</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><div class=3D"">Hi =
Marius,<br class=3D""><br class=3D"">Since all ADs read *many* =
documents, it is universally<br class=3D"">preferred to respond quickly =
as you've done, and follow-up<br class=3D"">with a revised version in =
this case (and when requested<br class=3D"">in the steps that follow). =
<br class=3D""><br class=3D"">When the revised version is available, the =
next step will <br class=3D"">likely be an IETF-wide Last Call. This is =
when the various <br class=3D"">Directorates that help the Area =
Directors will review <br class=3D"">the draft, and there will be some =
more comments as a result.<br class=3D""><br class=3D"">After Last Call, =
the draft will be scheduled for an IESG<br class=3D"">telechat, and the =
other Area Directors will likely provide <br class=3D"">even more =
comments. Prompt replies and resolving text will<br class=3D"">again =
keep your draft moving.<br class=3D""><br class=3D"">If you have any =
questions as we proceed, don't hesitate to<br class=3D"">get in =
touch.<br class=3D""><br class=3D"">regards,<br class=3D"">Al<br =
class=3D"">bmwg co-chair and doc shepherd<br class=3D""><br class=3D""><br=
 class=3D""><br class=3D""><blockquote type=3D"cite" =
class=3D"">-----Original Message-----<br class=3D"">From: bmwg [<a =
href=3D"mailto:bmwg-bounces@ietf.org" =
class=3D"">mailto:bmwg-bounces@ietf.org</a>] On Behalf Of Marius =
Georgescu<br class=3D"">Sent: Saturday, April 15, 2017 5:35 AM<br =
class=3D"">To: Warren Kumari<br class=3D"">Cc: <a =
href=3D"mailto:bmwg-chairs@ietf.org" class=3D"">bmwg-chairs@ietf.org</a>; =
draft-ietf-bmwg-ipv6-tran-tech-<br class=3D""><a =
href=3D"mailto:benchmarking@ietf.org" =
class=3D"">benchmarking@ietf.org</a>; <a href=3D"mailto:bmwg@ietf.org" =
class=3D"">bmwg@ietf.org</a><br class=3D"">Subject: Re: [bmwg] AD =
Evaluation for draft-ietf-bmwg-ipv6-tran-tech-<br =
class=3D"">benchmarking<br class=3D""><br class=3D"">Hello Warren,<br =
class=3D""><br class=3D"">Thank you very much for your kind words and =
for the detailed review.<br class=3D"">We will revise the draft =
considering the Shepherd writeup and your<br class=3D"">comments.<br =
class=3D"">Is there a deadline we should be aware of?<br class=3D""><br =
class=3D"">Best regards,<br class=3D"">Marius<br class=3D""><br =
class=3D""><blockquote type=3D"cite" class=3D"">On Apr 14, 2017, at 7:46 =
PM, Warren Kumari &lt;<a href=3D"mailto:warren@kumari.net" =
class=3D"">warren@kumari.net</a>&gt; wrote:<br class=3D""><br =
class=3D"">Hi there,<br class=3D""><br class=3D"">First off, thanks for =
a really well written document -- I've done some<br =
class=3D"">benchmarking of routers and switches and similar, but never =
and<br class=3D"">translation devices. This document explains how I =
would benchmark this<br class=3D"">nicely.<br class=3D""><br class=3D"">I =
had a few minor nits which I think would be nice if you could<br =
class=3D"">address before I start IETF LC - this should help the =
document sail<br class=3D"">though the process.<br class=3D"">While =
addressing these, there are also 2 editorial notes in the<br =
class=3D"">Shepherd writeup, both (IMO) easy to address.<br class=3D""><br=
 class=3D""><br class=3D"">W<br class=3D""><br class=3D"">Section 1:<br =
class=3D"">"ability to gracefully accommodate greater numbers of =
flows<br class=3D""> &nbsp;than the maximum number of flows which the =
DUT can operate<br class=3D""></blockquote>normally."<br =
class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">[O] =
DUT<br class=3D"">[P] [spell out DUT]<br class=3D"">[R] first use of the =
acronym<br class=3D""><br class=3D""><br class=3D"">Section 1.1. IPv6 =
Transition Technologies<br class=3D"">"4. Encapsulation: The production =
network is assumed to have all three<br class=3D"">domains, Domains A =
and B are IPvX specific, while the core ..."<br class=3D""><br =
class=3D"">[O] three domains, Domains A and B<br class=3D"">[P] three =
domains; Domains A and B<br class=3D"">[R] grammar<br class=3D""><br =
class=3D""><br class=3D"">Section 5.1. Frame Formats and Sizes<br =
class=3D""><br class=3D"">"The two documents can be referred for the =
dual-stack transition<br class=3D""></blockquote>technologies."<br =
class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">[O] =
referred<br class=3D"">[P] referenced<br class=3D"">[R] not sure what is =
meant; referenced is just a guess here. Or<br class=3D"">"referred to"? =
Nither sounds great.<br class=3D""><br class=3D""><br class=3D""><br =
class=3D"">"The calculation method for the Ethernet, as well as a =
calculation<br class=3D"">example are detailed in Appendix A. "<br =
class=3D""><br class=3D"">[O] calculation example are<br class=3D"">[P] =
calculation example, are<br class=3D"">[R] grammar<br class=3D""><br =
class=3D""><br class=3D"">Section 6. Modifiers<br class=3D""><br =
class=3D""> &nbsp;The idea of testing under different operational =
conditions was first<br class=3D""> &nbsp;introduced in =
[RFC2544](Section 11) and represents an important<br class=3D""> =
&nbsp;aspect of benchmarking network elements, as it emulates to some<br =
class=3D""> &nbsp;extent the conditions of a production environment. =
Section 6 of<br class=3D""><br class=3D"">[O] &nbsp;as it emulates to =
some extent the<br class=3D"">[P] as it emulates, to some extent, the<br =
class=3D"">[R] grammar<br class=3D""><br class=3D""><br class=3D"">Section=
 8. Additional Benchmarking Tests for Stateful IPv6 Transition<br =
class=3D"">Technologies<br class=3D""><br class=3D""> &nbsp;This section =
describes additional tests dedicated to the stateful<br class=3D""> =
&nbsp;IPv6 transition technologies. For the tests described in this<br =
class=3D""> &nbsp;section the DUT devices SHOULD follow the test setup =
and test<br class=3D""><br class=3D"">[O] section the DUT device<br =
class=3D"">[P] section, the DUT device<br class=3D"">[R] grammar<br =
class=3D""><br class=3D""><br class=3D"">- If the DNS64 server =
implements caching and there is a cache hit<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;then step 1 is followed by step 6 =
(and steps 2 through 5 are<br class=3D""> =
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;omitted).<br class=3D"">- If the =
domain name has an AAAA record then it is returned in<br class=3D""><br =
class=3D"">[O] If the domain name has an AAAA record then it is =
returned<br class=3D"">[P] If the domain name has an AAAA record, then =
it is returned<br class=3D""><br class=3D""><br class=3D""><br =
class=3D"">(When all the domain names are cached then the results do not =
depend<br class=3D"">on what percentage of the ...<br class=3D""><br =
class=3D"">[O] are cached then the<br class=3D"">[P] are cached, the<br =
class=3D"">[R] readability<br class=3D""><br class=3D""><br class=3D""><br=
 class=3D"">(When all the domain names<br class=3D""> &nbsp;are cached =
then the results do not depend on what percentage of the<br class=3D""> =
&nbsp;domain names have AAAA records, thus these combinations are not<br =
class=3D""> &nbsp;worth testing one by one.)<br class=3D"">[O] are =
cached then the<br class=3D"">[P] are cached, the<br class=3D"">[R] =
readability<br class=3D""><br class=3D""><br class=3D"">queries at the =
required frequency using up not more than the half of<br class=3D"">the =
timeout time.<br class=3D""><br class=3D""> &nbsp;Remark: a sample =
open-source test program, dns64perf++ is available<br class=3D""><br =
class=3D"">[O] dns64perf++ is<br class=3D"">[P] dns64perf++, is<br =
class=3D"">[R] grammar<br class=3D""><br class=3D""><br class=3D""><br =
class=3D""> For encapsulation transition technologies a m:n setup can =
be<br class=3D""><br class=3D"">[O] &nbsp;For encapsulation transition =
technologies<br class=3D"">[P] &nbsp;For encapsulation transition =
technologies,<br class=3D"">[R] grammar<br class=3D""><br class=3D""> =
&nbsp;created, where m is the number of flows applied to the same =
client<br class=3D""> &nbsp;device and n the number of client devices =
connected to the same<br class=3D""> &nbsp;server device.<br class=3D""> =
&nbsp;For the translation based transition technologies the client =
devices<br class=3D""><br class=3D"">[O] For the translation based =
transition technologies the client<br class=3D"">[P] For the translation =
based transition technologies, the client<br class=3D"">[R] grammar<br =
class=3D""><br class=3D""><br class=3D""><br class=3D"">In other words, =
if flow I is started at time x, flow i+1<br class=3D""><br class=3D"">[O] =
flow I is started at time x, flow i+1<br class=3D"">[R] I and i should =
be consistent; either use upper or lower case. Same<br class=3D"">for =
other variables.<br class=3D""><br class=3D""><br class=3D""><br =
class=3D"">Section 11. NAT44 and NAT66<br class=3D""><br class=3D""> =
&nbsp;Although these technologies are not the primarily scope of this<br =
class=3D""><br class=3D"">[O] the primarily scope<br class=3D"">[P] the =
primary scope<br class=3D"">[R] word choice<br class=3D""><br =
class=3D""><br class=3D"">Section 12. Summarizing function and =
variation<br class=3D""><br class=3D"">For a fine grain analysis of the =
frequency distribution of the data,<br class=3D""><br class=3D"">[O] =
fine grain analysis<br class=3D"">[P] fine grained analysis<br =
class=3D"">[R] word choice<br class=3D""><br class=3D""><br =
class=3D"">--<br class=3D"">I don't think the execution is relevant when =
it was obviously a bad<br class=3D"">idea in the first place.<br =
class=3D"">This is like putting rabid weasels in your pants, and later =
expressing<br class=3D"">regret at having chosen those particular rabid =
weasels and that pair<br class=3D"">of pants.<br class=3D""> =
&nbsp;---maf<br class=3D""></blockquote><br =
class=3D"">_______________________________________________<br =
class=3D"">bmwg mailing list<br class=3D""><a =
href=3D"mailto:bmwg@ietf.org" class=3D"">bmwg@ietf.org</a><br =
class=3D"">https://urldefense.proofpoint.com/v2/url?u=3Dhttps-<br =
class=3D"">3A__www.ietf.org_mailman_listinfo_bmwg&amp;d=3DDwICAg&amp;c=3DL=
FYZ-<br =
class=3D"">o9_HUMeMTSQicvjIg&amp;r=3DOfsSu8kTIltVyD1oL72cBw&amp;m=3DAUvQF5=
fSs3nruuM6VEndHNo4IEW<br =
class=3D"">FISyixgNCDlimEFk&amp;s=3DwgtEceg7IsJIzEfu1N0VBNJcuYUx4X7fqNHAJm=
sV2AE&amp;e=3D<br =
class=3D""></blockquote></div></div></blockquote></div><br =
class=3D""></div></body></html>=

--Apple-Mail=_1435E284-C185-44DE-A770-626C1B94F2ED--


From nobody Tue Apr 18 04:51:15 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E046E129B63; Tue, 18 Apr 2017 04:51:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.411
X-Spam-Level: 
X-Spam-Status: No, score=-3.411 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t5tvUNPAzR2E; Tue, 18 Apr 2017 04:51:10 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C0438129B84; Tue, 18 Apr 2017 04:51:10 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net [127.0.0.1]) by m0048589.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v3IBjY2p026390; Tue, 18 Apr 2017 07:51:08 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0048589.ppops.net-00191d01. with ESMTP id 29w84usrvm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 18 Apr 2017 07:51:07 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3IBp6uJ018722; Tue, 18 Apr 2017 07:51:06 -0400
Received: from mlpi409.sfdc.sbc.com (mlpi409.sfdc.sbc.com [130.9.128.241]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3IBotTe018556 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 18 Apr 2017 07:50:57 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi409.sfdc.sbc.com (RSA Interceptor); Tue, 18 Apr 2017 11:50:47 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3IBokAg018419; Tue, 18 Apr 2017 06:50:46 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3IBoeYD018133; Tue, 18 Apr 2017 06:50:41 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id AE040E0425; Tue, 18 Apr 2017 07:50:39 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 18 Apr 2017 07:50:39 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Marius Georgescu <marius.georgescu@rcs-rds.ro>, Warren Kumari <warren@kumari.net>
CC: "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org" <draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
Thread-Index: AQHStT7P6GGywmc5/UW/liz/M3vuoqHGb7iA///xyCCABKnkAP//+jgQ
Date: Tue, 18 Apr 2017 11:50:38 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F7151C@njmtexg5.research.att.com>
References: <CAHw9_iJ+PcuJVS3ougbFUQkT0OwmEXbRVqy=44bORwUHar-FzQ@mail.gmail.com> <ADC8E1B6-CEC2-42CE-9BB8-F4A8EE8630EF@rcs-rds.ro> <4D7F4AD313D3FC43A053B309F97543CF25F70C23@njmtexg5.research.att.com> <C7D080C6-E54B-49F9-BB09-B7F6126D6237@rcs-rds.ro>
In-Reply-To: <C7D080C6-E54B-49F9-BB09-B7F6126D6237@rcs-rds.ro>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.254.167]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF25F7151Cnjmtexg5researc_"
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-18_10:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704180096
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/pWY9omeo2S2FVvN9nwG-YryNXSs>
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchmarking
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Apr 2017 11:51:14 -0000

--_000_4D7F4AD313D3FC43A053B309F97543CF25F7151Cnjmtexg5researc_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Marius,

thanks for your editing efforts. I think you
have handled all of Warren's editorial comments and the
two small issues identified in the shepherd's form (which I will
remove and indicate the new version).

@Warren - we're on our way again,
thanks for your review.

Al
doc shepherd

From: Marius Georgescu [mailto:marius.georgescu@rcs-rds.ro]
Sent: Tuesday, April 18, 2017 3:57 AM
To: MORTON, ALFRED C (AL)
Cc: Warren Kumari; bmwg-chairs@ietf.org; draft-ietf-bmwg-ipv6-tran-tech-ben=
chmarking@ietf.org; bmwg@ietf.org
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-benchm=
arking

Hello Al,

Thank you for your kind help with this draft.
The revised version covers the received comments.
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06<=
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__tools.ietf.org_html_=
draft-2Dietf-2Dbmwg-2Dipv6-2Dtran-2Dtech-2Dbenchmarking-2D06&d=3DDwMFAg&c=
=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D2HUOwbd3l5OJGvE94o=
eM8_Eh2GN0L9juGEi8A0uBzaw&s=3Du3qaLjeINQmyjGL26zFUOWhnBDBY8kcE9Mh8oRXALjQ&e=
=3D>
If there are any other issues, please let us know.

Best regards,
Marius

On Apr 15, 2017, at 3:56 PM, MORTON, ALFRED C (AL) <acmorton@att.com<mailto=
:acmorton@att.com>> wrote:

Hi Marius,

Since all ADs read *many* documents, it is universally
preferred to respond quickly as you've done, and follow-up
with a revised version in this case (and when requested
in the steps that follow).

When the revised version is available, the next step will
likely be an IETF-wide Last Call. This is when the various
Directorates that help the Area Directors will review
the draft, and there will be some more comments as a result.

After Last Call, the draft will be scheduled for an IESG
telechat, and the other Area Directors will likely provide
even more comments. Prompt replies and resolving text will
again keep your draft moving.

If you have any questions as we proceed, don't hesitate to
get in touch.

regards,
Al
bmwg co-chair and doc shepherd




-----Original Message-----
From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Marius Georgescu
Sent: Saturday, April 15, 2017 5:35 AM
To: Warren Kumari
Cc: bmwg-chairs@ietf.org<mailto:bmwg-chairs@ietf.org>; draft-ietf-bmwg-ipv6=
-tran-tech-
benchmarking@ietf.org<mailto:benchmarking@ietf.org>; bmwg@ietf.org<mailto:b=
mwg@ietf.org>
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-
benchmarking

Hello Warren,

Thank you very much for your kind words and for the detailed review.
We will revise the draft considering the Shepherd writeup and your
comments.
Is there a deadline we should be aware of?

Best regards,
Marius


On Apr 14, 2017, at 7:46 PM, Warren Kumari <warren@kumari.net<mailto:warren=
@kumari.net>> wrote:

Hi there,

First off, thanks for a really well written document -- I've done some
benchmarking of routers and switches and similar, but never and
translation devices. This document explains how I would benchmark this
nicely.

I had a few minor nits which I think would be nice if you could
address before I start IETF LC - this should help the document sail
though the process.
While addressing these, there are also 2 editorial notes in the
Shepherd writeup, both (IMO) easy to address.


W

Section 1:
"ability to gracefully accommodate greater numbers of flows
 than the maximum number of flows which the DUT can operate
normally."


[O] DUT
[P] [spell out DUT]
[R] first use of the acronym


Section 1.1. IPv6 Transition Technologies
"4. Encapsulation: The production network is assumed to have all three
domains, Domains A and B are IPvX specific, while the core ..."

[O] three domains, Domains A and B
[P] three domains; Domains A and B
[R] grammar


Section 5.1. Frame Formats and Sizes

"The two documents can be referred for the dual-stack transition
technologies."


[O] referred
[P] referenced
[R] not sure what is meant; referenced is just a guess here. Or
"referred to"? Nither sounds great.



"The calculation method for the Ethernet, as well as a calculation
example are detailed in Appendix A. "

[O] calculation example are
[P] calculation example, are
[R] grammar


Section 6. Modifiers

 The idea of testing under different operational conditions was first
 introduced in [RFC2544](Section 11) and represents an important
 aspect of benchmarking network elements, as it emulates to some
 extent the conditions of a production environment. Section 6 of

[O]  as it emulates to some extent the
[P] as it emulates, to some extent, the
[R] grammar


Section 8. Additional Benchmarking Tests for Stateful IPv6 Transition
Technologies

 This section describes additional tests dedicated to the stateful
 IPv6 transition technologies. For the tests described in this
 section the DUT devices SHOULD follow the test setup and test

[O] section the DUT device
[P] section, the DUT device
[R] grammar


- If the DNS64 server implements caching and there is a cache hit
      then step 1 is followed by step 6 (and steps 2 through 5 are
      omitted).
- If the domain name has an AAAA record then it is returned in

[O] If the domain name has an AAAA record then it is returned
[P] If the domain name has an AAAA record, then it is returned



(When all the domain names are cached then the results do not depend
on what percentage of the ...

[O] are cached then the
[P] are cached, the
[R] readability



(When all the domain names
 are cached then the results do not depend on what percentage of the
 domain names have AAAA records, thus these combinations are not
 worth testing one by one.)
[O] are cached then the
[P] are cached, the
[R] readability


queries at the required frequency using up not more than the half of
the timeout time.

 Remark: a sample open-source test program, dns64perf++ is available

[O] dns64perf++ is
[P] dns64perf++, is
[R] grammar



For encapsulation transition technologies a m:n setup can be

[O]  For encapsulation transition technologies
[P]  For encapsulation transition technologies,
[R] grammar

 created, where m is the number of flows applied to the same client
 device and n the number of client devices connected to the same
 server device.
 For the translation based transition technologies the client devices

[O] For the translation based transition technologies the client
[P] For the translation based transition technologies, the client
[R] grammar



In other words, if flow I is started at time x, flow i+1

[O] flow I is started at time x, flow i+1
[R] I and i should be consistent; either use upper or lower case. Same
for other variables.



Section 11. NAT44 and NAT66

 Although these technologies are not the primarily scope of this

[O] the primarily scope
[P] the primary scope
[R] word choice


Section 12. Summarizing function and variation

For a fine grain analysis of the frequency distribution of the data,

[O] fine grain analysis
[P] fine grained analysis
[R] word choice


--
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
 ---maf

_______________________________________________
bmwg mailing list
bmwg@ietf.org<mailto:bmwg@ietf.org>
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DAUvQF5fSs3nruuM6VEndHNo4IE=
W
FISyixgNCDlimEFk&s=3DwgtEceg7IsJIzEfu1N0VBNJcuYUx4X7fqNHAJmsV2AE&e=3D


--_000_4D7F4AD313D3FC43A053B309F97543CF25F7151Cnjmtexg5researc_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
	{mso-style-priority:99;
	mso-style-link:"Balloon Text Char";
	margin:0in;
	margin-bottom:.0001pt;
	font-size:8.0pt;
	font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
	{mso-style-name:"Balloon Text Char";
	mso-style-priority:99;
	mso-style-link:"Balloon Text";
	font-family:"Tahoma","sans-serif";}
span.EmailStyle19
	{mso-style-type:personal-reply;
	font-family:"Courier New";
	color:black;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Hi Marius,
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">thanks for your editing efforts. I think you<o=
:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">have handled all of Warren&#8217;s editorial c=
omments and the<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">two small issues identified in the shepherd&#8=
217;s form (which I will
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">remove and indicate the new version).<o:p></o:=
p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">@Warren &#8211; we&#8217;re on our way again,<=
o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">thanks for your review.<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">Al<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black">doc shepherd<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Co=
urier New&quot;;color:black"><o:p>&nbsp;</o:p></span></p>
<div style=3D"border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in =
4.0pt">
<div>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal"><b><span style=3D"font-size:10.0pt;font-family:&quot=
;Tahoma&quot;,&quot;sans-serif&quot;">From:</span></b><span style=3D"font-s=
ize:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> Marius G=
eorgescu [mailto:marius.georgescu@rcs-rds.ro]
<br>
<b>Sent:</b> Tuesday, April 18, 2017 3:57 AM<br>
<b>To:</b> MORTON, ALFRED C (AL)<br>
<b>Cc:</b> Warren Kumari; bmwg-chairs@ietf.org; draft-ietf-bmwg-ipv6-tran-t=
ech-benchmarking@ietf.org; bmwg@ietf.org<br>
<b>Subject:</b> Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech=
-benchmarking<o:p></o:p></span></p>
</div>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Hello Al,<o:p></o:p></p>
<div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class=3D"MsoNormal">Thank you for your kind help with this draft.<o:p></=
o:p></p>
</div>
<div>
<p class=3D"MsoNormal">The revised version covers the received comments.<o:=
p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><a href=3D"https://urldefense.proofpoint.com/v2/url?=
u=3Dhttps-3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Dipv6-2Dtran-2Dtech-=
2Dbenchmarking-2D06&amp;d=3DDwMFAg&amp;c=3DLFYZ-o9_HUMeMTSQicvjIg&amp;r=3DO=
fsSu8kTIltVyD1oL72cBw&amp;m=3D2HUOwbd3l5OJGvE94oeM8_Eh2GN0L9juGEi8A0uBzaw&a=
mp;s=3Du3qaLjeINQmyjGL26zFUOWhnBDBY8kcE9Mh8oRXALjQ&amp;e=3D">https://tools.=
ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06</a><o:p></o:p>=
</p>
</div>
<div>
<p class=3D"MsoNormal">If there are any other issues, please let us know.<o=
:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class=3D"MsoNormal">Best regards,<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">Marius<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<div>
<blockquote style=3D"margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class=3D"MsoNormal">On Apr 15, 2017, at 3:56 PM, MORTON, ALFRED C (AL) &=
lt;<a href=3D"mailto:acmorton@att.com">acmorton@att.com</a>&gt; wrote:<o:p>=
</o:p></p>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<div>
<p class=3D"MsoNormal">Hi Marius,<br>
<br>
Since all ADs read *many* documents, it is universally<br>
preferred to respond quickly as you've done, and follow-up<br>
with a revised version in this case (and when requested<br>
in the steps that follow). <br>
<br>
When the revised version is available, the next step will <br>
likely be an IETF-wide Last Call. This is when the various <br>
Directorates that help the Area Directors will review <br>
the draft, and there will be some more comments as a result.<br>
<br>
After Last Call, the draft will be scheduled for an IESG<br>
telechat, and the other Area Directors will likely provide <br>
even more comments. Prompt replies and resolving text will<br>
again keep your draft moving.<br>
<br>
If you have any questions as we proceed, don't hesitate to<br>
get in touch.<br>
<br>
regards,<br>
Al<br>
bmwg co-chair and doc shepherd<br>
<br>
<br>
<br>
<br>
<o:p></o:p></p>
<p class=3D"MsoNormal">-----Original Message-----<br>
From: bmwg [<a href=3D"mailto:bmwg-bounces@ietf.org">mailto:bmwg-bounces@ie=
tf.org</a>] On Behalf Of Marius Georgescu<br>
Sent: Saturday, April 15, 2017 5:35 AM<br>
To: Warren Kumari<br>
Cc: <a href=3D"mailto:bmwg-chairs@ietf.org">bmwg-chairs@ietf.org</a>; draft=
-ietf-bmwg-ipv6-tran-tech-<br>
<a href=3D"mailto:benchmarking@ietf.org">benchmarking@ietf.org</a>; <a href=
=3D"mailto:bmwg@ietf.org">
bmwg@ietf.org</a><br>
Subject: Re: [bmwg] AD Evaluation for draft-ietf-bmwg-ipv6-tran-tech-<br>
benchmarking<br>
<br>
Hello Warren,<br>
<br>
Thank you very much for your kind words and for the detailed review.<br>
We will revise the draft considering the Shepherd writeup and your<br>
comments.<br>
Is there a deadline we should be aware of?<br>
<br>
Best regards,<br>
Marius<br>
<br>
<br>
<o:p></o:p></p>
<p class=3D"MsoNormal">On Apr 14, 2017, at 7:46 PM, Warren Kumari &lt;<a hr=
ef=3D"mailto:warren@kumari.net">warren@kumari.net</a>&gt; wrote:<br>
<br>
Hi there,<br>
<br>
First off, thanks for a really well written document -- I've done some<br>
benchmarking of routers and switches and similar, but never and<br>
translation devices. This document explains how I would benchmark this<br>
nicely.<br>
<br>
I had a few minor nits which I think would be nice if you could<br>
address before I start IETF LC - this should help the document sail<br>
though the process.<br>
While addressing these, there are also 2 editorial notes in the<br>
Shepherd writeup, both (IMO) easy to address.<br>
<br>
<br>
W<br>
<br>
Section 1:<br>
&quot;ability to gracefully accommodate greater numbers of flows<br>
&nbsp;than the maximum number of flows which the DUT can operate<o:p></o:p>=
</p>
<p class=3D"MsoNormal">normally.&quot;<br>
<br>
<o:p></o:p></p>
<p class=3D"MsoNormal"><br>
[O] DUT<br>
[P] [spell out DUT]<br>
[R] first use of the acronym<br>
<br>
<br>
Section 1.1. IPv6 Transition Technologies<br>
&quot;4. Encapsulation: The production network is assumed to have all three=
<br>
domains, Domains A and B are IPvX specific, while the core ...&quot;<br>
<br>
[O] three domains, Domains A and B<br>
[P] three domains; Domains A and B<br>
[R] grammar<br>
<br>
<br>
Section 5.1. Frame Formats and Sizes<br>
<br>
&quot;The two documents can be referred for the dual-stack transition<o:p><=
/o:p></p>
<p class=3D"MsoNormal">technologies.&quot;<br>
<br>
<o:p></o:p></p>
<p class=3D"MsoNormal"><br>
[O] referred<br>
[P] referenced<br>
[R] not sure what is meant; referenced is just a guess here. Or<br>
&quot;referred to&quot;? Nither sounds great.<br>
<br>
<br>
<br>
&quot;The calculation method for the Ethernet, as well as a calculation<br>
example are detailed in Appendix A. &quot;<br>
<br>
[O] calculation example are<br>
[P] calculation example, are<br>
[R] grammar<br>
<br>
<br>
Section 6. Modifiers<br>
<br>
&nbsp;The idea of testing under different operational conditions was first<=
br>
&nbsp;introduced in [RFC2544](Section 11) and represents an important<br>
&nbsp;aspect of benchmarking network elements, as it emulates to some<br>
&nbsp;extent the conditions of a production environment. Section 6 of<br>
<br>
[O] &nbsp;as it emulates to some extent the<br>
[P] as it emulates, to some extent, the<br>
[R] grammar<br>
<br>
<br>
Section 8. Additional Benchmarking Tests for Stateful IPv6 Transition<br>
Technologies<br>
<br>
&nbsp;This section describes additional tests dedicated to the stateful<br>
&nbsp;IPv6 transition technologies. For the tests described in this<br>
&nbsp;section the DUT devices SHOULD follow the test setup and test<br>
<br>
[O] section the DUT device<br>
[P] section, the DUT device<br>
[R] grammar<br>
<br>
<br>
- If the DNS64 server implements caching and there is a cache hit<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;then step 1 is followed by step 6 (and =
steps 2 through 5 are<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;omitted).<br>
- If the domain name has an AAAA record then it is returned in<br>
<br>
[O] If the domain name has an AAAA record then it is returned<br>
[P] If the domain name has an AAAA record, then it is returned<br>
<br>
<br>
<br>
(When all the domain names are cached then the results do not depend<br>
on what percentage of the ...<br>
<br>
[O] are cached then the<br>
[P] are cached, the<br>
[R] readability<br>
<br>
<br>
<br>
(When all the domain names<br>
&nbsp;are cached then the results do not depend on what percentage of the<b=
r>
&nbsp;domain names have AAAA records, thus these combinations are not<br>
&nbsp;worth testing one by one.)<br>
[O] are cached then the<br>
[P] are cached, the<br>
[R] readability<br>
<br>
<br>
queries at the required frequency using up not more than the half of<br>
the timeout time.<br>
<br>
&nbsp;Remark: a sample open-source test program, dns64perf&#43;&#43; is ava=
ilable<br>
<br>
[O] dns64perf&#43;&#43; is<br>
[P] dns64perf&#43;&#43;, is<br>
[R] grammar<br>
<br>
<br>
<br>
For encapsulation transition technologies a m:n setup can be<br>
<br>
[O] &nbsp;For encapsulation transition technologies<br>
[P] &nbsp;For encapsulation transition technologies,<br>
[R] grammar<br>
<br>
&nbsp;created, where m is the number of flows applied to the same client<br=
>
&nbsp;device and n the number of client devices connected to the same<br>
&nbsp;server device.<br>
&nbsp;For the translation based transition technologies the client devices<=
br>
<br>
[O] For the translation based transition technologies the client<br>
[P] For the translation based transition technologies, the client<br>
[R] grammar<br>
<br>
<br>
<br>
In other words, if flow I is started at time x, flow i&#43;1<br>
<br>
[O] flow I is started at time x, flow i&#43;1<br>
[R] I and i should be consistent; either use upper or lower case. Same<br>
for other variables.<br>
<br>
<br>
<br>
Section 11. NAT44 and NAT66<br>
<br>
&nbsp;Although these technologies are not the primarily scope of this<br>
<br>
[O] the primarily scope<br>
[P] the primary scope<br>
[R] word choice<br>
<br>
<br>
Section 12. Summarizing function and variation<br>
<br>
For a fine grain analysis of the frequency distribution of the data,<br>
<br>
[O] fine grain analysis<br>
[P] fine grained analysis<br>
[R] word choice<br>
<br>
<br>
--<br>
I don't think the execution is relevant when it was obviously a bad<br>
idea in the first place.<br>
This is like putting rabid weasels in your pants, and later expressing<br>
regret at having chosen those particular rabid weasels and that pair<br>
of pants.<br>
&nbsp;---maf<o:p></o:p></p>
<p class=3D"MsoNormal"><br>
_______________________________________________<br>
bmwg mailing list<br>
<a href=3D"mailto:bmwg@ietf.org">bmwg@ietf.org</a><br>
<a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-">https://url=
defense.proofpoint.com/v2/url?u=3Dhttps-</a><br>
3A__www.ietf.org_mailman_listinfo_bmwg&amp;d=3DDwICAg&amp;c=3DLFYZ-<br>
o9_HUMeMTSQicvjIg&amp;r=3DOfsSu8kTIltVyD1oL72cBw&amp;m=3DAUvQF5fSs3nruuM6VE=
ndHNo4IEW<br>
FISyixgNCDlimEFk&amp;s=3DwgtEceg7IsJIzEfu1N0VBNJcuYUx4X7fqNHAJmsV2AE&amp;e=
=3D<o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</div>
</div>
</body>
</html>

--_000_4D7F4AD313D3FC43A053B309F97543CF25F7151Cnjmtexg5researc_--


From nobody Tue Apr 18 07:11:45 2017
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 84D4E127BA3; Tue, 18 Apr 2017 07:11:43 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: "IETF-Announce" <ietf-announce@ietf.org>
X-Test-IDTracker: no
X-IETF-IDTracker: 6.49.1
Auto-Submitted: auto-generated
Precedence: bulk
CC: bmwg-chairs@ietf.org, acmorton@att.com, Al Morton <acmorton@att.com>, bmwg@ietf.org, warren@kumari.net, draft-ietf-bmwg-ipv6-tran-tech-benchmarking@ietf.org
Reply-To: ietf@ietf.org
Sender: <iesg-secretary@ietf.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <149252470353.16122.4815057235609635083.idtracker@ietfa.amsl.com>
Date: Tue, 18 Apr 2017 07:11:43 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/XDtHcpv9Vaq9E_L4gKtT0vkzSUI>
Subject: [bmwg] Last Call: <draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06.txt> (Benchmarking Methodology for IPv6 Transition Technologies) to Informational RFC
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Apr 2017 14:11:44 -0000

The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Benchmarking Methodology for IPv6 Transition Technologies'
  <draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06.txt> as Informational
RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-05-02. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a metric for benchmarking load
   scalability.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/ballot/


No IPR declarations have been submitted directly on this I-D.





From nobody Tue Apr 18 17:49:44 2017
Return-Path: <sbanks@encrypted.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B2D1212945B; Tue, 18 Apr 2017 17:49:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level: 
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lxXQEOndEKzj; Tue, 18 Apr 2017 17:49:40 -0700 (PDT)
Received: from aws.hosed.org (aws.hosed.org [50.16.104.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 57AE8129454; Tue, 18 Apr 2017 17:49:40 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by aws.hosed.org (Postfix) with ESMTP id 7EB6280376; Tue, 18 Apr 2017 20:49:39 -0400 (EDT)
X-Virus-Scanned: Debian amavisd-new at aws.hosed.org
Received: from aws.hosed.org ([127.0.0.1]) by localhost (aws.hosed.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LXvOUzMhlzpn; Tue, 18 Apr 2017 20:49:39 -0400 (EDT)
Received: from [10.0.0.102] (c-67-164-26-160.hsd1.ca.comcast.net [67.164.26.160]) by aws.hosed.org (Postfix) with ESMTPSA id 025D580374; Tue, 18 Apr 2017 20:49:38 -0400 (EDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Sarah B <sbanks@encrypted.net>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF25F710A7@njmtexg5.research.att.com>
Date: Tue, 18 Apr 2017 17:49:37 -0700
Cc: "bmwg@ietf.org" <bmwg@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <7AC78C9E-BA60-401E-BC50-0FC91C4D5389@encrypted.net>
References: <149244388572.17763.6511045661474480726@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25F710A7@njmtexg5.research.att.com>
To: ALFRED MORTON <acmorton@att.com>
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0QCmOdccSSBwp1oyrxIMy42U3gk>
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-vswitch-opnfv-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Apr 2017 00:49:43 -0000

Thanks for that, Al. I have an action item from IETF98,where I need to =
call consensus (and I'm doing so) that we are ready to move this draft =
through to publication. Please stand by for the incoming shepherd write =
up.=20

BMWG, thanks for your time, your reviews, and your feedback on this =
work. It's kindly appreciated.

Thanks
Sarah

> On Apr 17, 2017, at 9:02 AM, MORTON, ALFRED C (AL) <acmorton@att.com> =
wrote:
>=20
> BMWG,
> This draft expired long after WGLC,=20
> so I've submitted a trivial update=20
> to change the date and fix some URLs
> that broke following OPNFV's Wiki Migration.
> Al
>=20
> -----Original Message-----
> From: I-D-Announce [mailto:i-d-announce-bounces@ietf.org] On Behalf Of =
internet-drafts@ietf.org
> Sent: Monday, April 17, 2017 11:45 AM
> To: i-d-announce@ietf.org
> Cc: bmwg@ietf.org
> Subject: I-D Action: draft-ietf-bmwg-vswitch-opnfv-02.txt
>=20
>=20
> A New Internet-Draft is available from the on-line Internet-Drafts =
directories.
> This draft is a work item of the Benchmarking Methodology of the IETF.
>=20
>        Title           : Benchmarking Virtual Switches in OPNFV
>        Authors         : Maryam Tahhan
>                          Billy O'Mahony
>                          Al Morton
> 	Filename        : draft-ietf-bmwg-vswitch-opnfv-02.txt
> 	Pages           : 21
> 	Date            : 2017-04-17
>=20
> Abstract:
>   This memo describes the progress of the Open Platform for NFV =
(OPNFV)
>   project on virtual switch performance "VSWITCHPERF".  This project
>   intends to build on the current and completed work of the
>   Benchmarking Methodology Working Group in IETF, by referencing
>   existing literature.  The Benchmarking Methodology Working Group has
>   traditionally conducted laboratory characterization of dedicated
>   physical implementations of internetworking functions.  Therefore,
>   this memo begins to describe the additional considerations when
>   virtual switches are implemented in general-purpose hardware.  The
>   expanded tests and benchmarks are also influenced by the OPNFV
>   mission to support virtualization of the "telco" infrastructure.
>=20
>=20
>=20
> The IETF datatracker status page for this draft is:
> =
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.or=
g_doc_draft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv_&d=3DDwICAg&c=3DLFYZ-o9_HUMeMT=
SQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9=
vIX_AlGVc&s=3D6vW7Rn1CLNS6IMn-CzUvahpfx1IRG-vLu6OyC6dxwac&e=3D=20
>=20
> There are also htmlized versions available at:
> =
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__tools.ietf.org_html=
_draft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv-2D02&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTS=
QicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9v=
IX_AlGVc&s=3DysSokYvIEmIcJi3xXgExYO8C0Lq4HZ7pdnYtrktbvNM&e=3D=20
> =
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__datatracker.ietf.or=
g_doc_html_draft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv-2D02&d=3DDwICAg&c=3DLFYZ-=
o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBB=
HSTfeH8p9vIX_AlGVc&s=3DObVw1_C8LZ4Xm4hbr3dFH0y5nSEx22buynICB3jqe2k&e=3D=20=

>=20
> A diff from the previous version is available at:
> =
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.ietf.org_rfcdif=
f-3Furl2-3Ddraft-2Dietf-2Dbmwg-2Dvswitch-2Dopnfv-2D02&d=3DDwICAg&c=3DLFYZ-=
o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBB=
HSTfeH8p9vIX_AlGVc&s=3DUNtcjwlxwySrNLcieT_MhCpq8LhnrLtcc-N3BiGCtHA&e=3D=20=

>=20
>=20
> Please note that it may take a couple of minutes from the time of =
submission
> until the htmlized version and diff are available at tools.ietf.org.
>=20
> Internet-Drafts are also available by anonymous FTP at:
> =
https://urldefense.proofpoint.com/v2/url?u=3Dftp-3A__ftp.ietf.org_internet=
-2Ddrafts_&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cB=
w&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3Dzq0CKZEiTbED5gL-XhNF=
7n9ZOq0aNI9slkN6R5s_MQA&e=3D=20
>=20
> _______________________________________________
> I-D-Announce mailing list
> I-D-Announce@ietf.org
> =
https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.ietf.org_mailma=
n_listinfo_i-2Dd-2Dannounce&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsS=
u8kTIltVyD1oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3DWK_=
aU9ZL3BR_cvzJcffJymLc5Keu9XAOdOladHCo8Cw&e=3D=20
> Internet-Draft directories: =
https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.ietf.org_shadow.=
html&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3D=
DDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3DNBq2Ejodc_989q2agGgcGXlHcy=
Wp3j7ZkiSOPGOWLPU&e=3D=20
> or =
https://urldefense.proofpoint.com/v2/url?u=3Dftp-3A__ftp.ietf.org_ietf_1sh=
adow-2Dsites.txt&d=3DDwICAg&c=3DLFYZ-o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1=
oL72cBw&m=3DDDtoKys_BskNl-NLiOZ1SQEBBHSTfeH8p9vIX_AlGVc&s=3DbxjbjC90oyZ74N=
saJKf-DoL3cwejnWbj-1jHA-OEMfY&e=3D=20
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


From nobody Wed Apr 19 08:55:35 2017
Return-Path: <sbanks@encrypted.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E6B83129B15 for <bmwg@ietfa.amsl.com>; Wed, 19 Apr 2017 08:55:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level: 
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dhjpb78ROE_9 for <bmwg@ietfa.amsl.com>; Wed, 19 Apr 2017 08:55:32 -0700 (PDT)
Received: from aws.hosed.org (aws.hosed.org [50.16.104.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 03A69129B0A for <bmwg@ietf.org>; Wed, 19 Apr 2017 08:55:32 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by aws.hosed.org (Postfix) with ESMTP id 4E27180374 for <bmwg@ietf.org>; Wed, 19 Apr 2017 11:55:31 -0400 (EDT)
X-Virus-Scanned: Debian amavisd-new at aws.hosed.org
Received: from aws.hosed.org ([127.0.0.1]) by localhost (aws.hosed.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mf11jklZLmvu for <bmwg@ietf.org>; Wed, 19 Apr 2017 11:55:31 -0400 (EDT)
Received: from [10.0.0.102] (c-67-164-26-160.hsd1.ca.comcast.net [67.164.26.160]) by aws.hosed.org (Postfix) with ESMTPSA id 106D780373 for <bmwg@ietf.org>; Wed, 19 Apr 2017 11:55:30 -0400 (EDT)
From: Sarah B <sbanks@encrypted.net>
Content-Type: multipart/alternative; boundary="Apple-Mail=_2A3BA52D-010A-4208-9B47-28C24B67B467"
Date: Wed, 19 Apr 2017 08:55:30 -0700
References: <149261730594.19284.12023646415352221144.idtracker@ietfa.amsl.com>
To: bmwg@ietf.org
Message-Id: <9B0C7F25-A0F5-49C2-91FC-C91AAC692051@encrypted.net>
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/e_5qlgpXO0cPJn7htM9AiwuRnZ8>
Subject: [bmwg] Fwd: Publication has been requested for draft-ietf-bmwg-vswitch-opnfv-02
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Apr 2017 15:55:34 -0000

--Apple-Mail=_2A3BA52D-010A-4208-9B47-28C24B67B467
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii



> Begin forwarded message:
>=20
> From: Sarah Banks <sbanks@encrypted.net>
> Subject: Publication has been requested for =
draft-ietf-bmwg-vswitch-opnfv-02
> Date: April 19, 2017 at 8:55:05 AM PDT
> To: <warren@kumari.net>
> Cc: Sarah Banks <sbanks@encrypted.net>, sbanks@encrypted.net, =
bmwg-chairs@ietf.org, iesg-secretary@ietf.org
>=20
> Sarah Banks has requested publication of =
draft-ietf-bmwg-vswitch-opnfv-02 as Informational on behalf of the BMWG =
working group.
>=20
> Please verify the document's state at =
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/


--Apple-Mail=_2A3BA52D-010A-4208-9B47-28C24B67B467
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><br class=3D""><div><br class=3D""><blockquote type=3D"cite" =
class=3D""><div class=3D"">Begin forwarded message:</div><br =
class=3D"Apple-interchange-newline"><div style=3D"margin-top: 0px; =
margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span=
 style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">From: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D"">Sarah Banks &lt;<a =
href=3D"mailto:sbanks@encrypted.net" =
class=3D"">sbanks@encrypted.net</a>&gt;<br class=3D""></span></div><div =
style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
margin-left: 0px;" class=3D""><span style=3D"font-family: =
-webkit-system-font, Helvetica Neue, Helvetica, sans-serif; =
color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Subject: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D""><b class=3D"">Publication has =
been requested for draft-ietf-bmwg-vswitch-opnfv-02</b><br =
class=3D""></span></div><div style=3D"margin-top: 0px; margin-right: =
0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span =
style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Date: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D"">April 19, 2017 at 8:55:05 AM =
PDT<br class=3D""></span></div><div style=3D"margin-top: 0px; =
margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span=
 style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">To: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D"">&lt;<a =
href=3D"mailto:warren@kumari.net" class=3D"">warren@kumari.net</a>&gt;<br =
class=3D""></span></div><div style=3D"margin-top: 0px; margin-right: =
0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span =
style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, =
sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Cc: =
</b></span><span style=3D"font-family: -webkit-system-font, Helvetica =
Neue, Helvetica, sans-serif;" class=3D"">Sarah Banks &lt;<a =
href=3D"mailto:sbanks@encrypted.net" =
class=3D"">sbanks@encrypted.net</a>&gt;, <a =
href=3D"mailto:sbanks@encrypted.net" class=3D"">sbanks@encrypted.net</a>, =
<a href=3D"mailto:bmwg-chairs@ietf.org" =
class=3D"">bmwg-chairs@ietf.org</a>, <a =
href=3D"mailto:iesg-secretary@ietf.org" =
class=3D"">iesg-secretary@ietf.org</a><br class=3D""></span></div><br =
class=3D""><div class=3D""><div class=3D"">Sarah Banks has requested =
publication of draft-ietf-bmwg-vswitch-opnfv-02 as Informational on =
behalf of the BMWG working group.<br class=3D""><br class=3D"">Please =
verify the document's state at <a =
href=3D"https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/" =
class=3D"">https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/=
</a><br class=3D""></div></div></blockquote></div><br =
class=3D""></body></html>=

--Apple-Mail=_2A3BA52D-010A-4208-9B47-28C24B67B467--


From nobody Wed Apr 19 16:02:07 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 42817129C11 for <bmwg@ietfa.amsl.com>; Wed, 19 Apr 2017 16:02:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WTCI3uAktEH0 for <bmwg@ietfa.amsl.com>; Wed, 19 Apr 2017 16:02:04 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 977A8129C67 for <bmwg@ietf.org>; Wed, 19 Apr 2017 16:02:04 -0700 (PDT)
Received: from pps.filterd (m0053301.ppops.net [127.0.0.1]) by mx0a-00191d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v3JN0wLc037044 for <bmwg@ietf.org>; Wed, 19 Apr 2017 19:02:02 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by mx0a-00191d01.pphosted.com with ESMTP id 29xh6pr0ub-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Wed, 19 Apr 2017 19:02:02 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3JN21GM012534 for <bmwg@ietf.org>; Wed, 19 Apr 2017 19:02:01 -0400
Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3JN1rGY012429 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Wed, 19 Apr 2017 19:01:55 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi408.sfdc.sbc.com (RSA Interceptor) for <bmwg@ietf.org>; Wed, 19 Apr 2017 23:01:43 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3JN1hON032411 for <bmwg@ietf.org>; Wed, 19 Apr 2017 18:01:43 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3JN1cMD031749 for <bmwg@ietf.org>; Wed, 19 Apr 2017 18:01:38 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 83E2FF045D for <bmwg@ietf.org>; Wed, 19 Apr 2017 19:01:37 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Wed, 19 Apr 2017 19:01:37 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Draft Agenda for IETF-98 (minutes available)
Thread-Index: AdK5XfGIN1TTtxwBR32qvoBRVaq/Vg==
Date: Wed, 19 Apr 2017 23:01:37 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F7265A@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [130.10.249.244]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-19_16:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704190190
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Twtf1F1a4YQEUP0wU05prIaV9gc>
Subject: Re: [bmwg] Draft Agenda for IETF-98 (minutes available)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Apr 2017 23:02:06 -0000

Hi Folks,

A big thank you to Sarah for preparing/submitting our minutes [0]
in time for the proceedings cutoff date!=20
Also, thanks to our note takers and jabber scribe
(Jacob, Ramki, and Jay).

I remember offering to share the public URL of two
ETSI NFV Testing Working Group Specifications I've been
pushing up the hill. I wanted to wait till recent
releases of each of these specs available, and that
happened today.  The TST004 spec [1] is about various
metrics for establishing data paths through the
NFV Infrastructure, and their dataplane performance
once established. This is the spec that came-up in=20
connection with Ramki's ACL question on=20
the OPNFV vsperf draft.

ETSI NFV TST WG also has a draft of normative metrics for
fundamental NFVI platform measures (like CPU Util. and network
interface performance).  This one is called TST008 [2].

Both these draft specs are available in the NFV Open area
at the links below (grab the one with the latest date).

regards,
Al

[0] https://www.ietf.org/proceedings/98/minutes/minutes-98-bmwg-00.pdf

[1] https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST004_NFVI_PATH_TEST_repor=
t

[2] https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST008_NFVI_Metrics_Spec


> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED C
> (AL)
> Sent: Saturday, March 11, 2017 3:09 PM
> To: bmwg@ietf.org
> Subject: [bmwg] Draft Agenda for IETF-98
>=20
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more
> information.
>=20
> BMWG,
>=20
> The draft agenda for our session at IETF-98 in Chicago
> is available with the other meeting materials:
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__datatracker.ietf.org_meeting_98_agenda_bmwg_&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DCmiLBGBsDQpsyFiColhl8DgK=
OAj
> w6UDOM7ZvR4uO8f8&s=3DMB6sKDxDvSqqR8wmWLxXmIzxdG4bYlULcTSrzDB_Hjk&e=3D
>=20
> Please let me know of any:
>=20
> - requests for Remote Presentation (before March 24)
> - changes to Presenter Names
> - revised drafts (after today)
> - additional requests for agenda time
> - other Agenda Bashing...
>=20
> thanks, and see you in Chicago (UTC-5 when we get there).
> Al
> bmwg co-chair
>=20
>=20
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=3Dhttps-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=3DDwICAg&c=3DLFYZ-
> o9_HUMeMTSQicvjIg&r=3DOfsSu8kTIltVyD1oL72cBw&m=3DCmiLBGBsDQpsyFiColhl8DgK=
OAj
> w6UDOM7ZvR4uO8f8&s=3DcywZqwp8FeifSSIc8WopFxpHLSgfm5UOhvMtpqYeU70&e=3D


From nobody Sun Apr 23 07:41:07 2017
Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB1FE129564 for <bmwg@ietfa.amsl.com>; Sun, 23 Apr 2017 07:41:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dtqs6vPMBSEz for <bmwg@ietfa.amsl.com>; Sun, 23 Apr 2017 07:41:03 -0700 (PDT)
Received: from mail-qt0-x22e.google.com (mail-qt0-x22e.google.com [IPv6:2607:f8b0:400d:c0d::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C409C12955B for <bmwg@ietf.org>; Sun, 23 Apr 2017 07:41:02 -0700 (PDT)
Received: by mail-qt0-x22e.google.com with SMTP id m36so97522188qtb.0 for <bmwg@ietf.org>; Sun, 23 Apr 2017 07:41:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=ZPXrA+yX1AwhPeJnrNRlnEmXnRYZwNO0++GR9ratqmo=; b=C4DbSU0DhVWaEIEUKv1ldgbD1gT+RfcNtHts1UjoR649GS3Q5UZeqVuk50MFccl4id B2cw4msp8fpDeju9UGTUX4XwlQBdPss1l3Ds6mogJ4NnK+i25JyWPClFo5wgUx9hSoL3 xWWlH54JWCIVFJv4GuI0J56F2yXZP1yl+9Jp3R/lAcGLG9dU0F4VIIBS/ehVqnLG8CYy Lc+o36EFrHBTp1zTgtl9KhCfsprQrb1boD0FA1jIrhHJGeEuHMNBUW/dkN0tNIuDCMZ7 5KiwhbS/9Au4o+WxtpQ6GPoBHVpRABg+9hFnI1ju5dDvaoAhHkYeAWygtYjfMeJCgrKs FQJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ZPXrA+yX1AwhPeJnrNRlnEmXnRYZwNO0++GR9ratqmo=; b=FEQa7kx/R7O7DxCA/aFSggfWCslCG1nmXAfThls+ehbBGp3BUkl9yMvrsgB+I35zOl 8Fe6Y8uUXFJDAENy6Ddcx78VPMSNc+LUFDWLr074MbF9veqGO/CPbojjiJhdHPv7Wh24 xT8dtvhophojJsRMeN4BPA5mBFQ8c9ho3qhOFJq7/3HAA6Jh2MvqQeowTBL6FwMlH741 OhyuCXEltYnEbp0uJQkNDtIhQ1fFwpI0JGG3Atvu/H5yK7LcLToHOgOnG6v0Mhua6QDg QsBaOGULA13TEM0CggA8Du2aole1qIsGs7JP8DjCihN7UpkBnbyKGzw//MhnZDEAEYtH C/Vw==
X-Gm-Message-State: AN3rC/7Tijyjb0xvPSToj+E98cjK1UeDwKq9xYKQhAB4OYX5VKrd/dFA hbw439E5lkiwcExNeCYD8URDYwPaNRRXkJeCXg==
X-Received: by 10.200.51.70 with SMTP id u6mr21293641qta.122.1492958461482; Sun, 23 Apr 2017 07:41:01 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.12.158.154 with HTTP; Sun, 23 Apr 2017 07:40:21 -0700 (PDT)
From: Warren Kumari <warren@kumari.net>
Date: Sun, 23 Apr 2017 10:40:21 -0400
Message-ID: <CAHw9_iLhzXyV42a9cMVLMCcxNFDgFUODqXLyFnsRrq1XpXiQvA@mail.gmail.com>
To: draft-ietf-bmwg-vswitch-opnfv@ietf.org, sbanks@encrypted.net,  bmwg@ietf.org
Content-Type: text/plain; charset=UTF-8
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/YAhU7RwfGyEPP5uIpJ_4mf28ex8>
Subject: [bmwg] AD Eval of draft-ietf-bmwg-vswitch-opnfv
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Apr 2017 14:41:06 -0000

Hi there,

I've just done the AD eval of draft-ietf-bmwg-vswitch-opnfv - thank
you for writing this, I think that it is a useful document.

I had a few questions and comments which I think  it would be useful
to address before starting IETF LC. I'm not a benchmarking (or NFV /
vswitch person), so some of these questions / comments may be very
basic.

I've noted my questions with [WK: ...]. My questions / comments look
long, because I kept much of the text to make readability easier.

---------


2.  Scope

   The primary purpose and scope of the memo is to inform the industry
   of work-in-progress that builds on the body of extensive BMWG
   literature and experience, and describe the extensions needed for
   benchmarking virtual switches.  Inital feedback indicates that many

[WK: s/Inital/Initial/]

   of these extensions may be applicable beyond the current scope (to
   hardware switches in the NFV Infrastructure and to virtual routers,
   for example).  Additionally, this memo serves as a vehicle to include
   more detail and commentary from BMWG and other Open Source
   communities, under BMWG's chartered work to characterize the NFV

   Infrastructure (a virtual switch is an important aspect of that
   infrastructure).

   The benchmarking covered in this memo should be applicable to many
   types of vswitches, and remain vswitch-agnostic to great degree.

[WK: Please expand / define vswitch - perhaps in the parenthesis above.]

   There has been no attempt to track and test all features of any
   specific vswitch implementation.

...

3.1.  Comparison with Physical Network Functions

   To compare the performance of virtual designs and implementations
   with their physical counterparts, identical benchmarks are needed.
   BMWG has developed specifications for many network functions this
   memo re-uses existing benchmarks through references, and expands them
   during development of new methods.

[ WK: This sentence makes no sense / is missing some words / should be
2 sentences / something....]

   A key configuration aspect is the
   number of parallel cores required to achieve comparable performance
   with a given physical device, or whether some limit of scale was
   reached before the cores could achieve the comparable level.

   It's unlikely that the virtual switch will be the only application
   running on the SUT, so CPU utilization, Cache utilization, and Memory
   footprint should also be recorded for the virtual implementations of
   internetworking functions.

...

3.3.  New Configuration Parameters

   A key consideration when conducting any sort of benchmark is trying
   to ensure the consistency and repeatability of test results.  When
   benchmarking the performance of a vSwitch

[ WK: Please choose a single term (vswitch / vSwitch) ]

   there are many factors that
   can affect the consistency of results, one key factor is matching the
   various hardware and software details of the SUT.  This section lists
   some of the many new parameters which this project believes are
   critical to report in order to achieve repeatability.

   Hardware details including:

   o  Platform details

   o  Processor details

   o  Memory information (type and size)

[ WK: I would also think that the memory layout is important - if not,
ignore. Oh, it is mentioned below ("Memory DIMM configurations"). This
would be easier to read if you grouped memory together, processor /
cores together, etc.]


   o  Number of enabled cores

   o  Number of cores used for the test

   o  Number of physical NICs, as well as their details (manufacturer,
      versions, type and the PCI slot they are plugged into)

   o  NIC interrupt configuration

[WK: What about other NIC features (DDP, checksum magic,
scatter-gather, etc)? Or are these not applicable? (if not, ignore) ]

   o  BIOS version, release date and any configurations that were
      modified

   o  CPU microcode level

   o  Memory DIMM configurations (quad rank performance may not be the
      same as dual rank) in size, freq and slot locations

   o  PCI configuration parameters (payload size, early ack option...)

   o  Power management at all levels (ACPI sleep states, processor
      package, OS...)

   Software details including:

   o  OS parameters and behavior (text vs graphical no one typing at the
      console on one system)

   o  OS version (for host and VNF)

   o  Kernel version (for host and VNF)

   o  GRUB boot parameters (for host and VNF)

   o  Hypervisor details (Type and version)

   o  Selected vSwitch, version number or commit id used

   o  vSwitch launch command line if it has been parameterised

   o  Memory allocation to the vSwitch

   o  which NUMA node it is using, and how many memory channels

   o  DPDK or any other SW dependency version number or commit id used

   o  Memory allocation to a VM - if it's from Hugpages/elsewhere

[WK: Hugpages? It this the emo version of hugepages? :-) ]


   o  VM storage type: snapshot/independent persistent/independent non-
      persistent

   o  Number of VMs

   o  Number of Virtual NICs (vNICs), versions, type and driver


[ WK: What about IO virtualization stuff like SR-IOV and similar. Or
not applicable?]

   o  Number of virtual CPUs and their core affinity on the host

   o  Number vNIC interrupt configuration

   o  Thread affinitization for the applications (including the vSwitch
      itself) on the host

   o  Details of Resource isolation, such as CPUs designated for Host/
      Kernel (isolcpu) and CPUs designated for specific processes
      (taskset). - Test duration. - Number of flows.

   Test Traffic Information:

   o  Traffic type - UDP, TCP, IMIX / Other


[ WK: I don't think that IMIX is a well known acronym. I think either
define it (or ref: RFC6985?), or drop it (the section is "some of the
many new parameters"...)]

   o  Packet Sizes

   o  Deployment Scenario

3.4.  Flow classification

   Virtual switches group packets into flows by processing and matching
   particular packet or frame header information, or by matching packets
   based on the input ports.  Thus a flow can be thought of a sequence
   of packets that have the same set of header field values (5-tuple) or
   have arrived on the same port.  Performance results can vary based on
   the parameters the vSwitch uses to match for a flow.  The recommended
   flow classification parameters for any vSwitch performance tests are:
   the input port, the source IP address, the destination IP address and
   the Ethernet protocol type field.

[ WK: Is it useful to say "5-tuple" above? To me 5-tuple is src ip,
src port, dst ip, dst port, ip protocol. This says for vswitch it is
input port, src ip, dst ip and eth protocol. The two close together
may confuse readers - I think just drop "(5-tuple)". Also, this says
"input port" - is that actually input interface / should it be called
"physical port"? (when hearing IP and then port, people think L3
port)]


   o  Scalability Tests to understand how the virtual switch performs as
      the number of flows, active ports, complexity of the forwarding
      logic's configuration... it has to deal with increases.

[ WK: Looks like got sidetracked in this sentence. ]


---------

Thanks again,
W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf


From nobody Tue Apr 25 04:44:08 2017
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D39C12EC2D; Tue, 25 Apr 2017 04:44:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.4
X-Spam-Level: 
X-Spam-Status: No, score=-5.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZypTGAFNjDKJ; Tue, 25 Apr 2017 04:44:05 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C65FC12EC2F; Tue, 25 Apr 2017 04:44:04 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v3PBZW6k012878; Tue, 25 Apr 2017 07:44:01 -0400
Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0049463.ppops.net-00191d01. with ESMTP id 2a25as13nd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Apr 2017 07:44:00 -0400
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3PBi0ZT015785; Tue, 25 Apr 2017 07:44:00 -0400
Received: from mlpi407.sfdc.sbc.com (mlpi407.sfdc.sbc.com [130.9.128.239]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id v3PBho4f015716 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 25 Apr 2017 07:43:52 -0400
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by mlpi407.sfdc.sbc.com (RSA Interceptor); Tue, 25 Apr 2017 11:43:32 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3PBhWdr032158; Tue, 25 Apr 2017 06:43:32 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v3PBhOtl031783; Tue, 25 Apr 2017 06:43:24 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-azure.research.att.com (Postfix) with ESMTP id ACBE3E022F; Tue, 25 Apr 2017 07:43:23 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Tue, 25 Apr 2017 07:43:23 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Warren Kumari <warren@kumari.net>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, "sbanks@encrypted.net" <sbanks@encrypted.net>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: AD Eval of draft-ietf-bmwg-vswitch-opnfv
Thread-Index: AQHSvD+m1+lMKFsOy0SwDArzS5XAjKHV+UWw
Date: Tue, 25 Apr 2017 11:43:22 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25F89CE8@njmtexg4.research.att.com>
References: <CAHw9_iLhzXyV42a9cMVLMCcxNFDgFUODqXLyFnsRrq1XpXiQvA@mail.gmail.com>
In-Reply-To: <CAHw9_iLhzXyV42a9cMVLMCcxNFDgFUODqXLyFnsRrq1XpXiQvA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [192.144.77.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-25_07:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704250209
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/GulDAPxfbYzcGD7rpdAMsPoTDK8>
Subject: Re: [bmwg] AD Eval of draft-ietf-bmwg-vswitch-opnfv
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Apr 2017 11:44:07 -0000

VGhhbmtzIGZvciB5b3VyIHJldmlldyAmIGNvbW1lbnRzLCBXYXJyZW47IHRoZSBlZGl0cyB3aWxs
DQp0YWtlIGEgZmV3IGRheXMgdG8gcmVhY2ggdGhlIHRvcCBvZiB0aGUgVG9EbyBsaXN0Lg0KQWwN
Cg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBXYXJyZW4gS3VtYXJpIFtt
YWlsdG86d2FycmVuQGt1bWFyaS5uZXRdDQo+IFNlbnQ6IFN1bmRheSwgQXByaWwgMjMsIDIwMTcg
MTA6NDAgQU0NCj4gVG86IGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2QGlldGYub3JnOyBz
YmFua3NAZW5jcnlwdGVkLm5ldDsNCj4gYm13Z0BpZXRmLm9yZw0KPiBTdWJqZWN0OiBBRCBFdmFs
IG9mIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9wbmZ2DQo+IA0KPiBIaSB0aGVyZSwNCj4gDQo+
IEkndmUganVzdCBkb25lIHRoZSBBRCBldmFsIG9mIGRyYWZ0LWlldGYtYm13Zy12c3dpdGNoLW9w
bmZ2IC0gdGhhbmsNCj4geW91IGZvciB3cml0aW5nIHRoaXMsIEkgdGhpbmsgdGhhdCBpdCBpcyBh
IHVzZWZ1bCBkb2N1bWVudC4NCj4gDQo+IEkgaGFkIGEgZmV3IHF1ZXN0aW9ucyBhbmQgY29tbWVu
dHMgd2hpY2ggSSB0aGluayAgaXQgd291bGQgYmUgdXNlZnVsDQo+IHRvIGFkZHJlc3MgYmVmb3Jl
IHN0YXJ0aW5nIElFVEYgTEMuIEknbSBub3QgYSBiZW5jaG1hcmtpbmcgKG9yIE5GViAvDQo+IHZz
d2l0Y2ggcGVyc29uKSwgc28gc29tZSBvZiB0aGVzZSBxdWVzdGlvbnMgLyBjb21tZW50cyBtYXkg
YmUgdmVyeQ0KPiBiYXNpYy4NCj4gDQo+IEkndmUgbm90ZWQgbXkgcXVlc3Rpb25zIHdpdGggW1dL
OiAuLi5dLiBNeSBxdWVzdGlvbnMgLyBjb21tZW50cyBsb29rDQo+IGxvbmcsIGJlY2F1c2UgSSBr
ZXB0IG11Y2ggb2YgdGhlIHRleHQgdG8gbWFrZSByZWFkYWJpbGl0eSBlYXNpZXIuDQo+IA0KPiAt
LS0tLS0tLS0NCj4gDQo+IA0KPiAyLiAgU2NvcGUNCj4gDQo+ICAgIFRoZSBwcmltYXJ5IHB1cnBv
c2UgYW5kIHNjb3BlIG9mIHRoZSBtZW1vIGlzIHRvIGluZm9ybSB0aGUgaW5kdXN0cnkNCj4gICAg
b2Ygd29yay1pbi1wcm9ncmVzcyB0aGF0IGJ1aWxkcyBvbiB0aGUgYm9keSBvZiBleHRlbnNpdmUg
Qk1XRw0KPiAgICBsaXRlcmF0dXJlIGFuZCBleHBlcmllbmNlLCBhbmQgZGVzY3JpYmUgdGhlIGV4
dGVuc2lvbnMgbmVlZGVkIGZvcg0KPiAgICBiZW5jaG1hcmtpbmcgdmlydHVhbCBzd2l0Y2hlcy4g
IEluaXRhbCBmZWVkYmFjayBpbmRpY2F0ZXMgdGhhdCBtYW55DQo+IA0KPiBbV0s6IHMvSW5pdGFs
L0luaXRpYWwvXQ0KPiANCj4gICAgb2YgdGhlc2UgZXh0ZW5zaW9ucyBtYXkgYmUgYXBwbGljYWJs
ZSBiZXlvbmQgdGhlIGN1cnJlbnQgc2NvcGUgKHRvDQo+ICAgIGhhcmR3YXJlIHN3aXRjaGVzIGlu
IHRoZSBORlYgSW5mcmFzdHJ1Y3R1cmUgYW5kIHRvIHZpcnR1YWwgcm91dGVycywNCj4gICAgZm9y
IGV4YW1wbGUpLiAgQWRkaXRpb25hbGx5LCB0aGlzIG1lbW8gc2VydmVzIGFzIGEgdmVoaWNsZSB0
byBpbmNsdWRlDQo+ICAgIG1vcmUgZGV0YWlsIGFuZCBjb21tZW50YXJ5IGZyb20gQk1XRyBhbmQg
b3RoZXIgT3BlbiBTb3VyY2UNCj4gICAgY29tbXVuaXRpZXMsIHVuZGVyIEJNV0cncyBjaGFydGVy
ZWQgd29yayB0byBjaGFyYWN0ZXJpemUgdGhlIE5GVg0KPiANCj4gICAgSW5mcmFzdHJ1Y3R1cmUg
KGEgdmlydHVhbCBzd2l0Y2ggaXMgYW4gaW1wb3J0YW50IGFzcGVjdCBvZiB0aGF0DQo+ICAgIGlu
ZnJhc3RydWN0dXJlKS4NCj4gDQo+ICAgIFRoZSBiZW5jaG1hcmtpbmcgY292ZXJlZCBpbiB0aGlz
IG1lbW8gc2hvdWxkIGJlIGFwcGxpY2FibGUgdG8gbWFueQ0KPiAgICB0eXBlcyBvZiB2c3dpdGNo
ZXMsIGFuZCByZW1haW4gdnN3aXRjaC1hZ25vc3RpYyB0byBncmVhdCBkZWdyZWUuDQo+IA0KPiBb
V0s6IFBsZWFzZSBleHBhbmQgLyBkZWZpbmUgdnN3aXRjaCAtIHBlcmhhcHMgaW4gdGhlIHBhcmVu
dGhlc2lzIGFib3ZlLl0NCj4gDQo+ICAgIFRoZXJlIGhhcyBiZWVuIG5vIGF0dGVtcHQgdG8gdHJh
Y2sgYW5kIHRlc3QgYWxsIGZlYXR1cmVzIG9mIGFueQ0KPiAgICBzcGVjaWZpYyB2c3dpdGNoIGlt
cGxlbWVudGF0aW9uLg0KPiANCj4gLi4uDQo+IA0KPiAzLjEuICBDb21wYXJpc29uIHdpdGggUGh5
c2ljYWwgTmV0d29yayBGdW5jdGlvbnMNCj4gDQo+ICAgIFRvIGNvbXBhcmUgdGhlIHBlcmZvcm1h
bmNlIG9mIHZpcnR1YWwgZGVzaWducyBhbmQgaW1wbGVtZW50YXRpb25zDQo+ICAgIHdpdGggdGhl
aXIgcGh5c2ljYWwgY291bnRlcnBhcnRzLCBpZGVudGljYWwgYmVuY2htYXJrcyBhcmUgbmVlZGVk
Lg0KPiAgICBCTVdHIGhhcyBkZXZlbG9wZWQgc3BlY2lmaWNhdGlvbnMgZm9yIG1hbnkgbmV0d29y
ayBmdW5jdGlvbnMgdGhpcw0KPiAgICBtZW1vIHJlLXVzZXMgZXhpc3RpbmcgYmVuY2htYXJrcyB0
aHJvdWdoIHJlZmVyZW5jZXMsIGFuZCBleHBhbmRzIHRoZW0NCj4gICAgZHVyaW5nIGRldmVsb3Bt
ZW50IG9mIG5ldyBtZXRob2RzLg0KPiANCj4gWyBXSzogVGhpcyBzZW50ZW5jZSBtYWtlcyBubyBz
ZW5zZSAvIGlzIG1pc3Npbmcgc29tZSB3b3JkcyAvIHNob3VsZCBiZQ0KPiAyIHNlbnRlbmNlcyAv
IHNvbWV0aGluZy4uLi5dDQo+IA0KPiAgICBBIGtleSBjb25maWd1cmF0aW9uIGFzcGVjdCBpcyB0
aGUNCj4gICAgbnVtYmVyIG9mIHBhcmFsbGVsIGNvcmVzIHJlcXVpcmVkIHRvIGFjaGlldmUgY29t
cGFyYWJsZSBwZXJmb3JtYW5jZQ0KPiAgICB3aXRoIGEgZ2l2ZW4gcGh5c2ljYWwgZGV2aWNlLCBv
ciB3aGV0aGVyIHNvbWUgbGltaXQgb2Ygc2NhbGUgd2FzDQo+ICAgIHJlYWNoZWQgYmVmb3JlIHRo
ZSBjb3JlcyBjb3VsZCBhY2hpZXZlIHRoZSBjb21wYXJhYmxlIGxldmVsLg0KPiANCj4gICAgSXQn
cyB1bmxpa2VseSB0aGF0IHRoZSB2aXJ0dWFsIHN3aXRjaCB3aWxsIGJlIHRoZSBvbmx5IGFwcGxp
Y2F0aW9uDQo+ICAgIHJ1bm5pbmcgb24gdGhlIFNVVCwgc28gQ1BVIHV0aWxpemF0aW9uLCBDYWNo
ZSB1dGlsaXphdGlvbiwgYW5kIE1lbW9yeQ0KPiAgICBmb290cHJpbnQgc2hvdWxkIGFsc28gYmUg
cmVjb3JkZWQgZm9yIHRoZSB2aXJ0dWFsIGltcGxlbWVudGF0aW9ucyBvZg0KPiAgICBpbnRlcm5l
dHdvcmtpbmcgZnVuY3Rpb25zLg0KPiANCj4gLi4uDQo+IA0KPiAzLjMuICBOZXcgQ29uZmlndXJh
dGlvbiBQYXJhbWV0ZXJzDQo+IA0KPiAgICBBIGtleSBjb25zaWRlcmF0aW9uIHdoZW4gY29uZHVj
dGluZyBhbnkgc29ydCBvZiBiZW5jaG1hcmsgaXMgdHJ5aW5nDQo+ICAgIHRvIGVuc3VyZSB0aGUg
Y29uc2lzdGVuY3kgYW5kIHJlcGVhdGFiaWxpdHkgb2YgdGVzdCByZXN1bHRzLiAgV2hlbg0KPiAg
ICBiZW5jaG1hcmtpbmcgdGhlIHBlcmZvcm1hbmNlIG9mIGEgdlN3aXRjaA0KPiANCj4gWyBXSzog
UGxlYXNlIGNob29zZSBhIHNpbmdsZSB0ZXJtICh2c3dpdGNoIC8gdlN3aXRjaCkgXQ0KPiANCj4g
ICAgdGhlcmUgYXJlIG1hbnkgZmFjdG9ycyB0aGF0DQo+ICAgIGNhbiBhZmZlY3QgdGhlIGNvbnNp
c3RlbmN5IG9mIHJlc3VsdHMsIG9uZSBrZXkgZmFjdG9yIGlzIG1hdGNoaW5nIHRoZQ0KPiAgICB2
YXJpb3VzIGhhcmR3YXJlIGFuZCBzb2Z0d2FyZSBkZXRhaWxzIG9mIHRoZSBTVVQuICBUaGlzIHNl
Y3Rpb24gbGlzdHMNCj4gICAgc29tZSBvZiB0aGUgbWFueSBuZXcgcGFyYW1ldGVycyB3aGljaCB0
aGlzIHByb2plY3QgYmVsaWV2ZXMgYXJlDQo+ICAgIGNyaXRpY2FsIHRvIHJlcG9ydCBpbiBvcmRl
ciB0byBhY2hpZXZlIHJlcGVhdGFiaWxpdHkuDQo+IA0KPiAgICBIYXJkd2FyZSBkZXRhaWxzIGlu
Y2x1ZGluZzoNCj4gDQo+ICAgIG8gIFBsYXRmb3JtIGRldGFpbHMNCj4gDQo+ICAgIG8gIFByb2Nl
c3NvciBkZXRhaWxzDQo+IA0KPiAgICBvICBNZW1vcnkgaW5mb3JtYXRpb24gKHR5cGUgYW5kIHNp
emUpDQo+IA0KPiBbIFdLOiBJIHdvdWxkIGFsc28gdGhpbmsgdGhhdCB0aGUgbWVtb3J5IGxheW91
dCBpcyBpbXBvcnRhbnQgLSBpZiBub3QsDQo+IGlnbm9yZS4gT2gsIGl0IGlzIG1lbnRpb25lZCBi
ZWxvdyAoIk1lbW9yeSBESU1NIGNvbmZpZ3VyYXRpb25zIikuIFRoaXMNCj4gd291bGQgYmUgZWFz
aWVyIHRvIHJlYWQgaWYgeW91IGdyb3VwZWQgbWVtb3J5IHRvZ2V0aGVyLCBwcm9jZXNzb3IgLw0K
PiBjb3JlcyB0b2dldGhlciwgZXRjLl0NCj4gDQo+IA0KPiAgICBvICBOdW1iZXIgb2YgZW5hYmxl
ZCBjb3Jlcw0KPiANCj4gICAgbyAgTnVtYmVyIG9mIGNvcmVzIHVzZWQgZm9yIHRoZSB0ZXN0DQo+
IA0KPiAgICBvICBOdW1iZXIgb2YgcGh5c2ljYWwgTklDcywgYXMgd2VsbCBhcyB0aGVpciBkZXRh
aWxzIChtYW51ZmFjdHVyZXIsDQo+ICAgICAgIHZlcnNpb25zLCB0eXBlIGFuZCB0aGUgUENJIHNs
b3QgdGhleSBhcmUgcGx1Z2dlZCBpbnRvKQ0KPiANCj4gICAgbyAgTklDIGludGVycnVwdCBjb25m
aWd1cmF0aW9uDQo+IA0KPiBbV0s6IFdoYXQgYWJvdXQgb3RoZXIgTklDIGZlYXR1cmVzIChERFAs
IGNoZWNrc3VtIG1hZ2ljLA0KPiBzY2F0dGVyLWdhdGhlciwgZXRjKT8gT3IgYXJlIHRoZXNlIG5v
dCBhcHBsaWNhYmxlPyAoaWYgbm90LCBpZ25vcmUpIF0NCj4gDQo+ICAgIG8gIEJJT1MgdmVyc2lv
biwgcmVsZWFzZSBkYXRlIGFuZCBhbnkgY29uZmlndXJhdGlvbnMgdGhhdCB3ZXJlDQo+ICAgICAg
IG1vZGlmaWVkDQo+IA0KPiAgICBvICBDUFUgbWljcm9jb2RlIGxldmVsDQo+IA0KPiAgICBvICBN
ZW1vcnkgRElNTSBjb25maWd1cmF0aW9ucyAocXVhZCByYW5rIHBlcmZvcm1hbmNlIG1heSBub3Qg
YmUgdGhlDQo+ICAgICAgIHNhbWUgYXMgZHVhbCByYW5rKSBpbiBzaXplLCBmcmVxIGFuZCBzbG90
IGxvY2F0aW9ucw0KPiANCj4gICAgbyAgUENJIGNvbmZpZ3VyYXRpb24gcGFyYW1ldGVycyAocGF5
bG9hZCBzaXplLCBlYXJseSBhY2sgb3B0aW9uLi4uKQ0KPiANCj4gICAgbyAgUG93ZXIgbWFuYWdl
bWVudCBhdCBhbGwgbGV2ZWxzIChBQ1BJIHNsZWVwIHN0YXRlcywgcHJvY2Vzc29yDQo+ICAgICAg
IHBhY2thZ2UsIE9TLi4uKQ0KPiANCj4gICAgU29mdHdhcmUgZGV0YWlscyBpbmNsdWRpbmc6DQo+
IA0KPiAgICBvICBPUyBwYXJhbWV0ZXJzIGFuZCBiZWhhdmlvciAodGV4dCB2cyBncmFwaGljYWwg
bm8gb25lIHR5cGluZyBhdCB0aGUNCj4gICAgICAgY29uc29sZSBvbiBvbmUgc3lzdGVtKQ0KPiAN
Cj4gICAgbyAgT1MgdmVyc2lvbiAoZm9yIGhvc3QgYW5kIFZORikNCj4gDQo+ICAgIG8gIEtlcm5l
bCB2ZXJzaW9uIChmb3IgaG9zdCBhbmQgVk5GKQ0KPiANCj4gICAgbyAgR1JVQiBib290IHBhcmFt
ZXRlcnMgKGZvciBob3N0IGFuZCBWTkYpDQo+IA0KPiAgICBvICBIeXBlcnZpc29yIGRldGFpbHMg
KFR5cGUgYW5kIHZlcnNpb24pDQo+IA0KPiAgICBvICBTZWxlY3RlZCB2U3dpdGNoLCB2ZXJzaW9u
IG51bWJlciBvciBjb21taXQgaWQgdXNlZA0KPiANCj4gICAgbyAgdlN3aXRjaCBsYXVuY2ggY29t
bWFuZCBsaW5lIGlmIGl0IGhhcyBiZWVuIHBhcmFtZXRlcmlzZWQNCj4gDQo+ICAgIG8gIE1lbW9y
eSBhbGxvY2F0aW9uIHRvIHRoZSB2U3dpdGNoDQo+IA0KPiAgICBvICB3aGljaCBOVU1BIG5vZGUg
aXQgaXMgdXNpbmcsIGFuZCBob3cgbWFueSBtZW1vcnkgY2hhbm5lbHMNCj4gDQo+ICAgIG8gIERQ
REsgb3IgYW55IG90aGVyIFNXIGRlcGVuZGVuY3kgdmVyc2lvbiBudW1iZXIgb3IgY29tbWl0IGlk
IHVzZWQNCj4gDQo+ICAgIG8gIE1lbW9yeSBhbGxvY2F0aW9uIHRvIGEgVk0gLSBpZiBpdCdzIGZy
b20gSHVncGFnZXMvZWxzZXdoZXJlDQo+IA0KPiBbV0s6IEh1Z3BhZ2VzPyBJdCB0aGlzIHRoZSBl
bW8gdmVyc2lvbiBvZiBodWdlcGFnZXM/IDotKSBdDQo+IA0KPiANCj4gICAgbyAgVk0gc3RvcmFn
ZSB0eXBlOiBzbmFwc2hvdC9pbmRlcGVuZGVudCBwZXJzaXN0ZW50L2luZGVwZW5kZW50IG5vbi0N
Cj4gICAgICAgcGVyc2lzdGVudA0KPiANCj4gICAgbyAgTnVtYmVyIG9mIFZNcw0KPiANCj4gICAg
byAgTnVtYmVyIG9mIFZpcnR1YWwgTklDcyAodk5JQ3MpLCB2ZXJzaW9ucywgdHlwZSBhbmQgZHJp
dmVyDQo+IA0KPiANCj4gWyBXSzogV2hhdCBhYm91dCBJTyB2aXJ0dWFsaXphdGlvbiBzdHVmZiBs
aWtlIFNSLUlPViBhbmQgc2ltaWxhci4gT3INCj4gbm90IGFwcGxpY2FibGU/XQ0KPiANCj4gICAg
byAgTnVtYmVyIG9mIHZpcnR1YWwgQ1BVcyBhbmQgdGhlaXIgY29yZSBhZmZpbml0eSBvbiB0aGUg
aG9zdA0KPiANCj4gICAgbyAgTnVtYmVyIHZOSUMgaW50ZXJydXB0IGNvbmZpZ3VyYXRpb24NCj4g
DQo+ICAgIG8gIFRocmVhZCBhZmZpbml0aXphdGlvbiBmb3IgdGhlIGFwcGxpY2F0aW9ucyAoaW5j
bHVkaW5nIHRoZSB2U3dpdGNoDQo+ICAgICAgIGl0c2VsZikgb24gdGhlIGhvc3QNCj4gDQo+ICAg
IG8gIERldGFpbHMgb2YgUmVzb3VyY2UgaXNvbGF0aW9uLCBzdWNoIGFzIENQVXMgZGVzaWduYXRl
ZCBmb3IgSG9zdC8NCj4gICAgICAgS2VybmVsIChpc29sY3B1KSBhbmQgQ1BVcyBkZXNpZ25hdGVk
IGZvciBzcGVjaWZpYyBwcm9jZXNzZXMNCj4gICAgICAgKHRhc2tzZXQpLiAtIFRlc3QgZHVyYXRp
b24uIC0gTnVtYmVyIG9mIGZsb3dzLg0KPiANCj4gICAgVGVzdCBUcmFmZmljIEluZm9ybWF0aW9u
Og0KPiANCj4gICAgbyAgVHJhZmZpYyB0eXBlIC0gVURQLCBUQ1AsIElNSVggLyBPdGhlcg0KPiAN
Cj4gDQo+IFsgV0s6IEkgZG9uJ3QgdGhpbmsgdGhhdCBJTUlYIGlzIGEgd2VsbCBrbm93biBhY3Jv
bnltLiBJIHRoaW5rIGVpdGhlcg0KPiBkZWZpbmUgaXQgKG9yIHJlZjogUkZDNjk4NT8pLCBvciBk
cm9wIGl0ICh0aGUgc2VjdGlvbiBpcyAic29tZSBvZiB0aGUNCj4gbWFueSBuZXcgcGFyYW1ldGVy
cyIuLi4pXQ0KPiANCj4gICAgbyAgUGFja2V0IFNpemVzDQo+IA0KPiAgICBvICBEZXBsb3ltZW50
IFNjZW5hcmlvDQo+IA0KPiAzLjQuICBGbG93IGNsYXNzaWZpY2F0aW9uDQo+IA0KPiAgICBWaXJ0
dWFsIHN3aXRjaGVzIGdyb3VwIHBhY2tldHMgaW50byBmbG93cyBieSBwcm9jZXNzaW5nIGFuZCBt
YXRjaGluZw0KPiAgICBwYXJ0aWN1bGFyIHBhY2tldCBvciBmcmFtZSBoZWFkZXIgaW5mb3JtYXRp
b24sIG9yIGJ5IG1hdGNoaW5nIHBhY2tldHMNCj4gICAgYmFzZWQgb24gdGhlIGlucHV0IHBvcnRz
LiAgVGh1cyBhIGZsb3cgY2FuIGJlIHRob3VnaHQgb2YgYSBzZXF1ZW5jZQ0KPiAgICBvZiBwYWNr
ZXRzIHRoYXQgaGF2ZSB0aGUgc2FtZSBzZXQgb2YgaGVhZGVyIGZpZWxkIHZhbHVlcyAoNS10dXBs
ZSkgb3INCj4gICAgaGF2ZSBhcnJpdmVkIG9uIHRoZSBzYW1lIHBvcnQuICBQZXJmb3JtYW5jZSBy
ZXN1bHRzIGNhbiB2YXJ5IGJhc2VkIG9uDQo+ICAgIHRoZSBwYXJhbWV0ZXJzIHRoZSB2U3dpdGNo
IHVzZXMgdG8gbWF0Y2ggZm9yIGEgZmxvdy4gIFRoZSByZWNvbW1lbmRlZA0KPiAgICBmbG93IGNs
YXNzaWZpY2F0aW9uIHBhcmFtZXRlcnMgZm9yIGFueSB2U3dpdGNoIHBlcmZvcm1hbmNlIHRlc3Rz
IGFyZToNCj4gICAgdGhlIGlucHV0IHBvcnQsIHRoZSBzb3VyY2UgSVAgYWRkcmVzcywgdGhlIGRl
c3RpbmF0aW9uIElQIGFkZHJlc3MgYW5kDQo+ICAgIHRoZSBFdGhlcm5ldCBwcm90b2NvbCB0eXBl
IGZpZWxkLg0KPiANCj4gWyBXSzogSXMgaXQgdXNlZnVsIHRvIHNheSAiNS10dXBsZSIgYWJvdmU/
IFRvIG1lIDUtdHVwbGUgaXMgc3JjIGlwLA0KPiBzcmMgcG9ydCwgZHN0IGlwLCBkc3QgcG9ydCwg
aXAgcHJvdG9jb2wuIFRoaXMgc2F5cyBmb3IgdnN3aXRjaCBpdCBpcw0KPiBpbnB1dCBwb3J0LCBz
cmMgaXAsIGRzdCBpcCBhbmQgZXRoIHByb3RvY29sLiBUaGUgdHdvIGNsb3NlIHRvZ2V0aGVyDQo+
IG1heSBjb25mdXNlIHJlYWRlcnMgLSBJIHRoaW5rIGp1c3QgZHJvcCAiKDUtdHVwbGUpIi4gQWxz
bywgdGhpcyBzYXlzDQo+ICJpbnB1dCBwb3J0IiAtIGlzIHRoYXQgYWN0dWFsbHkgaW5wdXQgaW50
ZXJmYWNlIC8gc2hvdWxkIGl0IGJlIGNhbGxlZA0KPiAicGh5c2ljYWwgcG9ydCI/ICh3aGVuIGhl
YXJpbmcgSVAgYW5kIHRoZW4gcG9ydCwgcGVvcGxlIHRoaW5rIEwzDQo+IHBvcnQpXQ0KPiANCj4g
DQo+ICAgIG8gIFNjYWxhYmlsaXR5IFRlc3RzIHRvIHVuZGVyc3RhbmQgaG93IHRoZSB2aXJ0dWFs
IHN3aXRjaCBwZXJmb3JtcyBhcw0KPiAgICAgICB0aGUgbnVtYmVyIG9mIGZsb3dzLCBhY3RpdmUg
cG9ydHMsIGNvbXBsZXhpdHkgb2YgdGhlIGZvcndhcmRpbmcNCj4gICAgICAgbG9naWMncyBjb25m
aWd1cmF0aW9uLi4uIGl0IGhhcyB0byBkZWFsIHdpdGggaW5jcmVhc2VzLg0KPiANCj4gWyBXSzog
TG9va3MgbGlrZSBnb3Qgc2lkZXRyYWNrZWQgaW4gdGhpcyBzZW50ZW5jZS4gXQ0KPiANCj4gDQo+
IC0tLS0tLS0tLQ0KPiANCj4gVGhhbmtzIGFnYWluLA0KPiBXDQo+IA0KPiAtLQ0KPiBJIGRvbid0
IHRoaW5rIHRoZSBleGVjdXRpb24gaXMgcmVsZXZhbnQgd2hlbiBpdCB3YXMgb2J2aW91c2x5IGEg
YmFkDQo+IGlkZWEgaW4gdGhlIGZpcnN0IHBsYWNlLg0KPiBUaGlzIGlzIGxpa2UgcHV0dGluZyBy
YWJpZCB3ZWFzZWxzIGluIHlvdXIgcGFudHMsIGFuZCBsYXRlciBleHByZXNzaW5nDQo+IHJlZ3Jl
dCBhdCBoYXZpbmcgY2hvc2VuIHRob3NlIHBhcnRpY3VsYXIgcmFiaWQgd2Vhc2VscyBhbmQgdGhh
dCBwYWlyDQo+IG9mIHBhbnRzLg0KPiAgICAtLS1tYWYNCg==


From nobody Wed Apr 26 09:05:27 2017
Return-Path: <rjsparks@nostrum.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 960B11300E8; Wed, 26 Apr 2017 09:05:12 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Robert Sparks <rjsparks@nostrum.com>
To: <gen-art@ietf.org>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking.all@ietf.org, ietf@ietf.org, bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149322271242.5954.6564310624067283031@ietfa.amsl.com>
Date: Wed, 26 Apr 2017 09:05:12 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/NUBEEN2tkGx_1G1qcgT81RRHlO0>
Subject: [bmwg] Genart last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Apr 2017 16:05:22 -0000

Reviewer: Robert Sparks
Review result: Ready with Issues

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
Reviewer: Robert Sparks
Review Date: 2017-04-26
IETF LC End Date: 2017-05-02
IESG Telechat date: Not scheduled for a telechat

Summary: Essentially Ready for publication as an Informational RFC,
but with some minor issues to address before publication

This document is (with exceptions noted below) straightforward and
easy to follow

Minor Issues:

Section 8 is very confusing - I suspect it has been made so by
removing things that were in it earlier. Right now it claims to
provide additional tests, but the only content is about testing things
with firewalls, along with a statement that this document is only
targeting network devices that do not have a firewall function. I
think you can keep most of the text here (except the statement that
you aren't talking about things with firewalls) and remove the
confusion by changing the section heading to something like "Tests in
the presence of a firewall function".

It's unclear how to apply the formula in Section 10.2.1 to the results
that come out of, say, Section 7.3.2, where you are reporting a
(minimum, median, maximum) tuple. Some discussion about the
applicability of the tests where you recommend against reporting a
single number to the methods in this section would help. It would also
help to point out that the Xpd result can be go negative (it will go
negative for things that become smaller as the number of flows
increase, and positive for things that get bigger). If I read this
correctly, if throughput (for example) goes to 0 as n increases, Xpd
will go to -100%. Similary if latency doubles as n increases, Xpd will
go to +100% (and will go to +200% if latency triples).

Nits:

There are several places where you point to Section 6 where I think
you meant to point to Section 7. See the Procedure: line in 10.2.1 for
an example. Also make sure 9 is correct where you say "Sections 6
through 9" in section 12.

Please double-check that you meant "larger MTU" in the last paragraph
of 5.1. It might be correct, but I find the paragraph confusing.

In 8.1 you meant to point to 5.2, not 5.3

Please try to simplify this sentence: 
  "The duration of each trial SHOULD be at least 60 seconds to reduce
the potential gain of a DNS64 server, which is able to exhibit higher
performance by storing the requests and thus utilizing also the
timeout time for answering them."

(Style comment - please feel free to ignore): Consider deleting "as
well" everywhere it occurs in the document - most of the places it is
used, the sentence works just as well, and sometimes better, without
it.

Typos:
end of section 5.1.1: Appendix A1 should be Appendix A
please remove the comma in the first sentence of the second paragraph
of section 5.2



From nobody Wed Apr 26 09:28:00 2017
Return-Path: <sarah.banks@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1AF73131491; Wed, 26 Apr 2017 09:27:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level: 
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9,  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UngOZXbxf_3e; Wed, 26 Apr 2017 09:27:39 -0700 (PDT)
Received: from mail-oi0-x242.google.com (mail-oi0-x242.google.com [IPv6:2607:f8b0:4003:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 196D0124B0A; Wed, 26 Apr 2017 09:27:36 -0700 (PDT)
Received: by mail-oi0-x242.google.com with SMTP id m34so1441646oik.2; Wed, 26 Apr 2017 09:27:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;  h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IKasMraTRyvHWJ8U7oqEhFKH4QCq8fy89khGsA2KoEA=; b=q+Khg7PM6cHv+h6kIywxgD3N625KZZ+0dgw/qFTDraneg+3tohcuDHhLcDyidvxRaW VeKtoKnGUEWVqzdpwe7qYxS5gbh8PXMMkYCLW1FYRwJQB/5JLTezRgimyQ813GB8DjeS rgU0uLip2qVlgdbuk7JPaC+h3riN6K7UVy14Gp8B6nZ4sixtdCxPd7bRxuLYdCGDiGNh p/YKXKYJo/nOzDb7TTKASrmqkKLDoJnS4gUnsoL7AU08hssnUMTeezHlCE1DmOVbDtX1 s+VKscfvOQlQ3y8Cr/Q/i8OLjC0YfNhb83k2ndhf/iFpJQye7mEzXypibG8asHfCFg6K KKHg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IKasMraTRyvHWJ8U7oqEhFKH4QCq8fy89khGsA2KoEA=; b=Ooc4CkMXonMYH9V0F8ELNksjQFeG7/wbzvAVdFy8dDBjt4Wsc//O/8ju2D2khD89Et 1rq/boqVlAxXViOlfXz1Fi1Duage+k9Rh60TdYz+AfELueqeJ7yaz5tsNxgefiOH883R zwUl/dRZ81BW16q52bVkxNOf3Bx3gI+Ofq3yM9Ko3qRkEXQ1/Ma3RK43qgtwzn0xJ9AB /eBoXPXIPHjwwdicueTKOetinkmywsh8p+8S/BpFQzkisgb26vGoVCoFf1XAchiwTVq0 ZZNkkSEBtpA9RalZPq67OGS7PVnI5vScR18777hstl4ev3PFcDy9GdVmwHmjKSgmCQs/ QIxA==
X-Gm-Message-State: AN3rC/62HfgqOSf85sOZzfuiVkfmCfuyXhEabmdTDcDk2Vo5xUiGrNui xwFVYsu+NdaR/e0pytQ=
X-Received: by 10.157.49.17 with SMTP id e17mr405178otc.42.1493224055173; Wed, 26 Apr 2017 09:27:35 -0700 (PDT)
Received: from [10.211.187.101] (mobile-166-176-123-51.mycingular.net. [166.176.123.51]) by smtp.gmail.com with ESMTPSA id j51sm291042otc.47.2017.04.26.09.27.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Apr 2017 09:27:34 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Sarah Banks <sarah.banks@gmail.com>
X-Mailer: iPhone Mail (14E304)
In-Reply-To: <149322271242.5954.6564310624067283031@ietfa.amsl.com>
Date: Wed, 26 Apr 2017 12:27:33 -0400
Cc: gen-art@ietf.org, draft-ietf-bmwg-ipv6-tran-tech-benchmarking.all@ietf.org, ietf@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <85B5ED62-14E2-4103-9B59-20DAA8763E01@gmail.com>
References: <149322271242.5954.6564310624067283031@ietfa.amsl.com>
To: Robert Sparks <rjsparks@nostrum.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/myOFqBBqMzepIQU5AK4FWz0atk4>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Apr 2017 16:27:41 -0000

Hi Robert,
    Thank you for your review; we appreciate your time and comments, and the=
 authors will review in due course.=20

Thanks
Sarah

> On Apr 26, 2017, at 12:05 PM, Robert Sparks <rjsparks@nostrum.com> wrote:
>=20
> Reviewer: Robert Sparks
> Review result: Ready with Issues
>=20
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>=20
> For more information, please see the FAQ at
>=20
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>=20
> Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
> Reviewer: Robert Sparks
> Review Date: 2017-04-26
> IETF LC End Date: 2017-05-02
> IESG Telechat date: Not scheduled for a telechat
>=20
> Summary: Essentially Ready for publication as an Informational RFC,
> but with some minor issues to address before publication
>=20
> This document is (with exceptions noted below) straightforward and
> easy to follow
>=20
> Minor Issues:
>=20
> Section 8 is very confusing - I suspect it has been made so by
> removing things that were in it earlier. Right now it claims to
> provide additional tests, but the only content is about testing things
> with firewalls, along with a statement that this document is only
> targeting network devices that do not have a firewall function. I
> think you can keep most of the text here (except the statement that
> you aren't talking about things with firewalls) and remove the
> confusion by changing the section heading to something like "Tests in
> the presence of a firewall function".
>=20
> It's unclear how to apply the formula in Section 10.2.1 to the results
> that come out of, say, Section 7.3.2, where you are reporting a
> (minimum, median, maximum) tuple. Some discussion about the
> applicability of the tests where you recommend against reporting a
> single number to the methods in this section would help. It would also
> help to point out that the Xpd result can be go negative (it will go
> negative for things that become smaller as the number of flows
> increase, and positive for things that get bigger). If I read this
> correctly, if throughput (for example) goes to 0 as n increases, Xpd
> will go to -100%. Similary if latency doubles as n increases, Xpd will
> go to +100% (and will go to +200% if latency triples).
>=20
> Nits:
>=20
> There are several places where you point to Section 6 where I think
> you meant to point to Section 7. See the Procedure: line in 10.2.1 for
> an example. Also make sure 9 is correct where you say "Sections 6
> through 9" in section 12.
>=20
> Please double-check that you meant "larger MTU" in the last paragraph
> of 5.1. It might be correct, but I find the paragraph confusing.
>=20
> In 8.1 you meant to point to 5.2, not 5.3
>=20
> Please try to simplify this sentence:=20
>  "The duration of each trial SHOULD be at least 60 seconds to reduce
> the potential gain of a DNS64 server, which is able to exhibit higher
> performance by storing the requests and thus utilizing also the
> timeout time for answering them."
>=20
> (Style comment - please feel free to ignore): Consider deleting "as
> well" everywhere it occurs in the document - most of the places it is
> used, the sentence works just as well, and sometimes better, without
> it.
>=20
> Typos:
> end of section 5.1.1: Appendix A1 should be Appendix A
> please remove the comma in the first sentence of the second paragraph
> of section 5.2
>=20
>=20
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg


From nobody Wed Apr 26 23:10:25 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id DF8E4120326; Wed, 26 Apr 2017 23:10:23 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149327342386.2907.10285958194610420783@ietfa.amsl.com>
Date: Wed, 26 Apr 2017 23:10:23 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/xLKKbGRe95KDwErWV4ymstQvpEE>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-terminology-07.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Apr 2017 06:10:24 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Terminology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-terminology-07.txt
	Pages           : 16
	Date            : 2017-04-26

Abstract:
The purpose of this informational document is to establish definitions,
discussion and measurement techniques for data center benchmarking.
Also, it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts when
one is interested in benchmarking network switches and routers in the
data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-terminology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-07
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-terminology-07

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-terminology-07


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Wed Apr 26 23:10:44 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 764C91294A0; Wed, 26 Apr 2017 23:10:34 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149327343446.3048.16998045525795276276@ietfa.amsl.com>
Date: Wed, 26 Apr 2017 23:10:34 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/bf4azP7jgCmnwD4fXt4yBdy4kzI>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-dcbench-methodology-04.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Apr 2017 06:10:34 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Data Center Benchmarking Methodology
        Authors         : Lucien Avramov
                          Jacob Rapp
	Filename        : draft-ietf-bmwg-dcbench-methodology-04.txt
	Pages           : 18
	Date            : 2017-04-26

Abstract:
   The purpose of this informational document is to establish test and
   evaluation methodology and measurement techniques for physical
   network equipment in the data center.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-04
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-dcbench-methodology-04

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-dcbench-methodology-04


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Sat Apr 29 00:09:25 2017
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C417D1294EF; Sat, 29 Apr 2017 00:09:23 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: <i-d-announce@ietf.org>
Cc: bmwg@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.50.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <149344976376.2975.12745727120053963921@ietfa.amsl.com>
Date: Sat, 29 Apr 2017 00:09:23 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/AB1NNsM9tXe8duhMbgyvMsr_74A>
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 29 Apr 2017 07:09:24 -0000

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition Technologies
        Authors         : Marius Georgescu
                          Liviu Pislaru
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07.txt
	Pages           : 28
	Date            : 2017-04-29

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a metric for benchmarking load
   scalability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From nobody Sat Apr 29 00:49:46 2017
Return-Path: <marius.georgescu@rcs-rds.ro>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3D604129B5A; Sat, 29 Apr 2017 00:49:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level: 
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rcs-rds.ro
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9XS4QChOOzzf; Sat, 29 Apr 2017 00:49:41 -0700 (PDT)
Received: from mailproxy5.rcs-rds.ro (mailproxy5.rcs-rds.ro [212.54.124.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 44AC5129568; Sat, 29 Apr 2017 00:47:22 -0700 (PDT)
Received: from [10.252.6.123] (unknown [10.252.6.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mailproxy5.rcs-rds.ro (Postfix) with ESMTPSA id F0E262836A1; Sat, 29 Apr 2017 10:47:18 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rcs-rds.ro; s=MailProxy; t=1493452039; bh=Zb8otDAsXI/AYQAqTa++lYnEj6tDRePlfRNT1gEps+E=; h=Subject:To:References:Cc:From:Date:In-Reply-To; b=aY0Um0bFsdNK2zVbOn4Q7zu+/6QtbUWypmp/mUcvcIrgdWiMK4WNzel9YF6zE2j4u KYXd8B7K+DLjRemnhVImorOVYi6SD1UZlXe38DJfZxjikFPvvcMd2K1hHBUNfH0wAv rcUklW2plV3m7Hi6CkwwEY+KSvxPc/wTUsJtAc4w=
To: Robert Sparks <rjsparks@nostrum.com>, gen-art@ietf.org
References: <149322271242.5954.6564310624067283031@ietfa.amsl.com>
Cc: draft-ietf-bmwg-ipv6-tran-tech-benchmarking.all@ietf.org, ietf@ietf.org, bmwg@ietf.org
From: Marius Georgescu <marius.georgescu@rcs-rds.ro>
Message-ID: <303391a2-84f1-31dc-1b67-e3cc8b0aea3c@rcs-rds.ro>
Date: Sat, 29 Apr 2017 10:52:23 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <149322271242.5954.6564310624067283031@ietfa.amsl.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/bTAlin0oVtwOxTKeJOlM2oNKhGs>
Subject: Re: [bmwg] Genart last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 29 Apr 2017 07:49:44 -0000

Hello Robert,

Thank you very much for your detailed review.

We have revised the draft according to your comments.

The result was uploaded as:

https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07

Please find inline a punctual overview of the changes.

On 4/26/2017 7:05 PM, Robert Sparks wrote:
> Reviewer: Robert Sparks
> Review result: Ready with Issues
>
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
> Reviewer: Robert Sparks
> Review Date: 2017-04-26
> IETF LC End Date: 2017-05-02
> IESG Telechat date: Not scheduled for a telechat
>
> Summary: Essentially Ready for publication as an Informational RFC,
> but with some minor issues to address before publication
>
> This document is (with exceptions noted below) straightforward and
> easy to follow
>
> Minor Issues:
>
> Section 8 is very confusing - I suspect it has been made so by
> removing things that were in it earlier. Right now it claims to
> provide additional tests, but the only content is about testing things
> with firewalls, along with a statement that this document is only
> targeting network devices that do not have a firewall function. I
> think you can keep most of the text here (except the statement that
> you aren't talking about things with firewalls) and remove the
> confusion by changing the section heading to something like "Tests in
> the presence of a firewall function".
- we have removed the second paragraph , which was indeed confusing.
The point, which may be clearer in Section 1.1 is that we need
additional benchmarks for stateful IPv6 transition technologies,
which may be (or not) labeled as having a firewall function.  
> It's unclear how to apply the formula in Section 10.2.1 to the results
> that come out of, say, Section 7.3.2, where you are reporting a
> (minimum, median, maximum) tuple. Some discussion about the
> applicability of the tests where you recommend against reporting a
> single number to the methods in this section would help.
+ we added a note in the reporting format part stating: "

*In the case of benchmarks for which more than one value is reported
(e.g. IPDV Section 7.3.2), a column for each of the values SHOULD be
included.*

"
>  It would also
> help to point out that the Xpd result can be go negative (it will go
> negative for things that become smaller as the number of flows
> increase, and positive for things that get bigger). If I read this
> correctly, if throughput (for example) goes to 0 as n increases, Xpd
> will go to -100%. Similary if latency doubles as n increases, Xpd will
> go to +100% (and will go to +200% if latency triples).
+ This was one of the important points which needed clarification. Thank
you for pointing it out.
In order to keep the  performance degradation results in the Lower is
better  direction, we proposed a secondary calculation formula for
higher is better benchmarks (e.g. Throughput). This makes, in our
opinion, the results easier to interpret.

> Nits:
>
> There are several places where you point to Section 6 where I think
> you meant to point to Section 7. See the Procedure: line in 10.2.1 for
> an example. Also make sure 9 is correct where you say "Sections 6
> through 9" in section 12.
+Corrected
>
> Please double-check that you meant "larger MTU" in the last paragraph
> of 5.1. It might be correct, but I find the paragraph confusing.
We find this part to be correct. However, keeping only the 1280 +
overhead recommendation might make it less confusing.
Do you think that would be a viable solution?
> In 8.1 you meant to point to 5.2, not 5.3
>
> Please try to simplify this sentence: 
>   "The duration of each trial SHOULD be at least 60 seconds to reduce
> the potential gain of a DNS64 server, which is able to exhibit higher
> performance by storing the requests and thus utilizing also the
> timeout time for answering them."
+corrected
>
> (Style comment - please feel free to ignore): Consider deleting "as
> well" everywhere it occurs in the document - most of the places it is
> used, the sentence works just as well, and sometimes better, without
> it.
+corrected
> Typos:
> end of section 5.1.1: Appendix A1 should be Appendix A
+corrected

Best regards,
Marius

