
From ilya@nobulus.com  Mon Jul  2 07:55:35 2012
Return-Path: <ilya@nobulus.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AA88121F85A8 for <bmwg@ietfa.amsl.com>; Mon,  2 Jul 2012 07:55:35 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.598
X-Spam-Level: 
X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ehj3cOlk00I4 for <bmwg@ietfa.amsl.com>; Mon,  2 Jul 2012 07:55:33 -0700 (PDT)
Received: from nobulus.com (nobulus.com [IPv6:2001:6f8:892:6ff::11:152]) by ietfa.amsl.com (Postfix) with ESMTP id EAEAF21F86A4 for <bmwg@ietf.org>; Mon,  2 Jul 2012 07:55:31 -0700 (PDT)
Received: from nobulus.com (localhost [127.0.0.1]) by nobulus.com (Postfix) with ESMTP id 9445E1714A; Mon,  2 Jul 2012 16:55:34 +0200 (CEST)
X-Virus-Scanned: amavisd-new at nobulus.com
Received: from nobulus.com ([127.0.0.1]) by nobulus.com (nobulus.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id U1HQ+mRP4eGr; Mon,  2 Jul 2012 16:55:27 +0200 (CEST)
Received: from HNIVARLAS2 (unknown [IPv6:2001:6f8:892:6f8:a0a1:604:2fa:a2e8]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by nobulus.com (Postfix) with ESMTPSA id E9B3D17593; Mon,  2 Jul 2012 16:55:26 +0200 (CEST)
From: "Ilya Varlashkin" <ilya@nobulus.com>
To: "'Al Morton'" <acmorton@att.com>, <bmwg@ietf.org>
References: <201206281523.q5SFNq2L003397@alpd052.aldc.att.com>
In-Reply-To: <201206281523.q5SFNq2L003397@alpd052.aldc.att.com>
Date: Mon, 2 Jul 2012 16:55:26 +0200
Message-ID: <010901cd5862$b75c6f20$26154d60$@nobulus.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_010A_01CD5873.7AEAE470"
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AQIvIJeuphSC1GDy2STSyKkWFus9mZZSgCTQ
Content-Language: en-gb
Subject: Re: [bmwg] Adoption of Router/Basic BGP Convergence Drafts
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Jul 2012 14:55:35 -0000

This is a multipart message in MIME format.

------=_NextPart_000_010A_01CD5873.7AEAE470
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit

Hello all,

 

I've got stuck with my draft - the traffic generator/protocol simulator has
serious issues when I need to simulate same exit/border router reachable via
multiple ports, this severely limit type of test scenarios that can be
simulated. The problem #1 is that BGP session cannot be transferred from one
port to another, so primary path failure results in primary BGP session
going down. The second issue is with trying to work around #1 - when IGP (in
my case it was ISIS) on two ports advertises same node/router, which I
thought I could use as next-hop in BGP updates without actually establishing
session to it from DUT, the two ports advertised actually two independent
routers sharing most of information but having different (and ever
increasing) Sequence Number. The ever increasing Sequence Number caused DUT
to run new SPF right after previous one finished, which in turn kicked up
exponential back-off resulting in much slower convergence and thus
diminishing usefulness of the tests.

 

I'm now trying to figure out alternative setup that would involve additional
real routers, but need to see if results will be meaningful.

 

Based on this I'd ask not to adopt my draft at this moment as WG item.

 

/iLya

 

From: bmwg-bounces@ietf.org [mailto:bmwg-bounces@ietf.org] On Behalf Of Al
Morton
Sent: Thursday, June 28, 2012 5:25 PM
To: bmwg@ietf.org
Subject: [bmwg] Adoption of Router/Basic BGP Convergence Drafts

 

BMWG,

We (still) have a charter item on BGP Convergence:
Basic BGP Convergence Benchmarking Methodology to IESG Review (overdue)

Over the last year or so, we've discussed two drafts that
will help to satisfy the milestone.  We've also seen some results of 
implementation and testing at our last face2face session.

The two drafts are:

Convergence benchmarking on contemporary routers
http://tools.ietf.org/id/draft-varlashkin-router-conv-bench-00.txt

Basic BGP Convergence Benchmarking Methodology for Data Plane Convergence

 
 <http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence-03> 
http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence-03
 <http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence-03>  
 

The combination of these drafts address both the milestone and
issues with non-convergence path restoration that has come-up 
in IESG comments in the past.

Therefore, I now ask whether the working group members feel we
should adopt these as WG drafts. So:

1. Please Read the Drafts by July 19th, 2012.

2. Respond with your review, and 

3. give your impressions on whether they are sufficiently clear in their 
approach and direction, such that we can complete them in a timely 
fashion in the working group.

That's it!

regards,
Al
bmwg chair


------=_NextPart_000_010A_01CD5873.7AEAE470
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta =
http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii"><meta name=3DGenerator content=3D"Microsoft Word 14 =
(filtered medium)"><style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
	{font-family:Consolas;
	panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
pre
	{mso-style-priority:99;
	mso-style-link:"HTML Preformatted Char";
	margin:0cm;
	margin-bottom:.0001pt;
	font-size:10.0pt;
	font-family:"Courier New";}
span.HTMLPreformattedChar
	{mso-style-name:"HTML Preformatted Char";
	mso-style-priority:99;
	mso-style-link:"HTML Preformatted";
	font-family:Consolas;}
span.EmailStyle19
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-GB link=3Dblue =
vlink=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'>Hello all,<o:p></o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'>I&#8217;ve got stuck with my draft &#8211; the traffic =
generator/protocol simulator has serious issues when I need to simulate =
same exit/border router reachable via multiple ports, this severely =
limit type of test scenarios that can be simulated. The problem #1 is =
that BGP session cannot be transferred from one port to another, so =
primary path failure results in primary BGP session going down. The =
second issue is with trying to work around #1 &#8211; when IGP (in my =
case it was ISIS) on two ports advertises same node/router, which I =
thought I could use as next-hop in BGP updates without actually =
establishing session to it from DUT, the two ports advertised actually =
two independent routers sharing most of information but having different =
(and ever increasing) Sequence Number. The ever increasing Sequence =
Number caused DUT to run new SPF right after previous one finished, =
which in turn kicked up exponential back-off resulting in much slower =
convergence and thus diminishing usefulness of the =
tests.<o:p></o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'>I&#8217;m now trying to figure out alternative setup that would =
involve additional real routers, but need to see if results will be =
meaningful.<o:p></o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'>Based on this I&#8217;d ask not to adopt my draft at this moment as =
WG item.<o:p></o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'>/iLya<o:p></o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'><o:p>&nbsp;</o:p></span></p><div =
style=3D'border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm =
4.0pt'><div><div style=3D'border:none;border-top:solid #B5C4DF =
1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=3DMsoNormal><b><span =
lang=3DEN-US =
style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span>=
</b><span lang=3DEN-US =
style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> =
bmwg-bounces@ietf.org [mailto:bmwg-bounces@ietf.org] <b>On Behalf Of =
</b>Al Morton<br><b>Sent:</b> Thursday, June 28, 2012 5:25 =
PM<br><b>To:</b> bmwg@ietf.org<br><b>Subject:</b> [bmwg] Adoption of =
Router/Basic BGP Convergence Drafts<o:p></o:p></span></p></div></div><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>BMWG,<br><br>We (still) have a charter item on BGP =
Convergence:<br>Basic BGP Convergence Benchmarking Methodology to IESG =
Review (overdue)<br><br>Over the last year or so, we've discussed two =
drafts that<br>will help to satisfy the milestone.&nbsp; We've also seen =
some results of <br>implementation and testing at our last face2face =
session.<br><br>The two drafts are:<br><br>Convergence benchmarking on =
contemporary routers<br><a =
href=3D"http://tools.ietf.org/id/draft-varlashkin-router-conv-bench-00.tx=
t">http://tools.ietf.org/id/draft-varlashkin-router-conv-bench-00.txt</a>=
<br><br>Basic BGP Convergence Benchmarking Methodology for Data Plane =
Convergence<o:p></o:p></p><pre><o:p>&nbsp;</o:p></pre><pre><a =
href=3D"http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence=
-03"><o:p></o:p></a></pre><pre><span class=3DMsoHyperlink><a =
href=3D"http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence=
-03">http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence-03=
<o:p></o:p></a></span></pre><pre><span class=3DMsoHyperlink><a =
href=3D"http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence=
-03"><o:p><span =
style=3D'text-decoration:none'>&nbsp;</span></o:p></a></span></pre><pre><=
o:p>&nbsp;</o:p></pre><p class=3DMsoNormal>The combination of these =
drafts address both the milestone and<br>issues with non-convergence =
path restoration that has come-up <br>in IESG comments in the =
past.<br><br>Therefore, I now ask whether the working group members feel =
we<br>should adopt these as WG drafts. So:<br><br>1. Please Read the =
Drafts by July 19th, 2012.<br><br>2. Respond with your review, and =
<br><br>3. give your impressions on whether they are sufficiently clear =
in their <br>approach and direction, such that we can complete them in a =
timely <br>fashion in the working group.<br><br>That's =
it!<br><br>regards,<br>Al<br>bmwg =
chair<o:p></o:p></p></div></div></body></html>
------=_NextPart_000_010A_01CD5873.7AEAE470--


From acmorton@att.com  Mon Jul  2 10:49:11 2012
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F83821F8771 for <bmwg@ietfa.amsl.com>; Mon,  2 Jul 2012 10:49:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.963
X-Spam-Level: 
X-Spam-Status: No, score=-103.963 tagged_above=-999 required=5 tests=[AWL=-1.021, BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.457, MIME_QP_LONG_LINE=1.396, MSGID_FROM_MTA_HEADER=0.803, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6jTaGTviNEQ3 for <bmwg@ietfa.amsl.com>; Mon,  2 Jul 2012 10:49:10 -0700 (PDT)
Received: from nbfkord-smmo06.seg.att.com (nbfkord-smmo06.seg.att.com [209.65.160.94]) by ietfa.amsl.com (Postfix) with ESMTP id 449FC21F875C for <bmwg@ietf.org>; Mon,  2 Jul 2012 10:49:10 -0700 (PDT)
Received: from unknown [144.160.128.153] (EHLO flpi408.enaf.ffdc.sbc.com) by nbfkord-smmo06.seg.att.com(mxl_mta-6.11.0-10) over TLS secured channel with ESMTP id 51fd1ff4.0.287888.00-378.775547.nbfkord-smmo06.seg.att.com (envelope-from <acmorton@att.com>);  Mon, 02 Jul 2012 17:49:16 +0000 (UTC)
X-MXL-Hash: 4ff1df1c3084e574-95da1cc75fea570f562a047d5ef9554675429b0f
Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flpi408.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id q62Hn8wO026176 for <bmwg@ietf.org>; Mon, 2 Jul 2012 10:49:09 -0700
Received: from fflint03.pst.cso.att.com (fflint03.pst.cso.att.com [150.234.39.63]) by flpi408.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id q62Hmvs0026009 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <bmwg@ietf.org>; Mon, 2 Jul 2012 10:49:04 -0700
Received: from alpd052.aldc.att.com (alpd052.aldc.att.com [130.8.42.31]) by fflint03.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Mon, 2 Jul 2012 10:48:41 -0700
Received: from aldc.att.com (localhost.localdomain [127.0.0.1]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q62HmeAM013037 for <bmwg@ietf.org>; Mon, 2 Jul 2012 13:48:40 -0400
Received: from mailgw1.maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q62Hma8P012873 for <bmwg@ietf.org>; Mon, 2 Jul 2012 13:48:36 -0400
Message-Id: <201207021748.q62Hma8P012873@alpd052.aldc.att.com>
Received: from acmt.att.com (dn135-16-251-71.dhcpn.ugn.att.com[135.16.251.71](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20120702174428gw10060lr4e>; Mon, 2 Jul 2012 17:44:28 +0000
X-Originating-IP: [135.16.251.71]
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Mon, 02 Jul 2012 13:49:56 -0400
To: "Ilya Varlashkin" <ilya@nobulus.com>, <bmwg@ietf.org>
From: Al Morton <acmorton@att.com>
In-Reply-To: <010901cd5862$b75c6f20$26154d60$@nobulus.com>
References: <201206281523.q5SFNq2L003397@alpd052.aldc.att.com> <010901cd5862$b75c6f20$26154d60$@nobulus.com>
Mime-Version: 1.0
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)]
X-MAIL-FROM: <acmorton@att.com>
X-SOURCE-IP: [144.160.128.153]
X-AnalysisOut: [v=1.0 c=1 a=dit6Jc5sqrAA:10 a=eLynkO_QMvYA:10 a=ofMgfj31e3]
X-AnalysisOut: [cA:10 a=BLceEmwcHowA:10 a=8nJEP1OIZ-IA:10 a=xwOvzTHDVLE4u4]
X-AnalysisOut: [nGvK72ag==:17 a=48vgC7mUAAAA:8 a=nlaYq9vyUcXnH3aRz18A:9 a=]
X-AnalysisOut: [wPNLvfGTeEIA:10 a=_W_S_7VecoQA:10 a=lZB815dzVvQA:10 a=XCDX]
X-AnalysisOut: [ZVy3xw33xmOs:21 a=kEozUSKTgG8iQwy9:21]
Subject: Re: [bmwg] Adoption of Router/Basic BGP Convergence Drafts
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Jul 2012 17:49:11 -0000

<html>
<body>
Ok Ilya, thanks for the update.<br>
BMWG will postpone adoption your draft at this time.<br><br>
If anyone has suggestions for Ilya, I'm sure they<br>
will be appreciated.<br><br>
regards,<br>
Al<br>
bmwg chair<br><br>
At 10:55 AM 7/2/2012, Ilya Varlashkin wrote:<br>
<blockquote type=3Dcite class=3Dcite cite=3D"">Content-Type:
multipart/alternative;<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>
boundary=3D&quot;----=3D_NextPart_000_010A_01CD5873.7AEAE470&quot;<br>
Content-Language: en-gb<br><br>
Hello all,<br>
&nbsp;<br>
I=92ve got stuck with my draft =96 the traffic generator/protocol simulator
has serious issues when I need to simulate same exit/border router
reachable via multiple ports, this severely limit type of test scenarios
that can be simulated. The problem #1 is that BGP session cannot be
transferred from one port to another, so primary path failure results in
primary BGP session going down. The second issue is with trying to work
around #1 =96 when IGP (in my case it was ISIS) on two ports advertises
same node/router, which I thought I could use as next-hop in BGP updates
without actually establishing session to it from DUT, the two ports
advertised actually two independent routers sharing most of information
but having different (and ever increasing) Sequence Number. The ever
increasing Sequence Number caused DUT to run new SPF right after previous
one finished, which in turn kicked up exponential back-off resulting in
much slower convergence and thus diminishing usefulness of the
tests.<br>
&nbsp;<br>
I=92m now trying to figure out alternative setup that would involve
additional real routers, but need to see if results will be
meaningful.<br>
&nbsp;<br>
Based on this I=92d ask not to adopt my draft at this moment as WG
item.<br>
&nbsp;<br>
/iLya<br>
&nbsp;<br>
<b>From:</b> bmwg-bounces@ietf.org
[<a href=3D"mailto:bmwg-bounces@ietf.org" eudora=3D"autourl">
mailto:bmwg-bounces@ietf.org</a>] <b>On Behalf Of </b>Al Morton<br>
<b>Sent:</b> Thursday, June 28, 2012 5:25 PM<br>
<b>To:</b> bmwg@ietf.org<br>
<b>Subject:</b> [bmwg] Adoption of Router/Basic BGP Convergence
Drafts<br>
&nbsp;<br>
BMWG,<br><br>
We (still) have a charter item on BGP Convergence:<br>
Basic BGP Convergence Benchmarking Methodology to IESG Review
(overdue)<br><br>
Over the last year or so, we've discussed two drafts that<br>
will help to satisfy the milestone.&nbsp; We've also seen some results of
<br>
implementation and testing at our last face2face session.<br><br>
The two drafts are:<br><br>
Convergence benchmarking on contemporary routers<br>
<a href=3D"http://tools.ietf.org/id/draft-varlashkin-router-conv-bench-00.tx=
t">
http://tools.ietf.org/id/draft-varlashkin-router-conv-bench-00.txt</a><br>
<br>
Basic BGP Convergence Benchmarking Methodology for Data Plane
Convergence<br><br>
<pre>&nbsp;</pre><br><br>
<br><br>
<pre>
<a href=3D"http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence=
-03">
http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence-03</a>
</pre><br><br>
<pre>
<a href=3D"http://tools.ietf.org/html/draft-papneja-bgp-basic-dp-convergence=
-03">
&nbsp;</a></pre><br><br>
<pre>&nbsp;</pre><br>
The combination of these drafts address both the milestone and<br>
issues with non-convergence path restoration that has come-up <br>
in IESG comments in the past.<br><br>
Therefore, I now ask whether the working group members feel we<br>
should adopt these as WG drafts. So:<br><br>
1. Please Read the Drafts by July 19th, 2012.<br><br>
2. Respond with your review, and <br><br>
3. give your impressions on whether they are sufficiently clear in their
<br>
approach and direction, such that we can complete them in a timely <br>
fashion in the working group.<br><br>
That's it!<br><br>
regards,<br>
Al<br>
bmwg chair<br>
_______________________________________________<br>
bmwg mailing list<br>
bmwg@ietf.org<a href=3D"https://www.ietf.org/mailman/listinfo/bmwg" eudora=
=3D"autourl">
https://www.ietf.org/mailman/listinfo/bmwg</a></blockquote></body>
</html>


From internet-drafts@ietf.org  Thu Jul  5 08:00:40 2012
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CCA3421F8757; Thu,  5 Jul 2012 08:00:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.518
X-Spam-Level: 
X-Spam-Status: No, score=-102.518 tagged_above=-999 required=5 tests=[AWL=0.081, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lflw9AdCYL8w; Thu,  5 Jul 2012 08:00:40 -0700 (PDT)
Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D3FA21F86CF; Thu,  5 Jul 2012 08:00:40 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: internet-drafts@ietf.org
To: i-d-announce@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 4.30
Message-ID: <20120705150040.4941.50768.idtracker@ietfa.amsl.com>
Date: Thu, 05 Jul 2012 08:00:40 -0700
Cc: bmwg@ietf.org
Subject: [bmwg] I-D Action: draft-ietf-bmwg-imix-genome-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Jul 2012 15:00:41 -0000

A New Internet-Draft is available from the on-line Internet-Drafts director=
ies.
 This draft is a work item of the Benchmarking Methodology Working Group of=
 the IETF.

	Title           : IMIX Genome: Specification of variable packet sizes for =
additional testing
	Author(s)       : Al Morton
	Filename        : draft-ietf-bmwg-imix-genome-02.txt
	Pages           : 9
	Date            : 2012-07-05

Abstract:
   Benchmarking Methodologies have always relied on test conditions with
   constant packet sizes, with the goal of understanding what network
   device capability has been tested.  Tests with constant packet size
   reveal device capabilities but differ significantly from the
   conditions encountered in operational deployment, and so additional
   tests are sometimes conducted with a mixture of packet sizes, or
   "IMIX".  The mixture of sizes a networking device will encounter is
   highly variable and depends on many factors.  An IMIX suited for one
   networking device and deployment will not be appropriate for another.
   However, the mix of sizes may be known and the tester may be asked to
   augment the fixed size tests.  To address this need, and the
   perpetual goal of specifying repeatable test conditions, this draft
   defines a way to specify the exact repeating sequence of packet sizes
   from the usual set of fixed sizes, and other forms of mixed size
   specification.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-imix-genome

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-bmwg-imix-genome-02

A diff from previous version is available at:
http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-bmwg-imix-genome-02


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From wwwrun@rfc-editor.org  Tue Jul 10 11:00:00 2012
Return-Path: <wwwrun@rfc-editor.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8D43C21F862F; Tue, 10 Jul 2012 10:59:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.174
X-Spam-Level: 
X-Spam-Status: No, score=-102.174 tagged_above=-999 required=5 tests=[AWL=0.426, BAYES_00=-2.599, NO_RELAYS=-0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MOWz23kyuWib; Tue, 10 Jul 2012 10:59:58 -0700 (PDT)
Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1890:123a::1:2f]) by ietfa.amsl.com (Postfix) with ESMTP id D06E421F8628; Tue, 10 Jul 2012 10:59:57 -0700 (PDT)
Received: by rfc-editor.org (Postfix, from userid 30) id 5172172F655; Tue, 10 Jul 2012 11:00:13 -0700 (PDT)
To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
From: rfc-editor@rfc-editor.org
Message-Id: <20120710180014.5172172F655@rfc-editor.org>
Date: Tue, 10 Jul 2012 11:00:13 -0700 (PDT)
Cc: bmwg@ietf.org, rfc-editor@rfc-editor.org
Subject: [bmwg] RFC 6645 on IP Flow Information Accounting and Export Benchmarking Methodology
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Jul 2012 18:00:00 -0000

A new Request for Comments is now available in online RFC libraries.

        
        RFC 6645

        Title:      IP Flow Information Accounting and 
                    Export Benchmarking Methodology 
        Author:     J. Novak
        Status:     Informational
        Stream:     IETF
        Date:       July 2012
        Mailbox:    janovak@cisco.com
        Pages:      39
        Characters: 87875
        Updates/Obsoletes/SeeAlso:   None

        I-D Tag:    draft-ietf-bmwg-ipflow-meth-10.txt

        URL:        http://www.rfc-editor.org/rfc/rfc6645.txt

This document provides a methodology and framework for quantifying
the performance impact of the monitoring of IP flows on a network device
and the export of this information to a Collector.  It identifies the rate
at which the IP flows are created, expired, and successfully exported
as a new performance metric in combination with traditional
throughput.  The metric is only applicable to the devices compliant
with RFC 5470, "Architecture for IP Flow Information Export".  The
methodology quantifies the impact of the IP flow monitoring process
on the network equipment.  This document is not an Internet Standards 
Track specification; it is published for informational purposes.

This document is a product of the Benchmarking Methodology Working Group of the IETF.


INFORMATIONAL: This memo provides information for the Internet community.
It does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

This announcement is sent to the IETF-Announce and rfc-dist lists.
To subscribe or unsubscribe, see
  http://www.ietf.org/mailman/listinfo/ietf-announce
  http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist

For searching the RFC series, see http://www.rfc-editor.org/rfcsearch.html.
For downloading RFCs, see http://www.rfc-editor.org/rfc.html.

Requests for special distribution should be addressed to either the
author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
specifically noted otherwise on the RFC itself, all RFCs are for
unlimited distribution.


The RFC Editor Team
Association Management Solutions, LLC



From jakarthi@cisco.com  Fri Jul 13 18:54:38 2012
Return-Path: <jakarthi@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A9F5521F84D6; Fri, 13 Jul 2012 18:54:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.598
X-Spam-Level: 
X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hnS7VPsqUcmD; Fri, 13 Jul 2012 18:54:36 -0700 (PDT)
Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) by ietfa.amsl.com (Postfix) with ESMTP id 9F1AA21F8453; Fri, 13 Jul 2012 18:54:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=jakarthi@cisco.com; l=8165; q=dns/txt; s=iport; t=1342230913; x=1343440513; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=+ET8XZ0xej1ZGS8WFDlgcYPwloSRTpvynAzpxlmj0IU=; b=FsE0Wz8RWotLN/WocH+uR771Iqo1nMnD5x0dfGcmO7e8013Kk2Rv2FaX tUFNhxuIvlFFg6M/ty34L2gJH0i0ml3Tqbe+0LWzo1SHZyRXDKJC/GRIM /37jxH51dQgwiUw30UWC0tNnk8ypGvh4bDy1aL7IG9D2oo8M078j3Ucnm s=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av4EAH3QAFCtJV2d/2dsb2JhbABFgkq1WoEHgicSAV8HEgEIOz0lAgQOJ4drm3ugHJE+A5U6jiCBZoJf
X-IronPort-AV: E=Sophos;i="4.77,583,1336348800";  d="scan'208,217";a="101771342"
Received: from rcdn-core-6.cisco.com ([173.37.93.157]) by rcdn-iport-2.cisco.com with ESMTP; 14 Jul 2012 01:55:13 +0000
Received: from xhc-rcd-x02.cisco.com (xhc-rcd-x02.cisco.com [173.37.183.76]) by rcdn-core-6.cisco.com (8.14.5/8.14.5) with ESMTP id q6E1tCji014945 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sat, 14 Jul 2012 01:55:13 GMT
Received: from xmb-aln-x10.cisco.com ([169.254.5.146]) by xhc-rcd-x02.cisco.com ([173.37.183.76]) with mapi id 14.02.0298.004; Fri, 13 Jul 2012 20:55:12 -0500
From: "Jay Karthik (jakarthi)" <jakarthi@cisco.com>
To: "gregory.mirsky@ericsson.com" <gregory.mirsky@ericsson.com>
Thread-Topic: [bmwg] Last Call: <draft-ietf-bmwg-protection-meth-09.txt> (Methodology for benchmarking MPLS protection mechanisms) to Informational RFC
Thread-Index: AQHNYWO0kGwKRFph702BQ4vGfDLCsA==
Date: Sat, 14 Jul 2012 01:55:12 +0000
Message-ID: <CC264611.10247%jakarthi@cisco.com>
In-Reply-To: <CC25A814.F5BB%jakarthi@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.1.120420
x-originating-ip: [10.86.254.224]
x-tm-as-product-ver: SMEX-10.2.0.1135-7.000.1014-19036.004
x-tm-as-result: No--40.284900-8.000000-31
x-tm-as-user-approved-sender: No
x-tm-as-user-blocked-sender: No
Content-Type: multipart/alternative; boundary="_000_CC26461110247jakarthiciscocom_"
MIME-Version: 1.0
Cc: "mpls@ietf.org" <mpls@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Subject: Re: [bmwg] Last Call: <draft-ietf-bmwg-protection-meth-09.txt> (Methodology for benchmarking MPLS protection mechanisms) to Informational RFC
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Jul 2012 01:54:39 -0000

--_000_CC26461110247jakarthiciscocom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Greg,

First of all, apologies in getting back to you this late. Even though we in=
corporated your comments in the most recent version (-10, published on 5/29=
), a formal response to your comments should have been made sooner.

Please see inline marked (JK) for the specific response to your comments.

Thanks for your thorough review.

Best,
Jay

I'm concerned with the very title of the document, Methodology for benchmar=
king MPLS protection mechanisms, even though only MPLS-TE FRR being conside=
red while LSP end-to-end and segment protection implicitly being kept out o=
f the scope.

JK: Modified the title to be more explicit in the most recent version poste=
d.

I've found several textual inaccuracies related to both MPLS and BFD to mak=
e me wonder if the document was liasioned to MPLS WG.
List of acronyms is missing - PLR, OIR, LOS, AIS, etc.

JK: PLR is defined in Term RFC 6414 that we are referencing. OIR Is mention=
ed in Appendix B. Other acronyms such as AIS and LOS are added in rev 10.

Introduction. I think that for "planned link or node failure" MBB is more e=
fficient and useful than FRR. But MBB is not being mentioned in the documen=
t.

JK: We have used the term Revision which implies MBB

Introduction. "A correlated failure is the simultaneous occurrence of two o=
r more failures." Personally, as correlated events I consider events with c=
ause-effect relationship.
Introduction. Path restoration after FRR discussion does not appear logical=
, in the scope of benchmarking document.

JK: We have modified section 5.6

Document Scope. "Protection from Bi-directional Forwarding Detection (BFD) =
is outside the scope of this document." I frankly couldn't decode this sent=
ense.

JK: Addressed in rev 10

Document Scope. Several references to Path Restoration as Re-optimization -=
 doubt that it belongs in the document at all.

JK: Section 5.6 adds more color to this discussion

Sections 6.1.1 through 6.2.4 - what is relevance of listing numbers of labe=
ls in the stack?

JK: Convergence, performance may be dependent on the label stack depth.

Peerformance of control plane should be outside of the scope of benchmarkin=
g as it is end-to-end metrics, not explicitly of DUT.

JK: Scope modified

--_000_CC26461110247jakarthiciscocom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <2787D579A0B32A4EA450683550BC67E9@cisco.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; font-size: 14px; font-family: Calibri, sans-ser=
if; ">
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
Greg,</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
<br>
</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
First of all, apologies in getting back to you this late. Even though we in=
corporated your comments in the most recent version (-10, published on 5/29=
), a formal response to your comments should have been made sooner.&nbsp;</=
div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
<br>
</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
Please see inline marked (JK) for the specific response to your comments.</=
div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
<br>
</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
Thanks for your thorough review.</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
<br>
</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
Best,</div>
<div style=3D"font-size: 14px; font-family: Calibri, sans-serif; color: rgb=
(0, 0, 0); ">
Jay</div>
<span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); ">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div><font><font><br>
I'm concerned with the very title of the document, Methodology for benchmar=
king MPLS protection mechanisms, even though only MPLS-TE FRR being conside=
red while LSP end-to-end and segment protection implicitly being kept out o=
f the scope.<br>
</font></font></div>
</div>
</span></div>
</div>
</span>
<div style=3D"color: rgb(0, 0, 0); "><br>
</div>
<div style=3D"color: rgb(0, 0, 0); ">JK: Modified the title to be more expl=
icit in the most recent version posted.</div>
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div><font class=3D"Apple-style-span" color=3D"#ff0000"><br>
</font></div>
</div>
</span><span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); ">
<div>
<div>I've found several textual inaccuracies related to both MPLS and BFD t=
o make me wonder if the document was liasioned to MPLS WG.<br>
List of acronyms is missing - PLR, OIR, LOS, AIS, etc.<br>
<br>
<font color=3D"#FF0000" style=3D"color: rgb(0, 0, 0); ">JK: PLR is defined =
in Term RFC 6414 that we are referencing. OIR Is mentioned in Appendix B. O=
ther acronyms such as AIS and LOS are added in rev 10.<br>
</font><br>
Introduction. I think that for &quot;planned link or node failure&quot; MBB=
 is more efficient and useful than FRR. But MBB is not being mentioned in t=
he document.<br>
<br>
<font>JK: We have used the term Revision which implies MBB</font><br>
<br>
Introduction. &quot;A correlated failure is the simultaneous occurrence of =
two or more failures.&quot; Personally, as correlated events I consider eve=
nts with cause-effect relationship.<br>
<font color=3D"#FF0000" style=3D"color: rgb(0, 0, 0); "></font>Introduction=
. Path restoration after FRR discussion does not appear logical, in the sco=
pe of benchmarking document.<br>
<br>
JK: We have modified section 5.6<font><font class=3D"Apple-style-span" colo=
r=3D"#ff0000"><br>
</font></font><font style=3D"color: rgb(0, 0, 0); "><br>
Document Scope. &quot;Protection from Bi-directional Forwarding Detection (=
BFD) is outside the scope of this document.&quot; I frankly couldn't decode=
 this sentense.<br>
</font></div>
</div>
</span></div>
</div>
</span>
<div style=3D"color: rgb(0, 0, 0); "><br>
</div>
<div style=3D"color: rgb(0, 0, 0); ">JK: Addressed in rev 10</div>
<span id=3D"OLK_SRC_BODY_SECTION" style=3D"color: rgb(0, 0, 0); ">
<div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space; ">
<span id=3D"OLK_SRC_BODY_SECTION">
<div>
<div><font><font><font class=3D"Apple-style-span" color=3D"#ff0000"><br>
</font>Document Scope. Several references to Path Restoration as Re-optimiz=
ation - doubt that it belongs in the document at all.<br>
<br>
<font color=3D"#FF0000" style=3D"color: rgb(0, 0, 0); ">JK: Section 5.6 add=
s more color to this discussion<br>
</font><br>
Sections 6.1.1 through 6.2.4 - what is relevance of listing numbers of labe=
ls in the stack?<br>
<br>
<font color=3D"#FF0000" style=3D"color: rgb(0, 0, 0); ">JK: Convergence, pe=
rformance may be dependent on the label stack depth.
<br>
</font><br>
Peerformance of control plane should be outside of the scope of benchmarkin=
g as it is end-to-end metrics, not explicitly of DUT.<br>
<br>
<font color=3D"#FF0000" style=3D"color: rgb(0, 0, 0); ">JK: Scope modified<=
br>
</font></font></font></div>
</div>
</span></div>
</div>
</span>
</body>
</html>

--_000_CC26461110247jakarthiciscocom_--

From davids@iit.edu  Mon Jul 16 11:22:33 2012
Return-Path: <davids@iit.edu>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 84A4F11E8292 for <bmwg@ietfa.amsl.com>; Mon, 16 Jul 2012 11:22:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.185
X-Spam-Level: 
X-Spam-Status: No, score=-0.185 tagged_above=-999 required=5 tests=[BAYES_40=-0.185]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HM7ttolTRxY7 for <bmwg@ietfa.amsl.com>; Mon, 16 Jul 2012 11:22:32 -0700 (PDT)
Received: from spamblock-1.iit.edu (spamblock-1.iit.edu [216.47.143.127]) by ietfa.amsl.com (Postfix) with ESMTP id 6774F11E8293 for <bmwg@ietf.org>; Mon, 16 Jul 2012 11:22:32 -0700 (PDT)
X-ASG-Debug-ID: 1342462990-03d9b5362c4d9fb0001-1nqPoS
Received: from atlas-v-iv.iit.edu (atlas-p.iit.edu [216.47.143.155]) by spamblock-1.iit.edu with ESMTP id DZepqP7d6F4V9cg6 for <bmwg@ietf.org>; Mon, 16 Jul 2012 13:23:10 -0500 (CDT)
X-Barracuda-Envelope-From: davids@iit.edu
X-Barracuda-Apparent-Source-IP: 216.47.143.155
MIME-version: 1.0
Content-transfer-encoding: 7BIT
Content-type: text/plain; charset=US-ASCII
Received: from [192.168.0.197] (c-67-184-3-189.hsd1.il.comcast.net [67.184.3.189]) by atlas-v.iit.edu (Sun Java(tm) System Messaging Server 6.3-4.01 (built Aug 3 2007; 32bit)) with ESMTPA id <0M7900KM3N3WVS40@atlas-v.iit.edu> for bmwg@ietf.org; Mon, 16 Jul 2012 13:23:59 -0500 (CDT)
User-Agent: Microsoft-MacOutlook/14.2.3.120616
Date: Mon, 16 Jul 2012 13:23:06 -0500
From: Carol Davids <davids@iit.edu>
X-ASG-Orig-Subj: Re: SIP Benchmarking methodology and terminology - version 4 updates
To: "Aamer Akhter (aakhter)" <aakhter@cisco.com>
Message-id: <CC29C169.213ED%davids@iit.edu>
Thread-topic: SIP Benchmarking methodology and terminology - version 4 updates
In-reply-to: <7F298ACC76CC154F832B6D02852D169F078D5B4F@XMB-RCD-101.cisco.com>
X-Barracuda-Connect: atlas-p.iit.edu[216.47.143.155]
X-Barracuda-Start-Time: 1342462990
X-Barracuda-URL: http://216.47.143.127:8000/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at iit.edu
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=7.0 tests=BSF_SC0_MISMATCH_TO
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.102886 Rule breakdown below pts rule name              description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
Cc: bmwg@ietf.org
Subject: Re: [bmwg] SIP Benchmarking methodology and terminology - version 4 updates
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Jul 2012 18:22:33 -0000

Aamer,

Thank you very much for your discussion comments. We greatly appreciate
them.  Please see our responses below.


Comments and responses:
1. Can we please add a statement with regards to the relationship
between this work and rfc6076? (something more deeper than BMWG is in
the lab).

CD/VG: RFC6076 relates to end-to-end performance metrics. It includes
measurements of the percent of successful calls to total attempted calls,
for example. The current drafts relate to a steady-state measurement in
which no calls fail. We seek the maximum delivery rate that the system
can sustain without failure. Some of the metrics to be observed in
RFC6076 are ones that are set as input parameters to the tests defined in
our SIP Benchmarking Methodology draft. We added this information to the
new draft.

2. Have anybody checked to see if the metrics out of rfc6076 and this
work are comparable?

CD/VG: Yes, we did check. See 1) above.

3. "However, benchmarking SIP
presence is not a part of this work item.". Is there a reason for
this. Presence would be one of the more interesting cases to understand.

CD/VG: It could be done.  We thought the the application had too many
variables and
did not belong in a basic SIP benchmarking specification.  If the
Emulated Agent sends a SUBSCRIBE message to a presence server, the proxy
function within the DUT/SUT will forward that request to the presence
server, either statefully or statelessly. If the DUT/SUT contains the
presence server then of course it will work very hard. If it does not
contain the presence server then it will not work very hard. But if you
want to compare two different devices that both contain a presence server
there seemed to be too many variables.


To accurately benchmark presence requires a separate architecture,
processing elements, processing rules, and filtering rules as
required by the SIMPLE WG. Our aim in this particular work was
to get baseline SIP multimedia session setup benchmarked, with
a few simple extensions such as IM. Presence is far more
complex to be included in this work.

4. "Only session establishment is considered for the performance
benchmarks.  Session disconnect is not considered in the scope of
this work item." I would think that given connectivity churn
(links going up and down, power outages, etc), both establishment as
well as disconnect are interest items. Consider the cleanup work needed
after a disconnect... some of this might be a high workload than simply
maintaining a connection.

CD/VG: The cleanup work is taken into account in the measurement already.
We are
setting up and tearing down calls in order to find the rate at which we
can do so without any failures. So if we indicate that a call will last
for 5 seconds, that includes the fact that it will be disconnected after 5
seconds. The metric we are interested in is the session throughput. We
will add a sentence that explains this.

5. term-04, figure 12 seems misformatted, or it needs a better
explanation of how to read it...

CD/VG: Thanks. There appear to be a number of these. Our vertical lines
are not
lining up as intended. We will need to fix this.

6. For the case of video conferencing the SDP can get quite large. I am
wondering if this (SDP size) is an interesting axis to explore.

CD/VG: If the DUT/SUT processes the SDP it might be interesting. The size
of the
SDP will be affected by the choice of codec and the consequent number of
parameter lines. We could add the 'codec type' as a parameter of test.

Note that if the DUT is a user agent, then the benchmark
numbers derived from it already include the processing of SDP
since user agents (end points) must process the SDP to set up
sessions.


Best,

Carol








Carol Davids
Professor &  Director RTC Lab
Illinois Institute of Technology
Office: 630-682-6024
Mobile: 630-292-9417
Email: davids@iit.edu
Skype: caroldavids1
Web: rtc-lab.itm.iit.edu





>



From davids@iit.edu  Mon Jul 16 12:09:17 2012
Return-Path: <davids@iit.edu>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DFB211E82E8 for <bmwg@ietfa.amsl.com>; Mon, 16 Jul 2012 12:09:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.392
X-Spam-Level: 
X-Spam-Status: No, score=-1.392 tagged_above=-999 required=5 tests=[AWL=1.207,  BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QTo97qUfVE6S for <bmwg@ietfa.amsl.com>; Mon, 16 Jul 2012 12:09:16 -0700 (PDT)
Received: from spamblock-1.iit.edu (spamblock-1.iit.edu [216.47.143.127]) by ietfa.amsl.com (Postfix) with ESMTP id 1851811E82E4 for <bmwg@ietf.org>; Mon, 16 Jul 2012 12:09:16 -0700 (PDT)
X-ASG-Debug-ID: 1342465799-03d9b5362c4f0de0001-1nqPoS
Received: from atlas-v-iv.iit.edu (atlas-p.iit.edu [216.47.143.155]) by spamblock-1.iit.edu with ESMTP id RYMMyAP2CKRXHDB5; Mon, 16 Jul 2012 14:09:59 -0500 (CDT)
X-Barracuda-Envelope-From: davids@iit.edu
X-Barracuda-Apparent-Source-IP: 216.47.143.155
MIME-version: 1.0
Content-transfer-encoding: 7BIT
Content-type: text/plain; charset=US-ASCII
Received: from [192.168.0.197] (c-67-184-3-189.hsd1.il.comcast.net [67.184.3.189]) by atlas-v.iit.edu (Sun Java(tm) System Messaging Server 6.3-4.01 (built Aug 3 2007; 32bit)) with ESMTPA id <0M7900K19P9VVS50@atlas-v.iit.edu>; Mon, 16 Jul 2012 14:10:48 -0500 (CDT)
User-Agent: Microsoft-MacOutlook/14.2.3.120616
Date: Mon, 16 Jul 2012 14:09:54 -0500
From: Carol Davids <davids@iit.edu>
X-ASG-Orig-Subj: Re: [bmwg] RAI-ART review of draft-ietf-bmwg-sip-bench-term-04 and draft-ietf-bmwg-sip-bench-meth-04
To: "Worley, Dale R (Dale)" <dworley@avaya.com>, "rai@ietf.org" <rai@ietf.org>, Al Morton <acmorton@att.com>, "draft-ietf-bmwg-sip-bench-term@tools.ietf.org" <draft-ietf-bmwg-sip-bench-term@tools.ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Message-id: <CC29C656.21417%davids@iit.edu>
Thread-topic: [bmwg] RAI-ART review of draft-ietf-bmwg-sip-bench-term-04 and draft-ietf-bmwg-sip-bench-meth-04
In-reply-to: <CD5674C3CD99574EBA7432465FC13C1B22726A0A88@DC-US1MBEX4.global.avaya.com>
X-Barracuda-Connect: atlas-p.iit.edu[216.47.143.155]
X-Barracuda-Start-Time: 1342465799
X-Barracuda-URL: http://216.47.143.127:8000/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at iit.edu
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=7.0 tests=BSF_SC0_MISMATCH_TO
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.102890 Rule breakdown below pts rule name              description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
Cc: Mary Barnes <mary.ietf.barnes@gmail.com>
Subject: Re: [bmwg] RAI-ART review of draft-ietf-bmwg-sip-bench-term-04 and draft-ietf-bmwg-sip-bench-meth-04
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Jul 2012 19:09:17 -0000

Dale,

Thanks very much for your careful review of the drafts.  Please see
below for our responses and descriptions of what we have done to address
the issues. We highlight your comments under RAI-ART REVIEW COMMENT:, and
indicate our responses under
RESPONSE:CD/VG:

We also appreciate your pointing out the need for a final editorial review
and have begun that work using the detail you provided.  We plan to post
the edited version in time for IETF 85 in November.

Best,

Carol 


Carol Davids
Email: davids@iit.edu
Skype: caroldavids1
..................................................................

RAI-ART REVIEW COMMENT:
I. Technical issues
A. Media
The drafts seem to be undecided as to whether they want to benchmark
media processing or not.  At one point, it says that the benchmarking
is entirely in the signaling plane.  At another point, it is specified
how to set the number of media streams per INVITE dialog.  I suspect
the conflict is that there are devices the WG wants to benchmark which
do handle the media and their media handling affects performance
significantly, but the WG is unsure how to parametrize media
benchmarking.

RESPONSE: CD/VG: 
The media parameters need to be specified because, if
the DUT/SUT processes media, the media processing will impact the SIP
Performance, using cycles that could otherwise have been used for
SIP-related processing.  We do not measure the performance of the audio
and/or video.  But we need to document the conditions under which the test
was conducted if we are to be able to compare test results.


We are measuring the signaling-plane throughput, but thisthroughput will
vary depending upon the conditions of test.  One
important condition of test is media processing:  Is the DUT processing
media streams and if so, how many and what parameters define them? We note
the presence and character of the media streams being processed by the
device without measuring the quality of the resulting media.

We define the the media according to the packet size, type of codec, and
number of streams.  These parameters describe the conditions under which
the test is
performed.  But we do not measure
the quality or other characteristics of the resulting media.  We expect
that the more work we ask the
device under test to do, the lower will be its SIP throughput.



RAI-ART REVIEW COMMENT:


The simplest situation, of course, is if the DUT is a proxy and the
media bypasses it entirely.  Beyond that, even in the simplest media
relaying situations, significant processing costs can arise.  Even if
the DUT isn't transcoding, it may have to rewrite the RTP codec
numbers (as it may have modified the SDP passing through it).  In more
complex situations, the DUT may have to transcode the codecs.

Some consideration has to be made of how to specify exactly what the
media loads are.  SDP already has ways to specify the exact encoding
of media streams, including the number of samples per packet, which
implies the packet size and packet rate.  These drafts only allow
specifying the packet size as a benchmark parameter, but packet size
is notoriously ambiguous -- which layers of encapsulation of the
encoded media are counted?  But specifying the detailed encoding
side-steps that question.

RESPONSE:CD/VG: 
We will add a note that the testing organization may
choose to specify more characteristics of the associated media and may keep
records of comparative results.  Our goal
in this draft is to describe a method for measuring SIP throughput of the
device under test.  Organizations are
encouraged to identify boundary conditions that they deem important and to
perform tests under as many variants of these boundary conditions as they
wish.




RAI-ART REVIEW COMMENT:

The draft wants to consider the duration of a media stream to be
separately adjustable from the duration of the containing dialog, but
the draft explicitly places out of scope the re-INVITE which is
necessary to accomplish that realistically (that is, with the
signaling matching the presented media packets).


RESPONSE:CD/VG: 
The parameters of test are recorded in the test setup report.  The
session attempt rate and the total number of sessions to be attempted are
identified in this report.  These two numbers determine the total length
of the test.  The duration of a session is also identified in this report,
but the name we assigned that parameter is "Media session hold time."  We
will change the name of the parameter to "Session hold time." When the
INVITE-initiated session includes media, then the session hold time
represents the duration of the media session.  Whether or not we include
media, the session ends when the BYE message is received by the emulated
agent. We do not include the duration of a stream among the parameters of
test. We do allow there to be multiple media streams per session, but the
session is ended by a BYE.

Re-INVITEs are a different question.  They do not affect the duration of
the session.  But it is true that they consume processing cycles. We will
discuss this further and appreciate your identifying the issue.

We will also add a test parameter that describes the time between
successive call attempts by the emulated agent. We recommend setting this
parameter to 0 since setting it at a higher value will make the
testing-to-failure take longer.



RAI-ART REVIEW COMMENT:


B. INVITE dialog structure
The drafts seem to want to consider the establishment (or failure
thereof) of an INVITE dialog to be instantaneous, after which the
dialog continues for a chosen length of time, and then vanishes
instantly.  Little or no consideration is given to the various
scenarios of call establishment, including the most common case:
INVITE sent -- 183 response -- significant delay -- 200 response.
Dialog teardown is not conceptualized as being a processing step that
involves significant cost and may fail:  "Session disconnect is not
considered in the scope of this work item."

This lack of consideration is enhanced in the forking cases, as the
variety of scenarios (and their durations) increases.

In addition, the drafts only consider forking which is done within the
DUT, whereas it will be common in practice for forking to be done
downstream of the DUT, presenting the DUT with a stream of 1xx
responses from multiple endpoints, with a 2xx after an extended delay.

Also, in regard to signaling benchmarking, INVITEs that ultimately
fail are likely to be as costly as INVITEs that succeed, but there
doesn't seem to be a defined parameter "fraction of attempted calls
which succeed" (which controls the callee EAs).


RESPONSE:CD/VG: 

RE DELAY: 
The delay in responding with a 200 OK after getting an
INVITE is not specified in
the current methodology document and is assumed to be instantaneous. Our
intent is to stress the DUT as quickly as possible.  Introducing a delay
serves to increase the time before the DUT produces its first
stress-induced error. A high interval (> 32s) may cause the DUT to enter a
stable state and not be subject to stress. For this reason, we
intentionally chose not to introduce delays before issuing a 200 OK.

Note that many user agents automatically introduce delays by first sending
a 180-ringing, etc.  Any additional
artificial delays, while easy to introduce, would be an additional
tuning parameter that is subject to differing interpretations.


RE FORKING: 
Regarding the forking case being done in the DUT ---
this was a design
choice since we wanted to model the complexity and delay caused by
forking N branches downstream and collating the responses, etc. at the
DUT.  In other words, our assumption is that the DUT is a proxy (or a
B2BUA) and is doing forking and response collating.
The event that the tester arranges forking to happen downstream
of the DUT is automatically captured when the device acting as a
DUT is a user agent client and the next downstream SIP entity is forking
and presenting responses to the DUT.


RE THE COST OF FAILURES:
Regarding the
contention that "INVITEs that ultimately fail are likely
to be as costly as INVITEs that succeed, but there doesn't seem to be a
Defined parameter "fraction of attempted calls which succeed" (which
controls the callee EAs)" --- it seems to us that the Session
Establishment Performance benchmark (please see Section 5.2) covers
this.


RAI-ART REVIEW COMMENT:


C. Loop detection
All discussions of loop detection needs to based on the revised loop
detection requirements in RFC 5393.


RESPONSE:CD/VG: 
We will update the methodology document to ensure that loop
detection
is based on the revised loop detection requirements in rfc5393.  This
is a good catch! 



RAI-ART REVIEW COMMENT:

D. Authentication
In some SIP operations, authentication is commonly done.  This can
have various effects on the message flows that need to be taken into
account in the benchmarks.

For instance, a registrar may require that the registering UA
authenticate itself.  Commonly, the UA sends a REGISTER request, which
is rejected with 401 because it contains a nonce that is too old.  The
UA then immediately sends another REGISTER with the nonce provided in
the 401 response, and that request receives a 200 response.

In this scenario, the number of effective REGISTER requests is half of
the total REGISTER requests, leading to an apparent attempt failure
rate of 50%, even though the middlebox is doing the Right Thing 100%
of the time.  This suggests that the definition of "attempt failure"
needs to be updated so that a 4xx response "passed upstream" by the
DUT is not counted as an attempt failure.

In other scenarios, the DUT itself might be expected to enforce SIP
authentication, which would require a somewhat different definition of
attempt failure, and would be expected to have lower throughput.

So some thought needs to be given as to whether these scenarios are
to be benchmarked, and to document how authentication is to be handle
in whatever benchmarks are defined.

RESPONSE:CD/VG: 


We agree with your analysis, and in fact, the terminology document already
recognizes this through its scope (please see Section 2.1
of the terminology document).  It says,
    o  REGISTER and INVITE requests may be challenged or remain
      
unchallenged for authentication purpose as this may impact the
      
performance benchmarks.  Any observable performance degradation
      
due to authentication is of interest to the SIP community.
      
Whether or not the REGISTER and INVITE requests are challenged is
      
a condition of test and will be recorded and reported.
However, in the methodology document, we do not include any further
guidance for this.  As you point out, there is a need for some guidance.
To remedy this, we propose that we add the following in Section 5.1 of
The methodology document:

    Authentication
option = ___________________________________
    (on|off;
if on, please see Note-2 below).
    Number
of responses of the following type:
      
401:       _____________  (if
authentication turned on; N/A
                                  otherwise)
      
407:       _____________  (if
authentication turned on; N/A
                                  otherwise)
      
2xx-class: _____________
      
1xx-class: _____________
      
Others:    _____________
This information will provide the tester to analyze how many 401/407
were received and adjust the metrics in Section 5.2 and 5.3 accordingly.











>



From internet-drafts@ietf.org  Mon Jul 16 14:58:27 2012
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF6E211E8279; Mon, 16 Jul 2012 14:58:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.506
X-Spam-Level: 
X-Spam-Status: No, score=-102.506 tagged_above=-999 required=5 tests=[AWL=0.093, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ui4s0LsrpsOW; Mon, 16 Jul 2012 14:58:27 -0700 (PDT)
Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6591611E8270; Mon, 16 Jul 2012 14:58:27 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: internet-drafts@ietf.org
To: i-d-announce@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 4.30p3
Message-ID: <20120716215827.23714.53692.idtracker@ietfa.amsl.com>
Date: Mon, 16 Jul 2012 14:58:27 -0700
Cc: bmwg@ietf.org
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Jul 2012 21:58:28 -0000

A New Internet-Draft is available from the on-line Internet-Drafts director=
ies.
 This draft is a work item of the Benchmarking Methodology Working Group of=
 the IETF.

	Title           : Benchmarking Methodology for Content-Aware Network Devic=
es
	Author(s)       : Mike Hamilton
                          Sarah Banks
	Filename        : draft-ietf-bmwg-ca-bench-meth-02.txt
	Pages           : 19
	Date            : 2012-07-16

Abstract:
   This document defines a set of test scenarios and metrics that can be
   used to benchmark content-aware network devices.  The scenarios in
   the following document are intended to more accurately predict the
   performance of these devices when subjected to dynamic traffic
   patterns.  This document will operate within the constraints of the
   Benchmarking Working Group charter, namely black box characterization
   in a laboratory environment.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ca-bench-meth

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-bmwg-ca-bench-meth-02

A diff from previous version is available at:
http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-bmwg-ca-bench-meth-02


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From acmorton@att.com  Tue Jul 17 06:46:21 2012
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B948721F86E3 for <bmwg@ietfa.amsl.com>; Tue, 17 Jul 2012 06:46:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -105.696
X-Spam-Level: 
X-Spam-Status: No, score=-105.696 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, MSGID_FROM_MTA_HEADER=0.803, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xmljmrlCNqzn for <bmwg@ietfa.amsl.com>; Tue, 17 Jul 2012 06:46:21 -0700 (PDT)
Received: from nbfkord-smmo04.seg.att.com (nbfkord-smmo04.seg.att.com [209.65.160.86]) by ietfa.amsl.com (Postfix) with ESMTP id C773621F86D9 for <bmwg@ietf.org>; Tue, 17 Jul 2012 06:46:20 -0700 (PDT)
Received: from unknown [144.160.128.153] (EHLO flpi408.enaf.ffdc.sbc.com) by nbfkord-smmo04.seg.att.com(mxl_mta-6.11.0-10) over TLS secured channel with ESMTP id cdc65005.0.489912.00-445.1339260.nbfkord-smmo04.seg.att.com (envelope-from <acmorton@att.com>);  Tue, 17 Jul 2012 13:47:08 +0000 (UTC)
X-MXL-Hash: 50056cdc2f96be3d-33ca93fa51c4403ac02d93fd7ca536aad32e27b0
Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flpi408.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id q6HDl7Zj023879 for <bmwg@ietf.org>; Tue, 17 Jul 2012 06:47:07 -0700
Received: from fflint04.pst.cso.att.com (fflint04.pst.cso.att.com [150.234.39.64]) by flpi408.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id q6HDkpf8023547 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <bmwg@ietf.org>; Tue, 17 Jul 2012 06:47:01 -0700
Received: from alpd052.aldc.att.com (alpd052.aldc.att.com [130.8.42.31]) by fflint04.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Tue, 17 Jul 2012 06:46:23 -0700
Received: from aldc.att.com (localhost.localdomain [127.0.0.1]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q6HDkMPK014076 for <bmwg@ietf.org>; Tue, 17 Jul 2012 09:46:22 -0400
Received: from dns.maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q6HDkGm5013845 for <bmwg@ietf.org>; Tue, 17 Jul 2012 09:46:16 -0400
Message-Id: <201207171346.q6HDkGm5013845@alpd052.aldc.att.com>
Received: from acmt.att.com (vpn-135-70-203-35.vpn.east.att.com[135.70.203.35](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20120717134155gw100n982ae>; Tue, 17 Jul 2012 13:41:55 +0000
X-Originating-IP: [135.70.203.35]
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Tue, 17 Jul 2012 09:47:37 -0400
To: bmwg@ietf.org
From: Al Morton <acmorton@att.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)]
X-MAIL-FROM: <acmorton@att.com>
X-SOURCE-IP: [144.160.128.153]
X-AnalysisOut: [v=1.0 c=1 a=nmcWtPEZ_3EA:10 a=ofMgfj31e3cA:10 a=BLceEmwcHo]
X-AnalysisOut: [wA:10 a=kj9zAlcOel0A:10 a=xwOvzTHDVLE4u4nGvK72ag==:17 a=48]
X-AnalysisOut: [vgC7mUAAAA:8 a=tuzqY95TdnpSGJkzvxAA:9 a=CjuIK1q_8ugA:10 a=]
X-AnalysisOut: [Qv5F4YHriPYA:10]
Subject: [bmwg] Meeting Prep and Draft Agenda
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 17 Jul 2012 13:46:21 -0000

BMWG,

There are a changes to our meeting preparation this time:

- meeting time is earned in a sense, by e-mail discussion

We have discussed many of our existing drafts, either on bmwg-list
or other IETF list (like the IESG-list), and those that
are eligible are on the draft agenda.

- meeting time is shorter than usual

When there was some difficulty scheduling our session to avoid
conflicts, I offered to reduce our time request to one hour.
We haven't used the full two hour slot at the last two IETFs
(for various reasons, such as last-minute presenter unavailability),
so let's help our ADs at IETF-84.

I also have no information on proposed work at this time,
but it's possible I overlooked something in the flood of
drafts over the last 2 weeks.  Let us all know on-list if I
omitted something.

- the meeting materials page is not yet available

I can submit, but you can't see it, so I've appended the draft
agenda below. Comments are welcome.

Note that remote access will be according to:
http://www.ietf.org/meeting/84/remote-participation.html

regards,
Al
bmwg chair

=========================================================

Benchmarking Methodology WG (BMWG)
Thursday August 2, 2012
1730-1830  Afternoon Session III
Regency A       	OPS 	bmwg

Remote Participation:
http://www.ietf.org/meeting/84/remote-participation.html

0. Agenda Bashing

1. WG Status and Milestones

Published (?):
IP Flow Information Accounting and Export Benchmarking Methodology


Drafts not presented at this meeting:

1B. IETF Last Call comments on: <draft-ietf-bmwg-protection-meth-09.txt>
     from Joel M. Halpern and Gregory Mirsky
Status?  Waiting for AD go-ahead...

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2. SIP performance benchmarking terms and methods
    Presenter: Vijay Gurbani
http://tools.ietf.org/html/draft-ietf-bmwg-sip-bench-meth-04
http://tools.ietf.org/html/draft-ietf-bmwg-sip-bench-term-04
WGLC comments, including Expert review. Revised drafts needed.


3. Benchmarking Methodology for Content-Aware Network Devices
    Presenter: Mike Hamilton
There are some updates in the draft in response to ???
Some comments were not yet addressed in the previous version due to
time constraints.

4. IMIX Genome
    Presenter: Al
    Comments on methods to determine the mix (Aamer) were addressed

5. RFC 2544 Applicability Statement:
    Presenter: Al
    IETF Last Call Comments, and discussion for resolution (if not 
yet achieved)

6. Basic BGP Convergence Benchmarking Methodology status
    Presenter:
draft-papneja-bgp-basic-dp-convergence
draft-varlashkin-router-conv-bench
Status: Call for review and adoption...


7. Milestone Review

8. New Work Proposal Review/Matrix

LAST. AOB




From talexander@ixiacom.com  Mon Jul 23 18:39:55 2012
Return-Path: <talexander@ixiacom.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A191C11E80F3 for <bmwg@ietfa.amsl.com>; Mon, 23 Jul 2012 18:39:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.999
X-Spam-Level: 
X-Spam-Status: No, score=-5.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_33=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BLBeIl16WJTN for <bmwg@ietfa.amsl.com>; Mon, 23 Jul 2012 18:39:54 -0700 (PDT)
Received: from tx2outboundpool.messaging.microsoft.com (tx2ehsobe001.messaging.microsoft.com [65.55.88.11]) by ietfa.amsl.com (Postfix) with ESMTP id 8231F11E80E2 for <bmwg@ietf.org>; Mon, 23 Jul 2012 18:39:54 -0700 (PDT)
Received: from mail270-tx2-R.bigfish.com (10.9.14.249) by TX2EHSOBE013.bigfish.com (10.9.40.33) with Microsoft SMTP Server id 14.1.225.23; Tue, 24 Jul 2012 01:39:53 +0000
Received: from mail270-tx2 (localhost [127.0.0.1])	by mail270-tx2-R.bigfish.com (Postfix) with ESMTP id 7DE951200315; Tue, 24 Jul 2012 01:39:53 +0000 (UTC)
X-Forefront-Antispam-Report: CIP:157.56.244.149; KIP:(null); UIP:(null); IPV:NLI; H:CH1PRD0611HT002.namprd06.prod.outlook.com; RD:none; EFVD:NLI
X-SpamScore: 0
X-BigFish: PS0(zzc85fhzz1202hzzz2fh2a8h668h839hd25hf0ah107ah34h)
Received-SPF: pass (mail270-tx2: domain of ixiacom.com designates 157.56.244.149 as permitted sender) client-ip=157.56.244.149; envelope-from=talexander@ixiacom.com; helo=CH1PRD0611HT002.namprd06.prod.outlook.com ; .outlook.com ; 
Received: from mail270-tx2 (localhost.localdomain [127.0.0.1]) by mail270-tx2 (MessageSwitch) id 1343093991432585_14404; Tue, 24 Jul 2012 01:39:51 +0000 (UTC)
Received: from TX2EHSMHS036.bigfish.com (unknown [10.9.14.247])	by mail270-tx2.bigfish.com (Postfix) with ESMTP id 67C3B194004B; Tue, 24 Jul 2012 01:39:51 +0000 (UTC)
Received: from CH1PRD0611HT002.namprd06.prod.outlook.com (157.56.244.149) by TX2EHSMHS036.bigfish.com (10.9.99.136) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 24 Jul 2012 01:39:51 +0000
Received: from CH1PRD0611MB420.namprd06.prod.outlook.com ([169.254.8.229]) by CH1PRD0611HT002.namprd06.prod.outlook.com ([10.255.148.165]) with mapi id 14.16.0175.005; Tue, 24 Jul 2012 01:39:50 +0000
From: Tom Alexander <talexander@ixiacom.com>
To: Mike Hamilton <mhamilton@breakingpoint.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
Thread-Index: AQHNAIhwQ06zgXVRJ0GGJq/Ccz5bAZZnE0EAgAA3vICA0S128A==
Date: Tue, 24 Jul 2012 01:39:49 +0000
Message-ID: <1113A19AC2AB4C4FB562BDA89BAD16061DD66430@CH1PRD0611MB420.namprd06.prod.outlook.com>
References: <CB83BC6B.ADAF%mhamilton@breakingpoint.com> <73D3A52E-2CE5-4991-A84B-408F0334444B@nobulus.com>
In-Reply-To: <73D3A52E-2CE5-4991-A84B-408F0334444B@nobulus.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-originating-ip: [69.54.136.121]
Content-Type: multipart/mixed; boundary="_002_1113A19AC2AB4C4FB562BDA89BAD16061DD66430CH1PRD0611MB420_"
MIME-Version: 1.0
X-MS-Exchange-CrossPremises-AuthAs: Internal
X-MS-Exchange-CrossPremises-AuthMechanism: 04
X-MS-Exchange-CrossPremises-AuthSource: CH1PRD0611HT002.namprd06.prod.outlook.com
X-MS-Exchange-CrossPremises-SCL: -1
X-MS-Exchange-CrossPremises-messagesource: StoreDriver
X-MS-Exchange-CrossPremises-BCC: 
X-MS-Exchange-CrossPremises-rules-execution-history: Support - Juniper
X-MS-Exchange-CrossPremises-processed-by-journaling: Journal Agent
X-MS-Exchange-CrossPremises-ContentConversionOptions: False; 00160000; True; ; iso-8859-1
X-OrganizationHeadersPreserved: CH1PRD0611HT002.namprd06.prod.outlook.com
X-OriginatorOrg: ixiacom.com
X-Mailman-Approved-At: Mon, 23 Jul 2012 19:32:54 -0700
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Jul 2012 01:41:04 -0000

--_002_1113A19AC2AB4C4FB562BDA89BAD16061DD66430CH1PRD0611MB420_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hello, Mike,

I've read the latest posted ca-bench-meth-02 draft (finally! sorry for the =
delay) and I appreciate your incorporating my feedback on your previous ver=
sion. It looks quite good. Enclosed are some remaining comments on the draf=
t.

I assume the terminology draft will be forthcoming?

Best regards,

- Tom Alexander


--_002_1113A19AC2AB4C4FB562BDA89BAD16061DD66430CH1PRD0611MB420_
Content-Type: text/plain; name="comments on ca-bench-meth-02.txt"
Content-Description: comments on ca-bench-meth-02.txt
Content-Disposition: attachment;
	filename="comments on ca-bench-meth-02.txt"; size=3757;
	creation-date="Tue, 24 Jul 2012 01:39:07 GMT";
	modification-date="Tue, 24 Jul 2012 01:35:52 GMT"
Content-Transfer-Encoding: base64

Q29tbWVudHMgb24gZHJhZnQtaWV0Zi1ibXdnLWNhLWJlbmNoLW1ldGgtMDIudHh0DQotLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KMS4gSW4gU2VjdGlv
biAzLCAiVGVzdCBTZXR1cCIsIEkgcmVhbGl6ZSB0aGF0IG90aGVyIGNvbW1lbnRlcnMNCmhhdmUg
YWxyZWFkeSBjb21tZW50ZWQgb24gdGhpcyBwYXJ0aWN1bGFyIHRvcGljLiBIb3dldmVyLCB0aGUN
Cmxhbmd1YWdlIG9mIHRoZSBkcmFmdCBjdXJyZW50bHkgYWxsb3dzIHBlb3BsZSB0byBwdWJsaXNo
IHRoZWlyDQp0ZXN0IHJlc3VsdHMgd2l0aG91dCBhbnkgZGVzY3JpcHRpb24gb2YgaG93IHRoZSBE
VVQgd2FzIGNvbmZpZ3VyZWQsDQp3aGljaCBkb2Vzbid0IGV2ZW4gYWxsb3cgcGVvcGxlIHRvIHVu
ZGVyc3RhbmQgdGhlIHRlc3Qgc2V0dXAsDQpsZXQgYWxvbmUgcmVjcmVhdGUgdGhlIHRlc3QuIFNv
LCBhcyBhIGNvbXByb21pc2UsIEkgc3VnZ2VzdCBjaGFuZ2luZw0KdGhlIGxhc3QgdHdvIHNlbnRl
bmNlcyBvZiB0aGlzIHNlY3Rpb24gdG8gcmVhZDoNCg0KIkEgc3VtbWFyeSBvZiB0aGUgRFVUIGNv
bmZpZ3VyYXRpb24gTVVTVCBiZSBwdWJsaXNoZWQgd2l0aCB0aGUNCmZpbmFsIGJlbmNobWFya2lu
ZyByZXN1bHRzLiBJbiBvcmRlciB0byBpbXByb3ZlIHJlcGVhdGFiaWxpdHksDQp0aGUgcHVibGlz
aGVkIGNvbmZpZ3VyYXRpb24gaW5mb3JtYXRpb24gU0hPVUxEIGluY2x1ZGUgY29tbWFuZC0NCmxp
bmUgc2NyaXB0cyB1c2VkIHRvIGNvbmZpZ3VyZSB0aGUgRFVULCBpZiBhbnksIGFuZCBTSE9VTEQg
YWxzbw0KaW5jbHVkZSBhbnkgY29uZmlndXJhdGlvbiBpbmZvcm1hdGlvbiBmb3IgdGhlIHRlc3Qg
ZXF1aXBtZW50IHVzZWQuIg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCjIuIEluIFNlY3Rpb24gMy4zLCBjYW4g
d2UgYXQgbGVhc3QgcmVxdWlyZSBwZW9wbGUgdG8gcmVwb3J0IHRoZQ0Kc3BlY2lmaWMgdHJhZmZp
YyBtaXggdGhhdCB0aGV5IGhhdmUgZWxlY3RlZCB0byB1c2U/IFNvbWV0aGluZw0KbGlrZTogIklu
IGFsbCBjYXNlcywgdXNlcnMgTVVTVCByZXBvcnQgdGhlIHRyYWZmaWMgbWl4IHVzZWQgaW4NCnRo
ZSB0ZXN0LCBhbmQgU0hPVUxEIHJlcG9ydCB0aGlzIHVzaW5nIGEgdGVtcGxhdGUgc2ltaWxhciB0
bw0KdGhhdCBpbiBBcHBlbmRpeCBBLiINCg0KQnkgdGhlIHdheSwgSSBiZWxpZXZlIFNlY3Rpb24g
MS4xIG9mIFJGQyAyNzI0IGlzIGEgbG90IHNpbXBsZXINCmFuZCBjbGVhcmVyIGRlZmluaXRpb24g
b2YgYSAiZmxvdyIgdGhhbiBzZWN0aW9uIDIuMSBvZiBSRkMgMjcyMi4NClBlcmhhcHMgeW91IG1p
Z2h0IGNvbnNpZGVyIGNoYW5naW5nIHRoZSByZWZlcmVuY2U/DQoNCi0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KMy4g
SW4gU2VjdGlvbiAzLjU6DQoNCkkgc3VnZ2VzdCByZXBsYWNpbmcgIkZsb3cgU2l6ZSBpbiBCaXRz
IiB3aXRoICJEYXRhIEV4Y2hhbmdlZA0KQnkgRmxvdywgQml0cyIuDQoNClRoZSB0ZXJtICJBZ2dy
ZWdhdGUgRmxvd3MiIGRvZXNuJ3Qgc2VlbSB0byBiZSBkZWZpbmVkIGFueXdoZXJlIC0NCndoYXQg
aXMgYW4gIkFnZ3JlZ2F0ZSBGbG93IiBhbmQgd2hhdCBpcyBpdCBhIHBlcmNlbnRhZ2Ugb2Y/IEkN
CnN1c3BlY3QgdGhhdCBpdCdzIGFjdHVhbGx5IHRoZSBudW1iZXIgb2YgdGhlc2UgcGFydGljdWxh
cg0KYXBwbGljYXRpb24gZmxvd3MgaW4gdGhlIG1peCBkaXZpZGVkIGJ5IHRoZSB0b3RhbCBudW1i
ZXIgb2YNCmFwcGxpY2F0aW9uIGZsb3dzLCBleHByZXNzZWQgYXMgYSBwZXJjZW50YWdlLiBIb3dl
dmVyLCBJJ20gbm90DQpzdXJlIHRoaXMgaXMgd2hhdCB3YXMgaW50ZW5kZWQgZWl0aGVyLiBBbiBl
eHBsYW5hdG9yeSBub3RlIHdvdWxkDQpiZSB2ZXJ5IGdvb2QuDQoNCkFsc28sIEkgc3VnZ2VzdCBs
ZWF2aW5nIG91dCB3aGF0IGFwcGVhcnMgdG8gYmUgcmVxdWlyZWQgdmFsdWVzIGZvcg0KdGhlIHZh
cmlvdXMgZW50cmllcyBpbiB0aGUgdGFibGUsIGFzIHRoZXkgYXJlIHNvbWV3aGF0IGNvbmZ1c2lu
Zy4NClRodXMgaW5zdGVhZCBvZjoNCg0KICAgbyAgRmxvdyBTaXplIGluIEJpdHMNCiAgIG8gIFBl
cmNlbnRhZ2Ugb2YgQWdncmVnYXRlIEZsb3dzOiAyNSUNCiAgIG8gIFRyYW5zcG9ydCBQcm90b2Nv
bChzKTogVENQLFVEUA0KICAgbyAgRGVzdGluYXRpb24gUG9ydChzKTogODANCg0KeW91IG1pZ2h0
IGNvbnNpZGVyIGhhdmluZzoNCg0KICAgbyAgRGF0YSBFeGNoYW5nZWQgQnkgRmxvdywgQml0cw0K
ICAgbyAgUGVyY2VudGFnZSBvZiBBZ2dyZWdhdGUgRmxvd3MNCiAgIG8gIFRyYW5zcG9ydCBQcm90
b2NvbChzKQ0KICAgbyAgRGVzdGluYXRpb24gVENQL1VEUCBQb3J0KHMpDQoNCi0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0N
Cg0KNC4gSW4gU2VjdGlvbiAzLjYsIEkgcmVjb21tZW5kIGNoYW5naW5nIHRoZSBsYXN0IHNlbnRl
bmNlIHRvICJGbG93DQppbml0aWF0aW9uIFNIT1VMRCBiZSBpbiBhIHBzZXVkby1yYW5kb20gbWFu
bmVyIGFjcm9zcyBpbmdyZXNzIHBvcnRzLiINCmFzIHBlciB5b3VyIHJlc3BvbnNlIHRvIG15IGxh
c3Qgc2V0IG9mIGNvbW1lbnRzLg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCjUuIFdhcyB0aGVyZSBhbnkgaW50
ZW50IG9mIHNwZWNpZnlpbmcgbW9yZSBwYXJhbWV0ZXJzIHRoYW4NCiJBcHBsaWNhdGlvbi1MYXll
ciBQYXJhbWV0ZXJzIiBpbiBTZWN0aW9uIDQuMS4yPyBJZiBub3QsIHRoZW4gSQ0Kc3VnZ2VzdCB0
aGF0IFNlY3Rpb24gNC4xLjIuMSBiZSBkZWxldGVkLCBhbmQgU2VjdGlvbiA0LjEuMiBiZQ0KY2hh
bmdlZCB0byByZWFkOg0KDQoiNC4xLjIgU2V0dXAgUGFyYW1ldGVycw0KDQpUaGUgcGFyYW1ldGVy
cyBvZiBlYWNoIGFwcGxpY2F0aW9uIHByb3RvY29sIGluIHVzZSBkdXJpbmcgdGhlIHRlc3QNCnJ1
biBTSE9VTEQgYmUgcmVwb3J0ZWQgZm9yIGFsbCB0ZXN0cy4gUmVwb3J0aW5nIFNIT1VMRCBmb2xs
b3cgdGhlDQp0YWJsZSBwcm92aWRlZCBpbiBTZWN0aW9uIDMuNS4iDQoNCkFsc28sIFNlY3Rpb24g
NC4yLjIuMSBzaG91bGQgYmUgZGVsZXRlZCBhbmQgU2VjdGlvbiA0LjIuMiBzaG91bGQgYmUNCmNo
YW5nZWQgdG8gcmVhZDoNCg0KIjQuMi4yIFNldHVwIFBhcmFtZXRlcnMNCg0KVGhlIHNhbWUgcGFy
YW1ldGVyIHJlcG9ydGluZyBwcm9jZWR1cmUgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC4xLjINClNI
T1VMRCBiZSB1c2VkIGZvciBhbGwgdGVzdHMuIg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCjYuIFNlY3Rpb24g
NC4zIGlzIE1VQ0ggYmV0dGVyIG5vdyEgU28gaXMgQXBwZW5kaXggQS4NCg0KLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0K
DQo3LiBJbiBBcHBlbmRpeCBBLCBjb3VsZCB5b3UgcHJvdmlkZSBhIHJlZmVyZW5jZSAoaW4gdGhl
IHJlZmVyZW5jZXMNCnNlY3Rpb24gb2YgdGhlIGRyYWZ0KSBmb3IgdGhpcyBTYW5kdmluZSByZXNl
YXJjaD8gQWxzbywgaXMgdGhpcw0KcmVmZXJlbmNlIHB1YmxpY2x5IGF2YWlsYWJsZT8NCg==

--_002_1113A19AC2AB4C4FB562BDA89BAD16061DD66430CH1PRD0611MB420_--

From gregory.mirsky@ericsson.com  Wed Jul 25 16:20:02 2012
Return-Path: <gregory.mirsky@ericsson.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6819321F849C; Wed, 25 Jul 2012 16:20:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.598
X-Spam-Level: 
X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IQYSwEDuswvN; Wed, 25 Jul 2012 16:20:01 -0700 (PDT)
Received: from imr4.ericy.com (imr4.ericy.com [198.24.6.9]) by ietfa.amsl.com (Postfix) with ESMTP id 1B8B621F849B; Wed, 25 Jul 2012 16:20:01 -0700 (PDT)
Received: from eusaamw0706.eamcs.ericsson.se ([147.117.20.31]) by imr4.ericy.com (8.14.3/8.14.3/Debian-9.1ubuntu1) with ESMTP id q6PNJwF7010762; Wed, 25 Jul 2012 18:20:00 -0500
Received: from EUSAACMS0715.eamcs.ericsson.se ([169.254.1.13]) by eusaamw0706.eamcs.ericsson.se ([147.117.20.31]) with mapi; Wed, 25 Jul 2012 19:19:55 -0400
From: Gregory Mirsky <gregory.mirsky@ericsson.com>
To: "Jay Karthik (jakarthi)" <jakarthi@cisco.com>
Date: Wed, 25 Jul 2012 19:19:54 -0400
Thread-Topic: [bmwg] Last Call: <draft-ietf-bmwg-protection-meth-09.txt> (Methodology for benchmarking MPLS protection mechanisms) to Informational RFC
Thread-Index: AQHNYWO0kGwKRFph702BQ4vGfDLCsJct7k9w
Message-ID: <FE60A4E52763E84B935532D7D9294FF13924C82A1F@EUSAACMS0715.eamcs.ericsson.se>
References: <CC25A814.F5BB%jakarthi@cisco.com> <CC264611.10247%jakarthi@cisco.com>
In-Reply-To: <CC264611.10247%jakarthi@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_FE60A4E52763E84B935532D7D9294FF13924C82A1FEUSAACMS0715e_"
MIME-Version: 1.0
Cc: "mpls@ietf.org" <mpls@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Subject: Re: [bmwg] Last Call: <draft-ietf-bmwg-protection-meth-09.txt> (Methodology for benchmarking MPLS protection mechanisms) to Informational RFC
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Jul 2012 23:20:02 -0000

--_000_FE60A4E52763E84B935532D7D9294FF13924C82A1FEUSAACMS0715e_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Jay, et al.,
thank you for your careful consideration of my comments. Couple more notes =
on -10 version follow:

 *
"MPLS-FRR" might be interpretted as both MPLS-TE-FRR and LDP-FRR. I would c=
onsider using in the document MPLS-TE-FRR or RSVP-TE-FRR.
 *
Section 5.2 I think that "Ethernet based links do not have layer 2 failure =
indicators, and therefore relies on layer 3 signaling for failure detection=
" is not entirely accurate. I'd point to IEEE 802.1ag/CFM and Y.1731 that c=
an be used to monitor continuity among MEPs of specified MA/MEG. Thus use o=
f Layer 3 is not required but optional.
 *
Section 5.2 In reference to BFD "These methods can be used for the layer 3 =
failure indicators ..." I'd note that BFD can be used in ACH, i.e. non-IP, =
encapsulation to monitor LSP, Segment or Section.
 *
Section 5.6 Restoration and Reversion. Restoration in MPLS-TE FRR network r=
elies on the control plane signalling. As I interpret Section 2. Document S=
cope, the document concentrates on defining benchmarking test cases to meas=
ure performance of facility backup FRR as defined in RFC 4090. I think that=
 Restoration and Reversion are out of that scope. If that is the case, then=
 statement "It is important to include Restoration and Reversion as a step =
in each test case to measure the amount of packet loss, out of order packet=
s, or duplicate packets that occurs in this process" is confusing.
 *
Section 6.1. I find most test cases very confusing. As stated in the Sectio=
n 2, the scope of this document is on facility backup method of MPLS-TE FRR=
. Thus backup tunnel is bypass backup tunnel that is encapsulating protecte=
d LSP(s).
 *
Section 6.1 and 6.2 I don't see enough rational for so many test cases. IMO=
, PLR performance in protection switchover is not affected by topology of a=
 bypass tunnel. And I'm concerned that failure detection has not been separ=
ated from protection switchover itself and will make benchmark testing resu=
lts, e.g. larger BFD interval being used larger variation in detection time=
 will be in the tests. Hence, after a switchover protected LSP would have a=
n extra label on its stack.
 *
Section 7.1 Benchmarking MPLS Performance seems outside of scope for this d=
ocument. Can it be spawn into a separate document and then referenced?

    Regards,
        Greg

________________________________
From: Jay Karthik (jakarthi) [mailto:jakarthi@cisco.com]
Sent: Friday, July 13, 2012 6:55 PM
To: Gregory Mirsky
Cc: bmwg@ietf.org; mpls@ietf.org
Subject: Re: [bmwg] Last Call: <draft-ietf-bmwg-protection-meth-09.txt> (Me=
thodology for benchmarking MPLS protection mechanisms) to Informational RFC

Greg,

First of all, apologies in getting back to you this late. Even though we in=
corporated your comments in the most recent version (-10, published on 5/29=
), a formal response to your comments should have been made sooner.

Please see inline marked (JK) for the specific response to your comments.

Thanks for your thorough review.

Best,
Jay

I'm concerned with the very title of the document, Methodology for benchmar=
king MPLS protection mechanisms, even though only MPLS-TE FRR being conside=
red while LSP end-to-end and segment protection implicitly being kept out o=
f the scope.

JK: Modified the title to be more explicit in the most recent version poste=
d.

I've found several textual inaccuracies related to both MPLS and BFD to mak=
e me wonder if the document was liasioned to MPLS WG.
List of acronyms is missing - PLR, OIR, LOS, AIS, etc.

JK: PLR is defined in Term RFC 6414 that we are referencing. OIR Is mention=
ed in Appendix B. Other acronyms such as AIS and LOS are added in rev 10.

Introduction. I think that for "planned link or node failure" MBB is more e=
fficient and useful than FRR. But MBB is not being mentioned in the documen=
t.

JK: We have used the term Revision which implies MBB

Introduction. "A correlated failure is the simultaneous occurrence of two o=
r more failures." Personally, as correlated events I consider events with c=
ause-effect relationship.
Introduction. Path restoration after FRR discussion does not appear logical=
, in the scope of benchmarking document.

JK: We have modified section 5.6

Document Scope. "Protection from Bi-directional Forwarding Detection (BFD) =
is outside the scope of this document." I frankly couldn't decode this sent=
ense.

JK: Addressed in rev 10

Document Scope. Several references to Path Restoration as Re-optimization -=
 doubt that it belongs in the document at all.

JK: Section 5.6 adds more color to this discussion

Sections 6.1.1 through 6.2.4 - what is relevance of listing numbers of labe=
ls in the stack?

JK: Convergence, performance may be dependent on the label stack depth.

Peerformance of control plane should be outside of the scope of benchmarkin=
g as it is end-to-end metrics, not explicitly of DUT.

JK: Scope modified

--_000_FE60A4E52763E84B935532D7D9294FF13924C82A1FEUSAACMS0715e_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dus-ascii">
<META content=3D"MSHTML 6.00.6002.18616" name=3DGENERATOR></HEAD>
<BODY=20
style=3D"FONT-SIZE: 14px; FONT-FAMILY: Calibri, sans-serif; WORD-WRAP: brea=
k-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<DIV dir=3Dltr align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DA=
rial=20
size=3D2>Hi Jay, et al.,</FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DA=
rial=20
size=3D2>thank you for your careful consideration of my comments. Couple mo=
re=20
notes on -10 version follow:</FONT></SPAN></DIV>
<UL dir=3Dltr>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial=20
  size=3D2>"MPLS-FRR" might be interpretted as both MPLS-TE-FRR and LDP-FRR=
. I=20
  would consider using in the document MPLS-TE-FRR or=20
  RSVP-TE-FRR.</FONT></SPAN></DIV>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial siz=
e=3D2>Section=20
  5.2 I think that "Ethernet based links do not have layer 2 failure indica=
tors,=20
  and therefore relies on layer 3 signaling for failure detection" is not=20
  entirely accurate. I'd point to IEEE 802.1ag/CFM and Y.1731 that can be u=
sed=20
  to monitor continuity&nbsp;among MEPs of specified MA/MEG. Thus use of La=
yer 3=20
  is not required but optional.</FONT></SPAN></DIV>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial siz=
e=3D2>Section=20
  5.2 In reference to BFD "These methods can be used for the layer 3 failur=
e=20
  indicators ..." I'd note that BFD can be used in ACH, i.e. non-IP,=20
  encapsulation to monitor LSP, Segment or Section.</FONT></SPAN></DIV>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial siz=
e=3D2>Section=20
  5.6 Restoration and Reversion. Restoration in MPLS-TE FRR network relies =
on=20
  the control plane signalling. As I interpret Section 2. Document Scope, t=
he=20
  document concentrates on defining benchmarking test cases to measure=20
  performance of&nbsp;facility backup FRR as defined in RFC 4090. I think t=
hat=20
  Restoration and Reversion are out of that scope. If that is the case, the=
n=20
  statement&nbsp;"It is important to include Restoration and Reversion as a=
 step=20
  in each test case to measure the amount of packet loss, out of order pack=
ets,=20
  or duplicate packets that <SPAN class=3Dinsert>occurs in this process" is=
=20
  confusing.</SPAN></FONT></SPAN></DIV>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial siz=
e=3D2><SPAN=20
  class=3Dinsert>Section 6.1. I find most test cases very confusing.&nbsp;A=
s=20
  stated in the Section 2, the scope of this document is on facility backup=
=20
  method of&nbsp;MPLS-TE FRR. Thus backup tunnel is bypass backup tunnel th=
at is=20
  encapsulating protected LSP(s). </SPAN></FONT></SPAN></DIV>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial siz=
e=3D2><SPAN=20
  class=3Dinsert>Section 6.1 and 6.2 <FONT color=3D#0000ff><FONT color=3D#0=
00000>I=20
  don't see enough rational for so many test cases. IMO, PLR performance in=
=20
  protection switchover is not affected by topology of a bypass tunnel. And=
 I'm=20
  concerned that failure detection has not been separated from protection=20
  switchover itself and will make benchmark testing results, e.g. larger BF=
D=20
  interval being used larger variation in detection time will be in the=20
  tests</FONT></FONT>. Hence, after a switchover protected LSP would have a=
n=20
  extra label on its stack.</SPAN></FONT></SPAN></DIV>
  <LI>
  <DIV align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DArial siz=
e=3D2><SPAN=20
  class=3Dinsert>Section 7.1 Benchmarking MPLS Performance seems outside of=
 scope=20
  for this document. Can it be spawn into a separate document and then=20
  referenced?</SPAN></FONT></SPAN></DIV></LI></UL>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DA=
rial=20
size=3D2><SPAN class=3Dinsert>&nbsp;&nbsp;&nbsp; Regards,</SPAN></FONT></SP=
AN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D069521220-17072012><FONT face=3DA=
rial=20
size=3D2><SPAN class=3Dinsert>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=20
Greg</SPAN></FONT></SPAN></DIV><BR>
<DIV class=3DOutlookMessageHeader lang=3Den-us dir=3Dltr align=3Dleft>
<HR tabIndex=3D-1>
<FONT face=3DTahoma size=3D2><B>From:</B> Jay Karthik (jakarthi)=20
[mailto:jakarthi@cisco.com] <BR><B>Sent:</B> Friday, July 13, 2012 6:55=20
PM<BR><B>To:</B> Gregory Mirsky<BR><B>Cc:</B> bmwg@ietf.org;=20
mpls@ietf.org<BR><B>Subject:</B> Re: [bmwg] Last Call:=20
&lt;draft-ietf-bmwg-protection-meth-09.txt&gt; (Methodology for benchmarkin=
g=20
MPLS protection mechanisms) to Informational RFC<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if">Greg,</DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if"><BR></DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if">First=20
of all, apologies in getting back to you this late. Even though we incorpor=
ated=20
your comments in the most recent version (-10, published on 5/29), a formal=
=20
response to your comments should have been made sooner.&nbsp;</DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if"><BR></DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if">Please=20
see inline marked (JK) for the specific response to your comments.</DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if"><BR></DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if">Thanks=20
for your thorough review.</DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if"><BR></DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if">Best,</DIV>
<DIV=20
style=3D"FONT-SIZE: 14px; COLOR: rgb(0,0,0); FONT-FAMILY: Calibri, sans-ser=
if">Jay</DIV><SPAN=20
id=3DOLK_SRC_BODY_SECTION style=3D"COLOR: rgb(0,0,0)">
<DIV>
<DIV=20
style=3D"WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-brea=
k: after-white-space"><SPAN=20
id=3DOLK_SRC_BODY_SECTION>
<DIV>
<DIV><FONT size=3D+0><FONT size=3D+0><BR>I'm concerned with the very title =
of the=20
document, Methodology for benchmarking MPLS protection mechanisms, even tho=
ugh=20
only MPLS-TE FRR being considered while LSP end-to-end and segment protecti=
on=20
implicitly being kept out of the=20
scope.<BR></FONT></FONT></DIV></DIV></SPAN></DIV></DIV></SPAN>
<DIV style=3D"COLOR: rgb(0,0,0)"><BR></DIV>
<DIV style=3D"COLOR: rgb(0,0,0)">JK: Modified the title to be more explicit=
 in the=20
most recent version posted.</DIV><SPAN id=3DOLK_SRC_BODY_SECTION>
<DIV>
<DIV=20
style=3D"WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-brea=
k: after-white-space"><SPAN=20
id=3DOLK_SRC_BODY_SECTION>
<DIV>
<DIV><FONT class=3DApple-style-span=20
color=3D#ff0000><BR></FONT></DIV></DIV></SPAN><SPAN id=3DOLK_SRC_BODY_SECTI=
ON=20
style=3D"COLOR: rgb(0,0,0)">
<DIV>
<DIV>I've found several textual inaccuracies related to both MPLS and BFD t=
o=20
make me wonder if the document was liasioned to MPLS WG.<BR>List of acronym=
s is=20
missing - PLR, OIR, LOS, AIS, etc.<BR><BR><FONT style=3D"COLOR: rgb(0,0,0)"=
=20
color=3D#ff0000>JK: PLR is defined in Term RFC 6414 that we are referencing=
. OIR=20
Is mentioned in Appendix B. Other acronyms such as AIS and LOS are added in=
 rev=20
10.<BR></FONT><BR>Introduction. I think that for "planned link or node fail=
ure"=20
MBB is more efficient and useful than FRR. But MBB is not being mentioned i=
n the=20
document.<BR><BR><FONT size=3D+0>JK: We have used the term Revision which i=
mplies=20
MBB</FONT><BR><BR>Introduction. "A correlated failure is the simultaneous=20
occurrence of two or more failures." Personally, as correlated events I con=
sider=20
events with cause-effect relationship.<BR><FONT style=3D"COLOR: rgb(0,0,0)"=
=20
color=3D#ff0000></FONT>Introduction. Path restoration after FRR discussion =
does=20
not appear logical, in the scope of benchmarking document.<BR><BR>JK: We ha=
ve=20
modified section 5.6<FONT size=3D+0><FONT class=3DApple-style-span=20
color=3D#ff0000><BR></FONT></FONT><FONT style=3D"COLOR: rgb(0,0,0)"><BR>Doc=
ument=20
Scope. "Protection from Bi-directional Forwarding Detection (BFD) is outsid=
e the=20
scope of this document." I frankly couldn't decode this=20
sentense.<BR></FONT></DIV></DIV></SPAN></DIV></DIV></SPAN>
<DIV style=3D"COLOR: rgb(0,0,0)"><BR></DIV>
<DIV style=3D"COLOR: rgb(0,0,0)">JK: Addressed in rev 10</DIV><SPAN=20
id=3DOLK_SRC_BODY_SECTION style=3D"COLOR: rgb(0,0,0)">
<DIV>
<DIV=20
style=3D"WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-brea=
k: after-white-space"><SPAN=20
id=3DOLK_SRC_BODY_SECTION>
<DIV>
<DIV><FONT size=3D+0><FONT size=3D+0><FONT class=3DApple-style-span=20
color=3D#ff0000><BR></FONT>Document Scope. Several references to Path Resto=
ration=20
as Re-optimization - doubt that it belongs in the document at all.<BR><BR><=
FONT=20
style=3D"COLOR: rgb(0,0,0)" color=3D#ff0000>JK: Section 5.6 adds more color=
 to this=20
discussion<BR></FONT><BR>Sections 6.1.1 through 6.2.4 - what is relevance o=
f=20
listing numbers of labels in the stack?<BR><BR><FONT style=3D"COLOR: rgb(0,=
0,0)"=20
color=3D#ff0000>JK: Convergence, performance may be dependent on the label =
stack=20
depth. <BR></FONT><BR>Peerformance of control plane should be outside of th=
e=20
scope of benchmarking as it is end-to-end metrics, not explicitly of=20
DUT.<BR><BR><FONT style=3D"COLOR: rgb(0,0,0)" color=3D#ff0000>JK: Scope=20
modified<BR></FONT></FONT></FONT></DIV></DIV></SPAN></DIV></DIV></SPAN></BO=
DY></HTML>

--_000_FE60A4E52763E84B935532D7D9294FF13924C82A1FEUSAACMS0715e_--

From acmorton@att.com  Mon Jul 30 10:38:14 2012
Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A227511E80AE for <bmwg@ietfa.amsl.com>; Mon, 30 Jul 2012 10:38:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -105.647
X-Spam-Level: 
X-Spam-Status: No, score=-105.647 tagged_above=-999 required=5 tests=[AWL=0.149, BAYES_00=-2.599, MSGID_FROM_MTA_HEADER=0.803, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ucyYYzfMb6O5 for <bmwg@ietfa.amsl.com>; Mon, 30 Jul 2012 10:38:13 -0700 (PDT)
Received: from nbfkord-smmo06.seg.att.com (nbfkord-smmo06.seg.att.com [209.65.160.94]) by ietfa.amsl.com (Postfix) with ESMTP id 4966D11E808E for <bmwg@ietf.org>; Mon, 30 Jul 2012 10:38:13 -0700 (PDT)
Received: from unknown [144.160.128.153] (EHLO flpi408.enaf.ffdc.sbc.com) by nbfkord-smmo06.seg.att.com(mxl_mta-6.11.0-10) over TLS secured channel with ESMTP id 486c6105.0.256461.00-469.696352.nbfkord-smmo06.seg.att.com (envelope-from <acmorton@att.com>);  Mon, 30 Jul 2012 17:38:13 +0000 (UTC)
X-MXL-Hash: 5016c6857b9ac7a9-6ca8c8f83f5b7c12646b09c0684ef551d6415f97
Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flpi408.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id q6UHcAJW018214 for <bmwg@ietf.org>; Mon, 30 Jul 2012 10:38:12 -0700
Received: from fflint03.pst.cso.att.com (fflint03.pst.cso.att.com [150.234.39.63]) by flpi408.enaf.ffdc.sbc.com (8.14.5/8.14.5) with ESMTP id q6UHc0jt017463 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <bmwg@ietf.org>; Mon, 30 Jul 2012 10:38:06 -0700
Received: from alpd052.aldc.att.com (alpd052.aldc.att.com [130.8.42.31]) by fflint03.pst.cso.att.com (RSA Interceptor) for <bmwg@ietf.org>; Mon, 30 Jul 2012 10:37:41 -0700
Received: from aldc.att.com (localhost.localdomain [127.0.0.1]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q6UHbeqM017290 for <bmwg@ietf.org>; Mon, 30 Jul 2012 13:37:40 -0400
Received: from dns.maillennium.att.com (mailgw1.maillennium.att.com [135.25.114.99]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q6UHbb6u017122 for <bmwg@ietf.org>; Mon, 30 Jul 2012 13:37:37 -0400
Message-Id: <201207301737.q6UHbb6u017122@alpd052.aldc.att.com>
Received: from acmt.att.com (vpn-135-70-228-129.vpn.east.att.com[135.70.228.129](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20120730173306gw10025c65e>; Mon, 30 Jul 2012 17:33:06 +0000
X-Originating-IP: [135.70.228.129]
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Mon, 30 Jul 2012 13:37:21 -0400
To: bmwg@ietf.org
From: Al Morton <acmorton@att.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)]
X-MAIL-FROM: <acmorton@att.com>
X-SOURCE-IP: [144.160.128.153]
X-AnalysisOut: [v=1.0 c=1 a=4i23c6CIpeYA:10 a=wstEu69ZDhQA:10 a=ofMgfj31e3]
X-AnalysisOut: [cA:10 a=BLceEmwcHowA:10 a=kj9zAlcOel0A:10 a=xwOvzTHDVLE4u4]
X-AnalysisOut: [nGvK72ag==:17 a=48vgC7mUAAAA:8 a=jGqiy6jZAAAA:8 a=pGLkceIS]
X-AnalysisOut: [AAAA:8 a=UyXn0u00t8FPcUfw2wgA:9 a=CjuIK1q_8ugA:10 a=PgV7WA]
X-AnalysisOut: [adS9kA:10 a=U2-5aeDKFN8A:10 a=EMjICKLqsV8A:10 a=lZB815dzVv]
X-AnalysisOut: [QA:10 a=MSl-tDqOz04A:10]
Subject: [bmwg] Fwd: Need volunteers for the NomCom
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 30 Jul 2012 17:38:14 -0000

>X-Originating-IP: [150.234.117.113]
>DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ietf.org; s=ietf1;
>         t=1343666324; bh=V1/Lh13xCgFVckioZjA+x8OGQHibuYuqDRY5TT03s4U=;
>         h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
>         Subject:Message-ID:Date:List-Id:List-Unsubscribe:List-Archive:
>         List-Post:List-Help:List-Subscribe:Sender;
>         b=u1gv5uwVLr7tHLFiCiQg9+yX3uEtmJj2f9V82P0LiU7dREF9sJtG7KBmiuH1Kc3wI
>         +7hbq5yP7M7fQoqlmb9mO0lkhCj5Ojh1zsM0TBdX8QDlySM8a0knz50hGCDPYQpgA/
>         hYgiupn7u29ROTWk8OXul+vVi/FEPdkLgp8uK7gU=
>X-Original-To: wgchairs@ietfa.amsl.com
>Delivered-To: wgchairs@ietfa.amsl.com
>X-Virus-Scanned: amavisd-new at amsl.com
>X-Spam-Flag: NO
>X-Spam-Score: -102.599
>X-Spam-Level:
>X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5
>         tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100]
>From: NomCom Chair <nomcom-chair@ietf.org>
>To: Working Group Chairs <wgchairs@ietf.org>
>Subject: Need volunteers for the NomCom
>X-Test-IDTracker: no
>X-IETF-IDTracker: 4.32
>Date: Sun, 29 Jul 2012 20:22:33 -0700
>X-Mailman-Approved-At: Mon, 30 Jul 2012 09:38:40 -0700
>X-BeenThere: wgchairs@ietf.org
>X-Mailman-Version: 2.1.12
>List-Id: Working Group Chairs <wgchairs.ietf.org>
>List-Unsubscribe: <https://www.ietf.org/mailman/options/wgchairs>,
>         <mailto:wgchairs-request@ietf.org?subject=unsubscribe>
>List-Archive: <http://www.ietf.org/mail-archive/web/wgchairs>
>List-Post: <mailto:wgchairs@ietf.org>
>List-Help: <mailto:wgchairs-request@ietf.org?subject=help>
>List-Subscribe: <https://www.ietf.org/mailman/listinfo/wgchairs>,
>         <mailto:wgchairs-request@ietf.org?subject=subscribe>
>Sender: wgchairs-bounces@ietf.org
>X-Spam: [F=0.0967741935; B=0.500(0); spf=0.500; STSI=0.500(-49); 
>STSM=0.300(-49); CM=0.500; MH=0.500(2012073010); S=0.200(2010122901); SC=none]
>X-MAIL-FROM: <wgchairs-bounces@ietf.org>
>X-SOURCE-IP: [12.22.58.30]
>X-AnalysisOut: [v=1.0 c=1 a=LSVET5z0puwA:10 a=BLceEmwcHowA:10 a=wPDyFdB5xv]
>X-AnalysisOut: [gA:10 a=IkcTkHD0fZMA:10 a=rIEbu7p4NK8A:10 a=GU4rwa5YvsQnE1]
>X-AnalysisOut: [5YQEkOQw==:17 a=48vgC7mUAAAA:8 a=pGLkceISAAAA:8 a=-pkKFdpO]
>X-AnalysisOut: [DCBK8kvJeqcA:9 a=QEXdDO2ut3YA:10 a=MSl-tDqOz04A:10 a=lZB81]
>X-AnalysisOut: [5dzVvQA:10]
>
>We are currently looking for volunteers to serve on the 2012-2013 NomCom.
>As you know, the success of the NomCom process depends crucially on
>having a large pool of volunteers from throughout the IETF community.
>In particular, it is valuable for the pool of volunteers to have strong
>representation from all of the technical areas within the IETF.
>
>I understand that not all IETF participants read the IETF announce list
>frequently. Therefore, if you would be willing to inform active participants
>in your working groups about this year's call for NomCom volunteers, I
>would greatly appreciate it.
>
>The NomCom 2012-2013 Call for Volunteers is open until this Sunday,
>August 5. Details can be found at: 
>https://datatracker.ietf.org/ann/nomcom/49851/
>
>Thank you for your help,
>- Matt Lepinski
>   mlepinski.ietf@gmail.com
>   nomcom-chair@ietf.org


From internet-drafts@ietf.org  Tue Jul 31 08:24:49 2012
Return-Path: <internet-drafts@ietf.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2961A21F8620; Tue, 31 Jul 2012 08:24:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.333
X-Spam-Level: 
X-Spam-Status: No, score=-102.333 tagged_above=-999 required=5 tests=[AWL=0.266, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bkfbwc7IdThR; Tue, 31 Jul 2012 08:24:48 -0700 (PDT)
Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 783D021F861E; Tue, 31 Jul 2012 08:24:48 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: internet-drafts@ietf.org
To: i-d-announce@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 4.33
Message-ID: <20120731152448.19816.81394.idtracker@ietfa.amsl.com>
Date: Tue, 31 Jul 2012 08:24:48 -0700
Cc: bmwg@ietf.org
Subject: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-03.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 31 Jul 2012 15:24:49 -0000

A New Internet-Draft is available from the on-line Internet-Drafts director=
ies.
 This draft is a work item of the Benchmarking Methodology Working Group of=
 the IETF.

	Title           : Benchmarking Methodology for Content-Aware Network Devic=
es
	Author(s)       : Mike Hamilton
                          Sarah Banks
	Filename        : draft-ietf-bmwg-ca-bench-meth-03.txt
	Pages           : 19
	Date            : 2012-07-31

Abstract:
   This document defines a set of test scenarios and metrics that can be
   used to benchmark content-aware network devices.  The scenarios in
   the following document are intended to more accurately predict the
   performance of these devices when subjected to dynamic traffic
   patterns.  This document will operate within the constraints of the
   Benchmarking Working Group charter, namely black box characterization
   in a laboratory environment.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ca-bench-meth

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-bmwg-ca-bench-meth-03

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-bmwg-ca-bench-meth-03


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


From mhamilton@breakingpoint.com  Tue Jul 31 08:27:07 2012
Return-Path: <mhamilton@breakingpoint.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BD64D21F869E for <bmwg@ietfa.amsl.com>; Tue, 31 Jul 2012 08:27:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L6JnSa2MoVSG for <bmwg@ietfa.amsl.com>; Tue, 31 Jul 2012 08:27:07 -0700 (PDT)
Received: from mail.breakingpoint.com (mail.breakingpoint.com [65.36.7.12]) by ietfa.amsl.com (Postfix) with ESMTP id 4431621F869A for <bmwg@ietf.org>; Tue, 31 Jul 2012 08:27:07 -0700 (PDT)
Received: from EXCHANGE.securitytestsystems.com ([::1]) by EXCHANGE.securitytestsystems.com ([::1]) with mapi id 14.01.0355.002; Tue, 31 Jul 2012 10:27:01 -0500
From: Mike Hamilton <mhamilton@breakingpoint.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
Thread-Index: AQHNaT09HIL4wPedW0+mXUhExWj++ZdDj1wA
Date: Tue, 31 Jul 2012 15:26:59 +0000
Message-ID: <CC3D6320.1912A%mhamilton@breakingpoint.com>
In-Reply-To: <1113A19AC2AB4C4FB562BDA89BAD16061DD66430@CH1PRD0611MB420.namprd06.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: Microsoft-MacOutlook/14.2.3.120616
x-originating-ip: [10.10.12.196]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <860D766CEC13D544984A89D978880CB0@breakingpoint.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: Tom Alexander <talexander@ixiacom.com>
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 31 Jul 2012 15:27:07 -0000

Hi Tom,=20
Please see the responses below.

Comment 1:  I've adopted the text you suggest as written.
Comment 2:  Agreed and both comments adopted as suggested.
Comment 3:  I've removed what you suggested.  Also, I've modified the PoAF
language to say "Offered Percentage of Total Flows".
Comment 4:  Agreed and adopted text as you suggest.
Comment 5:  Totally agreed and changed as you suggest.
Comment 6:  I was trying to figure out the XML format for external
references when I was up against the deadline.  I've incorporated a proper
external reference.  Yes, this reference is publicly available at no
cost/subscription/etc.

I really appreciate your feedback and this has all been incorporated into
the -03 version that I just submitted.

Mike




-----Original Message-----
From: Tom Alexander <talexander@ixiacom.com>
Date: Monday, July 23, 2012 8:39 PM
To: Mike Hamilton <mhamilton@breakingpoint.com>, "bmwg@ietf.org"
<bmwg@ietf.org>
Subject: RE: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt

>Hello, Mike,
>
>I've read the latest posted ca-bench-meth-02 draft (finally! sorry for
>the delay) and I appreciate your incorporating my feedback on your
>previous version. It looks quite good. Enclosed are some remaining
>comments on the draft.
>
>I assume the terminology draft will be forthcoming?
>
>Best regards,
>
>- Tom Alexander
>


From talexander@ixiacom.com  Tue Jul 31 11:21:30 2012
Return-Path: <talexander@ixiacom.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1709821F890C for <bmwg@ietfa.amsl.com>; Tue, 31 Jul 2012 11:21:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.599
X-Spam-Level: 
X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hIBMsyO6-GOY for <bmwg@ietfa.amsl.com>; Tue, 31 Jul 2012 11:21:29 -0700 (PDT)
Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe006.messaging.microsoft.com [216.32.180.16]) by ietfa.amsl.com (Postfix) with ESMTP id 054C421F88F9 for <bmwg@ietf.org>; Tue, 31 Jul 2012 11:21:28 -0700 (PDT)
Received: from mail230-va3-R.bigfish.com (10.7.14.252) by VA3EHSOBE007.bigfish.com (10.7.40.11) with Microsoft SMTP Server id 14.1.225.23; Tue, 31 Jul 2012 18:21:28 +0000
Received: from mail230-va3 (localhost [127.0.0.1])	by mail230-va3-R.bigfish.com (Postfix) with ESMTP id 56A15140111; Tue, 31 Jul 2012 18:21:28 +0000 (UTC)
X-Forefront-Antispam-Report: CIP:157.56.244.149; KIP:(null); UIP:(null); IPV:NLI; H:CH1PRD0611HT002.namprd06.prod.outlook.com; RD:none; EFVD:NLI
X-SpamScore: -28
X-BigFish: PS-28(zz9371I148cI542M1432Izz1202hzz1033IL8275bh8275dhz2fh2a8h668h839h944hd25hf0ah107ah)
Received-SPF: pass (mail230-va3: domain of ixiacom.com designates 157.56.244.149 as permitted sender) client-ip=157.56.244.149; envelope-from=talexander@ixiacom.com; helo=CH1PRD0611HT002.namprd06.prod.outlook.com ; .outlook.com ; 
Received: from mail230-va3 (localhost.localdomain [127.0.0.1]) by mail230-va3 (MessageSwitch) id 1343758886156758_4810; Tue, 31 Jul 2012 18:21:26 +0000 (UTC)
Received: from VA3EHSMHS025.bigfish.com (unknown [10.7.14.241])	by mail230-va3.bigfish.com (Postfix) with ESMTP id 1A9DF38004D; Tue, 31 Jul 2012 18:21:26 +0000 (UTC)
Received: from CH1PRD0611HT002.namprd06.prod.outlook.com (157.56.244.149) by VA3EHSMHS025.bigfish.com (10.7.99.35) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 31 Jul 2012 18:21:25 +0000
Received: from CH1PRD0611MB420.namprd06.prod.outlook.com ([169.254.8.164]) by CH1PRD0611HT002.namprd06.prod.outlook.com ([10.255.148.165]) with mapi id 14.16.0175.005; Tue, 31 Jul 2012 18:21:25 +0000
From: Tom Alexander <talexander@ixiacom.com>
To: Mike Hamilton <mhamilton@breakingpoint.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
Thread-Index: AQHNAIhwQ06zgXVRJ0GGJq/Ccz5bAZZnE0EAgAA3vICA0S128IAL6FaAgAAtziA=
Date: Tue, 31 Jul 2012 18:21:24 +0000
Message-ID: <1113A19AC2AB4C4FB562BDA89BAD16061DD6A61B@CH1PRD0611MB420.namprd06.prod.outlook.com>
References: <1113A19AC2AB4C4FB562BDA89BAD16061DD66430@CH1PRD0611MB420.namprd06.prod.outlook.com> <CC3D6320.1912A%mhamilton@breakingpoint.com>
In-Reply-To: <CC3D6320.1912A%mhamilton@breakingpoint.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [50.43.73.138]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossPremises-AuthAs: Internal
X-MS-Exchange-CrossPremises-AuthMechanism: 04
X-MS-Exchange-CrossPremises-AuthSource: CH1PRD0611HT002.namprd06.prod.outlook.com
X-MS-Exchange-CrossPremises-SCL: -1
X-MS-Exchange-CrossPremises-messagesource: StoreDriver
X-MS-Exchange-CrossPremises-BCC: 
X-MS-Exchange-CrossPremises-rules-execution-history: Support - Juniper
X-MS-Exchange-CrossPremises-processed-by-journaling: Journal Agent
X-MS-Exchange-CrossPremises-ContentConversionOptions: False; 00160000; True; ; iso-8859-1
X-OrganizationHeadersPreserved: CH1PRD0611HT002.namprd06.prod.outlook.com
X-OriginatorOrg: ixiacom.com
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 31 Jul 2012 18:21:30 -0000

Hello, Mike,

Thanks for incorporating my feedback. The -03 draft looks very good and I h=
ave no further comments.

Best regards,

- Tom

-----Original Message-----
From: Mike Hamilton [mailto:mhamilton@breakingpoint.com]=20
Sent: Tuesday, July 31, 2012 8:27 AM
To: bmwg@ietf.org
Cc: Tom Alexander
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt

Hi Tom,
Please see the responses below.

Comment 1:  I've adopted the text you suggest as written.
Comment 2:  Agreed and both comments adopted as suggested.
Comment 3:  I've removed what you suggested.  Also, I've modified the PoAF =
language to say "Offered Percentage of Total Flows".
Comment 4:  Agreed and adopted text as you suggest.
Comment 5:  Totally agreed and changed as you suggest.
Comment 6:  I was trying to figure out the XML format for external referenc=
es when I was up against the deadline.  I've incorporated a proper external=
 reference.  Yes, this reference is publicly available at no cost/subscript=
ion/etc.

I really appreciate your feedback and this has all been incorporated into t=
he -03 version that I just submitted.

Mike




-----Original Message-----
From: Tom Alexander <talexander@ixiacom.com>
Date: Monday, July 23, 2012 8:39 PM
To: Mike Hamilton <mhamilton@breakingpoint.com>, "bmwg@ietf.org"
<bmwg@ietf.org>
Subject: RE: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt

>Hello, Mike,
>
>I've read the latest posted ca-bench-meth-02 draft (finally! sorry for=20
>the delay) and I appreciate your incorporating my feedback on your=20
>previous version. It looks quite good. Enclosed are some remaining=20
>comments on the draft.
>
>I assume the terminology draft will be forthcoming?
>
>Best regards,
>
>- Tom Alexander
>



