Subject: [lmap] Advertised rate tests

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 3 Dec 2012 04:10:24 +0000

Changing the subject to reflect one topic.

Currently, the FCC MBA effort measures long-term TCP throughput on a reasonably well-defined sub-path, roughly reflecting the part of the path seen as being under the direct control of the consumer ISP. The measurements indicate that this tends to agree with the advertised rates. In the old days, it made sense to compare this to "line rate", but that's clearly not useful any more, given that even DSL is now a variable-rate (line-noise dependent) L2 service. The rate is essentially determined, at least for cable and fiber, by rate controllers.

Are you suggesting that the long-term TCP throughput is not a good headline metric?

For the model-based metric, do they provide a similar Mb/s metric that is likely to correspond to the advertised throughput? In other words, would a model-based metric for a "20 Mb/s" service yield 20 Mb/s or some other number?

Henning

________________________________________
From: [email-masked] [[email-masked]] on behalf of Al Morton [[email-masked]]
Sent: Sunday, December 02, 2012 10:02 PM
To: Henning Schulzrinne; Matt Mathis
Cc: [email-masked]; Nicholas Weaver; Brian Trammell; [email-masked]
Subject: Re: [lmap] [ippm]  Focus on Tests, Not Architectures...

Regarding "advertised rates":

- I think there's variation among the underlying units of measure for
   the numbers advertised today - this is the point I tried to make
   during Henning's Transport Area presentation. It would certainly help
   with service verification if the parameters describing service were
   clearly defined for those performing the verification - I appreciate that
   most consumers won't care much about the units of measure, but verification
   should be scientific (e.g., conducted at the same layer as the spec).

- The model-based metrics have appeal to me, in that they are derived
   from fundamental metrics that we have a reasonable hope of specifying
   sufficiently such that independent implementations and different
   hardware/software platforms will be able to make equivalent measurements.
   It may even turn-out that similar methods can provide a basis for
   model-based estimates of multiple (application?) metrics (Matt's
   memo focuses on TCP, which is certainly enough for now).

---

From: Al Morton <[email-masked]>
Date: Mon, 03 Dec 2012 08:55:43 -0500

At 11:10 PM 12/2/2012, Henning Schulzrinne
wrote:

Currently, the FCC MBA effort measures long-term TCP throughput on a
reasonably well-defined sub-path, roughly reflecting the part of the path
seen as being under the direct control of the consumer ISP. The
measurements indicate that this tends to agree with the advertised rates.
In the old days, it made sense to compare this to "line rate",
but that's clearly not useful any more, given that even DSL is now a
variable-rate (line-noise dependent) L2 service. The rate is essentially
determined, at least for cable and fiber, by rate
controllers.

I think that line rate may be a worthwhile member of the

measurement list, especially when it is variable (capturing

variability within a given spec should be an important aspect

of this exercise, IMO).

Some ISPs are just selling an IP packet transfer service,

and the advertised rate may include IP header and payload.

TCP throughput is measured at a higher layer (just

the tip of the iceberg when it comes to differences).

Are you suggesting that the long-term TCP throughput is not a good
headline metric?

I would suggest that it is one of several, because especially with
TCP,

YMMV due to many factors along the transport connection's path.

TCP Throughput seems closer to "city" miles per gallon than
"highway",

so I think some highway-end metric may be useful, too.

For the model-based metric, do they provide a similar Mb/s metric
that is likely to correspond to the advertised throughput? In other
words, would a model-based metric for a "20 Mb/s" service yield
20 Mb/s or some other number?

The short answer is yes. I think it's reasonable to assume that what

we standardize will influence the specifications that ISPs
advertise.

For more details, see Matt's memo

http://tools.ietf.org/html/draft-mathis-ippm-model-based-metrics-00

and doc at

https://docs.google.com/document/d/1KDJMuZ2cWfXbnWjgwNDpN4w5smvZLfCL-gTk6Xgx1mI/edit#

regards,

Al

Henning

From: [email-masked] [[email-masked]] on behalf of Al
Morton [[email-masked]]

Subject: Re: [lmap] [ippm]  Focus on Tests, Not
Architectures...

- I think there's variation among the underlying units of measure
for

the numbers advertised today - this is the point I tried to
make

during Henning's Transport Area presentation. It would
certainly help

with service verification if the parameters describing
service were

clearly defined for those performing the verification - I
appreciate that

most consumers won't care much about the units of measure,
but verification

should be scientific (e.g., conducted at the same layer as
the spec).

- The model-based metrics have appeal to me, in that they are
derived

from fundamental metrics that we have a reasonable hope of
specifying

sufficiently such that independent implementations and
different

hardware/software platforms will be able to make equivalent
measurements.

It may even turn-out that similar methods can provide a
basis for

model-based estimates of multiple (application?) metrics
(Matt's

memo focuses on TCP, which is certainly enough for
now).

_______________________________________________

ippm mailing list

[email-masked]

https://www.ietf.org/mailman/listinfo/ippm