Subject: Re: [lmap] Comments on draft-schulzrinne-lmap-requirements-00

From: Al Morton <[email-masked]>
Date: Tue, 25 Sep 2012 08:50:49 -0400

As promised, a few more detailed comments,

Al

It would be good to clarify the term "sample"
in:

3.4. Data collector

The data collector collects time-stamped measurement samples from

measurement clients. It generally makes these measurement samples

available only to authorized users.

There are several possibilities, drawing on IPPM's RFC 2330
terminology:

A singleton is an individual measurement, sometimes involving one
packet

(as in one-way delay) or many packets (a throughput
measurement).

A sample is a set of singleton measurements.

A statistic is a metric derived from a sample of measurements (e.g.,
median).

Which does the term "sample" in section 3.4 line up with
best?

Section 6:

A-4: Measurement clients MUST be able perform both active and

passive measurements.
There is a small set of overlap capabilities between active and
passive

measurement techniques (as far as I know). It may help to keep the

requirements for active and passive capabilities separate, even
making

the agents that perform these measurements separable as I alluded to

in the earlier post (a user passive measurement agent).

In general, it's useful to have requirements for fail-over,

security/integrity and extensibility for the supported measurements.

The inevitable negotiation of capabilities might be simplified if

there are a small set of hierarchical "profiles", where the
higher function

profiles also support all the capabilities of lesser
profiles/categories.

As Ken pointed-out, it's not clear to me what was meant by D-6:

D-6: Clients MUST be able to interleave data samples from different

measurement metrics to the data collector.
especially true while waiting for the definition of
"sample".

In D-7:

D-7: The data collector SHOULD be able to ascertain whether the

measurement client clock is at least approximately synchronized to

its own.
I agree that synchronization status needs to be communicated, but

I see it as a MUST and also more than a binary status. "I just

found a timing source" is different from long-term association
with

multiple time servers along with estimates of time error at the
device.

In N-1 through N-4:

Need to include the possibility that a single subscriber may have
access

to multiple classes of service.

Need to allow for communication of the method to indicate service
class,

if multiple classes of service exist. This might be as simple as
indicating

the packet marking needed.

Agree that N-8 is needed, possibly a MUST (automatic update of network
params).

At 02:48 PM 9/24/2012, Al Morton wrote:

LMAP co-authors,

Some further comments on your Internet Draft (I also appreciate reading
your

views on this measurement architecture, it is a challenging topic as

you rather thoroughly described it in the 00 version).

The comments concern the measurement entity roles, both those
currently

identified and possible additions. Also, there are complexity
limitations

to consider, especially when the devices perform both active and
passive

measurements.

As Ken Ko pointed out, the names and definitions of measurement
client

and measurement server are somewhat ambiguous. Although they are
familiar

terms, they imply a hierarchy which may not be present. It seems
that

these two entities are most easily distinguished by their location
in

the architecture, and depending how the measurement protocol works
and

the metric measured, the entities could take on multiple roles. It
is

worth considering more generic names, like "user agent" and
"network agent".

Location names hold-up when testing between two network agents.

The generic names avoid two problems:

- Some candidate test protocols already use client and server

- IPPM's Source and Destination host designations were fine
for

one-way testing, but were much less clear with round-trip
metrics.

The names could be even more specific, i.e. "user active measurement
agent",

and this would help to categorize requirements when the details of

active and passive measurements are decided. One could imagine
categories

of user agents with ability to test rates up to XX Mbps, others up
to

YY (> XX) Mbps, while maintaining other accuracy
requirements.

It's also useful to consider less complex measurement participation

at the distant edge of the network, where the scale is the largest.

Some test protocols and performance metrics already take this
reality

into account, and should be candidates for the job.

I think the demands of timing distribution in a packet network

and the fundamental importance of time synchronization and stability

in measurements warrant adding a time distribution entity to the

architecture, with the additional need to identify the protocol
between

all entities requiring accurate estimates of time.

I have some detailed comments on the requirements to share, but

they will have to wait for a future message.

regards,

Al

At 11:09 AM 9/21/2012, KEN KO wrote:

Thank you for posting this draft
it’s an excellent introduction to the issues and requirements
associated with large scale network performance testing. I think it will
generate much needed discussion. To help kick that discussion off, here
are some comments based on an initial reading of the material.

In section 3, the descriptions of the measurement client and measurement
server seem to be inconsistent. The server can both sink and source
traffic but the client is only a source (or an observer for passive
measurements). I think both the client and the server will source and
sink traffic. A more general comment is the need for unambiguous
definition of terms including measurement client and measurement server.
This should probably be coordinated with framework definition work in
progress at the Broadband Forum.

In section 4, the “measurement client to measurement controller” protocol
is described in terms of “pull” that is, with each client querying the
controller for its measurement schedule. This may be preferable for an
architecture where the network is sparsely populated with measurement
clients and a high percentage of the installed clients participate in a
test program. Looking forward to an environment where test clients become
ubiquitous and the number of clients participating in test program may be
a tiny fraction of the total, a “push” protocol where the controller
initiates communication only to those clients actively involved in a test
program should result in far less control traffic. We may need to
consider both alternatives. A solution hinted at in Section 5 might be
where measurement clients initially register with a controller and are
notified of push or pull mode as part of the registration
process.

There is a need to identify different types of measurement clients to the
measurement controller. Some clients, for instance located at residential
or business gateways adjacent to UNIs, are primary candidates for
scheduled test activities. Other clients located at intermediate nodes
within the network may be used only for troubleshooting, or at least may
have a different set of tests scheduled than a gateway. Still others such
as software-based clients installed on fixed or mobile endpoint devices
might serve either purpose, depending on whether there the associated
gateway is a measurement client and whether the endpoint is considered
“reliable” for the purposes of a specific test program. Classification of
measurement client both by feature set and by network location may be
desirable.

Requirements comments:

A-3: Not really a requirement this is informative as a use case.

C-3: Depending on how registration gets defined, this may be a MUST.

D-6: It isn’t clear how “interleave” is being used here or why clients
MUST be able to interleave data samples from different measurement
metrics. What is the problem with a file transfer that groups samples by
metric type?

D-9: seems redundant given A-6.

N-1: Consider making this capability a MUST. Whether it is used in a
specific environment is out of standards scope.

N-6, N-7: redundant given A-6. Authentication features should be
MUSTs.

Finally, many of the descriptions and requirements in this document are
relevant to the performance testing architecture and framework being
developed in the Broadband Forum. I recommend liaising this document, or
the updated document resulting from the upcoming IETF meeting, to the
BBF.

Best regards,

Ken Ko

_______________________________________________

lmap mailing list

[email-masked]

https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________

lmap mailing list

[email-masked]

https://www.ietf.org/mailman/listinfo/lmap