Subject: [lmap] survey on large-scale measurement platforms and related standards

From: "Bajpai, Vaibhav" <[email-masked]>
Date: Tue, 7 Apr 2015 09:41:08 +0000

Dear LMAP WG,

I would like to share something with you today.

As you know, early discussions within the IETF LMAP working group often lead
to folks asking for feature sets and possibilities of each contemporary
performance measurement platform. So I started digging literature work in
this space and found that although there were surveys on topology-based
measurement platforms (such CAIDA Ark et al.); literature work on performance
measurement platforms (such as SamKnows, RIPE Atlas et al.) was missing.

Therefore, I started writing such a survey in 2013 to plug this gap and
also complement this with a summary of IETF standardization efforts happening
within the LMAP and IPPM working groups.

This work got published and came online this week.
I thought I would share this with you:

----------------8<-----------------8<-----------------8<-----------------8<
A Survey on Internet Performance Measurement Platforms and Related Standardization Efforts
Vaibhav Bajpai, Jürgen Schönwälder
IEEE Communications Surveys & Tutorials
April, 2015

A number of Internet measurement platforms have emerged in the last few years.
These platforms have deployed thousands of probes at strategic locations
within access and backbone networks and behind residential gateways. In this
paper we provide a taxonomy of these measurement platforms on the basis of
their deployment use-case. We describe these platforms in detail by exploring
their coverage, scale, lifetime, deployed metrics and measurement tools,
architecture and overall research impact. We conclude the survey by describing
current standardization efforts to make large-scale performance measurement
platforms interoperable.

DOI: http://dx.doi.org/10.1109/COMST.2015.2418435
Author Copy: http://vaibhavbajpai.com/documents/papers/proceedings/lsmp-comst-2015.pdf
----------------8<-----------------8<-----------------8<-----------------8<

Thanks!

Best, Vaibhav

=====================================================
Vaibhav Bajpai

Research I, Room 91
Computer Networks and Distributed Systems (CNDS) Lab
School of Engineering and Sciences
Jacobs University Bremen, Germany

www.vaibhavbajpai.com
=====================================================

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Mon, 13 Apr 2015 13:57:11 +0000

Just a note about something we've come across lately for the lmap group to consider / chew on a bit.
NOT for the current spec, but future "piece parts"...

Two subjects in ensure testing results are contextually well linked to something..
        - Technology Evolution and Business relationship evolution inspired dimensions to not having apples to apples test results over time....
        - Benching marking on the path performance, not application development for anything using jitter buffers.

Problem(s) -

a) Server change out
        -  I'm testing Video but the Video equipment and software changes every 1 - 1.5 years (It's highly likely the results are off a completely different "video server front end" one year to the next)

b) Server Location change out
        The CDN locations serving the customers moves every 2-3 years based on CDN contracts...
        Load balancing and using the "fixed bandwidth" components of a CDN also change the serving location
        Both result is path changes that change performance

c) Hot potato routing (testing directionality)
        The download path and the upload path aren't the same - so using and "upload" traceroute may not capture the actual path of the download traffic, but the download is where we are looking for performance.

possible solution to "a thru c" issues - is to place a test server at the CDN / Server locations that doesn't change over time, and can measure the path performance from end to end.   This would enable a control to reflect a path change vs. a front end change, and would also give the correct path for the download performance vs. the upload path.

d) Benchmarking on "expected path performance" vs. what any one application is set too.

        Analogously - shock absorbers are designed to correct the bumps in the road, and jitter buffers are designed to do the same.
        To ensure a "appropriate" jitter buffer "delay to X seconds or frames and then play" - one needs to understand how bumpy the road is vs. measuring the travel in a shock absorber to identify what the baseline should be.

        I'm going to ping some of the universities looking at the duration of a congestion event on WIFI to see what they came up with in terms of collective collision seconds behavior of WIFI for this one.
        At the end of the day we should identify what the normal period for interruption is in order to create a standard... this may change with technology, time, distance, ... don't know yet.
        So for now - it's the principle of creating the foundation for the bench mark vs. grabbing what's commonly used.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.