Subject: [hops] Proposal for HOPS RG

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 22 May 2015 13:14:25 +0200

Hi Lars,

as you might have seen in the mean time, we would like to propose a new HOPS research group. Based on the fruitful discuss we had at the BarBoF in Dallas and further discussions with individuals we think that this topic is appropriate for a research group and send a proposed charter text with this mail (see below). Our goal would be to provide a forum to exchange and discuss measurement results and methodology on middlebox impairments. Further we’d also like work on a common data format that can be used to specify impairment tests to achieve comparable results from different existing and new data sets independent of the technique that was used to collect this data. We think that a research group is also the appropriate form for this activity. See further details below.

In parallel we’ve been also talking to people how indicated interest at the BarBoF as well as further people from the research community to figure out if there is interest in giving a presentation at a potential rg meeting. We have received large interest and are positive that people will be able to actually present data at such a meeting. See a tentative agenda for a potentially first meeting in Prague below.

Let us know what you think or if you have any further questions!

Brian and Mirja

------------------------------------------

How Ossified is the Protocol Stack? (HOPS)
==========================================

There has been long term and increasing interest in deploying transport protocols with alternate dynamics and behaviors to TCP and UDP. The IETF has standardized several new protocols including DCCP, UDP-lite, SCTP and several changes to TCP including ECN and LEDBAT. All of these new technologies have resulted in deployment challenges blamed on intentional and unintentional interference by middleboxes such as NATs and firewalls. This has lead to approaches such as building new protocols over UDP or HTTP to make traffic look like something a middlebox would expect. However, both these approaches have shortcomings and a variety of ameliorating engineering approaches are being considered.

What is missing is a study with more than anecdotal evidence of the nature of the problem and the portions of the network in which it manifests. One of the best analyses to date is [1] which measures from a very small number of locations: 49 residential, 17 enterprise, and 142 locations in total. There are two more recent studies: [2] is focusing on TCP in-band testing while using PlanetLab as a measurement testbed; [2] use crowd-sourcing with in total 1,165 workers from 53 different countries covering a larger set of access networks including mobile access for TLS testing [3]. However, to be able to draw conclusions about the significance of a certain middlebox interference problem more data form a large variety of access networks is needed. In the interest of getting ground-truth data about the nature of the problem, the HOPS research group will provide a discussion forum to coordinate efforts in research and industry -- such as network stack, browser, and middlebox vendors, as well as network and service operators -- on collecting and reporting statistics about middlebox impact on transport sessions.

[1] M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M. Handley, and H. Tokuda.
Is it still possible to extend tcp? In Proc. ACM IMC, 2011.
<http://conferences.sigcomm.org/imc/2011/docs/p181.pdf>

[2] R. Craven, R. Beverly, and M. Allman.
A middlebox-cooperative TCP for a non end-to-end Internet. In Proc. ACM SIGCOMM, 2014.
<http://doi.acm.org/10.1145/2619239.2626321>

[3] A. M. Mandalari, M. Bagnulo, A. Lutu.
Informing Protocol Design Through Crowdsourcing: the Case of Pervasive Encryption.
ACM SIGCOMM Workshop on Crowdsourcing and crowdsharing of Big (Internet) Data (C2B(I)D), 2015.
<http://www.it.uc3m.es/amandala/c2bid.html>

Objective
---------

The HOPS research group follows from the successful BarBoF meeting organized by Aaron Falk at IETF92 in Dallas to bring more data on the nature and extent of middlebox interference to protocol design and engineering efforts. To this end, we aim to provide a forum for discussion and exchange of measurement insights, data, and techniques co-located with IETF meetings. The BarBoF identified the need for this forum, with many participants wanting a place to continue these discussions.

In addition, we have identified two near-term goals to be completed within the research group in support of this work.

First is the definition of a common format for reporting on middlebox impairments observed in the network, whether these observations are made passively, actively, or as a side effect of some other operation, e.g. taken from application or network stack error logs. This common format must provide for correlation and comparison among data from disjoint observations, and address end-user privacy and business confidentiality concerns, e.g. using path pseudonyms instead of paths identified by AS number and/or IP address where necessary. These measurements should be useful not only for detecting impairments but also for assessing the likelihood that a certain impairment will be experienced by traffic on certain types of networks and in the Internet as a whole. For this purpose aggregated statistics on the prevalence of certain middblebox behaviors are useful, even without revealing the underlying measurement data.

Second is the specification of methods for analysis of middlebox interference, and associated active measurement techniques, that can scale to much larger numbers of measured paths than those presently in the literature, while minimizing network measurement traffic load on the network. Similar techniques could be used for middlebox benchmarking. Focusing on active measurements that are considered feasible for inclusion in well-known end-systems (e.g. in browsers or the OS) has been identified as way forward to collect more data and reach a high coverage of different access networks, specifically including mobile and fixed, residential, as well as enterprise network. These active measurements will be useful for targeted questions about specific paths as well as to fill in data not available from passive measurement. Further, providing guidelines on with measurement and analysis techniques are needed to detect and classify certain impairments may increase the willingness of industry to share data that is already available.

Membership
----------

Membership in the HOPS RG is open.

Meetings
--------

The HOPS RG will meet one to three times per year, initially always co-located with IETF meetings, to foster exchange among researchers and between research and industry on middlebox measurement topics. Meetings may in the future be scheduled to provide additional interaction with the network operations community, should operationally relevant and useful results warrant this.

We aim to hold a first meeting at IETF 93 in Prague, and already have a number of presentation confirmed leading to the following tentative agenda:

1) Welcome and Intro

2) Passive Measurement Data
  - "Results from deployment of QUIC, a UDP-based transport" Jana Iyengar (Google)
  - Dave Plonka (Akamai) (not fully confirmed yet)

3) Active Measurements
  - Robert Kisteleki (RIPE) (title tbd)
  - "Lessons learnt from middlebox measurement” Michio Honda (NetApp)

4) Measurement Methodology
  - "Tracking Middleboxes with Tracebox" Korian Edeline (Université de Liège)

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 22 May 2015 13:21:18 +0000

Hi,

On 2015-5-22, at 13:14, Mirja Kühlewind <[email-masked]> wrote:
> Our goal would be to provide a forum to exchange and discuss measurement results and methodology on middlebox impairments. Further we’d also like work on a common data format that can be used to specify impairment tests to achieve comparable results from different existing and new data sets independent of the technique that was used to collect this data.

that seems like a reasonable goal.

Which projects are doing these measurements at the moment and into the future, and do you have buy-in from them? A lot of work in this space is one-off measurement activities that end as soon as a paper was published.

Lars

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 22 May 2015 15:46:51 +0200

Hi Lars,

there are people from RIPE who are interested in this work and were already at the BarBoF. Further we are also in contact which the people from CAIDA. And, as you can see on the agenda, we are also talking to Google and Akamai with people who were also at the BarBoF and it actually seems to be possible that theyl will release some data. In fact we also don’t necessarily need the data itself. If we come up with a common format where we can specific tests, it is probably sufficient to release the final evaluated results. 

Regarding one-off measurement, these are definitely also useful if comparable to other results. Further if the raw data can be made available, it might be possible to apply further tests to the collected data at a later time.

I personally don’t know if this activity and interest, we see right now, will stay for long. However, even if this might not lead to a long-time research group, I think it is worth it to use the momentum and collect (and analyze) the data we can get right now.

Mirja

> Am 22.05.2015 um 15:21 schrieb Eggert, Lars <[email-masked]>:
> 
> Hi,
> 
> _______________________________________________
> hops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hops

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 22 May 2015 14:06:14 +0000

Hi,

On 2015-5-22, at 15:46, Mirja Kühlewind <[email-masked]> wrote:
> there are people from RIPE who are interested in this work and were already at the BarBoF. Further we are also in contact which the people from CAIDA. And, as you can see on the agenda, we are also talking to Google and Akamai with people who were also at the BarBoF

so that's promising, but not actually a large number of folks. I wonder if a discussion among four groups really needs an RG established. Isn't this something that might as well be handled  ad hoc?

A second concern I have is that the topic here is fairly narrow in scope ("let's discuss data around how bad middleboxes break things"), and rather short-lived (i.e., once that is done, the group is done). The IRTF tries to charter groups that are long-lived and try to tackle problem areas of substantial size, and I wonder if this is the case here.

(Since I was not at the bar BOF, I may be fundamentally misunderstanding something about this proposal. I'm only going on what is in the charter text proposal.)

Lars

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 22 May 2015 14:19:00 +0000

Hi again,

I should probably clarify that I'm *not* pushing back on you guys having a meeting in Prague and possibly later IETFs to see if a community forms. I will definitely approve rooms.

Take my comments as early feedback on the charter text you sent. The actual chartering discussion will happen after you've tried to meet a few times and we see what has happened in those meetings.

Lars

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 22 May 2015 16:25:08 +0200

Hi Lars,

if things run well, I don’t consider this as a short-term group. The goal of collecting and providing this measurement data is two-fold: 1) this data should provide input for future protocol specification. As I don’t believe that all middleboxes will suddenly disappear or at least not do any ‚stupid‘ things anymore, I see this as a long-termn project because data needs to be updated continuously. Further as we are just at the beginning, it’s not only about collecting the data but in many cases also about developing the appropriate measurement methodology/technique to actually detect the impairments. And 2) it not only about detecting which impairments exist and how likely it is that traffic we experience these impairment. I also see as a goal to detect broken middleboxes and start activities to get them removed. I guess that’s really not a short-term goal. Maybe we can spell out this part more explicitly in the charter as well.

Mirja

> Am 22.05.2015 um 16:06 schrieb Eggert, Lars <[email-masked]>:
> 
> Hi,
>

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 22 May 2015 16:27:07 +0200

Hi Lars,

thanks, that’s absolutely in line with what we have in mind. We see a to of activity and interest at the moment and potential for a longer-term research group. However, you can never say if people will be able to actually release data and how and so we would like to have a few meeting and see if we ca come up with continuous work we would like to do here.

Mirja

> Am 22.05.2015 um 16:19 schrieb Eggert, Lars <[email-masked]>:
> 
> Hi again,
>

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 22 May 2015 16:38:53 +0200

I dont think this should be a short lived effort.
I mean, at the IETF there is plenty of times that we try to design a 
protocol or protocol extension and we dont really have the data to 
perform an informed decision.  think the HOPS RG could be a starting 
point for many protocol design efforts.
For instance, in the TCPINC BOF, the question about whether encrypted 
TCP connection would fly over different ports was raised and there was 
no data available. (and it was a fundamental question to understand if 
the whole effort was worthwhile)
Similar questions now are raised in TCPM when designing the extended 
option format. And again, there is little data around (at least for some 
aspects of it).
It would be nice to have a place where people that want to work on 
design can gather data about what works and what doesnt. It would be 
nice if that was the HOPS RG, i guess.

In other words, one way of doing this is for the HOPS RG t be a venue 
for people with interesting questions and people who want to measure 
intersting things (or for people with interesting questions and people 
who has data that can help them answer the questions)

So, imho, something like HOPS is really missing in the IETF protocol 
design approach. But maybe it is just me.

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 22 May 2015 16:40:01 +0200

Hi Lars again,

one more point: it’s not only these four groups. 

There was quite a large number of people at the BarBoF and we had some really good discussion. Initially the goal for the barBoF was not at all to start a research group. We actually only wanted to find a way to integrate passive measurements into e.g. browser traffic as those application often already perform certain fallbacks and so on to find a working connection. In these cases it would have been simply nice to collect ‚reports‘ about failures that were experienced in operation. This would be a much more limited scope and would not need a research group. 

However, there was much broader interest in middlebox measurements at the barBoF and we do think that it would be nice to have an official discuss forum for this work. We’ve only discussed with the mention people to far to organize a potential first meeting and I think we already have more people that are interest and will to present something that we have time in the first meeting. Further as the goal is to make measurement more comparable by using a common data format, we see it as our task to actively invite further people doing measurements and therefore provide a connection between research and the IETF.

Mirja

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 22 May 2015 10:43:42 -0400

On 22 May 2015, at 10:38, marcelo bagnulo braun wrote:

> option format. And again, there is little data around (at least for 
> some aspects of it).

agree.  others I can think of, such as multi path tcp, many rai 
protocols,

> design approach. But maybe it is just me.

not. count me in! deployability has not been so much considered. Many 
protocols have been designed and then a fallback to http/443 was added 
later which makes the protocol brittle.

Marc.

>>> there are people from RIPE who are interested in this work and were 
>>> already at the BarBoF. Further we are also in contact which the 
>>> people from CAIDA. And, as you can see on the agenda, we are also 
>>> talking to Google and Akamai with people who were also at the BarBoF
>> so that's promising, but not actually a large number of folks. I 
>> wonder if a discussion among four groups really needs an RG 
>> established. Isn't this something that might as well be handled  ad 
>> hoc?
>>
>> A second concern I have is that the topic here is fairly narrow in 
>> scope ("let's discuss data around how bad middleboxes break things"), 
>> and rather short-lived (i.e., once that is done, the group is done). 
>> The IRTF tries to charter groups that are long-lived and try to 
>> tackle problem areas of substantial size, and I wonder if this is the 
>> case here.
>>
>> (Since I was not at the bar BOF, I may be fundamentally 
>> misunderstanding something about this proposal. I'm only going on 
>> what is in the charter text proposal.)
>>
>> Lars
>>
>>