Subject: [apps-discuss] AD Fail - MPTCP API

From: Pete Resnick <[email-masked]>
Date: Wed, 14 Nov 2012 15:48:19 -0600

If I were a good AD, I'd have been monitoring this all along and would 
have given a long heads up, before IETF Last Call. If I were a good AD, 
I would have asked for an Apps Directorate review. Now, Last Call 
complete and less than 19 hours before an IESG Telechat, I come to you, 
head hung in shame. Bad Pete.

https://datatracker.ietf.org/doc/draft-ietf-mptcp-api/ is on tomorrow's 
telechat. I love MPTCP and think it is exactly the right way to deal 
with multiple interfaces, but some of the stuff in this particular 
document makes me nervous. Unfortunately, I have not written any code to 
a sockets API in too long. I have already bugged two people for comments 
(thanks Chris and Cyrus), but I wouldn't mind more people looking at 
this document. It's only Informational, but I am quite sure people will 
start coding this stuff. I'm personally more worried about the legacy 
API interactions (see section 4.2.2 in particular), but if you have 
thoughts on the new APIs as well, they would be welcome.

I'm going to be writing up my comments, but at this point I'm not 
planning on asking to DISCUSS it. But if anyone has concerns where you 
think, "Holy crap! You can't let people start coding that, even if it is 
just experimental!", you've got not much time to let me know.

Begging your forgiveness,

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Ned Freed <[email-masked]>
Date: Wed, 14 Nov 2012 19:06:32 -0800 (PST)

I noted this text:

   As this address may not be valid any more if the first subflow is
   closed, the MPTCP stack MAY close the whole MPTCP connection if the
   first subflow is closed (i.e. fate sharing between the initial
   subflow and the MPTCP connection as a whole).  Whether to close the
   whole MPTCP connection by default SHOULD be controlled by a local
   policy.  Further experiments are needed to investigate its
   implications.

Presumably this "local policy" would be set system-wide? If so, that's a poor
choice. Such choices need to be able to be made at a finer granularity than
this, because in many cases not all connections a system handles have the same
security considerations.

The lack of user control for things like this has rendered other protocols
a lot less useful than they might otherwise have been. 

				Ned

---

From: Pete Resnick <[email-masked]>
Date: Wed, 14 Nov 2012 21:35:57 -0600

> a lot less useful than they might otherwise have been.

Well, I'd certainly hope in the new APIs, you would be able to set an 
option to do this. (I expect this would be in the "Advanced API", which 
this document doesn't lay out.) But for the legacy API, which this 
paragraph is talking about, I don't see how it could be other than 
system-wide, since the legacy app is unaware that there's MPTCP going on 
underneath.

That said, I'm not sure why the suggestion is in there in the first 
place. Under what circumstances would anyone want fate-sharing of the 
first subflow?

Still finishing up my own review,

pr

---

From: Ned Freed <[email-masked]>
Date: Thu, 15 Nov 2012 03:37:56 -0800 (PST)

> system-wide, since the legacy app is unaware that there's MPTCP going on

Lots of ways: The obvious one is a sticky per-port default.

> place. Under what circumstances would anyone want fate-sharing of the

Because many applications make use of the remote IP address for various
reasons, and need to see the original address that was used. 

Really, this alone looks to me like a recipe for systems enabling MPTCP,
seeing a bunch of wierd random failures, and as a result disabling it
permanently.

				Ned

---

From: Pete Resnick <[email-masked]>
Date: Thu, 15 Nov 2012 09:19:21 -0600

On 11/15/12 5:37 AM, Ned Freed wrote:

>> Under what circumstances would anyone want fate-sharing of the
>> first subflow?
>

As proposed, the application *does* get to see the original address that 
was used, fate-sharing or not. I just don't understand why an 
application would care, on a connection that is still up and running, 
whether the remote end is still using that same remote IP address.

Example:

- I get a connection from 1.2.3.4. I note the remote IP address and am 
happy with it.
- MPTCP adds a subflow on my connection from 5.6.7.8. We continue talking.
- Remote side shuts down it's interface on 1.2.3.4 (walked out of 
Starbucks). 5.6.7.8 interface is still operational.

Scenario 1. Non-fate-shared: Connection continues from 5.6.7.8 to me. My 
OS reports back from getpeername() that I am connected to 1.2.3.4, even 
though that particular subflow is now gone.

Scenario 2. Fate-shared: Connection gets closed because the original 
subflow died.

Why would I ever want Scenario 2?

> seeing a bunch of wierd random failures, and as a result disabling it

If fate-sharing is enabled, that's exactly what I would expect. Are you 
saying that you think this will happen if fate-sharing is off?

pr

---

From: Marc Blanchet <[email-masked]>
Date: Thu, 15 Nov 2012 10:31:11 -0500

to me, it remains to be seen how the interaction of:
- happy eyeballs
- IPv4 vs IPv6 address family preference
- application using a non-80 port and then falling back to port 80/443 because of outgoing port blocking in some access networks (I wrote a draft on that)
- multiple interfaces which are pretty common these days, such as wifi and cell data-3G-LTE-...

all of those are common cases today,  many with specific APIs. Then you add MP-TCP on top of that.

I fail to see how those interactions will work. Each of them, taken separately, does a good job for its own purpose. but the common case shall include all of these. Not sure the underlying OS could do all the job for the APP, too.

so, no clear suggestion, but a concern about complexity of apps and how well these techniques will interact and work each other.

Marc.

Le 2012-11-15 à 10:19, Pete Resnick a écrit :

> As proposed, the application *does* get to see the original address that was used, fate-sharing or not. I just don't understand why an application would care, on a connection that is still up and running, whether the remote end is still using that same remote IP address.
> 
> Example:
> 
> - I get a connection from 1.2.3.4. I note the remote IP address and am happy with it.
> - Remote side shuts down it's interface on 1.2.3.4 (walked out of Starbucks). 5.6.7.8 interface is still operational.
> 
> Scenario 1. Non-fate-shared: Connection continues from 5.6.7.8 to me. My OS reports back from getpeername() that I am connected to 1.2.3.4, even though that particular subflow is now gone.
> 
> Scenario 2. Fate-shared: Connection gets closed because the original subflow died.
> 
> If fate-sharing is enabled, that's exactly what I would expect. Are you saying that you think this will happen if fate-sharing is off?
> 
> pr
> 
> -- 
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Ned Freed <[email-masked]>
Date: Thu, 15 Nov 2012 07:50:27 -0800 (PST)

My mistake, sorry. I misread the text on how this interacts with getpeername.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Thu, 15 Nov 2012 08:08:37 -0800

> policy.  Further experiments are needed to investigate its
...
> The lack of user control for things like this has rendered other
> protocols a lot less useful than they might otherwise have been.

On its face, the choice to default to closing an entire session because
one of several channels goes away seems exactly wrong to me.

I would think that a major motivating factor for a multi-channel TCP is
robustness against an outage.  What the current normative language does
is to encourage making multi-channel TCP as fragile as single-channel.

Or have I missed something basic?

d/

-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: Claudio Allocchio <[email-masked]>
Date: Thu, 15 Nov 2012 20:22:34 +0100 (CET)

> is to encourage making multi-channel TCP as fragile as single-channel.

yes, but we shall also point out clearly all potential 'problems' that can 
exist with MCTCP vs TCP. Not all appications are the same with respect to 
this change.

I indeed decided to examine this set of issues in my review (it will 
come... just after 9pm GMT!) as most of the other issues have already been 
clearly discussed on the apps=discuss list already.

Just a set of "caveats", nothing more...

> Or have I missed something basic?

------------------------------------------------------------------------------
Claudio Allocchio             G   A   R   R          [email-masked]
                         Senior Technical Officer
tel: +39 040 3758523      Italian Academic and       G=Claudio; S=Allocchio;
fax: +39 040 3758565        Research Network         P=garr; A=garr; C=it;

            PGP Key: http://www.cert.garr.it/PGP/keys.php3#ca

---

From: John Leslie <[email-masked]>
Date: Fri, 16 Nov 2012 23:30:28 -0500

Marc Blanchet <[email-masked]> wrote:
> - application using a non-80 port and then falling back to port
>   80/443 because of outgoing port blocking in some access networks
>   (I wrote a draft on that)
> I fail to see how those interactions will work. Each of them, taken
> separately, does a good job for its own purpose. but the common case
> shall include all of these. Not sure the underlying OS could do all
> the job for the APP, too.
> 
> so, no clear suggestion, but a concern about complexity of apps and
> how well these techniques will interact and work each other.

   I think this deserves some discussion (and I was hoping someone else
would start it. ;^)

   Understand, first, that MPTCP can only use multiple paths if

- one TCP connection is established with MP_CAPABLE negotiated by
  three-way handshake with the MP_CAPABLE TCP option (this presumably
  will be quite common, eventually);

- at least one endpoint originates another TCP connection with MP_JOIN
  (instead of MP_CAPABLE) negotiated by three-way handshake with the
  MP_JOIN TCP option.

(i.e. without three MP_CAPABLE TCP options, followed by three MP_JOIN
options, there can be no multiple paths involved.)

   The MP_CAPABLE handshake exchanges keys. If for any reason the
MP_JOIN reaches a different endpoint, the keys will not match.

   In the simple case, the host originating the MP_JOIN will send it to
the same remote address, but from a different interface using a different
source IP address. But there's also a way to advise the other endpoint
of another local address, using ADD_ADDR.

   With ADD_ADDR it is possible to enable multiple paths using both
IPv4 and IPv6, as well as different ports, though some implementations
may not support this, and even if supported by the endpoints, things
can go wrong in the middle.

   So, let me try to chip away at the cases Marc asked about...

> - happy eyeballs

   Happy eyeballs is implemented in different ways, but generally
overlaps an attempt to establish an IPv6 connection with an attempt to
establish an IPv4 connection, discarding the one which doesn't complete
first.

   Both attempted connections are likely to carry the MP_CAPABLE option,
and may or may not succeed in negotiating it. But only one will be used.
Attempting an MP_JOIN must happen later, though I suppose ADD_ADDRs
could be exchanged so that the other IP version could be tried for a
secondary path -- that probably won't be common -- and even with that
there would be no re-use of the connection which finished second.

> - IPv4 vs IPv6 address family preference

   Same as "happy eyeballs". (Note that ADD_ADDRs would need to be
exchanged to make use of both address families.)

> - application using a non-80 port and then falling back to port 
>   80/443 because of outgoing port blocking in some access networks 

   Really the same, also; though different ports within the same
address families are easier to implement.

>   wifi and cell data-3G-LTE-...

   It would be quite practical to establish the initial TCP connection
on a more expensive service; then do MP_JOINs on the less expensive
service, essentially dropping the more expensive service so long as
the less expensive one keeps working.

   Note also that MPTCP _intends_ to keep the same MPTCP connection
(group) functioning as an interface gets different IP addresses.

====

   (I have only answered for what the protocol allows for. YMMV with
differing implementations.)

   The draft named in this thread covers only a basic API, which
could give the endpoint application(s) better control over various
tradeoffs involved in the internals of an MPTCP implementation. The
protocol itself is found in draft-ietf-mptcp-multiaddressed, which
is already approved for publication as an Experimental track RFC.

--
John Leslie <[email-masked]>