From michawe@ifi.uio.no Sun Apr 3 02:18:11 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 007623A659A for ; Sun, 3 Apr 2011 02:18:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KKC57e8JKxFe for ; Sun, 3 Apr 2011 02:18:10 -0700 (PDT) Received: from mail-out1.uio.no (mail-out1.uio.no [129.240.10.57]) by core3.amsl.com (Postfix) with ESMTP id 614F93A676A for ; Sun, 3 Apr 2011 02:18:10 -0700 (PDT) Received: from mail-mx5.uio.no ([129.240.10.46]) by mail-out1.uio.no with esmtp (Exim 4.72) (envelope-from ) id 1Q6JTK-0003Dg-Sg for multipathtcp@ietf.org; Sun, 03 Apr 2011 11:19:50 +0200 Received: from cm-84.208.175.27.getinternet.no ([84.208.175.27] helo=[192.168.0.199]) by mail-mx5.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user michawe (Exim 4.72) (envelope-from ) id 1Q6JTK-0005FJ-EW for multipathtcp@ietf.org; Sun, 03 Apr 2011 11:19:50 +0200 Message-Id: From: Michael Welzl To: multipathtcp@ietf.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v936) Date: Sun, 3 Apr 2011 11:19:28 +0200 X-Mailer: Apple Mail (2.936) X-UiO-Ratelimit-Test: rcpts/h 3 msgs/h 3 sum rcpts/h 5 sum msgs/h 4 total rcpts 8178 max rcpts/h 36 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 054843B70C7A0F3C81C849C0FD2EAA79D96F30B9 X-UiO-SPAM-Test: remote_host: 84.208.175.27 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 3 total 854 max/h 18 blacklist 0 greylist 0 ratelimit 0 Subject: [multipathtcp] About my review of draft-ietf-mptcp-multiaddressed-03 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 09:18:11 -0000 Hi, At the WG meeting, I went to the mic and volunteered to review draft- ietf-mptcp-multiaddressed-03. On the way home, my colleague Stein Gjessing and I agreed that we would prefer to swap this task, i.e. have him do it instead of me. I hope this is ok with the group... Cheers, Michael From steing@ifi.uio.no Sun Apr 3 02:59:12 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1CE0A3A6975 for ; Sun, 3 Apr 2011 02:59:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ay1u9VawGRFV for ; Sun, 3 Apr 2011 02:59:11 -0700 (PDT) Received: from mail-out2.uio.no (mail-forward2.uio.no [IPv6:2001:700:100:10::71]) by core3.amsl.com (Postfix) with ESMTP id 029C43A6784 for ; Sun, 3 Apr 2011 02:59:10 -0700 (PDT) Received: from mail-mx2.uio.no ([129.240.10.30]) by mail-out2.uio.no with esmtp (Exim 4.74) (envelope-from ) id 1Q6K70-0006ka-Rd for multipathtcp@ietf.org; Sun, 03 Apr 2011 12:00:50 +0200 Received: from 16.79-161-30.customer.lyse.net ([79.161.30.16] helo=[10.0.1.3]) by mail-mx2.uio.no with esmtpsa (TLSv1:AES128-SHA:128) user steing (Exim 4.72) (envelope-from ) id 1Q6K70-0003Rv-Bo; Sun, 03 Apr 2011 12:00:50 +0200 From: Stein Gjessing Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Sun, 3 Apr 2011 12:01:34 +0200 Message-Id: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> To: multipathtcp@ietf.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) X-UiO-Ratelimit-Test: rcpts/h 2 msgs/h 1 sum rcpts/h 3 sum msgs/h 2 total rcpts 4264 max rcpts/h 96 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, TVD_RCVD_IP=0.001, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: 614A52D9ADABBEA22AFF077EF09BB31A6DCCD395 X-UiO-SPAM-Test: remote_host: 79.161.30.16 spam_score: -49 maxlevel 99990 minaction 1 bait 0 mail/h: 1 total 446 max/h 8 blacklist 0 greylist 0 ratelimit 0 Subject: [multipathtcp] draft-ietf-mptcp-congestion-02 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 09:59:12 -0000 Hi, I have read draft-ietf-mptcp-congestion-02, without really trying to = understand the algorithm. Anyhow I think I found some inconsistencies = in the text, and I also have a few suggestions for easier readability. 1. The term "path" is used so much in this paper that I have a problem = with it in Goal 2: o Goal 2 (Do no harm) A multipath flow should not take up more capacity on any one of its paths than if it was a single path flow using only that route. This guarantees it will not unduly harm other flows. I mean it would be more precise (and even more important, easier to = understand) if you write: o Goal 2 (Do no harm) A multipath flow should not take up more capacity from any of the resources shared by its different paths, = than if it was a single flow using only one of these paths. This guarantees it will not unduly = harm other flows. 2. It is strange that you write (notice "the same bandwidth"): Our solution sets the multipath flow's aggregate bandwidth to be the same bandwidth a regular TCP flow would get on the best path available to the multipath flow when you in the next paragraph write (notice "the multipath flow = throughput will be strictly higher than"): We note that in cases with low statistical multiplexing (where the multipath flow influences the loss rates on the path) the multipath throughput will be strictly higher than a single TCP would get on any of the paths. In particular, if using two idle paths, multipath throughput will be sum of the two paths' throughput. I think you should modify or quantify the statement about "the same = bandwidth" in the former paragraph. The latter paragraph also conflicts with a later paragraph (notice = "equal to"): "alpha" is a parameter of the algorithm that describes the aggresiveness of the multipath flow. To meet Goal 1 (improve throughput), the value of alpha is chosen such that the aggregate throughput of the multipath flow is equal to the rate a TCP flow would get if it ran on the best path. 3.=20 Before the definition of alpha at the end of section 3, you write: Hence, alpha must be computed for each multipath flow, based on the observed properties of the paths. It is very easy to read this as "for each multipath subflow", and then = think there should be an alpha_i. I suggest you remove "for each" , and write simply: Hence, alpha must be computed based on the observed properties of = the paths. Stein From scott.brim@gmail.com Sun Apr 3 05:19:13 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 511713A67DA for ; Sun, 3 Apr 2011 05:19:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.593 X-Spam-Level: X-Spam-Status: No, score=-103.593 tagged_above=-999 required=5 tests=[AWL=0.006, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jklVOMLnHlZF for ; Sun, 3 Apr 2011 05:19:11 -0700 (PDT) Received: from mail-iy0-f172.google.com (mail-iy0-f172.google.com [209.85.210.172]) by core3.amsl.com (Postfix) with ESMTP id 73D4B3A67D8 for ; Sun, 3 Apr 2011 05:19:11 -0700 (PDT) Received: by iye19 with SMTP id 19so5937183iye.31 for ; Sun, 03 Apr 2011 05:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=FAKRgozRBC5Y+k4rhndXPBJaARwi7k2hEo1Gw27Jalk=; b=lhAyn6dweJEgZGwnoX0r99wxZHkkitt5Xhql/9XY0ikiuOrT8YrMWjNdHdnXbGFAtZ z/Z4PkGbElwreWRzV23A+isOWfAzQd3TTNwSbq9s9Tuo+6jkBPQn369HzqE6tEXzDNRK purHOetRoq4KVJknXGHHguZM1310kcJTuR36U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=l2GHIpFCdHRlumejDvqbb6/kPKQLI2OIdsqxsWmb6rz5+h++r90IYplVCwWh4H3s1i 8NhiYzLH6JO7hM1USrSyoylulczHzLvn05YzG6dB37u9oFXvT9YdEQvixv1fSWHpQ+VS qjXaPnQ0Pbr8arCqVXawktSV/rodoBauAaoK8= Received: by 10.43.43.197 with SMTP id ud5mr6328502icb.147.1301833253044; Sun, 03 Apr 2011 05:20:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.42.225.133 with HTTP; Sun, 3 Apr 2011 05:20:33 -0700 (PDT) In-Reply-To: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> References: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> From: Scott Brim Date: Sun, 3 Apr 2011 08:20:33 -0400 Message-ID: To: Stein Gjessing Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] draft-ietf-mptcp-congestion-02 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 12:19:13 -0000 On Sun, Apr 3, 2011 at 06:01, Stein Gjessing wrote: > Hi, > > I have read =A0draft-ietf-mptcp-congestion-02, without really trying to u= nderstand the algorithm. =A0Anyhow I think I found some inconsistencies in = the text, and I also have a few suggestions for easier readability. > > 1. > The term "path" is used so much in this paper that I =A0have a problem wi= th it in Goal 2: > > =A0o =A0Goal 2 (Do no harm) A multipath flow should not take up more > =A0 =A0 =A0capacity on any one of its paths than if it was a single path = flow > =A0 =A0 =A0using only that route. =A0This guarantees it will not unduly h= arm > =A0 =A0 =A0other flows. > > I mean it would be more precise (and even more important, easier to under= stand) if you write: > > =A0 o =A0Goal 2 (Do no harm) A multipath flow should not take up more > =A0 =A0 =A0capacity from any of the resources shared by its different pat= hs, than if it was a single flow > =A0 =A0 =A0using only one of these paths. =A0This guarantees it will not = unduly harm > =A0 =A0 =A0other flows. Why should MPTCP cripple itself like this? The requirement is that MPTCP do no more harm to any individual TCP connection than if it were a single TCP connection itself, and a TCP connection only follows (and thus can only experience 'harm' on) a single path. Thanks ... Scott From sebastien.barre@uclouvain.be Mon Apr 4 06:19:41 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2AAF63A693B for ; Mon, 4 Apr 2011 06:19:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.299 X-Spam-Level: X-Spam-Status: No, score=-6.299 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m0EjqNuBN+cb for ; Mon, 4 Apr 2011 06:19:35 -0700 (PDT) Received: from smtp4.sgsi.ucl.ac.be (smtp.sgsi.ucl.ac.be [130.104.5.67]) by core3.amsl.com (Postfix) with ESMTP id A74933A6979 for ; Mon, 4 Apr 2011 06:19:34 -0700 (PDT) Received: from [192.168.1.102] (unknown [83.101.48.28]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sbarre@smtp4.sgsi.ucl.ac.be) by smtp4.sgsi.ucl.ac.be (Postfix) with ESMTPSA id D4FB6F28A7; Mon, 4 Apr 2011 15:20:59 +0200 (CEST) Message-ID: <4D99C5BA.7030608@uclouvain.be> Date: Mon, 04 Apr 2011 15:20:58 +0200 From: =?ISO-8859-1?Q?S=E9bastien_Barr=E9?= User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); nl; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: multipathtcp@ietf.org References: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: clamav-milter 0.97-exp at smtp-4.sipr-dc.ucl.ac.be X-Virus-Status: Clean Received-SPF: Pass (sender authenticated); receiver=; client-ip=83.101.48.28; helo=[192.168.1.102] Received-SPF: Pass (sender authenticated); receiver=; client-ip=83.101.48.28; envelope-from= X-Sgsi-Spamcheck: SASL authenticated, X-SGSI-MailScanner-ID: D4FB6F28A7.00000 X-SGSI-MailScanner: Found to be clean X-SGSI-From: sebastien.barre@uclouvain.be X-SGSI-Spam-Status: No Cc: Scott Brim Subject: Re: [multipathtcp] draft-ietf-mptcp-congestion-02 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 13:19:41 -0000 Hi, Here are some comments for Section 4: *=== alpha*bytes_acked*mss_i/tot_cwnd/alpha_scale === I guess you meant (alpha*bytes_acked*mss_i/tot_cwnd)/alpha_scale. This is logical from the context, but the current writing is ambiguous if read alone. *Now in section 4.1: === In the multipath case, cwnd_cnt_i is maintained for each subflow as above, and cwnd_i is increased by 1 when cwnd_cnt_i> alpha_scale * tot_cwnd / alpha. === This does not correspond to the formula in section 3. IMO, it should be: "Increment cwnd_i by 1 when: cwnd_cnt_i > max(alpha_scale*tot_cwnd*mss_i/alpha, cwnd_i*mss_i)" Note that this solves two mistakes: One is that the max() should be used, the other is that we should multiply by mss_i on the right size to get byte units on both sizes of the inequality. *Finally a typo: Just before section 5: cwnd_max*mss_max*mss_max -> cwnd_max*mss_max regards, Sébastien. Op 03-04-11 14:20, Scott Brim schreef: > On Sun, Apr 3, 2011 at 06:01, Stein Gjessing wrote: >> Hi, >> >> I have read draft-ietf-mptcp-congestion-02, without really trying to understand the algorithm. Anyhow I think I found some inconsistencies in the text, and I also have a few suggestions for easier readability. >> >> 1. >> The term "path" is used so much in this paper that I have a problem with it in Goal 2: >> >> o Goal 2 (Do no harm) A multipath flow should not take up more >> capacity on any one of its paths than if it was a single path flow >> using only that route. This guarantees it will not unduly harm >> other flows. >> >> I mean it would be more precise (and even more important, easier to understand) if you write: >> >> o Goal 2 (Do no harm) A multipath flow should not take up more >> capacity from any of the resources shared by its different paths, than if it was a single flow >> using only one of these paths. This guarantees it will not unduly harm >> other flows. > Why should MPTCP cripple itself like this? The requirement is that > MPTCP do no more harm to any individual TCP connection than if it were > a single TCP connection itself, and a TCP connection only follows (and > thus can only experience 'harm' on) a single path. > > Thanks ... Scott > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp -- Sébastien Barré Researcher, CSE department, UCLouvain, Belgium http://inl.info.ucl.ac.be/sbarre From Michael.Tuexen@lurchi.franken.de Fri Apr 8 12:05:50 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 409B73A69D1 for ; Fri, 8 Apr 2011 12:05:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.95 X-Spam-Level: X-Spam-Status: No, score=-1.95 tagged_above=-999 required=5 tests=[AWL=0.349, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z3XBONWEXVu6 for ; Fri, 8 Apr 2011 12:05:49 -0700 (PDT) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) by core3.amsl.com (Postfix) with ESMTP id ECE4D3A6962 for ; Fri, 8 Apr 2011 12:05:48 -0700 (PDT) Received: from [192.168.1.192] (p508F9C39.dip.t-dialin.net [80.143.156.57]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 382BB1C0B4606; Fri, 8 Apr 2011 21:07:32 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: =?iso-8859-1?Q?Michael_T=FCxen?= In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D680C8AE5A7@rsys005a.comm.ad.roke.co.uk> Date: Fri, 8 Apr 2011 21:07:30 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20110314120002.16130.75450.idtracker@localhost> <2181C5F19DD0254692452BFF3EAF1D680C8AE5A7@rsys005a.comm.ad.roke.co.uk> To: "Ford, Alan" X-Mailer: Apple Mail (2.1084) Cc: mptcp Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 19:05:50 -0000 Hi Alan, sorry for the late response... See my comments in-line. Best regards Michael On Mar 30, 2011, at 1:27 PM, Ford, Alan wrote: > Hi Michael, >=20 > Many thanks for your review. Comments inline... >=20 >> -----Original Message----- >> From: multipathtcp-bounces@ietf.org [mailto:multipathtcp- >> bounces@ietf.org] On Behalf Of Michael T=FCxen >> Sent: 30 March 2011 12:04 >> To: mptcp >> Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt >>=20 >> Dear all, >>=20 >> I read the ID and have the following comments: >>=20 >> * I'm not sure why you need REQ4. In SCTP we do have this for >> 1-to-many style sockets, but I do not see that you use this >> for MPTCP. On 1-to-1 style sockets, it is not needed for >> SCTP at least. >=20 > The logic for REQ4 (connection ID) is explained in the text below that = list: >=20 > Finally, it should be possible for the > application to retrieve a unique connection identifier (local to the > endpoint on which it is running) for the MPTCP connection. This is > equivalent to using the (address, port) pair for a connection=20 > identifier in single-path TCP, which is no longer static in MPTCP. >=20 > Since there is no longer a single, static (address, port) pair for a = connection, we should provide a static connection ID for MPTCP-aware = applications to track its connections. Hmm. I'm confused. On section 4.2.2 it is stated that for backwards = compatibility getsockname() and getpeername() return something unique for the MP-TCP = connection. So why don't you do this always? >=20 >> * Section 5.3.1: Why can't you use TCP_MUTLIPATH_REMOVE only after >> connection setup? >=20 > Not sure there's much of a call for it, but I guess there's no reason = to prevent its use in address list management before setting up a = connection. Assume I bind 3 addresses to a listening socket. Lateron I can remove = one of the addresses. If then a connection is accepted, only two addresses are used. At least this works on SCTP.... >=20 >> * Table 1: What is a "list of addresses"? >=20 > An abstract data structure consisting of a list of IP (v4 and/or v6) = addresses (and optionally ports). >=20 > Do you consider it necessary to be more explicit? The data structures = are explained in more detail in the following sections. When I would implement it, I would need a bit more specific information. Do you really mean a list (some strucuture with a next pointer)? Do you mean an array of struct sockaddr_in structures (for an AF_INET = socket), what is the number of addresses? Do you means an array of sockaddr_in6 structures (for an AF_INET6) socket? What is the number of addresses? Or do you use an array of sockaddr_storage structures? Is a mixture of sockaddr_in and sockaddr_in6 structures allowed? What about mapped addresses? >=20 >> * Table 1: What is a "list of pairs of addresses"? >=20 > See above, but this time with pairs within the list :) Same problem as above? >=20 >> * Section 5.3.5: Why is TCP_MULTIPATH_CONNID needed? >=20 > See my response to REQ4 above. >=20 >> * Section 6.1: Are you suggesting that sctp_bindx(), sctp_connectx(), >> sctp_getladdrs(), sctp_getpaddrs() are also provided for MPTCP? >> This sounds strange. >> Maybe one can introduce bindx(), connectx(), getladdrs(), = getpaddrs() >> which supports SCTP and MPTCP. Kacheong suggested that a while ago. >> Maybe it is time to do that. >> Please note that you missed sctp_freeladdrs() and sctp_freepaddrs(). >=20 > I'll leave Michael to answer that one :) >=20 > Cheers, > Alan >=20 >=20 From philip.eardley@bt.com Fri Apr 8 12:10:08 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 286B43A6975 for ; Fri, 8 Apr 2011 12:10:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.625 X-Spam-Level: X-Spam-Status: No, score=-101.625 tagged_above=-999 required=5 tests=[AWL=-0.179, BAYES_00=-2.599, HELO_MISMATCH_COM=0.553, J_CHICKENPOX_21=0.6, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MJtPYimLETYa for ; Fri, 8 Apr 2011 12:10:06 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp61.intersmtp.COM [62.239.224.234]) by core3.amsl.com (Postfix) with ESMTP id 2F4B93A69E2 for ; Fri, 8 Apr 2011 12:10:06 -0700 (PDT) Received: from EVMHT64-UKRD.domain1.systemhost.net (10.36.3.101) by RDW083A005ED61.smtp-e1.hygiene.service (10.187.98.10) with Microsoft SMTP Server (TLS) id 8.3.106.1; Fri, 8 Apr 2011 20:11:51 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.166]) by EVMHT64-UKRD.domain1.systemhost.net ([10.36.3.101]) with mapi; Fri, 8 Apr 2011 20:11:50 +0100 From: To: Date: Fri, 8 Apr 2011 20:11:49 +0100 Thread-Topic: MPTCP WG meeting - draft minutes Thread-Index: AcvvykxpXGmnQP3CSF6JtJhuBZqj8AGTgtHw Message-ID: <9510D26531EF184D9017DF24659BB87F32902FA928@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [multipathtcp] MPTCP WG meeting - draft minutes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 19:10:08 -0000 Here's a draft of the MPTCP WG minutes for our Prague meeting. Please send = corrections as soon as possible Thanks very much to Olivier for the detailed notes! Phil & Yoshifumi. --- * Chairs. Review milestones wg doing pretty well two rfcs published 6181 and 6182 protocol : need reviewers to review the document implementation available check inl.info.ucl.ac.be/mptcp * Alan Ford : Protocol update of the changes to the protocol changes to wire format, but no fundamental change, revisions based on implementation experience effort to make document more readable for reviewers and implementers since 02 single option type, which changed all option formats data level sequence numbers and acks in single option data fin option decoupled from subflow Fin security solution discussed in interim meeting checksum changes single option format on slides data sequence signal big option, merge of data sequence data ack and data fin all in one format on slides F flag similar to FIN flag on TCP reason for DataFIN : corner case during connection close was possible with the previous design connection level fin must be acked before subflow level fin, but optimisation remains possible security solution logic similar to 02 based feedback reduced to 3 packets exchange with no use of payload sha-1 is specified but there are other bits for crypto agility in the futur= e see slide for security solution example MAC truncated on server-client direction but a false server a less significant security problem than a false client that tries to create a lot of subflow attempts other changes checksum optional clarified rules on duplicate acks for signalling appendix on tcp control block data structures heuristics not a lot of text on this now without large scale deployment, difficult to make clear recommendations where next document fairly stable In need of reviewers. We hope to have detailed reviews at this stage Michael Welzl : I volunteer [later: instead Michael's colleague, Stein Gjessing, will review] questions : nobody in the room open issues : lars eggert : don't have feedback, but is there another implementation besides ucl's implementation? If you are doing an implementation, please send feedback * Mark Handley : MPTCP Proxies Classifies as open issue doing some work on mptcp as a mobility solution during the bof, we showed a scenario with a client using 3G and Wifi to a server to supplement or replace the 3G for better performance/robustness/battery power opportunistically use Wifi base stations seemed worth exploring and we did some simulations still preliminary, short paper see url on slide you can get better throughput, better robustness and save battery could be a key part of a mobility solution minor problem : cannot depend on servers for mptcp in the future who will use mptcp in near future ? Solution mptcp proxy idea mobile client and proxy provided by e.g. 3G provider connect to proxy and proxy connects to server when mobile moves, uses proxy server knows nothing about mptcp not ideal to realy everything through the proxy all the time can we avoid proxy all time on path assume server mp-capable, proxy can learn Q: Lars Eggert why wouldn't the mobile open connection to both proxy and server MH : need to kill state=20 Scott Brim :=20 cool thing is mobile IP homeagent, winning combination you want an initial rendez-vous point anyway for privacy mh : not exactly the same role, it is similar, but proxy would need to behave differently with normal server and mptcp server Randall Stewart : there is IPR on this at a former employer, did similar thing for SCTP Scott Brim : had no idea this patent covered this, we'll ask later extremely excited to see this model, you get 3 for 1, grab people who like mobile ip, don't need route optimisation, get privacy and get optimisation with mptcp mh : one thing we need to do is to tell the proxy where to connect to and you need to do this in the syn would like to put an address in the syn, but we don't have space for this principles are easy, but in practice not easy to find the bytes should we try this on the first path in the protocol spec ? Adam ? , cisco: you seem to indicate mobility use case, but mptcp could be a general proxy, in such case all connections will go through the proxy. If the proxy knows that the server is mptcp capable and it does not need the flow anymore mh : it might want to relay data at the beginning of the subflow to reduce latency you could do after the syn, but looks difficult Adam : take it offline Tim Shepard : mptcp proxy is used for everything and you have a long term relationship and have state in both ends, why not tunnel the information to the proxy mh : gre would not go to nat Tim : TCP goes through NAT, could tunnel over TCP ? MH : maybe over UDP ? Scott : mobile ip Tim : if you have wifi access point that is causing problem, does mip work mh : there is nat on all these links Tim : tunnel mh : udp makes sense, but don't like udp Lars Eggert : assuming you could reuse some bits of the source ports, fragment id,s .. Alan Ford : data in the syn, wait for discussion in tcpm and work on spdy in google Tim Shepard : you know you are talking to a special box and you could use special acks to discuss with this box, think more evil MH : Do we want this in the normal spec ? Scott Brim : keep separate Lars Eggert : agree with Scott, don't hold the draft in the wg longer Tim Shepard : should be ? in the spec to encourage people, only used for mobile clients than want to use a proxy Alan Ford : don't know if this will be useful and wireless is uncontrolled Phil Eardley (chair) : keep it separate but there is interest in the room * Michael Scharf, API authors have corrected some minor issues minor updates some functions return now both ip addresses and port numbers instead of only ip addresses additional entries in the candidate list for an extended api additional text on approaches like happy eyeballs, mentioned as a possibility, not mandated, more a heuristic for protocol than an API issue discussion of comments received from Javier Ubillos, Michael Tuexen document pretty stable, please provide further feedback no questions Phil (chair) : there will be a wg last call after the next revision * Mark Handley, congestion control 3 reviews received for previous cc draft, mainly wording and interpretation bytes/packets stacks. draft has been updated to cover both and avoid rounding errors. packet based version assumes that mss is same on all paths and draft describes how to handle the case if mss are different. NSDI2011 paper describes the algo in details and its performance some results from datacenter simulations see slides mptcp is good at pushing traffic out of hotspot paths, better than application level distribution over different tcp connections measurements on amazon EC2 datacenter where there are multiple paths running over 24 h measurements show that when vms are on same machine or host, mptcp same as tcp over ecmp paths, mptpcp with 2 or 4 flows gets much better performance, 3x more coupled congestion control find the available capacity in the datacenter * S=E9bastien Barr=E9 : demonstration of Linux MPTCP implementation setup : two servers in a LAN with a programmable router that inserts delay or losses between the two servers, live demo, servers are running latest mptcp linux kernel implementation look at bandwidth when the delay or bandwidth or losses change in realtime two separate 100 Mbps links iperf running between the two servers, gets almost 200 Mbps at the application level add losses on one path 1% throughput drops on the affected link but not the other and application continues to work with reduced bandwidth coupled congestion control is going away from congested path same loss for both pass, two subflows get roughly same bandwidth add delay on one link, one subflow is affected for some time and then recovers failure : stop one link, the other is still working and iperf only sees reduced bandwidth link comes back, need some time to find the link up due to exponential backoff Question : Tecco Boot why did loss on the blue link caused a reduction in bandwidth on the green one Sebastien : related to retransmission that are done on the other flow, retransmission timeout may cause delays and fill receive buffer MH : think you are asking a different questions congstion control is linked between the two subflows and this explains why we both are equal loss, they get the same performance loss is modelling congestion Tim : all point is to move traffic away from congested path Tecco Boot : I would like to have a solution to react differently to losses and congestion. Looking for solutions taking traffic from satcom to 3G to wifi, mptcp could do some job there, you should use reliable link MH : How do you detect whether the loss is due to congestion or due to something else ? Without this info, we need to assume that they are caused by congestion and not by link errors Georg Hampel, Alcatel : demo is very nice. What happens if there is a large delay, up to 3 or 5 seconds Yoshifumi : how do you create loss and delay Sebastien : tc, artificial Sebastien set delay to 3 second strange result Michael Scharf : like this demo, but independent of mptcp design, did the same demo with another protocol, could do it with a different protocol - distribution of live CDs with the implementation * Sebastien : presentation of guidelines for implementers implementation status objective of the document is to explain the choices made in the implementation and why and also the things that were abandoned hope to include feedback on other implementations later besides the real demo, MPTCP implementation is running as well on Nokia N900 and Nexus who are running Linux explanation of architecture explanation of path management explanation of send queue management need same mss for all subflows explanation of receive queue management * Rolf Winter, autoconfiguring single-homed end-systems to make use of network multihoming how to allow single homed host use multipath ? scenario : dhcp server, gateway with multiple ISPs and DHCP server mptcp works with multiple addresses, option 1 give multiple ip addresses to host, but gateway needs to do source routing solution: not same thing as multipath proxy create virtual interface that sends a dhcp request, add to dhcp request an option that indicates from which range you want the address mainly configuration tweaking, 20 lines of scripting and running in a small testbed other ways to achieve same result are discussed in the slides is it useful ? Questions : ? for experiments this is useful if you have a lot of time, but for normal usage, this is interesting for IPv6, multihoming on a single homed host with multiple addresses. how does mptcp work in this environment Rolf: other scenarios are discussed in this document some discussion on ipv4 versus ipv6 Phil Eardley (chair) : show of hands : a dozen hands * Rechartering ideas doing well on our charter from slide : How much energy is there ? Advice on implementation and heuristics ? Support for singl-homed end systems applicability and experience alternative security alternative congestion control advanced api proxy for mobility other topics Mark Handley : intend to spend some time on heuristics section, there is a lot we don't know, need good ways Scott Brim : I consider the following topics applicability and experience alternative congestion control advanced api proxy for mobility I hope somebody would work on these, I want to work on proxy for mobility Tim Shepard : there should be a venue to do interop testing if there is a second implementation. nice to continue with a venue Phil : default option is to finish charter before the next ietf and pause while waiting for experiements and usage Alan Ford : We'll get more feedback as implementations progress lars Eggert : nobody else is doing an implementation or knows another implementation ? bof was extremely popular and few Sowmini Varadhan from oracle : working on multipath for solaris Sebastien : a dozen people are helping in debuging and four are providing patches, others are downloading merge in main linux tree is too early, need to stabilise and follow the draft Lars : be great if more people could contribute From prvs=3080fe5ae4=alan.ford@roke.co.uk Sat Apr 9 09:53:12 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 936E83A6819 for ; Sat, 9 Apr 2011 09:53:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.419 X-Spam-Level: X-Spam-Status: No, score=-3.419 tagged_above=-999 required=5 tests=[AWL=-0.120, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rbbG0Kuf6yPE for ; Sat, 9 Apr 2011 09:53:11 -0700 (PDT) Received: from gse-mta-29.emailfiltering.com (gse-mta-29-tx.emailfiltering.com [194.116.198.160]) by core3.amsl.com (Postfix) with ESMTP id 54F113A680A for ; Sat, 9 Apr 2011 09:53:11 -0700 (PDT) Received: from salt-ext.roke.co.uk ([109.207.29.2]) by gse-mta-29.emailfiltering.com with emfmta (version 4.8.0.437) vanilla id 340709560 for Michael.Tuexen@lurchi.franken.de; ea6ec73a28da2347; Sat, 09 Apr 2011 17:54:55 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Sat, 9 Apr 2011 17:54:30 +0100 Message-ID: <2181C5F19DD0254692452BFF3EAF1D680C9997EC@rsys005a.comm.ad.roke.co.uk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt Thread-Index: Acv2IDfqtdNANOeTTR6eYDaOkBV9dwAtLRiA References: <20110314120002.16130.75450.idtracker@localhost> <2181C5F19DD0254692452BFF3EAF1D680C8AE5A7@rsys005a.comm.ad.roke.co.uk> From: "Ford, Alan" To: =?iso-8859-1?Q?Michael_T=FCxen?= Cc: mptcp Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 16:53:12 -0000 Hi Michael, Responses inline... > On Mar 30, 2011, at 1:27 PM, Ford, Alan wrote: > > Hi Michael, > > > > Many thanks for your review. Comments inline... > > > >> -----Original Message----- > >> From: multipathtcp-bounces@ietf.org [mailto:multipathtcp- > >> bounces@ietf.org] On Behalf Of Michael T=FCxen > >> Sent: 30 March 2011 12:04 > >> To: mptcp > >> Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt > >> > >> Dear all, > >> > >> I read the ID and have the following comments: > >> > >> * I'm not sure why you need REQ4. In SCTP we do have this for > >> 1-to-many style sockets, but I do not see that you use this > >> for MPTCP. On 1-to-1 style sockets, it is not needed for > >> SCTP at least. > > > > The logic for REQ4 (connection ID) is explained in the text below = that > list: > > > > Finally, it should be possible for the > > application to retrieve a unique connection identifier (local to = the > > endpoint on which it is running) for the MPTCP connection. This = is > > equivalent to using the (address, port) pair for a connection > > identifier in single-path TCP, which is no longer static in MPTCP. > > > > Since there is no longer a single, static (address, port) pair for a > connection, we should provide a static connection ID for MPTCP-aware > applications to track its connections. > > Hmm. I'm confused. On section 4.2.2 it is stated that for backwards > compatibility > getsockname() and getpeername() return something unique for the MP-TCP > connection. >=20 > So why don't you do this always? No, 4.2.2 doesn't say it is unique for a connection. Whilst you can = ensure local uniqueness (getsockname()), you cannot guarantee that the = remote (addr, port) will be unique for the lifetime of a connection. The key point of the getsockname/getpeername debate was about whether an = address remains valid as a connection identifier when the address is = removed from the connection. The conclusion from the very extensive = discussions here was that we simply don't know if it will cause problems = for applications, so we will leave it open as to whether a connection = can stay open after the loss of the first subflow, and await = implementation lessons. If it can, then the information returned by the non-MPTCP-aware = functions of getsockname and getpeername may not be accurate at the = given instant in time. We want to get people away from using these = functions with inaccurate assumptions, and instead use the new API for = address querying. The other purpose of such functions today would be for unique connection = identification. A local connection identifier, decoupled from the = concept of addresses, seems ideal for this purpose - that's what REQ4 is = all about. > >> * Table 1: What is a "list of addresses"? > > > > An abstract data structure consisting of a list of IP (v4 and/or v6) > addresses (and optionally ports). > > > > Do you consider it necessary to be more explicit? The data = structures are > explained in more detail in the following sections. > > When I would implement it, I would need a bit more specific = information. > Do you really mean a list (some strucuture with a next pointer)? Do > you mean an array of struct sockaddr_in structures (for an AF_INET = socket), > what is the number of addresses? Do you means an array of sockaddr_in6 > structures (for an AF_INET6) socket? What is the number of addresses? > Or do you use an array of sockaddr_storage structures? Is a mixture > of sockaddr_in and sockaddr_in6 structures allowed? What about mapped > addresses? Feedback during IETF79 was strongly against specifying to this level of = detail (we had previously done so - see the pre-WG-adoption versions). = The IETF is not in a position to specify concrete APIs, so we specified = it to the level of abstract definition you see currently: this specifies = the kinds of data that need to be transferred only. The term "list" seemed (to me at least) to be sufficiently abstract to = allow its actual implementation and API to be left to library authors, = in whatever is consistent with the rest of their API. I realise the concerns you raise are valid ones, but are they things = that should be dealt with in this draft, or left to software authors? = Our previous feedback has been very much towards the latter option. Thanks, Alan From Michael.Tuexen@lurchi.franken.de Sat Apr 9 10:16:51 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9454A3A6876 for ; Sat, 9 Apr 2011 10:16:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.966 X-Spam-Level: X-Spam-Status: No, score=-1.966 tagged_above=-999 required=5 tests=[AWL=0.333, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DLUUCxY+SjOL for ; Sat, 9 Apr 2011 10:16:50 -0700 (PDT) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) by core3.amsl.com (Postfix) with ESMTP id 8D92B3A6819 for ; Sat, 9 Apr 2011 10:16:48 -0700 (PDT) Received: from [192.168.1.192] (p508FD7AB.dip.t-dialin.net [80.143.215.171]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 2D84B1C0B4606; Sat, 9 Apr 2011 19:18:32 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: =?iso-8859-1?Q?Michael_T=FCxen?= In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D680C9997EC@rsys005a.comm.ad.roke.co.uk> Date: Sat, 9 Apr 2011 19:18:31 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20110314120002.16130.75450.idtracker@localhost> <2181C5F19DD0254692452BFF3EAF1D680C8AE5A7@rsys005a.comm.ad.roke.co.uk> <2181C5F19DD0254692452BFF3EAF1D680C9997EC@rsys005a.comm.ad.roke.co.uk> To: "Ford, Alan" X-Mailer: Apple Mail (2.1084) Cc: mptcp Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 17:16:51 -0000 On Apr 9, 2011, at 6:54 PM, Ford, Alan wrote: > Hi Michael, >=20 > Responses inline... >=20 >> On Mar 30, 2011, at 1:27 PM, Ford, Alan wrote: >>> Hi Michael, >>>=20 >>> Many thanks for your review. Comments inline... >>>=20 >>>> -----Original Message----- >>>> From: multipathtcp-bounces@ietf.org [mailto:multipathtcp- >>>> bounces@ietf.org] On Behalf Of Michael T=FCxen >>>> Sent: 30 March 2011 12:04 >>>> To: mptcp >>>> Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt >>>>=20 >>>> Dear all, >>>>=20 >>>> I read the ID and have the following comments: >>>>=20 >>>> * I'm not sure why you need REQ4. In SCTP we do have this for >>>> 1-to-many style sockets, but I do not see that you use this >>>> for MPTCP. On 1-to-1 style sockets, it is not needed for >>>> SCTP at least. >>>=20 >>> The logic for REQ4 (connection ID) is explained in the text below = that >> list: >>>=20 >>> Finally, it should be possible for the >>> application to retrieve a unique connection identifier (local to = the >>> endpoint on which it is running) for the MPTCP connection. This is >>> equivalent to using the (address, port) pair for a connection >>> identifier in single-path TCP, which is no longer static in MPTCP. >>>=20 >>> Since there is no longer a single, static (address, port) pair for a >> connection, we should provide a static connection ID for MPTCP-aware >> applications to track its connections. >>=20 >> Hmm. I'm confused. On section 4.2.2 it is stated that for backwards >> compatibility >> getsockname() and getpeername() return something unique for the = MP-TCP >> connection. >>=20 >> So why don't you do this always? >=20 > No, 4.2.2 doesn't say it is unique for a connection. Whilst you can = ensure local uniqueness (getsockname()), you cannot guarantee that the = remote (addr, port) will be unique for the lifetime of a connection. >=20 > The key point of the getsockname/getpeername debate was about whether = an address remains valid as a connection identifier when the address is = removed from the connection. The conclusion from the very extensive = discussions here was that we simply don't know if it will cause problems = for applications, so we will leave it open as to whether a connection = can stay open after the loss of the first subflow, and await = implementation lessons. >=20 > If it can, then the information returned by the non-MPTCP-aware = functions of getsockname and getpeername may not be accurate at the = given instant in time. We want to get people away from using these = functions with inaccurate assumptions, and instead use the new API for = address querying. >=20 > The other purpose of such functions today would be for unique = connection identification. A local connection identifier, decoupled from = the concept of addresses, seems ideal for this purpose - that's what = REQ4 is all about. .. so you are saying that some applications might break on some implementations of MP-TCP. Right? >=20 >>>> * Table 1: What is a "list of addresses"? >>>=20 >>> An abstract data structure consisting of a list of IP (v4 and/or v6) >> addresses (and optionally ports). >>>=20 >>> Do you consider it necessary to be more explicit? The data = structures are >> explained in more detail in the following sections. >>=20 >> When I would implement it, I would need a bit more specific = information. >> Do you really mean a list (some strucuture with a next pointer)? Do >> you mean an array of struct sockaddr_in structures (for an AF_INET = socket), >> what is the number of addresses? Do you means an array of = sockaddr_in6 >> structures (for an AF_INET6) socket? What is the number of addresses? >> Or do you use an array of sockaddr_storage structures? Is a mixture >> of sockaddr_in and sockaddr_in6 structures allowed? What about mapped >> addresses? >=20 > Feedback during IETF79 was strongly against specifying to this level = of detail (we had previously done so - see the pre-WG-adoption = versions). The IETF is not in a position to specify concrete APIs, so we = specified it to the level of abstract definition you see currently: this = specifies the kinds of data that need to be transferred only. >=20 > The term "list" seemed (to me at least) to be sufficiently abstract to = allow its actual implementation and API to be left to library authors, = in whatever is consistent with the rest of their API. >=20 > I realise the concerns you raise are valid ones, but are they things = that should be dealt with in this draft, or left to software authors? = Our previous feedback has been very much towards the latter option. OK. However, this means that you can not build programs which can be ported from one MP-TCP implementation to another. Maybe you can state that it is not the goal of the document to specify an API which allow writing portable applications. I'm not sure about the above raised point regarding IETF and APIs. At = least http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket gives some kind of specification (the intended status is Informational) and it was very useful even as an ID, since most kernel implementations provide that interface. Best regards Michael >=20 > Thanks, > Alan >=20 >=20 From prvs=4081a32bc3=alan.ford@roke.co.uk Sun Apr 10 05:38:46 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4C2A33A69DE for ; Sun, 10 Apr 2011 05:38:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.399 X-Spam-Level: X-Spam-Status: No, score=-3.399 tagged_above=-999 required=5 tests=[AWL=-0.100, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IfbhPKV9FbP2 for ; Sun, 10 Apr 2011 05:38:45 -0700 (PDT) Received: from gse-mta-29.emailfiltering.com (gse-mta-29-tx.emailfiltering.com [194.116.198.160]) by core3.amsl.com (Postfix) with ESMTP id 6A21A3A6A12 for ; Sun, 10 Apr 2011 05:38:44 -0700 (PDT) Received: from salt-ext.roke.co.uk ([109.207.29.2]) by gse-mta-29.emailfiltering.com with emfmta (version 4.8.0.437) vanilla id 341039056 for Michael.Tuexen@lurchi.franken.de; 3a823a0a9c085926; Sun, 10 Apr 2011 13:40:28 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 x-cr-hashedpuzzle: EQ8b GoXM HPKS IU84 I2Z1 JwIN J2No Kx2a NU8y PXEA QBOU SM/q Sqpq St14 VBc4 WDrl; 2; bQBpAGMAaABhAGUAbAAuAHQAdQBlAHgAZQBuAEAAbAB1AHIAYwBoAGkALgBmAHIAYQBuAGsAZQBuAC4AZABlADsAbQB1AGwAdABpAHAAYQB0AGgAdABjAHAAQABpAGUAdABmAC4AbwByAGcA; Sosha1_v1; 7; {D357C00A-D171-4E39-AB67-742A57591A91}; YQBsAGEAbgAuAGYAbwByAGQAQAByAG8AawBlAC4AYwBvAC4AdQBrAA==; Sun, 10 Apr 2011 12:39:57 GMT; UgBFADoAIABbAG0AdQBsAHQAaQBwAGEAdABoAHQAYwBwAF0AIABJAC0ARAAgAEEAYwB0AGkAbwBuADoAZAByAGEAZgB0AC0AaQBlAHQAZgAtAG0AcAB0AGMAcAAtAGEAcABpAC0AMAAxAC4AdAB4AHQA MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable x-cr-puzzleid: {D357C00A-D171-4E39-AB67-742A57591A91} Content-class: urn:content-classes:message Date: Sun, 10 Apr 2011 13:39:57 +0100 Message-ID: <2181C5F19DD0254692452BFF3EAF1D680C9997F1@rsys005a.comm.ad.roke.co.uk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt Thread-Index: Acv22ifvHjqxXb+DQRCPeFEQRD+uYAAoWmUA References: <20110314120002.16130.75450.idtracker@localhost> <2181C5F19DD0254692452BFF3EAF1D680C8AE5A7@rsys005a.comm.ad.roke.co.uk> <2181C5F19DD0254692452BFF3EAF1D680C9997EC@rsys005a.comm.ad.roke.co.uk> From: "Ford, Alan" To: =?iso-8859-1?Q?Michael_T=FCxen?= Cc: mptcp Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Apr 2011 12:38:46 -0000 Hi Michael, > On Apr 9, 2011, at 6:54 PM, Ford, Alan wrote: >=20 > > Hi Michael, > > > > Responses inline... > > > >> On Mar 30, 2011, at 1:27 PM, Ford, Alan wrote: > >>> Hi Michael, > >>> > >>> Many thanks for your review. Comments inline... > >>> > >>>> -----Original Message----- > >>>> From: multipathtcp-bounces@ietf.org [mailto:multipathtcp- > >>>> bounces@ietf.org] On Behalf Of Michael T=FCxen > >>>> Sent: 30 March 2011 12:04 > >>>> To: mptcp > >>>> Subject: Re: [multipathtcp] I-D = Action:draft-ietf-mptcp-api-01.txt > >>>> > >>>> Dear all, > >>>> > >>>> I read the ID and have the following comments: > >>>> > >>>> * I'm not sure why you need REQ4. In SCTP we do have this for > >>>> 1-to-many style sockets, but I do not see that you use this > >>>> for MPTCP. On 1-to-1 style sockets, it is not needed for > >>>> SCTP at least. > >>> > >>> The logic for REQ4 (connection ID) is explained in the text below = that > >> list: > >>> > >>> Finally, it should be possible for the > >>> application to retrieve a unique connection identifier (local to = the > >>> endpoint on which it is running) for the MPTCP connection. This = is > >>> equivalent to using the (address, port) pair for a connection > >>> identifier in single-path TCP, which is no longer static in = MPTCP. > >>> > >>> Since there is no longer a single, static (address, port) pair for = a > >> connection, we should provide a static connection ID for = MPTCP-aware > >> applications to track its connections. > >> > >> Hmm. I'm confused. On section 4.2.2 it is stated that for backwards > >> compatibility > >> getsockname() and getpeername() return something unique for the = MP-TCP > >> connection. > >> > >> So why don't you do this always? > > > > No, 4.2.2 doesn't say it is unique for a connection. Whilst you can = ensure > local uniqueness (getsockname()), you cannot guarantee that the remote = (addr, > port) will be unique for the lifetime of a connection. > > > > The key point of the getsockname/getpeername debate was about = whether an > address remains valid as a connection identifier when the address is = removed > from the connection. The conclusion from the very extensive = discussions here > was that we simply don't know if it will cause problems for = applications, so > we will leave it open as to whether a connection can stay open after = the loss > of the first subflow, and await implementation lessons. > > > > If it can, then the information returned by the non-MPTCP-aware = functions > of getsockname and getpeername may not be accurate at the given = instant in > time. We want to get people away from using these functions with = inaccurate > assumptions, and instead use the new API for address querying. > > > > The other purpose of such functions today would be for unique = connection > identification. A local connection identifier, decoupled from the = concept of > addresses, seems ideal for this purpose - that's what REQ4 is all = about. > > .. so you are saying that some applications might break on some > implementations of MP-TCP. Right? The short answer is, we don't know. The long answer: it is conceivable that there could be applications that = make assumptions about the answers returned by = getsockname()/getpeername() and which under certain circumstances may = break. We do not know, and require experimental feedback to learn = whether this is a problem, and if so, what the best way of solving it is = (whilst maintaining the widest access to MPTCP benefits). This is = covered in RFC6182 as well as the API draft. However, this is orthogonal to your original question, asking about REQ4 = - the concise answer to which is that we are trying to get away from = assumptions about IP addresses being static since that is no longer = necessarily the case with MPTCP (while maintaining as much backwards = compatibility as possible). > >>>> * Table 1: What is a "list of addresses"? > >>> > >>> An abstract data structure consisting of a list of IP (v4 and/or = v6) > >> addresses (and optionally ports). > >>> > >>> Do you consider it necessary to be more explicit? The data = structures are > >> explained in more detail in the following sections. > >> > >> When I would implement it, I would need a bit more specific = information. > >> Do you really mean a list (some strucuture with a next pointer)? Do > >> you mean an array of struct sockaddr_in structures (for an AF_INET > socket), > >> what is the number of addresses? Do you means an array of = sockaddr_in6 > >> structures (for an AF_INET6) socket? What is the number of = addresses? > >> Or do you use an array of sockaddr_storage structures? Is a mixture > >> of sockaddr_in and sockaddr_in6 structures allowed? What about = mapped > >> addresses? > > > > Feedback during IETF79 was strongly against specifying to this level = of > detail (we had previously done so - see the pre-WG-adoption versions). = The > IETF is not in a position to specify concrete APIs, so we specified it = to the > level of abstract definition you see currently: this specifies the = kinds of > data that need to be transferred only. > > > > The term "list" seemed (to me at least) to be sufficiently abstract = to > allow its actual implementation and API to be left to library authors, = in > whatever is consistent with the rest of their API. > > > > I realise the concerns you raise are valid ones, but are they things = that > should be dealt with in this draft, or left to software authors? Our = previous > feedback has been very much towards the latter option. > > OK. However, this means that you can not build programs which can be > ported from one MP-TCP implementation to another. >=20 > Maybe you can state that it is not the goal of the document to specify > an API which allow writing portable applications. >=20 > I'm not sure about the above raised point regarding IETF and APIs. At = least > http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket > gives some kind of specification (the intended status is > Informational) and it was very useful even as an ID, since most > kernel implementations provide that interface. OK, so what do we do? We should certainly specify the scope at the = start, and I would like somehow to cover a more concrete specification. However, it was a condition of the document's WG adoption that we made = it more abstract than the original specification (similar to what you = are asking for). Dave Thaler and others spoke very strongly against the = level of concrete spec that you are asking for during IETF79. An alternative approach - would it be appropriate to include a more = detailed spec with proposed code snippets in an appendix? The danger, of = course, is that if the API is changed by other standards = bodies/implementers the appendix becomes irrelevant or even misleading. Regards, Alan From Michael.Tuexen@lurchi.franken.de Sun Apr 10 13:06:08 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B0D1C3A6959 for ; Sun, 10 Apr 2011 13:06:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.98 X-Spam-Level: X-Spam-Status: No, score=-1.98 tagged_above=-999 required=5 tests=[AWL=0.319, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C1PCp1WQxO64 for ; Sun, 10 Apr 2011 13:06:07 -0700 (PDT) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) by core3.amsl.com (Postfix) with ESMTP id 9EEEA3A6955 for ; Sun, 10 Apr 2011 13:06:06 -0700 (PDT) Received: from [192.168.1.192] (p508FB819.dip.t-dialin.net [80.143.184.25]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id C53C71C0B461C; Sun, 10 Apr 2011 22:06:03 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: =?iso-8859-1?Q?Michael_T=FCxen?= In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D680C9997F1@rsys005a.comm.ad.roke.co.uk> Date: Sun, 10 Apr 2011 22:06:02 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <456EAADE-0AD3-4BBC-B2D2-309050E53262@lurchi.franken.de> References: <20110314120002.16130.75450.idtracker@localhost> <2181C5F19DD0254692452BFF3EAF1D680C8AE5A7@rsys005a.comm.ad.roke.co.uk> <2181C5F19DD0254692452BFF3EAF1D680C9997EC@rsys005a.comm.ad.roke.co.uk> <2181C5F19DD0254692452BFF3EAF1D680C9997F1@rsys005a.comm.ad.roke.co.uk> To: "Ford, Alan" X-Mailer: Apple Mail (2.1084) Cc: mptcp Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Apr 2011 20:06:08 -0000 On Apr 10, 2011, at 2:39 PM, Ford, Alan wrote: > Hi Michael, >=20 >> On Apr 9, 2011, at 6:54 PM, Ford, Alan wrote: >>=20 >>> Hi Michael, >>>=20 >>> Responses inline... >>>=20 >>>> On Mar 30, 2011, at 1:27 PM, Ford, Alan wrote: >>>>> Hi Michael, >>>>>=20 >>>>> Many thanks for your review. Comments inline... >>>>>=20 >>>>>> -----Original Message----- >>>>>> From: multipathtcp-bounces@ietf.org [mailto:multipathtcp- >>>>>> bounces@ietf.org] On Behalf Of Michael T=FCxen >>>>>> Sent: 30 March 2011 12:04 >>>>>> To: mptcp >>>>>> Subject: Re: [multipathtcp] I-D = Action:draft-ietf-mptcp-api-01.txt >>>>>>=20 >>>>>> Dear all, >>>>>>=20 >>>>>> I read the ID and have the following comments: >>>>>>=20 >>>>>> * I'm not sure why you need REQ4. In SCTP we do have this for >>>>>> 1-to-many style sockets, but I do not see that you use this >>>>>> for MPTCP. On 1-to-1 style sockets, it is not needed for >>>>>> SCTP at least. >>>>>=20 >>>>> The logic for REQ4 (connection ID) is explained in the text below = that >>>> list: >>>>>=20 >>>>> Finally, it should be possible for the >>>>> application to retrieve a unique connection identifier (local to = the >>>>> endpoint on which it is running) for the MPTCP connection. This = is >>>>> equivalent to using the (address, port) pair for a connection >>>>> identifier in single-path TCP, which is no longer static in MPTCP. >>>>>=20 >>>>> Since there is no longer a single, static (address, port) pair for = a >>>> connection, we should provide a static connection ID for = MPTCP-aware >>>> applications to track its connections. >>>>=20 >>>> Hmm. I'm confused. On section 4.2.2 it is stated that for backwards >>>> compatibility >>>> getsockname() and getpeername() return something unique for the = MP-TCP >>>> connection. >>>>=20 >>>> So why don't you do this always? >>>=20 >>> No, 4.2.2 doesn't say it is unique for a connection. Whilst you can = ensure >> local uniqueness (getsockname()), you cannot guarantee that the = remote (addr, >> port) will be unique for the lifetime of a connection. >>>=20 >>> The key point of the getsockname/getpeername debate was about = whether an >> address remains valid as a connection identifier when the address is = removed >> from the connection. The conclusion from the very extensive = discussions here >> was that we simply don't know if it will cause problems for = applications, so >> we will leave it open as to whether a connection can stay open after = the loss >> of the first subflow, and await implementation lessons. >>>=20 >>> If it can, then the information returned by the non-MPTCP-aware = functions >> of getsockname and getpeername may not be accurate at the given = instant in >> time. We want to get people away from using these functions with = inaccurate >> assumptions, and instead use the new API for address querying. >>>=20 >>> The other purpose of such functions today would be for unique = connection >> identification. A local connection identifier, decoupled from the = concept of >> addresses, seems ideal for this purpose - that's what REQ4 is all = about. >>=20 >> .. so you are saying that some applications might break on some >> implementations of MP-TCP. Right? >=20 > The short answer is, we don't know. >=20 > The long answer: it is conceivable that there could be applications = that make assumptions about the answers returned by = getsockname()/getpeername() and which under certain circumstances may = break. We do not know, and require experimental feedback to learn = whether this is a problem, and if so, what the best way of solving it is = (whilst maintaining the widest access to MPTCP benefits). This is = covered in RFC6182 as well as the API draft. OK, I see. >=20 > However, this is orthogonal to your original question, asking about = REQ4 - the concise answer to which is that we are trying to get away = from assumptions about IP addresses being static since that is no longer = necessarily the case with MPTCP (while maintaining as much backwards = compatibility as possible). OK. >=20 >>>>>> * Table 1: What is a "list of addresses"? >>>>>=20 >>>>> An abstract data structure consisting of a list of IP (v4 and/or = v6) >>>> addresses (and optionally ports). >>>>>=20 >>>>> Do you consider it necessary to be more explicit? The data = structures are >>>> explained in more detail in the following sections. >>>>=20 >>>> When I would implement it, I would need a bit more specific = information. >>>> Do you really mean a list (some strucuture with a next pointer)? Do >>>> you mean an array of struct sockaddr_in structures (for an AF_INET >> socket), >>>> what is the number of addresses? Do you means an array of = sockaddr_in6 >>>> structures (for an AF_INET6) socket? What is the number of = addresses? >>>> Or do you use an array of sockaddr_storage structures? Is a mixture >>>> of sockaddr_in and sockaddr_in6 structures allowed? What about = mapped >>>> addresses? >>>=20 >>> Feedback during IETF79 was strongly against specifying to this level = of >> detail (we had previously done so - see the pre-WG-adoption = versions). The >> IETF is not in a position to specify concrete APIs, so we specified = it to the >> level of abstract definition you see currently: this specifies the = kinds of >> data that need to be transferred only. >>>=20 >>> The term "list" seemed (to me at least) to be sufficiently abstract = to >> allow its actual implementation and API to be left to library = authors, in >> whatever is consistent with the rest of their API. >>>=20 >>> I realise the concerns you raise are valid ones, but are they things = that >> should be dealt with in this draft, or left to software authors? Our = previous >> feedback has been very much towards the latter option. >>=20 >> OK. However, this means that you can not build programs which can be >> ported from one MP-TCP implementation to another. >>=20 >> Maybe you can state that it is not the goal of the document to = specify >> an API which allow writing portable applications. >>=20 >> I'm not sure about the above raised point regarding IETF and APIs. At = least >> http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket >> gives some kind of specification (the intended status is >> Informational) and it was very useful even as an ID, since most >> kernel implementations provide that interface. >=20 > OK, so what do we do? We should certainly specify the scope at the = start, and I would like somehow to cover a more concrete specification. >=20 > However, it was a condition of the document's WG adoption that we made = it more abstract than the original specification (similar to what you = are asking for). Dave Thaler and others spoke very strongly against the = level of concrete spec that you are asking for during IETF79. I see. If it is the WG consensus, I'm fine with it. >=20 > An alternative approach - would it be appropriate to include a more = detailed spec with proposed code snippets in an appendix? The danger, of = course, is that if the API is changed by other standards = bodies/implementers the appendix becomes irrelevant or even misleading. In the case of SCTP, developers from FreeBSD, Solaris and Linux are = coauthors. So what these implementation do is really described well in the ID. And = the experience is that having this document for years was very helpful. I'm not sure which other standards bodies will start working on such = things. Maybe POSIX, I don't know. It seams that you are aware of the points I wanted to raise and the WG decided to do it differently. This is fine with me. Best regards Michael >=20 > Regards, > Alan >=20 >=20 >=20 From c.raiciu@cs.ucl.ac.uk Mon Apr 11 04:24:39 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 290F03A6B0B for ; Mon, 11 Apr 2011 04:24:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NuEMyqhKzOeF for ; Mon, 11 Apr 2011 04:24:38 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id E4DFF3A6B0A for ; Mon, 11 Apr 2011 04:24:33 -0700 (PDT) Received: from [141.85.157.57] (helo=[192.168.1.129]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1Q9FCS-0003v2-3C; Mon, 11 Apr 2011 12:22:32 +0100 Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Costin Raiciu In-Reply-To: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> Date: Mon, 11 Apr 2011 14:24:26 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <5E6F459A-DD29-449B-96F3-DC5B47834C7A@cs.ucl.ac.uk> References: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> To: Stein Gjessing X-Mailer: Apple Mail (2.1081) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] draft-ietf-mptcp-congestion-02 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Apr 2011 11:24:39 -0000 Hi Stein, Thanks for all your comments - we've updated the draft to take them into = account. Costin On 3 Apr 2011, at 13:01, Stein Gjessing wrote: > Hi, >=20 > I have read draft-ietf-mptcp-congestion-02, without really trying to = understand the algorithm. Anyhow I think I found some inconsistencies = in the text, and I also have a few suggestions for easier readability. >=20 > 1. > The term "path" is used so much in this paper that I have a problem = with it in Goal 2: >=20 > o Goal 2 (Do no harm) A multipath flow should not take up more > capacity on any one of its paths than if it was a single path = flow > using only that route. This guarantees it will not unduly harm > other flows. >=20 > I mean it would be more precise (and even more important, easier to = understand) if you write: >=20 > o Goal 2 (Do no harm) A multipath flow should not take up more > capacity from any of the resources shared by its different paths, = than if it was a single flow > using only one of these paths. This guarantees it will not = unduly harm > other flows. >=20 > 2. > It is strange that you write (notice "the same bandwidth"): >=20 > Our solution sets the multipath flow's aggregate bandwidth to be the > same bandwidth a regular TCP flow would get on the best path > available to the multipath flow >=20 > when you in the next paragraph write (notice "the multipath flow = throughput will be strictly higher than"): >=20 > We note that in cases with low statistical multiplexing (where the > multipath flow influences the loss rates on the path) the multipath > throughput will be strictly higher than a single TCP would get on = any > of the paths. In particular, if using two idle paths, multipath > throughput will be sum of the two paths' throughput. >=20 > I think you should modify or quantify the statement about "the same = bandwidth" in the former paragraph. >=20 > The latter paragraph also conflicts with a later paragraph (notice = "equal to"): >=20 > "alpha" is a parameter of the algorithm that describes the > aggresiveness of the multipath flow. To meet Goal 1 (improve > throughput), the value of alpha is chosen such that the aggregate > throughput of the multipath flow is equal to the rate a TCP flow > would get if it ran on the best path. >=20 > 3.=20 > Before the definition of alpha at the end of section 3, you write: >=20 > Hence, alpha must be > computed for each multipath flow, based on the observed properties = of > the paths. >=20 > It is very easy to read this as "for each multipath subflow", and then = think there should be an alpha_i. > I suggest you remove "for each" , and write simply: >=20 > Hence, alpha must be computed based on the observed properties of = the paths. >=20 > Stein >=20 > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From c.raiciu@cs.ucl.ac.uk Mon Apr 11 04:46:00 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A5D523A6B09 for ; Mon, 11 Apr 2011 04:46:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.449 X-Spam-Level: X-Spam-Status: No, score=-2.449 tagged_above=-999 required=5 tests=[AWL=-0.150, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6ACTHFmfHL0d for ; Mon, 11 Apr 2011 04:46:00 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 13F263A6B07 for ; Mon, 11 Apr 2011 04:46:00 -0700 (PDT) Received: from [141.85.157.57] (helo=[192.168.1.129]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1Q9FXG-0003vU-4P; Mon, 11 Apr 2011 12:44:02 +0100 Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Costin Raiciu In-Reply-To: <4D99C5BA.7030608@uclouvain.be> Date: Mon, 11 Apr 2011 14:45:58 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <6F6218FB-7171-4D1C-9F48-A79A9D85D2D5@cs.ucl.ac.uk> References: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> <4D99C5BA.7030608@uclouvain.be> To: =?iso-8859-1?Q?S=E9bastien_Barr=E9?= X-Mailer: Apple Mail (2.1081) Cc: multipathtcp@ietf.org, Scott Brim Subject: Re: [multipathtcp] draft-ietf-mptcp-congestion-02 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Apr 2011 11:46:00 -0000 Seb, thanks for your suggestions. > In the multipath case, cwnd_cnt_i is maintained for each subflow as > above, and cwnd_i is increased by 1 when cwnd_cnt_i> alpha_scale * > tot_cwnd / alpha. >=20 > =3D=3D=3D >=20 > This does not correspond to the formula in section 3. IMO, it should = be: > "Increment cwnd_i by 1 when: > cwnd_cnt_i > max(alpha_scale*tot_cwnd*mss_i/alpha, cwnd_i*mss_i)" I agree with the max - well spotted. However, there is no need to = include mss here. snd_cwnd_cnt is simply the count of the number of = segments acked since the last cwnd increase. > *Finally a typo: Just before section 5: > cwnd_max*mss_max*mss_max -> cwnd_max*mss_max Actually this is correct. If you look at the formula, it has mss^2 both an the numerator and the = denominator, so they cancel out, which is what we want.=20 Thanks, Costin From Internet-Drafts@ietf.org Mon Apr 11 05:00:04 2011 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D955C3A6A0E; Mon, 11 Apr 2011 05:00:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.588 X-Spam-Level: X-Spam-Status: No, score=-102.588 tagged_above=-999 required=5 tests=[AWL=0.011, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZI1N7LyPjXhS; Mon, 11 Apr 2011 05:00:03 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2FF1C3A6B07; Mon, 11 Apr 2011 05:00:03 -0700 (PDT) MIME-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" From: Internet-Drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.16 Message-ID: <20110411120003.25515.14061.idtracker@localhost> Date: Mon, 11 Apr 2011 05:00:03 -0700 Cc: multipathtcp@ietf.org Subject: [multipathtcp] I-D Action:draft-ietf-mptcp-congestion-03.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Apr 2011 12:00:05 -0000 --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Multipath TCP Working Group of the IETF. Title : Coupled Congestion Control for Multipath Transport Protocols Author(s) : C. Raiciu, et al. Filename : draft-ietf-mptcp-congestion-03.txt Pages : 12 Date : 2011-04-11 Often endpoints are connected by multiple paths, but communications are usually restricted to a single path per connection. Resource usage within the network would be more efficient were it possible for these multiple paths to be used concurrently. Multipath TCP is a proposal to achieve multipath transport in TCP. New congestion control algorithms are needed for multipath transport protocols such as Multipath TCP, as single path algorithms have a series of issues in the multipath context. One of the prominent problems is that running existing algorithms such as TCP New Reno independently on each path would give the multipath flow more than its fair share at a bottleneck link traversed by more than one of its subflows. Further, it is desirable that a source with multiple paths available will transfer more traffic using the least congested of the paths, hence achieving resource pooling. This would increase the overall efficiency of the network and also its robustness to failure. This document presents a congestion control algorithm which couples the congestion control algorithms running on different subflows by linking their increase functions, and dynamically controls the overall aggresiveness of the multipath flow. The result is a practical algorithm that is fair to TCP at bottlenecks while moving traffic away from congested links. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-mptcp-congestion-03.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Message/External-body; name="draft-ietf-mptcp-congestion-03.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2011-04-11045104.I-D@ietf.org> --NextPart-- From sowmini.varadhan@oracle.com Tue Apr 12 12:38:12 2011 Return-Path: X-Original-To: multipathtcp@ietfc.amsl.com Delivered-To: multipathtcp@ietfc.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfc.amsl.com (Postfix) with ESMTP id 8174CE08DA for ; Tue, 12 Apr 2011 12:38:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([208.66.40.236]) by localhost (ietfc.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K1kCymPGM+2s for ; Tue, 12 Apr 2011 12:38:10 -0700 (PDT) Received: from rcsinet10.oracle.com (rcsinet10.oracle.com [148.87.113.121]) by ietfc.amsl.com (Postfix) with ESMTP id C753CE08CE for ; Tue, 12 Apr 2011 12:38:10 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by rcsinet10.oracle.com (Switch-3.4.2/Switch-3.4.2) with ESMTP id p3CJc8Pe007497 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 12 Apr 2011 19:38:09 GMT Received: from quasimodo.us.oracle.com (quasimodo.us.oracle.com [10.152.12.37]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id p3CJc7h3029818 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 12 Apr 2011 19:38:08 GMT Received: from quasimodo.us.oracle.com (localhost [127.0.0.1]) by quasimodo.us.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id p3CJQu3P008572 for ; Tue, 12 Apr 2011 15:26:56 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.us.oracle.com (8.14.4+Sun/8.14.4/Submit) id p3CJQuoA008571 for multipathtcp@ietf.org; Tue, 12 Apr 2011 15:26:56 -0400 (EDT) X-Authentication-Warning: quasimodo.us.oracle.com: sowmini set sender to sowmini.varadhan@oracle.com using -f Date: Tue, 12 Apr 2011 15:26:56 -0400 From: sowmini.varadhan@oracle.com To: multipathtcp@ietf.org Message-ID: <20110412192656.GT7742@quasimodo.us.oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.14 (2007-03-31) X-Source-IP: quasimodo.us.oracle.com [10.152.12.37] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4DA4AA20.00BE,ss=1,fgs=0 Subject: [multipathtcp] some comments/questions on draft*multiaddressed-03 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Apr 2011 19:38:12 -0000 - Section 3.1, the description in paragraph 2 of pg 12 is ambiguous: > Furthermore, in order to ensure reliable delivery of the ACK > containing the MP_CAPABLE option, a server MUST respond with an ACK > segment on receipt of this, which may contain data, or will be a pure ^^^^ which (ACK?)? There are multiple ACKs in this paragraph, and some clarity would be helpful. Also, is this something that needs a note in Section 4? - I realize that the 4 vs 8 octet DSN/Data ACK was done in an effort to conserve option space, but is it worth the complexity? Would it be simpler to just keep both DSN/DACK at 8 octets? - Editorial: first line of Section 3.3.3 needs a "to". > In regular TCP a FIN announces (to) the receiver that the sender has no > more data to send. - Section 3.3.4 pg 26 the first paragraph does not parse: > ... a segment is > placed in the connection level in-order or out-of-order queue if it > is in-window at both connection and subflow level. The stack still > has to remember, for each subflow, which segments were received > succesfully so that it can ACK them at subflow level appropriately. > Typically, this will be implemented by keeping per subflow out-of- > order queues (containing only message headers, not the payloads) and > remembering the value of the cumulative ACK. I think what's intended is that a segment would be placed in the in-order queue if it is in-widnow at both connection and subflow level, and the remainder of the paragraph applies for the out-of-order queue? - Cross-referencing the api draft, per the mptcp-api draft, MPTCP will only be enabled if the application binds to INADDR_ANY. Per Section 3.7.1 of the mptcp-multiaddressed draft we're not mandating that the same port number be used. If different port numbers are used, then what would be returned as the port number by, say, rpcbind/portmapper? It might be simpler to make the same port number requirement a MUST. - Section 4 (pg 40) - I recognize that the note about duplicate ACKs is related to the discussion about fitting in options from Section 3. But what if the duplicate ACK with mptcp signalling is actually triggered by congestion? Do we need some more specific heuristics for identifying dup acks due to lost packets? --Sowmini From sebastien.barre@uclouvain.be Wed Apr 13 04:22:44 2011 Return-Path: X-Original-To: multipathtcp@ietfc.amsl.com Delivered-To: multipathtcp@ietfc.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfc.amsl.com (Postfix) with ESMTP id 97F10E0722 for ; Wed, 13 Apr 2011 04:22:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.298 X-Spam-Level: X-Spam-Status: No, score=-6.298 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([208.66.40.236]) by localhost (ietfc.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 95aBBEUO49D1 for ; Wed, 13 Apr 2011 04:22:43 -0700 (PDT) Received: from smtp1.sgsi.ucl.ac.be (smtp.sgsi.ucl.ac.be [130.104.5.67]) by ietfc.amsl.com (Postfix) with ESMTP id 81251E0710 for ; Wed, 13 Apr 2011 04:22:43 -0700 (PDT) Received: from [130.104.228.80] (unknown [130.104.228.80]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sbarre@smtp1.sgsi.ucl.ac.be) by smtp1.sgsi.ucl.ac.be (Postfix) with ESMTPSA id 19B89E8D49; Wed, 13 Apr 2011 13:22:32 +0200 (CEST) Message-ID: <4DA58777.2020006@uclouvain.be> Date: Wed, 13 Apr 2011 13:22:31 +0200 From: =?ISO-8859-1?Q?S=E9bastien_Barr=E9?= User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); nl; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: Costin Raiciu References: <6A20AA4A-9A92-4F47-85C0-751358E5D937@ifi.uio.no> <4D99C5BA.7030608@uclouvain.be> <6F6218FB-7171-4D1C-9F48-A79A9D85D2D5@cs.ucl.ac.uk> In-Reply-To: <6F6218FB-7171-4D1C-9F48-A79A9D85D2D5@cs.ucl.ac.uk> Content-Type: multipart/alternative; boundary="------------080109060406020803010500" X-Virus-Scanned: clamav-milter 0.97-exp at smtp-1.sipr-dc.ucl.ac.be X-Virus-Status: Clean Received-SPF: Pass (sender authenticated); receiver=; client-ip=130.104.228.80; helo= Received-SPF: Pass (sender authenticated); receiver=; client-ip=130.104.228.80; envelope-from= X-Sgsi-Spamcheck: SASL authenticated, X-SGSI-MailScanner-ID: 19B89E8D49.00000 X-SGSI-MailScanner: Found to be clean X-SGSI-From: sebastien.barre@uclouvain.be X-SGSI-Spam-Status: No Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] draft-ietf-mptcp-congestion-02 -03 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Apr 2011 11:22:44 -0000 This is a multi-part message in MIME format. --------------080109060406020803010500 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Hi Costin, Op 11-04-11 13:45, Costin Raiciu schreef: > === >> This does not correspond to the formula in section 3. IMO, it should be: >> "Increment cwnd_i by 1 when: >> cwnd_cnt_i> max(alpha_scale*tot_cwnd*mss_i/alpha, cwnd_i*mss_i)" > I agree with the max - well spotted. However, there is no need to include mss here. snd_cwnd_cnt is simply the count of the number of segments acked since the last cwnd increase. Then there is another (small) problem... Indeed you define snd_cwnd_cnt before, as a count of the number of segments. However, you also say in section 3 the following : === We assume appropriate byte counting (ABC, [RFC3465 ]) is used, hence the bytes_acked variable records the number of bytes newly acknowledged. If ABC is not used, bytes_acked SHOULD be set to mss_i. ==== Since there is no further ABC mention later in the document, I was assuming that your statement "we assume ABC is used" was valid throughout the document. If you count snd_cwnd_cnt in segments, you are not doing ABC. Still this formula is interesting IMO, but I think it should be mentioned that this is not doing ABC, and the equivalent formula for ABC should be put next to it (using bytes_acked instead of snd_cwnd_cnt to avoid ambiguities). What do you think ? >> *Finally a typo: Just before section 5: >> cwnd_max*mss_max*mss_max -> cwnd_max*mss_max > Actually this is correct. > If you look at the formula, it has mss^2 both an the numerator and the denominator, so they cancel out, which is what we want. Absolutely right. I was just reading too fast, and it was tempting to see such a repetition as a typo :-). Maybe mss_max^2 would be clearer ? thanks, Séb. > Thanks, > Costin > -- Sébastien Barré Researcher, CSE department, UCLouvain, Belgium http://inl.info.ucl.ac.be/sbarre --------------080109060406020803010500 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi Costin,

Op 11-04-11 13:45, Costin Raiciu schreef:
===
This does not correspond to the formula in section 3. IMO, it should be:
"Increment cwnd_i by 1 when:
cwnd_cnt_i > max(alpha_scale*tot_cwnd*mss_i/alpha, cwnd_i*mss_i)"

I agree with the max - well spotted. However, there is no need to include mss here. snd_cwnd_cnt is simply the count of the number of segments acked since the last cwnd increase.
Then there is another (small) problem... Indeed you define snd_cwnd_cnt before, as a count of the number of segments.
However, you also say in section 3 the following :
===
   We assume appropriate byte counting (ABC, [RFC3465]) is used, hence
   the bytes_acked variable records the number of bytes newly
   acknowledged.  If ABC is not used, bytes_acked SHOULD be set to
   mss_i.
====
Since there is no further ABC mention later in the document, I was assuming that your statement "we assume ABC is used" was valid throughout the document.
If you count snd_cwnd_cnt in segments, you are not doing ABC.
Still this formula is interesting IMO, but I think it should be mentioned that this is not doing ABC, and the equivalent formula for ABC should be put next to it (using bytes_acked instead of snd_cwnd_cnt to avoid ambiguities).

What do you think ? 

*Finally a typo: Just before section 5:
cwnd_max*mss_max*mss_max -> cwnd_max*mss_max

Actually this is correct.
If you look at the formula, it has mss^2 both an the numerator and the denominator, so they cancel out, which is what we want.
Absolutely right. I was just reading too fast, and it was tempting to  see such a repetition as a typo :-). Maybe mss_max^2 would be clearer ?

thanks,

Séb.
Thanks,
Costin



-- 
Sébastien Barré
Researcher,
CSE department, UCLouvain, Belgium
http://inl.info.ucl.ac.be/sbarre
--------------080109060406020803010500-- From yoshifumi.nishida@gmail.com Thu Apr 14 17:56:38 2011 Return-Path: X-Original-To: multipathtcp@ietfc.amsl.com Delivered-To: multipathtcp@ietfc.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfc.amsl.com (Postfix) with ESMTP id 4F3BAE078E for ; Thu, 14 Apr 2011 17:56:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.977 X-Spam-Level: X-Spam-Status: No, score=-102.977 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([208.66.40.236]) by localhost (ietfc.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4A5SOPg1opnx for ; Thu, 14 Apr 2011 17:56:36 -0700 (PDT) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by ietfc.amsl.com (Postfix) with ESMTP id E9AB4E065C for ; Thu, 14 Apr 2011 17:56:32 -0700 (PDT) Received: by pwi5 with SMTP id 5so1064068pwi.31 for ; Thu, 14 Apr 2011 17:56:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Q2tlYlUKSRa66RBLfSaY14aRH6m8tbT+E2hsRAPZ7JE=; b=sYeVNue48596nXiTeGij2c2hHfj3v86n7yhi/XDilL2RhmlXaRVJtUDhNyosVCG6Wd aowItRD8DPRfIk2j7tq1m5gbOJGdgb0tfdDi+knMPCEkRXUi0vSzjXgZCtTTbQ1gfZsJ hdwjI04une90Ft0E2gzlOuyniIpeLsyw/Jfik= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=hraxWhpy8ASHf6fj4g/SUjJIrRx4RGC8Awo7cZ/RjAbq4w+PVbccsJ/QBXhjcokMvH s0ZOFgxHGsFJzyNmCXAQ7EUlOUs1IyQXLDfiqG1JL5KICSMRQy1A2mOXacXrc9U81sgw mmm7GLL9bp93qCmwDqoGSfb5WdjRu1bEiETdY= MIME-Version: 1.0 Received: by 10.68.4.129 with SMTP id k1mr1005507pbk.72.1302828992403; Thu, 14 Apr 2011 17:56:32 -0700 (PDT) Sender: yoshifumi.nishida@gmail.com Received: by 10.68.57.105 with HTTP; Thu, 14 Apr 2011 17:56:32 -0700 (PDT) In-Reply-To: <20110412192656.GT7742@quasimodo.us.oracle.com> References: <20110412192656.GT7742@quasimodo.us.oracle.com> Date: Thu, 14 Apr 2011 17:56:32 -0700 X-Google-Sender-Auth: 4x5wOG0BawJebesBOvilGS0YDas Message-ID: From: Yoshifumi Nishida To: sowmini.varadhan@oracle.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] some comments/questions on draft*multiaddressed-03 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Apr 2011 00:56:38 -0000 Hi sowamini, 2011/4/12 : > - Section 3.1, the description in paragraph 2 of pg 12 is ambiguous: > =A0> Furthermore, in order to ensure reliable delivery of the ACK > =A0> containing the MP_CAPABLE option, a server MUST respond with an ACK > =A0> segment on receipt of this, which may contain data, or will be a pur= e > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^^^^ which (ACK?)? The= re are multiple ACKs > =A0in this paragraph, and some clarity would be helpful. Also, is this > =A0something that needs a note in Section 4? > > - I realize that the 4 vs 8 octet DSN/Data ACK was done in an effort to > =A0conserve option space, but is it worth the complexity? Would it be > =A0simpler to just keep both DSN/DACK at 8 octets? Hmm.. but, option space might be scarce resource.. One possible way I can think is to make 4 octets DSN/DataACK as an optional feature so that some implementations can choose to not support this feature.(just like DCCP's short seqnum) In order to do this, we will need to use one reserved bit in MP_CAPABLE option to indicate that it supports 4 octets DSN. > - Editorial: first line of Section 3.3.3 needs a "to". > =A0> In regular TCP a FIN announces (to) the receiver that the sender has= no > =A0> =A0more data to send. > > - Section 3.3.4 pg 26 the first paragraph does not parse: > =A0> ... a segment is > =A0> placed in the connection level in-order or out-of-order queue if it > =A0> is in-window at both connection and subflow level. =A0The stack stil= l > =A0> has to remember, for each subflow, which segments were received > =A0> succesfully so that it can ACK them at subflow level appropriately. > =A0> Typically, this will be implemented by keeping per subflow out-of- > =A0> order queues (containing only message headers, not the payloads) and > =A0> remembering the value of the cumulative ACK. > > =A0I think what's intended is that a segment would be placed in the in-or= der > =A0queue if it is in-widnow at both connection and subflow level, and the > =A0remainder of the paragraph applies for the out-of-order queue? > > - Cross-referencing the api draft, per the mptcp-api draft, MPTCP will > =A0only be enabled if the application binds to INADDR_ANY. Per Section 3.= 7.1 > =A0of the mptcp-multiaddressed draft we're not mandating that the same > =A0port number be used. If different port numbers are used, then what > =A0would be returned as the port number by, say, rpcbind/portmapper? > =A0It might be simpler to make the same port number requirement a MUST. In legacy applications, MPTCP will only be enabled if the application binds to INADDR_ANY. But, in case of MPTCP-aware applications, I think apps might want more flexibility and port numbers will be returned by mptcp-specific APIs. > - Section 4 (pg 40) - I recognize that the note about duplicate ACKs is > =A0related to the discussion about fitting in options from Section 3. > =A0But what if the duplicate ACK with mptcp signalling is actually > =A0triggered by congestion? Do we need some more specific heuristics for > =A0identifying dup acks due to lost packets? I guess that the authors also meant that receivers should not include MPTCP options in dupACKs when they have seen gaps in seqnums. Thanks, -- Yoshifumi Nishida nishida@sfc.wide.ad.jp From yoshifumi.nishida@gmail.com Thu Apr 14 21:20:59 2011 Return-Path: X-Original-To: multipathtcp@ietfc.amsl.com Delivered-To: multipathtcp@ietfc.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfc.amsl.com (Postfix) with ESMTP id 18DB6E0677 for ; Thu, 14 Apr 2011 21:20:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.677 X-Spam-Level: X-Spam-Status: No, score=-102.677 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, J_CHICKENPOX_21=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([208.66.40.236]) by localhost (ietfc.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pucIN6x9kA+E for ; Thu, 14 Apr 2011 21:20:57 -0700 (PDT) Received: from mail-px0-f182.google.com (mail-px0-f182.google.com [209.85.212.182]) by ietfc.amsl.com (Postfix) with ESMTP id 3D080E06A6 for ; Thu, 14 Apr 2011 21:20:57 -0700 (PDT) Received: by pxi20 with SMTP id 20so1473268pxi.27 for ; Thu, 14 Apr 2011 21:20:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; bh=uqE+m4F7auWwacsznDFWDy2t4tyRUjxcCSpUfVndOIo=; b=bJQZldKJggaNhfiaNaW7YQkJbsg7LIoDcr2YftUK7MwzDqIWwgPBjC8MMqxtTGFBe7 KuHgF+Mv3upEdLHVO48ROkQFAHaQ26AWpEscM54YZrh0nXSyZqpraWdmf+v0LzdwcU3x pU2tBFjg/K9lRI0nmEgJseZkt5nw3yiYVVPew= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=WHTbIdTxAGA0ai2IoQKCYS/0EpaXJcZVBgt7qGHk/fwPAVU8ALOW3MgJUyGmzcfkah 5xWggbmGEafQtdYToOYzck5Id4On9vfRcit9dYuDWjfKWq3JVDDOu0vL9AMhPakiGnN1 PjjGqvVzXe6kWPCwrIGQOdYFooj8KC/AIUAeY= MIME-Version: 1.0 Received: by 10.68.47.229 with SMTP id g5mr1245934pbn.407.1302841256634; Thu, 14 Apr 2011 21:20:56 -0700 (PDT) Sender: yoshifumi.nishida@gmail.com Received: by 10.68.57.105 with HTTP; Thu, 14 Apr 2011 21:20:56 -0700 (PDT) In-Reply-To: <9510D26531EF184D9017DF24659BB87F32902FA928@EMV65-UKRD.domain1.systemhost.net> References: <9510D26531EF184D9017DF24659BB87F32902FA928@EMV65-UKRD.domain1.systemhost.net> Date: Thu, 14 Apr 2011 21:20:56 -0700 X-Google-Sender-Auth: RSm7vmLxj2mnvV_mfibTxq32vAM Message-ID: From: Yoshifumi Nishida To: multipathtcp Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [multipathtcp] MPTCP WG meeting - draft minutes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Apr 2011 04:20:59 -0000 Hello, After minor revision and reformatting, I uploaded a draft minute for prague meeting on the following URL. http://www.ietf.org/proceedings/80/minutes/mptcp.txt We appreciate Olivier for note taking and Luigi for jabber support. We also would like to thank everyone at the meeting for active discussions. Thank you so much -- Yoshifumi & Phil 2011/4/8 : > Here's a draft of the MPTCP WG minutes for our Prague meeting. Please sen= d corrections as soon as possible > > Thanks very much to Olivier for the detailed notes! > > Phil & Yoshifumi. > > --- > > * Chairs. > > Review milestones > > wg doing pretty well > > two rfcs published 6181 and 6182 > > protocol : need reviewers to review the document > > implementation available check inl.info.ucl.ac.be/mptcp > > * Alan Ford =A0: Protocol > > update of the changes to the protocol > > changes to wire format, but no fundamental change, revisions based on > implementation experience > > effort to make document more readable for reviewers and implementers > > since 02 > > single option type, which changed all option formats > data level sequence numbers and acks in single option > data fin option decoupled from subflow Fin > security solution discussed in interim meeting > checksum changes > > single option > > format on slides > > data sequence signal > > big option, merge of data sequence data ack and data fin all in one > > format on slides > > F flag similar to FIN flag on TCP > > reason for DataFIN : corner case during connection close was possible > with the previous design > > connection level fin must be acked before subflow level fin, but > optimisation remains possible > > security solution > > logic similar to 02 > based feedback reduced to 3 packets exchange with no use of payload > sha-1 is specified but there are other bits for crypto agility in the fut= ure > > see slide for security solution example > > MAC truncated on server-client direction but a false server a less > significant security problem than a false client that tries to create a > lot of subflow attempts > > other changes > > checksum optional > clarified rules on duplicate acks for signalling > appendix on tcp control block data structures > > > heuristics > > not a lot of text on this now > without large scale deployment, difficult to make clear recommendations > > where next > > document fairly stable > > In need of reviewers. We hope to have detailed reviews at this stage > > Michael Welzl : I volunteer > [later: instead Michael's colleague, Stein Gjessing, will review] > > questions : > > nobody in the room > > open issues : > > lars eggert : don't have feedback, but is there another implementation > besides ucl's implementation? If you are doing an implementation, please > send feedback > > * Mark Handley : MPTCP Proxies > > Classifies as open issue > > doing some work on mptcp as a mobility solution > > during the bof, we showed a scenario with a client using 3G and Wifi to > a server to supplement or replace the 3G for better > performance/robustness/battery power > > opportunistically use Wifi base stations seemed worth exploring and we > did some simulations > > still preliminary, short paper see url on slide > > you can get better throughput, better robustness and save battery > > could be a key part of a mobility solution > > minor problem : cannot depend on servers for mptcp in the future > > who will use mptcp in near future ? > > Solution > > mptcp proxy > > idea mobile client and proxy provided by e.g. 3G provider > > connect to proxy and proxy connects to server > > when mobile moves, uses proxy > server knows nothing about mptcp > > not ideal to realy everything through the proxy all the time > > can we avoid =A0proxy all time on path > > assume server mp-capable, proxy can learn > > > Q: Lars Eggert why wouldn't the mobile open connection to both proxy and > server > > MH : need to kill state > > Scott Brim : > cool thing is mobile IP homeagent, winning combination > you want an initial rendez-vous point anyway for privacy > mh : not exactly the same role, it is similar, but proxy would need to > behave differently with normal server and mptcp server > > Randall Stewart : there is IPR on this at a former employer, did similar > thing for SCTP > > Scott Brim : had no idea this patent covered this, we'll ask later > extremely excited to see this model, you get 3 for 1, grab people who > like mobile ip, don't need route optimisation, get privacy and get > optimisation with mptcp > > mh : one thing we need to do is to tell the proxy where to connect to > and you need to do this in the syn > > would like to put an address in the syn, but we don't have space for this > principles are easy, but in practice not easy to find the bytes > should we try this on the first path in the protocol spec ? > > Adam ? , cisco: you seem to indicate mobility use case, but mptcp could > be a general proxy, in such case all connections will go through the > proxy. If the proxy knows that the server is mptcp capable and it does > not need the flow anymore > > mh : it might want to relay data at the beginning of the subflow to > reduce latency > > you could do after the syn, but looks difficult > > Adam : take it offline > > Tim Shepard : mptcp proxy is used for everything and you have a long > term relationship and have state in both ends, why not tunnel the > information to the proxy > > mh : gre would not go to nat > > Tim : TCP goes through NAT, could tunnel over TCP ? > > MH : maybe over UDP ? > > Scott : mobile ip > > Tim : if you have wifi access point that is causing problem, does mip wor= k > > mh : there is nat on all these links > > Tim : tunnel > > mh : udp makes sense, but don't like udp > > Lars Eggert : assuming you could reuse some bits of the source ports, > fragment id,s .. > > Alan Ford : data in the syn, wait for discussion in tcpm and work on > spdy in google > > Tim Shepard : you know you are talking to a special box and you could > use special acks to discuss with this box, think more evil > > MH : Do we want this in the normal spec ? > > Scott Brim =A0: keep separate > Lars Eggert : agree with Scott, don't hold the draft in the wg longer > > Tim Shepard : should be ? in the spec to encourage people, only used for > mobile clients than want to use a proxy > > Alan Ford : don't know if this will be useful and wireless is uncontrolle= d > > Phil Eardley (chair) : keep it separate but there is interest in the room > > * Michael Scharf, API > > authors have corrected some minor issues > > minor updates > > some functions return now both ip addresses and port numbers instead of > only ip addresses > > additional entries in the candidate list for an extended api > > additional text on approaches like happy eyeballs, mentioned as a > possibility, not mandated, more a heuristic for protocol than an API issu= e > > discussion of comments received from Javier Ubillos, Michael Tuexen > > document pretty stable, please provide further feedback > > no questions > > Phil (chair) : there will be a wg last call after the next revision > > > * Mark Handley, congestion control > > 3 reviews received for previous cc draft, mainly wording and interpretati= on > > bytes/packets stacks. draft has been updated to cover both and avoid > rounding errors. packet based version assumes that mss is same on all > paths and draft describes how to handle the case if mss are different. > > NSDI2011 paper describes the algo in details and its performance > > some results from datacenter simulations see slides > mptcp is good at pushing traffic out of hotspot paths, better than > application level distribution over different tcp connections > > measurements on amazon EC2 datacenter where there are multiple paths > running over 24 h > > measurements show that when vms are on same machine or host, mptcp same > as tcp > > over ecmp paths, mptpcp with 2 or 4 flows gets much better performance, > 3x more > > coupled congestion control find the available capacity in the datacenter > > * S=E9bastien Barr=E9 : demonstration of Linux MPTCP implementation > > setup : two servers in a LAN with a programmable router that inserts > delay or losses between the two servers, live demo, servers are running > latest mptcp linux kernel implementation > > look at bandwidth when the delay or bandwidth or losses change in realtim= e > > two separate 100 Mbps links > > iperf running between the two servers, gets almost 200 Mbps at the > application level > > add losses on one path 1% > > throughput drops on the affected link but not the other and application > continues to work with reduced bandwidth > coupled congestion control is going away from congested path > > same loss for both pass, two subflows get roughly same bandwidth > > add delay on one link, one subflow is affected for some time and then > recovers > > failure : stop one link, the other is still working and iperf only sees > reduced bandwidth > > link comes back, need some time to find the link up due to exponential > backoff > > > Question : > > > Tecco Boot why did loss on the blue link caused a reduction in bandwidth > on the green one > > Sebastien : related to retransmission that are done on the other flow, > retransmission timeout may cause delays and fill receive buffer > > MH : think you are asking a different questions > > congstion control is linked between the two subflows and this explains > why we both are equal loss, they get the same performance > > loss is modelling congestion > > Tim : all point is to move traffic away from congested path > > Tecco Boot : I would like to have a solution to react differently to > losses and congestion. Looking for solutions taking traffic from satcom > to 3G to wifi, mptcp could do some job there, you should use reliable lin= k > > MH : How do you detect whether the loss is due to congestion or due to > something else ? Without this info, we need to assume that they are > caused by congestion and not by link errors > > Georg Hampel, Alcatel : > > demo is very nice. What happens if there is a large delay, up to 3 or 5 > seconds > > Yoshifumi : how do you create loss and delay > > Sebastien : tc, artificial > > Sebastien set delay to 3 second > > strange result > > Michael Scharf : like this demo, but independent of mptcp design, did > the same demo with another protocol, could do it with a different protoco= l > > - distribution of live CDs with the implementation > > * Sebastien : presentation of guidelines for implementers > > implementation status > > objective of the document is to explain the choices made in the > implementation and why and also the things that were abandoned > > hope to include feedback on other implementations later > > besides the real demo, MPTCP implementation is running as well on Nokia > N900 and Nexus who are running Linux > > explanation of architecture > > explanation of path management > > explanation of send queue management > need same mss for all subflows > > explanation of receive queue management > > * Rolf Winter, autoconfiguring single-homed end-systems to make use of > network multihoming > > how to allow single homed host use multipath ? > > scenario : dhcp server, gateway with multiple ISPs and DHCP server > > mptcp works with multiple addresses, option 1 give multiple ip addresses > to host, but gateway needs to do source routing > > solution: > > not same thing as multipath proxy > > create virtual interface that sends a dhcp request, add to dhcp request > an option that indicates from which range you want the address > > mainly configuration tweaking, 20 lines of scripting and running in a > small testbed > > other ways to achieve same result are discussed in the slides > > is it useful ? > > Questions : > > ? for experiments this is useful if you have a lot of time, but for > normal usage, this is interesting for IPv6, multihoming on a single > homed host with multiple addresses. how does mptcp work in this environme= nt > > Rolf: other scenarios are discussed in this document > > some discussion on ipv4 versus ipv6 > > Phil Eardley (chair) : show of hands : a dozen hands > > * Rechartering ideas > > doing well on our charter > > > from slide : > > How much energy is there ? > Advice on implementation and heuristics ? > Support for singl-homed end systems > applicability and experience > alternative security > alternative congestion control > advanced api > proxy for mobility > other topics > > Mark Handley : > > intend to spend some time on heuristics section, there is a lot we don't > know, need good ways > > Scott Brim : I consider the following topics > > applicability and experience > alternative congestion control > advanced api > proxy for mobility > > I hope somebody would work on these, I want to work on proxy for mobility > > Tim Shepard : there should be a venue to do interop testing if there is > a second implementation. nice to continue with a venue > > Phil : default option is to finish charter before the next ietf and > pause while waiting for experiements and usage > > Alan Ford : We'll get more feedback as implementations progress > > lars Eggert : nobody else is doing an implementation or knows another > implementation ? > > bof was extremely popular and few > > Sowmini Varadhan from oracle : working on multipath for solaris > > Sebastien : a dozen people are helping in debuging and four are > providing patches, others are downloading > > merge in main linux tree is too early, need to stabilise and follow the > draft > > Lars : be great if more people could contribute > > > > > > > > > > > > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From prvs=8086c6266f=alan.ford@roke.co.uk Fri Apr 15 01:16:31 2011 Return-Path: X-Original-To: multipathtcp@ietfc.amsl.com Delivered-To: multipathtcp@ietfc.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfc.amsl.com (Postfix) with ESMTP id 1773FE0686 for ; Fri, 15 Apr 2011 01:16:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.209 X-Spam-Level: X-Spam-Status: No, score=-3.209 tagged_above=-999 required=5 tests=[AWL=0.390, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([208.66.40.236]) by localhost (ietfc.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Fn+iSQFzAytw for ; Fri, 15 Apr 2011 01:16:30 -0700 (PDT) Received: from ixe-mta-29.emailfiltering.com (ixe-mta-29-tx.emailfiltering.com [194.116.199.160]) by ietfc.amsl.com (Postfix) with ESMTP id D9E7EE065A for ; Fri, 15 Apr 2011 01:16:29 -0700 (PDT) Received: from salt-ext.roke.co.uk ([109.207.29.2]) by ixe-mta-29.emailfiltering.com with emfmta (version 4.8.1.33) vanilla id 345977398 for multipathtcp@ietf.org; c83b04bc1b213e20; Fri, 15 Apr 2011 09:16:28 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 15 Apr 2011 09:15:46 +0100 Message-ID: <2181C5F19DD0254692452BFF3EAF1D680CA17C55@rsys005a.comm.ad.roke.co.uk> In-Reply-To: <20110412192656.GT7742@quasimodo.us.oracle.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] some comments/questions on draft*multiaddressed-03 Thread-Index: Acv5SS0db5Khd3vkQwejcTLWij8MjAB+fDeQ References: <20110412192656.GT7742@quasimodo.us.oracle.com> From: "Ford, Alan" To: , Subject: Re: [multipathtcp] some comments/questions on draft*multiaddressed-03 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Apr 2011 08:16:31 -0000 Hi Sowmini, Sorry for the delay in replying; clarifications inline... > -----Original Message----- > From: multipathtcp-bounces@ietf.org [mailto:multipathtcp-bounces@ietf.org] On > Behalf Of sowmini.varadhan@oracle.com > Sent: 12 April 2011 20:27 > To: multipathtcp@ietf.org > Subject: [multipathtcp] some comments/questions on draft*multiaddressed-03 >=20 > - Section 3.1, the description in paragraph 2 of pg 12 is ambiguous: > > Furthermore, in order to ensure reliable delivery of the ACK > > containing the MP_CAPABLE option, a server MUST respond with an ACK > > segment on receipt of this, which may contain data, or will be a pure > ^^^^ which (ACK?)? There are multiple ACKs > in this paragraph, and some clarity would be helpful. Also, is this > something that needs a note in Section 4? This refers to receipt of the first pure ACK packet in the connection (i.e. the third packet in the SYN, SYN/ACK, ACK exchange). This contains the MP_CAPABLE option with both keys, and a receiver must ACK this in order to allow stateless operation at the sender. > - I realize that the 4 vs 8 octet DSN/Data ACK was done in an effort to > conserve option space, but is it worth the complexity? Would it be > simpler to just keep both DSN/DACK at 8 octets? We'd rather not mandate the use of this extra option space if we can avoid it. However, implementation feedback is interesting: is it really excessively complicated? The values must be stored in the 64-bit version internally, so it's only the mapping to/from the wire format where this matters. (Logic to ensure wrapping is handled correctly is fairly standard stuff, I would have thought.) > - Editorial: first line of Section 3.3.3 needs a "to". > > In regular TCP a FIN announces (to) the receiver that the sender has no > > more data to send. >=20 > - Section 3.3.4 pg 26 the first paragraph does not parse: > > ... a segment is > > placed in the connection level in-order or out-of-order queue if it > > is in-window at both connection and subflow level. The stack still > > has to remember, for each subflow, which segments were received > > succesfully so that it can ACK them at subflow level appropriately. > > Typically, this will be implemented by keeping per subflow out-of- > > order queues (containing only message headers, not the payloads) and > > remembering the value of the cumulative ACK. >=20 > I think what's intended is that a segment would be placed in the in-order > queue if it is in-widnow at both connection and subflow level, and the > remainder of the paragraph applies for the out-of-order queue? Yes. We will look at improving that text, thanks. > - Cross-referencing the api draft, per the mptcp-api draft, MPTCP will > only be enabled if the application binds to INADDR_ANY. Per Section 3.7.1 > of the mptcp-multiaddressed draft we're not mandating that the same > port number be used. If different port numbers are used, then what > would be returned as the port number by, say, rpcbind/portmapper? > It might be simpler to make the same port number requirement a MUST. In almost all circumstances I believe that will be the case. And indeed, for a listener, if you bind to INADDR_ANY today on some systems (e.g. Windows) you will have the same port reserved on all addresses (even those that appear later in a connection). The main reason it is a SHOULD is that there may be circumstances, either due to load balancing, or somehow a port is already in use on one address, where this is not possible. We could strengthen the text, thanks. > - Section 4 (pg 40) - I recognize that the note about duplicate ACKs is > related to the discussion about fitting in options from Section 3. > But what if the duplicate ACK with mptcp signalling is actually > triggered by congestion? Do we need some more specific heuristics for > identifying dup acks due to lost packets? If there is real congestion, the expectation is that the dup ACKs will not contain MPTCP options since there is no new information to signal at the connection-level. Many thanks for your feedback! Regards, Alan From sowmini.varadhan@oracle.com Fri Apr 15 13:10:40 2011 Return-Path: X-Original-To: multipathtcp@ietfc.amsl.com Delivered-To: multipathtcp@ietfc.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfc.amsl.com (Postfix) with ESMTP id 92EDFE0687 for ; Fri, 15 Apr 2011 13:10:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([208.66.40.236]) by localhost (ietfc.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mzQN-B-Qe2G1 for ; Fri, 15 Apr 2011 13:10:39 -0700 (PDT) Received: from rcsinet10.oracle.com (rcsinet10.oracle.com [148.87.113.121]) by ietfc.amsl.com (Postfix) with ESMTP id B9C97E0682 for ; Fri, 15 Apr 2011 13:10:39 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by rcsinet10.oracle.com (Switch-3.4.2/Switch-3.4.2) with ESMTP id p3FKAbfT007668 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 15 Apr 2011 20:10:38 GMT Received: from quasimodo.us.oracle.com (quasimodo.us.oracle.com [10.152.12.37]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id p3FKAYMO002271 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 15 Apr 2011 20:10:35 GMT Received: from quasimodo.us.oracle.com (localhost [127.0.0.1]) by quasimodo.us.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id p3FJxL77013069; Fri, 15 Apr 2011 15:59:21 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.us.oracle.com (8.14.4+Sun/8.14.4/Submit) id p3FJxK2c013068; Fri, 15 Apr 2011 15:59:20 -0400 (EDT) X-Authentication-Warning: quasimodo.us.oracle.com: sowmini set sender to sowmini.varadhan@oracle.com using -f Date: Fri, 15 Apr 2011 15:59:20 -0400 From: sowmini.varadhan@oracle.com To: "Ford, Alan" Message-ID: <20110415195920.GA12767@quasimodo.us.oracle.com> References: <20110412192656.GT7742@quasimodo.us.oracle.com> <2181C5F19DD0254692452BFF3EAF1D680CA17C55@rsys005a.comm.ad.roke.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D680CA17C55@rsys005a.comm.ad.roke.co.uk> User-Agent: Mutt/1.5.14 (2007-03-31) X-Source-IP: quasimodo.us.oracle.com [10.152.12.37] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4DA8A63B.00E1,ss=1,fgs=0 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] some comments/questions on draft*multiaddressed-03 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Apr 2011 20:10:40 -0000 On (04/15/11 09:15), Ford, Alan wrote: > > We'd rather not mandate the use of this extra option space if we can > avoid it. > > However, implementation feedback is interesting: is it really > excessively complicated? The values must be stored in the 64-bit version > internally, so it's only the mapping to/from the wire format where this > matters. (Logic to ensure wrapping is handled correctly is fairly > standard stuff, I would have thought.) yes, but it's just additional logic to check the flags and then map to/from the option both for parsing the options or even for debugging the snoop traces. It's not actually a big deal, but I was wondering how much of a tradeoff the simpler "always 64" was. Regarding rpcbind/portmapper return values for the port number for a service.. > In almost all circumstances I believe that will be the case. And indeed, > for a listener, if you bind to INADDR_ANY today on some systems (e.g. > Windows) you will have the same port reserved on all addresses (even > those that appear later in a connection). > > The main reason it is a SHOULD is that there may be circumstances, > either due to load balancing, or somehow a port is already in use on one > address, where this is not possible. > > We could strengthen the text, thanks. Ok. I guess this is somewhat related to the other questions around getsockname/getpeername when the first subflow itself is lost. At least for the service's port number case, things would be more straightforward if the port number is common to all subflows. > > - Section 4 (pg 40) - I recognize that the note about duplicate ACKs > is > > related to the discussion about fitting in options from Section 3. > > But what if the duplicate ACK with mptcp signalling is actually > > triggered by congestion? Do we need some more specific heuristics > for > > identifying dup acks due to lost packets? > > If there is real congestion, the expectation is that the dup ACKs will > not contain MPTCP options since there is no new information to signal at > the connection-level. Yes, I think Yoshifumi also mentioned the same thing. --Sowmini