From nobody Wed Mar 12 09:19:41 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B530F1A09DA; Wed, 12 Mar 2014 09:19:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HYTprVsS66Tz; Wed, 12 Mar 2014 09:19:35 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 1157A1A09CD; Wed, 12 Mar 2014 09:19:35 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: IETF Secretariat To: ehalep@ece.upatras.gr,joel.halpern@ericsson.com X-Test-IDTracker: no X-IETF-IDTracker: 5.1.0 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20140312161935.11177.72506.idtracker@ietfa.amsl.com> Date: Wed, 12 Mar 2014 09:19:35 -0700 Archived-At: http://mailarchive.ietf.org/arch/msg/forces/Ogbnz-mvsv-z0w_nm2rS2m-s1Ts Cc: akatlas@gmail.com, forces@ietf.org, ipr-announce@ietf.org Subject: [forces] IPR Disclosure: Telefonaktiebolaget LM Ericsson (publ)'s Statement about IPR related to draft-ietf-forces-packet-parallelization-00 X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Mar 2014 16:19:36 -0000 Dear Evangelos Haleplidis, Joel M. Halpern: An IPR disclosure that pertains to your Internet-Draft entitled "ForCES Packet Parallelization" (draft-ietf-forces-packet-parallelization) was submitted to the IETF Secretariat on 2014-03-12 and has been posted on the "IETF Page of Intellectual Property Rights Disclosures" (https://datatracker.ietf.org/ipr/2329/). The title of the IPR disclosure is "Telefonaktiebolaget LM Ericsson (publ)'s Statement about IPR related to draft- ietf-forces-packet-parallelization-00.""); The IETF Secretariat From nobody Mon Mar 17 05:31:54 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5B441A03FD; Mon, 17 Mar 2014 05:31:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9pNEgrTrs-uh; Mon, 17 Mar 2014 05:31:48 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B86E1A03F7; Mon, 17 Mar 2014 05:31:48 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 5.1.0p1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20140317123148.14751.87650.idtracker@ietfa.amsl.com> Date: Mon, 17 Mar 2014 05:31:48 -0700 Archived-At: http://mailarchive.ietf.org/arch/msg/forces/zu6cULgPx3jUvtckvD5jMw8UCGg Cc: forces@ietf.org Subject: [forces] I-D Action: draft-ietf-forces-protoextension-01.txt X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 12:31:50 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Forwarding and Control Element Separation Working Group of the IETF. Title : ForCES Protocol Extensions Author : Jamal Hadi Salim Filename : draft-ietf-forces-protoextension-01.txt Pages : 11 Date : 2014-03-17 Abstract: Experience in implementing and deploying ForCES architecture has demonstrated need for a few small extensions both to ease programmability and to improve wire efficiency of some transactions. This document describes extensions to the ForCES Protocol Specification[RFC 5810] semantics to achieve that end goal. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-forces-protoextension/ There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-forces-protoextension-01 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=draft-ietf-forces-protoextension-01 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Tue Mar 18 12:54:56 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9EEC61A0494 for ; Tue, 18 Mar 2014 12:54:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.378 X-Spam-Level: X-Spam-Status: No, score=-1.378 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, J_CHICKENPOX_32=0.6, RCVD_IN_DNSWL_LOW=-0.7] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7JfHNqMpYfx0 for ; Tue, 18 Mar 2014 12:54:54 -0700 (PDT) Received: from mail-ve0-f171.google.com (mail-ve0-f171.google.com [209.85.128.171]) by ietfa.amsl.com (Postfix) with ESMTP id BBA651A044D for ; Tue, 18 Mar 2014 12:54:54 -0700 (PDT) Received: by mail-ve0-f171.google.com with SMTP id cz12so7817915veb.30 for ; Tue, 18 Mar 2014 12:54:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=KDRK/RH8yzB9/vDryLMg2uhu8J94E49tOw4I6ERTViU=; b=bDkgmws/kmyD67pizXroNjJiwU4i2nF7jNxU76J45XlSoEztdJYp4J9aOYtI7fqvoo 8/OjR0/pKkxVBKZlntpKFvhiSOaDmtR50n4YA51kiS+cRXSy3uLKggdxG6kWDG1xZBac 5PS9OHH9ISfzBsOWB0MBeHxAW5BsBgzw9S25Y5xFTArOXhD6KjfKCcPi0YZPzLE1S93j xyz97Nx0DLpZBxC8wrBST0tMs4F+MdYqRX89MlQOQMopnVEBF06FzzF+YGv/3KTiQOHY S2Jtc24O2281K1AR3pudQlJktnFzhdF2lUV0Hpr5oY45KCtXkjGwI7H9PnXY+vNvHVe9 Lp8A== X-Gm-Message-State: ALoCoQmyKTTmIm5YK6CvGtMhIo7EBVfQ2QBF+3RZSnVWH3oPCs1DnOxOpdwQ6buT4iuWYW+fDSp7 X-Received: by 10.220.99.72 with SMTP id t8mr26541586vcn.10.1395172486155; Tue, 18 Mar 2014 12:54:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.2.196 with HTTP; Tue, 18 Mar 2014 12:54:25 -0700 (PDT) From: Jamal Hadi Salim Date: Tue, 18 Mar 2014 15:54:25 -0400 Message-ID: To: "forces@ietf.org" Content-Type: multipart/mixed; boundary=001a11c1d71c7079ab04f4e6e83b Archived-At: http://mailarchive.ietf.org/arch/msg/forces/GAf4hx6brGmPqe_4TWH1GKumWlg Subject: [forces] Large object dump X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 19:54:55 -0000 --001a11c1d71c7079ab04f4e6e83b Content-Type: text/plain; charset=ISO-8859-1 Folks, In the latest proto update draft I have tried to clarify what RFC 5810 didnt on dumping large objects. To quote from the draft text: -- Imagine a GET request to a path that is a table i.e a table dump. Such a request is sent to the FE with a specific correlator, say X. Imagine this table to have a large number of entries at the FE. For the sake of illustration, lets say millions of rows. This requires that the FE delivers the response over multiple messages, all using the same correlator X. --- My view (based on implementation) is to re-use the transaction flags since they have no semantic meaning on the direction of the FE->CE and they are not totally off in describing the progress of the dump. Here's what our implementation does (with intention to mirror what a transaction from CE->FE will do). The AT (Atomic Transaction) flag is used to indicate "multi message" The TP(Transaction Phase) flags SOT/MOT/EOT to indicate start, middle and End of dump. There is one caveat with two alternatives: 1) The last message which carries the EOT flag would need to go the CE without any data - likely with a result code. This would be the mirror of a 2PC commit where the last message is an empty commit message. OR: 2) The last message which carries the EOT flag would need to go the CE with data What do people think? We have implemented #2 - but it seems to me #1 would be better. Attaching a small diagram which shows alternative #1 just in case i wasnt clear. cheers, jamal --001a11c1d71c7079ab04f4e6e83b Content-Type: text/plain; charset=US-ASCII; name="alternative1.txt" Content-Disposition: attachment; filename="alternative1.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hsxlnd2r0 CgkgQ0UgUEwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IEZFIFBMCgogICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8CiAgICAgICAgICAgfCAoMCkgUXVlcnksIFBhdGgtdG8tYS1sYXJn ZS1vYmplY3QsIE9QPUdFVCAgICAgICAgICAgIHwKICAgICAgICAgICB8PC0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfAogICAgICAgICAgIHwgICAg ICAgICAgICAgICAgY29ycmVsYXR0b3IgPSBYICAgICAgICAgICAgICAgICAgICAgICB8CiAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHwKICAgICAgICAgICB8ICgxKSBRdWVyeS1SZXNwb25zZSwgU09ULEFULCBPUD1HRVQtUkVT UE9OU0UsIERBVEEgICAgfAogICAgICAgICAgIHw8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18CiAgICAgICAgICAgfCAgICAgICAgICAgICAgICBj b3JyZWxhdHRvciA9IFggICAgICAgICAgICAgICAgICAgICAgIHwKICAgICAgICAgICB8ICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAg ICAgIHwgKDIpIFF1ZXJ5LVJlc3BvbnNlLCBNT1QsQVQsIE9QPUdFVC1SRVNQT05TRSwgREFUQSAg ICB8CiAgICAgICAgICAgfDwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLXwKICAgICAgICAgICB8ICAgICAgICAgICAgICAgIGNvcnJlbGF0dG9yID0g WCAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CiAgICAgICAgICAgfCAoMykgUXVl cnktUmVzcG9uc2UsIE1PVCxBVCwgT1A9R0VULVJFU1BPTlNFLCBEQVRBICAgIHwKICAgICAgICAg ICB8PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t fAogICAgICAgICAgIHwgICAgICAgICAgICAgICAgY29ycmVsYXR0b3IgPSBYICAgICAgICAgICAg ICAgICAgICAgICB8CiAgICAgICAgICAgLiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIC4KICAgICAgICAgICAuICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLgogICAgICAgICAgIC4gICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAuCiAgICAgICAgICAg LiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC4K ICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgfAogICAgICAgICAgIHwgKE4pIFF1ZXJ5LVJlc3BvbnNlLCBFT1QsQVQsIE9QPUdF VC1SRVNQT05TRSwgREFUQSAgICB8CiAgICAgICAgICAgfDwtLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwKICAgICAgICAgICB8ICAgICAgICAgICAg ICAgIGNvcnJlbGF0dG9yID0gWCAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgICAgIHwg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CiAg ICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIHwK --001a11c1d71c7079ab04f4e6e83b-- From nobody Wed Mar 26 13:11:59 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E77F1A0392 for ; Wed, 26 Mar 2014 13:11:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.771 X-Spam-Level: X-Spam-Status: No, score=-0.771 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DSL=1.129, SPF_HELO_PASS=-0.001, UNPARSEABLE_RELAY=0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y8Vdbtd-Y1dx for ; Wed, 26 Mar 2014 13:11:51 -0700 (PDT) Received: from execdsl.com (remote.shinkuro.com [50.56.68.178]) by ietfa.amsl.com (Postfix) with ESMTP id D0CB41A01EC for ; Wed, 26 Mar 2014 13:11:50 -0700 (PDT) Received: from dummy.name; Wed, 26 Mar 2014 20:11:48 +0000 Message-ID: <53333483.8060001@stevecrocker.com> Date: Wed, 26 Mar 2014 16:11:47 -0400 From: Joel User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: "forces@ietf.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/forces/HPI1Dq85cnRfKkjPMUgwU-0ilbo Subject: Re: [forces] Large object dump X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 20:11:55 -0000 This looks like a real problem. Particularly for structures with multiple large tables, any other approach requires significant complexity on the CE. As far as I can tell, using these flags for this problem is as good an answer, or better than anything else I can find. So yes, lets do that. Yours, Joel On 3/18/14, 3:54 PM, Jamal Hadi Salim wrote: > Folks, > > In the latest proto update draft I have tried to clarify what RFC 5810 didnt on > dumping large objects. To quote from the draft text: > > -- > Imagine a GET request to a path that is a table i.e a table dump. > Such a request is sent to the FE with a specific correlator, say X. > Imagine this table to have a large number of entries at the FE. For > the sake of illustration, lets say millions of rows. This requires > that the FE delivers the response over multiple messages, all using > the same correlator X. > --- > > My view (based on implementation) is to re-use the transaction flags > since they have no semantic meaning on the direction of the FE->CE > and they are not totally off in describing the progress of the dump. > Here's what our implementation does (with intention to mirror what > a transaction from CE->FE will do). > > The AT (Atomic Transaction) flag is used to indicate "multi message" > The TP(Transaction Phase) flags SOT/MOT/EOT to indicate start, > middle and End of dump. > > There is one caveat with two alternatives: > 1) The last message which carries the EOT flag would need to go the CE > without any data - likely with a result code. This would be the mirror of > a 2PC commit where the last message is an empty commit message. > OR: > 2) The last message which carries the EOT flag would need to go the CE > with data > > What do people think? We have implemented #2 - but it seems to me #1 > would be better. > Attaching a small diagram which shows alternative #1 just in case i wasnt > clear. > > cheers, > jamal > > > > _______________________________________________ > forces mailing list > forces@ietf.org > https://www.ietf.org/mailman/listinfo/forces > From nobody Thu Mar 27 03:43:55 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 037851A0601 for ; Thu, 27 Mar 2014 03:43:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.978 X-Spam-Level: X-Spam-Status: No, score=-1.978 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AvhP1bcgAOws for ; Thu, 27 Mar 2014 03:43:50 -0700 (PDT) Received: from mail-ve0-f180.google.com (mail-ve0-f180.google.com [209.85.128.180]) by ietfa.amsl.com (Postfix) with ESMTP id 89AB11A01FF for ; Thu, 27 Mar 2014 03:43:49 -0700 (PDT) Received: by mail-ve0-f180.google.com with SMTP id jz11so3774415veb.25 for ; Thu, 27 Mar 2014 03:43:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=fcUPQD3Bxx7LyfsiNTWzNC/JynU0JLpjodha9eR1jeE=; b=Zc2Q/UoCIXtGH9Cdtxdjr9xlUDDrFVfGz+BYzbH6AXxCJmmMWJaFQAMnvf2blRce+w DL2vtj70UK1l3zI7DxLwiZQ6pjh5ZzrKiEXwAle7VNDmYr3aPUdwfXmJcOK1aPR5jn2B nXLaFE1IO0J0/TKPseR7CiWM1Tlheiv0XmySCBmn4oZ93Z+2JAuXJnUTqE0FqQIQPITP bojuxdEjlLEK4k1OPi5H4DsVurEVkweH+pkg0kYQBZy3NeurXUp0wlPG+9Qml3AGcdAf 9Z6CwAWBs6QXhHRQI9FPKmRUNIHG2PCDCexGeIIJmavjlIKUpyvrCcmHfvaTf4bReGad so9w== X-Gm-Message-State: ALoCoQk1QqeSPTKJt8gmLkbQoCru17YkK1g7LJcGbcr/k3q2XYbFsFGx+ifDTMeXTqvSmjTMM1AP X-Received: by 10.58.126.4 with SMTP id mu4mr797072veb.0.1395917027613; Thu, 27 Mar 2014 03:43:47 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.2.196 with HTTP; Thu, 27 Mar 2014 03:43:26 -0700 (PDT) In-Reply-To: <53333483.8060001@stevecrocker.com> References: <53333483.8060001@stevecrocker.com> From: Jamal Hadi Salim Date: Thu, 27 Mar 2014 06:43:26 -0400 Message-ID: To: Joel Content-Type: text/plain; charset=ISO-8859-1 Archived-At: http://mailarchive.ietf.org/arch/msg/forces/XOxnZC_C5bvgvOE9vv6Hg4OKG5g Cc: "forces@ietf.org" Subject: Re: [forces] Large object dump X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 10:43:52 -0000 Joel, I am leaning towards the last message which carries the EOT flag to go the CE without any data to mirror a 2PC commit where the last message is an empty commit message. It is not how our implementation works right now, but we can change it. Does that sound reasonable? cheers, jamal On Wed, Mar 26, 2014 at 4:11 PM, Joel wrote: > This looks like a real problem. Particularly for structures with multiple > large tables, any other approach requires significant complexity on the CE. > > As far as I can tell, using these flags for this problem is as good an > answer, or better than anything else I can find. So yes, lets do that. > > Yours, > Joel > > > On 3/18/14, 3:54 PM, Jamal Hadi Salim wrote: >> >> Folks, >> >> In the latest proto update draft I have tried to clarify what RFC 5810 >> didnt on >> dumping large objects. To quote from the draft text: >> >> -- >> Imagine a GET request to a path that is a table i.e a table dump. >> Such a request is sent to the FE with a specific correlator, say X. >> Imagine this table to have a large number of entries at the FE. For >> the sake of illustration, lets say millions of rows. This requires >> that the FE delivers the response over multiple messages, all using >> the same correlator X. >> --- >> >> My view (based on implementation) is to re-use the transaction flags >> since they have no semantic meaning on the direction of the FE->CE >> and they are not totally off in describing the progress of the dump. >> Here's what our implementation does (with intention to mirror what >> a transaction from CE->FE will do). >> >> The AT (Atomic Transaction) flag is used to indicate "multi message" >> The TP(Transaction Phase) flags SOT/MOT/EOT to indicate start, >> middle and End of dump. >> >> There is one caveat with two alternatives: >> 1) The last message which carries the EOT flag would need to go the CE >> without any data - likely with a result code. This would be the mirror of >> a 2PC commit where the last message is an empty commit message. >> OR: >> 2) The last message which carries the EOT flag would need to go the CE >> with data >> >> What do people think? We have implemented #2 - but it seems to me #1 >> would be better. >> Attaching a small diagram which shows alternative #1 just in case i wasnt >> clear. >> >> cheers, >> jamal >> >> >> >> _______________________________________________ >> forces mailing list >> forces@ietf.org >> https://www.ietf.org/mailman/listinfo/forces >> > > _______________________________________________ > forces mailing list > forces@ietf.org > https://www.ietf.org/mailman/listinfo/forces From nobody Thu Mar 27 06:24:56 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 999BF1A0347 for ; Thu, 27 Mar 2014 06:24:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.771 X-Spam-Level: X-Spam-Status: No, score=-0.771 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DSL=1.129, SPF_HELO_PASS=-0.001, UNPARSEABLE_RELAY=0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d3hMG0Va85pj for ; Thu, 27 Mar 2014 06:24:52 -0700 (PDT) Received: from execdsl.com (remote.shinkuro.com [50.56.68.178]) by ietfa.amsl.com (Postfix) with ESMTP id 995CF1A00FA for ; Thu, 27 Mar 2014 06:24:52 -0700 (PDT) Received: from dummy.name; Thu, 27 Mar 2014 13:24:50 +0000 Message-ID: <533426A0.1060601@stevecrocker.com> Date: Thu, 27 Mar 2014 09:24:48 -0400 From: Joel User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Jamal Hadi Salim References: <53333483.8060001@stevecrocker.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/forces/6BDZlUPxi6qxcduZ0fhaS2CXQR8 Cc: "forces@ietf.org" Subject: Re: [forces] Large object dump X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 13:24:53 -0000 I would certainly want to permit that. The question that is not obvious is whether we need to mandate it. Yours, Joel On 3/27/14, 6:43 AM, Jamal Hadi Salim wrote: > Joel, > > I am leaning towards the last message which carries the EOT flag to go the CE > without any data to mirror a 2PC commit where the last message is an empty > commit message. It is not how our implementation works right now, but we can > change it. > > Does that sound reasonable? > > cheers, > jamal > > On Wed, Mar 26, 2014 at 4:11 PM, Joel wrote: >> This looks like a real problem. Particularly for structures with multiple >> large tables, any other approach requires significant complexity on the CE. >> >> As far as I can tell, using these flags for this problem is as good an >> answer, or better than anything else I can find. So yes, lets do that. >> >> Yours, >> Joel >> >> >> On 3/18/14, 3:54 PM, Jamal Hadi Salim wrote: >>> >>> Folks, >>> >>> In the latest proto update draft I have tried to clarify what RFC 5810 >>> didnt on >>> dumping large objects. To quote from the draft text: >>> >>> -- >>> Imagine a GET request to a path that is a table i.e a table dump. >>> Such a request is sent to the FE with a specific correlator, say X. >>> Imagine this table to have a large number of entries at the FE. For >>> the sake of illustration, lets say millions of rows. This requires >>> that the FE delivers the response over multiple messages, all using >>> the same correlator X. >>> --- >>> >>> My view (based on implementation) is to re-use the transaction flags >>> since they have no semantic meaning on the direction of the FE->CE >>> and they are not totally off in describing the progress of the dump. >>> Here's what our implementation does (with intention to mirror what >>> a transaction from CE->FE will do). >>> >>> The AT (Atomic Transaction) flag is used to indicate "multi message" >>> The TP(Transaction Phase) flags SOT/MOT/EOT to indicate start, >>> middle and End of dump. >>> >>> There is one caveat with two alternatives: >>> 1) The last message which carries the EOT flag would need to go the CE >>> without any data - likely with a result code. This would be the mirror of >>> a 2PC commit where the last message is an empty commit message. >>> OR: >>> 2) The last message which carries the EOT flag would need to go the CE >>> with data >>> >>> What do people think? We have implemented #2 - but it seems to me #1 >>> would be better. >>> Attaching a small diagram which shows alternative #1 just in case i wasnt >>> clear. >>> >>> cheers, >>> jamal >>> >>> >>> >>> _______________________________________________ >>> forces mailing list >>> forces@ietf.org >>> https://www.ietf.org/mailman/listinfo/forces >>> >> >> _______________________________________________ >> forces mailing list >> forces@ietf.org >> https://www.ietf.org/mailman/listinfo/forces From nobody Thu Mar 27 07:13:02 2014 Return-Path: X-Original-To: forces@ietfa.amsl.com Delivered-To: forces@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 89F6B1A06DD for ; Thu, 27 Mar 2014 07:12:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6_CH5BIky5f0 for ; Thu, 27 Mar 2014 07:12:56 -0700 (PDT) Received: from mail-ee0-x229.google.com (mail-ee0-x229.google.com [IPv6:2a00:1450:4013:c00::229]) by ietfa.amsl.com (Postfix) with ESMTP id B89E91A00E2 for ; Thu, 27 Mar 2014 07:12:55 -0700 (PDT) Received: by mail-ee0-f41.google.com with SMTP id t10so2965917eei.0 for ; Thu, 27 Mar 2014 07:12:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=5z+JNKVrFGTgLmIt5VoUH3WSsUx5Xhj38J1IKXW8Zkc=; b=TVjrcoIPYbU1AqCkOZ5cTJXqiTw+xl1CM9hH9QAGS/fT7Jxr6uwMtsyfbZalBhTvIt B+7E4HCHw7IHNpwvaN4hf5HsoYWsI18FPZoz+JKBNFkTAclYtAr9drglFZLsHsDuHeMR vnIlxdDHpHc2foupiWlc2UpXjD1K4cJ3HvIuEihVZLS2glpPxrHZwlmY4aoXbrqe1DrB c0nJ6r+mcYY8gPceCcsuGg1H7kHcNlkXrKBUb3CF8ScvFpg7H4KCZEdySgnryn+YNW5o PhVMy6kwBViaJrJSe91mDMZWBPKPQVWPtlvk7jMl110oRA0zEarP7ONjlsvRN3cV/MR7 G2lA== X-Received: by 10.14.203.5 with SMTP id e5mr466968eeo.105.1395929573192; Thu, 27 Mar 2014 07:12:53 -0700 (PDT) Received: from EhalepXPS (ppp141237016155.access.hol.gr. [141.237.16.155]) by mx.google.com with ESMTPSA id u46sm4655300eel.1.2014.03.27.07.12.51 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Mar 2014 07:12:52 -0700 (PDT) From: "Haleplidis Evangelos" To: "'Joel'" , "'Jamal Hadi Salim'" References: <53333483.8060001@stevecrocker.com> <533426A0.1060601@stevecrocker.com> In-Reply-To: <533426A0.1060601@stevecrocker.com> Date: Thu, 27 Mar 2014 16:12:47 +0200 Message-ID: <013501cf49c6$a287e140$e797a3c0$@com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Ac9JwAQyuhNY3QFfQLOg5BbR5o58cgABcfUA Content-Language: el Archived-At: http://mailarchive.ietf.org/arch/msg/forces/PF0DqpnwelDi02OnqE8tWZZyra0 Cc: forces@ietf.org Subject: Re: [forces] Large object dump X-BeenThere: forces@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: ForCES WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 14:12:58 -0000 Greetings Joel and Jamal and to the list, I agree that this is something that we need to address and I also agree with the proposed solution. However I think that we would need to explicitly specify in the solution that the FE MUST NOT split a table row in the middle and send it within one of those messages. For example imagine having a table with each row being a large struct with let's say 10 components. Since the proposed solution does not offer any ordering in the messages (and indeed it's not needed as row indexes are packed into the data) the struct MUST NOT be split in two ForCES messages as there is no way to reconstruct the struct again. So the FE MUST take care to stop putting table rows if current_message_size+next_row_size>maximum_message_size. Regards, Evangelos Haleplidis. > -----Original Message----- > From: forces [mailto:forces-bounces@ietf.org] On Behalf Of Joel > Sent: Thursday, March 27, 2014 3:25 PM > To: Jamal Hadi Salim > Cc: forces@ietf.org > Subject: Re: [forces] Large object dump > > I would certainly want to permit that. The question that is not > obvious is whether we need to mandate it. > > Yours, > Joel > > On 3/27/14, 6:43 AM, Jamal Hadi Salim wrote: > > Joel, > > > > I am leaning towards the last message which carries the EOT flag to > go > > the CE without any data to mirror a 2PC commit where the last > message > > is an empty commit message. It is not how our implementation works > > right now, but we can change it. > > > > Does that sound reasonable? > > > > cheers, > > jamal > > > > On Wed, Mar 26, 2014 at 4:11 PM, Joel wrote: > >> This looks like a real problem. Particularly for structures with > >> multiple large tables, any other approach requires significant > complexity on the CE. > >> > >> As far as I can tell, using these flags for this problem is as good > >> an answer, or better than anything else I can find. So yes, lets do > that. > >> > >> Yours, > >> Joel > >> > >> > >> On 3/18/14, 3:54 PM, Jamal Hadi Salim wrote: > >>> > >>> Folks, > >>> > >>> In the latest proto update draft I have tried to clarify what RFC > >>> 5810 didnt on dumping large objects. To quote from the draft text: > >>> > >>> -- > >>> Imagine a GET request to a path that is a table i.e a table > dump. > >>> Such a request is sent to the FE with a specific correlator, > say X. > >>> Imagine this table to have a large number of entries at the > FE. For > >>> the sake of illustration, lets say millions of rows. This > requires > >>> that the FE delivers the response over multiple messages, all > using > >>> the same correlator X. > >>> --- > >>> > >>> My view (based on implementation) is to re-use the transaction > flags > >>> since they have no semantic meaning on the direction of the FE->CE > >>> and they are not totally off in describing the progress of the > dump. > >>> Here's what our implementation does (with intention to mirror what > a > >>> transaction from CE->FE will do). > >>> > >>> The AT (Atomic Transaction) flag is used to indicate "multi > message" > >>> The TP(Transaction Phase) flags SOT/MOT/EOT to indicate start, > >>> middle and End of dump. > >>> > >>> There is one caveat with two alternatives: > >>> 1) The last message which carries the EOT flag would need to go the > >>> CE without any data - likely with a result code. This would be the > >>> mirror of a 2PC commit where the last message is an empty commit > message. > >>> OR: > >>> 2) The last message which carries the EOT flag would need to go the > >>> CE with data > >>> > >>> What do people think? We have implemented #2 - but it seems to me > #1 > >>> would be better. > >>> Attaching a small diagram which shows alternative #1 just in case i > >>> wasnt clear. > >>> > >>> cheers, > >>> jamal > >>> > >>> > >>> > >>> _______________________________________________ > >>> forces mailing list > >>> forces@ietf.org > >>> https://www.ietf.org/mailman/listinfo/forces > >>> > >> > >> _______________________________________________ > >> forces mailing list > >> forces@ietf.org > >> https://www.ietf.org/mailman/listinfo/forces > > _______________________________________________ > forces mailing list > forces@ietf.org > https://www.ietf.org/mailman/listinfo/forces