From vrock28@gmail.com Tue Mar 25 04:57:09 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 653C81A0070 for ; Tue, 25 Mar 2014 04:57:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.951 X-Spam-Level: X-Spam-Status: No, score=0.951 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xx2uu6JaCich for ; Tue, 25 Mar 2014 04:57:07 -0700 (PDT) Received: from mail-ve0-x234.google.com (mail-ve0-x234.google.com [IPv6:2607:f8b0:400c:c01::234]) by ietfa.amsl.com (Postfix) with ESMTP id 3E44A1A006E for ; Tue, 25 Mar 2014 04:57:07 -0700 (PDT) Received: by mail-ve0-f180.google.com with SMTP id jz11so358091veb.39 for ; Tue, 25 Mar 2014 04:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=+ycl6xac1lGuK6wa6MocoqsY/vdtWeapAoVDMwe9Ekc=; b=0xWK+W/KoqMesPLGYGzRT1EwwVAy4HhYUFf6SN+vKFgAFEwIGH6ISr9XDzcXUvVcu+ mu9ipr9eIKAgGm/y7Q4ZPXu2xlf0xQQgnFvF8ZM37r9s/yf9aPnWW39nZmjbYHNilPwN tFdzbXIXatLMjopCsuDYaPqG0TwhmRFBWnBfccSvJbrSiP++rUAmdr+XPOJPu8gi2/Ec OrFAPAugBSuS7znHHIQqsy9R4oKLHaU1MQXQpS2nrjOHbM9RmhQDH5NvDV5RWk35MlbL bu5prmPnii37VRPAceohcNukfjdc2P+XTivRVNqhtxtX6PlB14CQsD6g3eB7vrumad3r uaPA== X-Received: by 10.58.38.166 with SMTP id h6mr7086252vek.22.1395748625534; Tue, 25 Mar 2014 04:57:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.92.233 with HTTP; Tue, 25 Mar 2014 04:56:45 -0700 (PDT) From: Rakshith Venkatesh Date: Tue, 25 Mar 2014 17:26:45 +0530 Message-ID: To: architecture-discuss@ietf.org Content-Type: multipart/alternative; boundary=089e013a0d4005643704f56d0de4 Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/SIOf9lhbnH93rQCGieS8qHXuKiA X-Mailman-Approved-At: Tue, 25 Mar 2014 09:03:26 -0700 Subject: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 11:59:17 -0000 --089e013a0d4005643704f56d0de4 Content-Type: text/plain; charset=ISO-8859-1 Hi, I was going through SPDY draft or the HTTP 2.0 initial draft. I had an idea which I think would require a change in the architecture of the same and so I am dropping this mail. Here is the idea: A Http client expects the server to send data in-order. (NOTE: When I say a server, I am referring to an appliance to which disks are attached and the file resides on the disk.). If there is a file let's say 10GB and if the client asks for this file from the server, the data has to be given in-order from the 1st byte till the last byte. Now let's say I implement an engine at the server side to do a parallel read on this huge file to fetch data at various offsets within the file, I will be able to fetch the data faster for sure but, I will not be able to send across the data immediately as and when I fetch it. I am expected to finish doing all parallel reads on the file till I read all the bytes, then send across the wire to the client. Now if we can have some tag or header which can be introduced as part of HTTP 2.0 which actually can help re-order the byte stream at the session layer or at a layer in between the application and session layer, we could potentially improve the performance for File reads using HTTP by making sure that the new module looks at this new tag and jumbles around the data based on this tag and eventually presents the data to HTTP to make it look all seamless. So the server can just do parallel reads on the same file at various offsets without worrying about ordering and send across the read chunks and this new module sitting at the client side can intervene and look at some form of tag/header to make sure it swaps the data accordingly and waits till all data is received and then present it to the Application protocol. NOTE: I am not referring to packet-reordering at TCP. By having this, servers can attain true parallelism in reading the file and can effectively improve file transfer rates. Thanks, Rakshith --089e013a0d4005643704f56d0de4 Content-Type: text/html; charset=ISO-8859-1

Hi,

 

I was going through SPDY draft or the HTTP 2.0 initial draft. I had an idea which I think would require a change in the architecture of the same and so I am dropping this mail. Here is the idea:

 

A Http client expects the server to send data in-order. (NOTE: When I say a server, I am referring to an appliance to which disks are attached and the file resides on the disk.). If there is a file let’s say 10GB and if the client asks for this file from the server, the data has to be given in-order from the 1st byte till the last byte. Now let’s say I implement an engine at the server side to do a parallel read on this huge file to fetch data at various offsets within the file, I will be able to fetch the data faster for sure but, I will not be able to send across the data immediately as and when I fetch it. I am expected to finish doing all parallel reads on the file till I read all the bytes, then send across the wire to the client.

 

Now if we can have some tag or header which can be introduced as part of HTTP 2.0 which actually can help re-order the byte stream at the session layer or at a layer in between the application and session layer, we could potentially improve the performance for File reads using HTTP by making sure that the new module looks at this new tag and jumbles around the data based on this tag and eventually presents the data to HTTP to make it look all seamless.

 

So the server can just do parallel reads on the same file at various offsets without worrying about ordering and send across the read chunks and this new module sitting at the client side can intervene and look at some form of tag/header to make sure it swaps the data accordingly and waits till all data is received and then present it to the Application protocol.

 

NOTE: I am not referring to packet-reordering at TCP.

 

By having this, servers can attain true parallelism in reading the file and can effectively improve file transfer rates.

 

Thanks,

Rakshith

--089e013a0d4005643704f56d0de4-- From nobody Tue Mar 25 14:17:49 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C06861A0241 for ; Tue, 25 Mar 2014 14:17:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DTnIrhZPU2I7 for ; Tue, 25 Mar 2014 14:17:45 -0700 (PDT) Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id A632E1A0239 for ; Tue, 25 Mar 2014 14:17:45 -0700 (PDT) Received: from [128.9.160.81] (nib.isi.edu [128.9.160.81]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id s2PLHG6o003559 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 25 Mar 2014 14:17:16 -0700 (PDT) Message-ID: <5331F25C.20803@isi.edu> Date: Tue, 25 Mar 2014 14:17:16 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rakshith Venkatesh , architecture-discuss@ietf.org References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/mmww-DeYEOeWETHy28BTiDQG3e0 Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 21:17:48 -0000 You sound like you're looking for NFS; maybe you should propose an "NSF over HTTP" variant where responses can be issued out of order? Joe On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: > Hi, > > I was going through SPDY draft or the HTTP 2.0 initial draft. I had an > idea which I think would require a change in the architecture of the > same and so I am dropping this mail. Here is the idea: > > A Http client expects the server to send data in-order. (NOTE: When I > say a server, I am referring to an appliance to which disks are attached > and the file resides on the disk.). If there is a file let’s say 10GB > and if the client asks for this file from the server, the data has to be > given in-order from the 1^st byte till the last byte. Now let’s say I > implement an engine at the server side to do a parallel read on this > huge file to fetch data at various offsets within the file, I will be > able to fetch the data faster for sure but, I will not be able to send > across the data immediately as and when I fetch it. I am expected to > finish doing all parallel reads on the file till I read all the bytes, > then send across the wire to the client. > > Now if we can have some tag or header which can be introduced as part of > HTTP 2.0 which actually can help re-order the byte stream at the session > layer or at a layer in between the application and session layer, we > could potentially improve the performance for File reads using HTTP by > making sure that the new module looks at this new tag and jumbles around > the data based on this tag and eventually presents the data to HTTP to > make it look all seamless. > > So the server can just do parallel reads on the same file at various > offsets without worrying about ordering and send across the read chunks > and this new module sitting at the client side can intervene and look at > some form of tag/header to make sure it swaps the data accordingly and > waits till all data is received and then present it to the Application > protocol. > > NOTE: I am not referring to packet-reordering at TCP. > > By having this, servers can attain true parallelism in reading the file > and can effectively improve file transfer rates. > > > Thanks, > > Rakshith > > > > _______________________________________________ > Architecture-discuss mailing list > Architecture-discuss@ietf.org > https://www.ietf.org/mailman/listinfo/architecture-discuss > From nobody Tue Mar 25 18:02:27 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E3E291A027E for ; Tue, 25 Mar 2014 18:02:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.749 X-Spam-Level: X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FU2TCGm6oV1u for ; Tue, 25 Mar 2014 18:02:20 -0700 (PDT) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) by ietfa.amsl.com (Postfix) with ESMTP id 034351A0275 for ; Tue, 25 Mar 2014 18:02:19 -0700 (PDT) Received: by mail-vc0-f176.google.com with SMTP id lc6so1539584vcb.21 for ; Tue, 25 Mar 2014 18:02:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=px7t9cg2F4V5ZNzZM8PsMLTiDLA1XQQrVU1DMN04SCw=; b=A6cBVGbiwCB+yuXnH5zijF96iut+F8lKLzk2IxyjZsE9pMaCRizaiOt0cnqb6pSd/D UcEIscQXbqBsU7llTGZoWgGK4KldCQ24fQrGzF0YLZp1gm+WyaC4YRWNrDXxHlVkWvYD KZY3Ut4uyAjF2aVJMnGVfyN+odNQcyyXUdNOSnUhcbITSifuFFkjueYvcrWPPTF4EbBV aQDky3HjybYl+N4iDBtrI38k3ea61Ohen7qqjjnvOe+G9CE2GP7QwdT+0Asa6Lq1lawg 5Hd5WwkKuENV+9X41KStDht7fc1ohc20K+yLzdwFlzKtpRogDdBmdUmp1bHR71a07woq 7l/A== X-Received: by 10.220.159.4 with SMTP id h4mr47190809vcx.1.1395795738415; Tue, 25 Mar 2014 18:02:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.92.233 with HTTP; Tue, 25 Mar 2014 18:01:58 -0700 (PDT) In-Reply-To: <5331F25C.20803@isi.edu> References: <5331F25C.20803@isi.edu> From: Rakshith Venkatesh Date: Wed, 26 Mar 2014 06:31:58 +0530 Message-ID: To: Joe Touch Content-Type: multipart/alternative; boundary=001a11c2ca0c2ad98b04f578059b Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/KK2BHpFBfY6_imzV5A-dSdzaGB0 Cc: architecture-discuss@ietf.org Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 01:02:23 -0000 --001a11c2ca0c2ad98b04f578059b Content-Type: text/plain; charset=ISO-8859-1 I am not sure if an NFS client can accept blocks out of order from a server.I need to check on that. If it does then NFS over HTTP sounds good. The way i see it is any File service protocol such as NFS, SMB (Large MTU's), HTTP, FTP should have a way which would help the server achieve true parallelism in reading a file and not worry about sending data/blocks in-order. Rakshith On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch wrote: > You sound like you're looking for NFS; maybe you should propose an "NSF > over HTTP" variant where responses can be issued out of order? > > Joe > > > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: > >> Hi, >> >> I was going through SPDY draft or the HTTP 2.0 initial draft. I had an >> idea which I think would require a change in the architecture of the >> same and so I am dropping this mail. Here is the idea: >> >> A Http client expects the server to send data in-order. (NOTE: When I >> say a server, I am referring to an appliance to which disks are attached >> and the file resides on the disk.). If there is a file let's say 10GB >> and if the client asks for this file from the server, the data has to be >> given in-order from the 1^st byte till the last byte. Now let's say I >> >> implement an engine at the server side to do a parallel read on this >> huge file to fetch data at various offsets within the file, I will be >> able to fetch the data faster for sure but, I will not be able to send >> across the data immediately as and when I fetch it. I am expected to >> finish doing all parallel reads on the file till I read all the bytes, >> then send across the wire to the client. >> >> Now if we can have some tag or header which can be introduced as part of >> HTTP 2.0 which actually can help re-order the byte stream at the session >> layer or at a layer in between the application and session layer, we >> could potentially improve the performance for File reads using HTTP by >> making sure that the new module looks at this new tag and jumbles around >> the data based on this tag and eventually presents the data to HTTP to >> make it look all seamless. >> >> So the server can just do parallel reads on the same file at various >> offsets without worrying about ordering and send across the read chunks >> and this new module sitting at the client side can intervene and look at >> some form of tag/header to make sure it swaps the data accordingly and >> waits till all data is received and then present it to the Application >> protocol. >> >> NOTE: I am not referring to packet-reordering at TCP. >> >> By having this, servers can attain true parallelism in reading the file >> and can effectively improve file transfer rates. >> >> >> Thanks, >> >> Rakshith >> >> >> >> _______________________________________________ >> Architecture-discuss mailing list >> Architecture-discuss@ietf.org >> https://www.ietf.org/mailman/listinfo/architecture-discuss >> >> --001a11c2ca0c2ad98b04f578059b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I am not sure if an NFS client can accept blocks out of or= der from a server.I need to check on that. If it does then NFS over HTTP so= unds good. The way i see it is any File service protocol such as NFS, SMB (= Large MTU's), HTTP, FTP should have a way which would help the server a= chieve true parallelism in reading a file and not worry about sending data/= blocks in-order.

Rakshith 

On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch <touch@= isi.edu> wrote:
You sound like you're looking for NFS; m= aybe you should propose an "NSF over HTTP" variant where response= s can be issued out of order?

Joe


On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote:
Hi,

I was going through SPDY draft or the HTTP 2.0 initial draft. I had an
idea which I think would require a change in the architecture of the
same and so I am dropping this mail. Here is the idea:

A Http client expects the server to send data in-order. (NOTE: When I
say a server, I am referring to an appliance to which disks are attached and the file resides on the disk.). If there is a file let’s say 10GB=
and if the client asks for this file from the server, the data has to be
given in-order from the 1^st byte till the last byte. Now let’s say I=

implement an engine at the server side to do a parallel read on this
huge file to fetch data at various offsets within the file, I will be
able to fetch the data faster for sure but, I will not be able to send
across the data immediately as and when I fetch it. I am expected to
finish doing all parallel reads on the file till I read all the bytes,
then send across the wire to the client.

Now if we can have some tag or header which can be introduced as part of HTTP 2.0 which actually can help re-order the byte stream at the session layer or at a layer in between the application and session layer, we
could potentially improve the performance for File reads using HTTP by
making sure that the new module looks at this new tag and jumbles around the data based on this tag and eventually presents the data to HTTP to
make it look all seamless.

So the server can just do parallel reads on the same file at various
offsets without worrying about ordering and send across the read chunks
and this new module sitting at the client side can intervene and look at some form of tag/header to make sure it swaps the data accordingly and
waits till all data is received and then present it to the Application
protocol.

NOTE: I am not referring to packet-reordering at TCP.

By having this, servers can attain true parallelism in reading the file
and can effectively improve file transfer rates.


Thanks,

Rakshith



_______________________________________________
Architecture-discuss mailing list
Architec= ture-discuss@ietf.org
https://www.ietf.org/mailman/listinfo/architecture-dis= cuss


--001a11c2ca0c2ad98b04f578059b-- From nobody Tue Mar 25 22:17:12 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 647491A00D7 for ; Tue, 25 Mar 2014 22:17:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fFgbmJDlxDlH for ; Tue, 25 Mar 2014 22:17:07 -0700 (PDT) Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id 4C1EE1A00C0 for ; Tue, 25 Mar 2014 22:17:07 -0700 (PDT) Received: from [192.168.1.91] (pool-71-105-87-112.lsanca.dsl-w.verizon.net [71.105.87.112]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id s2Q5Ghvl025734 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 25 Mar 2014 22:16:47 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Content-Type: text/plain; charset=windows-1252 From: Joe Touch In-Reply-To: Date: Tue, 25 Mar 2014 22:16:42 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <5EE5585A-19E4-4940-B21A-4BA208F08B78@isi.edu> References: <5331F25C.20803@isi.edu> To: Rakshith Venkatesh X-Mailer: Apple Mail (2.1874) X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/GmozKDPeOFdeMGJ7Xr8V4KRIKEs Cc: architecture-discuss@ietf.org Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 05:17:09 -0000 On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh = wrote: > I am not sure if an NFS client can accept blocks out of order from a = server. I need to check on that. NFS should be matching read responses to requests; order should not = matter, esp. because when using UDP the requests and responses can be = reordered by the network. > If it does then NFS over HTTP sounds good. The way i see it is any = File service protocol such as NFS, SMB (Large MTU's), HTTP, FTP should = have a way which would help the server achieve true parallelism in = reading a file and not worry about sending data/blocks in-order. FTP has block mode that should support any order of response, but if = they're all going over the same TCP connection it can help only with the = source access order rather than network reordering. HTTP has chunking that should help with this (e.g., using a chunk = extension field that indicates chunk offset). However, most file systems are engineered to manage files via in-order = access, even when parallelized, so I'm not sure how much this will all = help. I.e., I don't know if your assumption is valid, that overriding = the serial access will be faster. It might be useful to show that you can get to the components of a file = faster as you assume first. Joe >=20 > Rakshith=20 >=20 >=20 > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch wrote: > You sound like you're looking for NFS; maybe you should propose an = "NSF over HTTP" variant where responses can be issued out of order? >=20 > Joe >=20 >=20 > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: > Hi, >=20 > I was going through SPDY draft or the HTTP 2.0 initial draft. I had an > idea which I think would require a change in the architecture of the > same and so I am dropping this mail. Here is the idea: >=20 > A Http client expects the server to send data in-order. (NOTE: When I > say a server, I am referring to an appliance to which disks are = attached > and the file resides on the disk.). If there is a file let=92s say = 10GB > and if the client asks for this file from the server, the data has to = be > given in-order from the 1^st byte till the last byte. Now let=92s say = I >=20 > implement an engine at the server side to do a parallel read on this > huge file to fetch data at various offsets within the file, I will be > able to fetch the data faster for sure but, I will not be able to send > across the data immediately as and when I fetch it. I am expected to > finish doing all parallel reads on the file till I read all the bytes, > then send across the wire to the client. >=20 > Now if we can have some tag or header which can be introduced as part = of > HTTP 2.0 which actually can help re-order the byte stream at the = session > layer or at a layer in between the application and session layer, we > could potentially improve the performance for File reads using HTTP by > making sure that the new module looks at this new tag and jumbles = around > the data based on this tag and eventually presents the data to HTTP to > make it look all seamless. >=20 > So the server can just do parallel reads on the same file at various > offsets without worrying about ordering and send across the read = chunks > and this new module sitting at the client side can intervene and look = at > some form of tag/header to make sure it swaps the data accordingly and > waits till all data is received and then present it to the Application > protocol. >=20 > NOTE: I am not referring to packet-reordering at TCP. >=20 > By having this, servers can attain true parallelism in reading the = file > and can effectively improve file transfer rates. >=20 >=20 > Thanks, >=20 > Rakshith >=20 >=20 >=20 > _______________________________________________ > Architecture-discuss mailing list > Architecture-discuss@ietf.org > https://www.ietf.org/mailman/listinfo/architecture-discuss >=20 >=20 From nobody Tue Mar 25 23:21:27 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 20CB61A01DD for ; Tue, 25 Mar 2014 23:21:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.749 X-Spam-Level: X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HpsnWs02ihJG for ; Tue, 25 Mar 2014 23:21:20 -0700 (PDT) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) by ietfa.amsl.com (Postfix) with ESMTP id 50DEA1A01F0 for ; Tue, 25 Mar 2014 23:21:20 -0700 (PDT) Received: by mail-ve0-f173.google.com with SMTP id oy12so1810709veb.32 for ; Tue, 25 Mar 2014 23:21:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=wuxAgznTpLTiBjEAoZZnbQQPo+PytjzCCWmujFK7XBc=; b=WAJs+pfrN2Aib8Y1LOTgEy3CrC58ZO/GR4rQ7uJkNgGwYHYan0Khajb+RMZ3YeAiXc VPBvJA2C9XlCT/cCyh18WnvhcC0ShKYBUYUppx9vA4r6XvPNXRiiPrkYTuo2iHA+W1ls 8IwhM6H0HDsLQYCYoGLgMyjex3UQXm0PoaOcczGJA38/TdtQgNZoSQy7l6ObtYHwFyCV 9GW7/i4Cjx3dQ4X3SxpdzlVgLnQ6KPAzIa88PQoXrzj9Aj6MCkQoETjdP3M7rn0p5nWX N2wfxc6mIxlyI7BU4Ws3hO9xWEvN6Aa1A3baIrm8zhdtUHR6XcMvtwSQRODo5Rzxsb2O HtPw== X-Received: by 10.220.88.204 with SMTP id b12mr58413629vcm.3.1395814878789; Tue, 25 Mar 2014 23:21:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.92.233 with HTTP; Tue, 25 Mar 2014 23:20:58 -0700 (PDT) In-Reply-To: <5EE5585A-19E4-4940-B21A-4BA208F08B78@isi.edu> References: <5331F25C.20803@isi.edu> <5EE5585A-19E4-4940-B21A-4BA208F08B78@isi.edu> From: Rakshith Venkatesh Date: Wed, 26 Mar 2014 11:50:58 +0530 Message-ID: To: Joe Touch Content-Type: multipart/alternative; boundary=047d7b3a92ee05c6dc04f57c7af1 Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/DMlMQhHsfvKW7_Ic8-fpfBBPXLY Cc: architecture-discuss@ietf.org Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 06:21:24 -0000 --047d7b3a92ee05c6dc04f57c7af1 Content-Type: text/plain; charset=ISO-8859-1 I think i should be explaining what was done and what we saw as improvements. What i meant was lets say the block size of a file system is 64kB. Now, if a file is lets say 512GB, in the normal world, i would fetch 64kB of data sequentially until i read all of data. Now every 64KB data sent across the wire could get chunked by TCP and the chunks would get re-ordered at the client side to present the 64KB data to the application in-tact. Please note that i am not referring to this re-ordering as part of my idea. Further, what was done was divide the whole file into 64KB zones and have X workers waiting to fetch each zone. So 1 to X workers just went ahead reading the first X zones of data. Whoever came back first among these X workers was assigned the task of reading the X+1 zone and so on and so forth. This experiment saw a minimum performance improvement of 20-30% in terms of time taken to read the file. The only thing that stopped us from sending the reads across the wire was that the client expected the data bytes to be in-order. i.e: if the any worker came back first before the 1st worker, i could not send the data that was fetched by the that worker because of the constraint that i have mentioned previously. This also led to a situation where i had to hold onto the buffer resources even though i completed reading sub-portions of the file because fetching the whole file was yet to be finished. On Wed, Mar 26, 2014 at 10:46 AM, Joe Touch wrote: > > On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh wrote: > > > I am not sure if an NFS client can accept blocks out of order from a > server. I need to check on that. > > NFS should be matching read responses to requests; order should not > matter, esp. because when using UDP the requests and responses can be > reordered by the network. > > > If it does then NFS over HTTP sounds good. The way i see it is any File > service protocol such as NFS, SMB (Large MTU's), HTTP, FTP should have a > way which would help the server achieve true parallelism in reading a file > and not worry about sending data/blocks in-order. > > FTP has block mode that should support any order of response, but if > they're all going over the same TCP connection it can help only with the > source access order rather than network reordering. > > HTTP has chunking that should help with this (e.g., using a chunk > extension field that indicates chunk offset). > > However, most file systems are engineered to manage files via in-order > access, even when parallelized, so I'm not sure how much this will all > help. I.e., I don't know if your assumption is valid, that overriding the > serial access will be faster. > > It might be useful to show that you can get to the components of a file > faster as you assume first. > > Joe > > > > > > Rakshith > > > > > > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch wrote: > > You sound like you're looking for NFS; maybe you should propose an "NSF > over HTTP" variant where responses can be issued out of order? > > > > Joe > > > > > > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: > > Hi, > > > > I was going through SPDY draft or the HTTP 2.0 initial draft. I had an > > idea which I think would require a change in the architecture of the > > same and so I am dropping this mail. Here is the idea: > > > > A Http client expects the server to send data in-order. (NOTE: When I > > say a server, I am referring to an appliance to which disks are attached > > and the file resides on the disk.). If there is a file let's say 10GB > > and if the client asks for this file from the server, the data has to be > > given in-order from the 1^st byte till the last byte. Now let's say I > > > > implement an engine at the server side to do a parallel read on this > > huge file to fetch data at various offsets within the file, I will be > > able to fetch the data faster for sure but, I will not be able to send > > across the data immediately as and when I fetch it. I am expected to > > finish doing all parallel reads on the file till I read all the bytes, > > then send across the wire to the client. > > > > Now if we can have some tag or header which can be introduced as part of > > HTTP 2.0 which actually can help re-order the byte stream at the session > > layer or at a layer in between the application and session layer, we > > could potentially improve the performance for File reads using HTTP by > > making sure that the new module looks at this new tag and jumbles around > > the data based on this tag and eventually presents the data to HTTP to > > make it look all seamless. > > > > So the server can just do parallel reads on the same file at various > > offsets without worrying about ordering and send across the read chunks > > and this new module sitting at the client side can intervene and look at > > some form of tag/header to make sure it swaps the data accordingly and > > waits till all data is received and then present it to the Application > > protocol. > > > > NOTE: I am not referring to packet-reordering at TCP. > > > > By having this, servers can attain true parallelism in reading the file > > and can effectively improve file transfer rates. > > > > > > Thanks, > > > > Rakshith > > > > > > > > _______________________________________________ > > Architecture-discuss mailing list > > Architecture-discuss@ietf.org > > https://www.ietf.org/mailman/listinfo/architecture-discuss > > > > > > --047d7b3a92ee05c6dc04f57c7af1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I think i should be explaining what was done and what we saw as improvemen= ts. What i meant was lets say the block size of a file system is 64kB. Now,= if a file is lets say 512GB, in the normal world, i would fetch 64kB of da= ta sequentially until i read all of data. Now every 64KB data sent across t= he wire could get chunked by TCP and the chunks would get re-ordered at the= client side to present the 64KB data to the application in-tact. Please no= te that i am not referring to this re-ordering as part of my idea. 

=
 Further, w= hat was done was divide the whole file into 64KB zones and have X workers w= aiting to fetch each zone. So 1 to X workers just went ahead reading the fi= rst X  zones of data. Whoever came back first among these X workers wa= s assigned the task of reading the X+1 zone and so on and so forth. 

=
This experiment saw a minimum performance improvement of 20-30% in ter= ms of time taken to read the file. The only thing that stopped us from send= ing the reads across the wire was that the client expected the data bytes t= o be in-order. i.e: if the any worker came back first before the 1st worker= , i could not send the data that was fetched by the that worker because of = the constraint that i have mentioned previously.

This also led to a situation where i had to hold = onto the buffer resources even though i completed reading sub-portions of t= he file because fetching  the whole file was yet to be finished.


On Wed, Mar 2= 6, 2014 at 10:46 AM, Joe Touch <touch@isi.edu> wrote:

On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh <vrock28@gmail.com> wrote:

> I am not sure if an NFS client can accept blocks out of order from a s= erver. I need to check on that.

NFS should be matching read responses to requests; order should not matter,= esp. because when using UDP the requests and responses can be reordered by= the network.

> If it does then NFS over HTTP sounds good. The way i see it is any Fil= e service protocol such as NFS, SMB (Large MTU's), HTTP, FTP should hav= e a way which would help the server achieve true parallelism in reading a f= ile and not worry about sending data/blocks in-order.

FTP has block mode that should support any order of response, but if = they're all going over the same TCP connection it can help only with th= e source access order rather than network reordering.

HTTP has chunking that should help with this (e.g., using a chunk extension= field that indicates chunk offset).

However, most file systems are engineered to manage files via in-order acce= ss, even when parallelized, so I'm not sure how much this will all help= . I.e., I don't know if your assumption is valid, that overriding the s= erial access will be faster.

It might be useful to show that you can get to the components of a file fas= ter as you assume first.

Joe


>
> Rakshith
>
>
> On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch <touch@isi.edu> wrote:
> You sound like you're looking for NFS; maybe you should propose an= "NSF over HTTP" variant where responses can be issued out of ord= er?
>
> Joe
>
>
> On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote:
> Hi,
>
> I was going through SPDY draft or the HTTP 2.0 initial draft. I had an=
> idea which I think would require a change in the architecture of the > same and so I am dropping this mail. Here is the idea:
>
> A Http client expects the server to send data in-order. (NOTE: When I<= br> > say a server, I am referring to an appliance to which disks are attach= ed
> and the file resides on the disk.). If there is a file let’s say= 10GB
> and if the client asks for this file from the server, the data has to = be
> given in-order from the 1^st byte till the last byte. Now let’s = say I
>
> implement an engine at the server side to do a parallel read on this > huge file to fetch data at various offsets within the file, I will be<= br> > able to fetch the data faster for sure but, I will not be able to send=
> across the data immediately as and when I fetch it. I am expected to > finish doing all parallel reads on the file till I read all the bytes,=
> then send across the wire to the client.
>
> Now if we can have some tag or header which can be introduced as part = of
> HTTP 2.0 which actually can help re-order the byte stream at the sessi= on
> layer or at a layer in between the application and session layer, we > could potentially improve the performance for File reads using HTTP by=
> making sure that the new module looks at this new tag and jumbles arou= nd
> the data based on this tag and eventually presents the data to HTTP to=
> make it look all seamless.
>
> So the server can just do parallel reads on the same file at various > offsets without worrying about ordering and send across the read chunk= s
> and this new module sitting at the client side can intervene and look = at
> some form of tag/header to make sure it swaps the data accordingly and=
> waits till all data is received and then present it to the Application=
> protocol.
>
> NOTE: I am not referring to packet-reordering at TCP.
>
> By having this, servers can attain true parallelism in reading the fil= e
> and can effectively improve file transfer rates.
>
>
> Thanks,
>
> Rakshith
>
>
>
> _______________________________________________
> Architecture-discuss mailing list
> Architecture-discuss@= ietf.org
> https://www.ietf.org/mailman/listinfo/architecture-discu= ss
>
>


--047d7b3a92ee05c6dc04f57c7af1-- From nobody Wed Mar 26 06:21:51 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 394C41A00FB for ; Wed, 26 Mar 2014 06:21:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PZFg-tRSUu2u for ; Wed, 26 Mar 2014 06:21:46 -0700 (PDT) Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id EA1FB1A0100 for ; Wed, 26 Mar 2014 06:21:45 -0700 (PDT) Received: from [192.168.1.93] (pool-71-105-87-112.lsanca.dsl-w.verizon.net [71.105.87.112]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id s2QDLEHb016852 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 26 Mar 2014 06:21:17 -0700 (PDT) Message-ID: <5332D44B.1010003@isi.edu> Date: Wed, 26 Mar 2014 06:21:15 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rakshith Venkatesh References: <5331F25C.20803@isi.edu> <5EE5585A-19E4-4940-B21A-4BA208F08B78@isi.edu> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/XV1t5LOTfniYTdITehDDCCH3ZGs Cc: architecture-discuss@ietf.org Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 13:21:49 -0000 On 3/25/2014 11:20 PM, Rakshith Venkatesh wrote: > I think i should be explaining what was done and what we saw as > improvements. What i meant was lets say the block size of a file system > is 64kB. Now, if a file is lets say 512GB, in the normal world, i would > fetch 64kB of data sequentially until i read all of data. Now every 64KB > data sent across the wire could get chunked by TCP and the chunks would > get re-ordered at the client side to present the 64KB data to the > application in-tact. Please note that i am not referring to this > re-ordering as part of my idea. > > Further, what was done was divide the whole file into 64KB zones and > have X workers waiting to fetch each zone. So 1 to X workers just went > ahead reading the first X zones of data. Whoever came back first among > these X workers was assigned the task of reading the X+1 zone and so on > and so forth. That sounds like you tried to start all X workers at the same time, but such events are rarely truly simultaneous. I.e., the way you initiated your system may be causing the out-of-order responses. Or there could be a resource interaction in which the order you see is a side-effect of different processes or threads interacting with shared locks. > This experiment saw a minimum performance improvement of 20-30% in terms > of time taken to read the file. The only thing that stopped us from > sending the reads across the wire was that the client expected the data > bytes to be in-order. i.e: if the any worker came back first before the > 1st worker, i could not send the data that was fetched by the that > worker because of the constraint that i have mentioned previously. So let's assume you could transmit these blocks in the order they return from reads. Have you considered the impact on the receive side? I.e., you'd need to do the same study on the impact of out of order writes on performance. > This also led to a situation where i had to hold onto the buffer > resources even though i completed reading sub-portions of the file > because fetching the whole file was yet to be finished. Sure, but unless your send and receive side are exactly matched in terms of the order impact on performance, one side will end up holding buffers too. I.e., it sounds like you have some OS research to complete before you know you'll be able to take advantage of block-oriented framing in the transport protocol. Joe > > > On Wed, Mar 26, 2014 at 10:46 AM, Joe Touch > wrote: > > > On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh > wrote: > > > I am not sure if an NFS client can accept blocks out of order > from a server. I need to check on that. > > NFS should be matching read responses to requests; order should not > matter, esp. because when using UDP the requests and responses can > be reordered by the network. > > > If it does then NFS over HTTP sounds good. The way i see it is > any File service protocol such as NFS, SMB (Large MTU's), HTTP, FTP > should have a way which would help the server achieve true > parallelism in reading a file and not worry about sending > data/blocks in-order. > > FTP has block mode that should support any order of response, but if > they're all going over the same TCP connection it can help only with > the source access order rather than network reordering. > > HTTP has chunking that should help with this (e.g., using a chunk > extension field that indicates chunk offset). > > However, most file systems are engineered to manage files via > in-order access, even when parallelized, so I'm not sure how much > this will all help. I.e., I don't know if your assumption is valid, > that overriding the serial access will be faster. > > It might be useful to show that you can get to the components of a > file faster as you assume first. > > Joe > > > > > > Rakshith > > > > > > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch > wrote: > > You sound like you're looking for NFS; maybe you should propose > an "NSF over HTTP" variant where responses can be issued out of order? > > > > Joe > > > > > > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: > > Hi, > > > > I was going through SPDY draft or the HTTP 2.0 initial draft. I > had an > > idea which I think would require a change in the architecture of the > > same and so I am dropping this mail. Here is the idea: > > > > A Http client expects the server to send data in-order. (NOTE: When I > > say a server, I am referring to an appliance to which disks are > attached > > and the file resides on the disk.). If there is a file let’s say 10GB > > and if the client asks for this file from the server, the data > has to be > > given in-order from the 1^st byte till the last byte. Now let’s say I > > > > implement an engine at the server side to do a parallel read on this > > huge file to fetch data at various offsets within the file, I will be > > able to fetch the data faster for sure but, I will not be able to > send > > across the data immediately as and when I fetch it. I am expected to > > finish doing all parallel reads on the file till I read all the > bytes, > > then send across the wire to the client. > > > > Now if we can have some tag or header which can be introduced as > part of > > HTTP 2.0 which actually can help re-order the byte stream at the > session > > layer or at a layer in between the application and session layer, we > > could potentially improve the performance for File reads using > HTTP by > > making sure that the new module looks at this new tag and jumbles > around > > the data based on this tag and eventually presents the data to > HTTP to > > make it look all seamless. > > > > So the server can just do parallel reads on the same file at various > > offsets without worrying about ordering and send across the read > chunks > > and this new module sitting at the client side can intervene and > look at > > some form of tag/header to make sure it swaps the data > accordingly and > > waits till all data is received and then present it to the > Application > > protocol. > > > > NOTE: I am not referring to packet-reordering at TCP. > > > > By having this, servers can attain true parallelism in reading > the file > > and can effectively improve file transfer rates. > > > > > > Thanks, > > > > Rakshith > > > > > > > > _______________________________________________ > > Architecture-discuss mailing list > > Architecture-discuss@ietf.org > > https://www.ietf.org/mailman/listinfo/architecture-discuss > > > > > > From nobody Wed Mar 26 21:06:49 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7681B1A0293 for ; Wed, 26 Mar 2014 21:06:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.749 X-Spam-Level: X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VHtZv32JYQOC for ; Wed, 26 Mar 2014 21:06:42 -0700 (PDT) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) by ietfa.amsl.com (Postfix) with ESMTP id 3DFD31A0438 for ; Wed, 26 Mar 2014 21:06:42 -0700 (PDT) Received: by mail-vc0-f178.google.com with SMTP id im17so3564110vcb.37 for ; Wed, 26 Mar 2014 21:06:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=1AWP9jkSmlC+o+96qsRPnTBB5+p6w1UOHrzzlRSrWC0=; b=Hk6JB4aAlMVtfKXmSqsjBVR6IwVxMqnfZ8wO0HEC4vneDCmpzvq+wV2L100UIkHCFb g87B38l3F6YbveyGSCrlLMy4l+Bf7rKbFEwQpaFpHyJeHZU8YSOrVsFD7dlP/ocLBYrr 9r0VXi3/6tkPkD3vc69EkA2baLR1KXTMsCA27YGJn8GjFpeYM+m3rfjLVOk2DryHvlgH G938aKaxyPdXfUlTb5Qxc1hW0IJNw01ik4mJ91PouVBsh8TtG5srV+qvKZwBeCzTJllF UOwPQBqX16K2GUAA9/da1oNPJeKlqt1a9UVcUZB1bdJi4DWoni7dQUWkcAQ5chRv9Lac XDVQ== X-Received: by 10.221.27.8 with SMTP id ro8mr1851vcb.30.1395893200440; Wed, 26 Mar 2014 21:06:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.92.233 with HTTP; Wed, 26 Mar 2014 21:06:20 -0700 (PDT) In-Reply-To: <5332D44B.1010003@isi.edu> References: <5331F25C.20803@isi.edu> <5EE5585A-19E4-4940-B21A-4BA208F08B78@isi.edu> <5332D44B.1010003@isi.edu> From: Rakshith Venkatesh Date: Thu, 27 Mar 2014 09:36:20 +0530 Message-ID: To: Joe Touch Content-Type: multipart/alternative; boundary=001a11336baa5b95c404f58eb69e Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/7wibQyz_RkGMWolE0cvvRLtBiz0 Cc: architecture-discuss@ietf.org Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 04:06:45 -0000 --001a11336baa5b95c404f58eb69e Content-Type: text/plain; charset=ISO-8859-1 >the way you initiated your system may be causing the out-of-order responses. The main intention was to have a complete file lock to the reader and have these X workers do the job without having to worry about the order in which these workers return. >So let's assume you could transmit these blocks in the order they return from reads. Have you considered the impact on the receive side? Yes. I agree to your point that at the client side one must have enough resources to hold the out of order buffers without affecting other applications. So i wanted to know if its a good trade off between utilizing more resources at the client/Host side than on the server side while trying to achieve parallelism. What is your take on this and also how can i go ahead with this from here. Thanks, Rakshith On Wed, Mar 26, 2014 at 6:51 PM, Joe Touch wrote: > > > On 3/25/2014 11:20 PM, Rakshith Venkatesh wrote: > >> I think i should be explaining what was done and what we saw as >> improvements. What i meant was lets say the block size of a file system >> is 64kB. Now, if a file is lets say 512GB, in the normal world, i would >> fetch 64kB of data sequentially until i read all of data. Now every 64KB >> data sent across the wire could get chunked by TCP and the chunks would >> get re-ordered at the client side to present the 64KB data to the >> application in-tact. Please note that i am not referring to this >> re-ordering as part of my idea. >> >> Further, what was done was divide the whole file into 64KB zones and >> have X workers waiting to fetch each zone. So 1 to X workers just went >> ahead reading the first X zones of data. Whoever came back first among >> these X workers was assigned the task of reading the X+1 zone and so on >> and so forth. >> > > That sounds like you tried to start all X workers at the same time, but > such events are rarely truly simultaneous. > > I.e., the way you initiated your system may be causing the out-of-order > responses. Or there could be a resource interaction in which the order you > see is a side-effect of different processes or threads interacting with > shared locks. > > > This experiment saw a minimum performance improvement of 20-30% in terms >> of time taken to read the file. The only thing that stopped us from >> sending the reads across the wire was that the client expected the data >> bytes to be in-order. i.e: if the any worker came back first before the >> 1st worker, i could not send the data that was fetched by the that >> worker because of the constraint that i have mentioned previously. >> > > So let's assume you could transmit these blocks in the order they return > from reads. Have you considered the impact on the receive side? I.e., you'd > need to do the same study on the impact of out of order writes on > performance. > > > This also led to a situation where i had to hold onto the buffer >> resources even though i completed reading sub-portions of the file >> because fetching the whole file was yet to be finished. >> > > Sure, but unless your send and receive side are exactly matched in terms > of the order impact on performance, one side will end up holding buffers > too. > > I.e., it sounds like you have some OS research to complete before you know > you'll be able to take advantage of block-oriented framing in the transport > protocol. > > Joe > > >> >> On Wed, Mar 26, 2014 at 10:46 AM, Joe Touch > > wrote: >> >> >> On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh > > wrote: >> >> > I am not sure if an NFS client can accept blocks out of order >> from a server. I need to check on that. >> >> NFS should be matching read responses to requests; order should not >> matter, esp. because when using UDP the requests and responses can >> be reordered by the network. >> >> > If it does then NFS over HTTP sounds good. The way i see it is >> any File service protocol such as NFS, SMB (Large MTU's), HTTP, FTP >> should have a way which would help the server achieve true >> parallelism in reading a file and not worry about sending >> data/blocks in-order. >> >> FTP has block mode that should support any order of response, but if >> they're all going over the same TCP connection it can help only with >> the source access order rather than network reordering. >> >> HTTP has chunking that should help with this (e.g., using a chunk >> extension field that indicates chunk offset). >> >> However, most file systems are engineered to manage files via >> in-order access, even when parallelized, so I'm not sure how much >> this will all help. I.e., I don't know if your assumption is valid, >> that overriding the serial access will be faster. >> >> It might be useful to show that you can get to the components of a >> file faster as you assume first. >> >> Joe >> >> >> > >> > Rakshith >> > >> > >> > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch > > wrote: >> > You sound like you're looking for NFS; maybe you should propose >> an "NSF over HTTP" variant where responses can be issued out of order? >> > >> > Joe >> > >> > >> > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: >> > Hi, >> > >> > I was going through SPDY draft or the HTTP 2.0 initial draft. I >> had an >> > idea which I think would require a change in the architecture of >> the >> > same and so I am dropping this mail. Here is the idea: >> > >> > A Http client expects the server to send data in-order. (NOTE: >> When I >> > say a server, I am referring to an appliance to which disks are >> attached >> > and the file resides on the disk.). If there is a file let's say >> 10GB >> > and if the client asks for this file from the server, the data >> has to be >> > given in-order from the 1^st byte till the last byte. Now let's >> say I >> > >> > implement an engine at the server side to do a parallel read on >> this >> > huge file to fetch data at various offsets within the file, I will >> be >> > able to fetch the data faster for sure but, I will not be able to >> send >> > across the data immediately as and when I fetch it. I am expected >> to >> > finish doing all parallel reads on the file till I read all the >> bytes, >> > then send across the wire to the client. >> > >> > Now if we can have some tag or header which can be introduced as >> part of >> > HTTP 2.0 which actually can help re-order the byte stream at the >> session >> > layer or at a layer in between the application and session layer, >> we >> > could potentially improve the performance for File reads using >> HTTP by >> > making sure that the new module looks at this new tag and jumbles >> around >> > the data based on this tag and eventually presents the data to >> HTTP to >> > make it look all seamless. >> > >> > So the server can just do parallel reads on the same file at >> various >> > offsets without worrying about ordering and send across the read >> chunks >> > and this new module sitting at the client side can intervene and >> look at >> > some form of tag/header to make sure it swaps the data >> accordingly and >> > waits till all data is received and then present it to the >> Application >> > protocol. >> > >> > NOTE: I am not referring to packet-reordering at TCP. >> > >> > By having this, servers can attain true parallelism in reading >> the file >> > and can effectively improve file transfer rates. >> > >> > >> > Thanks, >> > >> > Rakshith >> > >> > >> > >> > _______________________________________________ >> > Architecture-discuss mailing list >> > Architecture-discuss@ietf.org > ietf.org> >> > https://www.ietf.org/mailman/listinfo/architecture-discuss >> > >> > >> >> >> --001a11336baa5b95c404f58eb69e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
>the way you initiated your system may be causing the out-of-order res= ponses.

The main intention was to have a complete file lock to the reader and ha= ve these X workers do the job without having to worry about the order in wh= ich these workers return.

>= ;So let&= #39;s assume you could transmit these blocks in the order they return from = reads. Have you considered the impact on the receive side? 

Yes= . I agree to your point that at the client side one must have enough resour= ces to hold the out of order buffers without affecting other applications.<= /span>

So = i wanted to know if its a good trade off between utilizing more resources a= t the client/Host side than on the server side while trying to achieve para= llelism. What is your take on this and also how can i go ahead with this fr= om here.

Tha= nks,
Rakshith



= On Wed, Mar 26, 2014 at 6:51 PM, Joe Touch <touch@isi.edu> wrote= :


On 3/25/2014 11:20 PM, Rakshith Venkatesh wrote:
I think i should be explaining what was done and what we saw as
improvements. What i meant was lets say the block size of a file system
is 64kB. Now, if a file is lets say 512GB, in the normal world, i would
fetch 64kB of data sequentially until i read all of data. Now every 64KB data sent across the wire could get chunked by TCP and the chunks would
get re-ordered at the client side to present the 64KB data to the
application in-tact. Please note that i am not referring to this
re-ordering as part of my idea.

  Further, what was done was divide the whole file into 64KB zones and=
have X workers waiting to fetch each zone. So 1 to X workers just went
ahead reading the first X  zones of data. Whoever came back first amon= g
these X workers was assigned the task of reading the X+1 zone and so on
and so forth.

That sounds like you tried to start all X workers at the same time, but suc= h events are rarely truly simultaneous.

I.e., the way you initiated your system may be causing the out-of-order res= ponses. Or there could be a resource interaction in which the order you see= is a side-effect of different processes or threads interacting with shared= locks.


This experiment saw a minimum performance improvement of 20-30% in terms of time taken to read the file. The only thing that stopped us from
sending the reads across the wire was that the client expected the data
bytes to be in-order. i.e: if the any worker came back first before the
1st worker, i could not send the data that was fetched by the that
worker because of the constraint that i have mentioned previously.

So let's assume you could transmit these blocks in the order they retur= n from reads. Have you considered the impact on the receive side? I.e., you= 'd need to do the same study on the impact of out of order writes on pe= rformance.


This also led to a situation where i had to hold onto the buffer
resources even though i completed reading sub-portions of the file
because fetching  the whole file was yet to be finished.

Sure, but unless your send and receive side are exactly matched in terms of= the order impact on performance, one side will end up holding buffers too.=

I.e., it sounds like you have some OS research to complete before you know = you'll be able to take advantage of block-oriented framing in the trans= port protocol.

Joe



On Wed, Mar 26, 2014 at 10:46 AM, Joe Touch <touch@isi.edu
<mailto:touch@isi.edu= >> wrote:


    On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh <vrock28@gmail.com
    <mailto:vrock28@gmail.com>> wrote:

     > I am not sure if an NFS client can accept blocks o= ut of order
    from a server. I need to check on that.

    NFS should be matching read responses to requests; order shou= ld not
    matter, esp. because when using UDP the requests and response= s can
    be reordered by the network.

     > If it does then NFS over HTTP sounds good. The way= i see it is
    any File service protocol such as NFS, SMB (Large MTU's),= HTTP, FTP
    should have a way which would help the server achieve true     parallelism in reading a file and not worry about sending
    data/blocks in-order.

    FTP has block mode that should support any order of response,= but if
    they're all going over the same TCP connection it can hel= p only with
    the source access order rather than network reordering.

    HTTP has chunking that should help with this (e.g., using a c= hunk
    extension field that indicates chunk offset).

    However, most file systems are engineered to manage files via=
    in-order access, even when parallelized, so I'm not sure = how much
    this will all help. I.e., I don't know if your assumption= is valid,
    that overriding the serial access will be faster.

    It might be useful to show that you can get to the components= of a
    file faster as you assume first.

    Joe


     >
     > Rakshith
     >
     >
     > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch <touch@isi.edu
=
    <mailto:touch@isi.edu>> wrote:
     > You sound like you're looking for NFS; maybe y= ou should propose
    an "NSF over HTTP" variant where responses can be i= ssued out of order?
     >
     > Joe
     >
     >
     > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote:      > Hi,
     >
     > I was going through SPDY draft or the HTTP 2.0 ini= tial draft. I
    had an
     > idea which I think would require a change in the a= rchitecture of the
     > same and so I am dropping this mail. Here is the i= dea:
     >
     > A Http client expects the server to send data in-o= rder. (NOTE: When I
     > say a server, I am referring to an appliance to wh= ich disks are
    attached
     > and the file resides on the disk.). If there is a = file let’s say 10GB
     > and if the client asks for this file from the serv= er, the data
    has to be
     > given in-order from the 1^st byte till the last by= te. Now let’s say I
     >
     > implement an engine at the server side to do a par= allel read on this
     > huge file to fetch data at various offsets within = the file, I will be
     > able to fetch the data faster for sure but, I will= not be able to
    send
     > across the data immediately as and when I fetch it= . I am expected to
     > finish doing all parallel reads on the file till I= read all the
    bytes,
     > then send across the wire to the client.
     >
     > Now if we can have some tag or header which can be= introduced as
    part of
     > HTTP 2.0 which actually can help re-order the byte= stream at the
    session
     > layer or at a layer in between the application and= session layer, we
     > could potentially improve the performance for File= reads using
    HTTP by
     > making sure that the new module looks at this new = tag and jumbles
    around
     > the data based on this tag and eventually presents= the data to
    HTTP to
     > make it look all seamless.
     >
     > So the server can just do parallel reads on the sa= me file at various
     > offsets without worrying about ordering and send a= cross the read
    chunks
     > and this new module sitting at the client side can= intervene and
    look at
     > some form of tag/header to make sure it swaps the = data
    accordingly and
     > waits till all data is received and then present i= t to the
    Application
     > protocol.
     >
     > NOTE: I am not referring to packet-reordering at T= CP.
     >
     > By having this, servers can attain true parallelis= m in reading
    the file
     > and can effectively improve file transfer rates.      >
     >
     > Thanks,
     >
     > Rakshith
     >
     >
     >
     > ___________________________________________= ____
     > Architecture-discuss mailing list
     > Architecture-discuss@ietf.org <mailto:Architecture-discuss@= ietf.org>
     > https://www.ietf.org/mailman/= listinfo/architecture-discuss
     >
     >



--001a11336baa5b95c404f58eb69e-- From nobody Wed Mar 26 21:17:17 2014 Return-Path: X-Original-To: architecture-discuss@ietfa.amsl.com Delivered-To: architecture-discuss@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 84B6D1A0099 for ; Wed, 26 Mar 2014 21:17:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.208 X-Spam-Level: X-Spam-Status: No, score=-4.208 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id koV6Cs3q08iO for ; Wed, 26 Mar 2014 21:17:12 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by ietfa.amsl.com (Postfix) with ESMTP id 644701A028C for ; Wed, 26 Mar 2014 21:17:12 -0700 (PDT) Received: from [192.168.1.97] (pool-71-105-87-112.lsanca.dsl-w.verizon.net [71.105.87.112]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id s2R4GWuJ001850 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Wed, 26 Mar 2014 21:16:43 -0700 (PDT) References: <5331F25C.20803@isi.edu> <5EE5585A-19E4-4940-B21A-4BA208F08B78@isi.edu> <5332D44B.1010003@isi.edu> In-Reply-To: Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: 7bit Content-Type: multipart/alternative; boundary=Apple-Mail-AD2F0FDF-9AA4-45BE-91E6-2C1F98DEBA26 Message-Id: <67B75A06-928A-4A8A-8481-E24CBE0E4545@isi.edu> X-Mailer: iPhone Mail (11D167) From: Joe Touch Date: Wed, 26 Mar 2014 21:16:32 -0700 To: Rakshith Venkatesh X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: http://mailarchive.ietf.org/arch/msg/architecture-discuss/kfDLhPlT8wFPXj9oqwF85Upjir8 Cc: "architecture-discuss@ietf.org" Subject: Re: [arch-d] HTTP 2.0 + Performance improvement Idea X-BeenThere: architecture-discuss@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: open discussion forum for long/wide-range architectural issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 04:17:15 -0000 --Apple-Mail-AD2F0FDF-9AA4-45BE-91E6-2C1F98DEBA26 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > On Mar 26, 2014, at 9:06 PM, Rakshith Venkatesh wrote:= >=20 > >the way you initiated your system may be causing the out-of-order respons= es. >=20 > The main intention was to have a complete file lock to the reader and have= these X workers do the job without having to worry about the order in which= these workers return. >=20 > >So let's assume you could transmit these blocks in the order they return f= rom reads. Have you considered the impact on the receive side?=20 >=20 > Yes. I agree to your point that at the client side one must have enough re= sources to hold the out of order buffers without affecting other application= s. >=20 > So i wanted to know if its a good trade off between utilizing more resourc= es at the client/Host side than on the server side while trying to achieve p= arallelism. What is your take on this and also how can i go ahead with this f= rom here. Frankly most of this is an OS problem that is out of scope for this list. Un= til you know that both ends can take advantage of out of order file operatio= ns it is premature to consider out of order file transfer protocols, but it l= ooks like what you light want already exists.=20 Joe >=20 > Thanks, > Rakshith >=20 >=20 >=20 >> On Wed, Mar 26, 2014 at 6:51 PM, Joe Touch wrote: >>=20 >>=20 >>> On 3/25/2014 11:20 PM, Rakshith Venkatesh wrote: >>> I think i should be explaining what was done and what we saw as >>> improvements. What i meant was lets say the block size of a file system >>> is 64kB. Now, if a file is lets say 512GB, in the normal world, i would >>> fetch 64kB of data sequentially until i read all of data. Now every 64KB= >>> data sent across the wire could get chunked by TCP and the chunks would >>> get re-ordered at the client side to present the 64KB data to the >>> application in-tact. Please note that i am not referring to this >>> re-ordering as part of my idea. >>>=20 >>> Further, what was done was divide the whole file into 64KB zones and >>> have X workers waiting to fetch each zone. So 1 to X workers just went >>> ahead reading the first X zones of data. Whoever came back first among >>> these X workers was assigned the task of reading the X+1 zone and so on >>> and so forth. >>=20 >> That sounds like you tried to start all X workers at the same time, but s= uch events are rarely truly simultaneous. >>=20 >> I.e., the way you initiated your system may be causing the out-of-order r= esponses. Or there could be a resource interaction in which the order you se= e is a side-effect of different processes or threads interacting with shared= locks. >>=20 >>=20 >>> This experiment saw a minimum performance improvement of 20-30% in terms= >>> of time taken to read the file. The only thing that stopped us from >>> sending the reads across the wire was that the client expected the data >>> bytes to be in-order. i.e: if the any worker came back first before the >>> 1st worker, i could not send the data that was fetched by the that >>> worker because of the constraint that i have mentioned previously. >>=20 >> So let's assume you could transmit these blocks in the order they return f= rom reads. Have you considered the impact on the receive side? I.e., you'd n= eed to do the same study on the impact of out of order writes on performance= . >>=20 >>=20 >>> This also led to a situation where i had to hold onto the buffer >>> resources even though i completed reading sub-portions of the file >>> because fetching the whole file was yet to be finished. >>=20 >> Sure, but unless your send and receive side are exactly matched in terms o= f the order impact on performance, one side will end up holding buffers too.= >>=20 >> I.e., it sounds like you have some OS research to complete before you kno= w you'll be able to take advantage of block-oriented framing in the transpor= t protocol. >>=20 >> Joe >>=20 >>>=20 >>>=20 >>> On Wed, Mar 26, 2014 at 10:46 AM, Joe Touch >> > wrote: >>>=20 >>>=20 >>> On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh >> > wrote: >>>=20 >>> > I am not sure if an NFS client can accept blocks out of order >>> from a server. I need to check on that. >>>=20 >>> NFS should be matching read responses to requests; order should not >>> matter, esp. because when using UDP the requests and responses can >>> be reordered by the network. >>>=20 >>> > If it does then NFS over HTTP sounds good. The way i see it is >>> any File service protocol such as NFS, SMB (Large MTU's), HTTP, FTP >>> should have a way which would help the server achieve true >>> parallelism in reading a file and not worry about sending >>> data/blocks in-order. >>>=20 >>> FTP has block mode that should support any order of response, but if= >>> they're all going over the same TCP connection it can help only with= >>> the source access order rather than network reordering. >>>=20 >>> HTTP has chunking that should help with this (e.g., using a chunk >>> extension field that indicates chunk offset). >>>=20 >>> However, most file systems are engineered to manage files via >>> in-order access, even when parallelized, so I'm not sure how much >>> this will all help. I.e., I don't know if your assumption is valid, >>> that overriding the serial access will be faster. >>>=20 >>> It might be useful to show that you can get to the components of a >>> file faster as you assume first. >>>=20 >>> Joe >>>=20 >>>=20 >>> > >>> > Rakshith >>> > >>> > >>> > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch >> > wrote: >>> > You sound like you're looking for NFS; maybe you should propose >>> an "NSF over HTTP" variant where responses can be issued out of orde= r? >>> > >>> > Joe >>> > >>> > >>> > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote: >>> > Hi, >>> > >>> > I was going through SPDY draft or the HTTP 2.0 initial draft. I >>> had an >>> > idea which I think would require a change in the architecture of t= he >>> > same and so I am dropping this mail. Here is the idea: >>> > >>> > A Http client expects the server to send data in-order. (NOTE: Wh= en I >>> > say a server, I am referring to an appliance to which disks are >>> attached >>> > and the file resides on the disk.). If there is a file let=E2=80=99= s say 10GB >>> > and if the client asks for this file from the server, the data >>> has to be >>> > given in-order from the 1^st byte till the last byte. Now let=E2=80= =99s say I >>> > >>> > implement an engine at the server side to do a parallel read on t= his >>> > huge file to fetch data at various offsets within the file, I wil= l be >>> > able to fetch the data faster for sure but, I will not be able to= >>> send >>> > across the data immediately as and when I fetch it. I am expected= to >>> > finish doing all parallel reads on the file till I read all the >>> bytes, >>> > then send across the wire to the client. >>> > >>> > Now if we can have some tag or header which can be introduced as >>> part of >>> > HTTP 2.0 which actually can help re-order the byte stream at the >>> session >>> > layer or at a layer in between the application and session layer,= we >>> > could potentially improve the performance for File reads using >>> HTTP by >>> > making sure that the new module looks at this new tag and jumbles= >>> around >>> > the data based on this tag and eventually presents the data to >>> HTTP to >>> > make it look all seamless. >>> > >>> > So the server can just do parallel reads on the same file at vari= ous >>> > offsets without worrying about ordering and send across the read >>> chunks >>> > and this new module sitting at the client side can intervene and >>> look at >>> > some form of tag/header to make sure it swaps the data >>> accordingly and >>> > waits till all data is received and then present it to the >>> Application >>> > protocol. >>> > >>> > NOTE: I am not referring to packet-reordering at TCP. >>> > >>> > By having this, servers can attain true parallelism in reading >>> the file >>> > and can effectively improve file transfer rates. >>> > >>> > >>> > Thanks, >>> > >>> > Rakshith >>> > >>> > >>> > >>> > _______________________________________________ >>> > Architecture-discuss mailing list >>> > Architecture-discuss@ietf.org >>> > https://www.ietf.org/mailman/listinfo/architecture-discuss >>> > >>> > >=20 --Apple-Mail-AD2F0FDF-9AA4-45BE-91E6-2C1F98DEBA26 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable


On Mar 26, 2014, at= 9:06 PM, Rakshith Venkatesh <vrock2= 8@gmail.com> wrote:

>t= he way you initiated your system may be causing the out-of-order responses.<= /span>

The main intention was to have a complete file lock to the reader and have= these X workers do the job without having to worry about the order in which= these workers return.

=
><= /span>So let's a= ssume you could transmit these blocks in the order they return from reads. H= ave you considered the impact on the receive side? 

=
Yes. I= agree to your point that at the client side one must have enough resources t= o hold the out of order buffers without affecting other applications.=

=
So i w= anted to know if its a good trade off between utilizing more resources at th= e client/Host side than on the server side while trying to achieve paralleli= sm. What is your take on this and also how can i go ahead with this from her= e.

Frankly most of t= his is an OS problem that is out of scope for this list. Until you know that= both ends can take advantage of out of order file operations it is prematur= e to consider out of order file transfer protocols, but it looks like what y= ou light want already exists. 

Joe
<= br>


=
Thank= s,
Rakshith

=


On= Wed, Mar 26, 2014 at 6:51 PM, Joe Touch <touch@isi.edu> wrote:


On 3/25/2014 11:20 PM, Rakshith Venkatesh wrote:
I think i should be explaining what was done and what we saw as
improvements. What i meant was lets say the block size of a file system
is 64kB. Now, if a file is lets say 512GB, in the normal world, i would
fetch 64kB of data sequentially until i read all of data. Now every 64KB
= data sent across the wire could get chunked by TCP and the chunks would
get re-ordered at the client side to present the 64KB data to the
application in-tact. Please note that i am not referring to this
re-ordering as part of my idea.

  Further, what was done was divide the whole file into 64KB zones and<= br> have X workers waiting to fetch each zone. So 1 to X workers just went
ahead reading the first X  zones of data. Whoever came back first among=
these X workers was assigned the task of reading the X+1 zone and so on
and so forth.

That sounds like you tried to start all X workers at the same time, but such= events are rarely truly simultaneous.

I.e., the way you initiated your system may be causing the out-of-order resp= onses. Or there could be a resource interaction in which the order you see i= s a side-effect of different processes or threads interacting with shared lo= cks.


This experiment saw a minimum performance improvement of 20-30% in terms
= of time taken to read the file. The only thing that stopped us from
sending the reads across the wire was that the client expected the data
bytes to be in-order. i.e: if the any worker came back first before the
1st worker, i could not send the data that was fetched by the that
worker because of the constraint that i have mentioned previously.

So let's assume you could transmit these blocks in the order they return fro= m reads. Have you considered the impact on the receive side? I.e., you'd nee= d to do the same study on the impact of out of order writes on performance.<= div class=3D"">

This also led to a situation where i had to hold onto the buffer
resources even though i completed reading sub-portions of the file
because fetching  the whole file was yet to be finished.

Sure, but unless your send and receive side are exactly matched in terms of t= he order impact on performance, one side will end up holding buffers too.
I.e., it sounds like you have some OS research to complete before you know y= ou'll be able to take advantage of block-oriented framing in the transport p= rotocol.

Joe



On Wed, Mar 26, 2014 at 10:46 AM, Joe Touch <touch@isi.edu
<mailto:touch@isi.edu<= /a>>> wrote:


    On Mar 25, 2014, at 6:01 PM, Rakshith Venkatesh <vrock28@gmail.com
=
    <mailto:vrock28@gmail.com>> wrote:

     > I am not sure if an NFS client can accept blocks ou= t of order
    from a server. I need to check on that.

    NFS should be matching read responses to requests; order shoul= d not
    matter, esp. because when using UDP the requests and responses= can
    be reordered by the network.

     > If it does then NFS over HTTP sounds good. The way i= see it is
    any File service protocol such as NFS, SMB (Large MTU's), HTTP= , FTP
    should have a way which would help the server achieve true
=     parallelism in reading a file and not worry about sending
    data/blocks in-order.

    FTP has block mode that should support any order of response, b= ut if
    they're all going over the same TCP connection it can help onl= y with
    the source access order rather than network reordering.

    HTTP has chunking that should help with this (e.g., using a ch= unk
    extension field that indicates chunk offset).

    However, most file systems are engineered to manage files via<= br>     in-order access, even when parallelized, so I'm not sure how m= uch
    this will all help. I.e., I don't know if your assumption is v= alid,
    that overriding the serial access will be faster.

    It might be useful to show that you can get to the components o= f a
    file faster as you assume first.

    Joe


     >
     > Rakshith
     >
     >
     > On Wed, Mar 26, 2014 at 2:47 AM, Joe Touch <touch@isi.edu
    <mailto:= touch@isi.edu>> wrote:
     > You sound like you're looking for NFS; maybe you sh= ould propose
    an "NSF over HTTP" variant where responses can be issued out o= f order?
     >
     > Joe
     >
     >
     > On 3/25/2014 4:56 AM, Rakshith Venkatesh wrote:
=      > Hi,
     >
     > I was going through SPDY draft or the HTTP 2.0 init= ial draft. I
    had an
     > idea which I think would require a change in the ar= chitecture of the
     > same and so I am dropping this mail. Here is the id= ea:
     >
     > A Http client expects the server to send data in-or= der. (NOTE: When I
     > say a server, I am referring to an appliance to whi= ch disks are
    attached
     > and the file resides on the disk.). If there is a f= ile let=E2=80=99s say 10GB
     > and if the client asks for this file from the serve= r, the data
    has to be
     > given in-order from the 1^st byte till the last byt= e. Now let=E2=80=99s say I
     >
     > implement an engine at the server side to do a para= llel read on this
     > huge file to fetch data at various offsets within t= he file, I will be
     > able to fetch the data faster for sure but, I will n= ot be able to
    send
     > across the data immediately as and when I fetch it.= I am expected to
     > finish doing all parallel reads on the file till I r= ead all the
    bytes,
     > then send across the wire to the client.
     >
     > Now if we can have some tag or header which can be i= ntroduced as
    part of
     > HTTP 2.0 which actually can help re-order the byte s= tream at the
    session
     > layer or at a layer in between the application and s= ession layer, we
     > could potentially improve the performance for File r= eads using
    HTTP by
     > making sure that the new module looks at this new t= ag and jumbles
    around
     > the data based on this tag and eventually presents t= he data to
    HTTP to
     > make it look all seamless.
     >
     > So the server can just do parallel reads on the sam= e file at various
     > offsets without worrying about ordering and send ac= ross the read
    chunks
     > and this new module sitting at the client side can i= ntervene and
    look at
     > some form of tag/header to make sure it swaps the d= ata
    accordingly and
     > waits till all data is received and then present it= to the
    Application
     > protocol.
     >
     > NOTE: I am not referring to packet-reordering at TC= P.
     >
     > By having this, servers can attain true parallelism= in reading
    the file
     > and can effectively improve file transfer rates.      >
     >
     > Thanks,
     >
     > Rakshith
     >
     >
     >
     > ____________________________________________= ___
     > Architecture-discuss mailing list
     > Architecture-discuss@ietf.org <mailto:Architecture-discuss@= ietf.org>
     > https://www.ietf.org/mailman/li= stinfo/architecture-discuss
     >
     >



= --Apple-Mail-AD2F0FDF-9AA4-45BE-91E6-2C1F98DEBA26--