From moore@cs.utk.edu Tue Apr 9 14:18:57 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15153 for ; Tue, 9 Apr 2002 14:18:56 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id OAA05249; Tue, 9 Apr 2002 14:18:46 -0400 (EDT) Date: Tue, 9 Apr 2002 14:18:46 -0400 (EDT) Message-Id: <200204091818.OAA05249@cs.utk.edu> From: rescap-REQUEST@cs.utk.edu To: rescap-archive@ietf.org Subject: subscription request rescap-archive@ietf.org has been added to the rescap@cs.utk.edu mailing list. ---- From owner-rescap@cs.utk.edu Wed Apr 10 10:17:22 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA20469 for ; Wed, 10 Apr 2002 10:17:22 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id KAA05862; Wed, 10 Apr 2002 10:17:01 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Wed, 10 Apr 2002 10:16:54 -0400 Received: from astro.cs.utk.edu (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id KAA05668; Wed, 10 Apr 2002 10:16:54 -0400 (EDT) Received: from astro.cs.utk.edu (160.36.58.43 -> astro.cs.utk.edu) by cs.utk.edu (smtpshim v1.0); Wed, 10 Apr 2002 10:16:54 -0400 Received: (from moore@localhost) by astro.cs.utk.edu (cf 8.9.3) id g3AEGrJ09409 for dist-rescap@cs.utk.edu; Wed, 10 Apr 2002 10:16:53 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Tue, 9 Apr 2002 23:51:36 -0400 Received: by cs.utk.edu (cf v2.9s-UTK) id XAA15628; Tue, 9 Apr 2002 23:51:35 -0400 (EDT) Received: from bailey.dscga.com (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id XAA15615; Tue, 9 Apr 2002 23:51:34 -0400 (EDT) Received: from bailey.dscga.com (198.78.11.130 -> bailey.neonym.net) by cs.utk.edu (smtpshim v1.0); Tue, 9 Apr 2002 23:51:34 -0400 Received: from bailey.dscga.com (localhost [127.0.0.1]) by bailey.dscga.com (8.12.1/8.12.1) with ESMTP id g3A3oc9X008653 for ; Tue, 9 Apr 2002 23:50:38 -0400 (EDT) Received: (from michael@localhost) by bailey.dscga.com (8.12.1/8.12.1/Submit) id g3A3ocVB008652 for rescap@cs.utk.edu; Tue, 9 Apr 2002 23:50:38 -0400 (EDT) Date: Tue, 9 Apr 2002 23:50:37 -0400 From: Michael Mealling To: rescap@cs.utk.edu Subject: questions on Keith's draft from an implementor.... Message-ID: <20020409235037.E6941@bailey.dscga.com> Reply-To: Michael Mealling Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.22.1i List-Owner: List-Post: List-Unsubscribe: Speaking as an implementor and not as a co-chair I've come up with some issues that I think might be worth discussing. I'm writting this as I'm writing my RC query/response encoder so intimate knowledge of the packet formats is needed to parse some of this stuff: 1) Support in the Query result structure for the QUERY_RECURSE flag being set is handled by an array of blob's that contain answers with the first being the one for the resource expressed in the query and subsequent array values being 'additional information'. This seems fine in the case where you have and want that additional information. But in the 'normal' case you end up 'wasting' a whole new set of blob headers for just one blob when in actuality it will always be there. Would it be more efficient to save a layer of encoded blobs and have the query_response structure look like this: BEGIN query_response string request_id int query_status int version_hi int version_low struct<> assertions struct<> signatures struct<> additional END Thus saving (if my math is right) 16 very redundant bytes for _every_ query response. 2) One thought keeps bugging me: will we have the case where a request for an attribute for a given resource needs arguments? Each time I've had the need for one I simply tagged it onto the end of the attribute name URI. But that just seems insufficiently specified for some gut reason. 3) The situation with not knowing what size UDP packet to use and just 'hoping' the other end gets all of the fragments is bothersome. I can't find any fragmentation delivery statistics but I've been told that anything above 1500 bytes creates 'problems' with failed delivery of all the subsequent fragments. The suggestion is to start out using TCP but that just seems overkill. Could we add something to the UDP packet that allows for a RESCAP response to be packaged for client reconstruction? Just thoughts I'd like some feedback on.... -MM -- -------------------------------------------------------------------------------- Michael Mealling | Vote Libertarian! | urn:pin:1 michael@neonym.net | | http://www.neonym.net From owner-rescap@cs.utk.edu Wed Apr 10 13:17:30 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA25744 for ; Wed, 10 Apr 2002 13:17:25 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id NAA14151; Wed, 10 Apr 2002 13:17:20 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Wed, 10 Apr 2002 13:17:17 -0400 Received: from astro.cs.utk.edu (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id NAA14120; Wed, 10 Apr 2002 13:17:17 -0400 (EDT) Received: from astro.cs.utk.edu (160.36.58.43 -> astro.cs.utk.edu) by cs.utk.edu (smtpshim v1.0); Wed, 10 Apr 2002 13:17:17 -0400 Received: (from moore@localhost) by astro.cs.utk.edu (cf 8.9.3) id g3AHHGS01484 for dist-rescap@cs.utk.edu; Wed, 10 Apr 2002 13:17:16 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Wed, 10 Apr 2002 11:32:21 -0400 Received: by cs.utk.edu (cf v2.9s-UTK) id LAA25086; Wed, 10 Apr 2002 11:32:20 -0400 (EDT) Received: from astro.cs.utk.edu (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id LAA25070; Wed, 10 Apr 2002 11:32:18 -0400 (EDT) Received: from astro.cs.utk.edu (160.36.58.43 -> astro.cs.utk.edu) by cs.utk.edu (smtpshim v1.0); Wed, 10 Apr 2002 11:32:18 -0400 Received: from astro.cs.utk.edu (localhost [127.0.0.1]) by astro.cs.utk.edu (cf 8.9.3) with ESMTP id g3AFWHZ21178; Wed, 10 Apr 2002 11:32:17 -0400 (EDT) Message-Id: <200204101532.g3AFWHZ21178@astro.cs.utk.edu> X-URI: http://www.cs.utk.edu/~moore/ From: Keith Moore To: Michael Mealling cc: rescap@cs.utk.edu Subject: Re: questions on Keith's draft from an implementor.... In-reply-to: (Your message of "Tue, 09 Apr 2002 23:50:37 EDT.") <20020409235037.E6941@bailey.dscga.com> Date: Wed, 10 Apr 2002 11:32:17 -0400 Sender: moore@cs.utk.edu List-Owner: List-Post: List-Unsubscribe: > 1) Support in the Query result structure for the QUERY_RECURSE flag being set > is handled by an array of blob's that contain answers with the first > being the one for the resource expressed in the query and subsequent > array values being 'additional information'. This seems fine in the case > where you have and want that additional information. But in the 'normal' > case you end up 'wasting' a whole new set of blob headers for just one > blob when in actuality it will always be there. Would it be more efficient > to save a layer of encoded blobs and have the query_response structure > look like this: > > BEGIN query_response > string request_id > int query_status > int version_hi > int version_low > struct<> assertions > struct<> signatures > struct<> additional > END > > Thus saving (if my math is right) 16 very redundant bytes for _every_ query > response. you're right about the bandwidth savings, but the server and client code are simpler if all responses have the same structure. I decided to compromise in favor of simplicity. in general, there are lost of ways to tweak the blob structure to be more efficient - e.g. for blobs < 64k, 2 bytes would suffice for each offset. again, it's a compromise - my choice was to keep blobs flexible enough to handle larger objects without making them more complex than necessary. > 2) One thought keeps bugging me: will we have the case where a request > for an attribute for a given resource needs arguments? someday you might want to match patterns of attribute names other than by prefix. a new RC function could be defined to do that; with old servers the client would just have to ask for all attributes and filter them itself. > 3) The situation with not knowing what size UDP packet to use and just > 'hoping' the other end gets all of the fragments is bothersome. I can't > find any fragmentation delivery statistics but I've been told that > anything above 1500 bytes creates 'problems' with failed delivery of all > the subsequent fragments. The suggestion is to start out using TCP but that > just seems overkill. Could we add something to the UDP packet that allows > for a RESCAP response to be packaged for client reconstruction? we definitely need to supply good advice about when to use TCP vs when to use UDP - my gut sense is that if the result is more than 3 IP fragments then you're better off using TCP than resending the request, but I'd like to see that verified by a statistical model. OTOH, I'm not sure about the value of adding fragmentation/reassembly to RC. I actually tried this in an earlier version, and it was a pain. It requires the server to maintain state about each pending request (since the data can change between the first and subsequent requests). and basically you need about the same services you would get from TCP anyway. Keith From owner-rescap@cs.utk.edu Wed Apr 10 17:23:46 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA03015 for ; Wed, 10 Apr 2002 17:23:45 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id RAA01327; Wed, 10 Apr 2002 17:23:41 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Wed, 10 Apr 2002 17:23:38 -0400 Received: by cs.utk.edu (cf v2.9s-UTK) id RAA01306; Wed, 10 Apr 2002 17:23:38 -0400 (EDT) Received: from bailey.dscga.com (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id RAA01288; Wed, 10 Apr 2002 17:23:34 -0400 (EDT) Received: from bailey.dscga.com (198.78.11.130 -> bailey.neonym.net) by cs.utk.edu (smtpshim v1.0); Wed, 10 Apr 2002 17:23:34 -0400 Received: from bailey.dscga.com (localhost [127.0.0.1]) by bailey.dscga.com (8.12.1/8.12.1) with ESMTP id g3ALMa9X011174; Wed, 10 Apr 2002 17:22:37 -0400 (EDT) Received: (from michael@localhost) by bailey.dscga.com (8.12.1/8.12.1/Submit) id g3ALMa4W011173; Wed, 10 Apr 2002 17:22:36 -0400 (EDT) Date: Wed, 10 Apr 2002 17:22:36 -0400 From: Michael Mealling To: Keith Moore Cc: Michael Mealling , rescap@cs.utk.edu Subject: Re: questions on Keith's draft from an implementor.... Message-ID: <20020410172236.E10914@bailey.dscga.com> Reply-To: Michael Mealling References: <20020409235037.E6941@bailey.dscga.com> <200204101532.g3AFWHZ21178@astro.cs.utk.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200204101532.g3AFWHZ21178@astro.cs.utk.edu> User-Agent: Mutt/1.3.22.1i List-Owner: List-Post: List-Unsubscribe: On Wed, Apr 10, 2002 at 11:32:17AM -0400, Keith Moore wrote: >> 1) Support in the Query result structure for the QUERY_RECURSE flag being set >> is handled by an array of blob's that contain answers with the first >> being the one for the resource expressed in the query and subsequent >> array values being 'additional information'. This seems fine in the case >> where you have and want that additional information. But in the 'normal' >> case you end up 'wasting' a whole new set of blob headers for just one >> blob when in actuality it will always be there. Would it be more efficient >> to save a layer of encoded blobs and have the query_response structure >> look like this: >> > > BEGIN query_response > > string request_id > > int query_status > > int version_hi > > int version_low > > struct<> assertions > > struct<> signatures > > struct<> additional > > END > > > > Thus saving (if my math is right) 16 very redundant bytes for _every_ query > > response. > > you're right about the bandwidth savings, but the server and client > code are simpler if all responses have the same structure. > I decided to compromise in favor of simplicity. But they still have the same structure. The issue is that, since _every_ query response will have at least one assertion for the URI in the query, why not break it out of the array of blobs into the query result blob itself. That way you only have to parse/use the blobs in the additional information if you need to.... > in general, there are lost of ways to tweak the blob structure to > be more efficient - e.g. for blobs < 64k, 2 bytes would suffice > for each offset. again, it's a compromise - my choice was to > keep blobs flexible enough to handle larger objects without making > them more complex than necessary. But this isn't a suggestion to tweak the blob structure but to tweak RC's query structure.... > > 2) One thought keeps bugging me: will we have the case where a request > > for an attribute for a given resource needs arguments? > > someday you might want to match patterns of attribute names other > than by prefix. a new RC function could be defined to do that; > with old servers the client would just have to ask for all attributes > and filter them itself. But I'm wondering if its worth doing now.... Has anyone else run into this situation? > > 3) The situation with not knowing what size UDP packet to use and just > > 'hoping' the other end gets all of the fragments is bothersome. I can't > > find any fragmentation delivery statistics but I've been told that > > anything above 1500 bytes creates 'problems' with failed delivery of all > > the subsequent fragments. The suggestion is to start out using TCP but that > > just seems overkill. Could we add something to the UDP packet that allows > > for a RESCAP response to be packaged for client reconstruction? > > we definitely need to supply good advice about when to use TCP vs > when to use UDP - my gut sense is that if the result is more than > 3 IP fragments then you're better off using TCP than resending > the request, but I'd like to see that verified by a statistical model. But why TCP just to get one more packet through? > OTOH, I'm not sure about the value of adding fragmentation/reassembly to RC. > I actually tried this in an earlier version, and it was a pain. It requires > the server to maintain state about each pending request (since the data > can change between the first and subsequent requests). and basically you > need about the same services you would get from TCP anyway. I'm not talking about full packet state and acks. I'm just talking about the server sending a packet sequence number so that the client can put blocks back together (and is also aware of what is being sent). If it fails then the client either resends the request or reconnects via TCP. The server wouldn't need to maintain any state. -MM -- -------------------------------------------------------------------------------- Michael Mealling | Vote Libertarian! | urn:pin:1 michael@neonym.net | | http://www.neonym.net From owner-rescap@cs.utk.edu Thu Apr 11 17:42:14 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA16103 for ; Thu, 11 Apr 2002 17:42:13 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id RAA07650; Thu, 11 Apr 2002 17:41:53 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Thu, 11 Apr 2002 17:41:41 -0400 Received: by cs.utk.edu (cf v2.9s-UTK) id RAA07609; Thu, 11 Apr 2002 17:41:40 -0400 (EDT) Received: from rwcrmhc51.attbi.com (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id RAA07592; Thu, 11 Apr 2002 17:41:38 -0400 (EDT) Received: from rwcrmhc51.attbi.com (204.127.198.38 -> rwcrmhc51.attbi.com) by cs.utk.edu (smtpshim v1.0); Thu, 11 Apr 2002 17:41:39 -0400 Received: from hephaestus ([24.98.218.176]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20020411214137.CWHC1143.rwcrmhc51.attbi.com@hephaestus>; Thu, 11 Apr 2002 21:41:37 +0000 Message-ID: <002601c1e1a2$e641ad60$0201a8c0@hephaestus> From: "Ben Schepens" To: "Keith Moore" Cc: , "Ben Schepens" Subject: ResCap doc Errors/Omission? Date: Thu, 11 Apr 2002 17:50:29 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 List-Owner: List-Post: List-Unsubscribe: Content-Transfer-Encoding: 7bit Keith, I am now working on actually building ResCap requests using the blob structures. I am reading the rescap document: http://www.ietf.org/internet-drafts/draft-ietf-rescap-rc-01.txt I found the following issues: 1) On page 13, section 4.2.1.3, the following statement exists: query_status A code describing the result of the attempt to query this resource_name. See section 3.5. I could find a section 3.5. Has it just not been added? Or am I missing something? 2) Is there a reason why the definition of Query_Type #4 is out of numerical order? Ben Schepens schepens@mindspring.com.nospam From owner-rescap@cs.utk.edu Thu Apr 11 22:03:59 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA00184 for ; Thu, 11 Apr 2002 22:03:59 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id WAA12600; Thu, 11 Apr 2002 22:03:56 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Thu, 11 Apr 2002 22:03:54 -0400 Received: by cs.utk.edu (cf v2.9s-UTK) id WAA12579; Thu, 11 Apr 2002 22:03:54 -0400 (EDT) Received: from astro.cs.utk.edu (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id WAA12555; Thu, 11 Apr 2002 22:03:52 -0400 (EDT) Received: from astro.cs.utk.edu (160.36.58.43 -> astro.cs.utk.edu) by cs.utk.edu (smtpshim v1.0); Thu, 11 Apr 2002 22:03:52 -0400 Received: from astro.cs.utk.edu (localhost [127.0.0.1]) by astro.cs.utk.edu (cf 8.9.3) with ESMTP id g3C23ol01010; Thu, 11 Apr 2002 22:03:50 -0400 (EDT) Message-Id: <200204120203.g3C23ol01010@astro.cs.utk.edu> X-URI: http://www.cs.utk.edu/~moore/ From: Keith Moore To: "Ben Schepens" cc: "Keith Moore" , rescap@cs.utk.edu Subject: Re: ResCap doc Errors/Omission? In-reply-to: (Your message of "Thu, 11 Apr 2002 17:50:29 EDT.") <002601c1e1a2$e641ad60$0201a8c0@hephaestus> Date: Thu, 11 Apr 2002 22:03:50 -0400 Sender: moore@cs.utk.edu List-Owner: List-Post: List-Unsubscribe: > 1) On page 13, section 4.2.1.3, the following statement exists: > > query_status > A code describing the result of the attempt to query this > resource_name. See section 3.5. > > I could find a section 3.5. Has it just not been added? Or am I missing > something? I think this is just supposed to be whatever section lists the error codes. It's probably been renumbered since I wrote that text, and I didn't catch it. > 2) Is there a reason why the definition of Query_Type #4 is out of numerical > order? probably because I added a new query type after the first version of the document, and I put it in the text where I thought it fit best, not in numerical order by query_type. Keith From owner-rescap@cs.utk.edu Mon Apr 22 16:48:56 2002 Received: from cs.utk.edu (cs.utk.edu [160.36.56.56]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA12503 for ; Mon, 22 Apr 2002 16:48:55 -0400 (EDT) Received: by cs.utk.edu (cf v2.9s-UTK) id QAA25362; Mon, 22 Apr 2002 16:48:33 -0400 (EDT) Received: by cs.utk.edu (bulk_mailer v1.16); Mon, 22 Apr 2002 16:48:26 -0400 Received: by cs.utk.edu (cf v2.9s-UTK) id QAA25334; Mon, 22 Apr 2002 16:48:25 -0400 (EDT) Received: from rwcrmhc53.attbi.com (marvin@localhost) by cs.utk.edu with ESMTP (cf v2.9s-UTK) id QAA25305; Mon, 22 Apr 2002 16:48:17 -0400 (EDT) Received: from rwcrmhc53.attbi.com (204.127.198.39 -> rwcrmhc53.attbi.com) by cs.utk.edu (smtpshim v1.0); Mon, 22 Apr 2002 16:48:17 -0400 Received: from hephaestus ([24.98.218.176]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20020422204814.NBEY12144.rwcrmhc53.attbi.com@hephaestus>; Mon, 22 Apr 2002 20:48:14 +0000 Message-ID: <000c01c1ea40$46889fd0$0201a8c0@hephaestus> From: "Ben Schepens" To: "Keith Moore" , "Rescap mailing list" Cc: "Ben Schepens" , "Michael Mealling" Subject: Blob validation clarification Date: Mon, 22 Apr 2002 16:57:11 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 List-Owner: List-Post: List-Unsubscribe: Content-Transfer-Encoding: 7bit In the ResCap -- BLOB document, one of the validation steps (on page 14) is a little vague. I believe the following statement: - The integer_pool_offset must be equal to the the number of arguments (decoded from array_counts_and_flags) multiplied by 4 plus 20 . would be much clearer if rewritten as below: - The integer_pool_offset must be equal to the the total number of arrays decoded from array_counts_and_flags + 1 (for the integer scalar array offset) + 1 (for the blob scalar array) + 1 (for the string scalar array offset) multiplied by 4 (Size in bytes of an unsigned int offset value) plus 20 (for the Blob Header Length) expectedOffset = [ (total_array_counts + 3) * 4] + 20 ~Ben Schepens schepens@verisignlabs.com