Subject: Ftp-WG: alternative thoughts on RETR/response variations

From: Stephen Head <[email-masked]>
Date: Wed, 7 Jun 2000 20:13:28 -0500

I originally thought I could try to live with the de-facto 150 response
but became concerned with parsing it correctly under the circumstances,
and/or parsing an extended version with funky characters in the pathname

and an MTDM value embedded somewhere.  It seemed a bit risky for
the ROI.

As a workaround for no ground given on the de-facto 150 response,
a proxy could transform a RETR pathname command into a bundled sequence
of commands to the origin server like

  (TYPE I)

...

  (receive RETR request from destination client)
...

  PWD
  MDTM pathname
  SIZE pathname
  RETR
...

  (send data to destination client, possibly transformed into ASCII,
  and optionally cache)

for each request from the destination client,
and process the responses as they come (ftp server daemon processes
could, independently, be trained to look for such sequences and respond
in kind with an aggregated response sequence if encountered).
(SIZE and response could be replaced with a standardized, reliable form
of the de-facto response which reported an accurate image size --
variation.)

An economization of this would be to have an extended version
of RETR, perhaps something like XRET, and pair with it
a correspondingly extended 15x-Ext response of the type
I mentioned earlier.  An XRET could also carry with it the notion
of switching momentarily to image type and then revert back
to the previous type -- a minor variation.  (Compression?)

Ideally, in both cases, the response would contain an exact size,
at least modulo the issues that have been brought up about that
so far (dynamic file generation, dynamic file modification, and
possibly stuff like structured file record deletes on OSs with those
types
of files).

Considering tradeoffs at a glance, the former alternative seems
to have somewhat more overhead even if requests and responses
are bundled on the control channel.

I guess no one uses non-Stream mode transfer so 12x-Ext
responses would be moot.  (?)

-s

---

From: Alun Jones <[email-masked]>
Date: Wed, 7 Jun 2000 20:42:27 -0500

>and an MTDM value embedded somewhere.  It seemed a bit risky for

Six of one, half a dozen of the other.  While it's easy to demonstrate a 
degenerative case, there are very few in actual circulation, and it's none 
too difficult to claim that a user creating that kind of file name deserves 
for it to cause problems with FTP clients (as long as the clients don't 
crash!).

>of commands to the origin server like

Why not PASV|PORT / MLST / [REST / ] RETR?

>of the de-facto response which reported an accurate image size --

I'm uncomfortable with this idea of batching requests and/or responses, but 
I can't _quite_ put it into words just yet.  Maybe someone else can?

>to the previous type -- a minor variation.  (Compression?)

MLST is one extra command, and provides essentially what you want.

>possibly stuff like structured file record deletes on OSs with those

For the vast majority of cases, MLST will provide exactly the information 
you're looking for, or close enough that a proxy can figure out whether to 
cache or not.  SIZE and MDTM will not practically provide you with more 
information, but they can take longer.  Note, also, that in the usual case 
where MLST responds with an incorrect size, ASCII mode transfer, it is not 
always possible to determine whether the size given from any one command 
will match the size of the file when stored on your proxy.  In that 
respect, it appears that the SIZE command is at least as inaccurate as the 
MLST facts!  [You _could_ call MLST _and_ SIZE, to find out how many 
carriage-return line-feed pairs there are, but again, I can provide you 
with a degenerate case to screw your calculations up.]

>are bundled on the control channel.

One or two extra commands are surely not that much of an overhead - and we 
are still talking about a proxy cache, which is not IMHO a widespread 
application for FTP.  You're asking for a change to be made that affects 
many clients, in order to help out a few?  Where's that Vulcan blood? :-)

>responses would be moot.  (?)

I've given some thought to implementing non-stream modes, but nobody's 
asked for it yet, so it isn't high on my list of priorities.  A client can 
ask for status information during the stream-mode transfer, but again, 
there is no standard for the response format.

Seems to me, you're asking the standard to be altered mainly so you (or 
some other proxy author) won't have to do as much work, and that work can 
be mostly offloaded to the server!  As a server author, I'm going to 
suggest that the proxy needs to do the work, and take the hits. :-)

Alun.
~~~~

--
Texas Imperial Software | Try WFTPD, the Windows FTP Server. Find us
1602 Harvest Moon Place | at web site http://www.wftpd.com or email
Cedar Park TX 78613     | us at [email-masked].  VISA / MC accepted.
Fax +1 (512) 378 3246   | NT-based ISPs, be sure to read details of
Phone +1 (512) 378 3246 | WFTPD Pro, NT service version - $100.

---

From: Stephen Head <[email-masked]>
Date: Wed, 7 Jun 2000 22:08:10 -0500

> for it to cause problems with FTP clients (as long as the clients don't

Sure.  I would tend to add any potential proxies to the list of those
applications that would tend not  towant to take any unnecessary chances,
though.  I would be reluctant to try to formalize the existing syntax
without allowances for any heretofore legal pathname, though.
People seem to be extending this in various different ad-hoc ways,
which will probably(?) lead to a mess sooner or later, but ... whatever...

> Why not PASV|PORT / MLST / [REST / ] RETR?

Sorry, yes, I think for what I was looking for,
SIZE and MDTM can be replaced by MLST,
but I was under the impression that MLST and friends were
not required to report a fully qualified pathname .  As symlink
can cause client side confusion about where in the server NVS
one is, I believe, a proxy would seem to want a PWD, at least
for each new remote directory entered, since my reading of
d-10 did not seem to indicate that the return of a _fully_ _qualified_
pathname was required:

7.2. Format of MLSx Response

   The format of a response to an MLSx command is as follows:

       ...

              entry            = [ facts ] SP pathname

although the example later appears to return a fully qualified pathname
(at least if it's a UNIX style NVS).  (Formal disambiguation would be nice
if so.)

I was deferring considering REST but my concept was if I could get
a sense of what to do / what would be best for RETR, REST would
most likely resolve itself as the issues are pretty much the same
as far as I can tell at a first pass.

> I can't _quite_ put it into words just yet.  Maybe someone else can?

Me too, for what it is worth, but what to do.  Maybe some people already
do this for other commands already, though.  Just as long as the _server_
doesn't crash, maybe... :-)

> MLST is one extra command, and provides essentially what you want.

Add PWD (?) and sometimes TYPE I (?) if I'm OK above.

  ...

...

> suggest that the proxy needs to do the work, and take the hits. :-)

There are I believe actually three kinds of work involved here,
implementation, transmission and computation.
Given a standard tends to live forever (RR track widths and Roman chariots,
etc.)
I put somewhat more weight on the latter two than the former.  Perhaps my
concerns
were misplaced in some way (eg idealistically slanted), but I don't think
grossly so
given the various issues and ambiguities involved.  If any of my arguments
hit any soft spots in the draft it may still be worthwhile to address those
at before going prime time with it.  I presumed the wg would find the concerns

of at least passing interest as a projected user (enthusiastic consumer? :-)
of the upcoming extension.

Re: Vulcan blood, please don't tempt me to respond in Klingon :-).

-s