Subject: Ftp-WG: WG last call comment updates for mlst-09

From: Robert Elz <[email-masked]>
Date: Mon, 10 Jan 2000 03:27:18 -0500

The WG last call period Paul announced several weeks ago has ended now.
The only comments I saw related to the SIZE command.

Currently, for the draft that will follow, I propose making just one
change.

The sentence that read ...

	This command is normally used in conjunction with the RESTART
	(REST) command.

I propose changing to ...

	This command is normally used in conjunction with the RESTART
	(REST) command when STORing a file to a remote server in STEAM
        mode, to determine the restart point.

if there are no objections.  This is to make it clearer that the reason
for using SIZE with REST isn't to determine if the file has changed.

I thought about the possibility of defining a 4xx error response, but I really
couldn't think of anything that makes sense, or with a definition I would
know what do do with (nor could I figure out when writing a server when I
would ever issue such a command, given the prime motivation seems to be to
avoid denial of service - making the server do loads of unnecessary work.

If I felt it necessary to limit SIZE for that reason, I'd be more tempted to
do it by refusing SIZE for any file older than a few days, on the assumption
that no-one would reasonably be attempting to restart a file store that had
been interrupted for any lengthy period of time.   But that wouldn't be a
"temporary" condition, it isn't going to just fix itself - so a 4xx error
wouldn't be appropriate there anyway.

On the other hand, the current definition of the 550 error covers almost any
conceivable reason for not responding to a SIZE command - and would cover
this one as well.   Any client treating a 550 to a SIZE as meaning "file 
doesn't exist" is straight out broken.   Given that 550 has been the (only
non-syntax related) error code in SIZE implementations for 15 years (or
however long it has really been - about that long) that the command has
existed, I would hesitate to go defining new error codes now personally.

So, aside from that one change, the only other thing I intend doing for the
(hopefully final) version of the draft, is to delete the "this paragraph will
be deleted from the final version of this document" paragraph from the
abstract (the one giving a summary of the changes).

I'll wait a few more days before I submit the -10 draft to determine whether
there are any objections to this, or whether anyone has any (specific, with
wording) suggestions for other changes.

kre

---

From: "Sergey Ayukov (mailing lists)" <[email-masked]>
Date: Mon, 10 Jan 2000 08:57:34 -0500

> The only comments I saw related to the SIZE command.

Hello,

I am sorry for being late... Unfortunately, I have only recently joined
this mailing list (but spent some time reading previous archived
discussions). Here are my comments regarding draft 9:

1) I (and probably some other client writers) would find useful command
for *setting* file modification time. It could be utilized for doing
mirroring-by-upload. AFAIK, among current implementations only ncftpd
supports SITE UTIME.

2) The draft gives examples of OS-specific facts as OS/2.ea, MACOS.rf and
UNIX.mode. How reasonable are the first two? I don't know much about
Macintosh resource forks, but OS/2 extended attributes can IIRC reach 64K.
Is this a reasonable way of handling them? (I did find comment about line
length not being limited). BTW, two BeOS vendors have recently developed
the extension (using SITE commands) to exchange attributes between client
and server (http://www.desertnights.com/attribute.shtml).

3) The draft gives no way of setting the facts for remote files.
Apparently not all facts would be settable, but at least some of them are.
When defining the way to retrieve the facts it could make sense to define
standard way to set them (that would also eliminate item 1, of course).
Taking again OS/2 EAs as an example, such asymmetry is going to create
some proprietary extensions for attaching attributes to files on server.

4) "Unique" fact is supposed to help mirroring programs. Currently they
rely on listings to detect and handle symbolic links. How these programs
are supposed to know how the links are organized? (I have noticed that
draft says that information about actual links is not available). Since
typical mirroring program tries to represent remote file structure as
close as possible, I would expect them (at least Unix ones) not using MLSx
since it will limit the functionality.

I understand that these comments are rather late and the committee is
gearing towards the final version. If the general opinion is that the RFC
must be published as soon as possible without discussions of any real
length, I will find it reasonable.

---------------------------------------------------------------------------
Dr. Sergey Ayukov                          Sternberg Astronomical Institute
http://www.ayukov.com                                        Moscow, Russia
http://crydee.sai.msu.ru/index-asv.html

---

From: Alun Jones <[email-masked]>
Date: Mon, 10 Jan 2000 09:50:04 -0500

>existed, I would hesitate to go defining new error codes now personally.

However, since it's clear that some clients _do_ treat 550 to mean the file 
does not exist, how about a brief sentence or two to indicate that SIZE 
returning 550 does not mean that the file can not be transferred?

Section 4.2 seems fairly limiting on what might cause the 550:

    Where the command is correctly parsed, but the size is not available,
    either because the pathname identifies no existing entity, or because
    the entity named cannot be transferred in the current MODE and TYPE
    (or at all), then a 550 reply should be sent.

Here, we're suggesting that 550 also could mean "I don't feel it's 
appropriate for me to go to the effort of calculating the size".

The wording I would suggest, would add on to the end of 4.2, and go 
something like:

"The presence of the 550 error response to a SIZE command MUST NOT be taken 
by the client as an indication that the file can not be transferred in the 
current MODE and TYPE.  A server may generate this error for other reasons 
- for instance if the processing overhead is considered too great."

Alun.
~~~~

--
Texas Imperial Software | Try WFTPD, the Windows FTP Server. Find it
1602 Harvest Moon Place | at web site http://www.wftpd.com or email
Cedar Park TX 78613     | us at [email-masked].  VISA / MC accepted.
Fax +1 (512) 378 3246   | NT based ISPs, be sure to read details of
Phone +1 (512) 378 3246 | WFTPD Pro, NT service version - $100.
*WFTPD and WFTPD Pro now available as native Alpha versions for NT*

---

From: Robert Elz <[email-masked]>
Date: Mon, 10 Jan 2000 10:15:00 -0500

Date:        Mon, 10 Jan 2000 09:50:04 -0500
    From:        Alun Jones <[email-masked]>
    Message-ID:  <[email-masked]>

  | However, since it's clear that some clients _do_ treat 550 to mean the file 
  | does not exist, how about a brief sentence or two to indicate that SIZE 
  | returning 550 does not mean that the file can not be transferred?

Seems reaosnable.

  | Section 4.2 seems fairly limiting on what might cause the 550:
  | 
  |     Where the command is correctly parsed, but the size is not available,
  |     either because the pathname identifies no existing entity, or because
  |     the entity named cannot be transferred in the current MODE and TYPE
  |     (or at all), then a 550 reply should be sent.

The either or case wasn't really meant to be an exhaustive list, just
examples of why "the size is not available" - "I don't want to make
it available" would be another possibility.   I might change "either" to
"perhaps" to make that clearer.

  | "The presence of the 550 error response to a SIZE command MUST NOT be taken 
  | by the client as an indication that the file can not be transferred in the 
  | current MODE and TYPE.  A server may generate this error for other reasons 
  | - for instance if the processing overhead is considered too great."

OK, I will wait for more comments, and if no more is added, will add that.

kre

ps: just in case someone else notices, Luke Mewburn already pointed out
the STEAM/STREAM typo in the sentence I sent in the previous message.

---

From: Robert Elz <[email-masked]>
Date: Mon, 10 Jan 2000 10:34:38 -0500

Date:        Mon, 10 Jan 2000 08:57:34 -0500
    From:        "Sergey Ayukov (mailing lists)" <[email-masked]>
    Message-ID:  <[email-masked]>

  | 1) I (and probably some other client writers) would find useful command
  | for *setting* file modification time. It could be utilized for doing
  | mirroring-by-upload. AFAIK, among current implementations only ncftpd
  | supports SITE UTIME.

I have seen clients (not sure about servers) attempt to pervert MDTM
by giving it two args (time & path).   One of the examples comes from
a client which once attempted to do that, and is there precisely to make
it clear that isn't rational.    And yes, a command for that certainly has
its uses - but wasn't really ever in the scope of what was being done

It can be argued (fairly easily) that there are two totally separate
things here - the mls* commands and the size/mdtm/rest add ons, and that
they should perhaps be in separate docs completely, with the former being
new stuff with a particularly limited scope, and the latter being simple
doc of existing functions long available and never properly documented.
Having them combined makes it appear that adding functionality to the
latter group would be reasonable for the WG to be doing, which I don't
think was ever in the game plan.  Setting mod times would/might fall
into that category.   Back when the group was newer, adding that as a
work item would probably have been accepted, now this group is really
badly running out of steam, and probably needs to be buried asap.

  | 2) The draft gives examples of OS-specific facts as OS/2.ea, MACOS.rf and
  | UNIX.mode. How reasonable are the first two?

Almost certainly not at all.   I know nothing at all about OS/2.  What
little I know about mac filesystems would suggest that the only rational
way of defining a MACOS.rf fact would be to define its value as being the
name of a "hidden" file (probably in the same virtual directory) which could
be retrieved (perhaps MLST'd as well) to obtain the resource fork data.
There's no real limit on how much that can be - for applications, all of the
code lives in the resorce fork.   But defining how that works, if it can
be made to work at all, is really for those who deal with mac servers
and clients.

  | 3) The draft gives no way of setting the facts for remote files.

No, listing formats were all we were doing (in this part of the work,
other results have already appeared as RFCs).   Again, the ability to set
some of the options would probably be useful, but I suspect, not by this
WG, a new one would be needed, after a small break probably.

  | 4) "Unique" fact is supposed to help mirroring programs. Currently they
  | rely on listings to detect and handle symbolic links. How these programs
  | are supposed to know how the links are organized?

In theory, they're not.  Unix style symbolic links are a horribly system
dependant beast.   In the virtual world, the best that can be done really
is to indicate that two names represent the same file, and leave it to the
client to duiplicate that the best it can.  From the ftp standpoint, it
would all look the same to another client - which name is the symlink, and
which is the actual file, really doesn't matter there.

For other applications than making a mirror of an ftp archive for use as
an ftp mirror server, you really can want better info than that - but it
is typically useful only on servers which can fairly accurately represent
the filesystem details of the server (often the same basic OS).  That's
where OS dependant facts are needed, and a fact to indicate that "this name
is a symlink and its value is xxxx" would be an entirely reaslnable unix
dependant fact to define sometime.   I don't think that more than what
now exists is reasonable for a filesystem independant fact set.

Thanks for the comments,

kre

---

From: "Sergey Ayukov (mailing lists)" <[email-masked]>
Date: Mon, 10 Jan 2000 13:04:29 -0500

> a client which once attempted to do that, and is there precisely to make

This is certainly no good, and I did not make such proposals (extending
MDTM in such way).

> they should perhaps be in separate docs completely, with the former being

Agreed.

> be made to work at all, is really for those who deal with mac servers

Ok (in other words, let vendor wildest fantasies unfold ;-) Since I am
doing FTP client implementation under several systems, and at least two of
them (OS/2 and BeOS) have attributes associated with files, I was really
interested how do I suppose to handle that according current draft 9.
 
> WG, a new one would be needed, after a small break probably.

I see.
 
> which is the actual file, really doesn't matter there.

Neverthess many new systems are picking up this "horribly system dependant
beast" because it appears to be fairly convenient. *IMHO* the balance
between an ideal world of pure FTP and real filesystems and their quirks
isn't right here; too much abstraction! (I have found however that the
current draft includes special type of OS.os-name=os-type which could
handle links just fine).

> now exists is reasonable for a filesystem independant fact set.

Maybe you're right. 

Thank you for the notes on my comments.

---

From: Robert Elz <[email-masked]>
Date: Mon, 10 Jan 2000 22:27:37 -0500

Date:        Mon, 10 Jan 2000 13:04:29 -0500
    From:        "Sergey Ayukov (mailing lists)" <[email-masked]>
    Message-ID:  <[email-masked]>

  | This is certainly no good, and I did not make such proposals (extending
  | MDTM in such way).

No, I know - I was just pointing out that there have been others who
have seen the need for a way to set the modify time (where that is even
possible) and that various approaches have been tried.   This one needs
to be worked out.   It is probably new enough that it could do with
some discussion at an actual face to face working group meeting, of some
working group - this group hasn't actually met in years now.

  | Ok (in other words, let vendor wildest fantasies unfold ;-)

Not quite - the OS specific extensions are supposed to be documented
so that everyone can agree on exactly what they mean, and how they
should be used.   The draft sets out the method for doing that.

  | Since I am
  | doing FTP client implementation under several systems, and at least two of
  | them (OS/2 and BeOS) have attributes associated with files, I was really
  | interested how do I suppose to handle that according current draft 9.

You're supposed to write down a proposal for the additional data
that you want to add for those systems, and submit it to IANA (after
this draft becomes an RFC).

  | (I have found however that the
  | current draft includes special type of OS.os-name=os-type which could
  | handle links just fine).

Yes, it can, though that should be used in addition to, not instead of
(perhaps unless some other negotiation is performed), listing the link as
a regular file (or directory) - that is, using the "two types for one object"
mechanisms.   Simply listing things as OS dependant types, and nothing else,
excludes them from any handing by clients that don't understand that OS,
and for something as simple as a linked file, that's not really necessary.

kre

---

From: "Sergey Ayukov (mailing lists)" <[email-masked]>
Date: Thu, 13 Jan 2000 09:59:51 -0500

> and for something as simple as a linked file, that's not really necessary.

Personally I did not like the idea of multiple entries per entity at all
in the current draft. They are confusing when read by unsuspecting humans,
and ugly when processed by an automatic parser. In the situation when
'unique' fact is not available or was not requested for performance
reasons, the ways to identify entries pointing to single entity become
rather unclear.

---

From: Robert Elz <[email-masked]>
Date: Thu, 13 Jan 2000 13:31:15 -0500

Date:        Thu, 13 Jan 2000 09:59:51 -0500
    From:        "Sergey Ayukov (mailing lists)" <[email-masked]>
    Message-ID:  <[email-masked]>

  | Personally I did not like the idea of multiple entries per entity at all
  | in the current draft.

They never really appealed to me all that much either - but do seem
to be the most obvious solution to an occasional, though existing,
problem.

  | They are confusing when read by unsuspecting humans,

MLS* output should never be "read by unsuspecting humans".   If the
output is just going to be dumped in front of a human (other than one
debugging an MLS* parser or similar who ought be presumed to not be
unsuspecting) then LIST ought be used instead - its output is typically
intended for human viewing.   On the other hand if the software is
going to reformat and "pretty up" the MLS* output, then it ought handle
this case.

  | and ugly when processed by an automatic parser.

Surely that depends upon the parser?

  | In the situation when
  | 'unique' fact is not available or was not requested for performance
  | reasons, the ways to identify entries pointing to single entity become
  | rather unclear.

No, this one is easy.   If the filename is the same (exactly the same)
then the two entries represent exactly the same file, and two types are
being represented for the same file.  If the filenames aren't identical
then two different objects are being represented.   Unique facts are
specifically not required for this.   Those are only to deal with multiple
names for the same file (however many types that file has).   Of course
if the unique fact is being returned on a multi-type multi-line listing
for one filename, then it will probably have the same value (though as
I think about it, I am not certain it always will, and I certainly would
not rely upon that).

kre

---

From: Alun Jones <[email-masked]>
Date: Thu, 13 Jan 2000 15:19:14 -0500

At 12:31 PM 1/13/2000 , you wrote:
>    From:        "Sergey Ayukov (mailing lists)"
>

Just to recap for the latecomers, the existing problem (as I posed it, at 
least) was that of a server that wants to treat zip files (or gz, tar, zoo, 
arc, etc - any archive format) both as file and as directory.  Hence, you 
could download the zip file, or you could 'CWD' into it and retrieve 
individual elements.

I haven't implemented this, and I'm not aware of any implementations, but 
it _is_ part of my "wish-list" of features that I will be implementing, and 
it causes issues with MLST, in that a file may be of two different types 
simultaneously; perhaps even with different attributes depending on how it 
is accessed.  It causes, of course, similar problems with LIST, since the 
first character of the de-facto standard 'ls -l' format only allows for 
file _or_ directory, and most GUI clients will not allow you to treat a 
file as both without some jumping through hoops.

Alun.
~~~~

---

From: Stephen Tihor <[email-masked]>
Date: Thu, 13 Jan 2000 15:23:37 -0500

The only question for this level of draft really is should 
multiple entryies returned for one entity be (a) explicitly forbidden
or (b) explicitly permitted (probably with a pointer that some OS specific
or generic note should follow in a suitable place (later RFC?), or (c)
benignly ignored.       There is certainly an arguement for noting (B) and
telling peopole that they should not blow up if the same name appears multiple times.

---

From: "Sergey Ayukov (mailing lists)" <[email-masked]>
Date: Fri, 14 Jan 2000 05:28:55 -0500

> MLS* output should never be "read by unsuspecting humans".   If the

If the output is human-readable (and MLST output is), it is going to be
read by people. Anyone not familiar with this RFC will be qualified as
"unsuspecting", although I agree that this is not important.

> Surely that depends upon the parser?

I meant that it breaks (rather established) paradigm of "one entry, one
entity".
 
> being represented for the same file.  If the filenames aren't identical

I see; thanks for clarification. I have misread the draft.

---

From: Robert Elz <[email-masked]>
Date: Fri, 14 Jan 2000 05:39:29 -0500

Date:        Fri, 14 Jan 2000 05:28:55 -0500
    From:        "Sergey Ayukov (mailing lists)" <[email-masked]>
    Message-ID:  <[email-masked]>

  | I see; thanks for clarification. I have misread the draft.

I will look and see if there is anything that should be improved in the
draft to make it less likely to be misread - this is exactly the kind of
thing that should be fixed now, before the RFC is done.

kre

---

From: "Sergey Ayukov (mailing lists)" <[email-masked]>
Date: Fri, 14 Jan 2000 05:43:42 -0500

> could download the zip file, or you could 'CWD' into it and retrieve 

I did read the archives and recall the idea about CWDable zip files.
However, this issue seems to be very different for me than symlinks. With
archive-as-directory, client is not required to identify both items as
being the same (and they are not the same in the sense of actions which
could be applied to them). With symlinks the situation is different: the
aware client must interpret them as pointing to the same entity on server
(by means of identical name).

> is accessed.  It causes, of course, similar problems with LIST, since the 

How are you going to distinguish "how it is accessed"? For example, if
"SIZE directoryname" command would be defined as returning full size of
the directory (sum of lengths of all files inside it; purely fictitious
example), what server should do if "SIZE filename.zip" is issued: respond
with size of the zip file or sum of sizes of entries in it? The usage of
one name to represent two essentially different things does not seem to be
reasonable for me; I would say that using different name for
archive-as-directory (say, appended with ".dir"?) would be better.

> file as both without some jumping through hoops.

It actually allows them to be 'l' (symbolic link) which can point to file
or directory. I have to agree that handling this situation in the client
is rather cumbersome.

---

From: "Sergey Ayukov (mailing lists)" <[email-masked]>
Date: Fri, 14 Jan 2000 05:48:26 -0500

> thing that should be fixed now, before the RFC is done.

I understand that this draft was in the works for years now, and most WG
members want it be published and be done with it. I have no objections
to that.