Subject: [ftpext] Fwd: Re: ftpext2 review of FTP HOST command?

From: Tony Hansen <[email-masked]>
Date: Mon, 12 Mar 2012 10:42:23 -0400

I received the following review this weekend about the HOST command draft.

     Tony

-------- Original Message --------
Subject: 	Re: ftpext2 review of FTP HOST command?
From: 	Patrik Fältström

I have had a look at the draft, and some of the comments are general that I presume you have looked at.

In no specific order:

1. Denial of service attack

What is people try to do a dictionary attack by repeated HOST commands before the authentication?

I think the server should have the right to "just say no" after a number of failed HOST or failed combinations of host/authentication tries.

2. HOST command required

The text says this:

>     A server-PI that receives a USER command to begin the authentication
>     sequence without having received a HOST command SHOULD NOT reject the
>     USER command.

Is SHOULD NOT good enough?

Should there not be a specific error code if the HOST command is required?

Or should there be always failed authentication if a virtual host is not given?

Hmm...I am thinking of what the situation might look like many years from now. I do not think the HTTP/1.1 solution is good enough where there is a default host that normally is either crap, or worse, disclose data that give hints on low security in the host.

So, is it not better to say for example:

A server-PI that receives a USER command to begin the authentication sequence without having received a HOST command must either reject the USER command with error code XXX or treat the authentication as if the user authenticates towards a default host or the union of all virtual hosts.

3. Syntax of hostname

Is this really correct?

domain        = sub-domain *("." sub-domain)

Should it not be

domain        = sub-domain 1*("." sub-domain)

If my memory of abnf does not fail me, i.e. at least two domain name parts?

4. IDN

Suggestion, replace:

Internationalization of domain names is only supported through the
use of Punycode as described in [RFC3492].

With

Internationalization of domain names is only supported through the use of an A-label as defined in [RFCXXXX] as the hostname parameter.

5. HOST command after authentication

The text says this:

>     As discussed in section 3 of this document, if a HOST command is sent
>     after a user has been authenticated, the server MUST treat the
>     situation as an invalid sequence of commands and return a 503 reply.

Is there no command where the user can return to an unauthenticated state, but still be connected to the host? If so, I think it should be possible to give the HOST command there. So the key is that the HOST command is only allowed in a non-authenticated state, right?

Hope this helps...

    Patrik

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 Mar 2012 12:32:40 -0400

--On Monday, March 12, 2012 10:42 -0400 Tony Hansen
<[email-masked]> wrote:

>...

Tony,

I've got several comments on both the draft and on Patrik's
comments, but I also have a prerequisite question.  Put in the
broadest possible way, "Is the WG actually sufficiently
functional to process drafts and, if so, which drafts is it
serious about processing?"

Even if I were more convinced that "HOST" was needed rather than
being something whose functionality should be embedded in USER
or ACCT (I'm a lot closer on that topic today than I was a few
years ago), I'm not convinced that it is inherently more
important than ...-hash, ...-typeu, or even ...-ftp64 (I believe
the latter is seriously defective).  I'd consider it
procedurally obnoxious and probably unacceptable if the only
draft the WG is willing to process is one on which the co-chair
is a co-author.

So:

	Is the WG really functional?
	
	If I update and re-post ...-typeu (today or after the
	25th) are enough people willing to review it to permit
	generating a WG Last Call?
	
	Do we have commitments to implement whatever the WG
	comes up with?

If the answer to some or all of the above is "no", should people
really spend time reviewing this document or should we be
thinking about concluding that FTP's detractors are correct
about level of interest and hence give up on this effort?

Overworked people who are trying to figure out where to spend
their time would like to know.

best,
   john

---

From: Tony Hansen <[email-masked]>
Date: Tue, 13 Mar 2012 01:20:56 -0400

John, thank you for posting this note, as well as your updated draft. 
Pete Resnick, Anthony and I have also been discussing this very question 
for some time. We've been trying some behind-the-scenes efforts to get 
some more participation, but have had little success. Frankly, Pete is 
ready to just pull the plug.

Folks, if you care about this working group's existence, please speak up 
now. Your thoughts are requested on the topics of:

     *) the group's existence
     *) the HOST document
     *) the TYPEU document just posted

Thank you. If no one responds, be assured that the group will shut down 
shortly.

     Tony Hansen
     your friendly neighborhood co-chair

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 13 Mar 2012 08:32:18 +0100 (CET)

> 	Do we have commitments to implement whatever the WG

If server softwares would implement HOST and if server admins would configure 
their servers to use it, then I'd be inclined to add support for it (in curl).

A lot of ifs in there and given history I'd say we're a loooong way from that 
situation.

> If the answer to some or all of the above is "no", should people really 
> spend time reviewing this document or should we be thinking about concluding 
> that FTP's detractors are correct about level of interest and hence give up 
> on this effort?

I'm sorry to agree: this is not a very functional WG. To me, it looks like 
there's not enough interest from implementors to update FTP.

-- 

  / daniel.haxx.se

---

From: John C Klensin <[email-masked]>
Date: Tue, 13 Mar 2012 10:10:20 -0400

--On Tuesday, March 13, 2012 08:32 +0100 Daniel Stenberg
<[email-masked]> wrote:

>> If the answer to some or all of the above is "no", should
>> people really  spend time reviewing this document or should
>> we be thinking about concluding  that FTP's detractors are
>> correct about level of interest and hence give up  on this
>> effort?
> 
> I'm sorry to agree: this is not a very functional WG. To me,
> it looks like there's not enough interest from implementors to
> update FTP.

Let me say something slightly more optimistic than the above,
even though I don't think it helps the WG very much.  I think we
have several implementers out there, each with a client/customer
base.  While those clusters of customers probably overlap enough
in their properties and needs to be different from a
niche-per-implementer/ vendor, they are different.  The result
is that vendor X wants extensions A, B, and C (which they may
have developed, implemented, and deployed in some form).  Vendor
Y wants extensions D, E, and C', where C' is functionally more
or less like C but not identical.  Maybe we could get them
together to work on a common C-approximation (although there
hasn't been evidence of that so far).  But X sees no commercial
value in working on, or even reviewing D or E and Y sees no
commercial value in working on A or B.  

Worse from our point of view, X's main interest, as a server
vendor, in bringing A and B to the IETF is to get them
standardized so they can beat up client vendors who haven't
implemented what would otherwise be a perfectly good proprietary
feature.  That not only gives them no incentive to invest energy
on D or E, it gives them significant incentive to push back on
any changes to their already-implemented versions of A and B
(using "running code" as an excuse).

I think (Anthony can correct me if needed) we can understand the
target audience for HOST in that context.  While it is one that
might occasionally require strong user authentication (including
the usual cryptographic isolation of the relevant information
and/or handshaking), and might require integrity protection of
the data stream (and that is where draft-ietf-ftpext2-hash comes
in) it would rarely require privacy protection.  I say
"occasionally" because there would be many cases where the
approach would be used anonymously or with a minimum of
identification (just like "normal" FTP).   In that sense, the
model is very similar to web pages that are accessed to retrieve
information rather that, e.g., to carry out transactions that
might require authentication and/or transmission of sensitive
information.

If that is the model, then the proposal in this spec is
reasonable and most of the concern about TLS, especially TLS
with strong certificate-based protection, is largely irrelevant.
However, that view raises the obvious and usual questions:

	(1) Will (and should) the IETF accept a proposal with
	known security weaknesses if the author says "just don't
	use it in contexts requiring strong data privacy
	protection".  
	
	(ii) Are the WG and the IETF willing to accept an FTP
	Extension spec that doesn't play well with extant
	extensions?  The charter is remarkably silent on that
	matter.
	
	(iii) What value can/does the IETF add in this type of
	limited scenario case?
	
	(iv) Why doesn't the specification describe the usage
	scenarios clearly enough that it can clearly exclude
	those cases that require strong security?   
	
Pragmatically, what this all amounts to, IMO, is "draft needs
work".  And the evidence remains that the WG doesn't have
whatever it takes to get that work done.

     john

p.s. On looking at the charter, I discovered that one of the
things the WG committed to do was 

	"* Review and confirm or reject errata of current FTP
	RFCs"

Circa 18 months after we got serious about chartering the group
(longer than that if one uses other criteria), that task hasn't
been started.  Another entry in the "not a good sign" list, IMO.

---

From: TJ Saunders <[email-masked]>
Date: Tue, 13 Mar 2012 09:16:53 -0700 (PDT)

>     *) the HOST document

Given the paucity of IPv4 addresses (and their widespread use), I still 
consider name-based virtual hosting support in FTP, via the proposed HOST 
command, to be of considerable value.

I will be implementing this command in proftpd for the next RC release; as 
such, I have been reading closely many of the comments on this topic.

My impression is that the HOST command proposal will end up being 
implemented whether or not the WG continues to exist.  Without the WG 
pushing the proposal through to Standard status, however, those 
implementations are more likely to differ.  To ameliorate that 
possibility, I for one would like to see the WG continue.

I am not much for document formatting and scheduling and such, but if the 
perceived lack of progress on the HOST draft is due to lack of 
implementations, then I can do my part to address that need for an 
implementation.  I suspect others have already implemented HOST, however.

There have been, and will continue to be, questions about how to handle 
HOST edge cases (REIN, close the connection, etc), and how it will 
interact with e.g. TLS SNI.  These are excellent questions, and should be 
addressed -- but please do not let the lack of resolution on these 
questions cause the HOST draft to be stopped altogether.  "Don't let the 
perfect be the enemy of the good", as it were.

Just my thoughts on this particular topic.

Cheers,
TJ
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   Drink to me only with thine eyes,
   And I will pledge with mine;
   Or leave a kiss but in the cup
   And I'll not look for wine.

   	-Ben Jonson

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---

From: SM <[email-masked]>
Date: Tue, 13 Mar 2012 09:24:58 -0700

Hi John,
>(using "running code" as an excuse).

It's up to X and Y to figure out what's in their mutual 
interest.  Otherwise, one ends up with the classic scenario of a 
non-working group.

>been started.  Another entry in the "not a good sign" list, IMO.

The WG doesn't have any RFC to show since it was chartered in 
November 2010.  That's not a good sign.

Regards,
-sm

---

From: Anthony Bryan <[email-masked]>
Date: Tue, 13 Mar 2012 15:55:12 -0400

> serious about processing?"

I would like to hear your comments on both, John...

this WG may not be functional enough, but as I understand it we will
still have the mailing list (if the WG is closed) & drafts could
progress through individual submission.

so, either way, your comments will improve the draft - despite your
concerns, wouldn't you rather it be as good as possible?

here are the drafts on our charter.

FTP consideration for IPv4/IPv6 transition
http://tools.ietf.org/html/draft-ietf-ftpext2-ftp64
FTP TYPE Extension for Internationalized Text
http://tools.ietf.org/html/draft-ietf-ftpext2-typeu
HASH Command for Cryptographic Hashes
http://tools.ietf.org/html/draft-ietf-ftpext2-hash
HOST Command for Virtual Hosts
http://tools.ietf.org/html/draft-ietf-ftpext2-hosts

> draft the WG is willing to process is one on which the co-chair

no one is saying one is more important than the other, but some have
more interest than others.

I can see how my draft progressing would be annoying but don't worry
about that, there either will be a WG or there won't.

>        generating a WG Last Call?

I have reviewed it but I don't know enough about this subject matter
to comment unfortunately.

please, everyone who is knowledgeable on it, review typeu!

> about level of interest and hence give up on this effort?

HOST has 6 or more implementations already.
HASH, where I did a good deal of advocacy, has a number of
implementations ready to upgrade from similar previous non-standard
commands.

I don't think I have read of implementations for the other 2 drafts.

> their time would like to know.

I hear you - I'm publicly apologizing for where I've dropped the ball
in this WG. we all have real life to deal with & things come up.

all volunteer work is appreciated & I think we are doing useful work
that will be used by people. :)

-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads

---

From: Anthony Bryan <[email-masked]>
Date: Tue, 13 Mar 2012 16:33:52 -0400

> such, I have been reading closely many of the comments on this topic.

thanks for letting us know.

here are the other implementations as of a year ago:

IIS FTP service
Serv-U server
WS_FTP service
SmartpFTP client
Beyond Compare client
FTP Voyager client.

> possibility, I for one would like to see the WG continue.

as it already is implemented, I believe implementations would grow.

& I assume the draft would be submitted as an individual submission
which I think is more work than a WG submission?

I see HOST documented in an RFC better than it not.

> perfect be the enemy of the good", as it were.

I agree, these issues must be addressed.

I think we ran into 2 problems here...

1) HOST passed IETF wide Last Call almost 2 years ago. so people are
not as interested in reviewing something that has been in limbo & held
up by being included in this WG.

2) non core FTP issues like internationalization & security where we
need outside help (& have gotten some).

> Just my thoughts on this particular topic.

thank you very much, they have been helpful!

---

From: Anthony Bryan <[email-masked]>
Date: Fri, 16 Mar 2012 18:01:50 -0400

> commercial value in working on A or B.

perhaps I'm misunderstanding, but what are C & C'?

HASH vs the many hash like commands (XCRC, XMD5, XSHA, etc)?

> (using "running code" as an excuse).

I (& my co-authors) are the only ones who have proposed more than one
command (HASH, RANG, LOCK) to be standardized.

we are all independent AFAIK but work on our own implementations
(curl, aria2, FileZilla).

if you're talking about HOST, it was created by Bero in the open
source realm quite some time ago.
it appears to first be specified in 1998, in the ID that became RFC
3659: http://tools.ietf.org/html/draft-ietf-ftpext-mlst-05

if you mean, as far as a person from Microsoft investing time to
standardize/finalize it, isn't that a good thing to be done at the
IETF out in the open than "beat[ing] up client vendors" with something
proprietary that is only known internally & documented in a .doc file?
:) [no offense I have no idea how things work there.]

> whatever it takes to get that work done.

I agree the draft needs work, but do you think it needs more than
these last reviews (& hopefully a later AppsDir & SecDir review) can
provide.

granted obviously the WG has been at death's door, but we should still
have the mailing list if it gets closed.

> been started.  Another entry in the "not a good sign" list, IMO.

I see that an errata discussion started in July 2010.

some were filed in 2010 against RFC 959 & 3659 by a quick look, & we
should follow up that thread. most were marked "held for document
update"

---

From: Pete Resnick <[email-masked]>
Date: Fri, 30 Mar 2012 11:45:49 +0200

On 3/13/12 6:20 AM, Tony Hansen wrote:
> Pete Resnick, Anthony and I have also been discussing this very 
> question for some time. We've been trying some behind-the-scenes 
> efforts to get some more participation, but have had little success. 
> Frankly, Pete is ready to just pull the plug.
>
> Folks, if you care about this working group's existence, please speak 
> up now. Your thoughts are requested on the topics of:
>
>     *) the group's existence
>     *) the HOST document
> Thank you. If no one responds, be assured that the group will shut 
> down shortly. 

Though there was a small burst of posting in the last 2 weeks, there has 
not been enough participation to convince me that there is energy to 
sustain a WG effort. I've decided to close the WG. Documents that the WG 
was working on will be moved to individual documents and will be 
replaced with renamed documents (without the "draft-ietf-" name) if and 
when they are updated. If publication of these documents is desired, 
they will need to be published as individual submissions sponsored by an 
Area Director. The mailing list shall remain open for discussion of 
these documents and any other FTP work.

Thanks to everyone (especially the chairs and document editors) for your 
efforts in getting this work to move forward. I think we may have more 
luck dealing with these documents in individual efforts.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Tony Hansen <[email-masked]>
Date: Fri, 30 Mar 2012 20:41:17 -0400

Well folks, we've officially shut the working group down. That does 
*not* mean that work on these drafts needs to die.

First off, this mailing list will stay open indefinitely. (After all, 
it's the same list we used for the FTPEXT working group many years ago.)

Second, Anthony and I are not abandoning you. We will continue to be 
available to help shepherd the drafts through the IETF process.

However, the onus is shifting from the working group back to you, the 
individual. It is now *your* responsibility to push your documents forward.

*IF* you care about your particular FTP extension draft, here is a way 
forward:

1) Repost your draft as an individual submission. Please use a name of 
"draft-YOURNAME-ftpext-TOPIC".
2) Please post a note to this mailing list about your updated document.
3) Solicit comments from people, particularly people who you know who 
either have implemented your extension or are willing to implement your 
extension. Also solicit comments from people who are good at looking for 
completeness of the specs and making sure that the nits are taken care of.
4) When you have at least 5 or 6 people who are willing to send an email 
to the mailing list saying that your document is ready to be published 
as an RFC, send an email message to Anthony and the mailing list asking 
for it to be published.
5) Anthony will work with me to fill out the proper shepherding forms to 
get the Area Directory to issue a IETF-wide last call process.

Let the good times roll.

     Your (former) friendly neighborhood FTPEXT2 co-chairs
     Tony Hansen and Anthony Bryan

On 3/30/2012 5:45 AM, Pete Resnick wrote:
>
> Though there was a small burst of posting in the last 2 weeks, there 
> has not been enough participation to convince me that there is energy 
> to sustain a WG effort. I've decided to close the WG. Documents that 
> the WG was working on will be moved to individual documents and will 
> be replaced with renamed documents (without the "draft-ietf-" name) if 
> and when they are updated. If publication of these documents is 
> desired, they will need to be published as individual submissions 
> sponsored by an Area Director. The mailing list shall remain open for 
> discussion of these documents and any other FTP work.
>
> Thanks to everyone (especially the chairs and document editors) for 
> your efforts in getting this work to move forward. I think we may have 
> more luck dealing with these documents in individual efforts.
>
> pr
>

---

From: John C Klensin <[email-masked]>
Date: Fri, 30 Mar 2012 21:56:13 -0400

Hi,

Per Tony's "WG Status" note of about an hour ago, I've reissued
the "TYPE U" draft with the name draft-klensin-ftpext-typeu.

This is identical to draft-ietf-ftpext2-typeu except for
	* The file name
	* The date
	* A revised description of the mailing list on which the
	    draft is being discussed
	* Removal of the name of the FTPEXT2 Working Group
	* A new change log section that provides a backtrace.
	
Quid Pro Quo offer: I'll review yours if you will either review
or implement mine :-).

Note that a valid minimal implementation of this extension for
systems that use Unicode internally and already support
conversion of whatever encoding is used internally to and from
UTF-8 for on the wire use consists of:

	* Advertising the extension if one is a server and
	     receives a FEAT command.
	* Sending the extended TYPE request if one is a client
	* Ensuring that lines end with CRLF if one is a sender
	(note that the relevant code for this purpose is in any
	implementation that supports TYPE A).
	
One should also normalize to NFC, but that is not required (the
spec deliberately says "SHOULD...if at all possible", not MUST).

Note also that any data stream that is valid for TYPE A is also
value for TYPE U.

Systems that do not support Unicode natively would have to do
some potentially-significant conversions to support this
command; they are not candidates for a trivial implementation
unless appropriate tools already exist or are easily adapted.

  best,
    john