Subject: Re: Late comments on the msgtrk documents

From: Eric Allman <[email-masked]>
Date: Fri, 12 Oct 2001 10:44:08 -0700

Philip, sorry to be so long in responding.  Your comments were very
good -- nothing like having a fresh pair of eyes.  I'll be distributing
new drafts to the list later today.  My comments follow.

: Comments on [SMTPEXT]
: 
: Section 3, paragraph labelled (4):
: 
:   (a) This paragraph talks about the "MAIL FROM command". In fact, all
:       other RFCs about SMTP call it the "MAIL command". This change is
:       needed throughout the document.

A quick scan shows that RFC numbers 1426, 1651, 1652, 1653, 1830, 1845,
1854, 1869, 1870, 1894, 2197, 2421, 2442, 2476, 2554, 2852, 2920, and
3030 all refer to the "MAIL FROM command".  However, the series that
this is most closely modeled on (DSNs, RFC 1891-4) do not use that
terminology.  For this reason, and because we refer to the RCPT command
(not the RCPT TO command), I've gone ahead and made the suggested
change, but it's hardly universal.  Anyone else want to throw anything
in on this?

:   (b) The second sentence reads "In addition, the ENVID and ORCPT
:       parameters (as defined in RFC 1891 sections 5.4 and 5.2
:       respectively) MUST be supported, with extensions as described
:       below." This implies that both these parameters are for the MAIL
:       command, but ORCPT is actually a parameter of the RCPT command.
:       Also, I can't see any extensions for the ORCPT command in the rest
:       of the document. Suggested rewording:
: 
:         In addition, the ENVID parameter of the MAIL command (as defined
:         in RFC 1891 section 5.4) MUST be supported, with extensions as
:         described below. The ORCPT parameter of the RCPT command (as
:         defined in RFC 1891 section 5.2) MUST also be supported.

Agreed.

: Section 3, paragraph labelled (5):
: 
:   This paragraph is talking about the length of the MAIL command, and
:   says "Note that a further extension of 614 characters for the OCRPT
:   and ENVID parameters is required..." This can't be right, because
:   ORCPT is not a parameter of MAIL.

Excellent point.  I've changed the wording as follows:

    (5)   The maximum length of a MAIL command line is increased by 40
          characters by the possible addition of the MTRK keyword  and
          value.   Note  that the 507 character extension of RCPT com-
          mands for the ORCPT parameter and the 107  character  exten-
          sion of MAIL commands for the ENVID parameter as mandated by
          RFC 1891 [RFC-DSN-SMTP] must also be included.

: Section 4.1:
: 
:   The first sentence reads "Any sender wishing to track a message
:   must...". I think it would be clearer to say "Any sender wishing to
:   request the retention of data for subsequent tracking of a message
:   must...".

Agreed.

:   A few paragraphs down there is "B is stored in tracking databases of
:   compliant MTAs..." I suggest a change to "compliant receiver MTAs" to
:   contrast with the start of the paragraph, which covers the sender.

Done.

: Section 4.2:
: 
:   The last paragraph reads:
: 
:            Any resubmissions of this message into the  message  trans-
:       mission  system  MUST  assign  a  new  ENVID.   In this context,
:       "resubmission" includes forwarding or resending a message from a
:       user  agent, but does not include MTA-level aliasing or forward-
:       ing where the message does not leave and  re-enter  the  message
:       transmission system.
: 
:   What does this mean in the context of an MTA aliasing that turns one
:   incoming address into two outgoing addresses, causing two different
:   copies of the message to be sent to two different MTAs? Does each copy
:   retain the ENVID? (This is probably answered by the following section,
:   but perhaps some words here might help the reader the first time
:   through, and emphasize the point about one-to-one redirection.)

I left this as is, partially because it is explained later, but
partially because the point here is when you MUST assign a new ENVID,
not when you need to delete or propagate it.

This brings up another point however.  RFC 1891 specifies the ENVID
value as being 100 characters.  Since we have changed this to
local-envid "@" fqhn, should we extend this (I claim yes)?  If so,
to what value?

: Section 4.3:
: 
:   The second and third paragraphs had me somewhat confused. I *think* I
:   now know what is meant, but I may be wrong.
: 
:            If aliasing, forwarding, or other redirection  of  messages
:       to  a single recipient occurs, then the MTA SHOULD treat this as
:       an ordinary hop-to-hop transfer and forward the  MTRK=,  ENVID=,
:       and ORCPT= values; these values MUST NOT be modified.
: 
:   "redirection of messages to a single recipient" can be read in one of
:   two ways:
: 
:       (a) A single envelope recipient (possibly one among many) is
:       redirected to a new single recipient, and that copy of the message
:       is what we are talking about.
: 
:       OR
: 
:       (b) A message contains only a single (incoming) envelope
:       recipient, and this paragraph covers just that case.
: 
:   I think it must be (a) that is intended. Is this wording better?
: 
:       If aliasing, forwarding, or other redirection of a recipient
:       occurs, and the result of the redirection is exactly one new
:       recipient, the MTA SHOULD treat this as...

Yes, I agree, except for the word "new".  That could imply that if
you forward mail to yourself and another address, this clause applies.
In fact it doesn't, except as modified below.

:   [This begs the question of the very common case where somebody who is
:   temporarily somewhere else sets up a .forward file that saves messages
:   in their local mailbox, and also forwards them to a single forwarding
:   address. Does this count as "one-to-one" redirection, or not? I
:   suppose it has to be not.]

Generally this would not count as one-to-one redirection.  But see below
for a possible exception designed specifically for this case.

:   The next paragraph reads thus:
: 
:            MTAs  MUST NOT copy MTRK certifiers when relaying a message
:       to multiple recipients.  An MTA MAY designate one recipient in a
:       multi-recipient alias as the "primary" recipient to which track-
:       ing requests shall  be  forwarded;  other  addresses  SHALL  NOT
:       receive  tracking certifiers.  MTAs MUST NOT forward MTRK certi-
:       fiers when doing mailing list expansion.
: 
:   Does "relaying a message to multiple recipients" mean that the message
:   envelope contains more than one recipient, and the MTA is passing it
:   on, or does it mean that a single envelope recipient is expanded to
:   more than one address, and these are passed on? I think the latter is
:   intended, in which case perhaps some wording that mirrors the previous
:   paragraph might be clearer:
: 
:       MTAs MUST NOT copy MTRK certifiers when a recipient is aliased,
:       forwarded, or otherwise redirected and the redirection results
:       in more than one new recipient. However, an MTA MAY designate
:       one of the new recipients as the "primary" recipient...

Yes, this is better.

:   [My case of "save in mailbox and forward" mentioned above is probably
:   an example of where the forward address should be "primary", I
:   suppose.]

Exactly.  Although it isn't clear which address should be "primary".
I would argue that if I leave a copy on my host and also forward my
mail to an agent that decides if it should be reformatted and forwarded
to my cell phone, then the host version is "primary".  But that's why
this is (intentionally) left undefined.

: Section 5.2:
: 
:   (Minor comment.) I felt it might be helpful to add a final paragraph
:   along these lines:
: 
:     Therefore, implementors are encouraged to provide mechanisms by
:     which site administrators can choose between these alternatives.

Done.

: Section 6:
: 
:   [RFC_SMTP] should refer to 2821, not 821

I've changed this.  However, it isn't clear to me if IETF policy is
to refer to the full standard (821) or the standards track (2821).

: ---------------------------------------------------------------------
: Comments on [TRKSTAT]
: 
: References to RFC 822 should refer to 2822 instead.

Changed (except on one case, where the document actually references
RFC 1894 section 2.1.3, title of which is ``Lexical tokens imported
from RFC 822).

: Section 3.3.2 REQUIRES the final-recipient field. I know that many
: administrators do not want to expose the results of redirection or other
: address rewriting. I suspect, therefore, that this field will be of very
: little use in practice.

We modeled this document heavily on the DSN series (1891-4).  In
particular, both 1891 and 1894 require final-recipient.  I didn't
see any reason to require it there and make it optional here.

The truly paranoid can declare delivery to their firewall as "final
delivery", in which case no internal topology is exposed.

: Section 3.3.5 (Remote-MTA field):
: 
: What if several MTAs have been tried, and the message is delayed because
: all are unreachable? Should this field be omitted, or should one MTA be
: chosen at random?

I'm confused -- the version that I'm look at doesn't say this at all.
The current wording is

      3.3.5.  Remote-MTA field

              The  Remote-MTA field is defined as in section Reference
         2.3.5 of [RFC-DSN-STAT].  This field MUST NOT be included  if
         no  delivery  attempts  have been made or if the Action field
         has value "opaque".  If delivery to some agent other than  an
         MTA (for example, a Local Delivery Agent) then this field MAY
         be included, giving the name of the host on which that  agent
         was contacted.

: Section 3.3.7 (Will-Retry-Until field) is hard for at least one MTA
: (Exim) to implement (because of the way its retrying is designed). By
: making this field REQUIRED, this specification assumes a certain kind of
: retry implementation.

I changed this to make it optional.  Any objections?  Wording is

      3.3.7.  Will-Retry-Until field

              The Will-Retry-Until field is defined as in section Ref-
         erence 2.3.8 of [RFC-DSN-STAT].  If the message is not in the
         local  queue or the Action field has the value ``opaque'' the
         Will-Retry-Until field MUST NOT be included; otherwise,  this
         field SHOULD be included.

eric

+-----------------------------------------------------------------------+
|Eric Allman              Sendmail, Inc.         +1-510/594-5501 (work)	|
|Chief Technical Officer  6425 Christie Ave      +1-510/594-5429 (fax)	|
[email-masked]        Emeryville, CA  94608  +1-510/843-9684 (home)	|
+-----------------------------------------------------------------------+

---

From: Lyndon Nerenberg <[email-masked]>
Date: Fri, 12 Oct 2001 12:26:01 -0600

> change, but it's hardly universal.  Anyone else want to throw anything

RFC2821 uses "MAIL command." Since that's the definitive reference we
should be consistent with it.

--lyndon

/* Pre-C-Preprocessor to translate ANSI trigraph idiocy in BUF
   before main CCCP processing.  Name `pcp' is also in honor of the
   drugs the trigraph designers must have been on. */

---

From: Philip Hazel <[email-masked]>
Date: Mon, 15 Oct 2001 09:05:14 +0100 (BST)

On Fri, 12 Oct 2001, Eric Allman wrote:

> Philip, sorry to be so long in responding.

No problem. I wasn't exactly waiting! But the IETF meeting in London now
feels as if it was in another universe and long ago.

Thanks for responding in detail. I don't think I've got any further
comments.

Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
[email-masked]      Cambridge, England. Phone: +44 1223 334714.

---

From: Gregory Neil Shapiro <[email-masked]>
Date: Mon, 15 Oct 2001 11:11:35 -0700

eric> This brings up another point however.  RFC 1891 specifies the ENVID
eric> value as being 100 characters.  Since we have changed this to
eric> local-envid "@" fqhn, should we extend this (I claim yes)?  If so, to
eric> what value?

Given that RFC 2822 section 4.5.3.1 states that the maximum length of he
domain portion is 255 characters, I would assume 256 (255 + 1 for "@")
would be a good number to add.

---

From: Eric Allman <[email-masked]>
Date: Mon, 15 Oct 2001 12:14:37 -0700

: eric> This brings up another point however.  RFC 1891 specifies the ENVID
: eric> value as being 100 characters.  Since we have changed this to
: eric> local-envid "@" fqhn, should we extend this (I claim yes)?  If so, to
: eric> what value?
: 
: Given that RFC 2822 section 4.5.3.1 states that the maximum length of he
: domain portion is 255 characters, I would assume 256 (255 + 1 for "@")
: would be a good number to add.

I was starting to make this change and suddenly realized that we may
have a conundrum here.  The obvious approach is to increase the length
of the ENVID parameter from 100 to 356.  However, how would I implement
this when initially submitting the message?  The obvious implementation
(if the submitter supports MSGTRK) would be to always create a fully
qualified ENVID.  But if that message gets relayed to a non-MSGTRK
agent, that ENVID could be too long, and so you've created a
non-compliant envelope.

I'm starting to lean toward leaving it at 100 characters for this
reason, although that could be problematic (disastrous?) if IDN goes
for ACE encoding, since the expected length of domain names will
go up.

Opinions?

eric

---

From: Gregory Neil Shapiro <[email-masked]>
Date: Mon, 15 Oct 2001 13:51:30 -0700

eric> I was starting to make this change and suddenly realized that we may
eric> have a conundrum here.  The obvious approach is to increase the length
eric> of the ENVID parameter from 100 to 356.  However, how would I implement
eric> this when initially submitting the message?  The obvious implementation
eric> (if the submitter supports MSGTRK) would be to always create a fully
eric> qualified ENVID.  But if that message gets relayed to a non-MSGTRK
eric> agent, that ENVID could be too long, and so you've created a
eric> non-compliant envelope.

eric> I'm starting to lean toward leaving it at 100 characters for this
eric> reason, although that could be problematic (disastrous?) if IDN goes
eric> for ACE encoding, since the expected length of domain names will
eric> go up.

eric> Opinions?

Yes, but it probably won't be liked.  This shows the danger of reusing an
existing protocol element for a new purpose.  I guess MsgTrk will have to
be limited.  C'est la vie.

---

From: "Vaudreuil, Greg M (Greg)" <[email-masked]>
Date: Mon, 15 Oct 2001 15:36:52 -0600

Why would ACE not cause this same problem with other uses of envid?  It
seems that this is a more general problem.   If we need to grow the
envelope-iD for ACE, we should enlarge it in the DSN specs as well.  

Lurkin' for too long.

Greg V.

-----Original Message-----
From: Gregory Neil Shapiro [mailto:[email-masked]]
Sent: Monday, October 15, 2001 3:52 PM
To: Eric Allman
Cc: [email-masked]
Subject: Re: Late comments on the msgtrk documents 

eric> I was starting to make this change and suddenly realized that we may
eric> have a conundrum here.  The obvious approach is to increase the length
eric> of the ENVID parameter from 100 to 356.  However, how would I
implement
eric> this when initially submitting the message?  The obvious
implementation
eric> (if the submitter supports MSGTRK) would be to always create a fully
eric> qualified ENVID.  But if that message gets relayed to a non-MSGTRK
eric> agent, that ENVID could be too long, and so you've created a
eric> non-compliant envelope.

eric> Opinions?

---

From: Gregory Neil Shapiro <[email-masked]>
Date: Mon, 15 Oct 2001 14:45:43 -0700

gregv> Why would ACE not cause this same problem with other uses of envid?  It
gregv> seems that this is a more general problem.   If we need to grow the
gregv> envelope-iD for ACE, we should enlarge it in the DSN specs as well.  

Envelope-ID pre-message track was immune to IDN as it didn't include a
domain name.  Actually, it doesn't in message track either, it's just
suggested that it can be made unique by adding an FQDN.  However, other
things in DSNs (and the protocol) will need to change if IDN gives us
longer hostnames.

gregv> Lurkin' for too long.

Welcome to the madness.

---

From: Eric Allman <[email-masked]>
Date: Mon, 15 Oct 2001 15:04:44 -0700

The current DSN standards say that ENVID doesn't have to have an
@domain part on it at all -- it only has to be unique to the sending
system.  We are trying to use it to be unique across all messages
that a server might see -- essentially an envelope-level Message-Id.
So I think we're clear for DSNs, but we have a headache for MSGTRK.

Another approach would be to define yet another envelope tag,
e.g., ENVIDprime, that would only be used for MSGTRK.  Seems kind
of ugly to me, but it does avoid this problem.

eric

============= In Reply To: ===========================================
: From:  "Vaudreuil, Greg M (Greg)" <[email-masked]>
: Subject:  RE: Late comments on the msgtrk documents 
: Date:  Mon, 15 Oct 2001 15:36:52 -0600

: 
: 
: Why would ACE not cause this same problem with other uses of envid?  It
: seems that this is a more general problem.   If we need to grow the
: envelope-iD for ACE, we should enlarge it in the DSN specs as well.  
: 
: Lurkin' for too long.
: 
: Greg V.
: 
: 
: -----Original Message-----
: From: Gregory Neil Shapiro [mailto:[email-masked]]
: Sent: Monday, October 15, 2001 3:52 PM
: To: Eric Allman
: Cc: [email-masked]
: Subject: Re: Late comments on the msgtrk documents 
: 
: 
: 
: eric> I was starting to make this change and suddenly realized that we may
: eric> have a conundrum here.  The obvious approach is to increase the length
: eric> of the ENVID parameter from 100 to 356.  However, how would I
: implement
: eric> this when initially submitting the message?  The obvious
: implementation
: eric> (if the submitter supports MSGTRK) would be to always create a fully
: eric> qualified ENVID.  But if that message gets relayed to a non-MSGTRK
: eric> agent, that ENVID could be too long, and so you've created a
: eric> non-compliant envelope.
: 
: eric> I'm starting to lean toward leaving it at 100 characters for this
: eric> reason, although that could be problematic (disastrous?) if IDN goes
: eric> for ACE encoding, since the expected length of domain names will
: eric> go up.
: 
: eric> Opinions?
: 
: Yes, but it probably won't be liked.  This shows the danger of reusing an
: existing protocol element for a new purpose.  I guess MsgTrk will have to
: be limited.  C'est la vie.

---

From: Eric Allman <[email-masked]>
Date: Mon, 15 Oct 2001 15:09:25 -0700

: Envelope-ID pre-message track was immune to IDN as it didn't include a
: domain name.  Actually, it doesn't in message track either, it's just
: suggested that it can be made unique by adding an FQDN.  However, other
: things in DSNs (and the protocol) will need to change if IDN gives us
: longer hostnames.

Actually, I think you'll find that draft-ietf-msgtrk-smtpext-03.txt
defines ENVID as including a FQHN (see section 4.2).

eric

---

From: Gregory Neil Shapiro <[email-masked]>
Date: Mon, 15 Oct 2001 15:14:43 -0700

gshapiro> Envelope-ID pre-message track was immune to IDN as it didn't
gshapiro> include a domain name.  Actually, it doesn't in message track
gshapiro> either, it's just suggested that it can be made unique by adding
gshapiro> an FQDN.

eric> Actually, I think you'll find that draft-ietf-msgtrk-smtpext-03.txt
eric> defines ENVID as including a FQHN (see section 4.2).

The text I am referring to in 4.2 is

          envid-parameter = "ENVID=" unique-envid
          unique-envid    = local-envid "@" fqhn
          local-envid     = xtext
          fqhn            = xtext
...
                                     In most cases, this means setting
      fqhn to be the fully qualified host name of the system  generat-
      ing  this  ENVID
...

"In most cases" leaves the door open for it not being required.  The
document never says what "fqhn" must be.  By my reading of the document, an
ENVID of "123@6425ChristieAvenueSuite4000EmeryvilleCA94608USA" is perfectly
legal as long as I can guarantee it's uniqueness.

---

From: Eric Allman <[email-masked]>
Date: Mon, 15 Oct 2001 15:28:58 -0700

Good point, and (as we discussed) in some cases you _can't_ use
your host name, as for example if you get a dynamic name from an
ISP (since it can be re-used you can't guarantee uniqueness of the
LHS).

eric

============= In Reply To: ===========================================
: From:  Gregory Neil Shapiro <[email-masked]>
: Subject:  Re: Late comments on the msgtrk documents 
: Date:  Mon, 15 Oct 2001 15:14:43 -0700

: The text I am referring to in 4.2 is
: 
:           envid-parameter = "ENVID=" unique-envid
:           unique-envid    = local-envid "@" fqhn
:           local-envid     = xtext
:           fqhn            = xtext
: ...
:                                      In most cases, this means setting
:       fqhn to be the fully qualified host name of the system  generat-
:       ing  this  ENVID
: ...
: 
: "In most cases" leaves the door open for it not being required.  The
: document never says what "fqhn" must be.  By my reading of the document, an
: ENVID of "123@6425ChristieAvenueSuite4000EmeryvilleCA94608USA" is perfectly
: legal as long as I can guarantee it's uniqueness.