Subject: PGP/MIME implementors: text mode vs. binary mode?

From: Thomas Roessler <[email-masked]>
Date: Tue, 13 Feb 2001 13:54:27 +0100

Please forward this message to any implementors of PGP/MIME you are
aware of.

To whom it may concern.

In the ietf-openpgp working group, we are about to finish a revised
version of RFC 2015.  While this looked like an easy task at first
sight, it is not, since OpenPGP breaks traditional pgp's text-mode
signatures.  This message is to solicit further input and comments
from other implementors of PGP/MIME.  (We thought we were done, but
then there were more arguments against the solution we believed to
have found...)

The current draft is draft-ietf-openpgp-mime-04.txt, and available
from: <http://www.ietf.org/internet-drafts/draft-ietf-openpgp-mime-04.txt>.

Please direct any answers to this message to the ietf-openpgp
mailing list.  In order to subscribe to that list, send a message to
<[email-masked]> with the single word "subscribe" in
the subject.

The Problem.

To pgp 2 and 5, a text-mode signature meant that line endings were
transformed to a CR-LF sequence, and the result was then hashed.
However, with OpenPGP, preparing a text-mode signature has another
effect: Trailing blanks (and tabs?) are stripped - this is what used
to be done as a clear-sign signature with traditional PGP.  Of
course, this is a bug in the spec, but one which won't be easily
fixed now that all the implementations are out there.

Now, for PGP/MIME, this turns into a really ugly problem: RFC 2015
does not specify which kind of signature to use. However, it does
specify how the signed material should be canonicalized before
hashing.  This method of canonicalization exactly matched what was
done by traditional PGP versions for text mode signatures, which
means that the very same hash value could be used to verify either a
text-mode or a binary-mode signature.  That way, PGP/MIME according
to RFC 2015 and with traditional PGP had the desired one-pass
properties.

With OpenPGP, this construction breaks badly as soon as trailing
white space is involved.  Since it doesn't look right or elegant to
ask implementors of one-pass parsers (if they exist) to calculate
two hashes in parallel, some decision has to be made on the kind of
signature which should be mandated for the future.

I'm listing the choices and the pros and cons I'm seeing:

* text-mode signatures

  + easy to implement if you invoke a command-line tool to which you
    can leave the details of canonicalization.
  - Ian Bell of turnpike.com tells me that it's hard to persuade the
    PGP SDK to produce detached text-mode signatures.
  + the new-style text-mode signatures will nicely ignore any
    whitespace changes which are irrelevant to the message's content
    (When used the right way, MIME is invariant under modifications of
    trailing whitespace.)
  - there are incompatibilities between implementations which use
    text-mode AND leave trailing white space in messages.
  - thus, clients will need additional code in order to avoid
    trailing whitespace (e.g., apply quoted-printable).
  - this will make any clients non-compliant which are using binary
    mode today.

* binary-mode signatures

  + easy to implement if you use the PGP SDK (I'm told).
  + clients are interoperable regardless of the back-end version
    used and regardless of the treatment of trailing whitespace.
  - those people who rely on "pgp -t" will have to add some code to
    pass canonicalized data to the back-end.  In some cases, this
    may badly break things.
  - signatures will break upon changes to trailing white space which
    don't affect the message's content or MIME semantics
  - thus, clients may need additional code in order to avoid
    trailing whitespace (e.g., apply quoted-printable).
  - this will make any clients non-compliant which are using text
    mode today.

Bottom line: The decision what to mandate boils down to the question
what implementors prefer.  And, yes, both alternatives are bad
solutions.

Thus, if you are an implementor of PGP/MIME, please subscribe to the
ietf-openpgp mailing list, and let us know what should be done in
your opinion, and why.  Also, please tell us what mode you are using
nowadays, so we get a more precise impression of what software we
are going to break.

Thanks, and kind regards,
-- 
Thomas Roessler			    <[email-masked]>

---

From: Bodo Moeller <[email-masked]>
Date: Tue, 13 Feb 2001 15:22:33 +0100

On Tue, Feb 13, 2001 at 01:54:27PM +0100, Thomas Roessler wrote:

[...]
> * text-mode signatures
[...]
>     trailing whitespace (e.g., apply quoted-printable).

There is no need to use quoted-printable to avoid trailing whitespace
in this case.  Applications that use text-mode signatures because they
consider trailing whitespace not significant can simply delete such
whitespace.  (Only in cases where you are worried about unauthorized
removal of whitespace but not about unauthorized addition of whitespace,
quoted-printable is required; e.g. "-- " signature separators.)

>   - this will make any clients non-compliant which are using binary

This is true only if text-mode signatures are made mandatory.  An
alternative is to allow both text-mode and binary signatures, but to
impose restrictions on the data to be signed so that the respective
hashes coincide -- i.e., disallow trailing whitespace unless encoded
such that it is no longer trailing whitespace as far as OpenPGP is
concerned.

> * binary-mode signatures
[...]
>     used and regardless of the treatment of trailing whitespace.

The same is true if text-mode signatures are used and senders strictly
avoid trailing whitespace.

[...]
>   - this will make any clients non-compliant which are using text

Again, this is only true only if binary-mode signatures are made
mandatory.  If both forms are legal, with the restriction the senders
have to avoid trailing unencoded whitespace (but recipients are not
required to strip any trailing whitespace before interpreting the
message), then it is up to the senders to decide if they want to use
binary-mode signatures as a countermeasure against addition of
whitespace in transit or if they think that text-mode signatures
suffice; and clients will still be able to verify signatures in a
single pass.

-- 
Bodo Möller <[email-masked]>
PGP http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/moeller/0x36d2c658.html
* TU Darmstadt, Theoretische Informatik, Alexanderstr. 10, D-64283 Darmstadt
* Tel. +49-6151-16-6628, Fax +49-6151-16-6036

---

From: Thomas Roessler <[email-masked]>
Date: Tue, 13 Feb 2001 16:58:37 +0100

On 2001-02-13 15:22:33 +0100, Bodo Moeller wrote:

> mandatory.  If both forms are legal, with the restriction the
> senders have to avoid trailing unencoded whitespace (but
> recipients are not required to strip any trailing whitespace
> before interpreting the message), then it is up to the senders to
> decide if they want to use binary-mode signatures as a
> countermeasure against addition of whitespace in transit or if
> they think that text-mode signatures suffice; and clients will
> still be able to verify signatures in a single pass.

This suggestion indeed sounds quite reasonable.  Thanks!

Actually, thinking a bit more about this, I think I found one case
in which trailing white space can make a semantic difference. (Which
would be another argument for mandating binary mode, however.)

More precisely, let's have a look at RFC822's grammar:

     field       =  field-name ":" [ field-body ] CRLF
     field-body  =  field-body-contents
     		    [CRLF LWSP-char field-body]
     field-body-contents = <the ASCII characters making up the
		    field-body, as defined in the following
		    sections, and consisting of combinations of
		    atom, quoted-string, and specials tokens, or
		    else consisting of texts>

field-body-contents can, in particular, consist entirely of
whitespace.

Example:

	^some-tag: some text$
	^    $
	^ some more text$
	^next-tag: ...$
	^$
	^body$

(^ marks the beginning of the line, $ the end of the line).

I.e., within an RFC822 header (which can in turn be any MIME header,
or a message header within a message/rfc822 body part), a line which
entirely consists of whitespace will be folded, and will - in
paritcular - _not_ separate the header from the body.  This
semantics will, of course, change if the MUA removes trailing white
space before the message is sent.  To make things still worse, we
can't even call quoted-printable or base64 to the rescue, since
nested encodings are explicitly prohibited.

What do you think - should we really worry about this, or should we
just put a warning into the Security Considerations section of the
text, pointing out that this case can happen and that
implementations should avoid it when sending messages, which is what
I'd suggest?

---

From: Bram Cohen <[email-masked]>
Date: Tue, 13 Feb 2001 11:57:46 -0800 (PST)

On Tue, 13 Feb 2001, Thomas Roessler wrote:

> signatures.

Good thing pgp is standardized, otherwise we might wind up with
incompatible implementations.

-Bram Cohen

"Markets can remain irrational longer than you can remain solvent"
                                        -- John Maynard Keynes

---

From: Derek Atkins <[email-masked]>
Date: 13 Feb 2001 14:08:36 -0500

> incompatible implementations.

The wonderful thing about standards... There are so many to choose from!
;)

-derek
-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [email-masked]                        PGP key available