Subject: Re: PGP/MIME implementors: text mode vs. binary mode?

From: [email-masked]
Date: Tue, 13 Feb 2001 11:07:30 -0800

There is something I'm still missing.

The signature block at the bottom of the PGP/MIME signed message has
a signature type byte in it, text vs binary.  Is it your assumption
that the value in this type byte should control the hashing of the
textual data which is earlier in the message?

I don't know if this has to be true, but it may be helpful for
implementations if it is true.

In that case there are two possibilities.  One is to mandate the value
in this type byte, and thereby mandate the hashing which is done in the
message.  Receivers would hash the message data according to the spec,
and then when they came to the type byte, they could either ignore it
or check it and complain if it is the wrong value.

The second possibility is to allow either value in this type byte, and
thereby require that the receiver read the signature before it goes back
and hashes the data (or else hash the data both ways).

Are these the alternatives as you see them?

Hal

---

From: Thomas Roessler <[email-masked]>
Date: Tue, 13 Feb 2001 20:51:55 +0100

On 2001-02-13 11:07:30 -0800, [email-masked] wrote:

> The signature block at the bottom of the PGP/MIME signed message
> has a signature type byte in it, text vs binary.  Is it your
> assumption that the value in this type byte should control the
> hashing of the textual data which is earlier in the message?

Yes, of course.

> implementations if it is true.

If I sign a document in binary-mode, I want to be informed if
someone changes white space, or if line ends change. That is, in my
understanding, the type byte in the signature must mandate the kind
of hashing to be done - otherwise, it wouldn't terribly useful.

> In that case there are two possibilities.  One is to mandate the
> value in this type byte, and thereby mandate the hashing which is
> done in the message.  Receivers would hash the message data
> according to the spec, and then when they came to the type byte,
> they could either ignore it or check it and complain if it is the
> wrong value.

Right.

> The second possibility is to allow either value in this type
> byte, and thereby require that the receiver read the signature
> before it goes back and hashes the data (or else hash the data
> both ways).

> Are these the alternatives as you see them?

Basically, yes.  Hashing the data in different ways doesn't look
very elegant, and going back to the data is not an option, either
(at least in theory).

In order to avoid these two approaches, we either have to mandate
the kind hash to be taken, or we have to make sure that both ways of
taking hashes are identical on the data to be considered.

Symbolically: Take the message m, and let || denote concatenation.
In this case, a hash used in a binary signature is just this:

(b)	hash-function (m || 
		some-properties (signature packet)),

A hash used in a text signature, however, is:

(t)	hash-function (strip-whitespace-and-canonicalize (m) ||
		some-properties (signature packet))

My approach was to mandate use of either (b) or (t). Bodo's
(better!) approach is that m is invariant under
strip-whitespace-and-canonicalize, i.e.,

	m = strip-whitespace-and-canonicalize (m),

by choosing m appropriately; this is the same basic approach which
is also used in RFC 2015.

(However, there it's not strip-whitespace-and-canonicalize, but just
canonicalize.  Also, the same canonicalization as everywhere else in
MIME world can be applied, which makes RFC 2015 more elegant than
its successor.)

Finally, since hash-function (a, b) can also be written in the form

	hash-function' (hash-function'' (a), b),

implementations can just calculate hash-function'' (m), and put in
some-properties (signature packet) when they encounter the
signature.

-- 
Thomas Roessler			    <[email-masked]>

---

From: Derek Atkins <[email-masked]>
Date: 13 Feb 2001 15:14:28 -0500

Hal,

The point is that when PGP/MIME was created with PGP 2.x (RFC1991),
the PGP/MIME canonicalization was such that it didn't matter which
type-byte you used; the hash would be the same.  The problem is that
OpenPGP (RFC2440) changes the definition of a text signature (please
excuse me for not quoting the exact type-byte value).

Because of this change, it is possible to create a text-signature with
OpenPGP that will fail with PGP 2.x (and vise-versa).  The major
difference is that RFC2440 specifies that you do not hash trailing
white-space (whereas in RFC1991 trailing white space was included in
the hash).

So here is the problem.  PGP/MIME specifies text canonicalization to
make binary and text signatures hash into the same value for RFC1991
signatures.  However, this is not the same for RFC2440 signatures.
The problem is that implementations want to be able to hash the
message in a single-pass, before they read the signature type-byte
from the signature.  Moreover, implementations want to hash the
message only once.

So, how do we maintain compatibility of PGP/MIME between RFC1991 and
RFC2440 versions of PGP, while maintaining the pre-hash, single-pass
processing?  As have been argued, there are a few ways to do this:

	a) specify a canonicalization and require binary sigs.
	   Unfortunately this invalidates a number of implementations
	   that currently use text sigs.

	b) specify a canonicalization that matches text and allow
	   either binary or text sigs.  Unfortunately this has the
	   problem that RFC1991 and RFC2440 have different ideas of
	   what should be included in a text-mode signature.

	c) Change the PGP/MIME canonicalization requirements to match
	   RFC2440 text-mode.  This has the effect that previous
	   messages (and probably many implementations) wont be
	   PGP/MIME compliant.

I don't know what the best way is.

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [email-masked]                        PGP key available

---

From: Thomas Roessler <[email-masked]>
Date: Tue, 13 Feb 2001 22:08:09 +0100

On 2001-02-13 15:14:28 -0500, Derek Atkins wrote:

>	   Unfortunately this invalidates a number of
>	   implementations that currently use text sigs.

>	   problem that RFC1991 and RFC2440 have different ideas
>	   of what should be included in a text-mode signature.

> 	c) Change the PGP/MIME canonicalization requirements to
>	   match RFC2440 text-mode.  This has the effect that
>	   previous messages (and probably many implementations)
>	   wont be PGP/MIME compliant.

What precisely is the difference between b) and c) supposed to be? I
seem to be missing some point here.

Anyway: When text and binary mode hashes (as I'll call them by abuse
of language) calculated according to RFC 2440 conincide, the text
hash from RFC 1991 coincides, too.  Additionally, protecting
trailing white space by way of an appropriate encoding has been a
good idea in the past, and was suggested and demonstrated in one of
the example messages in RFC 2015.

For this reason, we may actually be in the lucky situation that
implementations already fulfill most of the stricter requirements
suggested.

Maybe some implementors can say a word about this?

(To contribute my own part, mutt should be fine with it, and I'm
certainly willing to fix any holes through which trailing whitespace
might creep.)

---

From: Derek Atkins <[email-masked]>
Date: 13 Feb 2001 16:48:35 -0500

> seem to be missing some point here.

Sorry, the difference between b and c is that in b I implied that
PGP/MIME use the RFC1991 canonicalization scheme, whereas in c I imply
using the RFC2440 canonicalization scheme.  I'm sorry I wasn't more
clear.

-derek

---

From: Florian Weimer <[email-masked]>
Date: 15 Feb 2001 12:05:59 +0100

> > hashing of the textual data which is earlier in the message?

And thus breaking one-pass-processing, one of the fundamental design
goals of MIME? ;-)

You have to put this information into the micalg parameter, I think.

-- 
Florian Weimer 	                  [email-masked]
University of Stuttgart           http://cert.uni-stuttgart.de/
RUS-CERT                          +49-711-685-5973/fax +49-711-685-5898

---

From: Werner Koch <[email-masked]>
Date: Thu, 15 Feb 2001 13:30:39 +0100

> And thus breaking one-pass-processing, one of the fundamental design

It won't work anyway:

|   The "micalg" parameter for the "application/pgp-signature" protocol
|   MUST contain exactly one hash-symbol of the format "pgp-<hash-
                 ^^^^^^^^^^^
This does not allow to have more than 1 signature in the second part
with different hash algorithms.  I know that this is addressed in
the multisig draft but it would be nice to have 2 signatures in this
protocol.

So why shouldn't we allow a wildcard or a list.  The only drawback
is that one has to apply multiple hash algorithms and select later
which to use.

-- 
Werner Koch                                              <[email-masked]>
GNU Privacy Guard                                (http://www.gnupg.org)
Free Software Foundation Europe              (http://www.fsfeurope.org)
           [Please see X-* mail header for OpenPGP key info]

---

From: Thomas Roessler <[email-masked]>
Date: Thu, 15 Feb 2001 18:23:21 +0100

On 2001-02-15 12:05:59 +0100, Florian Weimer wrote:

>>> The signature block at the bottom of the PGP/MIME signed
>>> message has a signature type byte in it, text vs binary.  Is
>>> it your assumption that the value in this type byte should
>>> control the hashing of the textual data which is earlier in
>>> the message?

> And thus breaking one-pass-processing, one of the fundamental
> design goals of MIME? ;-)

Not really. ,-)

> You have to put this information into the micalg parameter, I

By mandating that the two hashes coincide on all allowed data, we
can circumvent this.