Subject: [openpgp] A way to securely define cleartext signature charset

From: Andre Heinecke <[email-masked]>
Date: Fri, 07 Sep 2018 15:52:43 +0200

Hi,

today I struggled for several hours with "charset guessing" code, that handles 
cleartext signatures in outlook and I thought that maybe this situation could 
be improved a bit in the future?

I dislike cleartext signatures as much as the next guy (probably more ;-) ).
The points made in [1] are valid and such messages should not be used.
But realistically I think that they won't go away.

My idea would be to define that after the Hash: header and the blank line 
(which starts the hashing) that there can be:

Optionally a "Charset" Armor Header followed by one blank line,
both included in the message digest.

So a message like:

Charset: UTF-8

This is än example mässäge.

An rfc4880 implementation would just show:
----
Charset: UTF-8

This is än example mässäge.
----

Ok that is slightly ugly but it's informative and the signature will still be 
verified correctly.
An rfc4880bis application could evaluate the header and omit it in the output.

Attached is a patch to the draft.

Best Regards,
Andre

1: https://dkg.fifthhorseman.net/notes/inline-pgp-harmful/
-- 
Andre Heinecke |  ++49-541-335083-262  | http://www.intevation.de/
Intevation GmbH, Neuer Graben 17, 49074 Osnabrück | AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

From=208ac7a240edeb28229cb5afe722484462703f9b4e Mon Sep 17 00:00:00 2001
From: Andre Heinecke <[email-masked]>
Date: Fri, 7 Sep 2018 15:44:33 +0200
Subject: [PATCH] Add optional charset specification

---
 middle.mkd | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/middle.mkd b/middle.mkd
index 9e5bb58..1347aea 100644
--- a/middle.mkd
+++ b/middle.mkd
@@ -3074,6 +3074,9 @@ The cleartext signed message consists of:
 
   - Exactly one blank line not included into the message digest,
 
+  - Optionally a "Charset" Armor Header followed by one blank line,
+    both included in the message digest,
+
   - The dash-escaped cleartext that is included into the message
     digest,
 
@@ -3090,6 +3093,15 @@ contains a comma-delimited list of used message digests.
 Current message digest names are described below with the algorithm
 IDs.
 
+If the "Charset" Armor Header is given, the specified message is
+to be expected in that encoding.  If the Header is omitted an
+implementation SHOULD try to guess the charset from the context
+of the message.  For example by using the systems local 8 bit
+charset or checking for the content-type in a MIME context.
+
+An implementation MAY assume UTF-8 encoding in case the "Charset"
+Header is omitted.
+
 An implementation SHOULD add a line break after the cleartext, but MAY
 omit it if the cleartext ends with a line break.  This is for visual
 clarity.
-- 
2.11.0

---

From: Peter Pentchev <[email-masked]>
Date: Sat, 8 Sep 2018 14:19:53 +0300

On Fri, Sep 07, 2018 at 03:52:43PM +0200, Andre Heinecke wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Charset: UTF-8
> 
> -----BEGIN PGP SIGNATURE-----

Hmm, is there any way to guard against a false positive identification of
an "old" message that just happens to start with such a line?  I can't
think of any off the top of my head...

Don't get me wrong, I *do* see the good things about your proposal.

Best regards,
Peter

-- 
Peter Pentchev  roam@{ringlet.net,debian.org,FreeBSD.org} [email-masked]
PGP key:        http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13

---

From: Marcus Brinkmann <[email-masked]>
Date: Sat, 8 Sep 2018 16:43:25 +0200

Why not a hashed signature subpacket?

On 09/07/2018 03:52 PM, Andre Heinecke wrote:> Hi,
> _______________________________________________
> openpgp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/openpgp
>

---

From: Andre Heinecke <[email-masked]>
Date: Sat, 08 Sep 2018 20:00:44 +0200

Hi,

> think of any off the top of my head...

I do not think so. Well you could put additional information in the signature 
that will identify it as a cleartext signature following rfc4880bis and only 
then handle the charset header. But I think that would overcomplicate it.

I do not think that a false positivie would not hurt much. PGP Inline charset 
handling is basically guessing so a false positive would just be a false 
guess.

And I think that if someone today signs a message that says

Charset: XYZ

And then continues with some text in another charset it would be weird anyway. 

> Don't get me wrong, I *do* see the good things about your proposal.

Thanks! 

Best Regards,
Andre

---

From: Andre Heinecke <[email-masked]>
Date: Sat, 08 Sep 2018 20:27:29 +0200

Hi,

> Why not a hashed signature subpacket?

Mostly because in an Application you can already use the information from the 
header before you do any OpenPGP parsing / signature verification.

E.g. in a MUA you usually want to show the data while you are verifying the 
signature. A charset Header could be easily parsed by a MUA and taken as a 
suggestion how to present the data.

There might also be the case where you know the charset was changed in 
transfer and you have to convert the charset back to get the correct 
bytestream that matches the signature before passing it to your OpenPGP 
backend.

That is not to say that I'm totally against a subpacket, if the correct 
charset would be known after verification / parsing it would also help.

Best Regards,
Andre

---

From: Werner Koch <[email-masked]>
Date: Tue, 11 Sep 2018 11:53:54 +0200

> header before you do any OpenPGP parsing / signature verification.

Verification tools already need to consider an unsigned armor header to
figure out the digest algorithm to use.  However, this is sometimes not
enough because some tools used to have peculiar interpretation of white
space and line endings or the "Hash" header line was missing.  Thus, for
one-pass processing running a second hash context was (or well, is)
useful.  Adding a "Charset" header and automatically try to convert
would lead to an even more complex verification step.  I don't think
that is justified.

Better have a way to sign the character set info and present this to the
user in the Good and in the Bad verification case.  On a bad
verification the user can then choose to convert and try a verification
again.  That would not be a one-pass processing anymore but for the ugly
cleartext signatures this seems to be acceptable.

I would thus suggest this new standard notation:

  ##### The 'charset' Notation
  
  The "charset" notation is a description of the character set used to
  encode the signed plaintext.  The default value is "UTF-8".  If used,
  the value MUST be encoded as human readable and MUST be present in the
  hashed subpacket section of the signature.  This notation is useful
  for cleartext signatures in cases where it is not possible to encode
  the text in UTF-8.  By having the used character set a part of the
  signed data, attacks exploiting different representation of code
  points will be mitigated.

Shalom-Salam,

   Werner
  
-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

---

From: Andre Heinecke <[email-masked]>
Date: Tue, 11 Sep 2018 12:14:43 +0200

> would lead to an even more complex verification step.  I don't think

Thinking more from the "backend" standpoint and less from the Application 
using the backend this makes sense to me. A minor issue is that my Application 
might temporarily show the wrong representation before the verification is done 
but I guess that is indeed minor.

> cleartext signatures this seems to be acceptable.

Yes, as for me the "User" would be my Application and not the person sitting 
in front of the Computer I think that is acceptable as it can be handled 
automatically.

>   points will be mitigated.

I like it.

"The default value is "UTF-8"" -> Do I understand this correctly that this 
basically means: If no charset notation is provided a cleartext signature MUST 
be in UTF-8?
That would be great.

Thanks and best regards,
Andre

---

From: Vincent Breitmoser <[email-masked]>
Date: Tue, 11 Sep 2018 12:50:11 +0200

> A minor issue is that my Application might temporarily show the wrong
> representation before the verification is done but I guess that is indeed
> minor.

Cleartext signatures are typically short. Is there a reason for even showing the
text before the signature is verified and the charset known?

 - V

---

From: Andre Heinecke <[email-masked]>
Date: Tue, 11 Sep 2018 13:01:30 +0200

> text before the signature is verified and the charset known?

In GnuPG we have the option "auto-key-retrieve" which I prefer to use. That 
option will fetch an unknown  signing key from remote sources e.g. Keyserver 
or WKD, which can take a while. But as I said, this is a minor issue.

Regards,
Andre

---

From: Werner Koch <[email-masked]>
Date: Tue, 11 Sep 2018 15:46:47 +0200

> basically means: If no charset notation is provided a cleartext signature MUST 

Yes, that is due to 

  ## {3.4} Text
  
  Unless otherwise specified, the character set for text is the UTF-8
  [](#RFC3629) encoding of Unicode [](#ISO10646).

this has not changed since 2440.  But agreed in the beginning the world
was US-centric and this fact was often overlooked. And PGP-2 had no such
definition.

Salam-Shalom,

   Werner

---

From: Jon Callas <[email-masked]>
Date: Fri, 14 Sep 2018 23:50:02 -0700

> On Sep 11, 2018, at 3:50 AM, Vincent Breitmoser <[email-masked]> wrote:
> 

Yes. Cleartext signatures exist so that someone with no PGP can read the message. Remember — DOS and BBSes. When you’re planning a meeting, you don’t want to isolate the people who don’t have enough space on their floppy disk for PGP *and* the BBS software.

	Jon