Subject: [Tools-discuss] non-ASCII characters in list archives

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 18 Oct 2013 11:06:11 -0600

The mailman archives don't show non-ASCII characters correctly.

For example, compare the following archives of the same message:

http://www.ietf.org/mail-archive/web/precis/current/msg00543.html

vs.

http://mail.jabber.org/pipermail/standards/2013-October/028099.html

Both systems (ietf.org and jabber.org) are running mailman, so
presumably there's a configuration difference here. I'm happy to help debug.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Adam Roach <[email-masked]>
Date: Fri, 18 Oct 2013 12:31:03 -0500

> presumably there's a configuration difference here. I'm happy to help debug.

Interestingly, the jabber.org HTTP server serves this up with a content 
type of "text/html; charset=iso-8859-1", while the ietf.org site simply 
says "text/html"

There's also been some transformation here, as the raw source from 
jabber reads: " in my example, a nickname of 
&quot;&#934;&#921;&#923;&#927;&#931; &#924;&#927;&#921;&quot;" where the 
ietf version reads " in my example, a nickname of 
&quot;&#xCE;&#xCE;&#xCE;&#xCE;&#xCE; &#xCE;&#xCE;&#xCE;&quot;"

Now, If I read things correctly, it looks like Peter's email client sent 
this message with a content type of "text/plain," but no indication of 
character encoding. I think it's debatable what a recipient should do 
under these circumstances.

 From where I'm sitting, it looks like the IETF and Jabber servers made 
different arbitrary guesses about which character encoding Peter was 
trying to use, and the Jabber server got lucky. I'm willing to put large 
money on the proposition that I could form an equally ambiguous email 
that would archive "correctly" on the IETF site and be corrupted beyond 
use on the Jabber site. I'm almost as confident that I could find an 
email client that does this by default (or, at least, in a commonly used 
configuration).

/a

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 18 Oct 2013 12:03:12 -0600

> email client that does this by default (or, at least, in a commonly used

Most probably. I'll test further.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 18 Oct 2013 20:08:42 +0200

>type of "text/html; charset=iso-8859-1", while the ietf.org site simply 

It seems the HTML content also does not declare a character encoding
like through a `<meta>` element. That is or at least used to be a XSS
vulnerability because you could make some browsers interpret the data
as UTF-7-encoded data and thereby smuggle in `<script>` elements.

>under these circumstances.

No, the mail above, [email-masked] declares UTF-8 in the
Content-Type header (I have that mail locally, mbox files are on the
ftp.ietf.org server to verify).
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 18 Oct 2013 13:11:10 -0600

> as UTF-7-encoded data and thereby smuggle in `<script>` elements.

Thanks for the tip. I'll check into the configuration there.

> ftp.ietf.org server to verify).

Right, it would be nice to preserve the UTF-8 encoded text at the archive.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/