Subject: [xml2rfc] #161: generated HTML is not really ASCII

From: "xml2rfc issue tracker" <[email-masked]>
Date: Thu, 11 Apr 2013 19:39:42 -0000

#161: generated HTML is not really ASCII

 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii" />

 OK, but then:

         <span class="org vcardline">Universität Bremen TZI</span>

 (This should be an &#xe4; or an &auml;, but certainly not raw UTF-8
 \303\244.)

-- 
---------------------------+----------------------------------
 Reporter:  [email-masked]   |      Owner:  [email-masked]
     Type:  defect         |     Status:  new
 Priority:  medium         |  Milestone:
Component:  Version 2 cli  |    Version:  2.4.x
 Keywords:                 |
---------------------------+----------------------------------

Ticket URL: <http://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/161>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 15 Apr 2013 10:08:40 +0900

On 2013/04/12 4:39, xml2rfc issue tracker wrote:
>   (This should be an&#xe4; or an&auml;, but certainly not raw UTF-8
>   \303\244.)

I agree that "charset=us-ascii" and "Universität" don't work together. 
The content produced is not valid, and we must clearly avoid that. 
However, replacing "ä" by "&#xe4;" seems backwards. While the document 
is then encoded in US-ASCII, it still contains non-ASCII characters. 
Actually, the document could say "charset=utf-8" and still contain no 
non-ASCII characters (escaped or not), and the ASCII-only old-hats 
should not have anything to complain.

The best solution, in my view, would be to change to "charset=utf-8", 
and encode the "ä" as UTF-8. That's my understanding of where we are 
moving to, so whatever we do, we should make it easy to get there even 
if we don't do it yet just now.

If producing data with non-ASCII characters in them (escaped or not) has 
to be avoided, then either produce an error message, or use some 
fallback (not escaping) that's clearly visible, e.g. something like
"UniversitU+00E4t" or so. But in any way, make this easy to switch off, 
so that you don't have to rework everything again when we upgrade to 
using Unicode characters.

Regards,   Martin.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 15 Apr 2013 10:12:05 +0200

The HTML code in XML2RFC is already using character references correctly.

E.g., in the initial page header:

  <td class="right">Universit&#228;t Bremen TZI</td>

It's just broken in the author info case.

I also don't agree at all there is anything wrong about charset=us-ascii and using unicode character references in HTML.
This continues to be the most painless way to ship HTML around, and with the secretariat's configured-for-ISO8859-1 servers, that is probably useful.
(I do agree we should move to UTF-8 everything, but that is a different discussion.)

Grüße, Carsten

On Apr 15, 2013, at 03:08, "Martin J. Dürst" <[email-masked]> wrote:

> I agree that "charset=us-ascii" and "Universität" don't work together. The content produced is not valid, and we must clearly avoid that. However, replacing "ä" by "&#xe4;" seems backwards. While the document is then encoded in US-ASCII, it still contains non-ASCII characters. Actually, the document could say "charset=utf-8" and still contain no non-ASCII characters (escaped or not), and the ASCII-only old-hats should not have anything to complain.
> 
> The best solution, in my view, would be to change to "charset=utf-8", and encode the "ä" as UTF-8. That's my understanding of where we are moving to, so whatever we do, we should make it easy to get there even if we don't do it yet just now.
> 
> If producing data with non-ASCII characters in them (escaped or not) has to be avoided, then either produce an error message, or use some fallback (not escaping) that's clearly visible, e.g. something like
> "UniversitU+00E4t" or so. But in any way, make this easy to switch off, so that you don't have to rework everything again when we upgrade to using Unicode characters.
> 
> Regards,   Martin.
> 
>

---

From: Julian Reschke <[email-masked]>
Date: Mon, 15 Apr 2013 10:44:00 +0200

> It's just broken in the author info case.

(shaking head; different code paths for different elements????)

> This continues to be the most painless way to ship HTML around, and with the secretariat's configured-for-ISO8859-1 servers, that is probably useful.

Serving UTF-8 labeled as UTF-8 in HTML and ISO-8859-1 in HTTP would 
indeed be bad.

> (I do agree we should move to UTF-8 everything, but that is a different discussion.)

+1

Best regards, Julian