Subject: Re: [xml2rfc] Umlauts in reference author names

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 01 Jul 2016 16:24:16 +0200

Hi Tony,

On 2016-07-01 05:32, HANSEN, TONY L wrote:
> Bibxml3 is created from 1id-index.txt and 1id-abstract.txt. Those
> files somehow got an umlaut in that field.
> 
> Henrik?

Hmm.  Yes.  The code that produces the 1id-* files uses a method which
wasn't intended to guarantee an ascii name, and this method changed a
few datatracker releases ago.  It was also using the name entry in the
datatracker person record as-is, which is bound to contain non-ascii for
anybody who have a name with non-ascii characters.

I've made some fixes, and will release them with the next patch release.

We'll need to change this again when non-ascii names _should_ be supported.

--------

Now, going off on a tangent from the bug above, and its fix.

FWIW, as test I made xml2rfc able to process unicode some time ago;
if anybody wants to try it, you could apply this patch to xml2rfc:

Index: xml2rfc/writers/raw_txt.py
===================================================================
--- xml2rfc/writers/raw_txt.py  (revision 2054)
+++ xml2rfc/writers/raw_txt.py  (working copy)
@@ -36,7 +36,7 @@
         self.list_counters = {} # Maintain counters for 'format' type lists
         self.edit_counter = 0   # Counter for edit marks
         # Set this to False to permit utf+8 output:
-        self.ascii = True       # Enable ascii flag
+        self.ascii = False      # Disable ascii flag
         self.cref_counter = 0   # Counter for cref anchors
         self.cref_list = []

Best regards,

	Henrik

> 
> 	Tony
> 
> On 6/29/16, 1:05 PM, "xml2rfc on behalf of Julian Reschke" <[email-masked] on behalf of [email-masked]> wrote:
> 
> On 2016-06-29 18:11, Ari Keränen wrote:
>>
>>> On 29 Jun 2016, at 19:00, Julian Reschke <[email-masked]> wrote:
>>>
>>> On 2016-06-29 16:54, Ari Keränen wrote:
>>>> Hi,
>>>>
>>>> Xml2rfc seems to get confused now with names with umlauts (like my last name) in references.
>>>>
>>>> For example,in draft-ietf-hip-native-nat-traversal-12 it says:
>>>>>   [I-D.ietf-ice-rfc5245bis]
>>>>>              Ker&#52292;nen, A., Holmberg, C., and J. Rosenberg,
>>>>
>>>> Any ideas how to fix that?
>>>
>>> For now, xml2rfc lives in a plain ASCII world. You'll have to substitute the "ä" with "ae" or "a" (depends on your locale, I guess).
>>
>> Actually, that has been the case for all references up until very recently (it's been "Keranen" in all drafts and RFCs). I wonder if the bibxml files, or how they are handled, have been changed recently?
> 
> That is indeed strange.
> 
> The ID being referenced is US-ASCII only, but
> 
>  
> http://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.draft-ietf-ice-rfc5245bis-02.xml
> 
> has the "ä" in the contact information. Where does this come from?
> 
> Best regards, Julian
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
> 
>