Subject: [Tools-discuss] idnits 2.16.02 counts line length using bytes and not characters (but xml2rfc 2.34.0 line-wraps counting chars, not bytes)

From: Daniel Kahn Gillmor <[email-masked]>
Date: Mon, 04 Nov 2019 15:51:54 -0500

i tried running idnits 2.16.02 on a local copy of an intermediate build
of draft-autocrypt-lamps-protected-headers (something between -00 and
-01).

idnits complained that one of my lines was > 72 chars long.

It was correct that the line was > 72 *bytes* long, but because the txt
was using UTF-8 (and not all characters are 7-bit clean), the line itself
was <= 72 characters.

the line in question was:

----
   In the diagrams below, "↧" (DOWNWARDS ARROW FROM BAR, U+21A7) is used
----

This .txt file was generated from xml2rfc 2.34.0-1 (as shipped in debian).

For draft -01 i've worked around it by adjusting the text so that it
reflows to have both the chars and bytes <= 72, so i'm not blocked on
the error, but i wanted to document it here (happy to also report it on
a bugtracker if you prefer, let me know).

My concern is that if xml2rfc is line-wrapping its txt output based on
the number of characters, then idnits should *also* be doing its
line-length-counting on the basis of characters, not the number of
bytes.

So i think the two tools should probably be better aligned if we want to
support non-ASCII characters in the published txt documents.

        --dkg

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 4 Nov 2019 22:07:00 +0100

Hi Daniel,

Agreed.  A rewrite of idnits which will do better in this respect is
in progress.

Best regards,

	Henrik

On 2019-11-04 21:51, Daniel Kahn Gillmor wrote:
> ___________________________________________________________
> Tools-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-discuss
> 
> Please report datatracker.ietf.org and mailarchive.ietf.org
> bugs at http://tools.ietf.org/tools/ietfdb
> or send email to [email-masked]
> 
> Please report tools.ietf.org bugs at
> http://tools.ietf.org/tools/issues
> or send email to [email-masked]
>

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Tue, 05 Nov 2019 07:34:53 -0500

> Agreed.  A rewrite of idnits which will do better in this respect is

great, thanks!

By the way, the incluson of UTF-8 characters has been really helpful for
this specific draft, as it permits pretty diagrams of MIME structure
that would be tedious to explain in text, or harder to follow visually
if we left out edges between parent and child nodes.

I really appreciate the work that goes into modernizing the IETF toolchain
to permit these improvements.

My only regret is that the htmlizer on tools.ietf.org seems to work from
the uploaded .txt variant, rather than building the html directly from
the .xml source.  "xml2rfc --html --v3" produces a document that is
*much* nicer to read (and better structured from an HTML point of view)
than the htmlized txt source.

Is there any plan to generate the HTML from the uploaded xml in this way
for future documents?

   --dkg

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 5 Nov 2019 17:01:37 +0100

Hi Daniel,

> Is there any plan to generate the HTML from the uploaded xml in this way

If you submit v3 XML, HTML is already being generated from the XML.

Right at the moment, it looks like the net (or wider area?) that
datatracker.ietf.org lives on is unreachable (I see this for multiple
machines in the same geographic area) but when it's reachable again,
you should see a link to the generated html on the datatracker page
for the document, and you should also find it next to the .txt and
.xml files in the repository:

  http://www.ietf.org/id/draft-autocrypt-lamps-protected-headers.html

Best regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 5 Nov 2019 17:22:43 +0100

Hi Daniel,

Below:

>   http://www.ietf.org/id/draft-autocrypt-lamps-protected-headers.html

Mmm.  That lacked the necessary version number, so was incorrect.  I also
see that the XML file is not pure v3, so HTML won't be generated by the
submission tool.

If you run the current XML file through 'xml2rfc --v2v3 ...' you should get
a pure v3 file you can use for future work.

Best regards,

	Henrik