Subject: [xml2rfc] XML2RFC PDF problem (was: Re: An IOT DIR review of draft-ietf-anima-autonomic-control-plane)

From: Toerless Eckert <[email-masked]>
Date: Mon, 11 Jun 2018 19:28:35 +0200

[ Moving to [email-masked] as this seems to be a tooling issue,
 Bcc dir/group mailing lists.] 

Thanks, Pascal

I can not find a problem in the xml source i uploaded
https://tools.ietf.org/id/draft-ietf-anima-autonomic-control-plane-16.xml
(but of course other eyes may see more..)

Note that i did not unload any conversions, just the XML to datatracker.

The "normal" html looks fine in the Introduction section:
https://tools.ietf.org/id/draft-ietf-anima-autonomic-control-plane-16.html

I think you are referring to the PDF conversion:
https://tools.ietf.org/pdf/draft-ietf-anima-autonomic-control-plane-16.pdf

Page 6 and 7 are completely messed up in that file, starting maybe as
you say from bullet point 2 in Section 1, Introduction. But for no
clear reason to me.

two Questions to xml2rfc gurus:

a) How can i even try to create this PDF format with hyperlinks ?

I tried all three PDF versions on https://xml2rfc.tools.ietf.org/
the first one creates perfect PDF without hyperlinks, the second one
creates perfect PDF from the perrfect html vrsion (which looks not really
good), and the third one creates some strange crap.

Without being able to create this hyperlink PDF i'd have to troubleshoot
by uploading a lot of versions to datatracker until i figure it out, so
thats not going to happen.

b) Anybody has a suggestion how to fix this formatting error ?

Note that this PDF conversion does seem to be doing some pattern matching:
In the introduction the XML source says:

Section 5 of <xref target="RFC7575"/>

Which that PDF shows as a single hyperlink (the HTML of course not).
Maybe this type of heuristic pattern matching fails somewhere.
But, see question a), without being able to troubleshoot without
creating useless draft versions on datatracker, i have no recourse to
fix.

Will open trac case if nobody has a clue.

Thanks a lot!
    Toerless

On Mon, Jun 11, 2018 at 09:55:29AM +0000, Pascal Thubert (pthubert) wrote:
> Dear Toerless and authors:
> 
> I reviewed -16 and I'm happy with the changes made for my comments. All set on my side.
> I noted that item 2 in the Introduction has a ref that opens a bracket that does not close; this creates a presentation issue on the htmlized version.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 11 Jun 2018 19:29:54 +0200

Thanks, Pascal

Thanks a lot!
    Toerless

-- 
---
[email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2018 20:12:53 +0200

On 2018-06-11 19:29, Toerless Eckert wrote:
> ...
> ...

One approach would be to ignore the PDF version(s).

Another one would be to create the PDF with different tools, such as 
with rfc2629.xslt. Will send example output directly to you.

Best regards, Julian

---

From: Fred Baker <[email-masked]>
Date: Mon, 11 Jun 2018 11:48:35 -0700

> On Jun 11, 2018, at 11:12 AM, Julian Reschke <[email-masked]> wrote:
> 

Agreed. That said, if I understand Toerless correctly, he is asking for the tool he is using to be fixed. His point isn't so much the PDF as the tool.

> Another one would be to create the PDF with different tools, such as with rfc2629.xslt. Will send example output directly to you.
> 
> Best regards, Julian
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 11 Jun 2018 22:21:29 +0200

Thanks Julian

As Fred said, i am primarily worried about the PDF conversion done internally
by datatracker after i upload an XML. If three is a version of this tool
i can use without creating a new draft version, i am happy to troublehoot
myself (like i could do with my prior 'needLines' problem). But so far
i do not know what this datatracker xml->PDF conversion tool is.

Nevertheless, i did like the PDF you send me, do you mind
telling me what toolchain created it ? Googling rfc2629.xslt didn't really
help.

Cheers
    Toerless

On Mon, Jun 11, 2018 at 08:12:53PM +0200, Julian Reschke wrote:
> Another one would be to create the PDF with different tools, such as with
> rfc2629.xslt. Will send example output directly to you.
> 
> Best regards, Julian

-- 
---
[email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2018 22:38:59 +0200

> telling me what toolchain created it ? Googling rfc2629.xslt didn't really

<https://greenbytes.de/tech/webdav/rfc2629xslt/rfc2629xslt.html>.

This generates HTML, which I then convert to PDF using 
<https://www.princexml.com/>.

Best regards, Julian

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 12 Jun 2018 06:35:16 +0000

Hello Toerless:

The HTML is in fact wrong already. What seems to screw things up is the link to section 3.2 below. This link is in fact pointing to an ID ref. Strange. Could you check again the xml at that very place?

Take care,

Pascal

> -----Original Message-----

> From: Toerless Eckert <[email-masked]>

> Sent: lundi 11 juin 2018 22:21

> To: Julian Reschke <[email-masked]>

> Cc: Toerless Eckert <[email-masked]>; Pascal Thubert (pthubert)

> <[email-masked]>; [email-masked]; draft-ietf-anima-autonomic-

> [email-masked]

> Subject: Re: [xml2rfc] XML2RFC PDF problem

> > > Note that this PDF conversion does seem to be doing some pattern

> matching:

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Jun 2018 08:57:25 +0200

On 2018-06-12 08:35, Pascal Thubert (pthubert) wrote:
> Hello Toerless:
> 
> The HTML is in fact wrong already. What seems to screw things up is the 
> link to section 3.2 below. This link is in fact pointing to an ID ref. 
> Strange. Could you check again the xml at that very place?
> 
> Take care,
> 
> Pascal

Indeed. In which case it's a bug in rfcmarkup 
(<https://tools.ietf.org/tools/rfcmarkup/>).

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 12 Jun 2018 12:32:53 +0200

> (<https://tools.ietf.org/tools/rfcmarkup/>).

Right.  I'll have to look at this.

Best regards,

	Henrik

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 12 Jun 2018 13:51:30 +0200

Rfcmarkup looks broken in the same way as the PDF: 
(except of course for the sizing issues in the PDF that seem like
 the result of rfcmarkup not folding the random/broken text
 it did insert)

https://tools.ietf.org/rfcmarkup?doc=draft-ietf-anima-autonomic-control-plane-16

I can not figure out how to apply rfcmarkup to a local file
though (not in the mood to install a webserver to get this to work),
so i can not troubleshoot it myself now.

WOuld be good if xml2rfc would allow to create rfcmarkup'ed
HTML and PDF files so one can locally verify that it will work
on a document. Even above URL is not exactly the desired output
because it includes the Versions header.

Cheers
    Toerless

P.S.: I have exhausted to no avail my mental capabilities staring
at the section in question in my draft, forcing a bug in
the source to come out and present itself to me.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Jun 2018 14:03:32 +0200

> so i can not troubleshoot it myself now.

If the draft is somewhere in svn or github, you could just use that URI.

> because it includes the Versions header.

rfcmarkup is something that won't be needed anymore once the new RFC 
formatting tools are in place.

> the source to come out and present itself to me.

Did you try to add a trailing dot in

>    2.  A controller or network management system can use it to securely
>        bootstrap network devices in remote locations, even if the
>        network in between is not yet configured; no Data-Plane dependent
>        bootstrap configuration is required.  An example of such a secure
>        bootstrap process is described in
>        [I-D.ietf-anima-bootstrapping-keyinfra]

?

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 12 Jun 2018 15:30:13 +0200

Hi Toerless,

> https://tools.ietf.org/rfcmarkup?doc=draft-ietf-anima-autonomic-control-plane-16

I've identified the cause of the problem (in rfcmarkup) and have a
fix under testing.

The problem is that rfcmarkup fails to find the reference
[I-D.ietf-anima-bootstrapping-keyinfra] in the reference section, and
instead uses the paragraph that starts with [I-D.ietf-anima-bootstrapping-keyinfra] in Appendix A.2.

> so i can not troubleshoot it myself now.

I expect to deploy the fixed rfcmarkup later today, or tomorrow.  There
should be no need for you to troubleshoot this further.

> because it includes the Versions header.

Mmm, right.  Will have to think about that.

Best regards,

	Henrik

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 12 Jun 2018 18:55:40 +0200

Thanks, Henrik, inline

> instead uses the paragraph that starts with [I-D.ietf-anima-bootstrapping-keyinfra] in Appendix A.2.

But there is no problem in the XML, right ? E.g.: ietf-anima-bootstrapping-keyinfra
is in the references section.

Is it because the reference is <?rfc include="reference.I-D.ietf-anima-bootstrapping-keyinfra.xml"?>
aka: with '.xml' ? xml2rfc doesn't complain about it, so i figured it doesn't
matter whether or not to add it...

> should be no need for you to troubleshoot this further.

Great!

> Mmm, right.  Will have to think about that.

Let me know if i should open an official request. The PDF+hyperlink format
is definitely a lot nicer than the two 'HTML' type formats of PDF produced
by xml2rfc. And its also the only PDF format generated by datatracker,
so authors can only check that that output would be correct if they
had a chance to locally generate it before uploading e.g., only the
XML to datatracker - of course they couldn't even upload such a hyperlink
PDF without a non-standard toolchain today in case they want to superceed
the datatracker generated one. Like the tooling Julian was suggesting.
But that one didn't have hyperlinks either.

Except for the bu, its cool tooling. Is three any documentation for the
patterns it tries to discover in the text ? Like my "Section 3.2 of [RFC7575]"
being a single hyperlink. Thats very nice. If i knew what patterns of
supplementry text it can recognize i could take care of such text
to happen in drafts.

Cheers
    Toerless

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Jun 2018 19:01:36 +0200

On 2018-06-12 18:55, Toerless Eckert wrote:
> ... > Let me know if i should open an official request. The PDF+hyperlink 
format
> But that one didn't have hyperlinks either.

It does. It's just not blue-with-underline, which would be very uncommon 
in PDF...

> supplementry text it can recognize i could take care of such text

AFAIU, rfcmarkup detects

  "Section x of [REF]"

and

  "[REF], Section x"

(plus the same variants with "Appendix").

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 13 Jun 2018 13:36:00 +0200

Hi Toerless,

Cc:ing Tony for the request below.

On 2018-06-12 18:55, Toerless Eckert wrote:
> But there is no problem in the XML, right ?

Correct.  This is purely a problem in the rfcmarkup tool.

> E.g.: ietf-anima-bootstrapping-keyinfra
> Is it because the reference is <?rfc
> include="reference.I-D.ietf-anima-bootstrapping-keyinfra.xml"?> aka:
> with '.xml' ? xml2rfc doesn't complain about it, so i figured it
> doesn't matter whether or not to add it...

No, that's fine as such.

> But that one didn't have hyperlinks either.

This email should be enough of a request.  I'm cc:ing Tony Hansen for
info -- he's the maintainer of the xml2rfc webservices.

> patterns it tries to discover in the text ?

Only in the code, around 1368 and following (before the coming changes).
Extracting comments from the code, mentioning the insertion of markup for
'section of' type references:

             # section x of rfc y markup
             # appendix x of rfc y markup
             # rfc y, section x markup
             # rfc y, appendix x markup
             # section x of? [rfc y] markup
             # appendix x of? [rfc y] markup
             # [rfc y], section x markup
             # [rfc y], appendix x markup

> Like my "Section 3.2 of [RFC7575]"

I hope the extract above helps.

Best regards,

	Henrik