Subject: [xml2rfc] Experience with --v2v3

From: Robert Moskowitz <[email-masked]>
Date: Sat, 14 Dec 2019 21:19:15 -0500

I suppose that it works well enough.  Technically it seems to produce 
xml that generates the same output.  Many things were changed that I 
would not have expected needing changing.

An example of a change was my <figure> <artwork> to <artwork> <[CDATA]> 
stuff.

OK.

The indentation methodology was not to my liking, and I edited the 
results to what I like for readability.  Also <sections> no longer seem 
to have the section name, that is now in <name>. It is a problem when 
this is put on a separate line and you want to collapse <section> in 
your editor.  You cannot see the name anymore.  So I had to put the 
<name> object on the same line as <section>.  This way collapsing the 
section maintains readability.

I really did not like it expanding <?rfc> references like:

     <?rfc include="reference.RFC.2119.xml"?>

The reason for the references rather than the content is that they can 
change (like author changes to drafts).  Thus I feel they are wrong to 
expand and should be left as is.  I manually put them back.

Fortunately I only have 4 drafts active, and they are small.  Plus two 
that I am helping the authors.

Bob

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Dec 2019 11:03:00 +0100

On 15.12.2019 03:19, Robert Moskowitz wrote:
>...
> ...

There's the "--add-xinclude" command line switch which is supposed to
change this:

>   V2-V3 Converter Options:
>     --add-xinclude                      replace reference elements with RFC
>                                         and Internet-Draft seriesInfo with the
>                                         appropriate XInclude element

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Sun, 15 Dec 2019 06:38:47 -0500

>>                                         appropriate XInclude element

I will give this a try shortly, but I feel that the switch is 
backwards.  The default should be NOT to expand xincludes.

rfc xincludes are used for a vary valid reason.  What is the reason to 
remove them for v3?  And making putting them back needing special action?

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Dec 2019 14:09:58 +0100

> remove them for v3?  And making putting them back needing special action?

We replaced a proprietary hack (a PI) with something standards-based
with greater utility.

WRT xml2rfc's defaults: you'll have to ask Henrik.

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Sun, 15 Dec 2019 09:09:07 -0500

> We replaced a proprietary hack (a PI) with something standards-based

Ah, I missed that it was a replacement of <?rfc with <xi:include

But this new form requires you to know where the rfc/ID is coming from 
with explicit URL.  So it goes...

> WRT xml2rfc's defaults: you'll have to ask Henrik.

Hopefully he will catch up with this thread and comment.

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 15 Dec 2019 14:20:43 -0500

> remove them for v3?  And making putting them back needing special action?

Do I understand correctly that the translation takes a wildcard 
reference (to the latest version of the draft) and binds it to a 
particular version at the time of the translation?

If so, that seems like the wrong thing to do.

	Thanks,
	Paul

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 16 Dec 2019 08:37:23 +1300

> If so, that seems like the wrong thing to do.

Utterly wrong. How does one refer to the latest version in v3 format?
Something like
<xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.draft-andrews-tcp-and-ipv6-use-minmtu.xml"/>
doesn't work.

   Brian

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 15 Dec 2019 21:21:12 +0100

Hi Bob,

> with explicit URL.  So it goes...

Yes.  v2 had default locations in which to look for bibxml files, something
I moved over to the v3 format's handling of <xi:include> entries.

However, Julian insisted that for <xi:include>, the URL had to be fully
specified, and having the same kind of fallback on default bibxml paths
was disallowed, so I had to take that out, with a great deal of regret.

> Hopefully he will catch up with this thread and comment.

The v2v3 converter work was part of the RFC format RFP statement of work,
which focused on the needs of the RPC.  When I introduced the --add-xinclude
switch, I checked with the RPC what their preference was, and it was at
that time to expand any references and inline them.  The preference has
shifted, and my intention is that when we transition to the xml2rfc 3.x
series, backwards incompatible changes can be permitted, and I would change
the default for the v2v3 converter to not include expanded references, but
instead insert <xi:include> entries.

Best regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 15 Dec 2019 21:28:23 +0100

Hi Paul,

> If so, that seems like the wrong thing to do.

I agree.  I see that this is a result of the way the current code tries
to add <xi:include> entries for any <reference> which refers to a draft
or RFC, whether it has been pulled in by reference or was present as a
fully expanded entry; this happening after the (current) default operation
of expanding all references during v2v3 conversion has taken place.

I'll change that in an upcoming release.

Best regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 15 Dec 2019 21:46:41 +0100

On 2019-12-15 20:37, Brian E Carpenter wrote:
> Utterly wrong.

Agreed.

> How does one refer to the latest version in v3 format?
> Something like

Given an existing URL, the handling of non-version-specific references in
v2 and v3 should be equivalent.  Any reference URL that works in v2 should
work in v3.  In v3, as in v2, the version-specific URLs include 'draft-'
in the name, while the non-version-specific URLs omits 'draft-'.  This is a
legacy from Marshall Rose's time, and the inconsistency has been the same
for years.

The correct <xi:include> entry for the non-version-specific reference should
be:

  <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.andrews-tcp-and-ipv6-use-minmtu.xml"/>

I've handled this differently in the reference entries generated on-the-fly
by the datatracker; there all of the following forms:

   bibxml3/reference.I-D.andrews-tcp-and-ipv6-use-minmtu.xml
   bibxml3/reference.I-D.draft-andrews-tcp-and-ipv6-use-minmtu.xml
   bibxml3/reference.I-D.andrews-tcp-and-ipv6-use-minmtu-04.xml
   bibxml3/reference.I-D.draft-andrews-tcp-and-ipv6-use-minmtu-04.xml

work.  Expanding the second to the full path gives:

https://datatracker.ietf.org/doc/bibxml3/reference.I-D.draft-andrews-tcp-and-ipv6-use-minmtu.xml

Best regards,

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Dec 2019 22:14:14 +0100

On 15.12.2019 21:21, Henrik Levkowetz wrote:
> ... > Yes.  v2 had default locations in which to look for bibxml files,
something
> was disallowed, so I had to take that out, with a great deal of regret.

The point of introducing xi:include was to introduce a standard
inclusion mechanism not specific to IETF work. Tweaking it to do
something else really really is a hack, and is bad because it makes it
impossible to use standard Xinclude processors/libraries.

I happen to care about that, so that's why I complained.

We can have a discussion about a different method which is
IETF-specific, but that, by definition, can not be called xi:include.

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Sun, 15 Dec 2019 16:17:24 -0500

> <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.draft-andrews-tcp-and-ipv6-use-minmtu.xml"/>

I just ran --v2v3 --add-xinclude on a draft

and got:

         <xi:include 
href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.draft-ietf-hip-dex-11.xml"/>
         <xi:include 
href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.draft-moskowitz-hip-hierarchical-hit-02.xml"/>
         <xi:include 
href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.draft-moskowitz-mobile-privacy-attack-01.xml"/>

that is so broken.  We have done so much to reference drafts by the name 
and let tools provide the current draft that this represents a step back.

I might thing some kind of wildcarding is needed for the version number.

My $.02 worth.

---

From: Robert Moskowitz <[email-masked]>
Date: Sun, 15 Dec 2019 16:22:17 -0500

> IETF-specific, but that, by definition, can not be called xi:include.

Perhaps if instead of an absolute URL there is a URI that provides the 
information so the latest draft info is retrieved?

Something like:

<xi:include 
href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/?ID=reference.I-D.moskowitz-hip-hierarchical-hit.xml"/>

Having I-D.draft seems redundant?

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 15 Dec 2019 22:50:14 +0100

Hi Bob,

> and let tools provide the current draft that this represents a step back.

This is a result of the current v2v3 converter.  Change it to the following,
and you're done, and can move forward:

   <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-hip-dex.xml"/>
   <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.moskowitz-hip-hierarchical-hit.xml"/>
   <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.moskowitz-mobile-privacy-attack.xml"/>

> I might thing some kind of wildcarding is needed for the version number.

You can use non-version-specific URLs for v3 as for v2.  The expansion to
a specific number happens only during the (current) v2v3 conversion.  But
with xi:include you have to take care that you have the full and correct
URL, as the xi:include mechanism won't do any path or other adjustments
for you, the way v2's inclusion mechanisms did.

Regards,

	Henrik

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 15 Dec 2019 22:53:04 +0100

On Dec 15, 2019, at 22:17, Robert Moskowitz <[email-masked]> wrote:
> so broken

Yep.  RFCXMLv3 appears to be a bit less of an authoring format than RFCXMLv2 was.
(Which is not surprising because it really was designed as a publishing format.)

The right way to solve that problem may be to use an actual authoring format for authoring (you know which one I’d recommend).

Grüße, Carsten

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 15 Dec 2019 22:54:57 +0100

Hi Bob,

> Having I-D.draft seems redundant?

Indeed.

I'd recommend this:

  <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-moskowitz-hip-hierarchical-hit.xml"/>

for reduced redundancy, but this also works, and uses the same reference
files as before:

(i.e., your proposal minus the '?ID=' part).

Regards,

	Henrik

---

From: Robert Moskowitz <[email-masked]>
Date: Sun, 15 Dec 2019 17:05:38 -0500

> (i.e., your proposal minus the '?ID=' part).

Ok that works.  But for those not following this thread, I would hope 
that v2v3 is improved with a real ?rfc to xi:include converter...

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 15 Dec 2019 19:31:41 -0500

> (i.e., your proposal minus the '?ID=' part).

For those of us who don't follow this closely, it would be very helpful 
to have guide on best practices for incorporating references into xmlv3 
documents, including:

- where to find the stuff for any desired document
- best way to incorporate when adding new references to a document
- how best to ensure this is done properly when migrating a document 
from v2, ore when reverse engineering a .txt document (when preparing 
for a bis.)

In v2 I've learned a way that "works" most of the time. But I have no 
confidence that it is the best way.

	Thanks,
	Paul

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Dec 2019 07:07:28 +0100

> for you, the way v2's inclusion mechanisms did.

Yes, but that means baking knowledge of the "right" URIs into the processor.

An IMHO better alternative would be to move these smarts into the HTTP
server that serves the references.

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 16 Dec 2019 12:42:23 +0100

Hi Paul,

> confidence that it is the best way.

The RFC-Editor has shared the FAQ they put together for internal use:

  https://www.rfc-editor.org/materials/FAQ-xml2rfcv3.html

Maybe that will help?

Regards,

	Henrik

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 16 Dec 2019 08:42:15 -0500

The draft had:

     <?rfc 
include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.6028/NIST.FIPS.202.xml?anchor=NIST.FIPS.202"?>

The --add-xinclude did not work.  I got:

         <reference anchor="NIST.FIPS.202" 
xml:base="https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.6028/NIST.FIPS.202.xml?anchor=NIST.FIPS.202">
           <front>
             <title>SHA-3 Standard: Permutation-Based Hash and 
Extendable-Output Functions</title>
             <seriesInfo name="DOI" value="10.6028/nist.fips.202"/>
             <seriesInfo name="National Institute of Standards and 
Technology" value="report"/>
             <author initials="M." surname="Dworkin" fullname="Morris J. 
Dworkin">
               <organization/>
             </author>
             <date year="2015" month="July"/>
           </front>
         </reference>

Should it have generated:

         <xi:include 
href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.6028/NIST.FIPS.202.xml?anchor=NIST.FIPS.202"/>

??

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 16 Dec 2019 15:09:38 +0100

Hi Bob,

> href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.6028/NIST.FIPS.202.xml?anchor=NIST.FIPS.202"/>

Not currently.  I'll look at making that possible when I rewrite the
<xi:include> handling.

Regards,

	Henrik

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 16 Dec 2019 09:19:21 -0500

> Not currently.  I'll look at making that possible when I rewrite the

Did I at least get the new format correct for a NIST reference in the 
xi:include manner?

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 16 Dec 2019 15:26:15 +0100

Hi Bob,

> Did I at least get the new format correct for a NIST reference in the 

Yes, it looks good :-)

	Henrik

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 16 Dec 2019 09:47:25 -0500

>> Did I at least get the new format correct for a NIST reference in the

Interestingly enough a slight difference in results:

The <?rfc method:

    [NIST.FIPS.202]
               Dworkin, M., "SHA-3 Standard: Permutation-Based Hash and
               Extendable-Output Functions", DOI 10.6028/nist.fips.202,
               National Institute of Standards and Technology report,
               July 2015, <https://doi.org/10.6028/nist.fips.202>.

the <xi method:

    [NIST.FIPS.202]
               Dworkin, M., "SHA-3 Standard: Permutation-Based Hash and
               Extendable-Output Functions", National Institute of
               Standards and Technology report,
               DOI 10.6028/nist.fips.202, July 2015,
               <https://doi.org/10.6028/nist.fips.202>.

:)

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 16 Dec 2019 11:08:47 -0500

>    https://www.rfc-editor.org/materials/FAQ-xml2rfcv3.html

Yes it does!

Is that a stable reference that will be maintained?

	Thanks,
	Paul

---

From: Sandy Ginoza <[email-masked]>
Date: Mon, 16 Dec 2019 11:18:46 -0800

Hi Paul,

> On Dec 16, 2019, at 8:08 AM, Paul Kyzivat <[email-masked]> wrote:
> 

Yes - we will continue to update the FAQ as needed.

Thank you,
Sandy 

> 
> 	Thanks,
> 	Paul
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 17 Dec 2019 08:19:40 +1300

> server that serves the references.

You mean something like
https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.fz-6man-ipv6-alt-mark.xml ?

   Brian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Dec 2019 20:26:29 +0100

> https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.fz-6man-ipv6-alt-mark.xml ?

I would argue that we should be able to set up servers so that all of these

   http://....ietf.org/bib/rfcnnnn.xml

   http://....ietf.org/bib/draft-name-foo.xml (latest)

   http://....ietf.org/bib/draft-name-foo-nn.xml

work. So authors should not even have to think about which "folder" the
reference is in...

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 16 Dec 2019 14:44:24 -0500

> work. So authors should not even have to think about which "folder" the

Got my vote.  But the NIST references from bibxml7 seem to work 
differently than the RFC and IDs:

<xi:include 

Would be nice if those and other orgs DOI references all work nicely 
without learning the specific magic.