Subject: [xml2rfc] FIPS and SP800 references

From: Robert Moskowitz <[email-masked]>
Date: Sun, 24 Sep 2017 18:27:15 -0400

I need a couple FIPS and SP800 references:

<reference anchor="FIPS.202.2015" 
target="http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf">
<front>
<title>SHA-3 Standard:  Permutation-Based Hash and Extendable-Output 
Functions</title>
<author>
<organization>National Institute of Standards and Technology</organization>
</author>
<date month="August" year="2015"/>
</front><seriesInfo name="FIPS" value="PUB 202"/></reference>

<reference anchor="SP.800_185.2016" 
target="http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-185.pdf">
<front>
<title>SHA-3 Derived Functions: cSHAKE, KMAC, TupleHash and 
ParallelHash</title>
<author>
<organization>National Institute of Standards and Technology</organization>
</author>
<date month="December" year="2016"/>
</front><seriesInfo name="Special Publication" value="SP 
800-185"/></reference>

Can these be added to http://xml2rfc.tools.ietf.org/public/rfc/bibxml2/ ?

Note the URL change for where FIPS docs are from the 'old'

http://csrc.nist.gov/publications/fips

Currently, I cannot find any SP in the bibxml area.

Maybe someone can talk to Tim Polk about this...

Bob

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 25 Sep 2017 02:21:46 +0200

On Sep 25, 2017, at 00:27, Robert Moskowitz <[email-masked]> wrote:
> <reference anchor="FIPS.202.2015" target="http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf">
> <front>
> <title>SHA-3 Standard:  Permutation-Based Hash and Extendable-Output Functions</title>
> <author>

    FIPS.202.2015: DOI.10.6028/NIST.FIPS.202

> <reference anchor="SP.800_185.2016" target="http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-185.pdf">
> <front>
> <title>SHA-3 Derived Functions: cSHAKE, KMAC, TupleHash and ParallelHash</title>
> <author>
> </front><seriesInfo name="Special Publication" value="SP 800-185"/></reference>

   SP800-185: DOI.10.6028/NIST.SP.800-185

Paste this into your markdown source.

Grüße, Carsten

> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Robert Moskowitz <[email-masked]>
Date: Sun, 24 Sep 2017 21:00:23 -0400

> Paste this into your markdown source.

I'm stubborn and use xml.  So I need an xml source...

---

From: Julian Reschke <[email-masked]>
Date: Mon, 25 Sep 2017 06:19:43 +0200

On 2017-09-25 00:27, Robert Moskowitz wrote:
> ...
> ...

There's nothing wrong in just having the referenced data inlined into 
your XML source.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 25 Sep 2017 08:02:08 +0200

> On Sep 25, 2017, at 03:00, Robert Moskowitz <[email-masked]> wrote:
> 
> 
> 
> I'm stubborn

I don’t know how to help you with that :-)

> and use xml.  So I need an xml source…

The doilit tool that comes with kramdown-rfc2629 spits out xml if you want:

$ doilit -x=FIPS.202.2015 10.6028/NIST.FIPS.202
<reference anchor="FIPS.202.2015" >
  <front>
    <title>SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions</title>
    <author initials="M." surname="Dworkin" fullname="Morris J. Dworkin">
      <organization></organization>
    </author>
    <date year="2015" month="July"/>
  </front>
  <seriesInfo name="[]" value=""/>
  <seriesInfo name="DOI" value="10.6028/nist.fips.202"/>
</reference>

As Julian says, you don’t lose anything by pasting this in, as the info behind the DOI is not really going to change.  You also can fix weirdness in the database (or, this time, in the tool: the series info with []; haven’t seen that before, to be fixed).

Or, if pasting XML like that is too radical, you can use bibxml7:

<!ENTITY DOI.10.6028_NIST.FIPS.202 SYSTEM "https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.6028/NIST.FIPS.202.xml?anchor=FIPS.202.2015”>

But then you have to live with any weirdness that may come up.

Grüße, Carsten

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 25 Sep 2017 13:32:55 -0400

>      FIPS.202.2015: DOI.10.6028/NIST.FIPS.202

I don't see how to include this in my xml.  I tried putting it the 
normative references section and got:

ERROR: Unable to validate the XML document: INPUT
  <string>: Line 242: Element references content does not follow the DTD, expecting (reference)+, got (CDATA reference)

---

From: Julian Reschke <[email-masked]>
Date: Mon, 25 Sep 2017 20:47:43 +0200

On 2017-09-25 19:32, Robert Moskowitz wrote:
>> On Sep 25, 2017, at 00:27, Robert Moskowitz<[email-masked]>  wrote:

Please provide the full file, or at least an excerpt that shows what you 
are doing...

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 25 Sep 2017 15:01:35 -0400

On 09/25/2017 02:47 PM, Julian Reschke wrote:
>>   <string>: Line 242: Element references content does not follow the 
>> DTD, expecting (reference)+, got (CDATA reference)
>
> Please provide the full file, or at least an excerpt that shows what 
> you are doing...
>
Here is the full xml.  I want to add the reference for FIPS 202 and SP 
800-185.

I can just add the xml to the normative reference section.  I would like 
to use a public standing xml.

Also I looked at the xml at:

https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.6028/NIST.FIPS.202.xml?anchor=FIPS.202.2015%E2%80%9D

and the reference anchor is rather not nice to have in the ID, as it has 
no punctuation.  There are other issues with it but that is the 'first'.

Bob

---

From: Julian Reschke <[email-masked]>
Date: Mon, 25 Sep 2017 21:42:33 +0200

What you did is that you added markdown text into XML. That doesn't 
work. You'll have to include the <reference> elements.

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 25 Sep 2017 15:54:07 -0400

> work. You'll have to include the <reference> elements.
Then what did you mean by:

"There's nothing wrong in just having the referenced data inlined into 
your XML source."

A few messages back.

That was the nature of my reply to Carsten.  I work in XML, he gave me a 
markdown answer.  Which also points to some orphaning of the xml capability.

Bob

Bob

---

From: Julian Reschke <[email-masked]>
Date: Mon, 25 Sep 2017 22:58:19 +0200

> markdown answer.  Which also points to some orphaning of the xml 

a) markdown is used by some people as *frontend*, not replacement

b) pleas go back and re-read what Carsten and I wrote

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 25 Sep 2017 23:16:42 +0200

On Sep 25, 2017, at 21:54, Robert Moskowitz <[email-masked]> wrote:
> 
> That was the nature of my reply to Carsten.  I work in XML, he gave me a markdown answer.  

Yes.  On this list, we are in different stages of transitioning away from XML as an *authoring* format.
Which is why I gave a markdown-specific answer first (and Julian pointed out how to do this in XML).

> Which also points to some orphaning of the xml capability.

XML (soon with a new DTD colloquially referred to as “RFCXML v3") will continue to be the preferred *submission* format, and of course authoring right in the submission format will always be possible.  That’s why I gave two ways to generate the references to NIST documents right in XML in my second answer: one for pasting in the RFCXML reference format, and one for doing the usual bibxml dance (“System entity reference” — well, I only gave the entity definition) at formatting time.

By the way, all the examples I gave allow you to change the anchor (label) as desired — for entity imports from bibxml this works with bibxml7, as these are dynamically generated, but not for most other bibxml categories.

Now for finding the seriesinfo strangeness...

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 25 Sep 2017 23:53:30 +0200

> Now for finding the seriesinfo strangeness...

Fixed in kramdown-rfc2629 1.2.6 (1.2.5 could not deal with empty container-titles).
Please gem update as convenient.

Grüße, Carsten

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 25 Sep 2017 17:56:24 -0400

Julian,

I am going back as you said...

On 09/25/2017 12:19 AM, Julian Reschke wrote:
>> Can these be added to 
>> http://xml2rfc.tools.ietf.org/public/rfc/bibxml2/ ?
>>

What data are you referring to that I can place inline in the XML? I am 
missing something here.

Bob

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 25 Sep 2017 18:06:25 -0400

> XML (soon with a new DTD colloquially referred to as “RFCXML v3") will continue to be the preferred *submission* format, and of course authoring right in the submission format will always be possib…

Since the bibxml url seems to have a strange xml for the nist docs, I 
have put some xml reference right into the draft for now.

> By the way, all the examples I gave allow you to change the anchor (label) as desired — for entity imports from bibxml this works with bibxml7, as these are dynamically generated, but not for most…

I have read the docs, and don't see, short of including the xml directly 
into the reference section to change the anchor label used in the ID.  
The one example in the docs show how to change RFC4306 to IKEv2 by 
including the whole xml.  This has always been a no fun issue with long 
draft names as references.

I had tried altering:

   <!ENTITY ieee_802_1ar SYSTEM 
"https://xml2rfc.tools.ietf.org/public/rfc/bibxml6/reference.IEEE.802.1AR_2009.xml">

To:

   <!ENTITY IEEE_802_1AR SYSTEM 
"https://xml2rfc.tools.ietf.org/public/rfc/bibxml6/reference.IEEE.802.1AR_2009.xml">

Which follows the IEEE naming requirement (1AR is a self-standing 
standard, not an addendum to another standard and thus MUST be 
capitalized when referenced), and then put

&IEEE_802_1AR

in the reference section and got an error for my efforts.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Sep 2017 06:17:47 +0200

> Yes.  On this list, we are in different stages of transitioning away from XML as an *authoring* format.

Clarifying: where one "stage" is: "not transitioning away at all".

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Sep 2017 06:25:59 +0200

> have put some xml reference right into the draft for now.

Yes. Why is that a problem?

>> By the way, all the examples I gave allow you to change the anchor 
>> (label) as desired — for entity imports from bibxml this works with 
>> bibxml7, as these are dynamically generated, but not for most other 
>> bibxml categories.
> 

The V3 vocabulary allows to rename references, see 
<https://www.greenbytes.de/tech/webdav/rfc7991.html#element.displayreference>. 
This is not implemented yet in xml2rfc, but you can use rfc2629.xslt as 
preprocessor already (see 
<https://www.greenbytes.de/tech/webdav/rfc2629xslt/rfc2629xslt.html>).

> in the reference section and got an error for my efforts.

1) I don't see how renaming the XML entity can possible affect the 
output. It's like renaming a variable a variable in a program.

2) I also don't see why it would cause an error. But if you don't show 
us either the source code or at least the error message, how are we 
supposed to help?

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 26 Sep 2017 12:44:17 +0200

> Clarifying: where one "stage" is: "not transitioning away at all".

:-)

Sure, I remember being in that stage, too.

Then, in 2010, I had to write/contribute to six I-Ds with a short deadline.

I just had gone through the experience of several non-trivial, collaborative (and heavily version-controlled) docbook-based documentation projects, and I remembered how much better these went once we had switched to ASCIIDOC as the authoring format.

So I gambled that I would save time by first writing a markdown to RFCXML conversion tool and then writing the six I-Ds.  Of course, I don’t have a control group, but I believe that this did save a lot of time.

The choice fell on markdown for that tool instead of ASCIIDOC — not because it is better (it is not), but it already was widely understood at the time, so it would be easier to collaborate on documents that way.  With the rise of git and github, both the version-controlled collaborative approach and the use of markdown as an authoring format have since received a considerable boost.

The fact that another markdown-based authoring tool as well as an org-mode based tool were created independently shows that other people are also seeing benefits in authoring using close-to-plaintext formats.

Our processing pipelines are now well-oiled for supporting these authoring languages.
The remaining problem happens when people want to collaborate on an RFC who have different authoring language preferences.  The version-controlled approach pretty much rules out constant up- and down-conversion, so it is usually necessary to agree on one language.  In the efforts I have been involved in, agreeing on markdown generally has not been too hard.  (Agreeing on American spelling has been harder :-)

Grüße, Carsten

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 26 Sep 2017 08:56:28 -0400

> Clarifying: where one "stage" is: "not transitioning away at all".

I am being pushed by one co-author that wants to write in kramdown, so I 
am going to have to learn a bit and a bit of github to send him updates.

Other than that, I am very comfortable with xml and using Geany as my 
editor.  I may only have a couple more active years here in the IETF, so 
I don't see the compelling need to change.  And it took me long enough 
to get into xml!

Bob

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 26 Sep 2017 09:06:58 -0400

> Yes. Why is that a problem?

If there is an 'official reference', I would rather use that then craft 
my own.

In this case, I don't think the official one is best.  It leaves out the 
organization, NIST.  The date is not the one in the pdf (off by a 
month).  It lacks the FIPS info (or the SP 800 info).  Never mind any 
discussion on the anchor name.

>> I have read the docs, and don't see, short of including the xml 
>> directly into the reference section to change the anchor label used 
>> in the ID.
>
> This is not implemented yet in xml2rfc, but you can use rfc2629.xslt 
> as preprocessor already (see 

I will look into this.

>> including the whole xml.  This has always been a no fun issue with 
>> long draft names as references.
>>
>> I had tried altering:
>>

See the attached.  I have renamed the anchor, IEEE.802.15.6_2012 to 
IEEE.802.15.6 in the ENTITY line and followed through in the reference.  
I get the error:

$ xml2rfc draft-moskowitz-small-crypto-00.xml
Parsing file draft-moskowitz-small-crypto-00.xml
ERROR: Unable to validate the XML document: 
draft-moskowitz-small-crypto-00.xml
  draft-moskowitz-small-crypto-00.xml: Line 364: IDREF attribute target 
references an unknown ID "IEEE.802.15.6"

Oh, and I tried it online from http://xml2rfc.tools.ietf.org/ and am 
getting the 500 server error.  Again.

Bob

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Sep 2017 15:18:37 +0200

On 2017-09-26 15:06, Robert Moskowitz wrote:
> ... >> 1) I don't see how renaming the XML entity can possible affect the

No, you renamed the entity, not the anchor - that one is defined in the 
XML that your're including from 
<http://xml2rfc.tools.ietf.org/public/rfc/bibxml6/reference.IEEE.802.15.6_2012.xml>.

> references an unknown ID "IEEE.802.15.6"

Yes, because your document does not contain a reference with that anchor.

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 26 Sep 2017 09:29:06 -0400

On 09/26/2017 09:18 AM, Julian Reschke wrote:
>>> 2) I also don't see why it would cause an error. But if you don't 
>>> show us either the source code or at least the error message, how 
>>> are we supposed to help?
>>
> No, you renamed the entity, not the anchor - that one is defined in 
> the XML that your're including from 

So there is, currently, no way to rename the anchor in an included 
reference xml...

OK

>>   draft-moskowitz-small-crypto-00.xml: Line 364: IDREF attribute 
>> target references an unknown ID "IEEE.802.15.6"
>> ...
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 26 Sep 2017 15:37:09 +0200

On Sep 26, 2017, at 15:06, Robert Moskowitz <[email-masked]> wrote:
>>> Since the bibxml url seems to have a strange xml for the nist docs, I have put some xml reference right into the draft for now.
>>> ...
>> 
> If there is an 'official reference', I would rather use that then craft my own.
> 
> In this case, I don't think the official one is best.  It leaves out the organization, NIST.  The date is not the one in the pdf (off by a month).  It lacks the FIPS info (or the SP 800 info).  Never mind any discussion on the anchor name.

The complete lack of the publisher name is a bug that I already have fixed; waiting for Henrik or Tony to install that fix.  However, the “FIPS PUB 202” or “Special Publication SP 800-185” you want is indeed not in the DOI database; that is a strong reason to maintain your own entry instead of using an ‘official’ one.

For bibxml7 DOI references, you do get to choose the anchor (the thing after the question mark in the URI); for manually included references, you can always change the anchor you give there.

> See the attached.  I have renamed the anchor, IEEE.802.15.6_2012 to IEEE.802.15.6 in the ENTITY line and followed through in the reference.  I get the error:

For directly including bibxml6 in XML via entity reference, you *don’t* get to choose the anchor right now.
(You probably have renamed the entity; the name of the entity is inconsequential; it’s just the equivalent of a filename — the anchor name is set in what is the content of the file.)

Again, if you paste in the reference, you can change the anchor as desired.
(Same if you use kramdown-rfc to do the pasting in for you.)

> ERROR: Unable to validate the XML document: draft-moskowitz-small-crypto-00.xml
> draft-moskowitz-small-crypto-00.xml: Line 364: IDREF attribute target references an unknown ID "IEEE.802.15.6"
> 
> Oh, and I tried it online from http://xml2rfc.tools.ietf.org/ and am getting the 500 server error.  Again.

Well, yes, because you don’t have a reference with the anchor "IEEE.802.15.6” (changing the entity name for that does nothing, and the reference in bibxml6 starts "<reference anchor='IEEE.802.15.6_2012' target=…").

BTW, in the markdown header, you would say

  IEEE.802.15.6: IEEE.802.15.6_2012

to change the anchor (this works with XML2RFC V2 already); for XML, just paste in the reference from bibxml6/reference.IEEE.802.15.6_2012.xml manually and change the anchor manually.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 26 Sep 2017 19:15:12 +0200

On Sep 26, 2017, at 15:29, Robert Moskowitz <[email-masked]> wrote:
> 
> So there is, currently, no way to rename the anchor in an included reference xml...

Currently only if that XML is freshly generated for you, as in bibxml7.
(Or by automatically rewriting it after generation as kramdown-rfc does.)

Tony has made noises that maybe he can provide the anchor query parameter for the other bibxmls, too, but as far as I know that hasn’t been implemented yet.

Grüße, Carsten