Subject: [xml2rfc] missing ref files

From: Randy Bush <[email-masked]>
Date: Mon, 27 Jun 2011 06:14:47 +0900

i am hitting more and more missing ref files.  i am starting to suspect
that they are for draft-ietf-wg---00s, but am not sure.  this is
starting to become painful.

anyone with clue?

randy

---

From: Alice Hagens <[email-masked]>
Date: Mon, 27 Jun 2011 09:57:42 -0400

Randy,

We have been having an issue with rsync1.xml.resource.org for 2 months:  In bibxml (RFCs), the most recent files we have are from 21 April. In bibxml3 (I-Ds), the most recent files are from 25 April (seems your issue is similar since you don't have recent -00s).

More recent files (e.g., http://xml.resource.org/public/rfc/bibxml/reference.RFC.6144.xml of 28-Apr-2011, http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-sidr-bgpsec-protocol.xml of 11-Jun-2011) are available via URL*, but not in our rsync'd repository.  

* So a work-around is using the full URL in the ENTITY form reference in your XML file. For example:   <!ENTITY bgpsec PUBLIC '' 'http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-sidr-bgpsec-protocol.xml'> then &bgpsec;)

xml.resource.org states:
> rsync access is available at two independent servers (rsync1.xml.resource.org::xml2rfc.bibxml, andrsync3.xml.resource.org::xml2rfc.bibxml) — after you pick one, you shouldn't switch. 

We have not tried rsync3.

Thanks,
Alice
for the RFC Production Center

On Jun 26, 2011, at 5:14 PM, Randy Bush wrote:

> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
> 

On Jun 24, 2011, at 9:51 PM, Randy Bush wrote:

> i do a nightly rsync
> 
>    /usr/local/bin/rsync -avz --delete-after \
> 	rsync1.xml.resource.org::xml2rfc.bibxml/bibxml/ \
> 	/usr/local/refs/rfc
>    /usr/local/bin/rsync -avz --delete-after \
> 	rsync1.xml.resource.org::xml2rfc.bibxml/bibxml3/ \
> 	/usr/local/refs/id
>    chmod -R og+rX /usr/local/refs
> 
> today, finishing off a draft, i hit a missing ref,
> 
>   I-D.ietf-sidr-bgpsec-protocol
> 
> which is in my internet-drafts rsync directory
> 
>    -rw-rw-r--  1 randy  staff  59025 Jun 12 13:04 ids/draft-ietf-sidr-bgpsec-protocol-00.txt
> 
> how am i supposed to debug, report, fix?
> 
> randy
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Nico Williams <[email-masked]>
Date: Mon, 27 Jun 2011 09:10:45 -0500

> * So a work-around is using the full URL in the ENTITY form reference in your XML file. For example:   <!ENTITY bgpsec PUBLIC '' 'http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-sidr-…

That's what I do, and it works out great.

I'd like xml2rfc to have a better way to handle this, so I'd not have
to specify the location of these references and instead xml2rfc could
just figure it out (try a local location configured or baked in, then
try the public reference repo).  That would make it much easier to
write .xmls that can be submitted as-is and work for all.

Nico
--

---

From: Julian Reschke <[email-masked]>
Date: Mon, 27 Jun 2011 16:25:18 +0200

On 2011-06-27 16:10, Nico Williams wrote:
> On Mon, Jun 27, 2011 at 8:57 AM, Alice Hagens<[email-masked]>  wrote:
>> * So a work-around is using the full URL in the ENTITY form reference in your XML file. For example:<!ENTITY bgpsec PUBLIC '' 'http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-sidr-bgpsec-protocol.xml'>  then&bgpsec;)
>
> ...

I wouldn't want to trade predictability vs convenience.

My preference is to actually *copy* the reference entries into my 
source. That way, the document is stand-alone, and will yield the same 
TXT independently of when and where it's generated.

Best regards, Julian

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 27 Jun 2011 09:01:10 -0700

On 06/27/2011 07:25 AM, Julian Reschke wrote:
>>> * So a work-around is using the full URL in the ENTITY form reference in your
>>> XML file. For example:<!ENTITY bgpsec PUBLIC ''
>>> 'http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-sidr-bgpsec-protocol.xml'> 
>>> then&bgpsec;)
>>
> My preference is to actually *copy* the reference entries into my source. That
> way, the document is stand-alone, and will yield the same TXT independently of
> when and where it's generated.

Yes, and this is, IMO, what people should do for XML documents before submitting
them to the IETF, so other format can later be generated with the correct
references.  I wrote a tool that is doing exactly this.  The document author
uses URNs, and the tool generates a stand alone XML file.  Here's an example
from a draft I recently published:

<references title="Normative References" xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="urn:ietf:rfc:2119" />
  <xi:include href="urn:ietf:id:ietf-p2psip-base" />
</references>

The next step was to use a cache for the resolution, but I have yet to implement
that.  There is more explanations on my blog, see in my signature below.

- -- 
Marc Petit-Huguenin
Personal email: [email-masked]
Professional email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org

---

From: Randy Bush <[email-masked]>
Date: Tue, 28 Jun 2011 05:01:20 +0900

> xml.resource.org states:
>> rsync access is available at two independent servers
>> (rsync1.xml.resource.org::xml2rfc.bibxml,
>> andrsync3.xml.resource.org::xml2rfc.bibxml) — after you pick one,
>> you shouldn't switch.

bingo!  3 works for me.

thank you thank you.

randy

---

From: Nico Williams <[email-masked]>
Date: Mon, 27 Jun 2011 23:37:09 -0500

On Mon, Jun 27, 2011 at 9:25 AM, Julian Reschke <[email-masked]> wrote:
> My preference is to actually *copy* the reference entries into my source.
> That way, the document is stand-alone, and will yield the same TXT
> independently of when and where it's generated.

I don't like that at all.  I mind not mind an option that outputs an
XML that includes references inline, or an archive that includes the
xml source and the reference files needed to render that xml.  But I
want an authoritative repository of references, not one I maintain by
hand.  This is 2011, for goodness' sake's!

Nico
--

---

From: Randy Bush <[email-masked]>
Date: Tue, 28 Jun 2011 15:26:59 +0900

> hand.  This is 2011, for goodness' sake's!

+1

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 28 Jun 2011 15:28:08 +0900

> TXT independently of when and where it's generated.

That's also my preference. It works even on a flight or when the 
connection is down temporarily.

Regards,   Martin.

---

From: Randy Bush <[email-masked]>
Date: Tue, 28 Jun 2011 15:29:54 +0900

>> source. That way, the document is stand-alone, and will yield the
>> same TXT independently of when and where it's generated.

notice the word 'rsync' in the $subject?

randy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 28 Jun 2011 08:56:49 +0200

On 2011-06-28 06:37, Nico Williams wrote:
> On Mon, Jun 27, 2011 at 9:25 AM, Julian Reschke<[email-masked]>  wrote:

I don't argue against the repository, I argue about requiring it to be 
*online* while transforming.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 28 Jun 2011 08:58:03 +0200

> notice the word 'rsync' in the $subject?

That still implies that any "consumer* of the XML version has a copy of 
the repository...

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Jun 2011 08:26:31 -0500

> *online* while transforming.

This is why I keep a local copy of the DNS.  Also wikipedia.  Also
everything else.

Copying references into the xml source is so /etc/hosts, so 1984.

Online infrastructure is a fact of life in 2011.

Nico
--

---

From: Michael Richardson <[email-masked]>
Date: Tue, 28 Jun 2011 08:44:33 -0400

>>>>> "Julian" == Julian Reschke <[email-masked]> writes:
    >> I don't like that at all.  I mind not mind an option that outputs
    >> an XML that includes references inline, or an archive that
    >> includes the xml source and the reference files needed to render
    >> that xml.  But I want an authoritative repository of references,
    >> not one I maintain by hand.  This is 2011, for goodness' sake's!

    Julian> I don't argue against the repository, I argue about
    Julian> requiring it to be *online* while transforming.

Some years ago I did some patches such that xml2rfc had a search path
while looking for references.  This allowed me to keep a local copy (for
offline, and much faster work), but still put the official URL into my
document.  

At the time, there was nobody who could accept my patches. I can try
again.

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition.

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 28 Jun 2011 07:34:54 -0700

On Jun 27, 2011, at 9:37 PM, Nico Williams wrote:

> But I

It's fine that you want that, but it's not what we have. resource.org is a voluntary service. At some point in the future, the IETF might support a replacement that will have different service level agreements.

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Jun 2011 09:54:26 -0500

> It's fine that you want that, but it's not what we have. resource.org is a voluntary service. At some point in the future, the IETF might support a replacement that will have different service leve…

If the current service stopped working we could make do -- worst case
scenario we'd all cobble together our own references entries from the
actual RFCs, I-Ds, etcetera that we want to reference.  Also, this is
hardly a critical service.  Therefore I would say that any lack of SLA
does not change the reality that this service *exists* and has been
working quite reliably for years.

I'm willing to chance using the references repository.  Indeed, I
fully intend to continue using it, at my own risk (quick, I better up
my life insurance policy! :)

Moreover, the fact that the service is not supported (is it?) is
hardly a reason for *not* building support for it into the tool -it's
already there-, much less for ripping it out (that might risk a
peasant's revolt the likes of which we've not seen in centuries).

Nico
--

---

From: Randy Bush <[email-masked]>
Date: Wed, 29 Jun 2011 00:13:48 +0900

> It's fine that you want that, but it's not what we have. resource.org
> is a voluntary service. At some point in the future, the IETF might
> support a replacement that will have different service level
> agreements.

some of us are old enough to remember that the internet is a voluntary
service.

and cache2 and cache3 are at tools, fwiw.

randy

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 30 Jun 2011 10:48:57 +0200

> Copying references into the xml source is so /etc/hosts, so 1984.

As long as I can convince my OS that my favourite text editor has write
access on /etc/hosts this is doubleclick analytics 2011 IpV6 ::1, and
quite the opposite of 1984 ;-)

Seriously, what Julian and Martin wrote is based on how XML entities
are supposed to work.  For "us" xml2rfc is only a tool and should work
without locally expanded bibxml snippets (presumably changing several
times until a draft is published as RFC).  IOW what "we" want is in
theory wrong, in practice it's fine if you know the limitation: no XML
tool without online access on the bibxml repo can actually do anything
useful with the bibxml snippets (apart from checking their syntax).

Copying the repo as suggested by Randy is not an option over slow or
expensive connections and/or with limited disk space.

-Frank

---

From: Nico Williams <[email-masked]>
Date: Thu, 30 Jun 2011 11:59:37 -0500

> quite the opposite of 1984 ;-)

:)

> useful with the bibxml snippets (apart from checking their syntax).

Before xml2rfc I used to write I-Ds as manually formatted, but not
paginated, text, which I'd then paginate (and stamp boilerplates on)
via a pagination script I wrote myself.  Heck, I could use LyX, render
as text, then use such a script again (or figure out how to get LaTeX
to output perfect I-Ds).

But guess what: xml2rfc is better because it's code that we all can
use, and because it solves all sorts of annoying little problems.
Finding and keeping references up to date is one of the most important
and useful features of xml2rfc -- dropping it (either from xml2rfc
itself, or from our habits) is simply out of the question.  Analogies
to many other modern conceniences that yes, we can do without, are
trivial, and in all cases we don't give up on modern conveniences just
because *sometimes* they're not available to us.

Yes please, let's be serious.  Specifically, let's not be luddites or,
more precisely, let the luddites among us do as they wish, but don't
force the rest of us to live what would now be an ascetic lifestyle to
us.

> expensive connections and/or with limited disk space.

Disk space is hardly limited any more.  And the initial copy is the
only slow operation.  Also, I seem to remember something about how the
network can optimize things when using HTTP...  But I digress from my
point above.

Nico
--

---

From: Tony Li <[email-masked]>
Date: Thu, 30 Jun 2011 12:04:59 -0700

> force the rest of us to live what would now be an ascetic lifestyle to

There's an old saying that all problems in computer science can be solved with another level of indirection. Is there some way that we could apply that here?

Tony

---

From: Julian Reschke <[email-masked]>
Date: Fri, 01 Jul 2011 00:11:17 +0200

> There's an old saying that all problems in computer science can be solved with another level of indirection. Is there some way that we could apply that here?

Except the one of having too many indirections.

---

From: Dzonatas Sol <[email-masked]>
Date: Thu, 30 Jun 2011 16:12:26 -0700

On 06/30/2011 03:11 PM, Julian Reschke wrote:
>> There's an old saying that all problems in computer science can be 
>> solved with another level of indirection. Is there some way that we 
>> could apply that here?
>> ...
>

What is expected with <!--1--2-->? An indirection in viral comments 
readability? "Ghost in the wires", misdirection.

-- 
--- http://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering
Ag-Biotech, Virtual Reality, Consultant