Subject: [Tools-discuss] use of <?rfc include= ?> processing instrucrtion

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 14 Sep 2004 19:02:01 +0200

Hi,

    In the requirements for an I-D submission tool which the tools-team
are currently looking at, we would like to provide the possibility to
submit the xml source file for a draft, at the same time as the text
file is sumbitted.  However, there is one potential problem with this
which I've run into - the use of <?rfc include= ?> for references which
can't be found in the standard reference libraries, or the use of this 
PI to put together an xml file from fragments.

If non-standard references or other include files are missing, the
usefulness of having the xml file diminishes...

(This happened with an xml file I'd submitted to the RFC editor, 
and I had to send in a new version with the references inline for
the RFC editor to be able to use it.)

I can think of some different ways of handling this.  They might be
combined, too.  What I wonder is what your thoughts are on the different possibilities:

1) The submission tool tries to run xml2rfc on the file, and reports
   failure or success
   a) if failure, rejects the document, but accepts the text document
   b) if failure, rejects the whole submission
   c) does nothing, leaving a broken xml file in the archive

2) Use of the <?rfc include= ?> PI is deprecated in xml2rfc, and a
   warning issued.  This should probably be combined with one of
   1) above

3) Use of <?rfc include= ?> is removed from xml2rfc, which with time
   will make 1) less and less needed

4) Some other resolution?

	Henrik

_______________________________________________
Tools-discuss mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tools-discuss

---

From: Rob Austein <[email-masked]>
Date: Tue, 14 Sep 2004 13:44:36 -0400

>    c) does nothing, leaving a broken xml file in the archive

(b) would be ok if combined with (4).

>    warning issued.  This should probably be combined with one of

No (see below).

>    will make 1) less and less needed

No no no no no.  Eg, the DNSSECbis specs (three documents) are broken
up into 85 .xml files, some of which are generated automatically,
including a set which are generated by postprocessing the output of a
reference implementation of part of the protocol.

If <?rfc include="" ?> weren't present, we would have had to invent it.

> 4) Some other resolution?

Like using the (undocumented) xml2rfc feature that already solves
this? :)

$ xml2rfc head.xml concatenated.xml

Where "head.xml" is the top-level intput file.

How about:

 -Submitting- xml that uses <?rfc include ?> is not allowed.

 Authors who use <?rfc include ?> must preprocess their xml files
 through xml2rfc to produce a single concatenated file.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 14 Sep 2004 11:49:43 -0600 (MDT)

>  c) does nothing, leaving a broken xml file in the archive

My strong preference would be for (b). A person should remove 
offending XML file from the submission if they want option (a). Option 
(c) seems like a bad idea on general principles of consistency and 
correctness.

Note that the tool can offer the author to remove the offending format 
(to help with getting to option (a)), but removal must not be 
automatic, IMO.

>  will make 1) less and less needed

Both put carriage in front of the horse, IMO. Include statements are 
very useful to authors. Xml2Rfc needs better include support, not less 
include support!

> 4) Some other resolution?

Preprocess XML to include all includes and submit the processed 
version. Many folks, myself included, preprocess xml2rfc sources 
anyway, for many reasons.

An alternative is to submit a tarball with all included files.

Alex.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 14 Sep 2004 23:14:43 +0200

Alex Rousskov wrote:
> My strong preference would be for (b). A person should remove offending 
> XML file from the submission if they want option (a). Option (c) seems 
> like a bad idea on general principles of consistency and correctness.

+1.

> (to help with getting to option (a)), but removal must not be automatic, 
> IMO.
> 

Include statements are useful, but *this* mechanism is borked. For 
instance, a document using reference includes by definition won't 
validate according to the DTD, and we *do* want people to validate, right?

> Preprocess XML to include all includes and submit the processed version. 
> Many folks, myself included, preprocess xml2rfc sources anyway, for many 
> reasons.

I think that's the best approach. In particular, the *author* should 
have both control and responsibility of what the references actually 
contain, and the resultant text should be immutable independant of when 
the xml2rfc-to-text conversion is run.

> An alternative is to submit a tarball with all included files.

-1

Best regards, Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 14 Sep 2004 15:33:10 -0600 (MDT)

On Tue, 14 Sep 2004, Julian Reschke wrote:

>> Both put carriage in front of the horse, IMO. Include statements 
>> are very useful to authors. Xml2Rfc needs better include support, 
>> not less include support!
>

If <include>s violate DTD, then we should fix DTD, <include>s, or 
both. What I am trying to say is that a mechanism to include external 
fragments into drafts is useful and needed. If the current mechanism 
is broken, it should be fixed one way or another. This is unrelated to 
the ID submission tool functionality.

Regardless of the above, authors should submit [preprocessed] XML that 
does not need external fragments. We seem to agree on this. If xml2rfc 
provides such preprocessing abilities natively, that would make users 
even happier. If not, a simple XSLT/Perl/etc. script can do the job.

Alex.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 14 Sep 2004 23:54:30 +0200

Alex Rousskov wrote:

> If <include>s violate DTD, then we should fix DTD, <include>s, or both. 
> What I am trying to say is that a mechanism to include external 
> fragments into drafts is useful and needed. If the current mechanism is 
> broken, it should be fixed one way or another. This is unrelated to the 
> ID submission tool functionality.

Correct. We had a discussion about *how* this could be done a few months 
ago, but with no result. I guess it's a good moment to start it again :-)

> even happier. If not, a simple XSLT/Perl/etc. script can do the job.

We agree here.

I find it particulary important that the submitted XML file will have 
the same contents in the references no matter *when* it is run thhrough 
the xml2rfc formatter, thus inclusion during authoring is good, but 
implicit inclusion on submission (or even later) IMHO is a Bad thing.

Julian

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 14 Sep 2004 16:01:55 -0600 (MDT)

On Tue, 14 Sep 2004, Julian Reschke wrote:

> I find it particulary important that the submitted XML file will 
> have the same contents in the references no matter *when* it is run 
> thhrough the xml2rfc formatter, thus inclusion during authoring is 
> good, but implicit inclusion on submission (or even later) IMHO is a 
> Bad thing.

Agreed. Note, however, that if a reference does not change with time 
(e.g., a reference to an RFC), then it is not a problem, in principle, 
to include such references dynamically (so to speak).

A perhaps nastier problem is xml2rfc versioning. We all know that 
rendered drafts may look rather different when xml2rfc versions 
change. I am not sure how to combat that (short of the submitter 
specifying the xml2rfc version to be used, which is kind of ugly).

Thanks,

Alex.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 16 Sep 2004 00:27:03 +0200

Rob Austein wrote:
 > ...

No, you wouldn't have to. You can just use the standard XML mechanisms 
instead (as documented on xml.resource.org).

>  through xml2rfc to produce a single concatenated file.

Yep.

Julian

---

From: David Meyer <[email-masked]>
Date: Wed, 15 Sep 2004 15:29:52 -0700

>>  through xml2rfc to produce a single concatenated file.

	Yep, seems a completely reasonable resolution.

	Dave

---

From: Scott W Brim <[email-masked]>
Date: Wed, 15 Sep 2004 18:46:28 -0400

>  through xml2rfc to produce a single concatenated file.

Good.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 16 Sep 2004 07:18:49 +0300 (EEST)

> (b) would be ok if combined with (4).
[...]
>  through xml2rfc to produce a single concatenated file.

If this is the case, I'd vote that the xml2rfc interface at 
xml.resource.org be able to do the postprocessing, and also burp out 
.xml files as .xml output (with the content set to octet-string, so 
the browsers don't try to render it).

There are folks who *don't* want to install xml2rfc, and just use the
web interface effectively.  If we want to get xml2rfc to wider use,
the number of these people should rise [because getting started on xml
is so much simpler if you don't need to worry about xml editors,
xml2rfc or whatnot at all], not decrease.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 08:06:53 +0200

Hi Pekka,

> xml2rfc or whatnot at all], not decrease.

I don't see a problem with having the xml2rfc which is run as part of
the I-D submission validation use the same reference libraries as the
web interface at xml.resouce.org (I'm assuming here that it uses the
publicly available bibxml reference libraries).  In this respect, the
two could be pretty similar - the xml.resource.org doesn't handle the
general <?rfc include ?> case either (as far as I know).

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Thu, 16 Sep 2004 08:56:39 +0200

> the browsers don't try to render it).

That's what the Content-Disposition response header is for...

> xml2rfc or whatnot at all], not decrease.

You don't necessarily need xml2rfc for that. For instance, and XSLT 
processor will do that nicely as well.

Julian

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Sep 2004 09:05:31 -0600 (MDT)

On Thu, 16 Sep 2004, Henrik Levkowetz wrote:

> I don't see a problem with having the xml2rfc which is run as part 
> of the I-D submission validation use the same reference libraries as 
> the web interface at xml.resouce.org (I'm assuming here that it uses 
> the publicly available bibxml reference libraries).

Whether there is a problem depends on the reference libraries you are 
talking about. If reference libraries contain only constant entries 
such as RFC info, there is no big problem in using them.

If reference libraries consist of changing entries such as 
Internet-Draft info, then it is a bad idea to use them because the 
meaning of submitted XML sources would depend on the time those 
sources are processed. We must avoid such changes in meaning, IMO.

To summarize: If a draft cites something that may change with time, 
that citation has to be statically embedded and not dynamically linked 
when posted.

If we agree with the above rule, the only remaining question is 
whether it is worth making an exception for constant <include>s? Would 
not it be easier if all <include>s are treated uniformly (i.e., 
require preprocessing)? Most drafts probably contain non-constant 
<include>s anyway, right?

Thanks,

Alex.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 17:35:44 +0200

Alex Rousskov wrote:
>> publicly available bibxml reference libraries).
> 
> 
> talking about. If reference libraries contain only constant entries such 
> as RFC info, there is no big problem in using them.
> 
> meaning of submitted XML sources would depend on the time those sources 
> are processed. We must avoid such changes in meaning, IMO.

I'm not that sure about this.  The primary use of the xml would be for
the RFC editor, and what is important that the RFC editor can process
the xml file without problems.  The RFC editor will probably have it's
own version of the draft reference files otherwise resolved through the
bibxml libraries, so it would actually be better to keep the include
statement in that case, so no manual changes of the references would be
needed.

> To summarize: If a draft cites something that may change with time, that 
> citation has to be statically embedded and not dynamically linked when 
> posted.

No, my viewpoint is that the important thing is that the RFC editor can
process the draft without problems, and with optimal results for the 
RFC editor.  

If we additionally consider having an official archive of XML sources
of RF Cs, we'd want the XML source there to be the last one used by the
RFC editor, post-processed to do all the includes, not the one submitted
by the author, I think.

> If we agree with the above rule, the only remaining question is whether 
> it is worth making an exception for constant <include>s? Would not it be 
> easier if all <include>s are treated uniformly (i.e., require 
> preprocessing)? Most drafts probably contain non-constant <include>s 
> anyway, right?

It also happens that RFC references in the bibxml libraries are corrected.
No reason to prevent those from being pulled in by the latest version when
the RFC editor is processing the xml file.

	Henrik

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Sep 2004 10:07:23 -0600 (MDT)

On Thu, 16 Sep 2004, Henrik Levkowetz wrote:

> The primary use of the xml would be for the RFC editor,

I strongly disagree. It might look like that today when virtually no 
IETF tools make use of XML sources until the draft reaches RFC Editor. 
Even RFC Editor was not using XML sources until recently. Hopefully, 
this is a temporary situation and we should design for long-term use, 
when many IETF tools and IETFers will use XML drafts for many 
purposes. (If that is a permanent situation, we should not even bother 
with accepting XML draft sources in the first place.)

> and what is important that the RFC editor can process the xml file 
> without problems.

What is important, IMO, is that anybody and anything can process XML 
sources without problems (that includes RFC Editor, of course).

> The RFC editor will probably have it's own version of the draft 
> reference files otherwise resolved through the bibxml libraries, so 
> it would actually be better to keep the include statement in that 
> case, so no manual changes of the references would be needed.

IMHO, we should minimize RFC Editor work and minimize chances for 
incorrect edits. No manual changes of the references should be needed 
for the final version of the draft submitted to the RFC Editor. That 
includes embedded references. If that is not something we can get 
today, we should at least optimize in that direction instead of giving 
RFC Editor more manual responsibilities and things to worry about.

>> that citation has to be statically embedded and not dynamically 
>> linked when posted.
>
> No, my viewpoint is that the important thing is that the RFC editor 
> can process the draft without problems, and with optimal results for 
> the RFC editor.

I do not think your point contradicts the above rule, but I believe we 
should optimize for XML draft use within IETF at large, not just for 
the last stage when the draft reaches RFC Editor. There is a lot more 
value in XML drafts than just making minor last step editing easier. 
We are not talking about book publication business; we have a 
different environment where a very active/important document stage is 
_before_ it gets to the RFC Editor table.

> If we additionally consider having an official archive of XML 
> sources of RFCs, we'd want the XML source there to be the last one 
> used by the RFC editor, post-processed to do all the includes, not 
> the one submitted by the author, I think.

Agreed (and that does not contradict the above rule). However, we 
should also minimize the differences between author-submitted sources 
and what comes out of RFC Editor queue.

>> whether it is worth making an exception for constant <include>s? 
>> Would not it be easier if all <include>s are treated uniformly 
>> (i.e., require preprocessing)? Most drafts probably contain 
>> non-constant <include>s anyway, right?
>
> It also happens that RFC references in the bibxml libraries are 
> corrected. No reason to prevent those from being pulled in by the 
> latest version when the RFC editor is processing the xml file.

The reason is simplicity and consistency of the approach and the ease 
of use for humans and tools processing drafts. I suspect that RFC 
Editor software will be able to check embedded references against a 
library and make whatever fixes are needed. That can and should be 
automated. Installing a local RFC library for every human or tool just 
to compile/interpret a draft cannot be automated.

Alex.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 18:16:38 +0200

>> reference files otherwise resolved through the bibxml libraries, so it 
>> would actually be better to keep the include statement in that case, 
>> so no manual changes of the references would be needed.
> 
> 
> includes embedded references. If that is not something we can get today, 
> we should at least optimize in that direction instead of giving RFC 
> Editor more manual responsibilities and things to worry about.
> 

Since the format and text the RFC editor uses for references to drafts
is different from what is used for draft references in drafts, it was
exactly this reasoning which was behind my suggestion of leaving the 
standard library references in there, un-expanded, so the RFC editor 
work could be done automatically, instead of hand editing the inlined 
xml source to fix the references.

And the same goes for the case when the bibxml text for a referenced
RFC has been updated.  Doing the inlining before the submission will
leave the old text in the xml file, instead of the updated one which
would be generated if the includes to the standard references were
kept unexpanded.

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 18:21:11 +0200

...
> The reason is simplicity and consistency of the approach and the ease of 
> use for humans and tools processing drafts. I suspect that RFC Editor 
> software will be able to check embedded references against a library and 
> make whatever fixes are needed. That can and should be automated. 
> Installing a local RFC library for every human or tool just to 
> compile/interpret a draft cannot be automated.

Oh, I'd say that leaving the standard references in place, to be pulled
in by an unmodified xml2RFC, is clearly a simpler approach than building
a new tool for the RFC editor to fix up outdated reference texts.

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Thu, 16 Sep 2004 18:31:32 +0200

> xml source to fix the references.

If these are references to Internet Drafts, it IMHO makes more sense to 
have them properly tagged with

	<seriesInfo name="ID" .../>

in which case they can be trivially postprocessed/updated automatically.

As stated before, relying on IDs to be in any way stable is risky. For 
instance, consider the case of rfc2396bis as submitted (last ID) and how 
it will get published after the AUTH48 period.

> would be generated if the includes to the standard references were

No, it wouldn't. It would just require a different tool to update it.

Best regards, Julian

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Sep 2004 10:33:00 -0600 (MDT)

On Thu, 16 Sep 2004, Henrik Levkowetz wrote:

>>> The RFC editor will probably have it's own version of the draft reference 
>>> files otherwise resolved through the bibxml libraries, so it would 
>>> actually be better to keep the include statement in that case, so no 
>>> manual changes of the references would be needed.
>> 
>> incorrect edits. No manual changes of the references should be 
>> needed for the final version of the draft submitted to the RFC 
>> Editor. That includes embedded references. If that is not something 
>> we can get today, we should at least optimize in that direction 
>> instead of giving RFC Editor more manual responsibilities and 
>> things to worry about.
>
> Since the format and text the RFC editor uses for references to 
> drafts is different from what is used for draft references in 
> drafts, it was exactly this reasoning which was behind my suggestion 
> of leaving the standard library references in there, un-expanded,

If RFC Editor format and text differs from what everybody else is 
using, would not making that format and text the same be the Right 
Thing to do, long-term?

> so the RFC editor work could be done automatically, instead of hand 
> editing the inlined xml source to fix the references.

It seems to me that whether the references are embedded or not, the 
RFC Editor work we are talking about here can be done automatically.
Am I missing something?

> Oh, I'd say that leaving the standard references in place, to be 
> pulled in by an unmodified xml2RFC, is clearly a simpler approach 
> than building a new tool for the RFC editor to fix up outdated 
> reference texts.

Not so clear to me. IMO, it is easier to build one tool for the RFC 
Editor than to worry about includes in every tool that processes XML 
sources! It looks like we are optimizing different things and, hence, 
cannot agree on a cost function.

Alex.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 16 Sep 2004 18:34:06 +0200

> a new tool for the RFC editor to fix up outdated reference texts.

I'll be happy to provide that tool for no charge in case somebody asks 
for it.

Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 19:46:33 +0200

...
>> standard library references in there, un-expanded,
> 
> 
> If RFC Editor format and text differs from what everybody else is using, 
> would not making that format and text the same be the Right Thing to do, 
> long-term?

Not if there is a good reason why they are different in the first 
place.  In a draft, you want to point out the revision of the draft
you're referring to, but in an RFC the RFC editor takes out that, and
just refers to it as a document by title and authors (work in progress).

> It seems to me that whether the references are embedded or not, the RFC 
> Editor work we are talking about here can be done automatically.
> Am I missing something?

You can automate most things :-) And with new tools, the premises
changes.  What is in existence at this moment is xml2rfc and either
includes in place, or hand correcting, I think.  (Input on
preferences from the RFC editor would be welcome, though :)

>> pulled in by an unmodified xml2RFC, is clearly a simpler approach than 
>> building a new tool for the RFC editor to fix up outdated reference 
>> texts.
> 
> 
> sources! 

Eh?  We currently have tools (xml2rfc and rfc2629xslt) which work.
How can continuing using these in an optimal manner be more complex
than arranging things so you have to build a new tool?

Once a putative new tool exists, naturally the best way of arranging
things can be reconsidered.

It looks like we are optimizing different things and, hence, 
> cannot agree on a cost function.

Could be.

	Henrik

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Sep 2004 14:22:00 -0600 (MDT)

On Thu, 16 Sep 2004, Henrik Levkowetz wrote:

>>> drafts, it was exactly this reasoning which was behind my 
>>> suggestion of leaving the standard library references in there, 
>>> un-expanded,
>>
> you're referring to, but in an RFC the RFC editor takes out that, 
> and just refers to it as a document by title and authors (work in 
> progress).

I do not see the difference as far as XML source is concerned. Both 
draft-name-version and draft-name references can be expressed with the 
same XML element. Why does something needs to be taken out or changed 
in the XML source?

 	[N.B. and I do not understand why I would "want to point" to
 	something different than what my RFC will point to, but that
 	is not important for this discussion]

> includes in place, or hand correcting, I think.

Not sure what you mean, but please note that I am not advocating 
changing xml2rfc. I am suggesting that XML draft sources submitted for 
posting should be complete. This will allow new tools (such as XML 
draft indexers, searchers, and validators) to deal with a complete 
document and not worry about some external library they need to keep 
up-to-date (or, worse, in sync with draft posting date??) to correctly 
interpret a given draft.

>> Editor than to worry about includes in every tool that processes 
>> XML sources!
>

If your assumption is that xml2rfc is the only family of tools we will 
have, then there is no need to talk about <include> rules for draft 
submissions. My hope is that populating IETF repositories with XML 
sources will allow for many more useful tools to work with those 
sources, to provide extra services to Secretariat, IESG, IETFers, and 
3rd parties. We do not know what those tools will be exactly, but we 
do know that there is a lot of information in drafts that cannot be 
conveniently accessed without XML sources.

I want to design with that tool expansion in mind because I do not see 
the value of posting XML draft sources otherwise. Do you?

Alex.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 23:32:11 +0200

...
> posting should be complete. This will allow new tools (such as XML draft 
> indexers, searchers, and validators) to deal with a complete document 
> and not worry about some external library they need to keep up-to-date 
> (or, worse, in sync with draft posting date??) to correctly interpret a 
> given draft.

I thought we'd already settled this point.  I agree with the format
the xml source is archived in, but that can be accomplished as well
by expanding the standard lib includes when the sources are put into
the archives, while providing the RFC editor with a format which with
the tools given today gives the least manual work.

>> Eh?  We currently have tools (xml2rfc and rfc2629xslt) which work. How 
>> can continuing using these in an optimal manner be more complex than 
>> arranging things so you have to build a new tool?
> 
> 
> 3rd parties. We do not know what those tools will be exactly, but we do 
> know that there is a lot of information in drafts that cannot be 

Finally I understand where we've been talking past one another.  I'm
*not* saying that the archived sources should contain include statements
- I'm saying that it would be OK and probably beneficial to let the 
*submitted* sources contain includes for standard bibxml references.
These can be included a split second after the submission, thereby 
reducing an extra step for the submitter (expansion) and retaining
a format which might serve the RFC editor better (if this draft goes
on to the RFC editor).

	Henrik

---

From: Michael Richardson <[email-masked]>
Date: Thu, 16 Sep 2004 10:12:39 -0400

-----BEGIN PGP SIGNED MESSAGE-----

a) can the xml2rfc that is available have an option to warn/error if
   there are any rfc include's still there?

b) it seems that it should really have the option of inlining 
   references as well. My experience is that IDs usually don't have
   completely up-to-date .xml files, and one may well have non-IETF
   references that one would like to share.

] Train travel features AC outlets with no take-off restrictions|  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
] panic("Just another Debian/notebook using, kernel hacking, security guy");  [

---

From: Michael Richardson <[email-masked]>
Date: Thu, 16 Sep 2004 16:33:27 -0400

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Sep 2004 15:46:56 -0600 (MDT)

On Thu, 16 Sep 2004, Henrik Levkowetz wrote:

> Finally I understand where we've been talking past one another. 
> I'm *not* saying that the archived sources should contain include 
> statements - I'm saying that it would be OK and probably beneficial 
> to let the *submitted* sources contain includes for standard bibxml 
> references. These can be included a split second after the 
> submission, thereby reducing an extra step for the submitter 
> (expansion) and retaining a format which might serve the RFC editor 
> better (if this draft goes on to the RFC editor).

I have no objections about submission-time expansion as a submission 
automation step, though I suspect many drafts include more than just 
standard references and, hence, will not benefit from it. We already 
know many authors that use <include>s to include pieces of documents 
and/or co-author data. Such documents would need to be expanded before 
the submission.

I am not sure what you mean by "retaining a format which might serve 
the RFC editor better". Do you want to archive two XML sources 
(original and expanded)? Or do you want to archive one XML source 
(expanded) and give the RFC Editor another (from author's private 
repository, not expanded)? Something else?

Alex.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Sep 2004 23:58:02 +0200

Alex Rousskov wrote:
>> *not* saying that the archived sources should contain include 
>> statements - I'm saying that it would be OK and probably beneficial to 
>> let the *submitted* sources contain includes for standard bibxml 
>> references. These can be included a split second after the submission, 
>> thereby reducing an extra step for the submitter (expansion) and 
>> retaining a format which might serve the RFC editor better (if this 
>> draft goes on to the RFC editor).
> 
> 

Yes.  If you submit something which includes stuff which is not in the
standard libraries, xml2rfc processing at submit time will fail, and the
submission will be rejected.

> I am not sure what you mean by "retaining a format which might serve the 
> RFC editor better". Do you want to archive two XML sources (original and 
> expanded)? Or do you want to archive one XML source (expanded) and give 
> the RFC Editor another (from author's private repository, not expanded)? 
> Something else?

I was thinking of retaining the latest version un-expanded, for forwarding
to the RFC editor if this turned out to be the version of the draft which
was finally approved by the IESG.  All versions going into the archive 
would be expanded.

	Henrik

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 16 Sep 2004 21:15:28 -0600 (MDT)

On Thu, 16 Sep 2004, Marshall Rose wrote:

>
> On Sep 16, 2004, at 14:58, Henrik Levkowetz wrote:
>
> is this suggestion, coupled with an option to turn off <?rfc 
> include='...'?>, the final consensus?

I do not think there is any need to add an option to turn off support 
for <include>s in xml2rfc. I may be missing something, but I think the 
conclusions of this thread imply that no changes to xml2rfc are 
necessary (which makes your point of not having such discussions on 
the xml2rfc list stronger).

The only xml2rfc-related issue is an apparent need to better document 
the use of xml2rfc as a preprocessor for <include> PIs.

Thank you,

Alex.

---

From: Pekka Savola <[email-masked]>
Date: Fri, 17 Sep 2004 08:03:31 +0300 (EEST)

Only tools-discuss,

> submission will be rejected.

I'd expect that 95% of drafts *only* include the reference includes, 
so if the tool is able to use the standard libraries (ones at 
xml.resource.org), that's good enough.

If someone is using xml2rfc to build their documents from parts, 
that's just fine .. but they can then be expected to also concatenate 
the final file manually.

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 17 Sep 2004 09:03:57 +0200

Alex Rousskov wrote:
>>> was finally approved by the IESG.  All versions going into the 
>>> archive would be expanded.
>>
>>
> necessary (which makes your point of not having such discussions on the 
> xml2rfc list stronger).

Agreed.

	Henrik

---

From: Marshall Rose <[email-masked]>
Date: Thu, 16 Sep 2004 16:17:09 -0700

/mtr

ps: and next time some starts a thread like this, can we please have it 
solely on the tools-discuss list...

---

From: Carl Malamud <[email-masked]>
Date: Fri, 17 Sep 2004 10:59:45 -0700 (PDT)

Hi -

I must admit I'm not quite sure I followed all the nuances of this thread.
Is this a correct summary?

The i-d submission tool will accept xml in rfc2629 format, including
the use of the <?rfc include?> PI.  However, authors should not use
XML entities as an alternative include mechanism (or, if they do,
they should provide an "expanded" version in a single file instead).

In terms of distribution, <?rfc include?> PIs are kept in place
all the way through submission to the RFC Editor (at which point,
at least with current policy, it is turned into nroff and then
edited).  However, the archived version of the i-d will expand 
the include PIs.

Did I get it right?

Carl

[ Charset ISO-8859-1 unsupported, converting... ]

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 17 Sep 2004 20:42:23 +0200

Close, I believe:

> the use of the <?rfc include?> PI

... for sources found in the standard bibxml libraries maintained
by Marshall.  Using <?rfc include ?> with other sources will cause a
failure.

> .  However, authors should not use

We've not really discussed XML entities - if they can be resolved
without any additional files, by a standard xml2rfc (with web access),
I don't now see any reason to prohibit their use.  Otherwise, an 
expanded file will be needed also in this case.

> edited).  However, the archived version of the i-d will expand 

Yes, that's right.

	Henrik

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 17 Sep 2004 21:29:33 +0200

I'm still confused by the discussion.
The gist seems to be that, for some forms of external links, it is 
useful to keep both the expanded form (based on the time of insertion 
in the archives) and the reference information (which makes it trivial 
to update the expanded information later).
Why don't we do exactly that?

Gruesse, Carsten

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 17 Sep 2004 14:27:54 -0600 (MDT)

On Fri, 17 Sep 2004, Carsten Bormann wrote:

> useful to keep both the expanded form (based on the time of 
> insertion in the archives) and the reference information (which 
> makes it trivial to update the expanded information later).

I think that is what Henrik wants and that is what we are likely to 
codify. We seem to agree that archived drafts contain no include PIs, 
just the references. I am not sure where/how Henrik wants to keep PIs 
for the RFC Editor, but I am sure that will become clear when we start 
formally documenting this. I am guessing there will be a copy of draft 
sources (with PIs) kept in the repository, invisible until RFC Editor 
needs it.

PIs in question are pointers to references in Marhall's database. 
Personally, I see no compelling reason to keep those PIs for the RFC 
Editor, but I cannot convince Henrik that RFC Editor should adjust 
their tools to process true/actual references rather than pointers to 
some semi-official database.

Alex.

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 18 Sep 2004 00:50:43 +0200

> sources (with PIs) kept in the repository, invisible until RFC Editor 

Right.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 18 Sep 2004 08:33:44 +0200

On Sep 17 2004, at 22:27 Uhr, Alex Rousskov wrote:

> I am guessing there will be a copy of draft sources (with PIs) kept in 
> the repository

Keeping copies of documents from different stages of the processing 
sequence is certainly one way to do this (this = keep both linking 
information and expanded information), but it has all the problems of 
keeping redundant information in a database.

The more obvious way would be to keep both in one document, e.g. by 
changing the link representation from a PI to an element that, in the 
most sourcy form, just contains the linking information (typically as 
attributes), and in bound (archived) form contains both this and the 
expanded information (typically as element content).  The presence of 
the element itself (as opposed to its content) would have no processing 
semantics except in a processing step that resolves links.  In essence, 
the link element, when resolved, becomes a container for a cache, which 
can be re-resolved at any time (e.g., by the RFC editor).

Gruesse, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Sat, 18 Sep 2004 10:47:31 +0200

> can be re-resolved at any time (e.g., by the RFC editor).

As mentioned before, we already have this for IETF documents through the 
seriesInfo information.  This can be used to pull in an updated 
reference anytime without having to rely on an inclusion mechanism.

Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 18 Sep 2004 12:56:55 +0200

Julian Reschke wrote:
>> the element itself (as opposed to its content) would have no 
>> processing semantics except in a processing step that resolves links.  
>> In essence, the link element, when resolved, becomes a container for a 
>> cache, which can be re-resolved at any time (e.g., by the RFC editor).
> 
> 

As soon as the RFC editor has put something in place that lets him
easily update the references in a document, I see no need to keep 
an extra copy.  I don't, however, think it's right to build the
ID submission tool so that it forces this change on the RFC editor.

	Henrik

---

From: Alex Rousskov <[email-masked]>
Date: Sat, 18 Sep 2004 14:36:53 -0600 (MDT)

On Sat, 18 Sep 2004, Henrik Levkowetz wrote:

> Julian Reschke wrote:
>> Carsten Bormann wrote:
>> 
>>> The more obvious way would be to keep both in one document
>> 
>> As mentioned before, we already have this for IETF documents 
>> through the seriesInfo information.
>
> easily update the references in a document, I see no need to keep an 
> extra copy.  I don't, however, think it's right to build the ID 
> submission tool so that it forces this change on the RFC editor.

I agree with all of the above.

If RFC Editor does not adjust their tools, they can get the sources 
they need (with includes) directly from authors, just like they are 
doing today. In that case, we did not help the RFC Editor, but we 
helped many other IETFers and the Secretariat, so we are making good 
progress. We force no changes on the RFC Editor.

I suggest that we leave the choice to the RFC Editor. They know their 
needs and resources. They can ask us for advice if needed. Let's trust 
them to make the right decision:

 	1) keep the current status quo; get sources from authors

 	2) ask Secretariat to store a hidden copy with <includes>

 	3) adjust RFC Editor tools to use references

Can we declare "leave the choice to the RFC Editor" as rough 
consensus? Are there any serious flaws in that plan?

Thank you,

Alex.

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 18 Sep 2004 23:50:29 +0200

Alex Rousskov wrote:

> If RFC Editor does not adjust their tools, they can get the sources they 
> need (with includes) directly from authors, just like they are doing 
> today. In that case, we did not help the RFC Editor, but we helped many 
> other IETFers and the Secretariat, so we are making good progress. We 
> force no changes on the RFC Editor.
> 
> Can we declare "leave the choice to the RFC Editor" as rough consensus? 
> Are there any serious flaws in that plan?

In order to leave the choice with the RFC editor, the tool should keep
a copy with <?rfc include ?> available.  If we can agree on that, fine :-)

	Henrik