Subject: Re: [Tools-team] XML validity when submitting

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 01 Dec 2004 10:15:02 +0100

this points out a generic problem with tools: version skew.

If the DTD/schema is widened to allow previously disallowed content, the 
widening has to happen first at the accepting site, then at the users.

If it is narrowed, the narrowing has to happen first at the users, then at 
the accepting site - this is harder.

When switching from a DTD to a schema, which (I believe) checks different 
things, we have even more trouble.

And when the validator tools start checking things that were not checked 
before, we will (as Bill noted) have even more fun.

My conclusion: We have to allow upload of XML that does not pass the 
current set of validations - there are too many cases where there is good 
reason to submit such. And in the deadline crunch, there's no time to go 
fight with the implementor of the checker.

We should, however, give very plain warning - "the following errors have 
been detected - do you still want to upload?" - the same way I think we're 
wanting to do for id-nits errors.

My opinion.

                   Harald

--On 30. november 2004 09:12 -0700 Alex Rousskov 
<[email-masked]> wrote:

> On Mon, 2004/11/29 (MST), <[email-masked]> wrote:
>
>> Is it enough for the file to be valid XML, or does it have to pass
>> the DTD?
>
> It has to pas the DTD, IMO.
>
> My primary concern are [future] tools that use XML libraries that will
> not  deal with broken XML. I suspect that some libraries will not deal
> with XML  that does not pass DTD validation. Also, there is little
> pragmatic  difference between enforcing basic XML validity and DTD-based
> validity. So  we should get all the benefits for about the same price...
>
> You should probably test against the latest DTD in [1], not the one in
> RFC  2629.
> [1] http://xml.resource.org/authoring/draft-mrose-writing-rfcs.html
>
> Alex.
>
> _______________________________________________
> Tools-team mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/tools-team
>
>

_______________________________________________
Tools-team mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tools-team

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 01 Dec 2004 08:38:32 -0700

On Wed, 2004/12/01 (MST), <[email-masked]> wrote:

> If it is narrowed, the narrowing has to happen first at the users, then  
> at the accepting site - this is harder.
>
> When switching from a DTD to a schema, which (I believe) checks  
> different things, we have even more trouble.

While I agree that the changes in DTD and/or schema need to be carefully  
propagated, I do not see a serious problem or trouble here. What  
specifically are you worried about?

> before, we will (as Bill noted) have even more fun.

If by "fun" you mean higher quality drafts, then I agree. If you mean  
complains from users, then we need to distinguish short-term growth  
problems from long-term inconveniences.

> current set of validations - there are too many cases where there is  
> good reason to submit such. And in the deadline crunch, there's no time  
> to go fight with the implementor of the checker.

I am afraid I do not see the "too many cases" you are referring to. I  
cannot think of a good reason to submit broken XML, regardless of the  
deadline proximity: If the draft submission tool rejects your XML, then  
(a) fix XML and resubmit, (b) use the manual Secretariat route, or (c) do  
not submit XML. What's wrong with that approach?

> been detected - do you still want to upload?"

Except for the manual Secretariat route, I do not see the point in  
uploading broken XML. Do you?

Thanks,

Alex.

P.S. If NEWTRK IDS make it, we will need a different set of acceptance  
criteria for them, provided their primary format remains XML. However, I  
would still argue that broken XML should not be auto-posted, _especially_  
when XML is the primary format! Accepting something that IETF tools cannot  
grok would cause nothing but trouble. Rejecting it should not cause any  
trouble, except in a few special cases that can be handled manually.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 01 Dec 2004 16:53:15 +0100

> not submit XML. What's wrong with that approach?

If you don't see it, look a little harder, please....

"too many cases" are for instance:

- XML validator has a bug, and rejects some valid XML with valid schema
- XML validator checks schema version 1.4, submitter conforms to 1.5
- XML validator checks schema version 1.5, submitter conforms to 1.4
- XML is produced by tool "X", which is emitting version 1.3-compliant code
- XML is produced by tool "Y", which puts -- inside an XML comment
- XML is produced by tool "Z", which produces an XML weirdness that was not 
checked for before, but the validator got upgraded 2 hours ago

And think through yourself as the submitter in all of these cases, 5 
minutes before the deadline.

And look at Bill's numbers - half the drafts that people have so far 
submitted, BELIEVING they were conformant, were not.

The result of refusing to accept XML that does not pass the validator at 
the crunch is "not submit XML" - and IMHO, it is *harder* to generate any 
sensible metadata from "no XML" than from "XML that did not pass the 
checker".

                     Harald

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 01 Dec 2004 09:42:41 -0700

> - XML validator has a bug, and rejects some valid XML with valid schema

Use the manual route or do not submit XML (and consider reporting a bug).
Once the toolset is stable, the bugs will be exceptions that Secretariat  
can dial with manually.

> - XML validator checks schema version 1.5, submitter conforms to 1.4

XML validator must use the IETF-approved schema(s). Non-conformant formats  
should not be accepted (for the reasons I stated earlier). If XML  
validator does not use the IETF-approved schema(s), it is a validator bug  
(see above). And, again, since schema/DTD changes would be quite  
infrequent, I do not see a reason to optimize for a rare conflict.

> - XML is produced by tool "Z", which produces an XML weirdness that was  
> not checked for before, but the validator got upgraded 2 hours ago

I do not think the tool producing XML matters in this context. I can only  
see two really different cases among all the cases you listed above:
	- validator is correct
	- validator is buggy

Once stable, the validator will be correct virtually all the time and  
exceptions can be dealt with manually. Why optimize for exceptions?

And, of course, upgrading validator just before the IETF deadline is a bad  
idea.

> minutes before the deadline.

If I have 5 minutes before the deadline, I will drop XML format or use the  
manual route (depending on the error message), no problem.

> submitted, BELIEVING they were conformant, were not.

Bill's numbers are irrelevant in this particular case because there was no  
validation. Once there is validation and the validator is stable, the  
numbers will be very different. Short-term (while the validator is not  
stable), there is no compelling reason to accept invalid drafts either. In  
fact, errors will prompt more folks to complain to validator implementor  
(and/or Marshall) and bugs will be fixed.

> the crunch is "not submit XML" - and IMHO, it is *harder* to generate  
> any sensible metadata from "no XML" than from "XML that did not pass the  
> checker".

Not for an automated tool that cannot "see" invalid XML (the parser simply  
returns an error, refusing to return an XML "tree" or a DOM object of  
sorts). And, again, all of your cases are exceptional, uncommon scenarious  
once things become stable (which should not take that long since we  
already have semi-stable tools to be re-used here).

Thanks,

Alex.

---

From: stanislav shalunov <[email-masked]>
Date: 01 Dec 2004 13:07:32 -0500

Harald Tveit Alvestrand <[email-masked]> writes:

> The result of refusing to accept XML that does not pass the
> validator at the crunch is "not submit XML" [...]

[Some violent agreement.]

I would further strike ``at the crunch'' from this sentence.  XML is
currently an optional format.  If people are kind enough to provide
the source that they use to create the draft, we should be grateful,
not ridiculously picky.

If a consumer of this format -- human, automated, bacterial, or alien
-- doesn't like some particular kind of the format -- not passing
validation, not satisfying the latest DTD, not satisfying the previous
version of the DTD, having lines that are too long, not having a
newline at the end of file, using splice commas or split infinitives,
or having too many capital letters -- the consumer can always choose
not to consume the source.  No such consumer choice is there if XML is
not there.

This would change if XML submission were required by policy.  Part of
this policy would then say what requirements the XML files need to
satisfy.  The usefulness of such a policy can be debated (probably not
here), but lack of its current existence cannot be.

When is the presence of XML confusing and harmful?  When it's not
actually the source.  When it's, say, the source of a different I-D or
something else entirely.  If a common version of xml2rfc takes the XML
and produces the plain text version from it, and it matches the
submitted plain text version, we know that the XML version is, indeed,
the source code for the draft.  It should then be accepted.  (Unless,
again, the policy is changed.)

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

Democracy is a form of government that substitutes election by the
incompetent many for appointment by the corrupt few.  -- G. B. Shaw

---

From: Larry Masinter <[email-masked]>
Date: Wed, 01 Dec 2004 11:45:37 -0800

If we accept XML files that don't pass validation, we could publish
the XML errors as well as the XML draft in the (XML) drafts index.

> something else entirely. 

Should this be a criteria for acceptance? 

Larry

---

From: stanislav shalunov <[email-masked]>
Date: 01 Dec 2004 15:25:03 -0500

> the XML errors as well as the XML draft in the (XML) drafts index.

Yes.

> Should this be a criteria for acceptance? 

Quite possibly.  At the very least, the provided XML should at least
be acceptable to xml2rfc as a valid input.

"The power of accurate observation is commonly called cynicism by
those who have not got it."			-- G. B. Shaw

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 01 Dec 2004 16:17:13 -0700

> the source that they use to create the draft, we should be grateful,

I do not consider a simple DTD or schema validation being "ridiculously  
picky". If we have to resort to undefined adjectives, I would say that  
accepting invalid XML is a "ridiculously careless" idea.

> not to consume the source.  No such consumer choice is there if XML is

In real world, the consumer will be forced to deal with broken XML. Tools  
we be written to convert broken drafts to valid ones. Other tools will not  
use the "best" XML library (based on some more important criteria) because  
it cannot deal with broken XML. These are _long-term_ overheads I am  
trying to avoid.

How many browsers do you know that reject broken HTML? Why do you think  
W3C has not used "be tolerant to crap" policy when defining newer formats  
such as XHTML? Because long-term overheads of supporting broken stuff are  
far more than short-term overheads of fixing it.

> here), but lack of its current existence cannot be.

I disagree that whether XML is required or not is important here. You can  
use the same line of arguments even if XML is required -- if xml2rfc  
works, we do not care about whether XML is valid.

> again, the policy is changed.)

While this is a good requirement in principle, I am not sure we should  
mandate xml2rfc as the only XML processor. There are already other tools  
that handle XML drafts. They can produce slightly different text, given  
the same XML, right? And if we accept the fact that there are many valid  
renderings of XML, I am not sure the submission tool will be able to  
validate whether the text matches the XML...

In the long run, a perhaps better approach would be not to accept plain  
text if XML is supplied (or not accept plain text at all). However, we  
cannot do that in the foreseeable future except for new things like NEWTRK  
ISDs.

Alex.

P.S. I wish XML would be some binary format that would not have a false  
implication that it is easy to parse regardless of structure validity!

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 01 Dec 2004 16:20:07 -0700

>> the XML errors as well as the XML draft in the (XML) drafts index.

And what would be the point? The authors already know their XML is buggy,  
why do we need to publish the errors? I would understand a boolean  
valid/invalid flag -- that can be used by tools to skip invalid XML. Who  
would use the published errors and how?

> At the very least, the provided XML should at least

I agree with that.

Alex.

---

From: stanislav shalunov <[email-masked]>
Date: 02 Dec 2004 17:03:53 -0500

"Alex Rousskov" <[email-masked]> writes:

> And what would be the point? The authors already know their XML is
> buggy,  why do we need to publish the errors? I would understand a
> boolean  valid/invalid flag -- that can be used by tools to skip
> invalid XML. Who  would use the published errors and how?

I don't feel strongly one way or the other.  I was simply agreeing
that we could, in fact, publish the errors without doing any obvious
harm.  It might be useful to publish at least a valid/invalid flag, so
that XML-consuming tools might be able to more easily avoid dealing
with bad XML.

This message is designed to be viewed in boustrophedon.

---

From: stanislav shalunov <[email-masked]>
Date: 02 Dec 2004 17:20:03 -0500

"Alex Rousskov" <[email-masked]> writes:

> In real world, the consumer will be forced to deal with broken XML.
> Tools we be written to convert broken drafts to valid ones.  Other
> tools will not use the "best" XML library (based on some more
> important criteria) because it cannot deal with broken XML.  These
> are _long-term_ overheads I am trying to avoid.

I understand your rationale and would agree with your position if it,
combined with the fact that XML is optional, would not lead to a
situation where users are discouraged from sharing the source (the
actual source, good or bad) of their drafts.

> How many browsers do you know that reject broken HTML?

A few complain.  I'd like the various ``web designers'' (hate that
term) to use them, but haven't had a lot of success even at my
employer.

> Why do you think W3C has not used "be tolerant to crap" policy when
> defining newer formats such as XHTML?  Because long-term overheads
> of supporting broken stuff are far more than short-term overheads of
> fixing it.

I understand your rationale.  You're preaching in the wrong place.  It
should be xml2rfc that need to reject bad XML.

> I disagree that whether XML is required or not is important here.
> You can use the same line of arguments even if XML is required -- if
> xml2rfc works, we do not care about whether XML is valid.

Here's why the optionality of XML is crucial: if XML is a required (or
perhaps the only accepted) format, then being picky about it within
reason is acceptable, as authors would face a different alternative:
fix XML or do not publish the draft.  As they evidently find some
communications value in I-D publication (otherwise they wouldn't have
bothered writing the draft), they would have an incentive to fix their
XML.

With optional XML, the obvious alternative is different: fix XML or do
not upload XML.  The second branch is way easier.

> While this is a good requirement in principle, I am not sure we
> should mandate xml2rfc as the only XML processor.  There are already
> other tools that handle XML drafts.  They can produce slightly
> different text, given the same XML, right?  And if we accept the
> fact that there are many valid renderings of XML, I am not sure the
> submission tool will be able to validate whether the text matches
> the XML...

Can we do some fuzzy matching or at least metadata matching?  (We
already need to be able to extract metadata from plain text.)  This
would prevent some errors.

> In the long run, a perhaps better approach would be not to accept
> plain text if XML is supplied (or not accept plain text at all).

I agree.

> However, we cannot do that in the foreseeable future except for new
> things like NEWTRK ISDs.

Why?  (Plain text versions generated on the server would then of
course be published as the official representation.)

> P.S. I wish XML would be some binary format that would not have a
> false implication that it is easy to parse regardless of structure
> validity!

Then it wouldn't be a source format, and we would not be having this
discussion.  (That's not to say that we wouldn't be having the same
discussion about the source code used to produce the binary format...)

---

From: Larry Masinter <[email-masked]>
Date: Fri, 03 Dec 2004 00:18:23 -0800

> > invalid XML. Who  would use the published errors and how?

Internet drafts are revised and updated, and are usually produced by
a community.  If there's a draft with bad XML and we make it
clear that there's an error in the XML, maybe someone will figure
out how to fix the XML and help the poor author with the next
version.

If we just post the bad XML blindly, or with just a comment
back to the author, maybe the author won't do anything.

I think if we go to the trouble of generating a status report
on the submitted document, posting the result will help,
both with peer pressure and community involvement.

I was thinking of something simple, e.g., the internet drafts
index could have 

   _draft-ietf-blah-bogey-02.txt_
   _draft-ietf-blah-bogey-02.xml_
  (errors detected, see _draft-ietf-blah-bogey-02.xml-error-log_)

Larry

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 03 Dec 2004 11:01:18 +0100

> implication that it is easy to parse regardless of structure validity!

been there, done that. ASN.1.

---

From: stanislav shalunov <[email-masked]>
Date: 03 Dec 2004 10:49:07 -0500

>   (errors detected, see _draft-ietf-blah-bogey-02.xml-error-log_)

I thought that any other possible warnings (non-XML-related) would be
published then, too.  The arguments about peer pressure and whatnot
apply to them equally well, don't they?

This message is designed to be viewed at an angle of 45 degrees.

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 03 Dec 2004 17:11:43 +0100

>   (errors detected, see _draft-ietf-blah-bogey-02.xml-error-log_)

This looks pretty good to me.  It could be part of the WG draft
pages as well as the internet-drafts index.

So, what about this as a resolution:

  In stage (a), we only handle text format submissions.

  In stage (b), we also accept other formats, including XML.  It is
  required that the XML can be converted to text by xml2rfc running
  under the submission tool.  In this phase, we do not require that
  it is well-formed and validates, but we require that the submission
  tool does validation and provide feedback, indicating the
  validation errors found.  We also provide a separate validation-
  only page.

  In stage (c), we require that the tool has the capability to reject
  XML sources that doesn't validate, and also that the tool has the
  option of only doing meta-data extraction from xml, not from text.
  This makes it possible to make the policy decision of making it
  easier to submit with validated XML source rather than with text
  only.

Personally, I think that given good validation feedback on submission
and having an online validation service will make most people pretty
quickly fix their xml errors.  I see how meticulous a lot of people
seem to be about fixing not only the errors given by idnits, but also
the warnings.  And having a notice about xml errors detected
alongside with the xml code associated with a draft on e.g. the
WG draft pages will probably lead to a lot of fixing and polishing.

	Henrik

---

From: Larry Masinter <[email-masked]>
Date: Fri, 03 Dec 2004 11:20:52 -0800

> apply to them equally well, don't they?

Agree

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 03 Dec 2004 20:50:56 +0100

> apply to them equally well, don't they?

Makes sense to me.  (and that's what I currently do with the idnits
results on the draft status pages at http://ietf.levkowetz.com/drafts/
currently.

	Henrik

---

From: stanislav shalunov <[email-masked]>
Date: 03 Dec 2004 14:52:15 -0500

>   validation errors found.  We also provide a separate validation-

What about making sure xml2rfc output matches the submitted plain
text?

I see a few ways of dealing with this issue.  In all cases below, XML,
if it is submitted, is required to be a working input to xml2rfc.

1. Always require plain text submission.  Do not do any checks to make
   sure the XML version is related to the plain text version.

2. Always require plain text submission.  Check that the optional XML,
   if submitted, converts with xml2rfc to exactly the same file as the
   submitted plain text.

3. Always require plain text submission.  Check that the optional XML,
   if submitted, has the same metadata as the plain text version.

4. Always require plain text submission.  Check that the optional XML,
   if submitted, has the same fuzzy checksums as the text file.  (To
   avoid too much development work, the fuzzy checksums could be
   similar to those used by, e.g., spam detectors.  Or they might be
   even more lax.)

5. Accept exactly one of plain text and XML.  If XML is submitted,
   generate the published plain text from XML with xml2rfc.

Personally, I most favor 5 [*] and least favor 1 and 2.  I don't think
there's a very large practical difference between 3 and 4.

                                                        --Stas

[*] If it is true that people often edit the plain text manually after
    generating it with xml2rfc or that many ways of converting XML to
    I-D format exist, then 3 or 4 should probably be used.  I don't
    know if the antecedent of the preceding sentence is true.

A fool's brain digests philosophy into folly, science into superstition,
and art into pedantry.  Hence University education.       -- G. B. Shaw

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 03 Dec 2004 21:45:02 +0100

> What about making sure xml2rfc output matches the submitted plain

I think there should be some kind of verification, but I'm undecided
on which one of your options I'd prefer.

>    generate the published plain text from XML with xml2rfc.

I'd add
  3f.Always require plain text submission.  Check that the optional
     XML, if submitted, has the same metadata as the plain text
     version.  Require that a diff between the two has less
     than N% diffing lines.

  6. Accept either plain text, plain text and XML, or XML only.
     If XML is submitted, 
     a) process according to 1 above
     b) process according to 2 above
     c) process according to 3 above
     d) process according to 4 above
     e) process according to 5 above
     f) (NEW) Require that extracted metadata from text and xml
	agree, and require that a diff between the two has less
	than N% diffing lines.

>From my experience, it's pretty usual with some slight fixup of the
text after it comes out of xml2rfc.  Some people post-process the 
table of contents, for instance, to make it look better. Others
tweak the format of the line giving the draft name to have the name
within "<" and ">".  I've seen other minor tweaks, too.  Out of all
options above, I'd prefer (in order) 6f, 3f, 3, I believe.

	Henrik

---

From: stanislav shalunov <[email-masked]>
Date: 03 Dec 2004 16:25:50 -0500

Henrik Levkowetz <[email-masked]> writes:

> >>   In stage (b), we also accept other formats, including XML. [...]
> > What about making sure xml2rfc output matches the submitted plain text?

Option 6 seems to be buggy (allows XML only on input, then doesn't say
what happens).

7. Accept at least one of XML and plain text.
   define fuzzy(x, y)
   if XML and xml2rfc fails then reject XML
   if XML and not plain text then generate plain text with xml2rfc
   else if XML and plain text then
     if metadata(plain text) != metadata(XML) \
        or not fuzzy(xml2rfc(XML), plain text) then
       reject submission
   else {plain text and not XML.  Do not use XML and proceed.}

Then we need to define fuzzy().  Percentage of differing lines being
below a threshold would work just as well as anything else to weed out
unrelated documents.  Weeding out versions that differ in a very minor
way without weeding out valid manual edits seems to be a task that
would require a human.  If manual fixes are common, we need to get the
human out of the loop, so we have to live with the small potential for
disagreeing XML and plain text.

Given that this potential remains by necessity, should we even bother
comparing anything but metadata?  Comparing metadata should exclude
accidental submission of unrelated documents, should it not?

8. Accept at least one of XML and plain text.  Generate plain text
   with xml2rfc if it isn't submitted.  If both are submitted, ensure
   that metadata are the same in the two files.

---

From: Larry Masinter <[email-masked]>
Date: Fri, 03 Dec 2004 13:52:24 -0800

Accept either plain text, plain text and XML, or XML only.

 If XML only is submitted, generate plain text using xml2rfc.
 If both are submitted, perform a diff between the generated
   text (from xml2rfc) and the submitted plain text.
   Include, in the internet drafts index, a report on whether
     the metadata appears to be different (WARNING) and how
     many lines are different (if over 5%, use WARNING).

This is a policy where we allow people to submit things
that we think might be broken, but we make it clear.

(By the way, Henrik: I think it would be more helpful
in your page of interest drafts if you didn't have a 
[nits] link if there were no nits detected. Put [no nits]
with no hyperlink in the index.

--------------------------

Consider the possibility of adding another "round trip"
when there are new significant warnings (see below for def).
The round trip asks the author "The following warnings
were detected for this draft, are you sure you want to
submit it?" (or some similar wording), and waits for
a confirmation before actually putting it into the internet
drafts directory.

A "new significant warning" consists of warnings that
we consider significant (invalid XML, differing metadata, etc.)
for
    New -00 drafts
    new versions of drafts, where there is an old version
      which didn't have significant warnings (this allows
      an author to ignore warnings once and have it stick).

This is under the theory that we allow people to submit
things that we think are broken, as long as they are aware
that we think they're broken.

Larry

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 03 Dec 2004 23:33:32 +0100

> that we think might be broken, but we make it clear.

Right, it seems we're in agreement here

> with no hyperlink in the index.

Good point.

I've changed the [nits] to [no nits] when there are no nits - but
it can still be of interest to see which version of the checker
generated the report, and what it said, so I'm keeping a link
to the report.

Yes, this is nice.  It adds a little added cost for submitting
something which is broken, and it also permits becoming gradually
stricter over time, and have that communicated back to the authors.

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 03 Dec 2004 23:45:35 +0100

> Option 6 seems to be buggy (allows XML only on input, then doesn't say

Eh?  6 had 6 different sub-options, and this is definitely not true
of all of them.

>    else {plain text and not XML.  Do not use XML and proceed.}

This in my understanding matches my 6.f for a particular definition
of fuzzy(x, y): 
      $(diff -C0 x y | wc -l) is greater than N% of $(wc -l x)

> accidental submission of unrelated documents, should it not?

I'd say we could use this criterium, and let a human (the
secretariat) handle cases where the diff was too large, but the
author maintained that the XML and text belonged together.

>    that metadata are the same in the two files.

I could go with this too, although I'd prefer 7 / 6f .

	Henrik

---

From: stanislav shalunov <[email-masked]>
Date: 03 Dec 2004 18:36:17 -0500

> Eh?  6 had 6 different sub-options, and this is definitely not true

OK, 6e will work.  That still leaves 6 buggy.  Let's not quibble.

> This [option 7] in my understanding matches my 6.f for a particular
> definition

Yes.  (Or, rather, matches the intent of 6f -- I tried to reformulate
6f to be unambiguous.)

---

From: stanislav shalunov <[email-masked]>
Date: 03 Dec 2004 18:40:14 -0500

Henrik Levkowetz <[email-masked]> writes:

> [an extra round-trip for serious warnings] also permits becoming
> gradually stricter over time [...]

In what way?  Incrementing the number of round-trips each month?
(You're being humorous, right?)

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 04 Dec 2004 01:36:27 +0100

> (You're being humorous, right?)

:-)  No, I actually meant that as time goes on, and conformance
levels rise, it is possible to change what is considered 'serious
warnings' so that we raise the bar, and gradually approach the
point where we can go to full validation required for acceptance.

	Henrik

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 15 Dec 2004 17:15:19 -0700

>    that metadata are the same in the two files.

I am not sure whether it is funny or sad, but the above resolution is  
exactly what the ID Submission draft version 06 already documents:

     The Check action preprocesses XML draft source (if any), generates
     plain text format (if needed, see R70), stores submitted draft (all
     formats) in the staging area, and then extracts meta-data and validates
     each format (R6/a).
	...
     It is an error to submit a draft which has neither plain text nor XML
     sources format (R68/a). XML source is acceptable without accompanying
     plain text only if the Toolset successfully generates a draft in plain
     text format from the XML source, as a part of the processing step
     documented below (R69/b).
	...
     All available meta-data entries must match across all
     submitted draft formats (R18/a). For example, if the
     interpreter managed to extract draft title from plain text and
     PDF format, both titles must match. This requirement prevents
     accidental submission of mismatching formats.	

I will add a requirement to generate a warning (and, eventually, a  
complete diff) when the produced plain text does not match the submitted  
one (even if metadata matches). I will also add a stage-3 suggestion to  
add a smart fuzzy match function to compare texts, but I hope that by the  
time we get to stage 3, there will not be enough plain text submissions to  
worry about them.

Thank you,

Alex.

P.S. Note that xml2rfc changes relatively often, without  
stable/development categories for releases. The Secretariat would have to  
deal with upset folks that use the latest and greatest xml2rfc only to get  
warnings from the submission tool. The tool might even try a few recent  
xml2rfc versions when comparing with plain text. This is not a big deal,  
of course, and I do not think we have to document it.

---

From: stanislav shalunov <[email-masked]>
Date: 15 Dec 2004 20:26:42 -0500

"Alex Rousskov" <[email-masked]> writes:

> [T]he above resolution is exactly what the ID Submission draft
> version 06 already documents:

If Henrik is OK with it, it works for me.

> stable/development categories for releases. The Secretariat would have
> to  deal with upset folks that use the latest and greatest xml2rfc
> only to get  warnings from the submission tool. The tool might even
> try a few recent  xml2rfc versions when comparing with plain
> text. This is not a big deal,  of course, and I do not think we have
> to document it.

I think that's a trick useful enough to document in the draft as a
suggestion for the implementor.

This message is designed to be viewed upside down.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 16 Dec 2004 06:24:30 +0100

Hi,

> exactly what the ID Submission draft version 06 already documents:

Oh, I think it indicates that we can come to similar conclusions by
different roads, which is good :-)

...
> time we get to stage 3, there will not be enough plain text submissions to  

Sounds good to me.

...
> of course, and I do not think we have to document it.

I agree with Stas' comment about this being worth an implementation
note.

	Henrik

---

From: Bill Fenner <[email-masked]>
Date: Mon, 20 Dec 2004 11:20:21 -0800

I wrote the lamest little validity checker in the world, at
http://rtg.ietf.org/~fenner/ietf/xml2rfc-valid/

I don't like xmllint's output, for example,

/tmp/xml2rfc-valid.cgi.q8Sqckpd/draft.xml:961: element front: validity error : Element front content does not follow the DTD, expecting (title , author+ , date , area* , workgroup* , keyword* , abstract? , note*), got (title author )

could be much nicer as "missing required element 'date'"...

I'm tossing over the idea of writing my own, since none of the validators
I've seen really do well at this.

  Bill