Subject: RE: [Tools-team] XML validity when submitting

From: Bill Fenner <[email-masked]>
Date: Thu, 18 Nov 2004 21:42:36 -0500

Larry,

  I'm sorry that you read "I think we should pander to those who don't
write valid XML".  What I meant was that people have their expectations
set by the tools that they use - for example, at the last meeting, the
secretariat rejected some I-D submissions that didn't use the boilerplate
that's in 1id-guidelines.txt, which Henrik's "idnits" tool doesn't check
for.  The user's expectation was, "I used this tool to check for problems,
and it didn't have any, so why did it get rejected?"

  I expect to see the same expectations set when the XML I-D submission
tool gets deployed - "the draft formatted when I ran xml2rfc, so why did
the web site reject it?".  Working towards setting expectations (and
in particular this expectation, since we know that it's likely to
actually happen) should be part of our charter - otherwise we're setting
ourselves up for failure.

  Bill

_______________________________________________
Tools-team mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tools-team

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 18 Nov 2004 20:04:05 -0700 (MST)

On Thu, 18 Nov 2004, Bill Fenner wrote:

> I expect to see the same expectations set when the XML I-D 
> submission tool gets deployed - "the draft formatted when I ran 
> xml2rfc, so why did the web site reject it?".  Working towards 
> setting expectations (and in particular this expectation, since we 
> know that it's likely to actually happen) should be part of our 
> charter - otherwise we're setting ourselves up for failure.

I agree. IMO, we should discuss this issue with Marshall (and xml2rfc 
community) and go from there. Any other suggestions? Do we at least 
have rough consensus that invalid XML should not be accepted 
long-term?

Alex.

---

From: Bill Fenner <[email-masked]>
Date: Thu, 18 Nov 2004 23:20:40 -0500

>Do we at least have rough consensus that invalid XML should not be
>accepted long-term?

I'd bet we have rough consensus that invalid XML should not be
accepted, ever.  I just think that we should have a plan for
dealing with the user expectations.

  Bill

---

From: Larry Masinter <[email-masked]>
Date: Thu, 18 Nov 2004 23:17:40 -0800

Marshall produced xml2rfc to solve a particular problem: let
people write documents in XML and then produce the actual
documents that people read: ASCII, nroff, and HTML.

We're trying to solve a slightly different problem, so we
shouldn't expect an exact match.

I see no reason to expect xml2rfc to do validation. Personally,
I prefer Julian Reschke's .xslt transformation to turn the
.xml into nice-looking HTML, but I imagine most xslt tools require
valid XML.

I imagine we could offer other transformation tools beyond
Marshall's if they were interesting and valuable.

We could set expectations by telling people other ways to validate
their XML. It's not a complex DTD.

Larry

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 19 Nov 2004 10:14:59 +0100

> have rough consensus that invalid XML should not be accepted 

I think that:

* Invalid XML should not be in the archives, long-term

* Short-term, rejecting sources which xml2rfc accepts is
  probably counter-productive

* We should talk to Marshall about the problem - any long-
  term resolution will have to be based on the tools people
  use giving them proper feedback, resulting in well-formed
  XML

* There may be an interim solution (haven't verified this)
  in running the HTML tidy program on the XML source to clean
  up the kind of errors which xml2rfc accepts - or maybe a
  custom script could do it.

	Henrik

---

From: Larry Masinter <[email-masked]>
Date: Fri, 19 Nov 2004 08:22:14 -0800

Consider the long list of available XML validation tools:

http://xml.coverpages.org/check-xml.html

(or Google for "XML validation tool", which is how I found this).

>   probably counter-productive

If we're able to give clear diagnostics about why the
XML fails some other validation, I don't think that would
be counter-productive. We could also point people at
XML validation tools and ask them to check their documents
before they submit, or we could offer a "checker" tool
which would allow authors to check their documents
*before* they were ready to send them in.

>   use giving them proper feedback, resulting in well-formed

I don't think we should rely on Marshall to provide the
validation tool. Marshall's tool is fine for what Marshall
intended it for.

>   custom script could do it.

"HTML tidy" only cleans up HTML, not XML.

Larry
-- 
http://larry.masinter.net

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 19 Nov 2004 17:57:31 +0100

> *before* they were ready to send them in.

This might be a workable road, but I'm not quite happy with it.

> validation tool. Marshall's tool is fine for what Marshall

Mmm, the most important point I was trying to make was 

   "						- any long-
   XML"

> "HTML tidy" only cleans up HTML, not XML.

Oh?  Are you quite sure?  I've used it to validate and
massage XML...

	Henrik

---

From: stanislav shalunov <[email-masked]>
Date: 19 Nov 2004 13:01:00 -0500

Henrik Levkowetz <[email-masked]> writes:

> * We should talk to Marshall about the problem - any long- term
>   resolution will have to be based on the tools people use giving them
>   proper feedback, resulting in well-formed XML

This matches my thinking.

>   custom script could do it.

A tool for fixing up XML would be nice.  I'm not sure, though, that it
needs to run on IETF I-D submission page.  Having it as part of
xml2rfc (which could then be more strict about what it accepts) would
seem like a better option.

That way, the IETF would not become responsible for maintaining a tool
that people will invariably find bugs in (the bugs could be not fixing
something that needs to be fixed, fixing it wrongly, or ``fixing''
something that's perfectly fine already).

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

Just my 0.086g of Ag.

---

From: stanislav shalunov <[email-masked]>
Date: 19 Nov 2004 13:16:00 -0500

Addendum: Just to be clear, I would favor rejecting XML failing
validation if XML were a required format for I-D submission or the
only format.  (I do not believe that being fussy about optional
formats is productive.)

---

From: "Alex Rousskov" <[email-masked]>
Date: Fri, 19 Nov 2004 11:18:36 -0700

>    use giving them proper feedback, resulting in well-formed

I agree, but have to note that the ID Submission Toolset might have enough  
power to force folks to use validation tools (or demand built-in  
validation from Marshall). For example, if IETF sees that XML sources are  
becoming increasingly useful, we can stop auto-posting drafts without  
them, essentially forcing most IETFers to use xml2rfc _and_ submit valid  
XML.

Whether and when the Toolset should do that is an open question.

Alex.

---

From: "Alex Rousskov" <[email-masked]>
Date: Fri, 19 Nov 2004 11:24:26 -0700

> seem like a better option.

It would be a better option for IETF, of course. However, Marshall does  
not work for us and may not want to spend his free time modifying the  
xml2rfc-embedded XML parser because he had more than enough troubles with  
it already. Perhaps, if we have strong consensus that xml2rfc must only  
accept valid XML, IETF can pay Marshall or his helper to do the work?

Alex.

---

From: Bill Fenner <[email-masked]>
Date: Fri, 19 Nov 2004 14:04:24 -0500

BTW, I've found that "xmllint" (which comes with libxml2) tends
to give excellent error messages, at least for the cases that
I've come across so far:

wonk.badxml:2: parser error : StartTag: invalid element name
<title>I'm going to barf < less than</title>
                          ^
wonk.badxml:3: parser error : xmlParseEntityRef: no name
<organziation>This also has a bare &.</organization>
                                    ^
wonk.badxml:3: parser error : Opening and ending tag mismatch: organziation line 3 and organization
<organziation>This also has a bare &.</organization>
                                                    ^

Especially if they're postprocessed to give some more info to an xml-newbie
(like the "no name" error could easily turn into "reminder: use &amp; to
represent &"), I think this kind of validation would be very useful.
xmllint also allows specifying a DTD to validate against.

  Bill