Subject: XML Validity vs Extensibility

From: "Lisa Dusseault" <[email-masked]>
Date: Fri, 1 Dec 2000 08:37:04 -0800

Changing the syntax of the PROPFIND and PROPPATCH in the manner you suggest
would break current implementations.  This would change the bits on the wire
for all PROP* messages, even for ones which include no extensions.

While it's arguable that it would be nice to have required valid XML when
RFC 2818 was written, changing the bits on the wire for Proposed Standard
protocols isn't done except for worse problems (or much smaller changes)
than this.

As far as validating XML to check interoperability, feel free:  RFC2518
includes a DTD, and implementors are welcome to check their implementation
against the DTD by validating XML bodies.

The cat's already out of the bag and can't be put back in.

Lisa

> -----Original Message-----
> From: "James J. Hunt" <[email-masked]>
> To: [email-masked]
> CC: [email-masked], [email-masked]
> Message-ID: <"i41fs3.ira.0039901:001201.012827"@ira.uka.de>
>
> Dear Colleges,
>
> >From the experience we have gathered trying to implement the DeltaV
> protocol, we have some basic comments and suggestions to make before the
> end of the last call period for the DeltaV protocol.  We regard these
> issues to be important for the future development of DeltaV and WebDAV.
> This message is cross posted to the WebDAV mailing list because the main
> points pertain as much to WebDAV as to DeltaV.
>
> Valid XML should be used in WebDAV instead of simply well formed XML
> ====================================================================
>
> Using simply well formed XML misses half the benefit of XML: the
> automatic detection of syntactic errors.  This is the best way to insure
> that a protocol stays open and correct.  Any client or server can be
> easily tested for at least syntactic conformance.  This does not means
> that every transaction must be validated, but ensuring that it can is
> important.
>
> On the WebDAV list, there has been some discussion about validating XML
> for WebDAV, for example, see
> http://lists.w3.org/Archives/Public/w3c-dist-auth/1999OctDec/0242.html.
> There seemed to be some agreement that the current WebDAV specification
> uses a syntax that is well-formed, but is not suitable for XML
> validation, and that it would not be too hard to make WebDAV to conform
> to XML validation rules.  However, some participants did not agree on
> the advantages of validation.
>
> Interoperability is one of the key requirement on Internet protocols for
> supporting a heterogeneous network.  This holds true for the WWW and its
> protocols including HTTP, WebDAV, and DeltaV.  As WebDAV matures several
> extension protocols are emerging, among them the bindings and
> redirect-reference protocols, DeltaV, the access control lists
> specification, and the advanced collections protocol.  All these
> extension protocols contribute to the XML syntax of WebDAV.  Moreover,
> it is quite likely that vendors will introduce proprietary extensions to
> the XML syntax.  For an idea of the extent to which this can occur, one
> just need look at the source code of the DeltaV specification at
> http://www.webdav.org/deltav/protocol/draft-ietf-deltav-versioning
> -10.5.htm
> and notice all the tags inserted by Word that are not part of the HTML
> specification.
>
> All in all, the XML syntax of WebDAV can no longer be viewed as being
> simple.  Validation can help us to identify bugs in WebDAV applications.
> Actually, this has already help us to find some bugs in the WebDAV
> specification itself (e.g. WebDAV issues DTD_BOOBOO, MISSING_NS_SPEC,
> cp. http://www.ics.uci.edu/pub/ietf/webdav/protocol/issues.html).
> Validation may also force implementors to handle XML more carefully and
> thus improve reliability.
>
> Note that we do not want to force a WebDAV implementation to validate
> XML.  We only suggest letting the implementor decide if and when to use
> validation.  Currently, this is not possible without modifying an
> existing XML parser or writing a non-conforming XML parser.  Therefore,
> we strongly suggest to fix the WebDAV specification and its extension
> protocols like DeltaV to conform to the XML specification--including
> XML validation.  Then an implementor may choose when to apply
> validation.
>
> Another problem is to track version information about the protocols.
> The DAV header as returned from an OPTIONS request (e.g.: "DAV: 1, 2,
> version-selector-checkout") does not help at all.  However, when using
> a DTD, the client and the server could agree on a protocol revision
> (or otherwise stop communication with an appropriate error message or
> log) by using the DTD's header information.
>
>
> Making WebDAV properties suitable for XML validation
> ====================================================
>
> WebDAV currently expresses resource properties through a variety of
> tags, for example:
>
> <D:displayname xmlns:D="DAV:">Hello, world!</D:displayname>
>
> In this example, the name of the property is "DAV:displayname", and its
> value is "Hello, world!".  This approach has some disadvantages.
> First of all, if you want to list property names only, you currently
> have to do something like:
>
> <D:displayname xmlns:D="DAV:"/>
>
> But, in an XML DTD, an element can not be defined to have empty
> content and non-empty content at the same time.  Hence, the way WebDAV
> currently handles properties is not suitable for validating XML.
> This problem has already been discussed as anomaly in the DAV:prop
> element usage (see
> http://lists.w3.org/Archives/Public/w3c-dist-auth/2000JanMar/0029.html).
>
> The simple solution here is to model the property name as an element
> attribute value or as PCDATA of a generic tag for all properties:
>
> Solution using XML attributes:
> - ------------------------------
>
> <!ELEMENT property (property-name | property-value)* >
> <!ELEMENT property-name EMPTY>
> <!ELEMENT property-value ANY>
> <!ATTLIST property-name
>           name  CDATA   #REQUIRED>
> <!ATTLIST property-value
>           name  CDATA   #REQUIRED>
>
> Example:
>
> <D:property xmlns:D="DAV:">
>   <D:property-name name="displayname"/> <!--specifying property
> name only-->
>   <D:property-value name="source">      <!--specifying property name &
> value-->
>     <D:href>http://www.foo.bar</D:href>
>   </D:property-value>
> </D:property>
>
> Alternative solution without use of XML attributes:
> - ---------------------------------------------------
>
> <!ELEMENT property (property-name, property-value?)* >
> <!ELEMENT property-name (#PCDATA)>
> <!ELEMENT property-value ANY>
>
> Example:
>
> <D:property xmlns:D="DAV:">
>   <D:property-name>displayname</D:property-name> <!--property name only-->
>   <D:property-name>source</D:property-name>      <!--property
> name & value-->
>   <D:property-value>
>     <D:href>http://www.foo.bar</D:href>
>   </D:property-value>
> </D:property>
>
>
> DTD for DeltaV
> ==============
>
> Having all of the above said, DeltaV should, similarly to WebDAV, of
> course specify an adequate DTD for all elements introduced in DeltaV.
> The DTD need not be sent with every message.  A reference to a URL at
> www.webdav.org would be sufficient.
>
> Labels in DeltaV
> ================
>
> One of the recent changes in DeltaV was to remove labels from the core
> versioning part and put them into the options part of the protocol.
> We strongly suggest to undo this change.  Even if there exist two or
> three revision control systems that do not use labels, labels are
> essential for identifying sets of associated files in a repository of
> versions.  And, actually, they are really easy to implement
> (especially on top of a WebDAV implementation that requires built-in
> mechanisms for property storage).
>
> Sincerely,
> James J. Hunt
> Jürgen Reuter
>
> P.S. I will be at the IETF meeting in San Diego from the evening of the
>      10th to the early morning of the 15th to discuss these issues.
>      --- JJHunt
>
> P.P.S I will send a second mail pertaining to smaller points we have
>       discussed here that concern only DeltaV.
>       --- Jürgen Reuter
>

---

From: Juergen Reuter <[email-masked]>
Date: Mon, 04 Dec 2000 12:08:44 +0100

> Changing the syntax of the PROPFIND and PROPPATCH in the manner you
> suggest would break current implementations.  This would change the bits
> on the wir= e for all PROP* messages, even for ones which include no
> extensions.

Not necessarily.  Just think of HTTP itself.  In HTTP/0.9, there were no
message headers (see Simple-Request/Response syntax in RFC1945).  It was
HTTP/1.0 that introduced message headers along with the HTTP-Version
field.  If a client does not provide the HTTP-Version field, the server
can assume the client to be an HTTP/0.9 client, and the server does not
serve any message header.  Most implementations still maintain this
behaviour:

# telnet www.webdav.org 80
Trying 206.54.183.234...
Connected to www.webdav.org.
Escape character is '^]'.
GET /
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
...
</html>
Connection closed by foreign host.
#

The HTTP-Version field is essential for the behaviour.  In HTTP/1.1, for
example, the Host message header became mandatory and thus effectively
introduced another incompatibility.  It is again the HTTP-Version field
that lets the client and server smartly agree on a protocol version
without breaking existing client or server implementations.

The same trick can be done with WebDAV:  The only thing to do is to
introduce -- for example -- a DAV-Version message header.  If a client
does not submit such a header, the server can assume that this client
only understands the current format of WebDAV as of RFC2518.  Vice versa,
if a client submits a DAV-Version header, but the server does not respond
with a DAV-Version header, the client can assume that the server only
understands the current format of WebDAV as of RFC2518.  This behaviour
will not break current implementations, but allows you to introduce new
features or new syntax that otherwise would break current implementations.

Actually, I think something like a DAV-Version message header is really
missing in WebDAV.

It is somewhat a pity that WebDAV has to undergo the same lessons that
HTTP already had.  It would be nice to collect such general lessons about
writing protocols and provide them for authors of other protocols --
maybe as an informational RFC for protocol authors?

> protocols isn't done except for worse problems (or much smaller changes)

In my opinion, using invalid XML is a really severe problem that probably
will lead to interoperability.  Already now there are frequent postings
on this list about WebDAV implementations that produce faulty output. 
And with all those extensions that are currently being developed, I think
the problems are getting worse.  And as far as I know, there is still no
reference implementation of WebDAV (though at least mod_dav may have good
chances to become a reference server implementation some day).  So, you
probably really want to check for correctness were possible.

> includes a DTD, and implementors are welcome to check their
> implementation against the DTD by validating XML bodies.

No, that is not true.  Did you try to apply the "DTD"?  I did.  RFC2518
contains lots of examples of invalid XML.  Most of them have been
discussed on this list during 1999.  WebDAV's DTD has a rather
informational than specifying character.  Currently, you are not able to
do any validation that conforms to the XML specification.  Actually,
WebDAV's "DTD" itself has some bugs in it; it does not conform to the XML
specification and can not be used for validating.  I finally used a
non-validating XML parser and manually added some methods that do some
kind of non-conforming validation.  Very ugly!

> The cat's already out of the bag and can't be put back in.

This sort of argumentation leads to those forever lasting nasty problems.
Just think of the 8+3 filename format that was invented around 1974 for
CP/M and which is -- for backwards compability -- still used today.  Just
a few days ago, I once again had to manually rename files that I had
copied from a linux workstation (not connected to the internet) to a
solaris machine, using floppy discs, because the upper/lower case
handling of those 8+3 filenames still does not properly work.

In other words: let us remove those nasty bugs from RFC2518 better now
than in 30 years!

Bye,
     Juergen