Subject: Re: rfc2518-bis-04 issues (part 1)

From: Jason Crawford <[email-masked]>
Date: Wed, 30 Jul 2003 12:20:23 -0400

I don't have the spec handy, so I can only comment on the items that 
include
enough comment...

> 03-C03
> 
> 4.4: "Note that the use of a new top-level URI identifier as a namespace 
is
> considered by many to be a bad thing?"
> 
> [as of draft 04 this now reads: "Note that "DAV:" is a top-level URI
> identifier that was defined
>    solely to provide a namespace for WebDAV XML elements and property
>    names.  This practice is discouraged in part because registration of
>    top-level URI identifiers is difficult. "DAV:" was defined as the
>    WebDAV namespace before standard best practices emerged, and this
>    namespace is kept and still used because of significant existing
>    deployments, but this should not be emulated. "]
> 
> Rewrite as:
> 
> "Note that both defining a new URI scheme just for the purpose of
> identifying protocol elements, and using just the scheme name as a 
namespace
> name is to be considered a bad practice, and should not be copied".
> 

The previous text seems clearer.  I'd not rewrite this.

> 03-C05
> 
> 4.5: "The value of a property appears inside the property name element. 
The
> value may be any text, including valid XML.  When the value is 
structured
> as XML, namespaces that are in scope for that part of the  XML document
> apply within the property value as well, and MUST be  preserved in 
server
> storage for retransmission later. Namespace prefixes need not be 
preserved
> due to the rules of prefix declaration in XML."
> 
> 1)    I think this needs to rephrased to use proper XML terminology, 
also
> 2)    I think that namespace prefixes within the property value do need 
to 
> be
> round.tripped.
> 
> Proposal:
> 
> "The value of a property appears inside the property name element and 
may be
> any kind of well-formed XML content, including both text-only and mixed
> content. When the property value contains further XML elements, 
namespaces
> and namespace prefixes that are in scope for that part of the XML 
document
> apply within the property value as well, and MUST be preserved in server
> storage for retransmission later."
> 
> [Issue 2 still needs to be resolved, the current text says: "Namespace
> prefixes need not be preserved due to the rules of prefix declaration in
> XML."]

I have no opinion on prefix preservation. 

> 03-C12:
> 
> 8.1.1.: "Some of the following new HTTP methods use XML as a request and
> response format.  All DAV compliant clients and resources MUST use  XML
> parsers that are compliant with [REC-XML]."
> 
> Add "?and [REC-XMLNS]".
> 
> We also need allow servers and clients to rejects a certain set of
> request/response that are indeed well-formed, in particular:
> 
> -     when it exceeds some predefined size or
Sounds fine, but it's just one of several reasons to reject
a request.  If possible, I'd like to declare these as out of
scope as long as we're clear how the server should respond
to problems of this class.  Is it clear how server writers
should respond to problematic situations that we don't
explicitly mention?

> 03-C17:
> 
> 8.1.5.: "Because clients may be forced to prompt users or throw away 
changed
> content if the ETag changes, a WebDAV server MUST not change the  ETag 
(or
> getlastmodified value) for a resource when only its property values 
change."
> 
> Some servers do, and I don't think we can change that. Therefore I think
> this change at least needs explicit consensus on the mailing list.

I vote for the wording that is in there.  I think we've already reached 
consensus
that changing property values should not be changing etags.

> 03-C21:
> 
> 8.2.: "Note that 'allprop' does not return values for all properties."
> 
> Change to:
> 
> "Note that 'allprop' does not return values for all live properties."

All dead properties must be returned?  I didn't pick that up in our 
discussions.

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 30 Jul 2003 20:15:54 +0200

> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of Jason Crawford
> Sent: Wednesday, July 30, 2003 6:20 PM
> To: Julian Reschke
> Cc: [email-masked]
> Subject: Re: rfc2518-bis-04 issues (part 1)
>
>
>
include
> > considered by many to be a bad thing…"
> >
> > identifying protocol elements, and using just the scheme name as
anamespace
> The previous text seems clearer.  I'd not rewrite this.

It may "seem" clearer, but it isn't. Mainly

1) usage of the term "top-level URI identifier" -- this isn't documented
anywhere. We're talking about the URI scheme name, and thus should use that
term.

2) the issues are exactly what I wrote: a) defining a new URI scheme without
actually needing one, and b) using just the scheme name as namespace URI
(which is illegal according to RFC2396).

Therefore, this section should be rewritten accordingly.

The
structured
server
preserved
also
> > 2)        I think that namespace prefixes within the property value do
need to be
> > round.tripped.
> >
> > Proposal:
> >
> > "The value of a property appears inside the property name
> element and may
> be
> > and namespace prefixes that are in scope for that part of the
> XML document

It was pointed out that the prefixes are irrelevant, *unfortunately* this is
not true, as they also may appear in attribute values (for instance in XSLT
and XML Schema datatypes).

> > Add "…and [REC-XMLNS]".

First of all, we should clarify that servers may reject requests for a
number of reasons we don't specify, such as to prevent DoS attacks.

In this particular case, I think the spec should actually *recommend* these
requests because they are a known XML protocol vulnerability.

> > content if the ETag changes, a WebDAV server MUST not change
> the  ETag (or

a) When and where?

b) Just for etags or for the last-modified date as well?

I just want to make sure that everybody is aware that this requirement will
make existing servers non-compliant, and it's unclear whether they'll be
able to become compliant. So we'll either see servers not being upgraded to
RFC2518bis, or servers claiming compliance to RFC2528bis but breaking it. I
don't see this as improvement.

> All dead properties must be returned?  I didn't pick that up in our

It never was discussed. RFC2518 guarantuees this and there never has been
any discussion about changing this (why?).

Julian

--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Elias Sinderson <[email-masked]>
Date: Wed, 30 Jul 2003 13:48:22 -0700

Comments inlined below...

Cheers,
Elias

Jason Crawford wrote:

> > 4.4: "Note that the use of a new top-level URI identifier as a 
> namespace is
> > considered by many to be a bad thing..."
> >

The existing text seems a bit 'heavy' (for lack of a better word), while 
the suggested replacement seems a bit 'light'. At the least, it makes 
sense to include some justification for the approach that was taken 
while making the case for others to not follow in suit. Further, saying 
something is 'bad practice' or discouraged without explaining why simply 
begs the question...

> > 03-C05
> >
> > 4.5: "The value of a property appears inside the property name 
> element.  The
> > 1)        I think this needs to rephrased to use proper XML 
> terminology, also
> > 2)        I think that namespace prefixes within the property value 
> do need to
> > be
> > round.tripped.
> >
> > Proposal:
> >
> > "The value of a property appears inside the property name element 
> and may be

My feeling is that while a server is allowed to rewrite XML as long as 
it preserves the semantics, dropping namespace prefixes is likely to 
cause problems. For example:

Storing
<Foo:bar>
</Foo:bar>
as
<Foo:bar/>
to save space is perfectly fine, while storing the above as
<bar/>
is not. The issue at hand is that namespaces serve to qualify the 
semantic interpretation of a given XML element, hence dropping that 
qualification can only lead to interoperability problems between clients 
which respect or, alternately, expect namespaces. Without the namespace 
qualifier, a client may not be able to know what is meant by an element. 
I would be happy to construct a more meaningful example to illustrate 
this, but I hope the above is sufficient. This is an important issue 
that we need to reach a concensus on.

> > Add "...and [REC-XMLNS]".
> >
> Sounds fine, but it's just one of several reasons to reject a request. 
>  If possible, I'd like to declare these as out of  scope as long as 
> we're clear how the server should respond to problems of this class. 
>  Is it clear how server writers should respond to problematic 
> situations that we don't explicitly mention?

I agree, adding the suggested text is a good idea. Re: Jason's comment, 
I think the best we can do is define a common response mechanism, as 
what qualifies as exceeding 'some predefined size' and what constitutes 
a denial of service attack will likely change from one server to another 
and over time as server hardware improves.

> > content if the ETag changes, a WebDAV server MUST not change the 
>  ETag (or
> I vote for the wording that is in there.  I think we've already 
> reached consensus that changing property values should not be changing 
> etags.

I also believe that there was already concensus re: property values not 
affecting ETags... Since doing a PUT of a resource doesn't affect 
properties and a GET doesn't retrieve properties, the ETag should not 
change when properties on a resource do. That is to say that ETags 
should only be modified when the enitity, itself, is modified (hence the 
name entity tag). We may want to consider, however, introducing an 
equivalent property tag, or PTag, which could be used to determine if 
properties on a given resource have changed. An argument with a (set of) 
use case(s) would have to be developed to make this a compelling idea.

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 30 Jul 2003 23:16:20 +0200

> Behalf Of Elias Sinderson
> Sent: Wednesday, July 30, 2003 10:48 PM
> To: [email-masked]
> ...
> My feeling is that while a server is allowed to rewrite XML as long as it
> preserves the semantics, dropping namespace prefixes is likely to cause
> problems. For example:
>
> Storing
> <Foo:bar>
> </Foo:bar>
> as
> <Foo:bar/>
> is not. The issue at hand is that namespaces serve to qualify the semantic
> interpretation of a given XML element, hence dropping that
> qualification can
> only lead to interoperability problems between clients which respect or,
> alternately, expect namespaces. Without the namespace qualifier, a client
> may not be able to know what is meant by an element. I would be happy to
> construct a more meaningful example to illustrate this, but I
> hope the above
> is sufficient. This is an important issue that we need to reach a
> concensus
> on.

It get's worse. Consider

<D:prop xmlns:E="whatever"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:xs="http://www.w3.org/2001/XMLSchema"><E:foo><E:bar
xsi:type="xs:boolean">true</E:bar></D:prop>

Not storing the namespace mapping for "xs" will simply break the content,
even if the prefix isn't used for any child element.

> ...
> consensus that changing property values should not be changing etags.

Where and when?

> properties
> and a GET doesn't retrieve properties, the ETag should not change when
> properties on a resource do. That is to say that ETags should only be

That's simply not true. You're argueing from a world view where the server
stores properties and content as separate entities. However, it's perfectly
legal to have a server that

- extracts some properties upon PUT (for instance metadata from a WORD
document) and/or
- changes the content upon PROPPATCH of one of the aforementioned
properties.

I think that we can't and shouldn't make this behaviour illegal.

> modified when the enitity, itself, is modified (hence the name
> entity tag).
> We may want to consider, however, introducing an equivalent
> property tag, or
> PTag, which could be used to determine if properties on a given resource
> have changed. An argument with a (set of) use case(s) would have to be
> developed to make this a compelling idea.

Yes, but that's a separate discussion.

Julian

---

From: Elias Sinderson <[email-masked]>
Date: Wed, 30 Jul 2003 16:38:35 -0700

Julian Reschke wrote:

>>><Julian Reschke> 03-C17:
>>>
>>>8.1.5.: "Because clients may be forced to prompt users or throw away changed
>>>      
>>>
>>>content if the ETag changes, a WebDAV server MUST not change the  ETag (or
>>>      
>>>
>>>getlastmodified value) for a resource when only its property values change."
>>>      
>>>
>><Jason Crawford> I vote for the wording that is in there.  I think we've already reached consensus that changing property values should not be changing etags.
>>    
>>
>
><Julian Reschke> Where and when?
>
While I had agreed with Jason and thought that there had been some 
concensus on this, an exhaustive search of the mailing list archives has 
not backed this up. I is entirely likely that I was misremembering the 
consensus that had been reached regarding the inclusion of stronger ETag 
requirements in 2518bis.

That being said, however, there has been a fair amount of discussion 
around this very issue and the majority opinion seems to be in favor of 
this interpretation... (more below). As an aside, after reading through 
the previous discussions, I now firmly believe that the specification of 
a property tag, PTag, would put this issue soundly to rest. The idea has 
been brought up before, and I think it deserves consideration.

>><Elias Sinderson> I also believe that there was already concensus re: property values not affecting ETags... Since doing a PUT of a resource doesn't affect properties
>>and a GET doesn't retrieve properties, the ETag should not change when properties on a resource do. That is to say that ETags should only be
>>    
>>
><Julian Reschke> That's simply not true. You're argueing from a world view where the server stores properties and content as separate entities. However, it's perfectly
No, implementation issues aside (that is, I don't care how a server 
stores properties as it is orthogonal to my position), ETags as defined 
in RFC 2616 have nothing to do with WebDAV properties. The fact that 
resources have subsequently been defined as having properties doesn't 
change this fact. More to the point, the WebDAV specification should not 
change the definition or semantic properties of ETags.

Let us, however, consider existing implementations where resource 
properties are stored as the same blob of bits as the resource entity. 
This situation does not imply that changing a property of a resource 
entails changing it's ETag. In this case, the server must know how to 
seperate the two when responding to a GET or PROPFIND request and is 
perfectly capable of modifying the ETag without taking property values 
into consideration. The fact that RFC 2518 doesn't treat this matter 
directly has certainly complicated the issue for implementors, as 
evidenced by the different approaches that server implementations have 
taken in generating ETags.

The two points you raise above are red herrings, if you ask me. In the 
first case you have stipulated that the client has done a PUT which 
affects some properties. This is, as you identify, a valid case for 
generating a new ETag, but this is simply because the client has done a 
PUT, not because of any property value(s) changing (i.e. you have 
refuted my assertion that a PUT won't change properties, but not 
provided a compelling reason why changing properties alone should affect 
the ETag).

In the second case you state that modifying some special properties may 
change the content of the resource. Again, I think this is a valid case 
for generating a new ETag, but not due to the property changing but 
because the resource contents have changed. In short, this is similar to 
doing a PUT and modifying the resource itself although it is 
accomplished via a PROPPATCH instead.

In short, my argument is this: ETags should only change when the 
contents of a resource change (e.g. when the entity itself changes). 
Generating strong ETags doesn't place an undue burden on server 
resources and neither does detecting when the actual contents of a 
resource have changed by any means. There will be some frustrations in 
updating existing server code to be compliant if we take a firm stand on 
this issue but, in the long run, the benefits to clients far outweigh 
any drawbacks. The alternative is to have different server 
implementations of ETags and, as a consequence, balkanizing the 
interoperability of clients and servers.

Apologies for the long wind, I feel this particular issue is quite 
important and deserving of a thorough treatment on the road to consensus.

Cheers,
Elias

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 31 Jul 2003 09:13:00 +0200

> Sent: Thursday, July 31, 2003 1:39 AM
> To: [email-masked]
> No, implementation issues aside (that is, I don't care how a server stores
> properties as it is orthogonal to my position), ETags as defined
> in RFC 2616
> have nothing to do with WebDAV properties. The fact that resources have
> subsequently been defined as having properties doesn't change this fact.
> More to the point, the WebDAV specification should not change the
> definition
> or semantic properties of ETags.

I didn't propose that. However, if you look at RFC2616 you will note that it
really doesn't say that the ETag MUST NOT change unless the entity changes.
It only defines the other direction (the etag must change if the entity
changes).

> are stored as the same blob of bits as the resource entity. This situation
> does not imply that changing a property of a resource entails
> changing it's
> ETag. In this case, the server must know how to seperate the two when
> responding to a GET or PROPFIND request and is perfectly capable of
> modifying the ETag without taking property values into consideration. The
> fact that RFC 2518 doesn't treat this matter directly has certainly
> complicated the issue for implementors, as evidenced by the different
> approaches that server implementations have taken in generating ETags.

Actually, the case I raised is the one where properties are *extracted* from
the content, *stay* in the content and the content is *updated* when
PROPPATCH is applied.

For instance, (upon PUT) a server might use the HTML Meta tag to extract a
"foo:author" property and expose that as DAV property (making it available
for DAV:basicsearch...). Applying PROPPATCH to this property would affect
the content that you'll get upon a subsequent GET as well.

Now if you tell me that this behaviour is non-compliant we are in deep
disagreement about WebDAV. I understand that many see WebDAV simply as a
drop-in for filesystems or FTP. This is just one use-case, but there are
other scenarios that RFC2616 and RFC2518 explicitly allow.

> The two points you raise above are red herrings, if you ask me.
> In the first
> case you have stipulated that the client has done a PUT which affects some
> properties. This is, as you identify, a valid case for generating a new
> ETag, but this is simply because the client has done a PUT, not because of

No, it may affect any number of properties that are protected on that
particular server.

> any property value(s) changing (i.e. you have refuted my assertion that a
> PUT won't change properties, but not provided a compelling reason why
> changing properties alone should affect the ETag).
>
> change the content of the resource. Again, I think this is a
> valid case for
> generating a new ETag, but not due to the property changing but
> because the
> resource contents have changed. In short, this is similar to
> doing a PUT and
> modifying the resource itself although it is accomplished via a PROPPATCH
> instead.
>
> contents of
> a resource change (e.g. when the entity itself changes). Generating strong

OK, let's make that very clear: ETags SHOULD not change unless the content
changes. *However*, it's completely up to the server *which* methods affect
the content. For instance (in the case mentioned above), a PROPPATCH/set to
foo:author may affect the META/name=author tag in the HTML content that you
GET.

> ETags doesn't place an undue burden on server resources and neither does
> detecting when the actual contents of a resource have changed by
> any means.

I think that's a separate dicussion. We still don't have any feedback from
the two major WebDAV implementions (moddav and IIS) about whether they plan
to implement this according to RFC2518bis.

> There will be some frustrations in updating existing server code to be
> compliant if we take a firm stand on this issue but, in the long run, the
> benefits to clients far outweigh any drawbacks. The alternative is to have
> different server implementations of ETags and, as a consequence,
> balkanizing
> the interoperability of clients and servers.

Well the alternative obviously is not to change the requirement, and to keep
things as they are right now. I'm not convinced the current situation is so
bad we need to break compatibility with both RFC2518 and lots of existing
servers.

> Apologies for the long wind, I feel this particular issue is
> quite important
> and deserving of a thorough treatment on the road to consensus.

Yes.