Subject: Stronger requirements for ETags

From: "Lisa Dusseault" <[email-masked]>
Date: Sat, 21 Jun 2003 16:27:15 -0700

Currently rfc2518bis-03 says:

"HTTP 1.1 suggests the use of the ETag header in responses to GET and PUT requests. Correct use of ETags is even more important in a distributed authoring environment, because ETags are necessary along with locks to avoid the lost-update problem.  A client might fail to renew a lock, for example when the lock times out and the client is accidentally offline or in the middle of a long upload.  When a client fails to renew the lock, it's quite possible the resource can still be relocked and the user can go on editing, as long as no changes were made in the meantime. ETags are required for the client to be able to distinguish this case. Otherwise, the client is forced to ask the user whether to overwrite the resource on the server without even being able to tell the user whether it has changed. Timestamps do not solve this problem nearly as well as ETags.

"WebDAV servers SHOULD support strong ETags for all resources that may be PUT.  If ETags are supported for a resource, the server MUST return the ETag header in all PUT and GET responses to that resource, as well as provide the same value for the 'getetag' property. 

"Because clients may be forced to prompt users or throw away changed content if the ETag changes, a WebDAV server MUST not change the ETag (or getlastmodified value) for a resource when only its property values change. The ETag represents the state of the body or contents of the resource. There is no similar way to tell if properties have changed."

Some history on how we got here:  RFC2518 doesn't require ETags, nor does HTTP.  But the overwrite problem isn't truly soluble without support for ETags (or some other untested new mechanism).  Thus, at the early-2003 interim meeting, the attendees agreed to strengthen requirements for ETags.

Draft 2518bis-02 required:

"WebDAV servers MUST support ETags correctly and MUST return the ETag header
in all GET and PUT responses. " 

However this was argued on the list to be too strong, and was weakened to the language already shown in -03.

So, what now?  Does the current language (03) make mod_dav or IIS (or other implementations) uncompliant with 03?  Is that a problem?  Would it be a bad idea to upgrade mod_dav or IIS to become compliant with RFC2518bis if it goes to RFC with these requirements?  I'd point out that there are other changes that already make servers slightly uncompliant with RFC2518bis -- e.g. the removal of specially-behaving lock-null resources makes existing servers supporting lock-null resources "uncompliant".  But being uncompliant with a spec that the server doesn't declare compliance to shouldn't be a problem!  A server will continue to be compliant with 2518 because that can never be replaced. Server implementors may upgrade their code/product to become compliant with RF2518BIS (whatever rfc number that turns out to be) and then declare compliance with that new, more stable standard.

IF the language must be weakened, is there any point to this at all?  I'm afraid we've got to make a hard choice.  Either we do something real to encourage ETag support, even though it makes existing servers "uncompliant".  Or we do nothing, and we leave the problem unmitigated.

As a server implementor, my personal opinion is that it's a good idea to make, and follow, stricter requirements for ETag support.  It's so helpful to clients and users (in terms of robust and reliable distributed authoring) that it's worth the cost.   

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Sun, 22 Jun 2003 16:06:55 +0200

> ...
>
> So, what now?  Does the current language (03) make mod_dav or IIS 
> (or other implementations) uncompliant with 03?  Is that a 

I think so, because both use a "promotion strategy", in which upon PUT only a weak ETag is returned, which, after some delay, is "promoted" into a strong ETag.

> problem?  Would it be a bad idea to upgrade mod_dav or IIS to 
> become compliant with RFC2518bis if it goes to RFC with these 

Of course that would be good, but these servers use this approach for a very good reason (being that they *can't' produce the strong variant at resource creation time).

> requirements?  I'd point out that there are other changes that 
> already make servers slightly uncompliant with RFC2518bis -- e.g. 
> the removal of specially-behaving lock-null resources makes 
> existing servers supporting lock-null resources "uncompliant".  

True.

> But being uncompliant with a spec that the server doesn't declare 
> compliance to shouldn't be a problem!  A server will continue to 
> be compliant with 2518 because that can never be replaced. Server 
> implementors may upgrade their code/product to become compliant 
> with RF2518BIS (whatever rfc number that turns out to be) and 
> then declare compliance with that new, more stable standard.

True as well.

My primary concern is that if we put in a change that isn't and furthermore *won't* be implemented by the two must widely used implementations, we have a severe problem with the spec. So it would be really good to understand the issue (maybe Greg Stein can say something about it) and find out whether at least Apache/mod_dav is going to be updated.

I personally see no use in the "promotion" strategy. If you can't produce a useful ETag upon PUT, don't.

> IF the language must be weakened, is there any point to this at 
> all?  I'm afraid we've got to make a hard choice.  Either we do 
> something real to encourage ETag support, even though it makes 
> existing servers "uncompliant".  Or we do nothing, and we leave 
> the problem unmitigated.

But then there's the risk that popular servers like mod_dav never will be updated to RFC2518bis conformance, just because it ask for things that can't be implemented. Or worse yet, the servers may claim compliance although they don't comply.

So I agree to any kind of text that explains *why* strong ETags are important, but I object to make them mandatory unless we can at least mod_dav support them properly.

> As a server implementor, my personal opinion is that it's a good 
> idea to make, and follow, stricter requirements for ETag support. 
>  It's so helpful to clients and users (in terms of robust and 
> reliable distributed authoring) that it's worth the cost.   

How do you calculate the cost in the case of mod_dav/fs, where clearly robust strong ETags can't be calculated without having additional metadata? Keep in mind that mod_dav is just an extension to the base HTTP server, such mod_dav's ETag handling must be identical to Apache/Httpd's.

Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Geoffrey M Clemm <[email-masked]>
Date: Mon, 23 Jun 2003 08:09:33 -0400

I agree with Julian that the most/best we can do is explain why
it is a SHOULD.  In particular, making it a MUST is very likely
to cause new client writers to write clients that will fail to
to work with a large number of existing servers, and that I think
would be a serious problem.

Cheers,
Geoff

[email-masked] wrote on 06/22/2003 10:06:55 AM:

> I think so, because both use a "promotion strategy", in which upon 
> PUT only a weak ETag is returned, which, after some delay, is 
> "promoted" into a strong ETag.
> 
> Of course that would be good, but these servers use this approach 
> for a very good reason (being that they *can't' produce the strong 
> variant at resource creation time).
> 
> My primary concern is that if we put in a change that isn't and 
> furthermore *won't* be implemented by the two must widely used 
> implementations, we have a severe problem with the spec. So it would
> be really good to understand the issue (maybe Greg Stein can say 
> something about it) and find out whether at least Apache/mod_dav is 
> going to be updated.
> 
> But then there's the risk that popular servers like mod_dav never 
> will be updated to RFC2518bis conformance, just because it ask for 
> things that can't be implemented. Or worse yet, the servers may 
> claim compliance although they don't comply.
> 
> So I agree to any kind of text that explains *why* strong ETags are 
> important, but I object to make them mandatory unless we can at 
> least mod_dav support them properly.
> 
> How do you calculate the cost in the case of mod_dav/fs, where 
> clearly robust strong ETags can't be calculated without having 
> additional metadata? Keep in mind that mod_dav is just an extension 
> to the base HTTP server, such mod_dav's ETag handling must be 
> identical to Apache/Httpd's.
> 
> Julian
> --

---

From: Elias Sinderson <[email-masked]>
Date: Tue, 24 Jun 2003 08:14:46 -0700

Hi,

Comments inlined below...

Lisa Dusseault wrote:

>[...]
>[...]
>So, what now?  Does the current language (03) make mod_dav or IIS (or other implementations) uncompliant with 03?
>
In this respect the -03 language doesn't make them uncompliant, just 
weakly compliant, IMO.

>Is that a problem?  Would it be a bad idea to upgrade mod_dav or IIS to become compliant with RFC2518bis if it goes to RFC with these requirements?
>
I don't see this as a problem and think it would be a good idea all 
around for mod_dav and IIS to upgrade and support ETags.

>I'd point out that there are other changes that already make servers slightly uncompliant with RFC2518bis [...] But being uncompliant with a spec that the server doesn't declare compliance to shouldn't be a problem!
>
Agreed.

>IF the language must be weakened, is there any point to this at all?  I'm afraid we've got to make a hard choice.  Either we do something real to encourage ETag support, even though it makes existin…
I wish there was something stronger than SHOULD and weaker than MUST, 
something along the lines of 'REALLY, REALLY SHOULD'...  :-) 
 Personally, I'm in favor of MUST, even with the difficulties it would 
create - growing pains are temporary and, as has been pointed out on 
several occasions, the benefits to both client authors and end users are 
clear.

Side note: Correct me if I'm wrong, but generating strong ETags just 
doesn't seem all that difficult to me...

Cheers,
Elias

---

From: "Julian Reschke" <[email-masked]>
Date: Tue, 24 Jun 2003 17:28:32 +0200

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Elias Sinderson
> Sent: Tuesday, June 24, 2003 5:15 PM
> To: Webdav WG
> Subject: Re: Stronger requirements for ETags
> 
> ...
> >

What is "weak" compliance?
 
> doesn't seem all that difficult to me...

Well, if this is the case, why do both mod_dav and IIS fail to do that?

Julian

---

From: Elias Sinderson <[email-masked]>
Date: Tue, 24 Jun 2003 09:15:12 -0700

>What is "weak" compliance?
Obviously not an *official* term!  :-)  Perhaps another way to put it 
would be to say that they would be 'barely' compliant? The general idea 
being that if a server were to satisfy all of the MUST level 
requirements but not the SHOULD level requirements it could be said to 
be compliant with the spec, but just barely compliant.

>Well, if this is the case, why do both mod_dav and IIS fail to do that?
I can't speak for the server implementors, but I imagine the issue is 
one of performance (as opposed to difficulty).

Cheers,
Elias