Subject: Re: [RFC2518 Issue] PROPFIND 'allprop' usage

From: [email-masked]
Date: Mon, 13 Nov 2000 15:17:40 +0000

...

These seem to be our options for modifying RFC2518 (remember, it has to be
a
simple mod):
 - deprecate 'allprop' and tell clients not to use it, but to use
'propname' instead

<tim>
This won't fix the problem, servers still have to deal with clients that
fail to follow the deprecation warning.  There would be no way to deprecate
the call anyway (other than on paper).
</tim>

 - define 'allprop' to be the set of properties required for DAV level 1
support (although servers could freely return more properties if desired)

<tim>
This is a breaking change to the spec.  I think it is unfair on clients to
change the spec. to this extent and at this stage.  To a 'legacy' client,
the request would appear to have succeeded but only returned partial
results, and there would be no indication that this had occurred.  The
would need to be some means for a client to determine of a server is
written to this new definition, which means it is a 'convenient'
deprecation.

I would support this if the server returned the subset of properties with a
new 200-series response code to indicate that the results were incomplete.
One could hope that some clients could continue to work in this environment
without modification.
</tim>

 - explicitly allow servers to return an error code (507?) for properties
that were too expensive to calculate for a 'allprop' request, but still
allowing the client to do property discovery through 'allprop'

<tim>
IMHO This would be a cleaner solution since clients are presumably already
dealing with server errors.  Clients would still learn the existance of
properties and their names, and smart clients could return to get their
values.

I suggest that the same error code be available for use with collections
that refuse to go deep.  It is astounding that a PROPFIND with no depth
header and no body is to be interpreted as all prop depth infinity -- the
server needs some mechanism to say 'be reasonable client'.
</tim>

---

From: "Humphreys, Richard" <[email-masked]>
Date: Mon, 13 Nov 2000 16:45:28 -0000

We spent a great deal of time debating this issue and have come to the
following conclusion:
 
'Allprop' is not the problem - 'depth: infinity' is the problem.
 
Allprop is a perfectly reasonable, even necessary, request.  Any client will
have a need to know about the complete property namespace for a particular
resource.  Why then should any sensibly written client need to issue a
propfind request with depth infinity?  Even if the server has the capacity
to build such a horrendous response, the response would probably crash the
client when it was received.
 
Two possible  solutions arose from our discussions, about which we would
welcome constructive criticism:
 
1- propfind should only be required to accept 'depth:0' and 'depth:1'.
Clients who really need more information can then issue requests as needed
to go further into the hierarchy. The client will then receive a '501 Not
implemented' if it attempts something that nasty.
 
2- if we have to keep 'depth: infinity', it should not be the default.
 
These suggestions seem reasonable (even ideal!) to us here, but we must
concede that we may be biased by the needs of our own particular project.

	Regards 

	Richard Humphreys 

	Xerox Europe

---

From: [email-masked]
Date: Tue, 14 Nov 2000 11:45:31 +0000

<Richard>
    We spent a great deal of time debating this issue and have
    come to the following conclusion:

</Richard>

<tim> I agree that depth infinity is not a good thing.</tim>

<Richard>
    Allprop is a perfectly reasonable, even necessary, request.
    Any client will have a need to know about the complete property
    namespace for a particular resource.  Why then should any
    sensibly written client need to issue a propfind request with
    depth infinity?  Even if the server has the capacity to build
    such a horrendous response, the response would probably crash the
    client when it was received.
</Richard>

<tim>
I dispute that allprop is necessary.  It is a convenient and even
reasonable short-hand for using propname and retrieving named properties.

The only excuse for a client to issue depth infinity requests is to
off-load the traversal work onto the server.  This is useful if the comms
are an issue, and also if the client is naive (their own depth infinity
traversal would have to retain state, watch out for Loop Detected cases,
etc.).

However, since there is inherently no limit to the number of properties
that can be set on a resource, the server has the problem of dealing with
potentially large responses (even without depth infinity) provoked by
clients that only sent (numerous) small requests.

Servers could refuse PROPPATCHes once the resource is 'full' thereby
limiting the size of responses.  The definition of 'full' would be a
function of the number of properties, the size of their values, and, for
collections, the 'fullness' of their members.
More likely, servers need some equivalent to 'that request is too expensive
for me to compute--ask me an easier one' status!
</tim>

<Richard>
    Two possible  solutions arose from our discussions, about which
    we would welcome constructive criticism:

    1- propfind should only be required to accept 'depth:0' and
    'depth:1'.  Clients who really need more information can then
    issue requests as needed to go further into the hierarchy.
    The client will then receive a '501 Not implemented' if it
    attempts something that nasty.
</Richard>

<tim>
Refusing to implement depth infinity would be fine.  Answering a 501 may be
misleading since it would imply that the PROPFIND method is not
implemented.  It would require some other status condition.
</tim>

<Richard>
    2- if we have to keep 'depth: infinity', it should not be the
    default.

    These suggestions seem reasonable (even ideal!) to us here,
    but we must concede that we may be biased by the needs of
    our own particular project.
</Richard>

<tim>
I agree it should not be the default.  I think an absent depth header
should default to depth zero, and an absent method body should default to
propnames.
</tim>

Tim

---

From: "Hall, Shaun" <[email-masked]>
Date: Wed, 22 Nov 2000 08:39:56 -0000

Hi,

We've discussed this and don't think that servers should compromise as
you've suggested.

Even if you permitted what you've suggested, the response could still
potentially be very large (depending on the non-locking properties defined
and the number of resources). We feel that the locking properties form only
a small number of all the properties stored.

If a client issues an "allprop", it should get all the properties defined.
Allowing the server to chooose what properties to return seems to convulent
the RFC/specification for example:
- how would the client know whether it actually received all the properties?
- would the client have to make further PROPFIND requests to retrieve the
extra properties?

If people want to allow the server to return a "subset" of all the
properties for an "allprop", the specification should "partition" the
potential response "in concrete" so there is no room for interpretation for
example:
- the server MUST return the WebDAV properties.
- the server MAY return all other properties. However, if it chooses to
return the non-WebDAV properties, it MUST return all non-WebDAV properties.
- if the specification allows servers to choose what properties to return,
there must be some well defined mechanism for the client to be able to
retrieve the non-returned proprties.

People must realise that allowing "allprop" with a depth infinity is going
to generate potentially large responses (which could be denial of service
attacks). If people don't like this, then consider dropping "infinite" depth
requests from the specification. This assumes that it doesn't break existing
products and one response for one product isn't necessarily a good
response/sample.

Regards

Shaun Hall
Xerox Europe

> -----Original Message-----
> From: Lisa Dusseault [mailto:[email-masked]]
> Sent: 21 November 2000 16:46
> To: Hartmut Warncke; WebDAV WG
> Subject: RE: [RFC2518 Issue] PROPFIND 'allprop' usage
> 
> 
> One more question, because maybe, as Richard Humphreys 
> suggests, "depth
> infinity" is the problem.
> 
> Hartmut, does GoLive5 use depth: infinity PROPFIND requests 
> at all?  If yes,
> does it use them for custom property discovery?
> 
> If not, then we could compromise our way through this by 
> stating that a
> WebDAV server SHOULD (MUST?) respond to a PROPFIND depth-0 
> 'allprop' request
> with all custom properties, but that it MAY respond to a 
> PROPFIND depth>0
> 'allprop' request with a more limited set of properties 
> (suggested to be the
> non-locking-related properties defined in RFC2518 presently).
> 
> Would that work?
> 
> Lisa
> 
> > -----Original Message-----
> > From: Hartmut Warncke [mailto:[email-masked]]
> > Sent: Monday, November 13, 2000 8:07 AM
> > To: Lisa Dusseault
> > Cc: WebDAV WG
> > Subject: Re: [RFC2518 Issue] PROPFIND 'allprop' usage
> >
> >
> >
> >
> > > To clarify, does the GoLive 5 WebDAV client rely on using
> > PROPFIND 'allprop'
> > > requests to get all custom properties on **any** webDAV server?
> >  or against
> > > a specific webDAV server?
> >
> > On  **any** WebDAV server.
> >
> > > Have you tested this against various servers?
> > > Does it work with all of them?
> >
> > Yes, it's working against IIS and mod_dav for example.
> >
> > Best, Hartmut
> >
> >
> > > > -----Original Message-----
> > > > From: [email-masked]
> > > > [mailto:[email-masked]]On Behalf Of 
> Hartmut Warncke
> > > > Sent: Monday, November 13, 2000 4:07 AM
> > > > To: Lisa Dusseault; WebDAV WG
> > > > Subject: Re: [RFC2518 Issue] PROPFIND 'allprop' usage
> > > >
> > > >
> > > >
> > > > Hi all,
> > > >
> > > > I have major concerns regarding the change of  the response
> > on a PROPFIND
> > > > 'allprop' request in the way you described it.
> > > >
> > > > Such a change would be  *v e r y  harmful*   for the GoLive 5
> > > > WebDAV client.
> > > > When we send a PROPFIND 'allprop' request we expect all
> > > > properties which are
> > > > defined on the resource, especially  the Lockproperties 
> and all custom
> > > > properties defined by GoLive 5 (which we have 
> PROPPATCHED before).
> > > >
> > > > We are probably able to replace PROPFIND 'allprop' requests by
> > > > PROPFIND 'prop'
> > > > requests in future GoLive releases (which would be indeed much
> > > > more efficient)
> > > > but the suggested change in the protocol would be a  *disaster*
> > > > for GoLive 5
> > > > which is already in the box.
> > > >
> > > > Best, Hartmut
> > > >
> > > >
> > > > Lisa Dusseault wrote:
> > > >
> > > > > Past discussions have shown that servers frequently 
> have trouble
> > > > > implementing PROPFIND 'allprop'.  Jim asked me to 
> summarize the past
> > > > > discussion & list the open issues so that we can get this
> > > > fixed, if it can
> > > > > be fixed, in revisions to 2518.
> > > > >
> > > > > There are already cases where not all properties will 
> be returned:
> > > > > RFC2518: "In the case of allprop and propname, if a principal
> > > > does not have
> > > > > the
> > > > >    right to know whether a particular property exists then
> > the property
> > > > >    should be silently excluded from the response."
> > > > >
> > > > > John Stracke's proposal for reducing/specifying the scope of
> > > > 'allprop', and
> > > > > discussion of the motivation:
> > > > >  -
> > > > 
> http://lists.w3.org/Archives/Public/w3c-dist-auth/1999JulSep/0092.html
> > > > >  -
> > > > 
> http://lists.w3.org/Archives/Public/w3c-dist-auth/1999JulSep/0310.html
> > > > >
> > > > > It has been a point of discussion for Advanced Collections:
> > > > >  -
> > > > 
> http://lists.w3.org/Archives/Public/w3c-dist-auth/1999JanMar/0008.html
> > > > > "Clients need to know whether the property is computed on
> > the fly before
> > > > > requesting it.  There is no way to find out.  The impact of
> > > > computing on the
> > > > > fly is especially significant when a client requests allprop.
> > > > There may be
> > > > > other properties that are computed on the fly as well.
> > DAV:getetag is
> > > > > computed, and some versioning history properties may also
> > be computed."
> > > > >
> > > > > Also in Versioning:
> > > > >  -
> > > 
> http://lists.w3.org/Archives/Public/w3c-dist-auth/1999JanMar/0075.html
> > > > "There has also been a massive growth in the number of 
> available DAV
> > > > properties.  PROPFIND allprop operations may lead to very large
> > > > responses even with Depth: 1, which would slow down performance
> > > > for users due to network speeds.  It might be 
> worthwhile to add this
> > > > facet to the open issue ALLPROP_AND_COMPUTED."
> > > >
> > > > Also in ACLs, Babich argues that clients who request 
> 'allprop' don't
> > > really
> > > > want to see the ACL property, thus they ought to specifically
> > ask for it.
> > > >  -
> http://lists.w3.org/Archives/Public/w3c-dist-auth/1998JulSep/0101.html
> > >
> > > Several server implementors have voiced the opinion that 'allprop'
> should
> > be
> > > "put out of its misery" (GMC) or at least weakened.  Often this is
> because
> > > of standard or custom properties that must be calculated 
> by the server
> > (e.g.
> > > 'lockdiscovery'), and the calculation can become extraordinarily
> expensive
> > > with an 'allprop' of depth: infinity.
> > >
> > > The only server-side argument for keeping 'allprop' is that
> > server-to-server
> > > COPY requires it; but if anybody has implemented this yet 
> and can't use
> > > 'propname' instead, please speak up.
> > >
> > > Summary:
> > > There thus seems to be a consensus among server 
> implementors and those
> > > designing new features for DAV.  What's missing in order 
> to resolve this
> > > issue for fixing RFC2518 is input from clients.
> > >
> > > 1.  Is anybody aware of clients that rely on particular 
> properties being
> > > returned in 'allprop'?  If the properties relied upon 
> include any more
> > than
> > > the set DAV:{creationdate, displayname, getcontentlanguage,
> > > getcontentlength, getcontenttype, getetag, getlastmodified} (the
> > properties
> > > required for DAV level 1 support) I would be very 
> surprised.  Thus,
> > servers
> > > may be able to restrict the required property set to this set.
> > >
> > > 2.  Is anybody aware of clients that rely on 'allprop', 
> rather than
> > > 'propname', for property discovery?  This would be a more 
> serious issue
> if
> > > major client implementations actually rely on doing 
> property discovery
> > using
> > > 'allprop', and attempt this against various 
> implementations of WebDAV
> > > servers.
> > >
> > > These seem to be our options for modifying RFC2518 
> (remember, it has to
> be
> > a
> > > simple mod):
> > 'propname'
> > > instead
> > > Please voice your preferences among these options, 
> objections, or other
> > > ideas.
> > >
> > > thanks,
> > > Lisa
> > > Xythos
>

---

From: "Peterson, Byron - [RHX]" <[email-masked]>
Date: Wed, 22 Nov 2000 10:43:16 -0500

could you please tell me how to unsubscribe to this list???
Thanks...

-----Original Message-----
From: Hartmut Warncke [mailto:[email-masked]]
Sent: Wednesday, November 22, 2000 8:12 AM
To: [email-masked]
Cc: WebDAV WG
Subject: Re: [RFC2518 Issue] PROPFIND 'allprop' usage

Lisa,

> One more question, because maybe, as Richard Humphreys suggests, "depth
> Hartmut, does GoLive5 use depth: infinity PROPFIND requests at all?  If
yes,
> does it use them for custom property discovery?

Yes, GoLive uses those infinity requests because it's a very effiicient way
to
synchronize the WebDAV server content with the content of a site on the
local
machine with  *o n e*  network request. We are requesting build-in DAV
properties (like "getlastmodified" or "lockdiscovery") as well as custom
properties (which were PROPPATCHED by GoLive before) with depth infinity.

To my mind depth infinity requests are -from the client perspective - a very
efficient and interesting WebDAV feature.
Dropping  infinite requests would break GoLive 5!

By the way if a server would be allowed to refuse depth infinity requests
(like
mod_dav with its "DavDepthInfinity" flag in its configuration file), the
client
needs a mechanism to figure out if the server is willing to respond to depth
infinity requests or not. This mechanism has to be precisely defined by
RFC2518.

> If not, then we could compromise our way through this by stating that a
> WebDAV server SHOULD (MUST?) respond to a PROPFIND depth-0 'allprop'
request
> with all custom properties, but that it MAY respond to a PROPFIND depth>0
> 'allprop' request with a more limited set of properties (suggested to be
the
> non-locking-related properties defined in RFC2518 presently).

I think "allprop" should always return all properties (custom and DAV
build-in).
Perhaps it makes sense to drop PROPFIND-ALLPROP requests with depth infinity
but
please keep the PROPFIND-PROP requests with depth infinity because they are
a
very powerful and efficient instrument. Dropping those requests would lead
us
-in some situations- into a time consuming disaster with a huge amount of
network traffic.

Best, Hartmut

> > > > [mailto:[email-masked]]On Behalf Of Hartmut Warncke
> > > > defined on the resource, especially  the Lockproperties and all
custom
> > > > properties defined by GoLive 5 (which we have PROPPATCHED before).
> > > >
> > > > > Past discussions have shown that servers frequently have trouble
> > > > > implementing PROPFIND 'allprop'.  Jim asked me to summarize the
past
> > > > > There are already cases where not all properties will be returned:
> > > > "There has also been a massive growth in the number of available DAV
> > > > for users due to network speeds.  It might be worthwhile to add this
> > > > Also in ACLs, Babich argues that clients who request 'allprop' don't
> > > really
> > > of standard or custom properties that must be calculated by the server
> > (e.g.
> > > COPY requires it; but if anybody has implemented this yet and can't
use
> > > There thus seems to be a consensus among server implementors and those
> > > designing new features for DAV.  What's missing in order to resolve
this
> > > 1.  Is anybody aware of clients that rely on particular properties
being
> > > returned in 'allprop'?  If the properties relied upon include any more
> > than
> > > required for DAV level 1 support) I would be very surprised.  Thus,
> > servers
> > > 2.  Is anybody aware of clients that rely on 'allprop', rather than
> > > 'propname', for property discovery?  This would be a more serious
issue
> if
> > > major client implementations actually rely on doing property discovery
> > using
> > > 'allprop', and attempt this against various implementations of WebDAV
> > > servers.
> > >
> > > These seem to be our options for modifying RFC2518 (remember, it has
to
> > > Please voice your preferences among these options, objections, or
other

---

From: Michael Hermann <[email-masked]>
Date: Wed, 22 Nov 2000 12:08:36 -0800

unsubscribe 

(sorry if this goes to the list, I forgot to record the admin address)

> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, November 22, 2000 12:00 PM
> To: [email-masked]
> I have to agree with Shaun that providing an unbounded 
> implementation of
> 'PROPFIND allprops depth infinity' would be brain dead.  
> Similarly, moving
> the goal posts to say that allprops no longer returns all the 
> properties
> would be 'unfavourable' to existing clients and servers.
> 
> The only tenable position is to allow the server to refuse 
> the request,
> either in its entirety (no all props and/or no depth infinity) or by
> allowing it to report partial results (a response status code that
> indicates not all properties were returned, and/or collection 
> members were
> not traversed) for a resource.
> 
> IMHO, attempting to synchronize thousands of files with a 
> single call to
> the server is not a fantastic idea.
> 
> Regards,
> 
> Tim
> 
> 
> Hartmut Warncke <[email-masked]> on 2000-11-22 04:58:26 PM
> 
> Please respond to Hartmut Warncke <[email-masked]>
> 
> To:   "Hall, Shaun" <[email-masked]>, WebDAV WG
>       <[email-masked]>
> cc:
> > The implications of PROPFIND with an infinite depth affect both the
> server
> > and/or the client. Supposing you have a large WebDAV 
> enabled repository.
> The
> > PROPFIND request (a few hundred bytes) could potentially 
> mean that (tens
> of)
> > GBs of data is to be returned. Even if the server could 
> generate such a
> > large response (which is a feat in its own right), it will load the
> network,
> > and what will happen when the client receives it? Pity the poor user
> sitting
> > there for a long time (hours?), only to be told "timed out" 
> or "out of
> > memory" or something similar.
> 
> Yes, of course, I completely understand your point of view. 
> But as a matter
> of
> fact we like this "depth infinity" feature very much because 
> we can use it
> in a
> very efficient way in some situations and therefore we are 
> struggleing so
> hard
> for keeping that feature.
> 
> > Changing the specification could break existing products, 
> which is a bad
> > thing. Backwards compatibilty should be maintained.
> 
> Good to hear that. But I am not quite sure how you will do 
> that because if
> a
> server has the possibility to refuse a depth infinity request 
> GoLive 5 will
> not
> be able to handle that situation (synchronization of client and server
> content
> will not work anymore).
> 
> > We think the way forward for the WebDAV specification is to 
> allow servers
> > the ability to refuse such requests and inform the client. 
> A mechanism
> > should be defined for the client to understand this. If the client
> received
> > a response which basically stated that the server was 
> refusing to service
> an
> > infinite depth request, it could issue multiple requests 
> with a Depth:1.
> 
> If you have to synchronize a very large site with thousands of files,
> replacing
> depth infinity requests by depth 1 requests would be a huge 
> performance
> disaster
> for us.
> 
> Best, Hartmut
> 
> 
> 
>

---

From: [email-masked]
Date: Wed, 22 Nov 2000 20:00:03 +0000

I have to agree with Shaun that providing an unbounded implementation of
'PROPFIND allprops depth infinity' would be brain dead.  Similarly, moving
the goal posts to say that allprops no longer returns all the properties
would be 'unfavourable' to existing clients and servers.

The only tenable position is to allow the server to refuse the request,
either in its entirety (no all props and/or no depth infinity) or by
allowing it to report partial results (a response status code that
indicates not all properties were returned, and/or collection members were
not traversed) for a resource.

IMHO, attempting to synchronize thousands of files with a single call to
the server is not a fantastic idea.

Regards,

Tim

server
> and/or the client. Supposing you have a large WebDAV enabled repository.
The
> PROPFIND request (a few hundred bytes) could potentially mean that (tens
of)
> GBs of data is to be returned. Even if the server could generate such a
network,
sitting
> there for a long time (hours?), only to be told "timed out" or "out of

Yes, of course, I completely understand your point of view. But as a matter
of
fact we like this "depth infinity" feature very much because we can use it
in a
very efficient way in some situations and therefore we are struggleing so
hard
for keeping that feature.

> Changing the specification could break existing products, which is a bad

Good to hear that. But I am not quite sure how you will do that because if
a
server has the possibility to refuse a depth infinity request GoLive 5 will
not
be able to handle that situation (synchronization of client and server
content
will not work anymore).

> We think the way forward for the WebDAV specification is to allow servers
> the ability to refuse such requests and inform the client. A mechanism
received
> a response which basically stated that the server was refusing to service
an
> infinite depth request, it could issue multiple requests with a Depth:1.

If you have to synchronize a very large site with thousands of files,
replacing
depth infinity requests by depth 1 requests would be a huge performance
disaster
for us.

Best, Hartmut

---

From: Terry Crowley <[email-masked]>
Date: Sun, 26 Nov 2000 19:22:07 -0800

I'd have to agree on the point that a 507 response isn't particularly
useful.  Just what is a client supposed to do with that?  How are they
supposed to recover?  What kind of feedback do they provide to the user that
they just got back a partial result set (and as Greg points out, it would be
very difficult to know how it was truncated).  Many clients would want to
just treat that as a complete failure and try a different approach, but in
the meantime they might have had to download and parse a very large (but not
large enough) response that they end up discarding.

Terry Crowley

>> Returning 507 would be a bit more difficult implementation-wise. However,
I
>> think we really shouldn't allow that mechanism. What is a client to do
when
>> it gets a 507? How does it know *what* was left out, and *how* to get
those
>> results? Did the server do a depth-first or a breadth-first response of
>> properties? Which collections did it recurse into and which did it not?
Did
>> it stop *partway* through a collection? How can a client tell?
>> -- 
>> Greg Stein, http://www.lyra.org/