Subject: Re: I-D ACTION:draft-ietf-ldup-lcup-04.txt

From: John McMeeking <[email-masked]>
Date: Mon, 10 Mar 2003 14:16:30 -0600

It seem like the log-based vs state-based distinctions may not be necessary
if we borrow some ideas from LDUP.

LDUP has the notion that deleted entries be maintained by the server for
some period of time.  A client search never returns these entries, but the
server keeps knowledge that the entry with UUID 'x-uuid' was deleted at
time 'x-deleted-time'.  This could be implemented using a log (search log
for deleted entries) or state (meta-data).  LDUP doesn't care (though at
some level it starts to be awfully hard to be state/log based agnostic).

If a cookie has some sort of time information, then a server can:
- find entries that have been added (createTimestamp attribute or
equivalent)
- find entries that have been modified (modifyTimestamp or equivalent)
- find entries that have been deleted (delete timestamp described above)

Modify DN presents problems.  Perhaps we could add a superior timestamp
(new superior) and RDN timestamp (rename).  Not knowing where the entry
came from, all modify DN operations (with appropriate time) would be
treated as "entered result set" if they are now in the result set, and as
"left result set" if they are not now in the result set.

I think modify would not follow the letter of the draft.  Any change to an
entry would have to be treated as a change to the attributes the client was
interested in.  But I don't think that should cause problems, unless there
are a lot of these changes relative to changes the client is really
interested in (whatever "a lot" is).

A server probably wouldn't want to keep deleted entry state information
around forever.  A server implementation should be able to prune this --
could be as arbitrary as discarding deleted entry information after a fixed
period.  Any LCUP cookie that corresponded to a time prior to this period
would require a resync.  Deletes should be relatively infrequent, so it
should be possible for this time period to be fairly long.

Assuming everyone views this as appropriate state information, a log-based
implementation is not required.  I doubt LCUP needs to say any of this, but
maybe others think such "hints" are appropriate.

John  McMeeking

                      "Kurt D.                                                                                                   
                      Zeilenga"                To:       John Merrells <[email-masked]>                                  
                      <[email-masked]        cc:       Rich Megginson <[email-masked]>, [email-masked]                  
                      g>                       Subject:  Re: I-D ACTION:draft-ietf-ldup-lcup-04.txt                              
                      Sent by:                                                                                                   
                      owner-ietf-ldup@m                                                                                          
                      ail.imc.org                                                                                                

                      03/10/2003 01:18                                                                                           
                      PM                                                                                                         

At 10:33 AM 3/10/2003, John Merrells wrote:
>Hmm. I can't resist dipping into this thread.
>>The most obvious problems are with the requests for content
>>refresh.
>>
>>Without historical information,
>In a state-based system the state records the history.

When I referred to a "state-based" system, I hoped it was
obvious that I was referring to a system which maintained
the current state of directory and meta information but
did not maintain historical state information, in particular
histories of state and/or content updates.  If this was
unclear, I apologize.

>>a server has no means to
>>determine whether or not an entry whose state indicates it was
>>updated (add,modify,rename) since the previous refresh
>>request has left the content.  It can only determine whether
>>the updated entry is currently in the content or not.  The
>>server must assume the entry was previously in the content.
>>Hence, the server must generate a message (update or entryLeft)
>>for each updated entry in the context or require a reload.
>A state-based system can record the deleted/left entires in its state.

I do not view such a system as being a state-based system.
I view it as a log-based system.   Do we need to more precisely
defined these terms?

>>But worse, if any entry was deleted from the context, a
>>server which maintains no historical information is forced
>>to respond that a reload is required.
>Too bad. The server implementation is up to the server implementor.

I disagree.  Every technical specifications limit implementation
approaches to those which can produce conformant implementations.
However, standard track technical specifications should avoid
placing undue constraints on implementation details.

Here I believe, as the author clarified, it was not/is not the
intent of the authors to preclude a state-based implementation
approach.  My follow-up message serves only to elaborate on why I
believe the technical specification does preclude state-based
implementation approaches.

It is possible that the authors misunderstood my initial question
in this thread.  If so, I assume they will post a clarification.

>The server can store no state, some state, or all state.

I believe that LCUP should be designed such that implementors may
choose whether to pursue a state-based or a log-based approach
(or, possibly, some hybrid approach).  I am attempting to demonstrate
how the current specification precludes the state-base (e.g., no
history) approach.

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Mon, 10 Mar 2003 14:46:28 -0800

[resent to fix some nasty typos]

>- find entries that have been deleted (delete timestamp described above)

Without additional historical information, the server however
would not, in general, be able to determine whether or not an
updated entry within the context was in the content at the time
indicated by the cookie.  The server must assume it was in the
content at the time indicated by the cookie and generate an
appropriate message to the client.

>interested in.

Yes.  Say the updated entry is within scope of the search but
currently doesn't match the search criteria.  Without additional
historical information, the server would not be able to determine
if the updated entry was in the content at the time indicated by
the cookie.  The server must assume it was in the content at the
time indicated by the cookie and generate an appropriate message
to the client.

>But I don't think that should cause problems, unless there

The problem here is that the server is unable to determine
whether an updated entry in the CONTEXT was previously in the
CONTENT without additional historical information and hence
must assume that it was in the CONTENT at the time indicated
by the cookie.

That is, the problem severity is relative to the changes to
the CONTEXT not just to CONTENT.

>maybe others think such "hints" are appropriate.

Well, I thin that if we believe that certain implementation
approaches would lead to implementations which would do harm,
that we should state these beliefs and, where appropriate,
detail implementation-approach-neutral imperatives of what
behaviors are considered harmful.  Lack of "eventual convergence
data consistency" and/or "overly chattiness" would be harmful.

Kurt

---

From: Jonghyuk Choi <[email-masked]>
Date: Tue, 11 Mar 2003 09:28:47 -0500

>In this case the server is using some historical information to optimize
the
>amount of information that needs to be sent back to the client. The more
>historical information stored the more you can optimize the amount of data
>that needs to be sent back.

No. Client cookie is not historical information. It's client state
information.
On the other hand, the maintained deleted entries are historical
information
to which the last sentence of the above holds.

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: [email-masked]
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979

---

From: Jonghyuk Choi <[email-masked]>
Date: Tue, 11 Mar 2003 11:00:39 -0500

>Exactly. There is a big tradeoff here between complexity of
>implementation and the efficiency of the LCUP protocol exchange. It is
>important to allow implementors to have some flexibility while still
>assuring that the protocol works reliably. In many cases, efficiency
>tradeoffs can and should be left in the hands of the implementors.

Yes. A protocol that exhibits refined complexity-efficiency characterics
for wide range of scenarios would be considered thorough. Efficiency
tradeoffs in the hands of the implementors are within the range that
the protocol offers.

>-Mark Smith
  >Netscape