Subject: What's going on?

From: Chris Apple <[email-masked]>
Date: Thu, 08 Jun 2000 15:59:59 -0400

I haven't seen any recent list activity other than two SPAMs since
April 25th. There are still deliverables that are/were due that
have not been published.

Please report on the status of your present (or pending) draft
to the list.

-- 
------------------------------------------------------------------------
Chris Apple                     Business Site: AnyWho Directory Service
Internet Directory Group                       http://www.anywho.com
AT&T Labs
[email-masked]
+1 973 236 6470                 Tired of slow directories?  Try AnyWho.
------------------------------------------------------------------------

---

From: "Albert Langer" <[email-masked]>
Date: Sat, 17 Jun 2000 18:27:20 +1000

Well, it's been a full week since this request for status reports. The only
responses I've seen have been:

a) Jim Sermersheim's reminder of offer to help with ID on Mandatory Replica
Management.
http://www.imc.org/ietf-ldup/mail-archive/msg00560.html

b) Roger Harrison's status report on LBURP.

http://www.imc.org/ietf-ldup/mail-archive/msg00558.html

As a new participant I have refrained from comment until the WG took a
decision concerning my objection to a final call on the requirements draft,
which is explained in the initial pages of my individual submission on "LDUP
Multiple Draft Conflict Resolution (MDCR)".

http://www.ietf.org/internet-drafts/draft-langer-ldup-mdcr-00.txt

Meanwhile, I have done penance for popping up with an objection to a final
call at the last moment without previous participation, by carefully reading
the entire archives of this list.

As the authors of the requirements draft have not responded to the request
for a status report, or participated in WG discussion of the objection to a
final call and the co-chairs have not announced any result of the WG
discussion, here's my view on its status.

1. It expired on 21 April 2000 although it is still visible at:

http://search.ietf.org/internet-drafts/draft-ietf-ldup-replica-req-02.txt

2. According to item 5 the draft minutes of the LDUP meeting at the 47th
IETF:

http://www.imc.org/ietf-ldup/mail-archive/msg00539.html

Recommendation of the chair and AD: Until an Internet draft
is an RFC there is always time to discuss things. However,
it is not fruitful to discuss anything until a formal
counter-proposal, in the form of an Internet Draft, is
submitted (as an individual submission) for consideration
by the working group. Thus, it is incumbent on Mr. Langer to
produce such an Internet Draft.

Once this new draft is submitted, the working group will
consider it. The first order of business is for the
replication requirements team and authors to respond on the
mail list, and then for the working group to consider
whether Mr. Langer’s concerns are justified. If they are,
then the requirements draft will have to be modified to take
this into account, and another working group last call
issued. If not, then the requirements draft will go to IETF
last call either as is, or with a small editorial
modification that further clarifies the requirements draft
in response to Mr. Langer’s objections. The working group
will make a decision by Easter. Note that this is
independent of the URP decision.

3. The discussion was opened on 11 April 2000 by WG co-chair John Strasser
as follows:

http://www.imc.org/ietf-ldup/mail-archive/msg00543.html

as mentioned in the draft minutes from our latest meeting in
Adelaide, here is the announcement of Albert's draft. Please
do read this as soon as possible (but not before you send
comments on the minutes ;-) ) and let's start discussing
this on this list. Remember, the goal is to reach a
consensus as to what actions to take by the end of this
month (that would be April, by the way ;-) ).

The questions on the floor that this draft addresses are:

  1) is the requirements document as currently written
     too vague? That is, should we delay sending it to
     IETF last call? Please read sections 1 and 3 of
     this draft to answer this question.
  2) should we add this draft as another possible
     conflict resolution method, or should we replace
     URP with this draft, or should URP stay as the
     only method of conflict resolution.

(BTW as requested I sent John a clarification to other parts of the minutes
explaining that my objection to the requirements draft was independent of
proposing MDCR and wider than the issue of atomic updates. No response
received.)

4. The replication requirements authors did not respond on the mailing list
as per the "first order of business" mentioned in the minutes. Perhaps as a
result, there was very little discussion. In that discussion no support was
expressed for proceeding to final call with no changes.

The only comments were as follows:

5. Ryan Moats said on 12 April:

http://www.imc.org/ietf-ldup/mail-archive/msg00544.html

If somebody can quote the portion of the
URP draft that shows that atomicity is maintained,
please do.  Otherwise, the URP draft needs to be edited
to make this concept explicit.

6. On 13 April, Alison Payne asked whether a requirement could be added
indicating that changes should not be rolled back and a reference to the
desirability of reducing the complexity of implementation.

http://www.imc.org/ietf-ldup/mail-archive/msg00546.html

7. On the same day Alison also submitted a detailed and very frank
"Contribution to Profiles Document (Consistency Discussion)". As well as
confirming that the URP draft cannot meet an atomicity requirement, in my
view it conclusively established that anyone considering deployment of a
multi-master directory system based on the URP draft would be wise to opt
for single master instead.

http://www.imc.org/ietf-ldup/mail-archive/msg00548.html

8. On the same day Harald Tveit Alvestrand said:

The important thing is IMHO documenting IN DETAIL what consistency the
procedures guarantee; the requirement that the procedure proposals do
should be in the requirements document.

Apart from that, I think requirements are ready to go.

http://www.imc.org/ietf-ldup/mail-archive/msg00549.html

9. There has been no further discussion on this agenda item since 13 April.

Incidentally, a year earlier, the Draft LDUP Minutes for the 44th IETF
(Oslo) recorded an agreement to change the name to "Design Considerations"
because an "Informational" RFC with the word "Requirements" in the title is
frowned upon.

http://www.imc.org/ietf-ldup/mail-archive/msg00294.html

My conclusion from the above is that unless the authors report on its status
promptly, after direct email from the WG chairs, the WG chairs should
declare the draft dead.

As well as numerous other flaws it simply does not contain any requirements
whatever concerning multi-master replication except for the following:

5.6  The replication model MUST support both master-slave and
           authoritative multi-updateable replica relationships.

The incoherence of those words is not just a matter of poor expression but
reflects fundamental misconceptions in the rest of the document. See for
example the definitions of "Updateable Replica" and "Atomicity".

There was no separate discussion of whether to add my MDCR draft as an
alternative to the URP draft for consideration of both by the WG, but
comments about the draft are included in the above messages, without
explicit statements for adding it or for leaving it as an individual
submission.

There is of course no possibility of two alternative methods of conflict
resolution being eventually accepted, and no proposal to simply drop the URP
draft from the WG agenda. The issue I am waiting for a decision on is
whether the WG wishes to consider my alternative. If it does, I am keen to
work with the URP authors on a combined proposal as we are all in Melbourne
and I have written to Alison proposing that Alison, Steve and I get together
to discuss the feasability. (Emailed twice, first with wrong phone number,
received only a delivery notice with no receipt notice or reply).

My understanding is that ALL deliverables listed in the WG charter:
http://www.ietf.org/html.charters/ldup-charter.html
are "unpublished".

In particular none can or should be published as RFCs until input on
requirements has been solicited by a formal LDUP Requirements RFC. The
reason for seeking comments on requirements before finalizing architecture,
let alone detailed protocol specifications, is fairly obvious.

The failure to think through and reach consensus on requirements first is, I
believe, after careful study of the archives, the reason why the WG has
ended up in its current state.

I propose that the WG should be re-chartered and should proceed through the
work it was originally chartered to do in the order it was chartered to do
it in, with whatever modifications appear necessary in discussion of the
re-chartering process.

In reviewing the archives I noted that there was a separate email list
established for the engineering team, presumably closed to avoid distraction
from other WG members while working.

If the archives of that list are available I have not been able to locate
them. They may, for all I know, shed a different light on what has happened
and why. Could the WG chairs please state where they are available or
arrange for them to be made publicly available (read only) if they are not?

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Chris Apple
Sent: Friday, June 09, 2000 6:00 AM
To: [email-masked]
Subject: What's going on?

---

From: "Ryan Moats" <[email-masked]>
Date: Sat, 17 Jun 2000 07:55:55 -0500

While I don't mind being quoted, it would be appropriate to
use full quotes, especially when the unquoted material is
germane to the discussion...  My original mail states in
addition to the portion that was quoted below:

"Once we've shown that URP maintains atomicity of LDAP changes,
I really don't see what else the MDCR draft is looking for..."

One reason I haven't said more is that I've been in
transition between companies, but I have vague memories
of discussions while with my old employeer about whether or
not MDCR would help (my memory is unclear, but I THINK 
that it wouldn't) or even whether atomicity in replication
would help or not (my memory is unclear here, but I 
THINK atomicity led to some thorny problems).  Hopefully, the
folks who were in those discussions and on this mailing
list can chime in with the actual scenarios that we discussed
because I frankly can't remember them and I admit that
without supporting scenarios to discuss my thoughts aren't
really useful.

Ryan Moats

P.S. Yes, I've removed quite a bit from the original, so
can be accused of being guilty of my own complaint :-)

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sat, 17 Jun 2000 05:57:36 -0700

>arrange for them to be made publicly available (read only) if they are not?

Per <http://www.ietf.org/html.charters/ldup-charter.html>,
the list archives of this WG are available at
<http://www.imc.org/ietf-ldup/>.

Kurt

---

From: "Albert Langer" <[email-masked]>
Date: Sat, 17 Jun 2000 23:14:21 +1000

Thanks Kurt, but unfortunately that is the WG archive which I mentioned I
had read the whole of and gave several links to. It is not the separate
engineering team list I was referring to. (Though you may be confirming that
I was mistaken no such list exists?)

-----Original Message-----
From: Kurt D. Zeilenga [mailto:[email-masked]]
Sent: Saturday, June 17, 2000 10:58 PM
To: [email-masked]
Cc: 'Chris Apple'; [email-masked]; [email-masked]
Subject: RE: What's going on? - Status of Requirements and MDCR and WG
charter

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sat, 17 Jun 2000 06:51:41 -0700

At 11:14 PM 6/17/00 +1000, Albert Langer wrote:
>engineering team list I was referring to.

Yes.  I should remember to finish my morning cola before posting...

>(Though you may be confirming that

I cannot confirm nor deny the existance of archives of this
list "engineering team" list.

Kurt

---

From: "Albert Langer" <[email-masked]>
Date: Sun, 18 Jun 2000 00:39:29 +1000

[KZ]

[AL]
Hmm, perhaps that also explains Ryan Moat's recent posting.

I think I'll wait and see if he posts any um "clarification" of what he
actually meant to say before responding.

[KL]

Aha! So it isn't a stuff up... it's a CONSPIRACY ;-)

Think I'll get some sleep myself...

PS The CC I originally included to John Strassner from his last email
bounces. (Different address in WG charter hasn't bounced yet, but still
silent).

---

From: "Ryan Moats" <[email-masked]>
Date: Sat, 17 Jun 2000 11:23:18 -0500

>[KZ] Yes.  I should remember to finish my morning cola before posting...
>
>[AL] Hmm, perhaps that also explains Ryan Moat's recent posting.

<soapbox>
While the spelling was probably only a typo, I still find it moderately
annoying.  It isn't that difficult a name to handle.
</soapbox>

>actually meant to say before responding.

Well, I thought I said precisely what I meant to say:  I was trying
to "tickle" the folks at my old company who were in the mentioned
discussions to join the thread because my memory of the technical
portions from them (as I sit here) are too tenuous for me to (a) be
sure what the issues were and (b) what the suggestions were for
dealing with them.

The problem is, I don't know that any of them will see this thread
before Monday AM, US EDT :-(.

Ryan

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 19 Jun 2000 01:49:50 +0200

I'll just repeat my earlier observation that both the LDUP protocol proposal
and the MDCR proposal allow a sequence of changes where the final state of
a set of multiple entries is not what either manager saw when he thought he 
had finished "his" updates to the entry set.

Since the problem of multiple-entry update consistency appears infeasible 
to solve in an LDAP context, I support going forward with the requirements 
document without changing the current text.

                    Harald A

--
Harald Tveit Alvestrand, EDB Maxware, Norway
[email-masked]

---

From: "Albert Langer" <[email-masked]>
Date: Mon, 19 Jun 2000 15:00:14 +1000

[HA]

                    Harald A

[AL]

Neither URP nor MDCR attempts to solve the problem of "multiple-entry update
consistency" because it is has never been part of the X.500 data model
supported by LDAP. Even on a single server with no replication, updates from
different DUAs to multiple entries may be interleaved, resulting in
inconsistency.

On server X, DUA A reads entries 1 and 2, DUA B reads entries 1 and 2, DUA A
changes entry 1, DUA B changes entry 1, DUA B changes entry 2, DUA A changes
entry 2. Result: entry 1 changed by B and entry 2 inconsistently changed by
A, although each changed both 1 and 2 consistently after first reading the
current state of both, and each immediately after reading both entries made
one change immediately followed by a second change without any intervening
delay.

(BTW DUAs making changes are now often driven by end users or unattended
applications, not necessarily by "managers").

As is correctly stated in the current architecture draft:

3.3  Document Non-Objectives

This document does not address the following issues, as they are
considered beyond the scope of the Working Group.
[...]
c) How transactions will be replicated. However, the architecture
  should not knowingly prevent or impede them, given the Working
  Group's incomplete understanding of the issues at this time.

http://www.ietf.org/internet-drafts/draft-ietf-ldup-model-03.txt

I would be interested in your view of the following comments "Re: LDUP
warmup exercise: atomicity in LDAPv3", from Tim Howes (16 December 1998):

"Each LDAP operation (add, modify, delete, moddn) as
a whole is atomic. The whole operation either happens
or it doesn't. Changes cannot be half-applied to any
single LDAP server.

The replication consistency model must assume and
build on this basic fact to define how multiple LDAP
replicas converge to the same state over time, in
the absence of additional changes. This kind of loose
consistency model is pretty fundamental to the notion
of a directory.

My two cents on what's important in a replication
consistency model are that it must be 1) predictable,
and that it should 2) make some kind of sense to
people using the system.

All this talk of consistency at different levels
(e.g., between applications using the directory at
the same time) is a red herring. Our job is to define
a consistency model for the directory itself. Some
applications may find this model sufficient for their
needs. Others may have to build more elaborate models
on top. But let's start with the basics.    -- Tim"

http://www.imc.org/ietf-ldup/mail-archive/msg00214.html

That seems consistent with the general internet architectural approach to
such issues.

Your proposal not to even state what guarantees are or are not provided, and
to refrain from providing any because we are not able to provide the highest
possible guarantees, does not seem consistent with the general internet
architectural approach to such issues.

It strikes me as rather like saying, either provide reliable network layer
connections or omit checksums from IP headers (and don't warn people about
the consequences of omitting checksums from UDP packets).

It is definately a red herring in the context of URP vs MDCR. How can the
fact that neither deals with it possibly be a basis for preferring one to
the other? Let alone a reason for not stating requirements that WOULD
distinguish between alternative proposals?

MDCR's consistency model is easy for users to understand - if somebody else
changed the same entry at about the same time, their change may have
succeeded and prevailed over yours as though yours had never happened. If
you want to know who or what is responsible for your change having
disappeared, simply look at the modifierName. If the changes are part of a
transaction affecting multiple entries, the DUA responsible for the
successful change for the root entry of the transaction is responsible for
the rollback.

That happens to be rather similar to what can also occur with single server
directories, since any change can be overwritten by another change to the
same entry. It will just happen more often with multi-master (since "about
the same time" is a longer interval).

Dropping packets is a lot more consistent with internet architectural
principles than merging the bits when too many arrive at once. It is also a
lot easier to implement.

The consistency model of URP, was summarized recently in Alison's
"Contribution to Profiles Document (Consistency Discussion)":

The attached Word version, with more easily read tables, is:

http://www.imc.org/ietf-ldup/mail-archive/doc00000.doc

In my view it is:

1) Unpredictable. See the description of "Extraordinary States" and
"Transient Extraordinary States".

As for "making some kind of sense to people using the system". In my view it
is:

2) "...preposterously complex. You seem to be tying yourselves in
knots with this...", to borrow a phrase used much earlier in a narrower
context from David Chadwick.

To find the actual context, grep through the archive. If you also read the
archive right through as I recently did, or even just the threads around
that comment, I think you will find that it is an understatement and
applicable to a much wider context than when originally made.

Many people seem to have expressed doubts about the consistency model but
since drifted away from active participation in LDUP work without proposing
an alternative.

Those remaining remind me of frogs succumbing to being boiled alive, while
others have hopped out of the pot as they noticed the temperature slowly
rising.

In fact the consistency model makes so little sense that I seriously doubt
that anyone COULD describe it in the requirements doc. That is not a ground
for simply omitting an essential feature of requirements (or "design
considerations"). It is grounds for thinking again.

OSI standards bodies are not notoriously reluctant to adopt standards that
are preposterously complex or do not make sense to people using the system,
but even they have dropped the URP proposal as a work item for X.500, after
it was initially introduced here as a joint work between IETF and ITU.

Returning to your comments on multiple-entry update consistency.

It might be interesting in an LDUP requirements RFC to seek input as to what
applications actually check after making changes to a set of multiple
entries, whether the results of reading back what they wrote actually
correspond to what they wrote. That would provide some guidance as to
existing applications that might be affected by the use of multi-master
instead of single master replication, since any such checks currently relied
on would cease to be conclusive with multi-master. However the WG has so far
shown no interest in obtaining any such input. Its requirements document is
not intended to solicit comments and was not prepared as required by the WG
charter, to precede architectural decisions, but is simply a formality, now
being rushed through without careful thought because the work that should
have been guided by requirements analysis has proceeded without any such
guidance.

Of course there may be many other applications that ought to check but don't
bother because the window for a race condition is fairly small. That is not
LDAP's problem since the data model never provided any such guarantee.

Dealing with "multiple-entry updates" is not within the scope of LDUP, but
explicitly left for future work on transactions. That does not mean it is
infeasible to solve within an LDAP context, just that it is not up to LDUP
to propose a solution, since it equally affects single server directories.
All LDUP has to do is avoid impeding a solution.

I understand there are also proposals for "grouped changes" which unlike
transactions, may not involve rollbacks, and may involve LDAP itself. I am
not familiar with them, but suspect that Active Directory (AD), as well as
URP might impede "grouped changes".

Where URP and MDCR differ, is that URP allows inconsistency between changes
to different attribute values of a single attribute of a single entry,
whereas MDCR does not allow conflicts to single entries at all, as in the
present LDAP/X.500 data model. AD takes an intermediate position. Because AD
only allows such inconsistency between entire attributes of a single entry,
while replicating each entire attribute as a whole, it does not impede
future work on transactions, whether or not it impedes future work on
"grouped changes".

This feature of URP breaks the LDAP/X.500 data model completely and even
makes it impossible to provide the mandatory "modifiersName" attribute,
because no particular DUA can be said to have modified an entry - it may be
a mixture of changes made by two or more DUAs. The same is true for the
intermediate position taken by AD.

Shouldn't a requirements RFC explicitly draw attention to this and solicit
input from other applications areas with applications that rely on the
existing data model and from operations and other areas as to whether they
have conflicting requirements for modifierName so as not to make life
difficult for sysadmins trying to resolve problems etc?

Microsoft and Novell can simply pass over the issue of modifiersName in
silence in their marketing blurbs and bury it deep in their technical
documents or not mention it at all. Is that the way the IETF should operate
too? What is the point of standards mandating modifiersName, if subsequent
"LDAP" standards can quietly drop it without at least presenting an
explanation for doing so? Do vendor initiated WGs have a special
dispensation?

Solutions for multiple-entry transactions, like those used with AD, are
likely to involve use of "child counts" and "consistency UUIDs" attached as
new attributes to each entry in a set of multiple entries affected by a
single transaction, so that those DUAs participating in a transaction can
roll it back in case of conflict. That is likely not to require changes to
LDAP itself, but either conventions or standards for applications that
require transactional support using an underlying directory service that
does not itself support any such concept as "multiple-entry updates",
because there are no multiple entry LDAP change operations (apart from the
extension for deleting a tree, which has a fairly simple solution for
conflicts).

As long as the LDAP/X.500 data model is not broken, applications can
implement such guarantees themselves, as Tim Howe points out in the quote
above. This may or may not require standardization.

Because URP mixes conflicting individual attribute values of a single entry
it is difficult to see how solutions such as child counts and consistency
uuids already deployed could be re-implemented at all. One cannot identify
the DUA reponsible for a change so it is difficult to see how to identify
the DUA responsible for initiating a transaction and therefore difficult to
see how to identify a transaction or roll it back. Therefore URP impedes
future work on "multiple-entry update consistency", while MDCR and AD do
not.

If you nevertheless support proceeding with URP as the only WG proposal for
replication, and leaving MDCR as an individual submission, you should also
support changing the following from the requirements draft rather than
leaving it unchanged:

      6.12 Multiple LDAP changes to a single server: If transactional
           consistency is propagated during replication, then multiple LDAP
           changes submitted to a single server SHOULD BE treated as a
           single 'atomic unit of work'.

The use of the word SHOULD is intended to require an explanation for
non-conformance. An explanation that a SHOULD requirement is not conformed
to because the method chosen for implementation does not support it is not
sufficient. If that was sufficient, the word MAY would be more appropriate.
Otherwise SHOULD means nothing at all since any implementation of any
requirement would conform to it.

An explanation that the requirement is not supported because there is no
feasible way to support it, is refuted by the fact that there are
applications using consistency uuids and child counts implemented on a
widely deployed commercial LDAP server (AD) that will be affected by failure
to support it. If you disagree and maintain there is no feasible way, then
you should support removing 6.12 entirely. What is the point of a SHOULD
with no feasible way to implement it?

You should also support an explicit statement in the requirements as to what
consistency guarantees LDUP does and does not offer.

That is what you said about requirements in your immediately preceding
message on this topic:

"The important thing is IMHO documenting IN DETAIL what consistency the
procedures guarantee; the requirement that the procedure proposals do
should be in the requirements document.

Apart from that, I think requirements are ready to go."

Could you please explain what has led you to change your view on that
between writing two successive messages on the same topic? It could not have
been the further discussion, because there hasn't been any. My guess is it
could be just because of the passage of another two months with nothing
happening. I take that as a sign of fundamental problems that will not be
fixed up by proceeding without changes. Do you really think the problems
affecting this WG will be reduced by issuing the requirements draft
unchanged?

Finally, I do not understand why you consider the requirements draft should
go forward without changing the current text, when you are the author of a
valuable draft on terminology concerning directories, and the current text
contains ridiculous definitions like these:

  Atomic operation - The ability to treat and contain several updates
  or attribute changes as a single operation for replication purposes
  to guarantee that the several updates or attribute changes are
  propagated to a replica as a single unit.

  Updateable Replica - A Non-authoritative read-writeable copy of the
  replicated information. Such that during conflict resolution a
  authoritative master takes precedents in resolving conflicts.

Please read it carefully again.

That definition of updateable replica could only have been written prior to
the group attaining any understanding at all of multi-master replication
issues, and if a better understanding has been acquired since, it looks like
the requirements draft has not been seriously reviewed after reaching that
understanding. (In fact it hasn't even been proof read).

The definition of "Atomic Operation" looks like a product of the Sirius
Cybernetics Marketing Division's weasel words department, deliberately
intended to obscure the issue, not like an IETF document at all. Even
Microsoft just shut up about it rather than inventing a new definition.

Every single scenario justifying requirements for multi-master replication
in the draft is irrelevant to the actual uses of multi-master replication.
The only scenario that does actually involve multi-master replication does
not identify it as having any connection with multi-master replication, and
the ONLY requirement stated for multi-master replication is:

The document was obviously (and verifiably) drafted before any serious
attention to the issues involved in multi-master replication, and has not
benefited from ANY user (as opposed to implementor) input as to what user
requirements might be.

---

From: "Albert Langer" <[email-masked]>
Date: Mon, 19 Jun 2000 17:19:42 +1000

Ryan,

I was hoping you might clarify what you want corrected as having been
misquoted.

All I know from either your previous message or two recent messages is that
I accurately quoted what you said then, prior to whatever private
discussions you tenuously recall having had since, about issues which you do
not recall and suggestions you do not recall for dealing with them.

On reflection I've decided to send a detailed refutation of your claim to
have been misquoted by private email first, in the hope that you might act
on my requests below, and thus avoid unnecessary feuding in the list:

If my summary does not accurately convey your previous or present views, it
should certainly be corrected.

Please note the way that Harald expressed his current view that the
requirements draft should go forward unchanged, which differs from the view
I quoted from him earlier, without throwing in an allegation that I had
misquoted him. Is there any reason why you could not do that?

1. If in fact you now believe the requirements doc should go to final call
without further substantive changes, or have no opinion either way yet,
please simply say so.

2. Again if your view now is that MDCR should be added to the WG agenda, or
should be left as an individual submission, or if you have no view either
way, please simply say so.

3. Please also withdraw your claim to have been misquoted, as you very
plainly stated that a requirement for atomicity should be made explicit,
whatever your views may be now, and you very plainly did not previously make
a clear statement either way as to what should happen to the MDCR draft, and
have still not done so.

As well as attempting to objectively summarize what has happened, I also
stated my view as to what should be done. If you are disagreeing with that
view, that has nothing to do with "misquoting".

If my view is wrong, it should be refuted (again, see Harald's message,
which I obviously disagree with, but is entirely germane to the topic).

In case you are unclear about whether URP does or does not maintain
atomicity, here is Alison's explicit statement that the whole point of the
"primitives" was a design choice not to do so.

"When you say "client operation atomicity" (a), do you mean the atomicity of
the operation accepted by the DSA before it is turned into primitives?
This is not maintained.  This is just why the primitives are needed.  The
atomicity of the primitives is maintained."

http://www.imc.org/ietf-ldup/mail-archive/msg00340.html

The consequences of that choice can be seen in their full glory by wading
through the email archives or more starkly in Alison's "Contribution to
Profiles Document (Consistency Discussion)":

A very brief summary is that with Single Master you get only (atomic and)
serialized operations, but with the (URP version of) Multi-Master you also
get "Extraodinary States" and "Transient Extraordinary States". But wait
there's more ... read the whole thing and think about what people are likely
to deploy and what people are likely to implement after considering the
description of their options in that profile. I don't think its going to be
(URP) Multi-Master if they can possibly avoid it.

As I don't like to leave a public allegation of misquoting unanswered, I am
also CCing the detailed refutation to the WG chairs as a substitute for
letting of steam in the list. No response from them is necessary.

Looking forward to a more productive exchange of views in future,

Seeya, Albert

---

From: "Ryan Moats" <[email-masked]>
Date: Mon, 19 Jun 2000 09:13:40 -0500

On the subject of the requirements / URP draft:

1. I still believe the requirements draft should be more
explicit about atomicity of operations being maintained
across replication.  In my various re-readings of this
draft, I have at times found justification for both
sides (maintain and do not maintain) and I still think
that maintaining atomicity is necessary.

2. The URP draft should be explicit in how it maintains
atomicity of operations across replication.  I'm pretty
sure from my last perusal it doesn't now.

3. Once URP maintains change atomicity, the "modifiersName"
issue in my mind goes away.  There may be others that
still remain...

On the MCDR draft, I'd like to take some time to get
some clarification on some things in the MDCR draft
that confuse/worry me...

1. The draft seems, from my reading, to be neutral about
whether the change tree carries around full objects, full
attributes, or attribute change lists.  Now, I may be
confused about draft's neutrality, but I'd certainly like
its position on this point to be more specific.  I would
argue for carrying the most efficient representation of
the change in the tree, which while I think should be
attribute change lists in most cases, I'm willing to
admit could be dependent on the nature of the change to
maintain atomicity.

2. The whole "weeding and summarizing" discussion left me
confused and therefore worried.  There seems to be an
unstated assumption that all replicas are "well-connected".
My concern is that if one or more replicas are
"sporadically-connected" then the size of the trees could
become an issue.  Again, I may be missing something in the
draft, but if so, I'd claim it is buried.  Because the
whole discussion left me confused, I think some clarification
in the form of an example would help.

Ryan

---

From: Russel F Weiser <[email-masked]>
Date: Mon, 19 Jun 2000 12:19:46 -0600

I'm sorry that I have been out of touch regarding the LDUP and thr replication
requirements issues.
I had assumed that it had gone to final call and there were no issues. I will
review your draft, and make comments. I will update the draft if it needs it to
prevent expiration.  Not being at the last IETF meeting I need to catchup on
this issue. I do find it interesting that I have had no direct email from you or
anyone else in LDUP  to notify me of any issues. I would assume that if you wish
express concerns that you might consider an direct EMAIL after all its on the
draft and also there has been very little activity of late.

cheers
RFW

---

From: "Albert Langer" <[email-masked]>
Date: Tue, 20 Jun 2000 08:25:23 +1000

[AL]
Ryan,

Thanks for your thoughtful comments in both private and public email.

I'm taking it that there is nothing you require corrected in my attempt to
summarize your view in a substitute for a status report - ie as I reported,
you did not support the requirements draft proceeding to final call
unchanged and you did not express any conclusion as to whether the MDCR
draft should be added to the WG agenda.

[RM]

[AL]
Understood.
URP certainly does not maintain atomicity and is explicit about that. This
problem is not in the URP draft, but in the requirements draft. If there was
a requirement to maintain atomicity there would be a completely different
URP (not necessarily MDCR).

In my reading of the requirements draft I found no useful discussion of the
arguments for and against maintaining atomicity at all. The discussion of
various "models" in "4. Applicability Statement" just obscures the issue by
confusing it with consistency between replicas and doesn't actually lead
anywhere relevant.

I think something like this should be in a new requirements draft:

"Changes to an LDAP directory can be:

a) Locally available
b) Atomic
c) Never revoked by the directory service

Pick any two.

This draft provides an explanation of why a choice must be made between b)
and c) for directory uses that require multi-master replication to achieve
a). It explains what sort of directory uses do require a) and explores the
implications of the possible choices between b) and c) and various
combinations and options for achieving them. Input is requested from other
areas as to what requirements they have for existing and future directory
applications and operational uses of the directory that should affect the
architectural choices now being made by the WG."

The architectural choice reflected in URP, flowing from the architecture
draft, is to achieve c) by not attempting b).

There are valid arguments for such a choice and there are valid (and in my
opinion stronger) arguments against it. Both have been repeatedly expressed
in the list but with no resolution,
and most importantly, no attempt to seek requirements input to guide
resolution.

Unfortunately most of the people arguing for atomicity seem to have done so
in the context of detail discussion of the URP draft, rather than as a
requirements issue, and seem to have lost interest without proposing a
concrete alternative.

My point in objecting to the requirements draft rather than waiting until
architecture or URP comes up for "final call", is that those choices should
be made on the basis of an analysis of user requirements, after seeking
input from the areas affected, not solely on the basis of implementor
preferences (while fully taking those into account).

[RM]

[AL]
Agreed. "modifiersName" is impossible without atomicity, and easily achieved
with it (as are many other things). It's maintaining atomicity in a
multi-master directory that is difficult and requires careful analysis of
the pros and cons. MDCR was intended to fill a void by showing that it is
possible to achieve atomicity. Any other means of doing so would equally
meet this objection to the requirements draft. (Though I actually wrote MDCR
as a simple extrapolation from that requirement for atomicity, to what would
have to be done to meet it without compromising other plausible
requirements, so I suspect any other solution would be pretty similar.)

[RM]

[AL]
Correct. I am not neutral about the granularity of the directory data model
being at the level of each entry being a single unit - as opposed to Active
Directory (AD) treating each attribute as an independent unit and URP
treating each attribute value as an independent unit. Both the latter
violate atomicity of directory operations which treat the entry as the unit.

However, as you noted, I mentioned in the MDCR draft that the replication
algorithms described there could also be applied at any of the 3 levels of
granularity.

Using the MDCR method at the level of attributes would I believe be better
than the way AD does it at that level, but would not achieve the main
objective of atomicity.

Using the MDCR method at the level of attribute values might also be an
improvement for URP, though carrying at least as much overhead as URP does.

However these possible contributions are not the basis for my objection to
the requirements.

As regards efficient propagation of the tree via the report propagation
protocol I believe the encoding on the wire is as efficient and as simple as
possible by just conveying the actual LDAP protocolOps for the changes. The
only redundancy is for the relatively small proportion of changes that
affect the Directory Information Tree (DIT), ie add, del and modifyDN,
rather than just the Directory Information Base (DIT), ie modify (for
changes to non-distinguished attributes and values).

Hmm, well actually including the targetDN in the protocolOP for modify as
well as the entryUUID  might technically be redundant but its necessary and
could occupy less space on the wire than the repetition of entryUUID for
each attribute value in URP.

Incidentally the MDCR draft also proposes adoption of the Coda report
propagation protocols  adopted by AD. I believe that would be a considerable
improvement without requiring any substantial change to either the
requirements or architecture and especially beneficial to the existing URP
design as there is a lot of avoidable complication there simply because
reports are propagated out of sequence by each replica.

That is completely independent of the issue of atomicity.

[RM]

Ryan

[AL] If the WG accepts the draft as being on its agenda for discussion I
think a lot more work will be needed on it, including detailed examples (and
diagrams) for the weeding and summarizing process as you suggest.

A simpler approach to conflict resolution would just resolve each conflict
immediately, at each DUA that encounters a conflicting change, by
suppressing the change with the lower version number etc and propagating
only 1 survivor (which may itself be suppressed at another DUA with the
survivor propagated from that conflict in turn reaching the previous DUA and
suppressing the previous survivor there). This is more or less what AD does
(for attributes). URP makes the fatal mistake of splitting the operations
into primitives so as to effectively merge conflicts instead of resolve
them, and adds the serious mistake of giving higher priority to timestamps
than to version numbers. This means the "survivor" at each DUA separately is
just the latest, with no actual conflict resolution at all.

The tree is especially important for "sporadically-connected" replicas since
they are the most likely to generate conflicts. In most cases the tree would
be trivial or very small, but if it is not recognized as a tree, that
"simplification" makes everything else become incredibly complicated.

I don't see a large tree as an implementation problem in itself - the
overhead is only about 12 bytes of RAM per conflicting change.

However a large tree would indicate an inappropriate use of multi-master
replication in situations where conflicting changes are common relative to
the replication interval. The directory model of loose consistency is
intended only for situations with a high ratio of reads to writes and
therefore a low proportion of conflicts relative to the replication
interval.

This is completely obscured in the requirements draft which treats
multi-master as solving all sorts of problems it has no relevance to
whatever, while ignoring its primary importance for enabling local
availability of changeable entries and not explaining that conflicting
changes to the same entry are an unavoidable consequence of that rather than
a useful feature.

Entries should be re-structured as families of entries for different sets of
attributes or attribute values where local availability of changes is
required with a ratio of conflicting changes high relative to the
replication interval (eg for maintaining distribution lists).

If that is not possible, multi-master replication should not be used at all.
(Or equivalently, a "primary replica" used for writes).

If a high rate of conflicts and therefore large trees is due to coordinated
transactions, these should use a database designed for that rather than a
directory. They will not work well when "sporadically-connected" however one
does it.

The conflicts form a tree in reality, whether it is recognized or not. If
that tree becomes large then URP would produce a high rate of "Extraordinary
States" and "Transient Extraordinary States" while MDCR would produce a high
rate of revocation notices because it has at least recognized the reality.
Neither is appropriate for directory applications. (In my view even an
extremely low rate of "Extraordinary States" without any means except manual
administration to recover from them is completely unacceptable).

---

From: "Albert Langer" <[email-masked]>
Date: Tue, 20 Jun 2000 08:53:04 +1000

[RFW]
I'm sorry that I have been out of touch regarding the LDUP and thr
replication
requirements issues.
I had assumed that it had gone to final call and there were no issues. I
will
review your draft, and make comments. I will update the draft if it needs it
to
prevent expiration.  Not being at the last IETF meeting I need to catchup on
this issue. I do find it interesting that I have had no direct email from
you or
anyone else in LDUP  to notify me of any issues. I would assume that if you
wish
express concerns that you might consider an direct EMAIL after all its on
the
draft and also there has been very little activity of late.

cheers
RFW

[AL]
Sorry I didn't send you an email. I would have done so if I had been
participating in the LDUP list prior to the IETF meeting. Unfortunately as
you can (now) see from the minutes and the acknowledgement in the draft I
only popped out of nowhere at the meeting and was then given a week to write
a draft. I copied it to Alison (for her and Steve) as most of it related to
URP and had already had discussions about it with them during the IETF
meeting, but I just assumed you and Ellen would get it from the formal
process process initiated in the minutes. I guess Chris and John must have
made the same assumption.

Anyway you're here now so I guess the requirements draft isn't dead and I
look forward to your comments after catching up. Is Ellen onboard too?

Although the other authors (except Roger) are still also AWOL it is for a
much shorter period, but it looks likely that there could be similar
communications failures.

Could Chris and John please confirm that they are sending direct emails to
all the authors of missing drafts now? A LOT of time can be wasted by these
sort of communication failures.

I get the impression from reviewing the archives that quite a few people
have dropped out of actively following the list, who were once closely
interested. Are they just lurking or should anything be done to draw their
attention to this discussion (apart from re-chartering the WG, already
proposed ;-)

Given that even authors of critical drafts have not even been lurking, I
think something should be done to make sure that everybody likely to be
interested in contributing has a further opportunity to do so.

In particular I am surprised at not having heard from Alison and Steve as
they indicated some interest in further discussions when we met at the IETF.

I really don't have much confidence in email for avoiding communication
failures - very inadequate multi-master replication protocol for that, due
to no closed loop for confirmations ;-)

---

From: "Steven Legg" <[email-masked]>
Date: Mon, 26 Jun 2000 15:37:22 +1000

Albert,

> [mailto:[email-masked]]On Behalf Of Albert Langer
> Sent: Tuesday, 20 June 2000 8:25
> To: 'Ryan Moats'
> Cc: [email-masked]
> Subject: RE: various comments (was Re: What's going on? - Status of
> Requirements...)

[snip]

> URP (not necessarily MDCR).

Atomicity is not the right concept to be arguing about in a multimaster
replication environment. Consider this example.

At server S1 at time t1 a user modify request on an entry adds attribute
A1 and replaces the existing values of attribute A2. At server S2 at
time t2 (t2 > t1) a user modify request replaces attribute A2. Some time
later the modify from server S1 arrives at S2. If S2 performs this
operation atomically it will replace the newer (time t2) value(s) of
attribute A2 with the older values (time t1). This is clearly the wrong
thing to do. If S2 adds the new attribute A1 but ignores the replacement
of A2 (as URP would do) then it produces the same outcome as the
serial execution of the two modify requests, though the action doesn't
fit the usual definition of "atomic".

If we are going to discuss atomicity in replication then we need a more
meaningful definition. URP makes all the replicas produce an outcome
that is equivalent to the serial atomic execution of all the updates.
That is as atomic as it needs to be.

The real question revolves around the preconditions of a user update
request. If we look at the equivalent serial processing of a collection
of updates performed at two or more multimaster replicas then some of those
updates will be "executed" against a different database state than actually
existed at the time they were really performed by one of the replicas.
The URP philosophy is that most of the time for most applications it doesn't
really matter. The MDCR philosophy is that it is better to completely
disregard the user's update, after the fact, just in case the state of the
entry did matter.

[snip]

> with it (as are many other things).

You seem to be assuming that the DSA maintained operational attributes
are being independently maintained by each replica. The intent in URP is
that the replica executing the user update request will modify the
operational attributes, like modifiersName and modifyTimestamp, as
required, and that these changes will also be reflected in the primitives
sent to the other replicas. Those other replicas will keep the values
of these operational attributes with the highest CSNs, which will
correspond to the latest change to the user attributes. Exactly the
same outcome as a serial execution of the updates in CSN order.

[snip]

> changes to non-distinguished attributes and values).

The replicating servers aren't necessarily LDAP or X.500 DSAs, so the update
protocol operations aren't necessarily just LDAP, DAP or DSP operations.
Also, these protocols are still subject to change so additional operations
may be defined in the future, or existing ones extended. LDAP also has
a means for vendors to define proprietary operations. We can't expect LDUP
implementors to cover all the possibilities.

The original request also doesn't carry the changes to operational
attributes or other vendor specific DSA invoked changes such as might
occur to maintain referential integrity of DNs.

Breaking all update requests into replication primitives gets around these
problems.

[snip]

> reports are propagated out of sequence by each replica.

Propagating the changes strictly in CSN order wouldn't make much difference.
The "complexity" arises from the requirement to process local changes out
of order with updates reported by other replicas. Whether those remote
changes are reported in order becomes irrelevant.

> diagrams) for the weeding and summarizing process as you suggest.

I too have some concerns about the "weeding and summarizing" stuff.
In particular it is not possible to guarantee that all replicas will
converge toward the same state. A version of an entry cannot be made
"Durable" until after some unspecified delay to see if any higher version
shows up. For an entry version to become the definitive (durable) version
depends not only on earlier events but also on events that are yet to occur!
If the waiting time is chosen badly the replicas will quickly diverge.
No matter what delay is chosen there is always a chance that a higher
version will pop up immediately afterward.

Also, I can also construct examples where two replicas endlessly flip
back and forth between two competing strands with no entry versions ever
becoming durable.

> suppressing the previous survivor there).

I don't think this is enough to solve the problems mentioned above.

> This is more or
> less what AD does
> (for attributes).
> URP makes the fatal mistake of splitting
> the operations
> than to version numbers.

The LDUP CSNs can be used either way. URP doesn't care, it just sees
a monotonically increasing value.

> This means the "survivor" at each
> DUA separately is

You would also need to store enough information to reconstruct the
previous versions of an entry on each strand. To change the current
entry version from one strand to another requires unwinding index changes,
etc, back to the common branching point and then applying the saved
change requests on the new strand. URP never has to go backwards.

[snip]

> administration to recover from them is completely unacceptable).

I don't expect many LDAP client application developers will want to
deal with the complexities of out-of-band revocation notices, or deal
with restoring the application context of the original change so it
can be repeated. The enterprising ones will probably just send a series
of spurious changes to an entry after each real change just to "up" the
version number and so increase the chance of the real change sticking.

In effect, what we have done with URP is hardwire a reasonable conflict
resolution mechanism (best effort merge) that is (we think) good enough
for most uses of the directory. But that doesn't leave the remaining uses
out in the cold. Alison and I have previously mentioned in passing that
we have an idea for providing strong consistency and transactions with URP.
This is a good time to sketch out what we mean.

This is the sequence of events for a user update request requiring or
requesting strong consistency:

The receiving replica (let's call it the primary) opens a session with
each of the other replicas and sends a request for all update primitives
up to and including the latest change to the target entry. This is just a
variation on a regular LDUP session. However the other replicas will also
lock the target entry to prevent any local changes to it. Some sort of
deadlock detection will be necessary.

The primary replica applies all the primitives it has received and then
attempts the user request. If it fails with an error the sessions with
the other replicas are closed and they drop the locks on the target entry.
If the request succeeds the primary server sends
the primitives up to and including the ones generated from the current
user request, then closes the sessions causing the locks to be
released. If the primary doesn't send the latest primitives the other
replicas will just import them the next time they action a strong
consistency update on the same entry. Two-phase commit isn't required.

Handling transactions is straightforward. The initial request from the
primary replica specifies a range of affected entries (maybe with
something like a search filter) instead of just the one target entry.
The primary is also allowed to send multiple requests within the same
session since it won't generally know all the entries affected by
a transaction at the beginning.

The above scheme requires all updatable replicas to be available to perform
a strong consistency update but there is a more general scheme that
allows some of the replicas to be unavailable. If there are N replicas
then it is only necessary to contact M (M > N/2) of them to make an update
provided N-M+1 of them are contacted before evaluating any strong
consistency query. The original description was the special case of M = N.

Regards,
Steven