Subject: Minutes from failover draft Nov. 14 conf. call

From: Kim Kinnear <[email-masked]>
Date: Wed, 15 Nov 2000 14:40:05 -0500

We held a conference call yesterday (Tuesday, Nov. 14 2000)
concerning the failover draft:

http://www.ietf.org/internet-drafts/draft-ietf-dhc-failover-07.txt

This was the final conference call before the -08 draft is
submitted.  We had a bit of a rocky start since I had managed to
mis-schedule the conference call bridge, but we finally got
started about 30 minutes late.

The following people attended some or all of the meeting (and a
big thanks to all of them):

Ralph Droms       DHC WG chair        EST   [email-masked]
Richard Jones     Checkpoint Software PST   [email-masked]
Kim Kinnear       Cisco Systems       EST   [email-masked]
Ted Lemon         Nominum             MST   [email-masked]
Greg Rabil        Lucent Technologies EST   [email-masked]
Mark Stapp        Cisco Systems       EST   [email-masked]
Sri Hary Vengadasubbu Cisco Systems   EST   [email-masked]
Bernie Volz       IPWorks             EST   [email-masked]

The agenda was:

   a) Issues learned from implementing the -06/07 draft.

Here are the minutes (which I've written in third person for,
believe it or not, clarity):

----------------------------------------------------------

This meeting largely revolved around Ted Lemon telling the rest
of us his comments on the draft, based on his experience of
implementing the major elements of the -06 and -07 versions of
the draft.

Ted's overall comments were that, in general, he found the draft
to be solid enough to implement from (modulo his further
comments).

We went through it page by page, and for completeness I will
replicate that information here.  The page numbers refer to the
-07 version of the draft.  I've ordered it by page number,
not the order of discussion in the meeting.

-----------------
Pg.  68: Table 7.9.1 Need to be fixed to say that the
reject-reason option MUST be sent if it is a rejected packet, and
MUST NOT be sent otherwise.

-----------------
Pg.  83: Startup doesn't talk about states than partner down.  In
particular, for other connected states (other than NORMAL, which
is mentioned), what do you do when you come up (since by
definition you are not connected then)?

Kim agreed to clarify that in the draft.

-----------------
Pg.  86: The state RESOLUTION-INTERRUPTED does not appear in the
table.

Kim said that the RESOLUTION-INTERRUPTED state was a new state,
and clearly he missed some places in integrating it into the
draft, but that he would ensure that it appeared in all of the
proper places.

-----------------
Pg.  87: Ted said that he thought that the time that is listed as
"the current time", should be "the time the server came up".
These could be different in this case because there is an
UPDREQALL->UPDDONE sequence between when the server came up and
the current time.

Kim said that he would think about this and almost certainly
change it to the "time the server came up", or explain why that
could not be done in the draft.

-----------------
Issue: Kim mentioned that a related issue to this section was
that many servers would implement failover with a
"partner-up-to-date" bit, to distinguish IP addresses which were
or were not up to date at the partner.  This approach works just
fine -- except when the partner server changes or has no memory
of any previous information.  While the UPDREQALL message was
invented to solve this problem, it doesn't cover many cases,
e.g., when due to a reconfiguration of a server an IP address
moves from one pool to another, and the pools have different
failover partners.  Kim offered a failover-endpoint sequence
number as a possible solution to this problem, and said that he
would be thinking about how much of this to put in the draft and
how much was just an implementation issue.  But that he would put
*something* in the draft so that people wouldn't have to stumble
over this problem as he did.

-----------------
Pg.  87: Ted said "What happens if the communications are
interrupted if you are in RECOVER state?".  Kim said that you
stay there, but Ted pointed out that this was not explicit in the
draft, just implicit from the lack of a transition arc from the
RECOVER state in the state transition diagram.

Kim said he would clarify this.

-----------------
Pg.  91: Ted said that he had a situation where two servers would
go into COMMUNICATIONS-INTERRUPTED state and NORMAL state and
back again.  He felt that it probably wasn't the draft's fault,
but it still happened and he finally fixed it.

Kim said that he would see if the draft could be tightened up in
this area.

-----------------
Pg.  91:  Ted said that if you were in PARTNER-DOWN state and
received a SHUTDOWN message (i.e., state) from the partner
server, that the transition into PARTNER-DOWN state was not well
documented.

Kim said that he would clarify this, at least in the text, and
*maybe* in the state transition diagram picture.

-----------------
Pg.  95: Ted said that, with loadbalancing enabled, if the two
servers transitioned into NORMAL state at different times (much
different anyway), that the loadbalancing wouldn't work right.

Kim and the group discussed this and decided to add a new state
something like POTENTIAL-CONFLICT-DONE for the primary, such that
it would be responsive to *all* requests (like
COMMUNICATIONS-INTERRUPTED), and it would hang out here until the
secondary was ready to go to NORMAL state, when they would both
go to NORMAL together.  This is analogous to the
RECOVER/RECOVER-DONE synchronization point (which is needed for
a different reason).

------------------
Pg.  97: Ted said that you could make transitions into SHUTDOWN
state from lots of other states, but that wasn't called out
anywhere.

Kim said that he would fix this problem.

-------------------
Pg.  108: In the hash bucket assignment algorithm, it doesn't say
what server the hash bucket bits are for -- the primary or
secondary.  It implies that they are the bits that are on for the
secondary.

Kim said that he would make this clearer, and add text on page 26
about this as well.

-------------------
Issue:  The group talked about the problem where a lease which
was leased by the secondary expires and goes back to the primary,
and then the client which wanted it asks the secondary for it
(because of load balancing).  We discussed that this was more or
less the way it had to be, because the more important issue is to
have the secondary have enough addresses to allocate to clients,
and to do that you have to give the secondary leases soon after
they are allocated and not necessarily wait for some leases to
expire.

That said, we decided to put words in the draft to the effect
that the primary should attempt to send leases to the secondary
that have been leases by the secondary in the past and who's most
recent client would end up being processed by the secondary due
to load balancing.

------------------
Issue:  Ted talked about how he was thinking of another state
like BACKUP-RESERVED for IP addresses which would indicate it was
leased as a dynamic BOOTP lease.  We talked about why he would
want to know this, and the reasons were strong (so that he could
consider a client with a client-id and one without a client-id to
be the "same" client if the MAC addresses compared equivalent and
it was a dynamic BOOTP lease).

This is another "flag" for an IP address, kind of like the
reserved flag.  Since Kim had promised Steve G. that he would
build in a flags byte if there was another flag (and remove the
BACKUP-RESERVED) state as a way to handle reserved leases, Kim
decided to add that flags byte now.  (Thought it may have 16 bits
to start).

-------------------------------

That's it for now.

A *big* thanks to all who attended, especially Ted
who dug through his notes and code to pass along this
really useful information.

Cheers -- Kim