Subject: RE: [AAA-WG]: Support for multi-process

From: Cain Brian-BCAIN1 <[email-masked]>
Date: Tue, 12 Jun 2001 16:54:42 -0500

> -----Original Message-----
> From: David Mitton [mailto:[email-masked]]
...
> There are no constraints on the value of the source port of 
> the connection. 
> It should be assumed to be an ephemeral port.  The well-known 
> port is in 
> use for listening.  A listening socket cannot make connections.
> 
> This is how most TCP protocols work.

Okay, so it would seem that duplicate connections are not preventable,
right?  Or at least, not for the common denominator.

So, given that, should we revert to Mark's latest offering for the state
machine?  (Or did I miss a "multiple transport connections are OK now"
verdict?)

Is my suggestion that implementations periodically check for duplicates
unreasonable/difficult to implement/inefficient?

-Brian

---

From: Pat Calhoun <[email-masked]>
Date: Tue, 12 Jun 2001 16:34:00 -0700

> right?  Or at least, not for the common denominator.

No, they are in fact preventable, by ensuring that the source port is NOT
the same as the destination port. So, if we require that the connection NOT
be established on the listening port, then we can ensure that both sides of
the connection will create a separate connection.

PatC

---

From: Jacques Caron <[email-masked]>
Date: Wed, 13 Jun 2001 15:42:34 +0200

>the connection will create a separate connection.

Isn't that exactly what we're trying to avoid?

Getting completely lost here :-(

Please select the right one:

1. We want to make sure there is never more than one active connection 
between any two processes
2. We don't care if there are multiple active connections between any two 
processes
3. The problem is something completely different?

As I understand it, we want (1) and the original problem was that the state 
machine uses/used IP addresses to detect duplicate connections. Just make 
it use the remote host identity (Host descriptor) received in the CER/CEA 
(the Host descriptor MUST be always the same for a given process). Don't 
know if that affects the state machine or not, though.

BTW, I'm not sure we need such a complex state machine. Connecting to a 
peer and receiving a connection from a peer should be two totally distinct 
state machines (pretty basic in operation, one is: connect, send CER, wait 
for CEA, and the other is accept, wait for CER, send CEA -- plus of course 
error handling at all intermediate steps). Just add that upon receipt of 
the remote's Host descriptor (in a CEA in the first case, in the CER in the 
other), it should be compared against those we already have a connection 
to, and if there is a match, the connection established by the peer with 
the lexically greater host descriptor has to be shut down).

I would add that having a single state machine for both is quite hard: how 
do you know a connection you receive actually goes into the same state 
machine as one you initiated until you received the Host descriptor?

Another option (now that we ruled out the "use same port for listening and 
connecting" because socket implementations are just too dumb and don't 
accept a perfectly legitimate TCP case) would be to have each server 
establish an outgoing connection, and use that one for sending, while still 
listening on both. Stupid waste of resources (sockets/file descriptors...) 
though.

Jacques.

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

---

From: Pat Calhoun <[email-masked]>
Date: Wed, 13 Jun 2001 06:43:33 -0700

> Isn't that exactly what we're trying to avoid?

We do want to avoid multiple connections, but we have identified that if all
of the ports are the same, both sides will think that they've setup the
connection, and this would either require:
1. additional massive state machine complications
2. change of CER/CEA message back to an indication-like message, or
3. Require that both sides issue CER, and CEA is just an ack and contains
   no capabilities.

> 3. The problem is something completely different?

We want a single connection, and we want to be able to identify when
multiple connections are active so we can close them. This is a separate
issue from what I was talking about.

> know if that affects the state machine or not, though.

According to Paul, the changes should be minimal.

> the lexically greater host descriptor has to be shut down).

but you need to handle the cases where unexpected events occur while in
intermediate (non-open) states. That's where the complexity occurs. I agree
that we could simplify the SM by allowing all connections to move towards the
open state, and close it when it reaches that point. That would reduce the
number of interim states that need to check for existing connections, but
you could *still* get into a race condition :(

> machine as one you initiated until you received the Host descriptor?

well, either you initiate the connection, or you don't. It doesn't seem
that hard to me :(

> listening on both. Stupid waste of resources (sockets/file descriptors...) 

I'd prefer a good solid state machine. Let's be honest here. I wrote the state
machine in -05 is less than a day. This isn't black magic, or rocket science :)

PatC

---

From: Jacques Caron <[email-masked]>
Date: Wed, 13 Jun 2001 16:55:57 +0200

At 15:43 13/06/01, Pat Calhoun wrote:
>connection

If you're referring to using the same port for listening and connecting, 
then the issue is mostly that most socket implementations just don't allow 
that. If they allowed it (as BSD stacks do with SO_REUSEPORT), then there 
could be only a single connection established, which would solve the problem.

>you could *still* get into a race condition :(

At any time you have an error (connect fails, timeout waiting for CER, 
connection dropped), shutdown the connection and go back to closed state. 
Still need some mechanism to introduce a delay before the next retry, 
though, not sure that's present anywhere in the current state machine or 
elsewhere in the draft?

>well, either you initiate the connection, or you don't. It doesn't seem

I meant to identify a connection that you receive as being associated to 
the same peer as one you already started establishing.

Imagine host A has processes A1 and A2, and host B has process B1.

B1 initiates a call to A1 and is somewhere in the state machine.
A2 initiates a call to B1. How does B1 know this is a connection from A2 
(which would be a R-Rcv-Conn-Req in state Closed) rather than from A1 
(which would be a R-Rcv-Conn-Req in some other state of the machine 
associated to the connection to A1)? When B1 accepts the call, it only 
knows the IP of A, which is the same for A1 and A2. It will know it's a 
different host only when it receives a CER from A2.

And if the only moment you can "match" two connections to the same host is 
when you receive the CER (or CEA), no problem, just do an election at that 
time and shutdown the connection that lost.

>machine in -05 is less than a day. This isn't black magic, or rocket 

Nope. Here is my proposal:

Initiating a connection state machine

       state            event              action         next state
       -----------------------------------------------------------------
       Closed           Start            Snd-Conn-Req     Wait-Conn-Ack

       Wait-Conn-Ack    Rcv-Conn-Ack     Snd-CER          Wait-CEA
                        Rcv-Conn-Nack    Cleanup          Closed
                        Timeout          Error            Closed

       Wait-CEA         Rcv-CEA          Process-CEA,     Elect
                                         Elect
                        Peer-Disc        Disc             Closed
                        Timeout          Error            Closed

       Elect            No-Elect         Nothing          Open
                        Win-Elect        Close other      Open
                        Lose-Elect       Error            Closed

       Open             Send-Message     Snd-Non-CER/CEA  Open
                        Rcv-Non-CER/CEA  Process          Open
                        WatchDog-Timer   Snd-DWR          Open
                        Rcv-DWA          Process-DWA      Open
                        Stop             Disc             Closed
                        Peer-Disc        Disc             Closed
                        Rcv-CER/CEA      Error            Closed

Receiving a connection state machine

       state            event              action         next state
       -----------------------------------------------------------------
       Closed           Rcv-Conn-Req     Snd-Conn-Ack     Wait-CER

       Wait-CER         Rcv-CER          Process_CER,     Elect
                                         Elect
                        Timeout          Error            Closed

       Elect            No-Elect         Snd-CEA          Open
                        Win-Elect        Close other,     Open
                                         Snd-CEA
                        Lose-Elect       Error            Closed

       Open             Send-Message     Snd-Non-CER/CEA  Open
                        Rcv-Non-CER/CEA  Process          Open
                        WatchDog-Timer   Snd-DWR          Open
                        Rcv-DWA          Process-DWA      Open
                        Stop             Snd-Disc         Closed
                        Peer-Disc        Disc             Closed
                        Rcv-CER/CEA      Error            Closed

No-Elect means no match was found. Win-Elect means this connection won. 
Lose-Elect means this connection lost. Election is based on Host received 
against Host local server would send.

Jacques.

---

From: David Mitton <[email-masked]>
Date: Wed, 13 Jun 2001 15:56:07 -0400

>the connection will create a separate connection.

The point I'm trying to make is that people are trying to make bad 
assumptions about port numbers relative to running server instances.

I think we want to leave actual port values out of it and use the contents 
of the message as the server identifier.

Dave.
----------------------------------------------------
David Mitton                            978-288-4570
Advisor, Nortel Networks
[email-masked]   or      [email-masked]

---

From: "David Mitton" <[email-masked]>
Date: Wed, 13 Jun 2001 16:10:58 -0400

>could be only a single connection established, which would solve the problem.

It's just not done that way, give it up.

>though, not sure that's present anywhere in the current state machine or

I'm not sure that a delay is required.

>time and shutdown the connection that lost.

Yes.  We've been here, done this.  That's the idea.
Initiator opens the connection, and sends the message, the listener looks 
at what he gets and proceeds.

And the only time you need to perform the election process is if you have 
more than one connection.  If you have no other connections up, it's 
pointless.

And it seems it's true that we need to use information in the message to 
identify the client end, since there may be multiple separate servers 
running on the same IP addressed system using configured and/or ephemeral 
non-default port numbers.

So what's the question?

>Nope. Here is my proposal:
<ignored... I've already spent too much time optimizing our prior work>

Dave.
------------------------------------------------------------
David Mitton                            ESN: 248-4570
Advisor, Nortel Networks                 978-288-4570
Wireless Solutions, IP Mobility
Billerica, MA 01821               [email-masked]