Subject: RE: [AAA-WG]: Support for multi-process

From: Cain Brian-BCAIN1 <[email-masked]>
Date: Tue, 5 Jun 2001 10:53:12 -0500

> -----Original Message-----
> From: Mark Eklund [mailto:[email-masked]]
> 
> Pat,
> 
> Here is my proposed solution for multi-process access.  

Looks good so far.  My only recommendation is non-technical -> call the Win/Lose Election events something more in line with their description, something like "Existing-Conn-Win" and "New-Conn-Win", maybe?

This is slightly off topic from the current thread, but I've been wondering a few things.  How do multihoming and ip aliasing interfere w/this model?

My understanding is that connections can be bound to any subset of the IPs available.  I've done a little digging, and this seems somewhat different for SCTP and TCP, but the conditions below are possible with both.

If that's the case, and the client listening on "w.x.y.z" is initiating connections from "p.q.r.s", it would seem that he could create undetectable duplicate connections.  It seems like quite a bit of effort has gone into the state machine to eliminate duplicate transport connections, so I figured this case needs to be mentioned.

I did a quick search of the list archive, and couldn't find any reference to this sort of situation.  Perhaps it should be explicitly disallowed in the spec, if it isn't already.  (e.g. "All initiator connections from a given AAA entity must be bound to the same IP(s) as that entity's responder connection(s)").  This does not seem to prevent duplicate connections entirely (apparently outgoing TCP connections can't really logically bind themselves to more than one address), but it does seem to eliminate a few extra possibilities.

Can anyone think of a logical reason to have a AAA entity that functions otherwise?

-Brian

---

From: Pat Calhoun <[email-masked]>
Date: Tue, 5 Jun 2001 10:43:58 -0700 (PDT)

> This is slightly off topic from the current thread, but I've been wondering
> a few things.  How do multihoming and ip aliasing interfere w/this model?

Not a problem since the peer's identity is used, not IP addresses.

> If that's the case, and the client listening on "w.x.y.z" is initiating
> connections from "p.q.r.s", it would seem that he could create undetectable
> duplicate connections.  It seems like quite a bit of effort has gone into
> the state machine to eliminate duplicate transport connections, so I figured
> this case needs to be mentioned.

ok, but again, since the state machine doesn't use IP addresses, but rather
peer identities, and of course assuming the same identity is sent regardless
of the source IP address, the state machine would work fine.

> 
> I did a quick search of the list archive, and couldn't find any reference to
> this sort of situation.  Perhaps it should be explicitly disallowed in the
> spec, if it isn't already.  (e.g. "All initiator connections from a given
> AAA entity must be bound to the same IP(s) as that entity's responder
> connection(s)").  This does not seem to prevent duplicate connections
> entirely (apparently outgoing TCP connections can't really logically bind
> themselves to more than one address), but it does seem to eliminate a few
> extra possibilities.
> 

multi-homing should be support (actually already is), and I wouldn't feel
comfortable eliminating this.

Having said that, for those of you have have seen my comments to the transport
draft, that spec implies that a AAA protocol MUST allow multiple connections
between peers for non SCTP transports in order to minimize head of line
blocking.

Now that's is completely against what we've been doing in the base protocol
spec, so what do folks think we should do? Single or multiple connections?

PatC

---

From: "Ruslan R. Laishev" <[email-masked]>
Date: Tue, 05 Jun 2001 22:08:51 +0400

> spec, so what do folks think we should do? Single or multiple connections?
	multiple.

-- 
Cheers, Ruslan.
+----------------pure personal opinion------------------------+
    RADIUS Server for OpenVMS project - www.radiusvms.com
      [email-masked] - Forum for ISP running OpenVMS
                Mobile: +7 (901) 971-3222

---

From: Mark Eklund <[email-masked]>
Date: Wed, 6 Jun 2001 10:39:47 -0400 (EDT)

Brian, 

> Looks good so far.  My only recommendation is non-technical
> -> call the Win/Lose Election events something more in line
> with their description, something like "Existing-Conn-Win"
> and "New-Conn-Win", maybe?

I changed it to be Peer-Wins and Local-Wins and changed the
description wording.  In changing that, I found that I had a
part of the election process backwards.  The diffs between my
original proposal and the corrected one summarize the entire
process and are listed below.  Could you give your O.K. on
those?

I have included the changed text below the diffs.

-mark

--- 1,3 ----
***************
*** 88,96 ****
        I-Wait-Lock      Locked           Elect            I-Wait-Returns
  
        I-Wait-Returns   No-Contest       Elect-Unlock     Open
!                        Win-Election     Elect-Unlock/    Open
                                          R-Disc           
!                        Lose-Election    Elect-Unlock/    Open
                                          I-Disc           
  
        R-Wait-DRR       Rcv-DRR          Elect-Lock,      R-Wait-Lock
--- 63,71 ----
        I-Wait-Lock      Locked           Elect            I-Wait-Returns
  
        I-Wait-Returns   No-Contest       Elect-Unlock     Open
!                        Local-Wins       Elect-Unlock/    Open
                                          R-Disc           
!                        Peer-Wins        Elect-Unlock/    Open
                                          I-Disc           
  
        R-Wait-DRR       Rcv-DRR          Elect-Lock,      R-Wait-Lock
***************
*** 100,109 ****
        R-Wait-Lock      Locked           Elect            R-Wait-Returns
  
        R-Wait-Returns   No-Contest       Elect-Unlock     Open
!                        Win-Election     Snd-DRA/         Open
                                          Elect-Unlock/
                                          I-Disc           
!                        Lose-Election    Elect-Unlock/    Open
                                          Snd-DRA-Exist
                         Not-Allowed      Elect-Unlock/    Closed
                                          Snd-DRA-Fail
--- 75,84 ----
        R-Wait-Lock      Locked           Elect            R-Wait-Returns
  
        R-Wait-Returns   No-Contest       Elect-Unlock     Open
!                        Peer-Wins        Snd-DRA/         Open
                                          Elect-Unlock/
                                          I-Disc           
!                        Local-Wins       Elect-Unlock/    Open
                                          Snd-DRA-Exist
                         Not-Allowed      Elect-Unlock/    Closed
                                          Snd-DRA-Fail
***************
*** 184,194 ****
        No-Contest     An election was held, and there was only one
                       connection to this peer.
  
!       Win-Election   An election was held, and the new connection to
!                      the peer was the winner.
  
!       Lose-Election  An election was held, and the existing connection to
!                      the peer was the winner.
  
        Not-Allowed    Before the election was held, it was determined that
                       the peer was not allowed.
--- 159,169 ----
        No-Contest     An election was held, and there was only one
                       connection to this peer.
  
!       Local-Wins     An election was held, and the connection initiated 
!                      locally wins.
  
!       Peer-Wins      An election was held, and the connection initiated by
! 		     the peer wins.
  
        Not-Allowed    Before the election was held, it was determined that
                       the peer was not allowed.
***************
*** 289,295 ****
     shorter OctetString to be null-padded to the length of the longer,
     then performing an octet by octet unsigned comparison with the
     first octet being most significant.  If the peer is the victor, 
!    Lose-Election is returned otherwise Win-Election is returned.
  
  8.5 Most Common State Transitions
  
--- 264,270 ----
     shorter OctetString to be null-padded to the length of the longer,
     then performing an octet by octet unsigned comparison with the
     first octet being most significant.  If the peer is the victor, 
!    Peer-Wins is returned otherwise Local-Wins is returned.
  
  8.5 Most Common State Transitions

******************* New Proposed Text *******************

8.0  Peer State Machine

   This section contains a finite state machine, that MUST be observed
   by all Diameter implementations. Each Diameter node MUST follow the
   state machine described below when communicating with each peer.
   Multiple actions are separated by commas, and may continue on
   succeeding lines as space requires. Similarly, state and next state
   may also span multiple lines as space requires.

   There may be at most one transport connection between any two peers
   over which Diameter messages may be passed. This state machine is
   intended to handle both the simple case, in which one peer initiates
   a connection to the other, and the complex case, in which each peer
   simultaneously initiates a connection to the other. In the complex
   case, an election occurs to determine which transport connection will
   survive.

   I- is used to represent the initiator (connecting) connection, while
   the R- is used to represent the responder (listening) connection. The
   lack of a prefix indicates that the event or action is the same
   regardless of the connection on which the event occurred.

   The stable states that a state machine may be in are Closed, and
   Open; all other states are intermediate.

   The DRR is always sent by the initiator immediately after a
   connection establishment.  After the election, the responder sends
   a DRA.  This DRA will contain either a Result-Code AVP of
   DIAMETER_SUCCESS, DIAMETER_SESSION_DENIED or
   DIAMETER_SESSION_EXISTS.  If the result is DIAMETER_SESSION_DENIED
   or DIAMETER_SESSION_EXISTS, the receiver of the DRA MUST close the
   connection.

   Since the identity of the peer is not known until either a DRR
   or DRA is received, it is possible for two connections to be briefly
   established.  This is why there is an election process for both
   the initiator and the responder.  This also means that each program
   may only have one election at a time.  This is the reason for the
   *-Wait-Lock state.

   The state machine constrains only the behavior of a Diameter
   implementation as seen by Diameter peers through events on the wire.
   Any implementation that produces equivalent results is considered
   compliant.

      state            event              action         next state
      -----------------------------------------------------------------
      Closed           Start            Snd-Conn-Req     I-Wait-Conn-Ack
                       Rcv-Conn-Req     Snd-Conn-Ack     R-Wait-DRR

      I-Wait-Conn-Ack  Rcv-Conn-Ack     Snd-DRR          I-Wait-DRA
                       Rcv-Conn-Nack    Cleanup          Closed
                       Timeout          Error            Closed

      I-Wait-DRA       Rcv-DRA          Elect-Lock       I-Wait-Lock
                       Rcv-DRA-Exist    I-Disc           Open
                       Rcv-DRA-Fail     I-Disc           Closed
                       Peer-Disc        Error            Closed
                       Timeout          Error            Closed

      I-Wait-Lock      Locked           Elect            I-Wait-Returns

      I-Wait-Returns   No-Contest       Elect-Unlock     Open
                       Local-Wins       Elect-Unlock/    Open
                                        R-Disc           
                       Peer-Wins        Elect-Unlock/    Open
                                        I-Disc           

      R-Wait-DRR       Rcv-DRR          Elect-Lock,      R-Wait-Lock
                       Timeout          Error            Closed
                       Rcv-Conn-Disc    Error            Closed

      R-Wait-Lock      Locked           Elect            R-Wait-Returns

      R-Wait-Returns   No-Contest       Elect-Unlock     Open
                       Peer-Wins        Snd-DRA/         Open
                                        Elect-Unlock/
                                        I-Disc           
                       Local-Wins       Elect-Unlock/    Open
                                        Snd-DRA-Exist
                       Not-Allowed      Elect-Unlock/    Closed
                                        Snd-DRA-Fail

      Open             Send-Message     Snd-Non-DRI      Open
                       Rcv-Non-DRI      Process          Open
                       WatchDog-Timer   Snd-DWR          Open
                       Rcv-DWA          Process-DWA      Open
                       Stop             Disc             Closed
                       Peer-Disc        Disc             Closed
                       Rcv-DRR          Error            Closed

8.1  States

   Following is a more detailed description of each automaton state.

      Closed         A peer is initially in the closed state, and no
                     transport connection exists with the peer.

      I-Wait-Conn-Ack
                     A transport connection has been initiated with the
                     peer, and an acknowledgment is pending.

      I-Wait-DRA     The local Diameter node is waiting for the peer to
                     issue a DRA.

      I-Wait-Lock    The initiator is waiting for an election lock.

      I-Wait-Returns The initiator is waiting on the results of the 
                     election returns.

      R-Wait-DRR     A transport connection indication has been received
                     from the peer, and a DRR has yet to be received by
                     the peer.

      R-Wait-Lock    The responder is waiting for an election lock.     

      R-Wait-Returns The responder is waiting on the results of the
                     election returns.

      Open           The peer connection is open.

8.2  Events

   Transitions and actions in the automaton are caused by events.

      Start          The Diameter application has signaled that a
                     connection should be initiated with the peer.

      Rcv-Conn-Req   A transport connection indication from the peer has
                     been received.

      Rcv-Conn-Ack   A positive acknowledgment was received to a
                     locally initiated transport connection.

      Rcv-Conn-Nack  A negative acknowledgment was received to a
                     locally initiated transport connection.

      Timeout        An application-defined timer has expired while
                     waiting for some event.

      Rcv-DRA        A DRA message from the peer was received.

      Rcv-DRA-Exist  A DRA failure of DIAMETER_SESSION_EXISTS was
                     received.

      Rcv-DRA-Fail   A DRA failure of DIAMETER_SESSION_DENIED was
                     received.

      Rcv-DRR        A DRR message from the peer was received.

      Peer-Disc      A disconnection indication from the peer was
                     received.

      Locked         The election locking mechanism succeeded.

      No-Contest     An election was held, and there was only one
                     connection to this peer.

      Local-Wins     An election was held, and the connection initiated 
                     locally wins.

      Peer-Wins      An election was held, and the connection initiated by
		     the peer wins.

      Not-Allowed    Before the election was held, it was determined that
                     the peer was not allowed.

      Send-Message   A Non-DRI message is to be sent.

      Rcv-Non-DRI    A Non-DRI message was received.

      WatchDog-Timer The Watchdog timer expired, indicating that a DWR
                     message is to be sent to the peer.

      Rcv-DWA        A DWA message was received.

      Stop           The Diameter application has signaled that a
                     connection should be terminated (e.g., on system
                     shutdown).

8.3  Actions

   Actions in the automaton are caused by events and typically indicate
   the transmission of packets and/or an action to be taken on the
   connection.

      Snd-Conn-Req   A transport connection is initiated with the peer.

      Snd-Conn-Ack   an acknowledgment is sent in response to a connect
                     request, confirming that the transport layer
                     connection is open.

      Snd-Conn-Nack  A negative acknowledgment is sent in response to a
                     connect request, indicating that the request was
                     refused.

      Snd-DRR        A DRR message is sent to the peer.

      Snd-DRA        A DRA message is sent to the peer.

      Cleanup        If necessary, the connection is shutdown, and any
                     local resources are freed.

      Error          The transport layer connection is disconnected,
                     either politely or abortively, in response to an
                     error condition. Local resources are freed.

      Process-DRI    A received DRI is processed.

      Disc           The transport layer connection is disconnected, and
                     local resources are freed.

      I-Disc         When two connections exist to the same peer, disconnect
                     the connection to the initiator.

      R-Disc         When two connections exist to the same peer, disconnect
                     the transport connection of the responder.

      Elect          An election occurs (see Section 8.4 for more
                     information).

      Elect-Lock     Block any other elections from happening.  If the
                     election can happen in multiple threads, some form
                     of locking is needed.  If it can only happen in one
                     thread, no action is needed.

      Elect-Unlock   If election can happen in multiple threads, the
                     remove the lock gotten from Elect-Lock.

      Error          Report an error and do cleanup.

      Snd-DRA        Send a DRA with a Result-Code AVP of DIAMETER_SUCCESS.

      Snd-DRA-Exist  Send a DRA with a Result-Code AVP of 
                     DIAMETER_SESSION_EXISTS.

      Snd-DRA-Fail   Send a DRA with a Result-Code AVP of
                     DIAMETER_SESSION_DENIED.

      Snd-Non-DRI    A non-DRI message is sent.

      Snd-DWR        A DWR message is sent.

      Process-DWA    The DWA message is serviced.

      Process        A non-DRI Diameter message is serviced.

8.4  The Election Process

   The election is performed after receipt of either a DRR or a DRA.
   If this is the receipt of a DRR and the peer is not allowed to
   connect, a result of Not-Allowed is returned.  If the Origin-FQDN
   received in the DRR/DRA is not a peer that already has an open
   connection, No-Contest is returned.  Otherwise, there is a
   comparison of the the Origin-FQDN sent by its peer with its own
   Origin-FQDN (which it may or may not have actually sent). The
   transport layer connection with the higher value of Origin-FQDN is
   the one that survives. The comparison proceeds by considering the
   shorter OctetString to be null-padded to the length of the longer,
   then performing an octet by octet unsigned comparison with the
   first octet being most significant.  If the peer is the victor, 
   Peer-Wins is returned otherwise Local-Wins is returned.

8.5 Most Common State Transitions

   Below is a drawing of the most common state transitions where
   either a connection initiated (left) or requested (right) and
   the connection is made with no problems.  States are boxed.
   Events are plain text.  Actions are in parenthesis.  All states
   are shown in this drawing but several events not show.

                        +------+         
                        |Closed|         
                        +------+         
                          |  |                           
              +-----------+  +-----------+
              |                          |
            Start                   Rcv-Conn-Req
              |                          |
        (Snd-Conn-Req)             (Snd-Conn-Ack)
              |                          |
      +---------------+                  |
      |I-Wait-Conn-Ack|                  |
      +---------------+                  |
              |                          |
         Rec-Conn-Ack                    |
              |                          |
          (Snd-DRR)                      |
              |                          |
        +----------+               +----------+
        |I-Wait-DRA|               |R-Wait-DRR|
        +----------+               +----------+
              |                          |
           Rcv-DRA                    Rcv-DRR
              |                          |
         (Elect-Lock)               (Elect-Lock)
              |                          |
        +-----------+              +-----------+
        |I-Wait-Lock|              |R-Wait-Lock|
        +-----------+              +-----------+ 
              |                          |      
            Locked                     Locked
              |                          |      
           (Elect)                    (Elect)
              |                          |
      +--------------+            +--------------+
      |I-Wait-Returns|            |R-Wait-Returns|
      +--------------+            +--------------+
              |                          |
          No-Contest                 No-Contest
              |                          |
        (Elect-Unlock)         (Snd-DRA/Elect-Unlock)
              |                          |
              +-----------+  +-----------+
                          |  |
                         +----+
                         |OPEN|
                         +----+

---

From: Jacques Caron <[email-masked]>
Date: Mon, 11 Jun 2001 14:27:27 +0200

Hi,

Coming a bit late into that discussion, but I just had a quick comment...

If we need to make sure we have only one TCP connection between two hosts, 
why not use the same port as source as the one we're listening on? Since 
TCP connections must be unique based on (souce addr, source port, dest 
addr, dest port), and only on that, it should make the problem trivial?

Still don't know whether we should have one single connection (better for 
congestion avoidance) or multiple ones (to avoid head of line blocking). 
Maybe this should be left as a configuration switch (as depending on the 
particular circumstances, one or the other might be better), the 
consequence of which could simply be to use the assigned port as source 
port or keep the random one.

Just my EUR 0.02.

Jacques.

>spec, so what do folks think we should do? Single or multiple connections?

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

---

From: Pat Calhoun <[email-masked]>
Date: Mon, 11 Jun 2001 06:32:31 -0700

> addr, dest port), and only on that, it should make the problem trivial?

Because that would not allow multiple processes on a box, since the second
Diameter process wouldn't be able to bind to the necessary port.

PatC

---

From: Mark Eklund <[email-masked]>
Date: Mon, 11 Jun 2001 09:44:28 -0400 (EDT)

Jacques,

> addr, dest port), and only on that, it should make the problem trivial?

The problem is that you can't accept and connect using the
same source port.  

We also want to be able to support two Diameter servers on the
same host.

> port or keep the random one.

I hear a lot of talk of head of line blocking.  Exactly what
does that mean?  I can think of two instances that don't require
multiple connections to fix it.  Is there something that would?

1. One packet takes 10 times as long as another packet to
   broadcast, thus backing up all the other packets?
   - Either use SCTP, or speed up the physical layer.

2. The server takes a long time to process incoming packets.
   - If this is as fast as the server goes, multiple connections
     won't help.
   - If the server takes a long time to process one type of
     packet, add extra threads to process the other packets.

-mark

---

From: Pat Calhoun <[email-masked]>
Date: Mon, 11 Jun 2001 07:03:47 -0700

>      packet, add extra threads to process the other packets.

I think the basic idea was to simulate SCTP streams by opening up
multiple TCP connections. In any case, we've already agreed that
this will not be done in Diameter.

PatC

---

From: Jacques Caron <[email-masked]>
Date: Mon, 11 Jun 2001 16:37:28 +0200

>Diameter process wouldn't be able to bind to the necessary port.

Well, to have multiple processes on a box, you surely need to use different 
ports (and/or IP addresses), and then you can have as many processes as you 
want?

Jacques.

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 11 Jun 2001 08:08:34 -0700 (PDT)

> >      packet, add extra threads to process the other packets.

It should be kept in mind that head of line blocking will
only occur when the time between requests is less than RTO, and
where timeouts are present. Thus, the problem will only manifest  on large
NASes where mechanisms such as fast re-transmit, SACK, or
limited-transmit cannot avoid a timeout.  

For a NAS communicating with a server through an agent, a slow server
will not create a head of line blocking problem. This is because the
agent will ACK at the transport layer, and will also respond to
Watchdog messages. The server application response will not be returned
in a timely way, but this will not result in failover or inability of
the NAS to send additional requests.

---

From: Patrice Calhoun <[email-masked]>
Date: Mon, 11 Jun 2001 09:27:03 -0700 (PDT)

> ports (and/or IP addresses), and then you can have as many processes as you 

Ah, then you are suggesting that both sides know that multiple processes are
used, and be configured for each process by a common port number. This seems
like alot of configuration to me :(

PatC

---

From: Jacques Caron <[email-masked]>
Date: Mon, 11 Jun 2001 18:50:44 +0200

>like alot of configuration to me :(

Errrr? Whatever happens, if you have multiple processes on the same box, 
they will have different port numbers, and their peers need to know that 
(that's part of the Host descricptor thing)...

The only thing I say is: "if you want to make sure only one connection is 
established, use the port number you're listening on as your source port 
also". Indeed, that would need to be done on the processes on both sides of 
the connection that is wished to be unique, but that is totally independent 
from the fact that one or multiple processes are running on any given box.

I just ran a very small test, which shows this:

1. I have a process listening for requests on port 1812 on the local box:
unix14: {121} netstat -anf inet | grep 1812
tcp        0      0  *.1812                 *.*                    LISTEN

2. I also have a process listening for requests on port 1812 on some remote 
box.

3. I then connect from the local box (using source port 1812) to the remote 
box (on port 1812 of course):

unix14: {123} netstat -anf inet | grep 1812
tcp        0      0  194.2.222.14.1812      194.150.27.98.1812     ESTABLISHED
tcp        0      0  *.1812                 *.*                    LISTEN

4. If I try to connect from the remote box (port 1812) to the local box 
(port 1812), it fails (Address already in use).

Of course, none of this is relevant if we actually want multiple 
connections, but I have to admit I'm completely lost on the final result of 
that part of the discussion...

Jacques.

---

From: Pat Calhoun <[email-masked]>
Date: Mon, 11 Jun 2001 10:38:10 -0700

> (that's part of the Host descricptor thing)...

Sorry, it appears as if I lost your train of thought. Let me try again.

> (port 1812), it fails (Address already in use).

But what if the remote box was periodically trying to connect to the local
box? That would always fail with 'Address already in use', since you'd already
bound a file descriptor on port 1812 for listening.

So we need to listen for connections, and initiate connections as well. We need
to allow for multiple connections to multiple peers, all listening on the
same port.

PatC

---

From: "David Mitton" <[email-masked]>
Date: Mon, 11 Jun 2001 16:14:13 -0400

> > (that's part of the Host descricptor thing)...

yes, if you want this difficult situation, you will have to figure out how 
to configure it.

><snip...>

No can do.

Only one process can be listening on a given port number at a time.   When 
the listening process accepts the connection (at least on Unix) a new 
socket is created, and typically it is given a new unique ephemeral port 
number. The original port number is not needed, as the connection has been 
made.

The listening process can spawn subprocesses or threads to handle that 
connection socket, but it still holds the listening well-known socket.

If you need to run different server "instances" on the same system, they 
have to listen to connections with distinct socket numbers, (that's how IP 
knows where to deliver your packets) or you partion your implementation 
internally.

So far, I thought the URI would include the ability to configure a 
non-default target (listening) port number which would be critical to doing 
this.

Dave.

         Dave.

------------------------------------------------------------
David Mitton                            ESN: 248-4570
Advisor, Nortel Networks                 978-288-4570
Wireless Solutions, IP Mobility
Billerica, MA 01821               [email-masked]

---

From: Jacques Caron <[email-masked]>
Date: Mon, 11 Jun 2001 22:15:13 +0200

>bound a file descriptor on port 1812 for listening.

Nope, it won't fail. It will fail only if there is already a connection 
between the two machines.

>to allow for multiple connections to multiple peers, all listening on the

Not a problem. Remember that the uniqueness requirement is on the tuple 
(local addr, local port, remote addr, remote port), not on (local addr, 
local port).

Jacques.

---

From: Pat Calhoun <[email-masked]>
Date: Mon, 11 Jun 2001 13:13:45 -0700

> >to allow for multiple connections to multiple peers, all listening on the

Of course, that's been my point all along.

> number. The original port number is not needed, as the connection has been 

Right, and this is where I got confused with Jacques e-mail that seemed to
state that this is what he wanted to do.

> non-default target (listening) port number which would be critical to doing 

It does, and these seem to be different issues. The primary issue here is how
to get a state machine that works to solve the problem where multiple 
transport connections could exist.

PatC

---

From: Jacques Caron <[email-masked]>
Date: Mon, 11 Jun 2001 22:47:00 +0200

I think there is some confusion here... There are two different issues at hand:
1. enable multiple processes to run on the same machine
2. make sure there is only one connection between any two processes (or 
not, but that's another subjet)

(1) requires that each process be assigned a distinct port or IP address. 
You cannot have two processes listening on port 1812. You could have one on 
1812 and one on 1813, for instance (don't know which other protocol I just 
bumped into...).

(2) requires either:
- the use of SCTP which apparently solves the problem magically (don't know 
how and why, haven't read the whole spec yet, but it was stated in some 
previous e-mail).
- the use of TCP with a complex state machine to make sure we tear down one 
of the two connections if two are established at the same time
- the use of TCP with the same port number of listening and accepting, as I 
described.

It is not a problem to have a process that:
- listens and accepts connections on port 1812
- connects FROM port 1812 to other machines, possibly on port 1812
[replace 1812 with another port number for any other process running on the 
same box]

On the other hand, a connection from port 1812 to another machine on port 
1812 will fail if there is already a connection between these two machines 
using the same port(s).

So, I can have:
- process A1 on box A listening on port 1812
- process A2 on box A listening on port 1813
- process B1 on box B listening on port 1812

Let's say they are brought up in that order (A1, A2, B1), and we want a 
direct connection between each pair. The following happens:

- A1 starts listening on A:1812
- A1 attempts to connect to A2 (A:1813) and B1 (B:1812) but fails (no one's 
listening)
- A2 is started.
- A2 starts listening on A:1813
- A2 attempts a connection to A:1812 (which succeeds)
- A2 attempts a connection to B:1812 (which fails, no one is listening)

[if A1 was to start a connection to A2 now, it would fail]

- B1 is started
- B1 starts listening on B:1812
- B1 attempts a connection to A:1812, which succeeds
- B1 attempts a connection to A:1813, which succeeds also

[if A1 or A2 were to start a connection B1 now, that would fail]

So, we end up with a all three servers listening on their respective ports, 
and a connection established between A1 and A2, A1 and B1, A2 and B1, 
respectively. Exactly what we wanted. And no messy state machine.

Of course, some bad TCP/sockets implementations might not support all this 
very well... To be checked [it worked perfectly on my good old FreeBSD boxes].

Hope it gets clearer now?

Jacques.

At 22:13 11/06/01, Pat Calhoun wrote:
> > >So we need to listen for connections, and initiate connections as 
> well. We
> > >need

---

From: Mark Eklund <[email-masked]>
Date: Mon, 11 Jun 2001 18:36:15 -0400 (EDT)

Jacques,

> [replace 1812 with another port number for any other process running on the 

From your example, it seemed that you could get the above to
work.  I have not had such luck.  Maybe I need to go back to TCP
101 :).  If I do the following pseudocode,

s = socket(AF_INET, SOCK_STREAM, IP_PROTOCOL_TCP);
s_addr = (AF_INET, 1812, 1.1.1.1);
bind(s, (AF_INET, s_addr, sizeof(s_addr));
d_addr = (AF_INET, 1812, *);
listen(s, 10);
accept(s, d_addr); 

I now can't get another thread bind to (AF_INET, 1812, 1.1.1.1)
so that I can request a connection to (AF_INET, 1812, 2.2.2.2).

Is this possible?

-mark

---

From: Jacques Caron <[email-masked]>
Date: Tue, 12 Jun 2001 00:48:01 +0200

Mark,

You need to setsockopt SO_REUSEADDR and SO_REUSEPORT to on(1). Not sure you 
actually need both, but it works that way :-) Note that it needs to be done 
on both sockets (the one listening/accepting and the one connecting).

I can send you some (very very very ugly) code I used for my tests if you 
want to try it out.

Of course, all of this is true on a good BSD stack. Don't know how it might 
turn out on other stacks that might make too strong asumptions about TCP 
requirements for uniqueness.

Jacques.