Subject: [AAA-WG]: Support for multi-process

From: Pat Calhoun <[email-masked]>
Date: Fri, 1 Jun 2001 14:01:28 -0700

Mark,

I am going through the last items that I haven't had a chance to address,
and this one in particular appears to be problematic. The suggested fix is
to remove an entry in the peer state machine, which would affect legitimate
cases where two connections are established by the same process. Should there
instead be some text that states that the peer's identity be examined, and not just
look at the IP Address?

What exactly was the suggested fix? The issue definition isn't really clear :(

Thanks,

PatC

---

From: Mark Eklund <[email-masked]>
Date: Fri, 1 Jun 2001 18:33:14 -0400 (EDT)

Pat,

Actually, at one time I thought I had a suggested fix, but now
I'm not sure.  Every time I think of changing the state machine,
I get a headache :(.  Here is the gist of the problem.

If we allow multiple peers at the same IP address, we have to
wait until receiving a DRI (or whatever it is now called) to
find out which peer it is.

Even then, how do we know which peer to which we are connecting?  
The diameter identity is [protocol] fqdn [ port ]. If I am a
server listening on port 1812, and I want to establish a
connection to another server on 1812, I can set the destination
port to 1812, but I cannot use 1812 on the source port.  Why?
Because I'm already listening to that port and can't bind to 
it for a connect.

So imagine one system a.b.c with a server (listening on port
1812) and a client (listening on port 2000) and system x.y.z
with a server (listening on port 1812).  Everything comes up at
the same time and attempts to make a connection.

Application  SAddr  SPort DAddr DPort Diameter Identity
a.b.c client a.b.c  3847  x.y.z 1812  a.b.c:3847
a.b.c server a.b.c  8852  x.y.z 1812  a.b.c:8852
x.y.z server x.y.z  9253  a.b.c 1812  x.y.z:9253
x.y.z.server x.y.z  9986  a.b.c 2000  x.y.z:9986

* The source port (SPort) is randomly chosen when binding to '*'.
* The identifier is the identifier sent in the DRI.
* when x.y.z receives "a.b.c:3847" and "a.b.c:8852", it doesn't
  know which application to associate it with.

My conclusions:
1. We cannot drop a connection until the DRI is received.

2. The state machine must be modified to reflect this. (commence headaches)

3. The Diameter Identifier port number must be the port on which
   the peer listens, not the source port that from which
   connecting.  For the example above, the a.b.c client would
   have a Diameter Identifier of "a.b.c:2000" even if the source
   port were 3847.

Sorry, no solutions at this point.

-mark

---

From: Mark Eklund <[email-masked]>
Date: Sat, 2 Jun 2001 15:53:17 -0400 (EDT)

Pat,

Here is my proposed solution for multi-process access.  

* The state table is massively modified.  

* I added a drawing of the state changes for an initiator and
  requester when everything goes correctly as Section 8.5.

* The Diameter-Identifier still has to use the port number which
  the Diameter server Listens which may not be the source port.

* I couldn't remember what DRI changed to so I called it DRR and
  DRA in this document.

* I couldn't remember if Origin-FQDN changed.  So it still has
  the same name.

8.0  Peer State Machine

   This section contains a finite state machine, that MUST be observed
   by all Diameter implementations. Each Diameter node MUST follow the
   state machine described below when communicating with each peer.
   Multiple actions are separated by commas, and may continue on
   succeeding lines as space requires. Similarly, state and next state
   may also span multiple lines as space requires.

   There may be at most one transport connection between any two peers
   over which Diameter messages may be passed. This state machine is
   intended to handle both the simple case, in which one peer initiates
   a connection to the other, and the complex case, in which each peer
   simultaneously initiates a connection to the other. In the complex
   case, an election occurs to determine which transport connection will
   survive.

   I- is used to represent the initiator (connecting) connection, while
   the R- is used to represent the responder (listening) connection. The
   lack of a prefix indicates that the event or action is the same
   regardless of the connection on which the event occurred.

   The stable states that a state machine may be in are Closed, and
   Open; all other states are intermediate.

   The DRR is always sent by the initiator immediately after a
   connection establishment.  After the election, the responder sends
   a DRA.  This DRA will contain either a Result-Code AVP of
   DIAMETER_SUCCESS, DIAMETER_SESSION_DENIED or
   DIAMETER_SESSION_EXISTS.  If the result is DIAMETER_SESSION_DENIED
   or DIAMETER_SESSION_EXISTS, the receiver of the DRA MUST close the
   connection.

   Since the identity of the peer is not known until either a DRR
   or DRA is received, it is possible for two connections to be briefly
   established.  This is why there is an election process for both
   the initiator and the responder.  This also means that each program
   may only have one election at a time.  This is the reason for the
   *-Wait-Lock state.

   The state machine constrains only the behavior of a Diameter
   implementation as seen by Diameter peers through events on the wire.
   Any implementation that produces equivalent results is considered
   compliant.

      state            event              action         next state
      -----------------------------------------------------------------
      Closed           Start            Snd-Conn-Req     I-Wait-Conn-Ack
                       Rcv-Conn-Req     Snd-Conn-Ack     R-Wait-DRR

      I-Wait-Conn-Ack  Rcv-Conn-Ack     Snd-DRR          I-Wait-DRA
                       Rcv-Conn-Nack    Cleanup          Closed
                       Timeout          Error            Closed

      I-Wait-DRA       Rcv-DRA          Elect-Lock       I-Wait-Lock
		       Rcv-DRA-Exist    I-Disc           Open
		       Rcv-DRA-Fail     I-Disc           Closed
                       Peer-Disc        Error            Closed
                       Timeout          Error            Closed

      I-Wait-Lock      Locked           Elect            I-Wait-Returns

      I-Wait-Returns   No-Contest       Elect-Unlock     Open
                       Win-Election     Elect-Unlock/    Open
					R-Disc           
		       Lose-Election    Elect-Unlock/    Open
                                        I-Disc		 

      R-Wait-DRR       Rcv-DRR          Elect-Lock,      R-Wait-Lock
                       Timeout          Error            Closed
                       Rcv-Conn-Disc    Error            Closed

      R-Wait-Lock      Locked           Elect            R-Wait-Returns

      R-Wait-Returns   No-Contest       Elect-Unlock     Open
                       Win-Election     Snd-DRA/         Open
                                        Elect-Unlock/
					I-Disc           
		       Lose-Election    Elect-Unlock/    Open
					Snd-DRA-Exist
                       Not-Allowed      Elect-Unlock/    Closed
					Snd-DRA-Fail

      Open             Send-Message     Snd-Non-DRI      Open
                       Rcv-Non-DRI      Process          Open
                       WatchDog-Timer   Snd-DWR          Open
                       Rcv-DWA          Process-DWA      Open
                       Stop             Disc             Closed
                       Peer-Disc        Disc             Closed
                       Rcv-DRR          Error            Closed

8.1  States

   Following is a more detailed description of each automaton state.

      Closed         A peer is initially in the closed state, and no
                     transport connection exists with the peer.

      I-Wait-Conn-Ack
                     A transport connection has been initiated with the
                     peer, and an acknowledgment is pending.

      I-Wait-DRA     The local Diameter node is waiting for the peer to
                     issue a DRA.

      I-Wait-Lock    The initiator is waiting for an election lock.

      I-Wait-Returns The initiator is waiting on the results of the 
                     election returns.

      R-Wait-DRR     A transport connection indication has been received
                     from the peer, and a DRR has yet to be received by
                     the peer.

      R-Wait-Lock    The responder is waiting for an election lock.     

      R-Wait-Returns The responder is waiting on the results of the
                     election returns.

      Open           The peer connection is open.

8.2  Events

   Transitions and actions in the automaton are caused by events.

      Start          The Diameter application has signaled that a
                     connection should be initiated with the peer.

      Rcv-Conn-Req   A transport connection indication from the peer has
                     been received.

      Rcv-Conn-Ack   A positive acknowledgment was received to a
                     locally initiated transport connection.

      Rcv-Conn-Nack  A negative acknowledgment was received to a
                     locally initiated transport connection.

      Timeout        An application-defined timer has expired while
                     waiting for some event.

      Rcv-DRA        A DRA message from the peer was received.

      Rcv-DRA-Exist  A DRA failure of DIAMETER_SESSION_EXISTS was
                     received.

      Rcv-DRA-Fail   A DRA failure of DIAMETER_SESSION_DENIED was
                     received.

      Rcv-DRR        A DRR message from the peer was received.

      Peer-Disc      A disconnection indication from the peer was
                     received.

      Locked         The election locking mechanism succeeded.

      No-Contest     An election was held, and there was only one
                     connection to this peer.

      Win-Election   An election was held, and the new connection to
                     the peer was the winner.

      Lose-Election  An election was held, and the existing connection to
                     the peer was the winner.

      Not-Allowed    Before the election was held, it was determined that
                     the peer was not allowed.

      Send-Message   A Non-DRI message is to be sent.

      Rcv-Non-DRI    A Non-DRI message was received.

      WatchDog-Timer The Watchdog timer expired, indicating that a DWR
                     message is to be sent to the peer.

      Rcv-DWA        A DWA message was received.

      Stop           The Diameter application has signaled that a
                     connection should be terminated (e.g., on system
                     shutdown).

8.3  Actions

   Actions in the automaton are caused by events and typically indicate
   the transmission of packets and/or an action to be taken on the
   connection.

      Snd-Conn-Req   A transport connection is initiated with the peer.

      Snd-Conn-Ack   an acknowledgment is sent in response to a connect
                     request, confirming that the transport layer
                     connection is open.

      Snd-Conn-Nack  A negative acknowledgment is sent in response to a
                     connect request, indicating that the request was
                     refused.

      Snd-DRR        A DRR message is sent to the peer.

      Snd-DRA        A DRA message is sent to the peer.

      Cleanup        If necessary, the connection is shutdown, and any
                     local resources are freed.

      Error          The transport layer connection is disconnected,
                     either politely or abortively, in response to an
                     error condition. Local resources are freed.

      Process-DRI    A received DRI is processed.

      Disc           The transport layer connection is disconnected, and
                     local resources are freed.

      I-Disc         When two connections exist to the same peer, disconnect
                     the connection to the initiator.

      R-Disc         When two connections exist to the same peer, disconnect
                     the transport connection of the responder.

      Elect          An election occurs (see Section 8.4 for more
                     information).

      Elect-Lock     Block any other elections from happening.  If the
                     election can happen in multiple threads, some form
                     of locking is needed.  If it can only happen in one
                     thread, no action is needed.

      Elect-Unlock   If election can happen in multiple threads, the
                     remove the lock gotten from Elect-Lock.

      Error          Report an error and do cleanup.

      Snd-DRA        Send a DRA with a Result-Code AVP of DIAMETER_SUCCESS.

      Snd-DRA-Exist  Send a DRA with a Result-Code AVP of 
                     DIAMETER_SESSION_EXISTS.

      Snd-DRA-Fail   Send a DRA with a Result-Code AVP of
                     DIAMETER_SESSION_DENIED.

      Snd-Non-DRI    A non-DRI message is sent.

      Snd-DWR        A DWR message is sent.

      Process-DWA    The DWA message is serviced.

      Process        A non-DRI Diameter message is serviced.

8.4  The Election Process

   The election is performed after receipt of either a DRR or a DRA.
   If this is the receipt of a DRR and the peer is not allowed to
   connect, a result of Not-Allowed is returned.  If the Origin-FQDN
   received in the DRR/DRA is not a peer that already has an open
   connection, No-Contest is returned.  Otherwise, there is a
   comparison of the the Origin-FQDN sent by its peer with its own
   Origin-FQDN (which it may or may not have actually sent). The
   transport layer connection with the higher value of Origin-FQDN is
   the one that survives. The comparison proceeds by considering the
   shorter OctetString to be null-padded to the length of the longer,
   then performing an octet by octet unsigned comparison with the
   first octet being most significant.  If the peer is the victor, 
   Lose-Election is returned otherwise Win-Election is returned.

8.5 Most Common State Transitions

   Below is a drawing of the most common state transitions where
   either a connection initiated (left) or requested (right) and
   the connection is made with no problems.  States are boxed.
   Events are plain text.  Actions are in parenthesis.  All states
   are shown in this drawing but several events not show.

       	       	        +------+	 
       	       	        |Closed|     	 
	   	        +------+	 
	   		  |  |		 		 
       	      +-----------+  +-----------+
              |                          |
            Start                   Rcv-Conn-Req
              |                          |
        (Snd-Conn-Req)             (Snd-Conn-Ack)
              |                          |
      +---------------+                  |
      |I-Wait-Conn-Ack|                  |
      +---------------+                  |
              |                          |
         Rec-Conn-Ack                    |
              |                          |
          (Snd-DRR)                      |
              |                          |
        +----------+               +----------+
        |I-Wait-DRA|               |R-Wait-DRR|
        +----------+               +----------+
              |                          |
           Rcv-DRA                    Rcv-DRR
              |                          |
         (Elect-Lock)               (Elect-Lock)
              |                          |
        +-----------+		   +-----------+
        |I-Wait-Lock|  	       	   |R-Wait-Lock|
        +-----------+		   +-----------+ 
       	      |                          |	
       	    Locked     	       	       Locked
       	      |                          |   	
           (Elect)                    (Elect)
              |                          |
      +--------------+            +--------------+
      |I-Wait-Returns|            |R-Wait-Returns|
      +--------------+            +--------------+
              |                          |
          No-Contest                 No-Contest
              |                          |
        (Elect-Unlock)         (Snd-DRA/Elect-Unlock)
              |                          |
              +-----------+  +-----------+
                          |  |
                         +----+
                         |OPEN|
                         +----+

-mark