Subject: Re: [Dime] CER/CEA on an open connection

From: Victor Fajardo <[email-masked]>
Date: Tue, 04 Apr 2006 17:19:06 -0400

After some offline discussion regarding CER/CEA exchange in the open 
connection state, a few of us have come up with the following 
application-id renegotiation proposal. It attempts to replicate existing 
CER/CEA exchanges as much as possible in order to minimize changes. The 
text is in its own section:

"5.3.8 Application Support Re-Negotiation

A Diameter node wishing to change application support MUST send a
Capabilities-Exchange-Req (CER) to all its peers for which it shares
an established connection. A change in application support constitutes
addition or termination of one or more application in a Diameter node.
In such a case, the Diameter node is expected to advertise the most
recent set of supported applications in a CER message while it is in
the open state as specified by the peer statemachine (see Sec 5.6).
The receiver of the CER MUST send a CEA with Result-Code AVP set to
DIAMETER_SUCCESS if it is able comply with the request. Note that peers
which successfully renegotiated the set of supported applications SHOULD
also update thier routing tables to reflect the change.

If the receiver of the CER is unable to comply with the request, it SHOULD
send a CEA with the Result-Code AVP set to DIAMETER_UNABLE_TO_COMPLY and
retain its pre-existing state with its peer. The receiver of the CEA can
decide wether to terminate or maintain the connection at the receipt of
this result code. If it decides to maintain the connection then it MUST
retain its pre-existing connection state with its peer prior sending the
CER.

A Diameter node SHOULD also determine wether changes in application
support results in having no common applications with one or more peers
prior to sending a CER. In such a case, it should send a 
Disconnect-Peer-Request
(DPR) to terminate the connection instead. The Disconnect-Cause AVP of the
DPR message MUST be set to DO_NOT_WANT_TO_TALK_TO_YOU.

A Diameter node that receives a CER from a peer from which it has sent its
own pending CER MUST be able serialize the simultaneous renegotiation. If
the Diameter node is acting as a responder (see Sec 5.6) then it MUST give
precedence to the renegotiation initiated by its peer and send a CEA 
accordingly.
If the Diameter node is acting as an initiator (see Sec 5.6) then it 
MUST delay
the renegotiation initiated by its peer (the responder) and send a CEA with
Result-Code AVP set to DIAMETER_UNABLE_TO_COMPLY in order to retain the
the current state between them. The receiver of this CEA (the responder)
SHOULD retain its current state and proceed with the ongoing renegotiation
initiated by its peer. It SHOULD retransmit its own CER to initiate another
renegotiation after the current renegotiation completes. Both 
renegotiations
SHOULD be treated as separate and sequential.

Diameter nodes that implement application renegotiation SHOULD check the
version information advertised by its peer in the diameter header of initial
CER/CEA exchange to determine if the peer supports renegotiation. The 
diameter
node MUST not attempt renegotiation if it is determined that peer has no 
support
for such scheme.

Capabilities exchange performed while in the open state SHOULD be limited
to modification of application support. Information added to the CER/CEA
messages by diameter extensions MAY be renegotiable though the extensions
themselves have to specify such schemes."

Hope this is helpful. Regards,
victor

> A simple alternative to renegotiation would be for nodes to terminate 
> and reestablish connections. Not desirable but given that this is 
> probably a rarely needed feature perhaps it could be acceptable.
>
> OTOH, if renegotiation is limited to changing supported applications 
> as proposed by Victor and Timothy, then that seems manageable.
>
> More inline...
>
> Victor Fajardo wrote:
>>>   
>>
>>
>> Comments inline:
>>
>>> Hmm, it seems to me that if we have to fix the state machine, that 
>>> might be a significant change as it would affect people 
>>> implementations.   Would specifying the renegotiation be something 
>>> that people are interested in?
>>
>>
>> It seems renegotiation has a valid point. To minimize the change in 
>> current implementations maybe we can add text that describes 
>> optional(??) functionality:
>>
>> 1. limiting scope of renegotiation to app id support
>> 2. sender of the cer wishes to add or remove app support with peer. 
>> if removal of an app results in no common app, the connection should 
>> be closed. This event need not be represented in the fsm since 2nd 
>> paragraph in sec 5.3 already describes the behavior.
>
> I think that in this case the guy removing support for the one shared 
> app should not send the CER but instead just terminate the connection. 
> (Possibly sending new Disconnect-Cause, say NO_SHARED_APP.)
>
>> 3. current stateful session affected by a removable will remain 
>> active until the session or the application itself ends. also applies 
>> to proxies. does not apply to stateless sessions, proxies or relays.
>
> We might want to specify a new non-2xxx result-code that the recipient 
> of an invalid "subsequent" CER can respond with. Invalid here meaning 
> that the CER differs from the CER/CEA originally sent by the peer in 
> aspects other than supported applications, e.g. different security or 
> different Origin-Host/Origin-Realm values. Of course the CER can be 
> rejected for other reasons, e.g. missing or illegal AVPs etc.
>
> We'll have to determine what happens with the connection in the case 
> where the CER is rejected. One option is to say that it stays open in 
> the state it was in prior to the failed CER. Both parties are then 
> free to terminate it themselves if they feel like it.
>
> Anders
>
>>
>> hope this is helpful. regards,
>> victor
>>
>>
>>>  
>>> John
>>>
>>>
>>>     Hi Anders,
>>>
>>>     I think there is a valid need for re-advertising capabilities.
>>>      Applications may come and go over time.  So this gives a means
>>>     for an application to join an existing connection without having
>>>     to bring it down and disrupt other applications that might already
>>>     be using that connection.  But I think the questions you raised
>>>     are valid with regard to the definition of this renegotiation.
>>>
>>>     It is subtle, but renegotiation is really defining an additional
>>>     state in the state machine.  If a CER is sent on an already
>>>     negotiated connection, you have entered a new
>>>     "Open-but-waiting_CEA" state.  The questions that you raise should
>>>     be addressed in that new state.
>>>
>>>     I think it is a good function, but agree that it needs to be
>>>     better defined.
>>>
>>>     Best Regards,
>>>     Timothy Smith
>>>
>>>     [email-masked]
>>>     (919) 254-4723
>>>
>>>
>>>
>>>     *Anders Kristensen <[email-masked]>*
>>>
>>>     03/13/2006 09:49 AM
>>>
>>>            To
>>>         [email-masked]
>>>     cc
>>>            Subject
>>>         [Dime] CER/CEA on an open connection
>>>
>>>
>>>
>>>       
>>>
>>>
>>>
>>>
>>>     In its description of the peer state machine RFC 3588 goes out 
>>> of its
>>>     way to allow CER requests on an already open connection (section
>>>     5.6).
>>>     Is the intention that either party can re-advertise its set of
>>>     capabilities at any time? This raises some questions. For 
>>> example, if
>>>     TLS was not listed as supported in the first CER/CEA but is now,
>>>     is it
>>>     possible to carry out a TLS handshake after the exchange of
>>>     non-CER/CEA
>>>     traffic? What if no response is received for a "non-initial" CER 
>>> (but
>>>     other traffic, including DWR/DEA, continues to be exchanged),
>>>     should the
>>>     connection be reset?
>>>
>>>     I wonder if the ability to exchange capabilities on an open
>>>     connection
>>>     is widely supported and whether it is really useful enough to
>>>     warrant it
>>>     being part of the spec. Should probably be part of the base 
>>> protocol
>>>     test suite if it is.
>>>
>>>     Anders
>>>
>>>     _______________________________________________
>>>     DiME mailing list
>>>     [email-masked]
>>>     https://www1.ietf.org/mailman/listinfo/dime
>>>
>>> ------------------------------------------------------------------------ 
>>>
>>>
>>> _______________________________________________
>>> DiME mailing list
>>> [email-masked]
>>> https://www1.ietf.org/mailman/listinfo/dime
>>>  
>>>
>>
>
>

_______________________________________________
DiME mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/dime

---

From: "Tolga Asveren" <[email-masked]>
Date: Tue, 4 Apr 2006 18:44:45 -0400

Hi Victor,

Overall, the idea of declaring unavailability of an application is really a
useful functionality. OTOH I believe being able to distinguish between
immediate and gracefull shutdowns is also important. As immediate shutdown,
I refer to cases like an application crash etc... Gracefull shutdown is
putting an application down in a controlled way for reasons like
maintanence. Considering its controlled behavior, it is important to
minimize message/state loss in that type of scenarios.

I believe, accepting messages for existing sessions till they finish but not
accepting messages for new sessions is a reasonable semantic for graceful
shutdown case. This type of operation probably will require knowledge about
session life cycle hence better be handled end-to-end rather than
hop-by-hop. To support gracefull failover, I suggest to introduce a new
error code -maybe a transient failure code?-. Something like:
DIAMETER_APPLICATION_SHUTTING_DOWN. This will be returned back when a
message initiating a new session is received, while the application is
gracefully shutting down. When requester receives an answer with this error
code, it shouldn't send any messages for new sessions, but it can continue
to send messages for existing sessions. When all existing sessions finish,
the application can be shut down and a CER declaring the unavailability of
the application can be sent to immediate neighbors. If this sounds
reasonable to people, I can provide text.

IMO, for immediate shutdown, the mechanism described below is a good way of
handling things. If there are intermediaries between client and server, the
immediate neigbor of the node, which declared unavailability of the
application, will reply with DIAMETER_UNABLE_TO_DELIVER for incoming
requests, maybe a clarification about this may be added.

Please see below for a few more questions as well.

   Thanks,
   Tolga

> -----Original Message-----
> From: Victor Fajardo [mailto:[email-masked]]
> Sent: Tuesday, April 04, 2006 5:19 PM
> To: [email-masked]
> Subject: Re: [Dime] CER/CEA on an open connection
>
>
[TOLGA]What do you mean with "pre-existing connection state"? Does this
imply the set of supported applications or only peer state as defined in
5.6?
Furthermore, when would a node reply with DIAMETER_UNABLE_TO_COMPLY? What
type of scenario do we foresee for that, only the one described below, i.e.
crossing CERs, or is this supposed to be a more generic rejection mechanism?

> node MUST not attempt renegotiation if it is determined that peer has no
[TOLGA]

---

From: Victor Fajardo <[email-masked]>
Date: Tue, 04 Apr 2006 21:02:05 -0400

Hi Tolga,

> imply the set of supported applications or only peer state as defined in
it means both supported application and peer state.
> crossing CERs, or is this supposed to be a more generic rejection mechanism?
its a generic rejection mechanism as stated above. it can be any 
scenario where a peer does not / cannot comply with the change request 
(probably based on some policy) and the renegotiation should not continue.

I'll try to review your graceful shutdown proposal.

regards,
victor

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 5 Apr 2006 13:41:09 -0400

Hi Victor,

[.. snip ..]
> (probably based on some policy) and the renegotiation should not continue.
[TOLGA]Isn't it a bit odd to keep the supported application set unmodifed at
the node, where application is no more existing? Even if the other side does
not agree -for whatever reason- for the new advertised application set, this
won't bring the application back and I think received requests for this
application will(should) be replied with DIAMETER_APPLICATION_UNSUPPORTED.

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 05 Apr 2006 14:55:50 -0400

Hi Tolga,
>>> Furthermore, when would a node reply with
>>>       
>> DIAMETER_UNABLE_TO_COMPLY? What
>>     
>>> type of scenario do we foresee for that, only the one described
>>>       
>> below, i.e.
>>     
>>> crossing CERs, or is this supposed to be a more generic
>>>       
>> rejection mechanism?
>>     
What is proposed is that a node initiating the change request should not 
terminate the supported application until the corresponding peer agrees. 
If the peer does not agree, the node initiating the change request has 
an alternative to gracefully terminate of the connection with the peer 
if it chooses to. Otherwise, it must honor its commitment to keep the 
application support.

regards,
victor

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 5 Apr 2006 16:26:55 -0400

Hi Victor,

[..snip..]
> if it chooses to. Otherwise, it must honor its commitment to keep the
[TOLGA]AFAICS, the moment you introduce some intelligence to that procedure,
e.g. receiver of CER aggreing with the application getting down, you need to
be sure that you are communicating with your application level peer. CER/CEA
is hop-to-hop, it probably won't do it for a generic solution, where we have
relay agents etc.. between the endpoints.
OTOH, the proposed solution is usefull for the cases, where a node is just
declaring the unavailability of a certian application to its immediate
neighbors, where there is no confirmation necessary from them. In any case,
it still can be used as a generic solution, just one needs to be aware that
CEA(Success) does not necessarily mean that the actual application layer
peer has agreed for the application going down.

Just to prevent a confusion, what I call as graceful shutdown is different
than the semantics you described, i.e. shutting down an application only
after receiving an acknowledgement from the peer. In graceful shutdown, the
application node is still in full control of when shutting down the
application, it is just indicating to the peer that it will accept messages
for existing sessions. This can be indicated just with an error code,
because it doesn't require a confirmation. I believe, the semantics you
described would require an end-to-end request/response transaction between
two application layer peers, which can be achieved with CER/CEA if
client/server are direct neighbors but can't be achieved with CEA/CER, if
there are relay agents between.

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 05 Apr 2006 17:23:17 -0400

Hi Tolga,
>>> ill(should) be replied with
>>>       
>> DIAMETER_APPLICATION_UNSUPPORTED.
>>     
renegotiation is done between peers (one hope) not between end-points 
and the decision on wether to continue support for an application is 
local to a node. how a node will deal with not supporting an application 
anymore is up to the application itself.

regards,
victor

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 5 Apr 2006 17:25:02 -0400

Hi Victor,

> Sent: Wednesday, April 05, 2006 5:23 PM
> To: Tolga Asveren
> Cc: [email-masked]
[TOLGA]Yes, this is also what I think, but I think thelast sentence of the
following passage from the proposed text is contradicting with this:

"If the receiver of the CER is unable to comply with the request, it SHOULD
CER."

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 05 Apr 2006 17:51:13 -0400

> retain its pre-existing connection state with its peer prior sending the
how is it contradicting ? CER/CEA are exchanged by peers and application 
support is decided locally.

regards,
victor

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 5 Apr 2006 17:38:33 -0400

"If it decides to maintain the connection then it MUST retain its
pre-existing connection state with its peer prior sending the CER."

According to the above sentence -or better said based on what I understand
from it-, if I support two applications, want to shutdown one, sent
corresponding CER and received a negative CEA, I MUST keep  pre-existing
connection state, i.e. I can't shut down the application.

> Sent: Wednesday, April 05, 2006 5:51 PM
> To: Tolga Asveren
> Cc: [email-masked]

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 05 Apr 2006 18:04:32 -0400

You understand correctly but I'm trying to understand what is your 
objection to this sentence if any ...
regards,
victor
>>>> local to a node. how a node will deal with not supporting an
>>>>         
>> application
>>     
>>> [TOLGA]Yes, this is also what I think, but I think thelast
>>>       
>> sentence of the
>>     
>>> "If the receiver of the CER is unable to comply with the
>>>       
>> request, it SHOULD
>>

---

From: <[email-masked]>
Date: Thu, 6 Apr 2006 06:11:54 +0300

Hi all,

>>> What is proposed is that a node initiating the change 
>request should 
>>> not terminate the supported application until the 
>corresponding peer agrees.
>>> If the peer does not agree, the node initiating the change request 
>>> has an alternative to gracefully terminate of the 
>connection with the 
>>> peer if it chooses to. Otherwise, it must honor its commitment to 
>>> keep the application support.
>>>     
>> [TOLGA]AFAICS, the moment you introduce some intelligence to that 
>> procedure, e.g. receiver of CER aggreing with the 
>application getting 
>> down, you need to be sure that you are communicating with your 
>> application level peer. CER/CEA is hop-to-hop, it probably 
>won't do it 
>> for a generic solution, where we have relay agents etc.. 
>between the endpoints.
>>   
>renegotiation is done between peers (one hope) not between 
>end-points and the decision on wether to continue support for 
>an application is local to a node. how a node will deal with 
>not supporting an application anymore is up to the application itself.

I agree with Victor.

John

---

From: "Tolga Asveren" <[email-masked]>
Date: Thu, 6 Apr 2006 08:15:42 -0400

This sentence I quoted does not allow the owner of the application to shut
it down, if the CEA result code is DIAMETER_UNABLE_TO_COMPLY.

I asked you whether this error code will be used for a generic purpose, and
your reply was yes, which means it does not necessarily mean that both sides
issued CER simultaneously.
I furthermore asked whether "pre-existing connection state" includes set of
supported applications, your reply was again yes.

OTOH, we want to decide for application support locally, but according to
the above statements rather than being a local decision it requires more an
agreement between two nodes, e.g. if CEA has result
code=DIAMETER_UNABLE_TO_COMPLY *for some reason*, issuer of CER shouldn't
shutdown the application.

And one idea for the simultaneous CER case:
Is this serialization really necessary or is it enough to kill one of the
negotiations? The CER/CEA for the first completed renegotiation transaction
will contain anyhow the supported application sets for both sides. Why do we
need the second one?

> Sent: Wednesday, April 05, 2006 6:05 PM
> To: Tolga Asveren
> Cc: [email-masked]

---

From: "Tolga Asveren" <[email-masked]>
Date: Thu, 6 Apr 2006 08:52:09 -0400

> will contain anyhow the supported application sets for both
[TOLGA]A further idea for this one: Considering that application support
decision is local, why do we care about simultaneous CERs at all? Each of
them will advertise the supported application set for both sides at that
moment. What is wrong with that?

---

From: Tina TSOU <[email-masked]>
Date: Fri, 07 Apr 2006 17:55:48 +0800

Hi,
I think the schema is ok but for simultaneous renegotiation
serialization -
"A Diameter node that receives a CER from a peer from which it has sent
its own pending CER MUST be able serialize the simultaneous
renegotiation. If the Diameter node is acting as a responder (see Sec
5.6) then it MUST give precedence to the renegotiation initiated by its
peer and send a CEA accordingly.
If the Diameter node is acting as an initiator (see Sec 5.6) then it
MUST delay the renegotiation initiated by its peer (the responder) and send 
a CEA
with Result-Code AVP set to DIAMETER_UNABLE_TO_COMPLY in order to retain
the current state between them. The receiver of this CEA (the
responder) SHOULD retain its current state and proceed with the ongoing
renegotiation initiated by its peer. It SHOULD retransmit its own CER to
initiate another renegotiation after the current renegotiation
completes. Both renegotiations SHOULD be treated as separate and 
sequential."

Does any one understand why this spl. handling is needed in the event of
a renegotiation cross over? Even without this, I think each CER will be
treated atomically by the receiver.

Tina

----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, April 06, 2006 11:11 AM
Subject: RE: [Dime] CER/CEA on an open connection

Hi all,

I agree with Victor.

John

---

From: Victor Fajardo <[email-masked]>
Date: Fri, 07 Apr 2006 09:24:31 -0400

Hi Tina,

Comments inline:
> MUST delay the renegotiation initiated by its peer (the responder) and 
> send a CEA
Maybe I'm wrong but I'm not clear on how multiple pending transactions 
can be resolved automatically without serialization or extra logic 
(i.e., election process in the rfc). Anyway, the basic idea of the text 
was to "assist" a diameter node in making sure serialization does occur. 
"Assistance" here implies that a diameter node need not be burdened with 
extra state/logic of handling simultaneous renegotiation since such 
renegotiation will be serialized. I don't have a strong opinion on this 
but if we get a consensus that we do not need to specify this behavior 
and let the implementations deal with it in their own way then we can 
make this text optional.

hope this helps,
victor
>
> Tina
>
> ----- Original Message ----- From: <[email-masked]>

---

From: "Tolga Asveren" <[email-masked]>
Date: Fri, 7 Apr 2006 09:13:47 -0400

Hi Victor,

Once there is the assumption that shutting down applications is completely
decided in a local way -which as far as I understand everybody agrees(?)-, I
don't see a need for any type of serialization. If you see a problem
scenario, could you provide message flow for it, otherwise IMHO there is no
need to complicate things.

Furthermore, there is also no need to do something special based on the
result code of CEA, if we want to honor "complete local decision for
application shutdown" principle. Regardless of CEA result, application will
be shut down -actually even wothout waiting for CEA-.

Probably what we need to say is just that renegotiations are limited to
advertise new set of supported applications.

    Thanks,
    Tolga

> Sent: Friday, April 07, 2006 9:25 AM
> To: Tina TSOU
> Cc: [email-masked]; [email-masked]; [email-masked]