Subject: [AAA-WG]: issue: real-time duplicate detection

From: <[email-masked]>
Date: Fri, 11 Oct 2002 14:54:40 +0300

Hi,

I apologize to open this discussion so late but in Nokia we just notice 
the we have a problem while discussing the possible error cases for the
draft Diameter Credit Control Application, which is also used in 3GPP 
Rel5 IMS (IP Multimedia subsystem) to realize real-time credit based 
accounting (i.e prepaid accounting, sometime called token based accounting).

The problem we have is related to real-time duplicate detection of event
accounting request. I briefly summarize the problem below.

In Credit Control Application is possible to perform one-time event for three
different purposes: direct debiting, refund and balance check. The problem
appears with direct debiting and refund.
One time event uses accounting request[event] message and basically when the
action is direct debiting the corresponding amount of money is directly
deducted from the user's account and when the action is refund the corresponding
amount of money is credited in the user's account. 

These use cases require real-time duplicate detection of the received event
request in order to avoid multiple debiting or multiple refund for the same
request. The duplicate detection must be performed and the proper answer 
must be given back to the client immediately so as to grant or to not grant
the service to the end user. In short, within few seconds the service need
to be granted or not granted to the end user, if the end user has to wait too
long before being able to use the requested service (e.g. Instant message)
the user experience will suffer too much.

With the current Diameter base protocol, for every single received request 
the Credit Control Server would need to search possible duplicate among other
request received within a certain backward time window (e.g a day). 
In current wireless GSM network there are several milion of prepaid 
subscriber and we have feeling that is an impossible task for the CC server
to perform real-time duplicate detection with the mentioned approach,
performances will suffer and the answer will be sent too late to the client.

Use of hashing techniques or other schemes help to eliminate the need to do 
full search of received accounting request within a time window as mentioned
in AnnexC.
However, the hashing approach to address real-time duplicate detection hasn't been 
tried in practise and hasn't been evaluated in full.
We beleive that hashing introduce additional complexity to server implementation and
also in finding the proper hashing function, hash table size etc. to minimize
for example collision cases. In contrast other techniques for duplicate detection
are already used, proven to work and operational in live networks with millions of
subscribers(e.g. 3GPP GPRS charging protocol).
This techniques consist in the client marking the possible duplicate upon re-transmission
of the messages. For those not familiar with this technique I suggest to read
the 3GPP technical specifications 32.215
(http://www.3gpp.org/ftp/Specs/archive/32_series/32.215/). Implementations in 3GPP use
this mechanism to also support hot-billing where real-time duplicate detection
is required and is proven to work in operational networks.

For the above mentioned reasons we think that there might be implementation that 
do not make use of hashing techniques and that even do not plan to support these 
techniques in near future.
Thus we believe that a more simple tool to enable optimization of real-time 
duplicate detection should be provided by the base protocol in addition to
hashing techniques or other complex schemes that a server may chose to implement.

A not so clean solution to this problem is the Re-transmission AVP defined
in the CC Application that however introduce some challenge with the management
of the base protocol pending message queue(i.e. CC messages in queue need to be 
deleted upon failover/failback execution to avoid their re-transmission without 
being marked with the AVP). In addition this solution does not solve the problem 
when relay/redirect agents are between client and server.
The CC servers implementation that decide to use the Re-transmission AVP will
search for possible duplicate only when a request message is received with this
AVP and not for every single received request. 

We feel that a better solution based on the base protocol may exist. We propose
that the base protocol to mark re-transmitted messages by dedicating one reserved
bit of the Diameter header to this purpose, server may use D-bit or hashing according
to their implementation. Please note that the way the server implement duplicate
detection (use of D-bit or not) is implementation dependent and does not affect 
interoperability at all.
This proposal was already discussed sometime ago (named D-bit)and was rejected because
the real-time duplicate detection was not seen necessary, we beleive that this requirement 
is now existing due to the above mentioned reasons and proven to work tool to implement 
simple real-time duplicate detection in the server (without adding implementation complexity)
should be offered by the base protocol.

In conclusion if you have better proposal let us know otherwise we will suggest small
changes to chapters 3, 5.5.4 and AppendixC on following e-mail for your review and comments.

With our best regards
Marco, Helen, JP

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 11 Oct 2002 06:12:24 -0700 (PDT)

> being marked with the AVP).

In practice, this is very difficult to achieve. All a Diameter client can
know is that it didn't receive an Application Layer ACK for the accounting message.
There can be Diameter agents between the client and server, so that
failover is initiated based on "liveness" of the Diameter agent, not
receipt of the application layer ACK. If the Diameter agent is "live" (as
indicated by receipt of DWRs) then it could be failing over to a different
accounting server, and the Diameter client will not know it -- and
therefore would be unable to mark the messages appropriately. And of
course, the Diameter client has no visibility into transport behavior
anywhere along the path.

Even with no Diameter agents, it is possible that the server
did send the ACK, but somehow it was never received, due to network or
other problems. The Diameter client does not know whether it received
a transport layer ACK for the accounting message, and whether retransmission
of the accounting message or ACK is occuring at the transport layer.

Based on the server not demonstrating "liveness" (such as lack of DWRs),
the Diameter client will failover. Since the failover time is less than TIME_WAIT, it
is conceivable that the failover connection will deliver the accounting
message (and application layer ACK) and that afterwards an application
layer ACK will finally be received from the failing connection. So in
practice, some sort of search is necessary in any case, within a time
window proportional to TIME_WAIT.

> AVP and not for every single received request.

Unfortunately, that approach is not reliable, due to the vagaries of
transport behavior -- and it fails completely where Diameter agents are
deployed.

> the real-time duplicate detection was not seen necessary

Actually, it was rejected because the proposed solution does not work.

---

From: David Mitton <[email-masked]>
Date: Fri, 11 Oct 2002 10:20:46 -0400

Utility request:
         Has anyone spread-sheeted the Diameter (and/or/both RADIUS) AVPs 
(and attributes)??

Even a dictionary in tabular (or CSV) form would help.

I'd appreciate a copy for checking tables and such.

Thanks,
         Dave.