Subject: [AAA-WG]: Issue 235 - Enabling efficient accounting record uniqueness checking

From: "Patil Basavaraj (NET/Dallas)" <[email-masked]>
Date: Wed, 16 Jan 2002 11:23:08 -0600

Hello,

At IETF52, Issue 235 was discussed and postponed in lieu of further
clarificaions. Below is a revised version of the issue and the
justifications. I will post further clarifications in a followup email.

-Basavaraj

----------------------------------------------------------------

Issue 235: Enabling efficient accounting record uniqueness checking
Submitter name: Basavaraj Patil
Submitter email address: [email-masked]
Date first submitted: November 9, 2001
Reference: 
Document: Base-08
Comment type: T
Priority: 1
Section: 3.0, 9.4
Rationale/Explanation of issue: 

If a Diameter server does not have any indicator in the received
Accounting Requests for the message being potentially duplicated,
the Diameter server (or a separate Accounting System) would have
to do a vast amount of unnecessary work just for the Accounting
Request uniqueness checking. 
With more and more "hot billing" systems and prepaid billing for
packet data becoming prevalent it becomes a requirement for accounting
messages (records) to be sent far more frequently and also enabling
speedy evaluation for accuracy and uniqueness.

For accounting, the section 9.4 (Fault Resilence) defines
that duplication detection must be based on the
inspection of the Session-Id and Accounting-Record-Number
AVP pairs. The current draft text assumes that every
accounting message would be cross-checked with these identifiers against
the others, to detect duplicate accounting messages.

However it is far more efficient to o first check if the Accounting
Request sender has flagged the message to be a potential duplicate.

(Duplicate messages may be generated in the
exceptional cases when a communication link between
Diameter peers goes down. This could happen due to
the sending node failure or the receiving node failure or
the communication link itself failing. e.g. physically
damaged. When the sending node is redirecting the traffic
to an alternate peer or able to communicate again
after a temporary link failure to the primary peer,
it may send again messages that have not yet
been acknowledged by a recipient node.)

Duplicates are generated very seldom. (Typically
just a few packets, for eg. after a link failure case.)
Therefore duplicate checking all-against-all, by
seems a processing intensive work and one that is unnecessary to be
performed by accounting servers.
(by the Diameter server or a billing system).

The 'D' bit would be for all such accounting
requests that were pending an application layer ack
when the connection went down, whether they are
resent on a connection to the same peer or an alternate
peer.

In failover scenarios, duplicate checking may not necessarily
be done in the Diameter Server. It may alternatively
be done in a billing engine. In that case the
billing engine should get the 'D' bit information
from the Diameter header, as delivered together with
the payload.

(It should be noted that the potentially duplicated
accounting message may arrive earlier at the end system
than a non-marked original accounting message. This is
the case with any duplicate occurrence scenario, but
is not a problem as the end system should anyway have
some time buffer, e.g. 24h or 12h, for the received
accounting records to be checked for duplicates.)

The significant benefit achieved here
is the decrease of uniqueness checking processing
comparisons 1) from e.g. 77 octets to 1 bit and
2) from cross-comparing all accounting messages
in the time buffer against each other to comparing
just the very few potentially duplicated accounting messages to all
other accounting messages in the uniqueness checking time buffer.

The End-to-End Identifier could in theory be also
utilized by the uniqueness checkings in accounting
but has not been defined. Additionally,
the End-to-End Identifier has recently been considered
to be unique only during a very limited time period,
e.g. 4 minutes, which would not always be long enough
when e.g. a sending Diameter client node is in temporary
isolation due to a network failure for several hours.

The scheme proposed here does does not limit
the Diameter Server or billing engine to do the
uniqueness checking in any other way.
(eg. all accounting records checked against each other,
by the long AVP pair octet strings).

Requested change: 
=================
In section 3.0  Diameter Header

...

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |R P E r r r r r|                  Command-Code                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

would be:

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |R P E D r r r r|                  Command-Code                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

and

         E(rror)     - If set, the message contains a protocol error,
                       and the message will not conform to the ABNF
                       described for this command.  Messages with the
                       'E' bit set is commonly referred to as an error
                       message. This bit MUST NOT be set in request
                       messages. See section 7.2.
         r(eserved)  - these flag bits are reserved for future use, and
                       MUST be set to zero, otherwise an error MUST be
                       sent to the sender.

would get one new flag bit 'D':

         E(rror)     - If set, the message contains a protocol error,
                       and the message will not conform to the ABNF
                       described for this command.  Messages with the
                       'E' bit set is commonly referred to as an error
                       message. This bit MUST NOT be set in request
                       messages. See section 7.2.
         (possibly) D(uplicated)
                     - This bit is defined only for Accounting Requests,
                       sent by a Diameter node or proxy.
                       If set, the message is possibly a duplicate,
                       and must be later checked by a recipient node
                       if it really is a duplicate. If sending a
                       message for first time, this bit MUST be
                       cleared.
                       This bit MUST NOT be set if an answer message
                       (about e.g. a protocol error) has been got for
                       an earlier similar message. It can be used only
                       in cases where no answer has been got from the
                       primary agent for a message and the message is
                       sent again (e.g. due to a failover, to an
                       alternate agent or to a recovered primary peer
                       node).
         r(eserved)  - these flag bits are reserved for future use, and
                       MUST be set to zero, otherwise an error MUST be
                       sent to the sender.

and in section 9.4  Fault Resilience
====================================
   ...
   Diameter peers acting as clients MUST implement the use of failover
   to guard against server failures and certain network failures.
   Diameter peers acting as agents or related off-line processing
   systems MUST detect duplicate accounting records caused by the
   sending of same record to several servers and duplication 
   of messages in transit. This detection MUST be based on the
   inspection of the Session-Id and Accounting-Record-Number AVP pairs.

the above paragraph would get one sentence to its end:

   The inspection MUST be performed at least for such records
   that arrived at the Diameter peer specifically in accounting
   messages which have the 'D' bit set.