Subject: RE: [AAA-WG]: Request for closure on issue 235

From: "Pat R. Calhoun" <[email-masked]>
Date: Wed, 27 Feb 2002 07:15:43 -0800

...
> 
> My problem with the D-bit is that it invites doing something in
> real  time that does not need to be done in real time, and puts a
> burden on  the sender while saving very little if anything for the
> receiver. The  biller is going to sort the records anyway, and
> finding adjacent 
> duplicates is trivial with or without the D-bit.
> 
> Barney Wolff

But there are cases when billing actually takes place in real time,
called prepaid charging. So the usage of the D-bit would make a
significant performance improvement for the receiver in these cases.

<PRC> I seriously don't understand the insistence on using User-Name
for the lookup. Not only is there a Session-Id, which is unique, but
there is also an accounting record number. Those two items are
guaranteed to be unique, and therefore are a great way of checking
for dups. I don't understand the mechanism by which records are put
in a "temporary" buffer, as opposed on written on disk. You need to
write ALL records in this "temporary" buffer (if it even exists) in
order to check for duplicates. 

What really confuses me here is how the checks are done. Using a
single bit 'D' isn't really useful for a lookup. Some additional data
is required, and whether the 'D' bit is set or not, the message must
be cached for duplicate check (to handle out of order packets).

Confused,

PatC

---

From: Barney Wolff <[email-masked]>
Date: Wed, 27 Feb 2002 18:42:16 -0500

I did not mean to imply that User-Name was the only key.  But records
will certainly be accumulated for a given user, whether because that's
how they are billed or for the bill detail.  So that sort (or index)
is already being done.  Given that, the universe of records that must
be checked for duplication is really quite small, which is why I don't
believe the D-bit actually saves anything noticeable.

Do people believe generating the D-bit reliably is a trivial burden on
the sender?  If so then it does no harm, because the receiver can
always ignore it.  But I don't think it's that trivial.  As a receiver
I'd be very wary of depending on the sender to trigger the dup check.

Barney

---

From: Jari Arkko <[email-masked]>
Date: Thu, 28 Feb 2002 23:33:20 +0200

Ok, so after watching the mailing list discussion and having wondered
about this for a while also in a conference call with the authors
yesterday... and my proposed secondary method... I think we are not
going to get consensus for this. Instead, we should add some text in
the Base that discusses duplicate elimination.

Some of the comments said that the method does not work. Maybe
we could have finally agreed that it does work, because it does appear
to work, at least my analysis said so. But even so, two concerns have
been raised that are harder to agree on: 1) true benefit of the RAM
check vs. the index check and 2) behaviour in reboot, RADIUS
interoperability etc. situations.

All is not lost however, for the D-bit forever. Specific applications
*can* introduce an AVP for the same purpose, if they like. However,
I hope the new text builds some understanding of the issues as well
so extension designers and implementors can take a wider understanding
of the issue. If my proposed scheme works -- I'm not 100% sure --
I would think that would be preferred, though.

Here's the proposed text. Add a reference in 9.4 to a new appendix B
'Appendix B discusses duplicate detection need and implementation
issues.'

Then, Appendix B:

As described in section 9.4, accounting record duplicate detection
is based on the session identifiers. Duplicates can appear for
various reasons:

- Failover to an alternate server. Where we close to real-time
   performance is expected, failover tresholds need to be kept
   low and this may lead to a relatively large likelihood of
   duplicates.

- A crash of a client at the time it just had managed to
   send a record from a non-volatile memory would likely
   cause the same record to be sent soon after the client
   has rebooted.

- Duplicates received from RADIUS gateways.

- Implementation problems and misconfiguration.

In some cases the Diameter accounting server can delay
the duplicate detection and accounting record processing
until a post-processing phase takes place. At that time
records are likely to be sorted according to the User-Name
contained in them and duplicate elimination is easy in
this case.

In other situations it may be necessary to perform real-time
duplicate detection, e.g. when the credit limits or fraud
attempts are being monitored in real time.

In general, only the duplicate generation at failover
case is something that can be reliably detected by
the Diameter client. The Diameter server is therefore
responsible for the duplicate detection process. When
real-time duplicate detection is required, this implies
a database-like search functionality to find duplicate
records. Implementors are advised, however, that
there exists ways to avoid expensive all-record searches.
For instance, it can be usually safely assumed that duplicates
appear within a time window of longest imaginable network
partition, perhaps a day as an example. So only records within
this time window need to be looked at. Secondly, hashing
techniques or other schemes may be used to eliminate the
need to do a full search even in this set except for rare
cases.