Subject: RE: [AAA-WG]: issue: real-time duplicate detection and proposed change.

From: <[email-masked]>
Date: Tue, 15 Oct 2002 12:56:41 +0300

Hi,
 
sequence number is not the same as Session-Id and record number?
If we consider the case of "one time event" direct debiting, the ACR[event, Session-Id, Record-number] is sent to
the server and the server loop back the same Session-Id and Record-number in the ACA.
am I right?
 
When the server receives the ACR money are deducted from the user's account, If the server doesn't know if the ACR is
a possible duplicate, it needs to perform database search among at least all the records received by this particular client 
within a reasonable time window (e.g. a day) to avoid double debiting, database searchthis is what we try to avoid.
 
Hashing may do it with certain memory requirements, D flag may do it with less memory requirements.
 
Br
Marco
 
 -----Original Message-----
From: ext Kuntal Chowdhury [mailto:[email-masked]]
Sent: 14. October 2002 22:44
To: Stura Marco (NET/Helsinki); [email-masked]; Loughney John (NRC/Helsinki)
Cc: [email-masked]
Subject: RE: [AAA-WG]: issue: real-time duplicate detection and proposed change.

Hello, 
As I understand the issue: we are trying to make sure that the (near)real-time accounting systems (e.g. pre-paid) do not debit/credit users accounts multiple times when the same message is re-transmitted. Can't this be solved by appending a message sequence number (string) with every request? The client will send the same sequence number as long as it does not receive a response containing the same sequence number. The server should loop back the same sequence number in the reply. If the same message sequence was received at the server before (a reasonable time window for real-time service) then the server replicates the earlier message and send back a response w/o crediting/debiting the user's account. upon receiving the response with a previously generated and acknowledged sequence number the recipient ignores the message.

I am not against the D-bit idea, however I think the message sequence numbers provide a good protection mechanism for avoiding erroneous accounting due to duplicate requests. 

Regards, 
Kuntal 

> -----Original Message----- 
> From: [email-masked] [ mailto:[email-masked]] 
> Sent: Monday, October 14, 2002 11:07 AM 
> To: [email-masked]; [email-masked] 
> Cc: [email-masked] 
> Subject: RE: [AAA-WG]: issue: real-time duplicate detection 
> and proposed change. 
> 
> 
> Hi Bernard, 
> 
> thanks for considering the issue. 
> 
> Actually the real benefits of the D flag is that duplicate 
> happen very seldom, so for all the D=0 records (say 90-95% of 
> the records) the server can immediately process the request 
> and answer to the client e.g. service granted or no money. 
> 
> D=1 records are statistically very few and duplicate search 
> only for those records do not affect the overall performance 
> of the system. 
> 
> Finally, your proposed text seems better than my original 
> proposal. However, I'm not yet fully sure it is acceptable to 
> delay the processing of all the D=1 records and don't have 
> time to think now since I have to harry on and go back home, 
> I'll come back tomorrow. I guess Time_Wait may be couple of 
> seconds, am I right? 
> 
> However proposal#2 could be just to replace the text I 
> proposed in Annex C that state that D falg may be used by 
> server implementation and specified in diameter application etc..etc 
> 
> John could you make some proposal to replace my text? 
> 
> 
> Br 
> Marco 
> 
> > -----Original Message----- 
> > From: ext Bernard Aboba [ mailto:[email-masked]] 
> > Sent: 14. October 2002 16:17 
> > To: Loughney John (NRC/Helsinki) 
> > Cc: [email-masked] 
> > Subject: RE: [AAA-WG]: issue: real-time duplicate detection 
> > and proposed 
> > change. 
> > 
> > 
> > > The alternatives I see are: 
> > > 
> > >   1) Not supporting the d-bit. 
> > >   2) Allow the setting of the d-bit, but either making 
> > the response 
> > >           implementation specific or dependent on futher 
> > specification. 
> > >   3) Supporting the d-bit and the behavior of responding 
> > to messages 
> > >           with the bit set. 
> > 
> > We have an issue filed (372). Approach #1 would be to reject 
> > the issue. 
> > Approach #3 appears to accept the changes requested in 
> Issue 372. What 
> > does approach #2 do? 
> > 
> > The text for Appendix C says: 
> > 
> > "Diameter server MAY check the D flag of the received message 
> > to determine 
> > if this record is a possible duplicate. If the D flag is set in the 
> > request message, the server search for the duplicate within 
> the above 
> > mentioned time window. This way database search is only 
> > required for those 
> > received records whose the D flag is set. Since in today 
> > networks link failures, server 
> > breakdown or other similar events are very rare, this 
> simple approach 
> > efficiently optimize performances of the duplicate 
> detection process. 
> > It may happen that the original record is received after the 
> > D flag marked 
> > record because they did experience different delay in their 
> way to the 
> > server. For this reason it is necessary for the server to 
> buffer the D 
> > flag marked records for a short time period, as an example 10 or 15 
> > minutes, and check every record received within this time 
> > period against 
> > the buffered D flag marked records." 
> > 
> > The problem is that failover can cause the accounting record 
> > to be sent to 
> > *another* server. For example, if a proxy fails, the Diameter 
> > client may 
> > go to another proxy, and the new proxy may set up a connection to a 
> > different accounting server. The approach described in the 
> > last sentence 
> > appears to assume that the duplicate arrives at the *same* 
> > server as the 
> > original record. 
> > 
> > I'd suggest that the last sentence say: 
> > 
> > "As a result, a simple approach is not foolproof, and may result in 
> > occasional duplicates being accepted by the system. If occasional 
> > duplicate records are not acceptable, then it may be difficult to 
> > retain realtime performance. If the server cannot locate 
> the original 
> > record corresponding to the D flag marked record, it is still 
> > possible for 
> > the original to arrive later. To ensure that the duplicate is 
> > detected, it 
> > is necessary to delay processing of the D flag marked record until 
> > the original record can be assured to have been received. 
> > This corresponds 
> > approximately to TIME_WAIT." 
> > 
> > My question is whether the above defeats the purpose of realtime 
> > processing. 
> > 
> > 
>

---

From: "Glen Zorn" <[email-masked]>
Date: Wed, 16 Oct 2002 11:26:41 -0700

[email-masked] writes:

> If we consider the case of "one time event" direct debiting, the
> ACR[event, Session-Id, Record-number] is sent to
> When the server receives the ACR money are deducted from the
> user's account, If the server doesn't know if the ACR is
> a possible duplicate, it needs to perform database search among
> at least all the records received by this particular client
> within a reasonable time window (e.g. a day)

This part has me confused.  We're talking about (near) real-time accounting
here and others have suggested that a delay in response of even 120 seconds
is too long.  So why are you going to search a day's worth of records for a
duplicate?  For that matter, couldn't you simply flag the requests that have
been replyed to in your database (you are keeping the requests, no?).

> to avoid double
> debiting, database searchthis is what we try to avoid.
>
> As I understand the issue: we are trying to make sure that the
> (near)real-time accounting systems (e.g. pre-paid) do not
> debit/credit users accounts multiple times when the same message
> is re-transmitted. Can't this be solved by appending a message
> sequence number (string) with every request? The client will send
> the same sequence number as long as it does not receive a
> response containing the same sequence number. The server should
> loop back the same sequence number in the reply. If the same
> message sequence was received at the server before (a reasonable
> time window for real-time service) then the server replicates the
> earlier message and send back a response w/o crediting/debiting
> the user's account. upon receiving the response with a previously
> generated and acknowledged sequence number the recipient ignores
> the message.
>
> I am not against the D-bit idea, however I think the message
> sequence numbers provide a good protection mechanism for avoiding
> erroneous accounting due to duplicate requests.
>
> Regards,
> Kuntal
>
>