Subject: RE: [AAA-WG]: Use of the D-Bit clarification

From: <[email-masked]>
Date: Thu, 17 Oct 2002 08:58:33 +0300

Hi Randy,

> so, please remind me why we need to make things more complicated?

As I understand, the proposal is to set a bit for retransmitted 
message requests:

	D(uplicate) - If set, the message is a possible duplicate request 
	due to failover/failback or re-sending of a
	record in storage. This flag MUST NOT be set in 
	answer messages.

That is all.  This, in my mind, is not so very complicated.

Some text is added to an appendix, suggesting what the receiver
should do with the d-bit, but let me outline a possible use of
this.

I am an end user with a certain amount of tokens on my account,
for using with services.  Whenever I make a service request,
my account is debitted a token, in real time.  If, because of
some network problems, an accounting request is retransmitted,
there stands the chance that my account will be doubly debitted,
and if I am low on tokens, then I may be wrongly denied the service.
In the real world, this is a good way to piss off customers.

The proposal is to use the d-bit with real-time accounting to
assign a priority to all accounting requests where the D-bit = 0,
as we can safely assume this message is not a duplicate request.
With out it, we can never assume that an accounting request 
is not a duplicate.

For an accounting request where the d-bit = 1, then the proposal is
not to process the messsage in real-time, but to have an optimized
duplicate detection algorithm and/or configurable based on service
provider's needs.  With any luck, messages with the d-bit = 1 should
an extremely small number.  This type of mechanism is already
in use in some GPRS networks and shown that the mechanism is
effective.

Of course, one could suggest having a hashed-based look-up to perform
the duplicate check, but that may increase the delay to the user; as
well as cause additionally hardware needs (memory & processor) to the
service provider to ensure that they are able to perform the duplicate
check in real-time.

I understand that this type of functionality may not be needed
in traditional IP-style accounting practices; but for cellular
packet services, where heavy use of pre-paid services is common,
this is needed.

So, my proposal is that we accept setting of the D-bit, as proposed
in issue 372.  Setting of this bit does not seem to break anything.
Further more, the Credit Based Charging Diameter Application has
been brought to the IETF, for standardization consideration.  My
recommendation is that this draft is the place where to handle the
duplicate-detection mechanism.

Does this sound reasonable?
John

---

From: <[email-masked]>
Date: Thu, 17 Oct 2002 09:41:42 +0300

Hi,

John made exactly the point.

> duplicate-detection mechanism.

I agree with John that the Diameter application specification is the place
where to provide eventually detailed description of how the Server
uses the D-flag, the base probably need to provide just an indicative
example.

Here follow additional considerations with regards to the end system discussion.

Best Regards
Marco

I guess that actually the D-flag make things much simpler, that's why 
the same mechanism is already in use for example in 3GPP GPRS network
around the globe (both 2.5G and 3G).

If you think of an end system (also the end system may be a Diameter
server, or e.g. the account holder in case of prepaid) receiving
millions of request, if D=0 (say 99% of the requests)the request
is immediately processed and answer is sent, if D=1 no harry to 
process the request. (here attached my previous mail with D-bit
usage clarification for your information).

If D-flag is not in use, the end system doesn't know if the incoming
request may be a duplicate thus need to check for every single received
request if this is a duplicate or not. For the end system, which is
the collector of all the requests generated in the network, is even
more critical if D-flag is not in use.

Setting the D-flag for the client and agents, is extremely simple
as compared to the extra work that the Diameter Sever and end
system would need to do if D-flag is not in use.

"The key is that once a record arriving from Diameter client to
a Diameter server and is marked with D=1, then ***if the original is not found 
in the +- time window (e.g. TIME_WAIT),then the D=1 mark is not removed*** 
from the accounting record when sending it further towards the end system. 
The network's accounting end system can be understood and implemented as 
a _central logical entity_(no matter if it would physically consist of several servers).
It could then be left to the end system to compare the possible
still existing (very, very few!) D=1 marked records against others
in a configurable +- time window."

> -----Original Message-----
> From: ext [mailto:[email-masked]]
> Sent: 17. October 2002 8:59
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: [AAA-WG]: Use of the D-Bit clarification
> 
> 
> Hi Randy,
> 

Subject: [AAA-WG]: Use of the D-Bit clarification
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>
Date: Wed, 16 Oct 2002 14:50:08 +0300

Hi again,

here the mail that try better explanation of the D-bit.

For real-time accounting applications, such as the Diameter Credit Control
draft, it is important to immediately process the original request (D=0)
since action by the client (action to the user) will be 
taken if the answer doesn't come within a certain time. As the user
cannot wait for the service to be granted for 20 or 30 seconds, usually
a real-time accounting application defines a "waiting timer" after which
expiration the service is granted or not granted to the end user. A 
value of the timer is about 10sec. 

The client behavior is usually controlled by some "failure handling"
parameter. For example the failure handling for "one time event"
direct debiting in case "waiting timer" expires could be "grant the
service and store the request" or "grant the service and forget the
request", this is configurable by the service provider.

Storing the request is a sort of back-up to give a chance to the 
service provider that the used service may be debited to the
user's account later and revenue is not lost.

If a duplicate happens to reach the server, the "waiting timer" is normally 
expired and proper action to the end user already been taken.

This "one time event" direct debiting could be applicable to messaging
services, for example. For session based services e.g. VoIP or data services the one
time event approach is likely not applicable since online monitoring of the ongoing
session is needed. In this cases start/interim/stop approach where money 
is reserved first and debited only when consumed by the service is the best solution. 
A stateful server would be used and also in this case is important to immediately 
process the original request (D=0). Sice stateful server would be used, moving the
accounting data stream to alternate server shouldn't be possible. 
However duplicate may happen also in this case.

In all cases when duplicate happen to reach the server, the client application
"waiting timer" is normally expired and proper action to the end user already been taken.

Conclusion of the above is:

1- original request (D=0) must be processed immediately.
2- no hurry to process the duplicate (D=1)

When D=0, no checking is needed in the Server and the record is immediately
processed.

Only when a D=1 accounting record is encountered, then the checkings are done 
(its AVP pair against the other Session-ID & Accounting-record-Number AVP pairs
 of the +/- time window). When D=1, its Session-ID & Accounting-Record-Number AVP pair
is buffered for a possible future check, but no check is needed in normal case. 
A link failover procedure (causing the D-bit to be used for the last few yet 
unacknowledged accounting records, the amount being in maximum the protocol 
window size of records only) occurs e.g. during 2-3 days per year, so the rareness 
of the D=1 case causes that
     1- most of the time there are no extra checkings at all
        (when using the D-bit).
     2- when there are checkings the search is done just for the few D=1
        and only within the configurable time window to the future & past. 
       (Of course, the checks can be Session-ID based).   
     3- only limited amount of resourses for duplicate detection are required
	  in the Server.

Out of order records shouldn't be an issue because of the +/- time window. 

The key is that once a record arriving from Diameter client to
a Diameter server and is marked with D=1, then ***if the original is not found 
in the +- time window (e.g. TIME_WAIT),then the D=1 mark is not removed*** 
from the accounting record when sending it further towards the end system. 
The network's accounting end system can be understood and implemented as 
a _central logical entity_(no matter if it would physically consist of several servers).
It could then be left to the end system to compare the possible
still existing (very, very few!) D=1 marked records against others
in a configurable +- time window.

I hope the above clarify some of the discussed topics.

Br
Marco

---

From: <[email-masked]>
Date: Fri, 18 Oct 2002 08:47:46 +0300

Hi,

> Kuntal Chowdhury wrote:
> 
> >  > For an accounting request where the d-bit = 1, then the
> >  > proposal is not to process the message in real-time, but to
> >  > have an optimized duplicate detection algorithm and/or
> >  > configurable based on service provider's needs. 
> > 
> > Here you are saying that D/T=1 messages wouldn't be processed in
> > real-time. What happens if the D/T=0 was never received at 
> the server.
> > The user will get service only when the server responds back for the
> > D/T=1 message which was delayed (non real-time) by several 
> seconds e.g.
> > 30 -120 secs at least. The user has to wait upto 2 mins to 
> get service.
> > Won't that piss off the user?

Please refer to my explanation of the use of the T-flag (at that time still D)
Mail subject "[AAA-WG]: Use of the D-Bit clarification" why the user wouldn't
be piss off and what usually are the application mechanisms defined
for real-time accounting (i.e. prepaid).

Jari Arkko wrote:

> I agree. The T bit is *only* for retransmission detection, not
> for priority.

Agreed, absolutely no priority meaning.

> (In fact, I have previously proposed a new AVP for specifically
> representing priority, so that proxies can schedule outgoing
> records in priority order. This is a much more general and useful
> mechanism, and can hopefully be done as a separate RFC at some
> point in time.)

Very interesting indeed.

> Yes. People have pointed out that duplicate detection can be
> performed without protocol extensions as well. The Nokia folks
> have stated that they believe they can do this detection faster
> or with less memory if the T bit is introduced to the protocol.
> *I* would choose other implementation tactics myself, but the
> matter is quite important for very large deployments and I'd
> hate to get into trouble later with this. Without specific
> measurements to prove the case one way or the other, I'm
> willing to accept an informational T bit in the header, as
> long as we don't mandate any specific duplicate detection
> algorithm on the server side. And I don't want any priority
> meanings for this bit either, for the reasons that you Kuntal
> brought up well in the above. If it turns out that most
> implementations don't use the T bit, at least we can get
> some useful information about the Diameter transport behaviour
> by observing the number of T=1 requests.

Right Jari! Nokia folks proposal was based on implementation experience
with existing charging protocols e.g. used in 3G/2G GPRS networks.
And since the matter is very important for very large deployments, you
exactly made the point.

Thanks Jari
Marco