Subject: RE: [AAA-WG]: issue: real-time duplicate detection and proposed change.

From: <[email-masked]>
Date: Tue, 15 Oct 2002 17:40:02 +0300

Hi,

I'm not sure my previous mail went through, so I re-send.

> It seems that there is an assumption that _all_ Diameter clients and
> agents mandatory support 'D' bit set when failover/failback occurs.
> Otherwise, the scheme will not work as is expected when
> failover/failback occurs at some node that does not set 'D' bit while
> the server supports D-bit handling, resulting in duplicated accounting
> request undetected.

Yes, the proposal is for diameter base (client and agents) to set the D flag when
failover/failback occur or, only for client, when a stored record is re-sent.
The server may support this scheme or other schemes as already mentioned 
in the Annex C.
I guess setting this flag is really minor change.

> If the assumption is correct, I'm not sure if the scheme is worth
> being defined in the base specification while other accounting
> application does not require this scheme and more generic scheme
> (i.e., hashing) may be able to solve the problem without changing the
> specification.

I guess hashing is placing additional memory requirement to implementation, here is some
rough number I drafted as an example in some other mail, how much those numbers are accurate
I don't know but they give an idea. The point is that not all the implementations are willing to 
use this memory size just to cover cases that happen very seldom , which also translate in cost 
for your product.

I don't know what it could be the hashing function but let's try to analyze what size 
the hashing table might be considering the multimedia messaging use case. 
Currently in GSM networks people send in average 1 SMS per hour.
If your server need to handle 4 millions of prepaid subscribers it means that in 24
hours it will get 4*24=96 millions requests. If we consider a Key of 32-bits(4 bytes)
the hashing table should be 96*4= around 370MB. Now if we want to limit the risk of collision
we may double the size to reduce to load factor 370*2=740MB. But I just considered one 
service, if we consider other services as well we can safely assume 2 or even 3 requests
per hour per subscriber.
Just with 2 requests per hour the size would double to 740*2= aroun 1.5GB in addition to the
disk database and the memory to run the other server SW.

Br
Marco

> -----Original Message-----
> From: ext Yoshihiro Ohba [mailto:[email-masked]]
> Sent: 14. October 2002 22:25
> To: Bernard Aboba
> Cc: Pat Calhoun; Stura Marco (NET/Helsinki); Loughney John
> (NRC/Helsinki); [email-masked]
> Subject: Re: [AAA-WG]: issue: real-time duplicate detection 
> and proposed
> change.
> 
> 
> It seems that there is an assumption that _all_ Diameter clients and
> agents mandatory support 'D' bit set when failover/failback occurs.
> Otherwise, the scheme will not work as is expected when
> failover/failback occurs at some node that does not set 'D' bit while
> the server supports D-bit handling, resulting in duplicated accounting
> request undetected.
> 
> If the assumption is correct, I'm not sure if the scheme is worth
> being defined in the base specification while other accounting
> application does not require this scheme and more generic scheme
> (i.e., hashing) may be able to solve the problem without changing the
> specification.
> 
> Yoshihiro Ohba
> 
> 
> On Mon, Oct 14, 2002 at 09:54:44AM -0700, Bernard Aboba wrote:
> > On 14 Oct 2002, Pat Calhoun wrote:
> > 
> > > But what would happen if two packets were received by a 
> Diameter server
> > > out of order, so the first packet received had the 'D' 
> bit set, while
> > > the second packet did not.
> > >
> > > PatC
> > 
> > That's where delaying by TIME_WAIT comes in, so that the 
> second packet
> > could be processed.
> > 
> > > D=1 records are statistically very few and duplicate search
> > > only for those records do not affect the overall performance
> > > of the system.
> > 
> > Question: so it wouldn't be an issue if some delay were required in
> > processing those records?
> > 
> > > I guess Time_Wait may be couple of seconds, am I right?
> > 
> > It varies between 30 seconds and 2 minutes.
> > 
> > > However proposal #2 could be just to replace the text I proposed
> > > in Annex C that state that D flag may be used by server 
> implementation
> > > and specified in diameter application etc.
> > 
> > If we're only talking about a few paragraphs in Appendix C, then I'd
> > suggest that we should probably try to hammer this out now. 
> However, if
> > using the "D" bit requires more than this, it might need to wait for
> > another application.
> > 
> > 
> > 
> > 
>

---

From: Yoshihiro Ohba <[email-masked]>
Date: Tue, 15 Oct 2002 13:08:22 -0400

Marco,

My point is that even if the discussion based on the performance point
of view is valid, it is weired that all Diameter clients and agents
mandatory support something that Diameter servers may not use.

Instead of defining a new flag in the Diameter header, I think it
would be better to define a new accounting application (i.e., realtime
or debit accounting) with defining a new AVP (with 'M'-bit set) that
is equivalent as D-bit in your proposal.  All Diameter clients,
intermediate agents, and servers that support the accounting
application MUST support the D-bit equivalent AVP.  The new accounting
application document can contain detailed processing rules instead of
describing it in the base specification.

What do you think?

Yoshihiro Ohba

---

From: Pat Calhoun <[email-masked]>
Date: 15 Oct 2002 10:19:09 +0000

If one assumes that a Diameter application is modular and "applications"
don't need to worry about the base (transport) of messages, then I think
that having an application specify the usage of a bit in the message
header is misguided.

PatC

---

From: Peter Deacon <[email-masked]>
Date: Tue, 15 Oct 2002 10:41:52 -0700 (Pacific Daylight Time)

> mandatory support something that Diameter servers may not use.

So anything that is retransmitted due to a lack of an application level
ack/nak gets the 'D' bit set.... If that's all it is ..seems pretty
reasonable to me.

Could make it optional but require the 'D' bit always set for servers not
supporting the hint.

Then folks would just need to keep a short-lived list to dupe check
against to handle the out of order senarios.

Have Fun!
Peter

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 15 Oct 2002 09:57:11 -0700 (PDT)

> Could make it optional but require the 'D' bit always set for servers not

Yes, that's a good point. Do all Diameter implementations need to support
this? I think that's the proposal in Issue 372.

---

From: Pat Calhoun <[email-masked]>
Date: 15 Oct 2002 11:01:46 +0000

optional != useful. I'd argue mandatory on clients, but optionally
supported on servers.

PatC