Subject: [AAA-WG]: Issue 372, where we stand

From: <[email-masked]>
Date: Mon, 21 Oct 2002 10:21:02 +0300

Hi all,

After some mail exchanges, I've adjusted the text to state the following.
Please let me know if this is OK, as I would like to get the draft submitted.

thanks,
John

Change chapter 3 as follow

3 Diameter Header

A summary of the Diameter header format is shown below. The fields are transmitted in network byte order.

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |      Ver      |                 Message Length                |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |R P E T r r r r|                  Command-Code                 |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                         Application-ID                        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                      Hop-by-Hop Identifier                    |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                      End-to-End Identifier                    |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |  AVPs ...
 +-+-+-+-+-+-+-+-+-+-+-+-+-

Version
This Version field MUST be set to 1 to indicate Diameter Version 1.

Message Length
The Message Length field is three octets and indicates the length of the Diameter message including the header fields.

Command Flags
The Command Flags field is eight bits. The following bits are assigned:

R(equest) - If set, the message is a request. If cleared, the message is an answer.

P(roxiable) - If set, the message MAY be proxied, relayed or redirected. If cleared, the message MUST be locally processed.

E(rror) - If set, the message contains a protocol error, and the message will not conform to the ABNF described for this command. Messages with the 'E' bit set are commonly referred to as error messages. This bit MUST NOT be set in request messages. See section 7.2.

T(Potentialy re-transmitted message)
This flag is defined only for request messages sent by Diameter clients or agents. This flag is used as an indication of an application layer retransmission event, e.g. due to failover to an alternate server.

If a Diameter client or agent knows that it is sending this request or accounting record contained in the request for the first time, it MUST reset this flag. Diameter agents only need to be concerned about the number of requests they send based on a single received request; retransmissions by other entities need not be tracked. However, Diameter agents that receive a request with the T flag set, MUST keep the T flag set in the forwarded request.

If request is either known to be a retransmission or the Diameter client or agent is unable to assure that it is the first such request, it MUST set this flag. For instance, after a reboot, a client may not know whether it has already tried to send the accounting records in its non-volatile memory before the reboot occurred.

Diameter servers MAY use the T flag as an aid when processing requests and detecting duplicate messages. However, servers that do this MUST ensure that duplicates are found even when the first transmitted request arrives at the server after the retransmitted request.

This flag MUST NOT be set if an error answer message (e.g. a protocol error) has been received for an earlier message. It can be used only in cases where no answer has been received from the Server for a request and the request is sent again, (e.g. due to a failover to an alternate peer, due to a recovered primary peer or due to a client re-sending a stored record from non-volatile memory such as after reboot of a client or agent).This flag MUST NOT be set in answer messages.

r(eserved) - these flag bits are reserved for future use, and MUST be set to zero, otherwise an error MUST be sent to the sender.

5.5.4 Failover and Failback Procedures

Change the paragraph

When a transport failure is detected, all messages in the queue are
sent to an alternate agent, if possible.
.....etc..

to

When a transport failure is detected, if possible all messages in the queue are sent to an alternate agent with the T flag set. On booting a Diameter client or agent, the T flag is also set on any records still remaining to be transmitted in non-volatile storage.

Change appendix C as follows:

Appendix C. Duplicate Detection

As described in section 9.4, accounting record duplicate detection is based on session identifiers. Duplicates can appear for various reasons:

- Failover to an alternate server. Where close to real-time performance is required, failover thresholds need to be kept low and this may lead to an increased likelihood of duplicates. Failover can occur at the client or within Diameter agents.

- Failure of a client or agent after sending of a record from non-volatile memory, but prior to receipt of an application layer ACK and deletion of the record to be sent. This will result in retransmission of the record soon after the client or agent has rebooted.

 - Duplicates received from RADIUS gateways. Since the retransmission behavior of RADIUS is not defined within [RFC2865], the likelihood of duplication will vary according to the implementation.

- Implementation problems and misconfiguration.

In some cases the Diameter accounting server can delay the duplicate detection and accounting record processing until a post-processing phase takes place. At that time records are likely to be sorted according to the included User-Name and duplicate elimination is easy in this case.

In other situations it may be necessary to perform real-time duplicate detection, such as when credit limits are imposed or real-time fraud detection is desired.

In general, only generation of duplicates due to failover or re-sending of records in non-volatile storage can be reliably detected by Diameter clients or agents. In such cases the Diameter client or agents can mark the message as possible duplicate by setting the T flag. Since the Diameter server is responsible for duplicate detection, it can choose to make use of the T flag or not, in order to optimize duplicate detection. Since the T flag does not affect interoperability, and may not be needed by some servers, generation of the T flag is REQUIRED for Diameter clients and agents, but MAY be implemented by Diameter servers.

As an example, it can be usually be assumed that duplicates appear within a time window of longest recorded network partition or device fault, perhaps a day. So only records within this time window need to be looked at in the backward direction. Secondly, hashing techniques or other schemes, such as the use of the T flag in the received messages, may be used to eliminate the need to do a full search even in this set except for rare cases.

The following is an example of how the T flag may be used by the server to detect duplicate requests.

A Diameter server MAY check the T flag of the received message to determine if the record is a possible duplicate. If the T flag is set in the request message, the server searches for a duplicate within a configurable duplication time window backward and forward. This limits database searching to those records where the T flag is set.  In a well-run network, network partitions and device faults will presumably be rare events, so this approach represents a substantial optimization of the duplicate detection process. During failover, it is possible for the original record to be received after the T flag marked record, due to differences in network delays experienced along the path by the original and duplicate transmissions. The likelihood of this occurring increase as the failover interval is decreased. In order to be able to detect out of order duplicates, the Diameter server should use backward and forward time windows when performing duplicate checking for the T flag marke!
 d request. For example, in order to allow time for the original record to exit the network and be recorded by the accounting server, the Diameter server can delay processing records with the T flag set until a time period TIME_WAIT + RECORD_PROCESSING_TIME has elapsed after the closing of the original transport connection.  After this time period has expired, then it may check the T flag marked records against the database with relative assurance that the original records, if sent, have been received and recorded.

---

From: <[email-masked]>
Date: Mon, 21 Oct 2002 10:58:10 +0300

Hi,

sounds OK.

However,  I think it is still better to change the sentence of T-flag description as suggested by Glen Zorn to avoid
wrong interpretation.

" This flag MUST NOT be set if the error answer message (e.g. a protocol error) has been received for the earlier message."

Glen Zorn wrote:

> 	This flag MUST NOT be set if an error answer message
> 	(about e.g. a protocol error) has been received for an
> 	previous message.

>Change "an" to "the", I think; otherwise, this could be interpreted as "if a
>protocol error ever happens, never set the 'T' bit again"

Br
Marco

> -----Original Message-----
> From: ext [mailto:[email-masked]]
> Sent: 21. October 2002 10:21
> To: [email-masked]
> Subject: [AAA-WG]: Issue 372, where we stand
> 
> 
> Hi all,
> 
> A summary of the Diameter header format is shown below. The 
> fields are transmitted in network byte order.
> 
> 
>  
>   0                   1                   2                   3
> The Message Length field is three octets and indicates the 
> length of the Diameter message including the header fields.
> 
> Command Flags
> P(roxiable) - If set, the message MAY be proxied, relayed or 
> redirected. If cleared, the message MUST be locally processed.
> 
> E(rror) - If set, the message contains a protocol error, and 
> the message will not conform to the ABNF described for this 
> command. Messages with the 'E' bit set are commonly referred 
> to as error messages. This bit MUST NOT be set in request 
> messages. See section 7.2.
> 
> This flag is defined only for request messages sent by 
> Diameter clients or agents. This flag is used as an 
> indication of an application layer retransmission event, e.g. 
> due to failover to an alternate server.
> 
> If a Diameter client or agent knows that it is sending this 
> request or accounting record contained in the request for the 
> first time, it MUST reset this flag. Diameter agents only 
> need to be concerned about the number of requests they send 
> based on a single received request; retransmissions by other 
> entities need not be tracked. However, Diameter agents that 
> receive a request with the T flag set, MUST keep the T flag 
> set in the forwarded request.
> 
> If request is either known to be a retransmission or the 
> Diameter client or agent is unable to assure that it is the 
> first such request, it MUST set this flag. For instance, 
> after a reboot, a client may not know whether it has already 
> tried to send the accounting records in its non-volatile 
> memory before the reboot occurred.
> 
> Diameter servers MAY use the T flag as an aid when processing 
> requests and detecting duplicate messages. However, servers 
> that do this MUST ensure that duplicates are found even when 
> the first transmitted request arrives at the server after the 
> retransmitted request.
> 
> This flag MUST NOT be set if an error answer message (e.g. a 
> protocol error) has been received for an earlier message. It 
> can be used only in cases where no answer has been received 
> from the Server for a request and the request is sent again, 
> (e.g. due to a failover to an alternate peer, due to a 
> recovered primary peer or due to a client re-sending a stored 
> record from non-volatile memory such as after reboot of a 
> client or agent).This flag MUST NOT be set in answer messages.
> 
> r(eserved) - these flag bits are reserved for future use, and 
> MUST be set to zero, otherwise an error MUST be sent to the sender.
> 
> When a transport failure is detected, if possible all 
> messages in the queue are sent to an alternate agent with the 
> T flag set. On booting a Diameter client or agent, the T flag 
> is also set on any records still remaining to be transmitted 
> in non-volatile storage.
> 
> 
> As described in section 9.4, accounting record duplicate 
> detection is based on session identifiers. Duplicates can 
> appear for various reasons:
> 
> - Failover to an alternate server. Where close to real-time 
> performance is required, failover thresholds need to be kept 
> low and this may lead to an increased likelihood of 
> duplicates. Failover can occur at the client or within 
> Diameter agents.
> 
> - Failure of a client or agent after sending of a record from 
> non-volatile memory, but prior to receipt of an application 
> layer ACK and deletion of the record to be sent. This will 
> result in retransmission of the record soon after the client 
> or agent has rebooted.
> 
>  - Duplicates received from RADIUS gateways. Since the 
> retransmission behavior of RADIUS is not defined within 
> [RFC2865], the likelihood of duplication will vary according 
> to the implementation.
> 
> In some cases the Diameter accounting server can delay the 
> duplicate detection and accounting record processing until a 
> post-processing phase takes place. At that time records are 
> likely to be sorted according to the included User-Name and 
> duplicate elimination is easy in this case.
> 
> In other situations it may be necessary to perform real-time 
> duplicate detection, such as when credit limits are imposed 
> or real-time fraud detection is desired.
> 
> In general, only generation of duplicates due to failover or 
> re-sending of records in non-volatile storage can be reliably 
> detected by Diameter clients or agents. In such cases the 
> Diameter client or agents can mark the message as possible 
> duplicate by setting the T flag. Since the Diameter server is 
> responsible for duplicate detection, it can choose to make 
> use of the T flag or not, in order to optimize duplicate 
> detection. Since the T flag does not affect interoperability, 
> and may not be needed by some servers, generation of the T 
> flag is REQUIRED for Diameter clients and agents, but MAY be 
> implemented by Diameter servers.
> 
> As an example, it can be usually be assumed that duplicates 
> appear within a time window of longest recorded network 
> partition or device fault, perhaps a day. So only records 
> within this time window need to be looked at in the backward 
> direction. Secondly, hashing techniques or other schemes, 
> such as the use of the T flag in the received messages, may 
> be used to eliminate the need to do a full search even in 
> this set except for rare cases.
> 
> A Diameter server MAY check the T flag of the received 
> message to determine if the record is a possible duplicate. 
> If the T flag is set in the request message, the server 
> searches for a duplicate within a configurable duplication 
> time window backward and forward. This limits database 
> searching to those records where the T flag is set.  In a 
> well-run network, network partitions and device faults will 
> presumably be rare events, so this approach represents a 
> substantial optimization of the duplicate detection process. 
> During failover, it is possible for the original record to be 
> received after the T flag marked record, due to differences 
> in network delays experienced along the path by the original 
> and duplicate transmissions. The likelihood of this occurring 
> increase as the failover interval is decreased. In order to 
> be able to detect out of order duplicates, the Diameter 
> server should use backward and forward time windows when 
> performing duplicate checking for the T flag marke!
>  d request. For example, in order to allow time for the 
> original record to exit the network and be recorded by the 
> accounting server, the Diameter server can delay processing 
> records with the T flag set until a time period TIME_WAIT + 
> RECORD_PROCESSING_TIME has elapsed after the closing of the 
> original transport connection.  After this time period has 
> expired, then it may check the T flag marked records against 
> the database with relative assurance that the original 
> records, if sent, have been received and recorded.
>

---

From: <[email-masked]>
Date: Mon, 21 Oct 2002 17:11:15 +0300

> Actually, what I think was intended was:
> 
> (e.g. a protocol error) has been received for the earlier
> message.

Got it.

John