Subject: [Dime] diameter overload control requirements review

From: Eric McMurry <[email-masked]>
Date: Tue, 30 Oct 2012 16:08:51 -0500

Janet Gunn has been kind enough to provide comments on the diameter overload control requirements.  My responses  are inline.

Thanks Janet!

Begin forwarded message:

> 
> On Oct 29, 2012, at 19:20 , Janet P Gunn <[email-masked]> wrote:
> 
> Eric, here are the rest of my comments.  They are just kind of jotted down rather than being properly organized. 
> 
> 
> Global editorial- change “an network” to “a network” 

fixed.  thanks

> 
> Sec 2.2 4th para. Editorial 
> “If one server notifies 
>    the agent that it has become overloaded, the notification should not 
>    be passed back to the client in a way where the client could 
>    mistakenly perceive the agent itself as being overloaded.” 
> Replace “where” by “that” 
> 

got it.

> Sec 2.3   Interconnect 
> 
> This section appears to talk about “server to server”.  The rest of the document is about client to/from server.  Does it matter? 
> 
> 

the servers on either end are likely proxies that are acting as both clients and servers and this use case is a bit different from the others.  I could use element in place of server, but that would be more inconsistent I think.

> paragraph after fig 7 (and elsewhere) Editorial 
> Replace “different than” with “different from”. 

ack.

> 
> In 
> “which limits how overload and loading information can be 
>    sent.“ 
> change” how” to “which’ (or “how much”) 

good catch.

> In 
>  “…the network operators may not want the 
>    interconnect to use overload or loading information intended to pass 
>    through the interconnect even if the elements in the interconnect 
>    network do support diameter overload control.” 
> Should that be “not intended to pass”? 

how about:

the network operators may not
         want the interconnect to use overload or loading information intended
         to to be conveyed to elements outside of the interconnect network,
         even if the elements in the interconnect network do support diameter
         overload control

is that more clear?

> 
> 3 existing mechanisms 
> In this section you talk about “peers” instead of clients and servers.  Which is it? (to me, clients/servers are not “peers) 

Diameter has applications that are bidirectional, which makes this a bit tricky.  We used the term "peer", which shows up throughout the document, generally to mean another element that is participating in a Diameter connection.  That could be a client or a server (or both) in any given case.

> 
> 4. issues with current mechanisms 
> What is the antecedent for “This” in the second sentence? 

how about this inadequacy instead of just this?
	This inadequacy may, in turn, contribute......

> 
> 5.1 
> Second para Editorial 
> Change 
> “did not have an equivalent load control mechanism which is provided 
>    in the more traditional SS7 elements in GSM” 
> to 
> “did not have an equivalent load control mechanism to the ones provided 
>    in the more traditional SS7 elements in GSM” 

got it.

> 
> third para 
> “Smartphones contribute much more heavily to. ..” 
> More heavily than what? GSM? 

non-smart phones.  clarified that.

> 
> Last set of bullets 
> “   o  first by generating excessive signaling load towards the HLR that 
>       is ten times that from a non-smartphone,” 
> Always/every time? On average? At worst? 

good question.  Ben, do you remember where this came from originally?

> 
> 5.2 
> What about the scenario of the registration storm when a power outage (or node failure ) ends, and the node is trying to get started? 

yes, that is mentioned in the study.  I'll mention it here also.

> 
> Last para 
> “While this study is concerned with…” 
> 
> What is “this study”?  This ID? The technical report mentioned 4 paragraphs earlier?  Need to make  antecedent clear. 
> 
> Sec 6 
> 
> 
> I am having difficulty with Req 8.  There are two quote different things, and they are not directly related. 
> 
> 1-        The mechanism MUST allow nodes to shed load without introducing oscillations. 
> To me, this is covered by Req 7.  “Not inducing oscillations” is one of the things covered by  “stable” .  You could add the reference to oscillations to the text in Req 7 for additional clarity. 
> 
> 2-        Avoiding stale information. 
>  This is important, but it has little, if anything, to do with avoiding oscillations 
>  I’d let it stand alone 
> “nodes MUST be able to 
>             distinguish current overload information from outdated or out of order 
>             information, and to make decisions using the most currently 
>             available information.” 
> 

okay

> I am not sure that I understand what you are looking for in Req 14. 
> 
> “ 
>    REQ 14:  Some scenarios that result in overload involve a rapid 
>             increase of traffic with little time between normal levels 
>             and overload inducing levels.  The mechanism SHOULD provide 
>             for increased feedback when traffic levels increase.  The 
>             mechanism MUST NOT do this in such a way that it increases 
>             the number of messages while at high loads.” 
> 
> If you have a sudden increase in load, you need a very RAPID feedback on the load changes, but I don’t see that you need INCREASED feedback. In fact “increased feedback” , by itself, would seem to violate REQ 13, not introducing additional work. 
> 
> It is fine to take advantage of an increased message rate if it exists.  But I do not think it is a good idea to REQUIRE an increased rate. 
> I also don’t think it Is a good idea to write the requirements so that only solutions that “piggyback” on existing messages can be considered. 

I agree.  13 covers the last part of that req and rapid would be a more precise term.  So it becomes:

Some scenarios that result in overload involve a rapid increase of
          traffic with little time between normal levels and overload inducing
          levels. The mechanism SHOULD provide for rapid feedback when
          traffic levels increase.

Ben, does this sound reasonable to you?

> 
> Req 17 says 
> “    In a mixed environment with nodes that support the overload 
>             control mechanism and that do not, the mechanism MUST NOT 
>             result in less useful throughput than would have resulted if 
>             it were not present.  It SHOULD result in less severe 
>             congestion in this environment.” 
> I find this confusing. 
>  …  less useful throughput than would have resulted if  WHAT were not present? The nonconforming nodes?  The mechanism? 
> … Less severe congestion than WHAT? Without the nonconforming nodes?  Without the mechanism? 
> 
> Perhaps 
> “    In a mixed environment with nodes that support the overload 
>             control mechanism and that do not, the mechanism MUST 
>             result in more  useful throughput than in an environment with no overload control.” 

I can change it to a positive statement and replace the last pronoun.  I think the "or equal to" part of the requirement should not be removed though.  How about:

In a mixed environment with nodes that support the overload control
          mechanism and that do not, the mechanism MUST result in at least as
          much useful throughput as would have resulted if the mechanism were
          not present.

> 
> 
> Req 25 
> “  The mechanism MUST provide a mechanism for indicating load 
>             levels even when not in an overloaded condition, to assist 
>             nodes making decisions to prevent overload conditions from 
>             occurring.” 
> I don’t think that this is needed, given that  Req 22 says that “Overload is not a binary state.” The only difference between “preventing overload conditions form occurring” and “degrees of overload” what you define as overload.  Maybe you should fold some of this text into Req 22 to make it clearer. 

22 covers signaling of overload.  25 is for signaling load, when not overloaded.

> 
> Req 27 
> 
> Add “not throttled” 
>    REQ 27:  The mechanism MUST NOT prevent a node from prioritizing 
>             requests based on any local policy, so that certain requests 
>             are given preferential treatment, given additional 
>             retransmission, not throttled,  or processed ahead of others. 

okay

> 
> 
> Req 34 
> “ The mechanism MUST provide a method for extending the 
>             information communicated and the algorithms used for 
>             overload control.” 
> 
> This is the first mention of such algorithms.  I think you need something earlier about supporting multiple algorithms, before you add a requirement about extending algorithms. 
> 

This probably could use some mention in the front matter.

> There are several requirements in the SIP Overload document that do not seem to have parallels in this document (but I may have missed them).  Most of them seem as if they would be relevant to DIME. 
> 
> SIP  REQ 6:  When overload is signaled by means of a specific message, the 
>       message must clearly indicate that it is being sent because of 
>       overload, as opposed to other, non overload-based failure 
>       conditions.  This requirement is meant to avoid some of the 
>       problems that have arisen from the reuse of the 503 response code 
>       for multiple purposes.  Of course, overload is also signaled by 
>       lack of response to requests.  This requirement applies only to 
>       explicit overload signals. 

REQ 20:  Any explicit overload indication MUST distinguish between
            actual overload, as opposed to other, non-overload related
            failures.

> 
> SIP   REQ 8:  The mechanism shall ensure that, when a request was not 
>       processed successfully due to overload (or failure) of a 
>       downstream element, the request will not be retried on another 
>       element that is also overloaded or whose status is unknown.  This 
>       requirement derives from REQ 1. 

REQ 23:  The mechanism MUST enable a supporting node to minimize the
            chance that retries due to an overloaded or failed node
            result in additional traffic to other overloaded nodes, or
            cause additional nodes to become overloaded.  Moreover, the
            mechanism SHOULD provide unambiguous directions to clients
            on when they should retry a request and when they should not
            considering the various causes of overload such as avalanche
            restart.

> 
> SIP  REQ 9:  That a request has been rejected from an overloaded element 
>       shall not unduly restrict the ability of that request to be 
>       submitted to and processed by an element that is not overloaded. 
>       This requirement derives from REQ 1 

REQ 24:  The mechanism MUST provide sufficient information to enable
            a load balancing node to divert messages that are rejected
            or otherwise throttled by an overloaded upstream node to
            other upstream nodes that are the most likely to have
            sufficient capacity to process them.

> 
> SIP   REQ 10:  The mechanism should support servers that receive requests 
>       from a large number of different upstream elements, where the set 
>       of upstream elements is not enumerable. (May not be relevant to Diameter) 
> 
> SIP   REQ 11:  The mechanism should support servers that receive requests 
>       from a finite set of upstream elements, where the set of upstream 
>       elements is enumerable. 

REQ 11:  The overload mechanism MUST be scalable.  That is, it MUST
            be able to operate in different sized networks.

> 
> SIP    REQ 16:  The mechanism should attempt to minimize the overhead of the 
>       overload control messaging. 

REQ 13:  The mechanism MUST NOT introduce substantial additional work
            for node in an overloaded state.  For example, a requirement
            for an overloaded node to send overload information every
            time it received a new request would introduce substantial
            work.  Existing messaging is likely to have the
            characteristic of increasing as an overload condition
            approaches, allowing for the possibility of increased
            feedback for information piggybacked on it.

> 
> SIP   REQ 23:  It must be possible for the overload mechanism to work in 
>       cases where there is a load balancer in front of a farm of 
>       proxies. 
> 

REQ 24:  The mechanism MUST provide sufficient information to enable
            a load balancing node to divert messages that are rejected
            or otherwise throttled by an overloaded upstream node to
            other upstream nodes that are the most likely to have
            sufficient capacity to process them.

> Janet 
> 
> 
> 
> This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind CSC to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. 
>

---

From: Ben Campbell <[email-masked]>
Date: Tue, 30 Oct 2012 16:19:34 -0500

Hi, replying to the items where you mentioned my name:

On Oct 30, 2012, at 2:50 PM, Eric McMurry <[email-masked]> wrote:

[...]

> good question.  Ben, do you remember where this came from originally?

That came from discussions with Martin, and his presentation in Vancouver. I suspect the 10x part was somewhat rhetorical, so it might make more sense to say "many times" or even "substantially more". IIRC, this referred to devices that hunt between the 3G and 2G mobile networks. While I guess the device doesn't have to be a smart-phone to do that, in practice those were the devices that did. I suspect newer LTE devices that handle voice by switching back to 3G may cause similar issues. 
 
Maybe it would be less hyperbolic to say something to the effect of "newer classes of mobile devices [may/often] create substantially higher signaling loads"?

[...]

> Ben, does this sound reasonable to you?

In principle, yes, although I worry that we will get wrapped around the meaning of rapid. Can we say "sufficiently rapid that [something]"?

Thanks!

Ben.

---

From: Janet P Gunn <[email-masked]>
Date: Tue, 30 Oct 2012 17:31:41 -0400

My further comments in line

In 

how about:

the network operators may not
         want the interconnect to use overload or loading information 
intended
         to to be conveyed to elements outside of the interconnect 
network,
         even if the elements in the interconnect network do support 
diameter
         overload control

is that more clear?

<JPG>  I don't think that helps  It still seems contradictory -  "may not 
want ... to use ...information  intended to be conveyed to elements 
outside..."
It seems that "information  intended to be conveyed to elements outside 
..." is EXACTLY  (and only) what you DO  WANT the interconnect to use.

...

Last para 

What is “this study”?  This ID? The technical report mentioned 4 
paragraphs earlier?  Need to make  antecedent clear. 

...

Req 25 
“  The mechanism MUST provide a mechanism for indicating load 
            levels even when not in an overloaded condition, to assist 
            nodes making decisions to prevent overload conditions from 
            occurring.” 
I don’t think that this is needed, given that  Req 22 says that “Overload 
is not a binary state.” The only difference between “preventing overload 
conditions form occurring” and “degrees of overload” what you define as 
overload.  Maybe you should fold some of this text into Req 22 to make it 
clearer. 

22 covers signaling of overload.  25 is for signaling load, when not 
overloaded.

<JPG> OK, but it seems to me that the distinction should be between
A-   "Indicating that I want the other end to do something to reduce the 
traffic it ends to me " (which could be "approaching overload" or "already 
in overload)
and
B- "Indicating that I support Overload control, but I don't need the other 
end to do anything about it at the moment."

rather than between  "load when overloaded " and "load when not 
overloaded"

Furthermore, I do not think there is a lot of value (for overload control) 
in "indicating load levels". (It might be valuable for load-balancing.) 
For overload control, the value is in telling the other end that they need 
to reduce their traffic to me (and how much).

...

There are several requirements in the SIP Overload document that do not 
seem to have parallels in this document (but I may have missed them). Most 
of them seem as if they would be relevant to DIME. 

OK on most of them- my brain was going in circles trying to go back and 
forth between the docs.

...

SIP   REQ 10:  The mechanism should support servers that receive requests 
      from a large number of different upstream elements, where the set 
      of upstream elements is not enumerable. (May not be relevant to 
Diameter) 

<JPG> The distinction  here is not the SIZE of the network.  It is whether 
you know, ahead of time, "who" your upstream (downstream for Diameter) 
nodes are.  I think this relates to your "Peer discovery" mechanism.  So I 
think it is covered by your REQ 5

SIP    REQ 16:  The mechanism should attempt to minimize the overhead of 
the 
      overload control messaging. 

<JPG> I guess I see a distinction between  "minimizing the overhead" and 
"NOT introducing substantial overhead", and I like "minimizing" better. 
But it is not a big deal.
Janet

---

From: Eric McMurry <[email-masked]>
Date: Tue, 30 Oct 2012 18:19:39 -0500

Thanks Janet.  comments inline.

On Oct 30, 2012, at 16:31 , Janet P Gunn <[email-masked]> wrote:

> <JPG>  I don't think that helps  It still seems contradictory -  "may not want ... to use ...information  intended to be conveyed to elements outside..." 
> It seems that "information  intended to be conveyed to elements outside ..." is EXACTLY  (and only) what you DO  WANT the interconnect to use. 

<em> well, information, and overload or loading information are not the same thing.  Taking the qualifiers off changes the meaning.  Even so, the text is apparently still not working.  How about:

the network operators may not 
         want the interconnect to use or act on overload or loading information intended 
         for consumption by elements outside of the interconnect network, 
         even if the elements in the interconnect network do support diameter 
         overload control 

> A-   "Indicating that I want the other end to do something to reduce the traffic it ends to me " (which could be "approaching overload" or "already in overload) 
> and 
> B- "Indicating that I support Overload control, but I don't need the other end to do anything about it at the moment." 
> 
> rather than between  "load when overloaded " and "load when not overloaded" 
> 
> Furthermore, I do not think there is a lot of value (for overload control) in "indicating load levels". (It might be valuable for load-balancing.) For overload control, the value is in telling the other end that they need to reduce their traffic to me (and how much). 

<em>  one of the primary goals of this is to give diameter nodes the ability to avoid overload when possible.  That can not be accomplished by signaling the degree of overload once it has already occurred.  Signaling the degree of overload once it has occurred does help to mitigate the results of the overload though, which is another goal.  The thinking was that these goals result in separate requirements and are not really the same thing.  That said, there is nothing stopping a mechanism from signaling the degree of overload and non-overloaded load level in the same way, depending on how the mechanism chooses to deal with actual overload.

> OK on most of them- my brain was going in circles trying to go back and forth between the docs.

<em>  I fully understand.  I had to go back and dig a bit to refresh my memory on those :-)

> <JPG> The distinction  here is not the SIZE of the network.  It is whether you know, ahead of time, "who" your upstream (downstream for Diameter)  nodes are.  I think this relates to your "Peer discovery" mechanism.  So I think it is covered by your REQ 5 
> 
> 
> 
> <JPG> I guess I see a distinction between  "minimizing the overhead" and "NOT introducing substantial overhead", and I like "minimizing" better.  But it is not a big deal. 

<em> I generally agree with you and think minimizing is a good goal, but I'm thinking not introducing substantial overhead is a bit easier to test as a requirement.