Subject: [dispatch] SIP load balancing

From: "Vijay K. Gurbani" <[email-masked]>
Date: Mon, 07 Mar 2011 16:26:38 -0600

Hello: The work going on in the SOC working group has highlighted
the need for some guidelines or strategies for SIP load balancing.

RjS had started some preliminary work on this way back in 2003 [1],
but that effort seems not to have proceeded further.  Furthermore,
our collective knowledge on SIP has increased since 2003 such that
some of the mechanisms discussed in the early work may no longer be
directly applicable (e.g., use of 503 and resulting problems with
overload, modifying route-sets may impinge on signing the route set
[2], etc.)

My co-author (Thierry Bessis) and I have put together a short draft
on load balancing that I have just submitted to the IETF
archives [3].  It will be good to have some folks take a look at it
and suggest improvements or to point out any shortcomings, of which
I am sure there will be many.

If there is interest, we can discuss this in the dispatch working
group in Prague.

[1] http://tools.ietf.org/html/draft-sparks-sipping-load-00
[2] http://tools.ietf.org/html/rfc5658
[3] 
http://tools.ietf.org/html/draft-bessis-dispatch-adaptive-load-balancing-00

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{bell-labs.com,acm.org} / [email-masked]
Web:   http://ect.bell-labs.com/who/vkg/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 07 Mar 2011 18:14:09 -0500

Vijay,

IIUC, the algorithm you have proposed is similar to the one Paul Jones 
has proposed - with the main difference being the numeric range of the 
weight. Do I have that right?

It isn't clear to me from your draft if you are specifying the algorithm 
by which the relative weights are used to determine the probability of a 
request going to one server or another. ISTM that it *does* matter. And 
it especially matters if there is an implication that the upstream 
node's algorithm may depend on some assumption or knowledge about the 
downstream nodes.

And of course this method of reporting weights isn't sufficient to drive 
a balancing algorithm that is sensitive to some requests requiring more 
or different resources than others. (That seems to be a real problem, 
though I don't know if we have a real solution for it.)

	Thanks,
	Paul

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Mon, 07 Mar 2011 17:37:43 -0600

Paul: Thank you for your input.  More inline.

On 03/07/2011 05:14 PM, Paul Kyzivat wrote:
> Vijay,
> IIUC, the algorithm you have proposed is similar to the one Paul
> Jones has proposed - with the main difference being the numeric range
> of the weight. Do I have that right?

I believe that Paul J.'s algorithm is more suited towards overload
control, not load balancing (and I will await Paul J.'s correction to
my assertion).

The intent in draft-bessis-dispatch-adaptive-load-balancing is to
*always* report the current load to give the upstream SIP proxy
an indication of the health of the SIP server in the cluster.  If one
waits until overload is imminent, then it may be too late since a sudden
and unanticipated burst of messages will put the SIP server in
overload rather quickly.

> It isn't clear to me from your draft if you are specifying the
> algorithm by which the relative weights are used to determine the
> probability of a request going to one server or another.

Yes, same way as RFC2782 does it.

> ISTM that it *does* matter. And it especially matters if there is an
> implication that the upstream node's algorithm may depend on some
> assumption or knowledge about the downstream nodes.

No knowledge, just a reported number that the upstream SIP proxy can
treat as a RFC2782-like weight indicator.  Today, rfc3263-resolution
starts off with the SRV indicators in RFC2782, without any regard
to additional knowledge or assumption about downstream nodes.
dispatch-adaptive-load-balancing is trying to use the same model,
except render the weight to be sensitive to the current traffic
in the cluster in real-time.

> And of course this method of reporting weights isn't sufficient to
> drive a balancing algorithm that is sensitive to some requests
> requiring more or different resources than others. (That seems to be
> a real problem, though I don't know if we have a real solution for
> it.)

Yes, that is a real problem, as is the problem of a SIP server in a
cluster that has a low weight and is thus not contacted with enough
frequency to merit more participation to the goodput.

It is well known in SIP that some methods are more onerous to process
than others --- from a proxying point of view, processing an INVITE
with forking and response aggregation is far more detrimental to
resource usage than is processing a BYE.  Some existing literature
techniques thus add a coefficient to the weight of sending an INVITE
versus a BYE.  At this time, the process outlined in 
dispatch-adaptive-load-balancing treats all requests the same, but we 
can definitely
look at ways to change this behaviour.

Thanks,

- vijay

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 8 Mar 2011 00:05:13 -0500

Vijay,

The Session Capacity Estimate (SCE) idea is actually quite similar to the
"adaptive load balancing" draft, from what I can see.  SCE was proposed as a
solution to the "overload" problem, the approach being one where we try to
avoid overload by selecting a device that reports a higher capacity value.
Thus, one could view it as "overload control through load balancing".  We
didn't explicitly call it load balancing, though, because we were not trying
to balance load (i.e., place the same load on all servers).  Rather, we just
wanted to direct traffic to devices that we knew had available capacity.

In practice, the way SCE would be employed is the following:
* SIP proxy, B2BUA, etc. (selector) would receive SCE capacity from
   peer devices
 * The selector would direct traffic toward the peer devices in a
   round-robin fashion, taking into consideration the SCE value so
   as to prevent exhausting resources on any given peer device
 * Once the peer's SCE value dropped below a given "low-water mark",
   the peer device would be less favored and not given traffic until
   the SCE value increases above the low-water mark (OPTIONS "ping"
   messages may be used to determine the SCE value in the event that
   there is no other traffic)
 * The current SOC overload control mechanism would be used to
   to prevent message overloading, as SCE only focuses on resource
   overloading (e.g., memory, CPU, disk space, DSPs)

Note that the "round-robin" approach may or may not be preferred and is not
required in our draft.  I've found this approach works pretty well, but one
may prefer to use a particular weighting that is more complex.  I don't
think local decisions in the box need to be subject to standardization: we
just need to standardize the signaling to enable such decisions.

In any case, it certainly looks like we're thinking along the same lines.

Paul

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Vijay K. Gurbani
> Sent: Monday, March 07, 2011 6:38 PM
> To: [email-masked]
> Subject: Re: [dispatch] SIP load balancing
> 
> control, not load balancing (and I will await Paul J.'s correction to my
> assertion).
> 
> *always* report the current load to give the upstream SIP proxy an
> indication of the health of the SIP server in the cluster.  If one waits
> until overload is imminent, then it may be too late since a sudden and
> unanticipated burst of messages will put the SIP server in overload
> rather quickly.
> 
> starts off with the SRV indicators in RFC2782, without any regard to
> additional knowledge or assumption about downstream nodes.
> dispatch-adaptive-load-balancing is trying to use the same model, except
> render the weight to be sensitive to the current traffic in the cluster
> in real-time.
> 
> > requiring more or different resources than others. (That seems to be a
> > real problem, though I don't know if we have a real solution for
> > it.)
> 
> than others --- from a proxying point of view, processing an INVITE with
> forking and response aggregation is far more detrimental to resource
> usage than is processing a BYE.  Some existing literature techniques
> thus add a coefficient to the weight of sending an INVITE versus a BYE.
> At this time, the process outlined in dispatch-adaptive-load-balancing
> treats all requests the same, but we can definitely look at ways to
> change this behaviour.
> 
> Thanks,
> 
> - vijay
> --
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Tue, 8 Mar 2011 15:22:09 +0530

Vijay,

As Paul K & Paul J mentioned, I'm looking at your proposal with lot of
similarity to SCE. One more key stuff, I could not understand the need
for two separate mechanism for

1) Load balancing ("weight")
2) Overload control ("oc")

In case feedback mechanism to the upstream is generic enough, the load
balancer will be able to balance based on value (capacity) received from
downstream and reduce the load drastically or stop in case the value is
too low in terms of capacity. Even though any overload control algorithm
is used ("oc" or "weight" or "SCE" or "resource-availability" other),
the locally overload control mechanism is required in downstream as per
mentioned in draft-ietf-soc-overload-design. I wish that we will look
into these aspect because overload control is the extreme case for load
balancing the cluster of downstream entity and load balancing mechanism
shall help to avoid the overload control based on feedback.

I guess that your mechanism assumes about downstream SIP UA as signaling
entity only and does not handle media. Please correct me in case I
wrongly understand. draft-partha-dispatch-resource-availability-00 was
written to handle both resource monitoring and load balancing of SIP
based media servers wherein load balancing has to take care of overload
as well.

I'm also interested in discussing SIP load balancing topic during
IETF-80 meeting.

Thanks
Partha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Paul E. Jones
Sent: Tuesday, March 08, 2011 10:35 AM
To: 'Vijay K. Gurbani'; [email-masked]
Subject: Re: [dispatch] SIP load balancing

Vijay,

The Session Capacity Estimate (SCE) idea is actually quite similar to
the "adaptive load balancing" draft, from what I can see.  SCE was
proposed as a solution to the "overload" problem, the approach being one
where we try to avoid overload by selecting a device that reports a
higher capacity value.
Thus, one could view it as "overload control through load balancing".
We didn't explicitly call it load balancing, though, because we were not
trying to balance load (i.e., place the same load on all servers).
Rather, we just wanted to direct traffic to devices that we knew had
available capacity.

Note that the "round-robin" approach may or may not be preferred and is
not required in our draft.  I've found this approach works pretty well,
but one may prefer to use a particular weighting that is more complex.
I don't think local decisions in the box need to be subject to
standardization: we just need to standardize the signaling to enable
such decisions.

In any case, it certainly looks like we're thinking along the same
lines.

Paul

> > Jones has proposed - with the main difference being the numeric 
> > range of the weight. Do I have that right?
> 
> indication of the health of the SIP server in the cluster.  If one 
> waits until overload is imminent, then it may be too late since a 
> sudden and unanticipated burst of messages will put the SIP server in 
> overload rather quickly.
> 

> except render the weight to be sensitive to the current traffic in the

> cluster in real-time.

> techniques thus add a coefficient to the weight of sending an INVITE
versus a BYE.

> Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent 1960 Lucent Lane, 
> Rm. 9C-533, Naperville, Illinois 60566 (USA)

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 08 Mar 2011 08:49:45 -0500

Vijay,

Thanks for clarifications. Now I think I understand what you are 
proposing. It is certainly simple, and better than nothing.

It *does* require that all the members of the load balancing set assign 
their own weights in a consistent way. This of course is no different 
than when the weights are assigned in DNS, except that its likely to be 
one person who puts all the weights into DNS, while the weights coming 
from the servers can be influenced by who implemented them. If they 
aren't all implemented consistently it can be a problem.

But this is a second order problem, and is probably unavoidable. (AFAIK 
every scheme for this space has a similar constraint.) It might need to 
be the subject of additional standardization/profiling, though it might 
not be ietf work.

	Thanks,
	Paul

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Tue, 8 Mar 2011 08:20:13 -0600

Is there a need to inter-twine media with signaling?

I think if there is an assumption that the media and signaling take the
same path, 
then you've lost some flexibility, and worse make sub-optimal choices
when 
one or the other is over-capacity and the other is not even in the
picture.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Parthasarathi R (partr)
Sent: Tuesday, March 08, 2011 4:52 AM
To: Paul E. Jones; Vijay K. Gurbani; [email-masked]
Subject: Re: [dispatch] SIP load balancing

Vijay,

Thanks
Partha

-----Original Message-----

Vijay,

Paul

versus a BYE.

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Tue, 8 Mar 2011 20:01:53 +0530

Mike,

The mechanism is required for the SIP based media servers to load
balancer. Media servers may be based on IETF mediactrl sort of
interfaces or SIP devices like SBC, PSTN GW. As you mentioned, the
problem in SIP based media server is complex as more resources involved
at the given time.  

Thanks
Partha

-----Original Message-----
From: Mike Hammer (hmmr) 
Sent: Tuesday, March 08, 2011 7:50 PM
To: Parthasarathi R (partr); Paul E. Jones; Vijay K. Gurbani;
[email-masked]
Subject: RE: [dispatch] SIP load balancing

I think if there is an assumption that the media and signaling take the
same path, then you've lost some flexibility, and worse make sub-optimal
choices when one or the other is over-capacity and the other is not even
in the picture.

Mike

-----Original Message-----

Vijay,

Thanks
Partha

-----Original Message-----

Vijay,

Paul

versus a BYE.

---

From: Simon Pietro Romano <[email-masked]>
Date: Tue, 08 Mar 2011 15:40:29 +0100

Hi Vijay,

interesting draft, indeed.
It brought to my mind that some of the ideas you are proposing can be 
found in Mobicent's implementation of Distributed Load Balancing, which 
makes use of 'Via' and 'Record-Route' headers to keep the Load Balancer 
in the loop, and exploits (in place of your piggybacking proposal) a 
proprietary 'heartbit' protocol (implemented with Java RMI) to report 
health status from SIP Servers to the SIP Proxy. Interestingly enough, 
inside Mobicents the balancer algorithms are 'pluggable': the original 
implementation is shipped with three different built-in algorithms, but 
nothing prevents implementers from proposing new ones.

As a general comment, I think that solutions like the one you are 
proposing have strong liaisons, besides Overload Control, with the 
broader issue of (transparent) fail-over management.

My 2 cents,

Simon

-- 
                             _\\|//_
                             ( O-O )
    ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~
                     Simon Pietro Romano
               Universita' di Napoli Federico II
                  Computer Science Department
         Phone: +39 081 7683823 -- Fax: +39 081 7684219
                 e-mail: [email-masked]
           http://www.comics.unina.it/simonpietro.romano

     <<Molti mi dicono che lo scoraggiamento è l'alibi degli
    idioti. Ci rifletto un istante; e mi scoraggio>>. Magritte.
                          oooO
    ~~~~~~~~~~~~~~~~~~~~~~(   )~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~
                           \ (    (   )
                            \_)    ) /
                                  (_/

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 08 Mar 2011 11:34:49 -0600

Paul,

Please let me address some of your comments.
I don't know what the SCE is, so please pardon me if I misunderstand 
something.

> wanted to direct traffic to devices that we knew had available capacity.
Agreed, the goal seems to be the same: avoiding or at least delaying 
overload.
And your other comment too: our proposal does try to balance the load on 
all the servers. The rational is that it is very easy for each server to 
know where it is in terms of resource usage (for instance: I'm 60 
percent resource loaded at this moment), and this is just the 
information you need to balance the traffic. Now it seems (but it is not 
clear, see below)  that the SCE "available capacity" is actually a real 
time feedback, so the information would be very similar actually. 
(What's the difference ? )
>     as to prevent exhausting resources on any given peer device
I see a contradiction here: if it is round-robin, it does not take into 
account the SCE value.
Is that something more like a weighted round-robin algorithm ?
>     there is no other traffic)
Why would we wait for a lower mark before to influence the distribution 
? Also is that less favored, or given no traffic ?
The later has stability issues.  And why do we need ping Options when 
there is no traffic ?
  If there is no traffic (or very little), should we care about optimal 
distribution from this source ?
>     overloading (e.g., memory, CPU, disk space, DSPs)
I don't see this difference here (message overload versus resource 
overload ? ) . To me, SOC focuses on source shedding, while SCE seems to 
focus on improved load distribution, just like our proposal.
> just need to standardize the signaling to enable such decisions.
I believe the signaling is not enough. The deep meaning of the feedback 
parameter needs to be clarified too.
For example, in our proposal, the feedback is simply the current 
utilization rate of the node (very simple).

Also the expectation needs to be clarified because it has a very 
important impact on the recommended behavior in case of overload.
The solution that we propose constantly tries to balance load in the 
cluster of servers. Therefore, in case the SOC shedding kicks in, there 
is NOT NEED to repeat the message elsewhere in the cluster because it is 
balanced already (So no other server has more capacity). This is a huge 
advantage for the network stability as message repeat tends to spread 
the overload and collapse the cluster.
> In any case, it certainly looks like we're thinking along the same lines.
Yes I agree.

Cordially,

Thierry

-- 
-- 
Cordially,
Thierry Bessis
IMS Solution: Architecture & Product Management - ALTA Member
   Organization: ALU > Carrier group > Wireline BU > **IMS Sol Arch, SE, PdM
Alcatel-Lucent
1960 Lucent Lane - Room 6L-517
   Naperville, IL  60563  U S
   Tel/Fax: +1 630 979 7989
   Corporate IM: tbessis    - Yahoo IM:     thierry_bessis

My WEB page:   http://ihgpweb.ih.lucent.com/~tbessis/
My Calendar:   www.google.com/calendar/embed?src=new_frs3175%40yahoo.fr
ALTA Hot Line: http://alta.all.alcatel-lucent.com/hotline

Conference information:
2801 2801 (US):+1 800 771 8734 - (F):+33 800 941 674 - Access Code: 9797989
others countries see: http://all.alcatel-lucent.com/audio

Upcoming planned Business Trip: 12 April - 14 April 2011
Upcoming planned Vacation: None

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 08 Mar 2011 12:17:10 -0600

Part,

On 08-Mar-2011 03:52, Parthasarathi R (partr) wrote:
> Vijay,
>
> As Paul K&  Paul J mentioned, I'm looking at your proposal with lot of
See my previous Email:
OC is a request to shed: Whatever you did before is not enough: you need 
to shed now.
Load Balancing is a mean to constantly balance the load, which is an 
optimized way to prevent localized overload (but won't prevent the 
cluster overload of course !).

However, the same transport mechanism can certainly be used and both 
mechanisms should work very well together,
> too low in terms of capacity.
You may decide shed from the load feedback value, but this complicated 
to setup: we did that before inside our IMS core product, but we decided 
that having a load indication separated from the shedding request is 
much easier and more efficient.
The main issue is to agree about the threshold: which load threshold do 
you use to decide to shed ? and how much to shed based on the load ? 
Some servers will want you to shed before others for the same indicated 
load and/or more (or less) than others.
The rejection rate allows each server to request exactly the source 
shedding that it thinks is needed: this is actually very powerful.
> Even though any overload control algorithm
> mentioned in draft-ietf-soc-overload-design.
Right. A difficulty is when a downstream server will have both oc 
compliant clients, and non oc-compliant. Obviously the local OLC needs 
to take into account that, otherwise "non compliant" will get a huge and 
unfair advantage (one stage OLC instead of two).
In our implementation, until we reach a critical level, we do NOT apply 
the local overload to olc compliant nodes (we trust them). we only apply 
it to non compliant.
> I wish that we will look
> entity only and does not handle media.
Yes this is correct. Although we plan to use the same thing in our MGCF 
which has media too (i.e. the resource feedback will ALSO take into 
account the available media count).

Cordially,

Thierry
> Please correct me in case I

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 08 Mar 2011 12:37:22 -0600

Paul,

> proposing. It is certainly simple, and better than nothing.
I think it is much better than nothing  ! :-)
It is true that some transactions use much more resources than others, 
but this does not damage the load balancing:
If one server is unlucky enough to get much more "loaded transactions" 
than others, it will quickly report a higher resource usage and will 
consequently receive less requests than others: despite the fact that 
one server get more of such transactions, the cluster will still be 
evenly loaded.

>
> It *does* require that all the members of the load balancing set 
> assign their own weights in a consistent way. This of course is no 
> different than when the weights are assigned in DNS, except that its 
> likely to be one person who puts all the weights into DNS, while the 
> weights coming from the servers can be influenced by who implemented 
> them. If they aren't all implemented consistently it can be a problem.
This is true, but the mechanism is still robust. In practice, the most 
important is that all use the same range. But the system is robust to 
things like linearity issues: as long as the feedback reported load 
curve is a monotonic function of the actual load, it works.

Cordially,

Thierry
>
> But this is a second order problem, and is probably unavoidable. 
> (AFAIK every scheme for this space has a similar constraint.) It might 
> need to be the subject of additional standardization/profiling, though 
> it might not be ietf work.
>
>     Thanks,
>     Paul
>

Conference information:

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 08 Mar 2011 14:05:02 -0600

> proposing.  It is certainly simple, and better than nothing.

Paul: Thanks for taking the time to tease out the larger picture.

More inline.

> them. If they aren't all implemented consistently it can be a

While I do not claim that the "weight" scheme is perfect, I think the
bounds for ambiguity are much less with this approach.  All the
upstream SIP proxy needs is a "weight" indicator that it will
use in a RFC2782-type algorithm to proportionally send as much traffic
to a downstream SIP server as it has indicated (i.e., higher "weight"
values will get proportionally more traffic).  This is much better
than round-robin, and furthermore, there is no requirement that each
downstream SIP server in the cluster set its "weight" in a manner
consistent with other servers, or even that the "weight" values of two
SIP servers be comparable.

I have also been thinking about the email exchange we had yesterday.

   > a real problem, though I don't know if we have a real solution for

I am starting to believe that this property --- being sensitive to
the processing requirements of some requests versus others --- is
more pertinent to overload control than it is to load balancing.
The "weight" approach normalizes the processing load at a SIP server
in the cluster and reports a canonical value regardless of the specific
*type* of request the SIP server can handle.  If the SIP server is
handling a lot of resource-heavy INVITE transactions, then it will
report a lower "weight" value.

Thanks,

- vijay

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 8 Mar 2011 15:47:42 -0500

Thierry,

Please see my comments below in green:

From: Thierry Bessis [mailto:[email-masked]] 
Sent: Tuesday, March 08, 2011 12:35 PM
To: Paul E. Jones
Cc: 'Vijay K. Gurbani'; [email-masked]
Subject: Re: [dispatch] SIP load balancing

Paul, 

Vijay,

Agreed, the goal seems to be the same: avoiding or at least delaying
overload. 
And your other comment too: our proposal does try to balance the load on all
the servers. The rational is that it is very easy for each server to know
where it is in terms of resource usage (for instance: I'm 60 percent
resource loaded at this moment), and this is just the information you need
to balance the traffic. Now it seems (but it is not clear, see below)  that
the SCE "available capacity" is actually a real time feedback, so the
information would be very similar actually. (What's the difference ? )

PEJ: I think it is very similar.  SCE provides real-time feedback of the
available resource capacity of a peer device.  The intent is to convey a
single integer value that represents an estimate of the number of sessions
the peer device can handle.  Note, this is not "message per second" or
other, but the number of sessions.  The focus on SCE is on devices like
"call agents", "session border controllers", stateful SIP proxies, etc. 

PEJ: As I mention below, the algorithm used to select a peer device is not a
part of the SCE draft.  One can use any mechanism.  The SCE draft merely
advertises a session capacity estimate.  We've found round-robin to be a
simple approach that works, as it distributes load relatively uniformly, but
stops directing traffic toward devices that report an SCE value that is
below a specified threshold. Other input could be considered, but we
deliberately left the algorithm outside the scope of SCE.

Why would we wait for a lower mark before to influence the distribution ?
Also is that less favored, or given no traffic ? 

PEJ: With our use of SCE, we're not trying to "balance" the load, per se.
One certainly could do that, but in order to balance the load on peer
devices when they are out of balance, one must direct multiple new sessions
toward the same device.  That is not always desirable since new sessions
usually consume more resources temporarily than established sessions.  So,
we try to burden a peer device by throwing a number of new sessions toward
the peer by distributing them in a round-robin fashion.  Again, though, the
approach taken is outside the scope of the SCE draft.

The later has stability issues.  And why do we need ping Options when there
is no traffic ?
 If there is no traffic (or very little), should we care about optimal
distribution from this source ?

PEJ: The assumption is that a peer device might also have several peers.
So, while device A1 may not be directing traffic toward device B, perhaps
A2, A3, and A4 are.  We can use OPTIONS "ping" messages to get current SCE
information.

I don't see this difference here (message overload versus resource overload
? ) . To me, SOC focuses on source shedding, while SCE seems to focus on
improved load distribution, just like our proposal.

PEJ: A peer device might be able to handle 18,000 new sessions, but perhaps
it can only handle them at a rate of 100 per second.  We would use SCE to
determine that a peer can handle new sessions, while we would use the
methods SOC is defining to prevent message overload so we do not send 200
INVITEs per second to the peer.  (Use of the SOC mechanism might prove to
work better than our current round-robin approach, but we've not tested it.)

I believe the signaling is not enough. The deep meaning of the feedback
parameter needs to be clarified too. 
For example, in our proposal, the feedback is simply the current utilization
rate of the node (very simple).

PEJ: That sounds similar to SCE.  An SCE-capable device advertises to its
peer how many new sessions it can handle given the number of sessions it is
currently handling. 

Also the expectation needs to be clarified because it has a very important
impact on the recommended behavior in case of overload. 
The solution that we propose constantly tries to balance load in the cluster
of servers. Therefore, in case the SOC shedding kicks in, there is NOT NEED
to repeat the message elsewhere in the cluster because it is balanced
already (So no other server has more capacity). This is a huge advantage for
the network stability as message repeat tends to spread the overload and
collapse the cluster. 

Yes I agree. 

PEJ: If you haven't already, have a look at our draft:

 <http://tools.ietf.org/html/draft-jones-sip-overload-sce-00>
http://tools.ietf.org/html/draft-jones-sip-overload-sce-00

It is now expired, but your draft suggests there is still life in this
approach.  We should look to harmonize them, perhaps.

Paul

Cordially, 

Thierry

Paul

-----Original Message-----

Vijay,

 
 
Thanks,
 
- vijay
--
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{bell-labs.com,acm.org} <mailto:vkg@%7bbell-labs.com,acm.org%7d>
/ [email-masked]
Web:   http://ect.bell-labs.com/who/vkg/
_______________________________________________
dispatch mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dispatch

-- 
-- 
Cordially, 
Thierry Bessis 

  IMS Solution: Architecture & Product Management - ALTA Member
  Organization: ALU > Carrier group > Wireline BU >  IMS Sol Arch, SE, PdM
Alcatel-Lucent
  1960 Lucent Lane - Room 6L-517
  Naperville, IL  60563  U S
  Tel/Fax: +1 630 979 7989
  Corporate IM: tbessis    - Yahoo IM:     thierry_bessis

Conference information:

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 08 Mar 2011 15:01:26 -0600

On 03/07/2011 11:05 PM, Paul E. Jones wrote:
> Vijay,
>
> The Session Capacity Estimate (SCE) idea is actually quite similar
> to the "adaptive load balancing" draft, from what I can see.

Paul: Thank you much for your input.  More inline.

> SCE was proposed as a solution to the "overload" problem, the
> approach being one where we try to avoid overload by selecting a
> device that reports a higher capacity value. Thus, one could view it
> as "overload control through load balancing".

I think overload control and load balancing are distinct, even though
it is natural to relate them.  Load balancing can delay, or prevent,
overload; but once a SIP host is overloaded, load balancing by itself
will not help too much.

I think it is best that we treat them separately, if not for anything
else than the fact that SOC is not chartered to create a solution for
load balancing.  We (i.e., the dispatch WG) can decide to send this
work to SOC, but I think that decision is a bit premature at this
point.  Furthermore, I understand that before the formation of the
SOC WG, there was some debate on whether or not to include load-
balancing in the charter and it was decided that load-balancing will not
be included.

I believe that we should take the view that load balancing and overload-
control are separate, and that when a cluster of SIP servers is put in
service, all of them start off providing load balancing information to
the upstream SIP proxy.  The upstream SIP proxy will do load balancing
to try to spread the load proportionally to the resources available at
the downstream SIP servers (not evenly, proportionally).  As time
progresses, some of them will enter an overload state and at this time,
the overload control kicks in.  When the overload condition is
alleviated, the overloaded peer goes back to reporting its load.

Now ... to the details of "sce" and "weight" as proposed in our draft.

In the large, I agree that we are thinking along the same lines.  This
is good.

In addition to the email exchange between you and Thierry, there
is one more property that is very attractive: statelessness.

I think that it is advantageous that we impose the property of
statelessness on a load balancing mechanism.  With "sce",
the upstream SIP proxy must keep track of how many requests it
has sent to a downstream SIP server since the latter sent it the
last "sce" value.  Furthermore, each time a request is sent to the
downstream SIP server, the SIP proxy must decrement its request count.
Consequently, the state on how many requests have been sent to the
downstream SIP server since the SIP proxy got the last "sce" parameter,
what the low water-mark is, is all lost if not replicated upon a
reboot.

With "weight", you bypass all of this state maintenance business.

All the SIP proxy needs is a "weight" indicator that it will use
in a RFC2782-type algorithm to proportionally send as much traffic
to a downstream SIP server as it has indicated (i.e., higher "weight"
values will get proportionally more traffic).  This is much better
than round-robin, and furthermore, there is no requirement that each
downstream SIP server in the cluster set its "weight" in a consistent
manner, or even that the "weight" values of two SIP servers be
comparable.

If a SIP server thinks that it has infinite resources, it
can set the "weight" to be 65,535.  Consequently, it will get a
proportionally more traffic if the "weight" of other SIP servers
in the cluster is set to, say, 10 or 20.  Similarly, if the SIP server
in a cluster wants to accept less number of messages, it sets a
lower, albeit non-zero, value for its "weight" parameter.   If the
cluster has capacity, other load control weights will be higher, and
will thus get a proportionate share of the traffic.  As all SIP servers
in the cluster reach capacity, the load distribution algorithm arranges
for a uniform load distribution to all of the SIP servers until either
the offered load subsides or the SIP servers enter into overload mode
and start shedding traffic using the mechanism defined in the overload
control draft.

> standardization: we just need to standardize the signaling to enable

I could not agree more.  Local decisions on how each SIP server in a
cluster generates a "weight" value are not subject to standardization.
Transporting this value and specifying a default algorithm to allow
a SIP proxy to perform load balancing seem to be a good avenue for
standardization.

Note that I think specifying a default algorithm to do load balancing
is important, and our draft chooses to use the well-understood RFC2782
usage of "weight" to do this.  This does not preclude any implementer
from coming up with their own algorithm and using that instead.

> In any case, it certainly looks like we're thinking along the same

Indeed.  If this work progresses, it'll be nice to have a coherent
view going forward.  If you will be in Prague, it will be good to
hook up and chat about this.

Once again, thanks for your input.

- vijay

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 08 Mar 2011 16:00:45 -0500

> SIP servers be comparable.

I don't understand that.

Suppose we have two servers that are identical except that they have 
different understandings of the weight assignment algorithm.

One thinks it is a powerful server, and sets its initial (unloaded) 
weight to 32k. The other thinks that it is not especially powerful as 
servers go, and so sets its weight to 4k.

That means that initially the first will get sent 8 times as much 
traffic as the other. So of course it will get more heavily loaded. As 
it does, it will presumably reduce its weight, but for a given load it 
will always have a larger number than the other.

> report a lower "weight" value.

The thing is, a server may be able to handle a lot of some type of load 
while being totally out of capacity for some other type of load.

(E.g. a box that is both an SBC to a provider sip trunk and a gw to some 
PSTN lines. It might be totally out of PSTN lines yet be able to handle 
lots of calls that happen to be addressed to that they go to the sip trunk.)

Also, shedding load is harder when the consumed resource is dialog stateful.

I think this is a hard problem.

	Thanks,
	Paul

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 08 Mar 2011 16:02:10 -0600

On 03/08/2011 03:00 PM, Paul Kyzivat wrote:
> traffic as the other. So of course it will get more heavily loaded.
> As it does, it will presumably reduce its weight, but for a given
> load it will always have a larger number than the other.

Paul: Good point.  However, note that if the server used AIMD strategy,
it will converge to an equilibrium point where it is reporting < 4K
"weight".  At that time, the other server becomes the preferred server.

I suspect that a good algorithm for reducing the "weight" value would
use some sort of AIMD strategy.

> The thing is, a server may be able to handle a lot of some type of
> load while being totally out of capacity for some other type of
> load.
>
> (E.g. a box that is both an SBC to a provider sip trunk and a gw to
> some PSTN lines. It might be totally out of PSTN lines yet be able to
> handle lots of calls that happen to be addressed to that they go to
> the sip trunk.)

True, but at least it has enough processing capacity to reject
the calls to PSTN lines and accept calls to the SIP trunk.  Rejecting
the PSTN calls at the sink is better than forcing an upstream server
to shed them.

> I think this is a hard problem.

No doubt, but I will sweep the which-message-to-shed problem
under the overload rug, and not worry about it when dusting the
load balancing rug ;-)

Thanks,

- vijay

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 08 Mar 2011 16:28:36 -0600

On 03/08/2011 03:52 AM, Parthasarathi R (partr) wrote:
>[...] Even though any overload control algorithm

Partha: Thank you for your input.  I do agree that overload control
is the extreme case for load balancing.  However, as my email to
Paul J. indicated, treating them differently may help, or we could
make load balancing an extension under overload control.  I don't
know what the answer is ... we'd have to hash it out in dispatch
on where to send load balancing to.  My understanding is that the
charter for SOC does not account for load balancing.

> wrongly understand.

Our mechanism does not distinguish between media or signaling in the
sense that the "weight" that is reported is a normalized value that
is calculated by the SIP server in a cluster.  The SIP server could be
a SIP bridge, for instance, in which case it may be able to handle a
lot of point-to-point calls, but can only handle few calls
requiring full mesh conferencing capabilities.  As long as the SIP
server is handling the low-resource point-to-point calls, it reports
a higher "weight" value.  But as soon as a conference call with a large
number of participants commences, using other resources like mixers
and transcoders, etc., the SIP server starts to report lower "weight"
values.

Thanks,

- vijay

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 08 Mar 2011 19:26:12 -0500

> "weight". At that time, the other server becomes the preferred server.

I guess I need more explanation of how that algorithm applies to this 
problem. Are you saying that the "weight" is the "window size", and that 
the server increases/decreases it whenever the incoming load is not 
keeping it at the optimal loading? Sounds interesting. But I know next 
to nothing about that algorithm or the statistical behavior of such 
algorithms.

> the PSTN calls at the sink is better than forcing an upstream server

Yes, up to a point. If we are balancing across a lot of servers, and 
most of them are out of pstn lines (because we are close to max 
capacity) then rejecting at the server may result in the same request 
having to be retried many times before finding a server that can handle 
the request. Once that number gets above a few the call wait time gets 
too high to be acceptable. You need a better way to pick the box that 
has resources to handle the particular request.

But I don't see how one characterizes the resource availability/limits 
in a way that the upstream node can effectively use to make a good decision.

	Thanks,
	Paul

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Wed, 09 Mar 2011 11:08:38 -0600

On 03/08/2011 06:26 PM, Paul Kyzivat wrote:
> I guess I need more explanation of how that algorithm applies to
> this problem.

Paul: I am referring to the stock algorithm in RFC2782 for
ordering RRs using the "weight" component.  Better algorithms
can be substituted, of course, but as a default, the one in
RFC2782 suffices.

> Are you saying that the "weight" is the "window size", and that the
> server increases/decreases it whenever the incoming load is not
> keeping it at the optimal loading? Sounds interesting.

Yes, sort of.  But because traffic is flowing one way --- from
the SIP proxy (the load balancer) to the SIP server in the
cluster --- there is only one window size (that of the latter).

> But I know next to nothing about that algorithm or the statistical
> behavior of such algorithms.

Taking your example, let's say that there are two servers and
srv1 advertises a "weight" of 32,000 and srv2 advertises a
"weight" of 4,000.  As load increases, srv1 decreases its
"weight" by 1/2 each time period (i.e., at time T0, "weight" was
32,000; at T1 it advertises a "weight" of 16,000; at T2 8,000;
at T3 4,000; at T4 2,000).  Let's also assume that srv2 keeps
its "weight" constant at 4,000.

At T4, the "weight" of srv1 is less than that of srv2.  Up until T4,
srv1 will get about 80% of the traffic (assuming only these two
servers and a constant "weight" of 4,000 for srv2).  At T4 and beyond,
srv2 picks up more than 90% of the traffic, relieving srv1.

Convergence is predicated on the delta between T_n and T_(n+1).
If the delta is larger, it will take a bit more time to converge.

Thanks,

- vijay

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Wed, 9 Mar 2011 12:10:18 -0500

This is a fascinating subject.  On the way home from work, I started to frame a set of differential equations to model it.

The key, of course, is to have all of the servers reach full capacity *at the same time* if the presented load increases steadily.  That is how we maximize goodput under such circumstances.  (If there is spare capacity within the cluster of servers, it doesn't matter if the load on the servers is balanced or not, as goodput equals presented load.)

The crucial questions are "What is the algorithm for generating the number that the server presents to the load balancer?" and "What algorithm does the load balancer use for assigning work, given the numbers presented by the servers?"

The draft (draft-bessis-dispatch-adaptive-load-balancing-00) seems to assume that the assignment algorithm is much the same as RFC 3263, to statistically send Wi/(sum(Wj)) of the load to server i (with Wi being the number reported by server i).  The draft gives little or no hint of what the generation algorithm will be.

In order for the mechanism to be standardized in a meaningful way, it must specify the generation algorithm sufficiently that a combination of servers that use generation algorithms in conformance with the standard will work together correctly -- as I said above, as the entire cluster nears full capacity, all servers much reach full capacity simultaneously.

t seems that we need to do a careful analysis of the behavior of the system near the point where "unused capacity of server i" is near 0 for all servers i.  The first question is to categorize generation functions by their behavior near the point "unused capacity = 0", that is, does the reported "weight" approach a non-zero number, does it approach zero linearly, does it approach zero quadratically, etc.?  I suspect that the system behaves well only when all servers use functions of the same form, and with the same constant on the leading term.  Intuitively, the correct answer seems to be to have the weight be "the unused capacity, measured in number of calls that can be handled".

This gets messier when we consider that the load is comes in quantized units (single calls) and has significant statistical fluctuations.  In particular, some functions that appear to work correctly in the "analytical" analysis outlined above my have bad behavior when presented with statistical fluctuations.

Additionally, the feedback from the servers to the load balancer will not be instantaneous, and the delays will introduce "control loop delays", which if not handled correctly might cause the system to oscillate.

A different problem results from the fact that different calls may present decidedly different loads to a server, and that the server may have several different types of resource.  How does the server sensibly report the nature of its resources and the loading on each?  How does the load balancer estimate the resources required by an individual call?  (Note that this problem is largely nonexistent in circuit-switched systems, as all calls present exactly the same load.)  (This is the problem that has bedevilled attempts to solve the problem in draft-partha-sip-overload-resource-availability.)

But I don't see any of the needed analysis in the draft, and I don't see any indication in the draft of a realization that this analysis needs to be done.  Until that is fixed, I predict that a solution will not be found, or worse, a solution that appears to work on paper but does not work in practice.

Dale

---

From: Thierry Bessis <[email-masked]>
Date: Wed, 09 Mar 2011 11:26:09 -0600

Thanks Paul for you detailed answer !

I've looked at your SCE proposal and your comments and my impression is 
that our two proposals are actually very different both from their 
intent perspective, and from their realization perspective.

The intent of our load balancing proposal is clearly continuous load 
balancing. Our suggested weight does try to figure any kind of remaining 
capacity in terms of sessions: it only indicates the instant (slightly 
smoothed) available resource, which integrates CPU, Memory and any other 
constrained resource. For the kind of usage we have in mind, (and for 
example), it does not matter if we have lot of sessions "left" when we 
have very little CPU to register those sessions: it is optimal to 
temporary offload this busy node rather than risking a CPU induced 
rejection or failure. When the node will have more CPU available, it 
will take more sessions, and may be even more than others, because it 
may have now more memory than others. If this node never have more CPU 
available, then its "session" capacity is actually unusable and the 
system correctly reflects that.

Also, the SOC was not intended for "dynamic" load distribution: it only 
provides the feedback when some shedding is expected (So already one or 
multiple nodes are about to be overloaded). Also, integrating the 
information from both SOC and SCE to take the distribution decision 
seems rather complicated to me. In our proposal, the distribution 
algorithm is simple, never changes, and is independent from SOC. We 
selected a node with weights always the same way, and we apply the SOC 
rejection when applicable to this node.

Cordially,

Thierry

On 08-Mar-2011 14:47, Paul E. Jones wrote:
> *From:*Thierry Bessis [mailto:[email-masked]]
> *Sent:* Tuesday, March 08, 2011 12:35 PM
> *To:* Paul E. Jones
> *Cc:* 'Vijay K. Gurbani'; [email-masked]
> *Subject:* Re: [dispatch] SIP load balancing
>
> Paul,
>
> And your other comment too: our proposal does try to balance the load 
> on all the servers. The rational is that it is very easy for each 
> server to know where it is in terms of resource usage (for instance: 
> I'm 60 percent resource loaded at this moment), and this is just the 
> information you need to balance the traffic. Now it seems (but it is 
> not clear, see below)  that the SCE "available capacity" is actually a 
> real time feedback, so the information would be very similar actually. 
> PEJ: I think it is very similar.  SCE provides real-time feedback of 
> the available resource capacity of a peer device.  The intent is to 
> convey a single integer value that represents an estimate of the 
> number of sessions the peer device can handle.  Note, this is not 
> "message per second" or other, but the number of sessions.  The focus 
> on SCE is on devices like "call agents", "session border controllers", 
> stateful SIP proxies, etc.
>
>   
> I see a contradiction here: if it is round-robin, it does not take 
> into account the SCE value.
> PEJ: As I mention below, the algorithm used to select a peer device is 
> not a part of the SCE draft.  One can use any mechanism.  The SCE 
> draft merely advertises a session capacity estimate.  We've found 
> round-robin to be a simple approach that works, as it distributes load 
> relatively uniformly, but stops directing traffic toward devices that 
> report an SCE value that is below a specified threshold. Other input 
> could be considered, but we deliberately left the algorithm outside 
> the scope of SCE.
>
> Why would we wait for a lower mark before to influence the 
> distribution ? Also is that less favored, or given no traffic ?
>
> PEJ: With our use of SCE, we're not trying to "balance" the load, per 
> se.  One certainly could do that, but in order to balance the load on 
> peer devices when they are out of balance, one must direct multiple 
> new sessions toward the same device.  That is not always desirable 
> since new sessions usually consume more resources temporarily than 
> established sessions.  So, we try to burden a peer device by throwing 
> a number of new sessions toward the peer by distributing them in a 
> round-robin fashion.  Again, though, the approach taken is outside the 
> scope of the SCE draft.
>
>
> PEJ: The assumption is that a peer device might also have several 
> peers.  So, while device A1 may not be directing traffic toward device 
> B, perhaps A2, A3, and A4 are.  We can use OPTIONS "ping" messages to 
> get current SCE information.
>
>
>
> overload ? ) . To me, SOC focuses on source shedding, while SCE seems 
> to focus on improved load distribution, just like our proposal.
>
> PEJ: A peer device might be able to handle 18,000 new sessions, but 
> perhaps it can only handle them at a rate of 100 per second.  We would 
> use SCE to determine that a peer can handle new sessions, while we 
> would use the methods SOC is defining to prevent message overload so 
> we do not send 200 INVITEs per second to the peer.  (Use of the SOC 
> mechanism might prove to work better than our current round-robin 
> approach, but we've not tested it.)
>
>   
> I believe the signaling is not enough. The deep meaning of the 
> feedback parameter needs to be clarified too.
> PEJ: That sounds similar to SCE.  An SCE-capable device advertises to 
> its peer how many new sessions it can handle given the number of 
> sessions it is currently handling.
>
> cluster of servers. Therefore, in case the SOC shedding kicks in, 
> there is NOT NEED to repeat the message elsewhere in the cluster 
> because it is balanced already (So no other server has more capacity). 
> This is a huge advantage for the network stability as message repeat 
> tends to spread the overload and collapse the cluster.
>
>     From:[email-masked]  <mailto:[email-masked]>  [mailto:[email-masked]] On
>
>     To:[email-masked]  <mailto:[email-masked]>
>
>     Email:vkg@{bell-labs.com,acm.org}  <mailto:vkg@%7bbell-labs.com,acm.org%7d>  /[email-masked]  <mailto:[email-masked]>
>
>     Web:http://ect.bell-labs.com/who/vkg/
>
>     [email-masked]  <mailto:[email-masked]>
>
> [email-masked]  <mailto:[email-masked]>
> *--
> Thierry Bessis*
> <http://ihgpweb.ih.lucent.com/%7Etbessis/>
> <http://www.google.com/calendar/embed?src=new_frs3175%40yahoo.fr>
> *Upcoming planned Business Trip: 12 April - 14 April 2011*
> *Upcoming planned Vacation: None*
>
>

Conference information:

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Wed, 9 Mar 2011 11:36:43 -0600

Agree with this.  If loads are not uniform, but bursty, then the
algorithm for the former may fail the latter.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Worley, Dale R (Dale)
Sent: Wednesday, March 09, 2011 12:10 PM
To: Paul Kyzivat (pkyzivat); Vijay K. Gurbani
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

This is a fascinating subject.  On the way home from work, I started to
frame a set of differential equations to model it.

The key, of course, is to have all of the servers reach full capacity
*at the same time* if the presented load increases steadily.  That is
how we maximize goodput under such circumstances.  (If there is spare
capacity within the cluster of servers, it doesn't matter if the load on
the servers is balanced or not, as goodput equals presented load.)

The crucial questions are "What is the algorithm for generating the
number that the server presents to the load balancer?" and "What
algorithm does the load balancer use for assigning work, given the
numbers presented by the servers?"

The draft (draft-bessis-dispatch-adaptive-load-balancing-00) seems to
assume that the assignment algorithm is much the same as RFC 3263, to
statistically send Wi/(sum(Wj)) of the load to server i (with Wi being
the number reported by server i).  The draft gives little or no hint of
what the generation algorithm will be.

In order for the mechanism to be standardized in a meaningful way, it
must specify the generation algorithm sufficiently that a combination of
servers that use generation algorithms in conformance with the standard
will work together correctly -- as I said above, as the entire cluster
nears full capacity, all servers much reach full capacity
simultaneously.

t seems that we need to do a careful analysis of the behavior of the
system near the point where "unused capacity of server i" is near 0 for
all servers i.  The first question is to categorize generation functions
by their behavior near the point "unused capacity = 0", that is, does
the reported "weight" approach a non-zero number, does it approach zero
linearly, does it approach zero quadratically, etc.?  I suspect that the
system behaves well only when all servers use functions of the same
form, and with the same constant on the leading term.  Intuitively, the
correct answer seems to be to have the weight be "the unused capacity,
measured in number of calls that can be handled".

This gets messier when we consider that the load is comes in quantized
units (single calls) and has significant statistical fluctuations.  In
particular, some functions that appear to work correctly in the
"analytical" analysis outlined above my have bad behavior when presented
with statistical fluctuations.

Additionally, the feedback from the servers to the load balancer will
not be instantaneous, and the delays will introduce "control loop
delays", which if not handled correctly might cause the system to
oscillate.

A different problem results from the fact that different calls may
present decidedly different loads to a server, and that the server may
have several different types of resource.  How does the server sensibly
report the nature of its resources and the loading on each?  How does
the load balancer estimate the resources required by an individual call?
(Note that this problem is largely nonexistent in circuit-switched
systems, as all calls present exactly the same load.)  (This is the
problem that has bedevilled attempts to solve the problem in
draft-partha-sip-overload-resource-availability.)

But I don't see any of the needed analysis in the draft, and I don't see
any indication in the draft of a realization that this analysis needs to
be done.  Until that is fixed, I predict that a solution will not be
found, or worse, a solution that appears to work on paper but does not
work in practice.

Dale

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 09 Mar 2011 12:43:22 -0500

> If the delta is larger, it will take a bit more time to converge.

Hmm. Clearly I'm missing something, since I see no mention of capacity 
in the above. Certainly remaining capacity of the server needs to enter 
into the computation of the weight.

I suspect maybe this needs to be thoroughly discussed over beer.
And with Dale in the discussion.

	Thanks,
	Paul

---

From: Thierry Bessis <[email-masked]>
Date: Wed, 09 Mar 2011 11:49:21 -0600

Hi Paul,

Please see my comment.
> servers go, and so sets its weight to 4k.
That is not how we implemented this weight: The weight is NOT an 
estimation of the capacity, it is a (smoothed) _measurement_ of the 
resource available.  Weight of 1 means that you have NO resource 
available (no CPU, or no Memory, or you are missing anything else 
required to process new sessions). Weight of <maximum> means that you 
have ALL your resources available (you are IDLE basically). This is a 
relative data: the same weight used by different servers does NOT 
represent the same capacity ! It represents the same relative load. This 
is consistent with the fact that we are trying to load all the server at 
the same relative load. In your example, the server does not need to 
know anything about its power or capacity, and even less about the 
capacity of the neighbors. It does not need to count anything, or 
understand why and how its resource vary with the number of instances: 
it just measures the resources.
> load while being totally out of capacity for some other type of load.
>
In this case, the mechanism should be applied in two places: one at the 
SBC ingress point, and another one after the PSTN breaking decision has 
been made: I don't see any other way ! :-)
> Also, shedding load is harder when the consumed resource is dialog 
I believe we should only shed at the source non Dialog related 
transactions, or new Dialogs.

Cordially,

Thierry

Conference information:

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Wed, 9 Mar 2011 13:39:29 -0500

________________________________________
From: Mike Hammer (hmmr) [[email-masked]]

(1) We *know* the loads are bursty.

(2) If the system has a uniform load that exceeds capacity, the administrators will install more capacity.

The only interesting case, in practice, is non-uniform loads near capacity.

(We can make a reasonable estimate of what is "near capacity".  If calls are assigned randomly in proportion to each server's total capacity (via RFC 3263), each server will on average get its fair share of the traffic, with a random variation around that average of approximately the square-root of the average traffic it receives.  So if a server can handle N calls, as long as the average load it is given is less than N minus square-root(N), then RFC 3263 will suffice.  The interesting problem is when the load presented to the system requires loading the servers in excess of that level.)

We still haven't demonstrated that the "obvious" strategy does not work in practice:  If a server is overloaded, it rejects the INVITE and the load balancer forks the INVITE to another server.

Dale

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Wed, 9 Mar 2011 13:02:17 -0600

Dale,

I am thinking of loads in two dimensions, if that is on the table:
 - signaling from the arrival rate of SIP INVITEs as an example, can be
bursty while the resource to handle SIP might be not so bursty,
 - media in terms of whether this is only uniform demand of PSTN ports
or voice calls, versus narrowband voice request versus HD video
Telepresence request, where the resources to meet that mix of demand
might be deemed bursty.

This thread has touched on several of these areas, much the way 4 guys
are describing the elephant. :)

Mike

-----Original Message-----
From: Worley, Dale R (Dale) [mailto:[email-masked]] 
Sent: Wednesday, March 09, 2011 1:39 PM
To: Mike Hammer (hmmr); Paul Kyzivat (pkyzivat); Vijay K. Gurbani
Cc: [email-masked]
Subject: RE: [dispatch] SIP load balancing

(2) If the system has a uniform load that exceeds capacity, the
administrators will install more capacity.

The only interesting case, in practice, is non-uniform loads near
capacity.

(We can make a reasonable estimate of what is "near capacity".  If calls
are assigned randomly in proportion to each server's total capacity (via
RFC 3263), each server will on average get its fair share of the
traffic, with a random variation around that average of approximately
the square-root of the average traffic it receives.  So if a server can
handle N calls, as long as the average load it is given is less than N
minus square-root(N), then RFC 3263 will suffice.  The interesting
problem is when the load presented to the system requires loading the
servers in excess of that level.)

We still haven't demonstrated that the "obvious" strategy does not work
in practice:  If a server is overloaded, it rejects the INVITE and the
load balancer forks the INVITE to another server.

Dale

---

From: Thierry Bessis <[email-masked]>
Date: Wed, 09 Mar 2011 14:20:56 -0600

Dale,

Please see my comments.

On 09-Mar-2011 11:10, Worley, Dale R (Dale) wrote:
> The crucial questions are "What is the algorithm for generating the number that the server presents to the load balancer?"
If you have read my previous comments, we suggest there is no algorithm 
per se: this is a measurement of the resource: If you measure the CPU 
and the Memory (for example), you'll take the smallest. It is better to 
smooth (with a 10s sliding window average for instance) the number.

> and "What algorithm does the load balancer use for assigning work, given the numbers presented by the servers?"
Isn't the one described in DNS explicit enough ? You just need to 
requesting a random number from 1 to the sum of the weights and choose 
the server accordingly.
> In order for the mechanism to be standardized in a meaningful way, it must specify the generation algorithm sufficiently that a combination of servers that use generation algorithms in conformance…
Exactly !
> t seems that we need to do a careful analysis of the behavior of the system near the point where "unused capacity of server i" is near 0 for all servers i.  The first question is to categorize gene…
This is not what we have in mind: the weight should be your relative 
resource usage. (0 - 100%). Using the proposed algorithm, this insure 
that servers reach full capacity simultaneously (even when they have 
vastly different capacities).
> This gets messier when we consider that the load is comes in quantized units (single calls) and has significant statistical fluctuations.  In particular, some functions that appear to work correctl…
Agreed. In a signaling network such as IMS, CPU usage typically needs to 
be averaged over 10s to smooth the curve. Memory usage is already an 
integration, so it can typically be used as is.
> Additionally, the feedback from the servers to the load balancer will not be instantaneous, and the delays will introduce "control loop delays", which if not handled correctly might cause the syste…
Yes, this is true. We are successfully using the simple algorithm 
mentioned with DNS (where we update the weight every minute). Otherwise 
they are algorithms that can be used to improve the oscillation in the 
feedback chain command. We think that since we are proposing a default 
one that works, this should be OK.
> A different problem results from the fact that different calls may present decidedly different loads to a server, and that the server may have several different types of resource.  How does the ser…
This is why I suggest that the weight should just be a measurement of 
the relative resource usage: then none of those issues seems to exist.
Another advantage is that it is far more robust: the capacity can evolve 
based on whatever is happening on the server, and everything is not call 
related (Consider the case where the server is doing a re-synch with its 
mate: during sometime, its ability to process new calls will be 
considerably less and it would be preferable that less calls get in 
during that time.
> But I don't see any of the needed analysis in the draft, and I don't see any indication in the draft of a realization that this analysis needs to be done.  Until that is fixed, I predict that a sol…
This solution works in practice: we are using it in ALU between 
components and nodes that are in the same NE (we currently use a 
proprietary transport mechanism to return the weights).

Cordially,

Thierry

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Wed, 09 Mar 2011 14:34:38 -0600

Dale,

> We still haven't demonstrated that the "obvious" strategy does not work in practice:  If a server is overloaded, it rejects the INVITE and the load balancer forks the INVITE to another server.
My opinion is the proposed SIP balancing mechanism should be employed 
together with the OLC to get the best results. Both systems are 
complementary and work well together.

The strategy to forks invite to other servers in a cluster in case of 
overload does not work in general. This has been demonstrated by 
simulation made by Volker Hilt, and we have experienced this issue as 
well in our Labs : When a cluster is near full capacity, and if nodes 
fork after overload rejection, the system collapses very fast. If the 
cluster is unevenly loaded, you don't even need to be close to full 
(cluster) capacity to collapse ! (Another good reason to be evenly loaded).

Cordially,

Thierry

Conference information:

---

From: "Paul E. Jones" <[email-masked]>
Date: Wed, 9 Mar 2011 19:47:17 -0500

Vijay,

> control are separate

They are different, but one can use overload prevention to balance load.
That was my point.  The focus of our draft was to avoid overload.  A
byproduct is that I can use it to perfectly balance load on a peer server.
 
> In addition to the email exchange between you and Thierry, there is one
> more property that is very attractive: statelessness.
> 
> statelessness on a load balancing mechanism.  With "sce", the upstream
> SIP proxy must keep track of how many requests it has sent to a
> downstream SIP server since the latter sent it the last "sce" value.

SCE does not propose keeping track of messages.  As a server replies
forwards a message to a peer, it inserts an SCE value: a value that
indicates how much additional capacity it can handle given its current load.
Imagine an SBC at the edge of my enterprise peering with AT&T.  Perhaps I
have 5 SBCs to which I can direct traffic.  As I forward traffic to the SBC,
I see in response messages SCE values like 1000, 1200, 2300, 1300, 850.
This tells my enterprise SBC roughly how many more sessions I can direct
toward each of those SBCs. The state maintained by the enterprise SBC is
perhaps an ordered list (or using whatever algorithm one chooses) to select
which SBC to direct the next session toward.

> Furthermore, each time a request is sent to the downstream SIP server,
> the SIP proxy must decrement its request count.

The enterprise SBC does not decrement counters, but it will likely be
provided with a new SCE value that might be influenced by other factors
other than the enterprise SBC.

> what the low water-mark is, is all lost if not replicated upon a reboot.

None of this information is needed.  When the enterprise SBC is rebooted, it
will discover the SCE values from its peers the next time it directs traffic
(either a call or an OPTIONS "ping").

> With "weight", you bypass all of this state maintenance business.

The SCE draft does not introduce any of the state you describe.  I agree
entirely that if I had to maintain all the state you described, it would be
complex.  But, SCE does not do it.

> All the SIP proxy needs is a "weight" indicator that it will use in a
> RFC2782-type algorithm to proportionally send as much traffic to a
> downstream SIP server as it has indicated (i.e., higher "weight"
> values will get proportionally more traffic).  This is much better than
> round-robin, and furthermore, there is no requirement that each

As RFC 2782 says, "Weight, the server selection field, is not quite
satisfactory, but the actual load on typical servers changes much too
quickly to be kept around in DNS caches."

That is precisely the case with high-transaction systems like SBCs.  It is
even more pronounced when those SBCs are handling phone calls of varying
duration, or processing media flows that vary significantly in terms of CPU
or bandwidth consumption, or when performing transcoding or transrating
functions.

I fully agree that a simple round-robin approach does not work.  We've
proven that.  What does work, though, is a combination of round-robin over a
set of peer devices that we know have available capacity.  That's where SCE
comes into play.  For over 10 years, we have performed peer selection using
an approach very much like SCE + round robin for 10 years now.  It works.
 
> If a SIP server thinks that it has infinite resources, it can set the
> "weight" to be 65,535.  Consequently, it will get a proportionally more
> traffic if the "weight" of other SIP servers in the cluster is set to,
> say, 10 or 20.  Similarly, if the SIP server in a cluster wants to
> accept less number of messages, it sets a

This is where the weighting mechanism starts to sound lot like SCE. Rather
than providing a "weight", we call it "sce".  I had proposed before that SCE
would be calculated as the minimum "capacity" value from a set of capacity
measurements. The device would estimate its capacity given current memory
utilization with respect to the current number of sessions.  The device
would estimate its capacity given current CPU utilization with respect to
the current number of sessions.  What the device would consider (CPU,
memory, DSPs, whatever) is outside the scope of the draft, but in the end,
the device determines these values and returns the minimum as the SCE value.
(After all, if memory is exhausted, then it does not matter how many other
resources are available: one is out of memory.)  This is the approach we
take, measuring resources on a set interval or certain events.  Again, all
of that is really a matter of implementation.

At the end, our comes an SCE value and it sounds a lot like the weight.
It's a number with which a peer can compare against similar numbers reported
from other peers.

> Indeed.  If this work progresses, it'll be nice to have a coherent view
> going forward.  If you will be in Prague, it will be good to hook up and
> chat about this.

I'd love to, but I will not be there, unfortunately. (I think 1999 was the
last time I had an opportunity to make it to an IETF meeting.)

Paul

---

From: [email-masked]
Date: Thu, 10 Mar 2011 11:22:56 +0800

http://tech.163.com/09/0121/14/506IMRFL000915BE.html

http://tech.qq.com/a/20090121/000204.htm

http://it.sohu.com/20090121/n261883817.shtml

===================================
 Zip    : 210012
 Tel    : 87211
 Tel2   :(+86)-025-52877211
 e_mail : [email-masked]
===================================

"Vijay K. Gurbani" <[email-masked]> 
发件人:  [email-masked]
2011-03-09 05:01

收件人
[email-masked]
抄送
"[email-masked]" <[email-masked]>
主题
Re: [dispatch] SIP load balancing

- vijay

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is solely property of the sender's organization. This mail communication is confidential. Recipients named above are obligated to maintain secrecy and are not permitted to disclose the contents of this communication to others.
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the originator of the message. Any views expressed in this message are those of the individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam system.

---

From: [email-masked]
Date: Thu, 10 Mar 2011 11:27:59 +0800

[email-masked] 
2011-03-10 11:22

收件人
"Vijay K. Gurbani" <[email-masked]>
抄送
[email-masked], "[email-masked]" <[email-masked]>
主题
[dispatch] F7的RFC发布新闻稿，供参考   //答复: Re:  SIP load balancing

收件人
[email-masked] 
抄送

- vijay

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is 
solely property of the sender's organization. This mail communication is 
confidential. Recipients named above are obligated to maintain secrecy and 
are not permitted to disclose the contents of this communication to 
others.
This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. 
If you have received this email in error please notify the originator of 
the message. Any views expressed in this message are those of the 
individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam 
system.
_______________________________________________
dispatch mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dispatch

---

From: "Paul E. Jones" <[email-masked]>
Date: Wed, 9 Mar 2011 23:15:03 -0500

Thierry,

I think there is more in common than not.

You compute a "weight" and we compute an "sce" that is, in my opinion, more
or less the same: it's a value that indicates how much traffic can be
directed toward the box relative to other peer devices.

You want to do "load balancing" whereas we want to do "overload prevention".
Preventing overload means distributing load across boxes.  We are not
proposing to perfectly balance the load, but the load is distributed
nonetheless.

Honestly, I just don't see a lot of difference, other than terminology.  The
algorithm employed by the device directing traffic, but that algorithm is
independent of the advertisement of a capacity/weight value.

Paul

From: Thierry Bessis [mailto:[email-masked]] 
Sent: Wednesday, March 09, 2011 12:26 PM
To: Paul E. Jones
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

I've looked at your SCE proposal and your comments and my impression is that
our two proposals are actually very different both from their intent
perspective, and from their realization perspective. 

The intent of our load balancing proposal is clearly continuous load
balancing. Our suggested weight does try to figure any kind of remaining
capacity in terms of sessions: it only indicates the instant (slightly
smoothed) available resource, which integrates CPU, Memory and any other
constrained resource. For the kind of usage we have in mind, (and for
example), it does not matter if we have lot of sessions "left" when we have
very little CPU to register those sessions: it is optimal to temporary
offload this busy node rather than risking a CPU induced rejection or
failure. When the node will have more CPU available, it will take more
sessions, and may be even more than others, because it may have now more
memory than others. If this node never have more CPU available, then its
"session" capacity is actually unusable and the system correctly reflects
that. 

Also, the SOC was not intended for "dynamic" load distribution: it only
provides the feedback when some shedding is expected (So already one or
multiple nodes are about to be overloaded). Also, integrating the
information from both SOC and SCE to take the distribution decision seems
rather complicated to me. In our proposal, the distribution algorithm is
simple, never changes, and is independent from SOC. We selected a node with
weights always the same way, and we apply the SOC rejection when applicable
to this node.

Cordially, 

Thierry

Thierry,

 

Paul, 

Vijay,

 

 

Yes I agree. 

Paul

Cordially, 

Thierry

Paul

-----Original Message-----

Vijay,

 
 
Thanks,
 
- vijay

-- 
-- 
Cordially, 
Thierry Bessis 

Conference information:

 

-- 
-- 
Cordially, 
Thierry Bessis 

Conference information:

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Thu, 10 Mar 2011 15:11:03 +0530

Vijay,

Please read inline

Thanks
Partha 

-----Original Message-----
From: Vijay K. Gurbani [mailto:[email-masked]] 
Sent: Wednesday, March 09, 2011 3:59 AM
To: Parthasarathi R (partr)
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

On 03/08/2011 03:52 AM, Parthasarathi R (partr) wrote:
>[...] Even though any overload control algorithm  is used ("oc" or 
>"weight" or "SCE" or "resource-availability" other),  the locally 
>overload control mechanism is required in downstream as per  mentioned 
>in draft-ietf-soc-overload-design. I wish that we will look  into these

>aspect because overload control is the extreme case for load  balancing

>the cluster of downstream entity and load balancing mechanism  shall 
>help to avoid the overload control based on feedback.

Partha: Thank you for your input.  I do agree that overload control is
the extreme case for load balancing.  However, as my email to Paul J.
indicated, treating them differently may help, or we could make load
balancing an extension under overload control.  I don't know what the
answer is ... we'd have to hash it out in dispatch on where to send load
balancing to.

<Partha> IMO, it is possible to have the single solution for overload
control which shall be used for load balancing as well in case the units
of the feedback are well defined and uniquely  identified across the
different devices. As Paul mentioned in the mail thread, it is worth to
have BAR BoF for this activity during IETF-80 in case lot of folks are
interested. </Partha>

  My understanding is that the charter for SOC does not account for load
balancing.

<Partha> It took long time for me to understand IETF SOC does not
considered load balancing  :-(...I agree with you now </Partha>

> I guess that your mechanism assumes about downstream SIP UA as 
> signaling entity only and does not handle media. Please correct me in 
> case I wrongly understand.

Our mechanism does not distinguish between media or signaling in the
sense that the "weight" that is reported is a normalized value that is
calculated by the SIP server in a cluster. 

<Partha> It will be really tough to generalize all devices under
"Weight" as a unit. Because "weight" understanding will vary heavy based
on the implementation of a specific devices. Value 1000 in one device
may not be same as 1000 in another device which may confuse load
balancer </Partha>

The SIP server could be a SIP bridge, for instance, in which case it may
be able to handle a lot of point-to-point calls, but can only handle few
calls requiring full mesh conferencing capabilities.  As long as the SIP
server is handling the low-resource point-to-point calls, it reports a
higher "weight" value.  But as soon as a conference call with a large
number of participants commences, using other resources like mixers and
transcoders, etc., the SIP server starts to report lower "weight"
values.

Thanks,

- vijay

---

From: Thierry Bessis <[email-masked]>
Date: Thu, 10 Mar 2011 12:16:53 -0600

Paul,

I agree they are many similitudes ! :-)
But the question I think is: Can we make the two approaches compatible ?

For instance, with our approach, it is fundamental that the weight 
represents a "Relative" occupation, and not an absolute capacity.
The first reason is that we found that loading continuously all the 
servers at the same utilization rate (same percentage) is the optimal 
way of loading a cluster. (We can discuss that if not agreed). The 
second reason is that a relative occupation is very easy to derive from 
information already available: any system knows where it is in terms of 
CPU or Memory usage. In contrast, deriving a left capacity in terms of 
remaining sessions can be difficult: Host can be used by different 
applications and perform different duties, which can change over the 
day, all of them impacting their true capacity at a certain moment. 
Beside, as already discussed, different sessions require different 
resources, so there is a whole complication of dealing with a multitude 
of call profiles, that (again) are not even constant over the day.

Also, we do not want to derive some shedding decision from the weight. 
We want to rely on the OLC mechanism for that. As I mentioned before, 
deriving shedding from the weight is very sensitive to the way weights 
are calculated: we would need to specify exactly how the weight is 
calculated: this is impractical ! Also everybody would need to agree 
about specific threshold: What is the weight value threshold for 
shedding ?: Again this is more difficult, and different servers will 
certainly want to use different thresholds.   Also there is a 
contradiction: whether you are using relative or absolute weights, your 
weight should  zero (100% busy, or 0 sessions available) at the moment 
you are getting overloaded. So if you are 100% busy (or in average no 
room for new sessions), how to you tell how many sessions need to be 
shed ? You could be 100% busy with an offered load of 300% nominal (then 
you need to shed 66%), or be 100% busy at just the nominal load, (then 
you don't need to shed). This again shows this threshold issue I 
mentioned. In our implementation, we used to use to a common 
"utilization" information to do both optimal distribution and shedding, 
but we moved away from that because of those reasons: we found that each 
node indicating its requested rejection rate is much more robust and 
requires very little agreements.

Cordially,

Thierry

On 09-Mar-2011 22:15, Paul E. Jones wrote:
> You compute a "weight" and we compute an "sce" that is, in my opinion, 
> more or less the same: it's a value that indicates how much traffic 
> can be directed toward the box relative to other peer devices.
>
> You want to do "load balancing" whereas we want to do "overload 
> prevention".  Preventing overload means distributing load across 
> boxes.  We are not proposing to perfectly balance the load, but the 
> load is distributed nonetheless.
>
> Honestly, I just don't see a lot of difference, other than 
> terminology.  The algorithm employed by the device directing traffic, 
> but that algorithm is independent of the advertisement of a 
> capacity/weight value.
>
> Paul
>
> *Sent:* Wednesday, March 09, 2011 12:26 PM
> *To:* Paul E. Jones
> *Cc:* [email-masked]
> I've looked at your SCE proposal and your comments and my impression 
> is that our two proposals are actually very different both from their 
> balancing. Our suggested weight does try to figure any kind of 
> remaining capacity in terms of sessions: it only indicates the instant 
> (slightly smoothed) available resource, which integrates CPU, Memory 
> and any other constrained resource. For the kind of usage we have in 
> mind, (and for example), it does not matter if we have lot of sessions 
> "left" when we have very little CPU to register those sessions: it is 
> optimal to temporary offload this busy node rather than risking a CPU 
> induced rejection or failure. When the node will have more CPU 
> available, it will take more sessions, and may be even more than 
> others, because it may have now more memory than others. If this node 
> never have more CPU available, then its "session" capacity is actually 
> unusable and the system correctly reflects that.
>
> Also, the SOC was not intended for "dynamic" load distribution: it 
> only provides the feedback when some shedding is expected (So already 
> one or multiple nodes are about to be overloaded). Also, integrating 
> the information from both SOC and SCE to take the distribution 
> decision seems rather complicated to me. In our proposal, the 
> distribution algorithm is simple, never changes, and is independent 
> from SOC. We selected a node with weights always the same way, and we 
> apply the SOC rejection when applicable to this node.
>
> Cordially,
>
> Thierry
>
> *Cc:* 'Vijay K. Gurbani'; [email-masked] <mailto:[email-masked]>

Conference information:

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Fri, 11 Mar 2011 00:39:33 +0530

Dale,
 
I like the way you described the problem statement. 

________________________________

From: [email-masked] on behalf of Worley, Dale R (Dale)
Sent: Wed 3/9/2011 10:40 PM
To: Paul Kyzivat (pkyzivat); Vijay K. Gurbani
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

<Partha> In case units (number) are well defined in the standard manner, it is possible to generate in the downstream entity and upstream (load balancer) will be able to understand and shall distribute the load based on the unit. IMO, The standard unit may be number of dialog or bandwidth available or any other . The selection of unit needs serious discussion to conclude somewhere. </Partha> 

<Partha> Considering calls as a unit works very well in deployed H.323 network wherein most of the calls consume resources lineraly (G711 or G729) calls </Partha> 

<Partha> This problem shall be solved by creating the lower watermark and upper watermark for the action to be taken. Say above 80 means stop forwarding and but when it comes back to 75 only start forwarding, this mechanism helps in avoiding the oscillation due to delay </partha>

A different problem results from the fact that different calls may present decidedly different loads to a server, and that the server may have several different types of resource.  How does the server sensibly report the nature of its resources and the loading on each?  How does the load balancer estimate the resources required by an individual call? 

<partha> Two aspects are required to solve this issue:

1) Report the current state of the resource (mediactrl has some mechanism for reporting resource, draft-partha-sip-overload-resource-availability mention one another mechanism)

2) Maximum Consumption vector for a dialog which is a prior knowledge exchanged between servers through other mechanism. 

In case the amount of available bandwidth required in the system as a unit, then the downstream shall report the available bandwidth and the amount of maximum consumption of the dialog shall be identified by maximum capacity mentioned in the dialog SDP. </Partha>

(Note that this problem is largely nonexistent in circuit-switched systems, as all calls present exactly the same load.)  (This is the problem that has bedevilled attempts to solve the problem in draft-partha-sip-overload-resource-availability.)

Dale

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Fri, 11 Mar 2011 07:39:07 +0530

Thierry,
 
Please read inline
 
Thanks
Partha

From: [email-masked] on behalf of Thierry Bessis
Sent: Thu 3/10/2011 11:46 PM
To: Paul E. Jones
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

Paul, 

For instance, with our approach, it is fundamental that the weight represents a "Relative" occupation, and not an absolute capacity. 
<Partha> The question is how to perform the "relative" operation without until understanding the capacity of the peer devices. Say Load balancer has 3 devices: 2 devices with 1x capacity and 1 device with 2x capacity. Here all devices will be able to provide the "relative" value without understanding the capacity of the other device. </Partha>

The first reason is that we found that loading continuously all the servers at the same utilization rate (same percentage) is the optimal way of loading a cluster. (We can discuss that if not agreed). 
<Partha> I agree that same utilization rate is the optimal way but the question is how to achieve it.  Round robin algorithm works in case all devices have same capacity and same amount of load per call. In reality, there are multiple varying factor like call duration, different vareity of platform within the cluster (say Generation-1 10 devices and Generation-2 5 devies), different resource usage for the call, rate at which calls generation. </Partha>
 
The second reason is that a relative occupation is very easy to derive from information already available: any system knows where it is in terms of CPU or Memory usage.
 
<Partha> Converting CPU or Memory to weight is not explained your draft. It is the real algorithm which needs lot of discussion. Because preparing the mapping between CPU & Memory to weight will be mind-blogging activity </Partha>
 
 In contrast, deriving a left capacity in terms of remaining sessions can be difficult: Host can be used by different applications and perform different duties, which can change over the day, all of them impacting their true capacity at a certain moment. Beside, as already discussed, different sessions require different resources, so there is a whole complication of dealing with a multitude of call profiles, that (again) are not even constant over the day.

<Partha>  In case you specify, IETF SoC does not allow and so we want another value, I think that we are in same page. If not, I could not understand your argument. As you mentioned, all system will know CPU or memory usage and it is possible to come up the static or refreshable value saying that load shed has to happen when 85% of CPU utilization or 90% of memory utilization. The different between our proposal is that there is no need of two value per message instead 1 value in each message & 1 value is static or in periodic manner  </Partha>
 
 We want to rely on the OLC mechanism for that. As I mentioned before, deriving shedding from the weight is very sensitive to the way weights are calculated: we would need to specify exactly how the weight is calculated: this is impractical ! Also everybody would need to agree about specific threshold: What is the weight value threshold for shedding ?: Again this is more difficult, and different servers will certainly want to use different thresholds.   Also there is a contradiction: whether you are using relative or absolute weights, your weight should  zero (100% busy, or 0 sessions available) at the moment you are getting overloaded. So if you are 100% busy (or in average no room for new sessions), how to you tell how many sessions need to be shed ? You could be 100% busy with an offered load of 300% nominal (then you need to shed 66%), or be 100% busy at just the nominal load, (then you don't need to shed). This again shows this threshold issue I mentioned. In our implementation, we used to use to a common "utilization" information to do both optimal distribution and shedding, but we moved away from that because of those reasons: we found that each node indicating its requested rejection rate is much more robust and requires very little agreements. 

Cordially, 

Thierry

	Thierry,

	You compute a "weight" and we compute an "sce" that is, in my opinion, more or less the same: it's a value that indicates how much traffic can be directed toward the box relative to other peer devices.

	You want to do "load balancing" whereas we want to do "overload prevention".  Preventing overload means distributing load across boxes.  We are not proposing to perfectly balance the load, but the load is distributed nonetheless.

	Honestly, I just don't see a lot of difference, other than terminology.  The algorithm employed by the device directing traffic, but that algorithm is independent of the advertisement of a capacity/weight value.

	Paul

	
	I've looked at your SCE proposal and your comments and my impression is that our two proposals are actually very different both from their intent perspective, and from their realization perspective. 
	
	The intent of our load balancing proposal is clearly continuous load balancing. Our suggested weight does try to figure any kind of remaining capacity in terms of sessions: it only indicates the instant (slightly smoothed) available resource, which integrates CPU, Memory and any other constrained resource. For the kind of usage we have in mind, (and for example), it does not matter if we have lot of sessions "left" when we have very little CPU to register those sessions: it is optimal to temporary offload this busy node rather than risking a CPU induced rejection or failure. When the node will have more CPU available, it will take more sessions, and may be even more than others, because it may have now more memory than others. If this node never have more CPU available, then its "session" capacity is actually unusable and the system correctly reflects that. 
	
	Also, the SOC was not intended for "dynamic" load distribution: it only provides the feedback when some shedding is expected (So already one or multiple nodes are about to be overloaded). Also, integrating the information from both SOC and SCE to take the distribution decision seems rather complicated to me. In our proposal, the distribution algorithm is simple, never changes, and is independent from SOC. We selected a node with weights always the same way, and we apply the SOC rejection when applicable to this node.
	
	Cordially, 
	
	Thierry

	Thierry,

	 

	Paul, 
	
	Please let me address some of your comments. 
	I don't know what the SCE is, so please pardon me if I misunderstand something. 

	Vijay,

	Agreed, the goal seems to be the same: avoiding or at least delaying overload. 
	And your other comment too: our proposal does try to balance the load on all the servers. The rational is that it is very easy for each server to know where it is in terms of resource usage (for instance: I'm 60 percent resource loaded at this moment), and this is just the information you need to balance the traffic. Now it seems (but it is not clear, see below)  that the SCE "available capacity" is actually a real time feedback, so the information would be very similar actually. (What's the difference ? )

	PEJ: I think it is very similar.  SCE provides real-time feedback of the available resource capacity of a peer device.  The intent is to convey a single integer value that represents an estimate of the number of sessions the peer device can handle.  Note, this is not "message per second" or other, but the number of sessions.  The focus on SCE is on devices like "call agents", "session border controllers", stateful SIP proxies, etc. 

	I see a contradiction here: if it is round-robin, it does not take into account the SCE value. 
	Is that something more like a weighted round-robin algorithm ? 

	PEJ: As I mention below, the algorithm used to select a peer device is not a part of the SCE draft.  One can use any mechanism.  The SCE draft merely advertises a session capacity estimate.  We've found round-robin to be a simple approach that works, as it distributes load relatively uniformly, but stops directing traffic toward devices that report an SCE value that is below a specified threshold. Other input could be considered, but we deliberately left the algorithm outside the scope of SCE.

	Why would we wait for a lower mark before to influence the distribution ? Also is that less favored, or given no traffic ? 

	PEJ: With our use of SCE, we're not trying to "balance" the load, per se.  One certainly could do that, but in order to balance the load on peer devices when they are out of balance, one must direct multiple new sessions toward the same device.  That is not always desirable since new sessions usually consume more resources temporarily than established sessions.  So, we try to burden a peer device by throwing a number of new sessions toward the peer by distributing them in a round-robin fashion.  Again, though, the approach taken is outside the scope of the SCE draft.

	The later has stability issues.  And why do we need ping Options when there is no traffic ?
	 If there is no traffic (or very little), should we care about optimal distribution from this source ?

	PEJ: The assumption is that a peer device might also have several peers.  So, while device A1 may not be directing traffic toward device B, perhaps A2, A3, and A4 are.  We can use OPTIONS "ping" messages to get current SCE information.

	I don't see this difference here (message overload versus resource overload ? ) . To me, SOC focuses on source shedding, while SCE seems to focus on improved load distribution, just like our proposal.

	PEJ: A peer device might be able to handle 18,000 new sessions, but perhaps it can only handle them at a rate of 100 per second.  We would use SCE to determine that a peer can handle new sessions, while we would use the methods SOC is defining to prevent message overload so we do not send 200 INVITEs per second to the peer.  (Use of the SOC mechanism might prove to work better than our current round-robin approach, but we've not tested it.) 

	I believe the signaling is not enough. The deep meaning of the feedback parameter needs to be clarified too. 
	For example, in our proposal, the feedback is simply the current utilization rate of the node (very simple).

	PEJ: That sounds similar to SCE.  An SCE-capable device advertises to its peer how many new sessions it can handle given the number of sessions it is currently handling. 
	
	Also the expectation needs to be clarified because it has a very important impact on the recommended behavior in case of overload. 
	The solution that we propose constantly tries to balance load in the cluster of servers. Therefore, in case the SOC shedding kicks in, there is NOT NEED to repeat the message elsewhere in the cluster because it is balanced already (So no other server has more capacity). This is a huge advantage for the network stability as message repeat tends to spread the overload and collapse the cluster. 

	Yes I agree. 

	http://tools.ietf.org/html/draft-jones-sip-overload-sce-00 <http://tools.ietf.org/html/draft-jones-sip-overload-sce-00> 

	It is now expired, but your draft suggests there is still life in this approach.  We should look to harmonize them, perhaps.

	Paul

	Cordially, 
	
	Thierry

	Paul

		-----Original Message-----
		From: [email-masked] [mailto:[email-masked]] On
		Behalf Of Vijay K. Gurbani
		Sent: Monday, March 07, 2011 6:38 PM
		To: [email-masked]
		Subject: Re: [dispatch] SIP load balancing

			Vijay,

		 
		 
		Thanks,
		 
		- vijay
		Email: vkg@{bell-labs.com,acm.org} <mailto:vkg@%7bbell-labs.com,acm.org%7d>  / [email-masked]

	-- 
	-- 
	Cordially, 
	Thierry Bessis 

	
	
	Conference information:

	 

	-- 
	-- 
	Cordially, 
	Thierry Bessis 

	
	
	Conference information:

 

Conference information:

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 11 Mar 2011 03:30:15 -0500

Thierry,

The SCE is not a fixed value that remains constant or decrements by 1,
necessarily, with each new session.  It is a value that is calculated
constantly (or at least with some frequency) and provided to peer devices.
This value can be compared with SCE values from other peer devices to
compare the relative capacities of peer devices at any given moment.

The specific ways through which the SCE value is computed may vary by
vendor, though the approach we have used is one where we calculate an
estimate of the number of sessions a device can handle given the current
number of sessions given current memory utilization.  We then do the same
for CPU and other system resources.  We then take the minimum value of each
and report that as the SCE value.  As a matter of course, our devices are
constantly measuring these resources, so computing the SCE if trivial.

Paul

From: Thierry Bessis [mailto:[email-masked]] 
Sent: Thursday, March 10, 2011 1:17 PM
To: Paul E. Jones
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

Paul, 

For instance, with our approach, it is fundamental that the weight
represents a "Relative" occupation, and not an absolute capacity.
The first reason is that we found that loading continuously all the servers
at the same utilization rate (same percentage) is the optimal way of loading
a cluster. (We can discuss that if not agreed). The second reason is that a
relative occupation is very easy to derive from information already
available: any system knows where it is in terms of CPU or Memory usage. In
contrast, deriving a left capacity in terms of remaining sessions can be
difficult: Host can be used by different applications and perform different
duties, which can change over the day, all of them impacting their true
capacity at a certain moment. Beside, as already discussed, different
sessions require different resources, so there is a whole complication of
dealing with a multitude of call profiles, that (again) are not even
constant over the day.

Also, we do not want to derive some shedding decision from the weight. We
want to rely on the OLC mechanism for that. As I mentioned before, deriving
shedding from the weight is very sensitive to the way weights are
calculated: we would need to specify exactly how the weight is calculated:
this is impractical ! Also everybody would need to agree about specific
threshold: What is the weight value threshold for shedding ?: Again this is
more difficult, and different servers will certainly want to use different
thresholds.   Also there is a contradiction: whether you are using relative
or absolute weights, your weight should  zero (100% busy, or 0 sessions
available) at the moment you are getting overloaded. So if you are 100% busy
(or in average no room for new sessions), how to you tell how many sessions
need to be shed ? You could be 100% busy with an offered load of 300%
nominal (then you need to shed 66%), or be 100% busy at just the nominal
load, (then you don't need to shed). This again shows this threshold issue I
mentioned. In our implementation, we used to use to a common "utilization"
information to do both optimal distribution and shedding, but we moved away
from that because of those reasons: we found that each node indicating its
requested rejection rate is much more robust and requires very little
agreements. 

Cordially, 

Thierry

Thierry,

Paul

Cordially, 

Thierry

Thierry,

 

Paul, 

Vijay,

 

 

Yes I agree. 

Paul

Cordially, 

Thierry

Paul

-----Original Message-----

Vijay,

 
 
Thanks,
 
- vijay

-- 
-- 
Cordially, 
Thierry Bessis 

Conference information:

 

-- 
-- 
Cordially, 
Thierry Bessis 

Conference information:

 

-- 
-- 
Cordially, 
Thierry Bessis 

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Fri, 11 Mar 2011 10:05:50 -0600

Paul,
> <Partha> The question is how to perform the "relative" operation 
> without until understanding the capacity of the peer devices. Say Load 
> balancer has 3 devices: 2 devices with 1x capacity and 1 device with 
> 2x capacity. Here all devices will be able to provide the "relative" 
> value without understanding the capacity of the other device. </Partha>
>
> way of loading a cluster. (We can discuss that if not agreed).
> <Partha> I agree that same utilization rate is the optimal way but the 
> question is how to achieve it.
Thierry>> we are on line on this one ! :-)
> Round robin algorithm works in case all devices have same capacity and 
> same amount of load per call.
Thierry >> Yes I agree it would work in this case. The reason we are not 
using it is that in the IMS network (where we are already using those 
principles), this is not the case: Devices do not have the same capacity 
because they are on different hardware, or (more often) because they are 
sharing resources with other devices that are doing different things at 
different moments. (which provide a huge resource saving !). The result 
is that it's impossible for any device to calculate its own capacity "in 
isolation": It would need to know exactly which other devices are 
sharing the same resources, their resource usage, and what they are 
doing at any time: an operational nightmare which would break all the 
software isolation rules ! :-) . In addition, we sometime shutdown some 
devices and re-enable them later, (or we grow more devices in the 
cluster): Round robin does not work at all in this case.

> In reality, there are multiple varying factor like call duration, 
> different vareity of platform within the cluster (say Generation-1 10 
> devices and Generation-2 5 devies), different resource usage for the 
> call, rate at which calls generation. </Partha>
Thierry>>  yes !
> The second reason is that a relative occupation is very easy to derive 
> from information already available: any system knows where it is in 
> terms of CPU or Memory usage.
> <Partha> Converting CPU or Memory to weight is not explained your 
> draft. It is the real algorithm which needs lot of discussion. Because 
> preparing the mapping between CPU & Memory to weight will be 
> mind-blogging activity </Partha>
Thierry>> We could be more explicit, but should we ? (see next 
paragraph). Memory and CPU are just the two most obvious examples or 
resources that you need to process sessions. For instance, some of our 
servers have a fixed amount of sessions registers (for instance 50 000). 
In this case, the weight also reflects that. The algorithm we use for 
weights is very simple: this is the smallest remaining (in percentage) 
of each resource that is (1) Required to process a call, (2) Scarce  
(not infinite). Let's consider the following system:
The server has 70% of its memory available for calls, an average of 30% 
CPU usage (average over the last 10s for instance), and 40% of its 
registers available (remember that those resources may be shared by 
other components or devices for any other usage: it does not matter).
Then the weight at this moment is MIN (70, 30, 40) = 30%. You then need 
to normalize this weight to match the weight range (an INT ? )

Now we are not saying that everybody should calculate their weight like 
that. What is important is that the weight reflects your current 
_relative_ capacity. In our case, this "algorithm" balances the load of 
our servers very well. Now if you use a server that has a fixed amount 
of resources and that is dimensioned to process let's say 50 000 
sessions, at a pace of 100 session/s, then you could probably derive a 
weight that would depend on both your count of sessions left, and your 
current rate of session/s.
>  In contrast, deriving a left capacity in terms of remaining sessions 
> can be difficult: Host can be used by different applications and 
> perform different duties, which can change over the day, all of them 
> impacting their true capacity at a certain moment. Beside, as already 
> discussed, different sessions require different resources, so there is 
> a whole complication of dealing with a multitude of call profiles, 
> that (again) are not even constant over the day.
>
> <Partha>  In case you specify, IETF SoC does not allow and so we want 
> another value, I think that we are in same page.
Thierry>> we are on the same page. I thought you suggested to use the 
same value.
> If not, I could not understand your argument. As you mentioned, all 
> system will know CPU or memory usage and it is possible to come up the 
> static or refreshable value saying that load shed has to happen when 
> 85% of CPU utilization or 90% of memory utilization. The different 
> between our proposal is that there is no need of two value per 
> message instead 1 value in each message & 1 value is static or in 
> periodic manner </Partha>
Thierry>> We use a single value: see the example above.

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Fri, 11 Mar 2011 10:41:14 -0600

Paul,
> estimate of the number of sessions a device can handle given the 
> current number of sessions given current memory utilization.  We then 
> do the same for CPU and other system resources.  We then take the 
> minimum value of each and report that as the SCE value.  As a matter 
> of course, our devices are constantly measuring these resources, so 
> computing the SCE if trivial.
>
So this mode of operation is very similar to what we also propose. Is 
that number a ratio (percentage of sessions available over the total 
number), or an absolute number ? If this is a ratio, then SCE and our 
load balancing are compatible: the same information and transport 
mechanism could be used for both: A single IETF proposal could be done.

Cordially,

Thierry
> *Sent:* Thursday, March 10, 2011 1:17 PM
> *To:* Paul E. Jones
> *Cc:* [email-masked]
> second reason is that a relative occupation is very easy to derive 
> terms of CPU or Memory usage. In contrast, deriving a left capacity in 
> terms of remaining sessions can be difficult: Host can be used by 
> different applications and perform different duties, which can change 
> over the day, all of them impacting their true capacity at a certain 
> moment. Beside, as already discussed, different sessions require 
> different resources, so there is a whole complication of dealing with 
> a multitude of call profiles, that (again) are not even constant over 
> the day.
>
> contradiction: whether you are using relative or absolute weights, 
> your weight should  zero (100% busy, or 0 sessions available) at the 
> moment you are getting overloaded. So if you are 100% busy (or in 
> average no room for new sessions), how to you tell how many sessions 
> nominal (then you need to shed 66%), or be 100% busy at just the 
> nominal load, (then you don't need to shed). This again shows this 
> threshold issue I mentioned. In our implementation, we used to use to 
> a common "utilization" information to do both optimal distribution and 
> shedding, but we moved away from that because of those reasons: we 
> found that each node indicating its requested rejection rate is much 
> more robust and requires very little agreements.
>
> Cordially,
>
> Thierry
>
>
> *Cc:* [email-masked] <mailto:[email-masked]>

Conference information:

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 11 Mar 2011 13:32:59 -0500

Thierry,

The SCE value is en estimate of the number of sessions the device can handle
given the current number of sessions giving respect to the current resource
utilization.  If there are 500 sessions, for example, and the CPU is at 50%
over the past n seconds, then one might assume the device can handle another
500 sessions.  Thus, the SCE might be set to 500.  However, this value is
subject to change every time resources are measured.  There are also "edge
cases" where the current number of sessions and the CPU is at 10%.  In such
cases where arithmetic would not yield a reasonable number, then the device
would need to assume a reasonable value (perhaps based on lab measurements
during development).  I would personally measure each important resource
type and calculate a possible SCE and then take the minimum of those SCE
values.

So, the value is not a ratio, but an estimate of the available capacity.

Now, looking at the peers to an SBC, what we see are multiple SCE values
advertised, such as (a) 400, (b) 600, (c) 800, and (d) 1000.  This tells us
that device D has the highest available capacity.  We could look at those as
"weights" and send a higher portion of traffic toward (d), but we do not.
Our approach is to then round-robin across those devices so as to avoid
overloading a device with too many SIP messages.  After all, just because a
device can handle more simultaneous calls does not mean that it has a higher
message-handling capacity: this is why we would want to complement SCE with
the current SOC work that is focused on message throttling.  Having said
that, nothing would require one to perform round-robin in SCE.  The
algorithm to select the peer given a set of SCE values was not described in
our document (see Section 6 - we leave that open to implementation).

Paul

From: Thierry Bessis [mailto:[email-masked]] 
Sent: Friday, March 11, 2011 11:41 AM
To: Paul E. Jones
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

Paul, 

So this mode of operation is very similar to what we also propose. Is that
number a ratio (percentage of sessions available over the total number), or
an absolute number ? If this is a ratio, then SCE and our load balancing are
compatible: the same information and transport mechanism could be used for
both: A single IETF proposal could be done. 

Cordially, 

Thierry

---

From: Thierry Bessis <[email-masked]>
Date: Fri, 11 Mar 2011 18:22:04 -0600

Paul,

I understand. It seems that the SCE feedback would be as dynamic as the 
it is in the load balancing.
I also agree that SCE could be used to do some load balancing based on 
the available sessions left on each node.

However SCE is using the weight as a representation of the number of 
available sessions, while load balancing is using the weight as an 
utilization ratio. The requirement we used for our load balancing 
requirement is based on the need to balance the utilization ratio on 
each node, (we think the system resists better and longer to 
near-overload situations this way) so the utilization rate must be 
available. In addition to provide what we need, such weight is very 
straightforward to calculate (it's actually more like a measurement): We 
do not have to come up with any assumption, rules or simulations to 
figure out how resources translate to sessions. Another thing is that 
the selection algorithm can be the same as with SRV: Very easy and cheap 
to implement, and still providing continuous load sharing. Also the load 
balancing works great with OLC, but independently: Both algorithm 
operate independently in a serial arrangement (First selection, then 
shedding), and there is no need for a re-selection when the request is 
shed, since the cluster is supposed to be balanced. Here again, the 
implementation can remain modular and very simple.

I agree that SCE can provide similar results, but you need to come up 
with remaining sessions. I agree we could do load distribution, but only 
based on remaining sessions (this probably would also work great, but we 
think load distribution based on utilization rate is better in our case, 
and we know it works). Finally, I think with SCE the algorithm to select 
the final destination when OLC is also used would be more complex.

So I think that the weight should represent the relative occupation of 
the node for those reasons. However we could add an option to provide 
also (or instead) an estimation of the number of sessions available, so 
that those who prefer to distribute based on that instead could do it.
We also did provide for more than one "variant" with OLC ...

Cordially,

Thierry

On 11-Mar-2011 12:32, Paul E. Jones wrote:
>
> Thierry,
>
> The SCE value is en estimate of the number of sessions the device can 
> handle given the current number of sessions giving respect to the 
> current resource utilization.  If there are 500 sessions, for example, 
> and the CPU is at 50% over the past n seconds, then one might assume 
> the device can handle another 500 sessions.  Thus, the SCE might be 
> set to 500.  However, this value is subject to change every time 
> resources are measured.  There are also "edge cases" where the current 
> number of sessions and the CPU is at 10%.  In such cases where 
> arithmetic would not yield a reasonable number, then the device would 
> need to assume a reasonable value (perhaps based on lab measurements 
> during development).  I would personally measure each important 
> resource type and calculate a possible SCE and then take the minimum 
> of those SCE values.
>
> Now, looking at the peers to an SBC, what we see are multiple SCE 
> values advertised, such as (a) 400, (b) 600, (c) 800, and (d) 1000.  
> This tells us that device D has the highest available capacity.  We 
> could look at those as "weights" and send a higher portion of traffic 
> toward (d), but we do not.  Our approach is to then round-robin across 
> those devices so as to avoid overloading a device with too many SIP 
> messages.  After all, just because a device can handle more 
> simultaneous calls does not mean that it has a higher message-handling 
> capacity: this is why we would want to complement SCE with the current 
> SOC work that is focused on message throttling.  Having said that, 
> nothing would require one to perform round-robin in SCE.  The 
> algorithm to select the peer given a set of SCE values was not 
> described in our document (see Section 6 -- we leave that open to 
> implementation).
>
> Paul
>
> *Sent:* Friday, March 11, 2011 11:41 AM
> *To:* Paul E. Jones
> *Cc:* [email-masked]

Conference information:

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 11 Mar 2011 19:38:11 -0500

On 3/11/2011 7:22 PM, Thierry Bessis wrote:

...

> The requirement we used for our load balancing
> each node

Ah!!! Finally I understand what you are up to!
It never made sense to me up till now.

	Thanks,
	Paul

---

From: Thierry Bessis <[email-masked]>
Date: Mon, 14 Mar 2011 15:28:56 -0500

Sorry Paul that it was not apparent from our draft that this was the goal.
This goal has also be chosen because the solution to get to this goal is 
pretty simple.

Cordially,

Thierry

Conference information:

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Mon, 14 Mar 2011 18:20:57 -0400

> From: Thierry Bessis [[email-masked]] 
> 
> If you have read my previous comments, we suggest there is no
> algorithm per se: this is a measurement of the resource: If you
> measure the CPU and the Memory (for example), you'll take the
> smallest. It is better to smooth (with a 10s sliding window average
> for instance) the number.

But "If you measure the CPU and the Memory (for example), you'll take
the smallest" *is* an algorithm.  Its name is "minimum".  There are an
infinite number of other algorithms.  And you do not state in the
draft that you've chosen "minimum".

Also, the particular smoothing chosen is not trivial, and should be
stated in the draft.

> vastly different capacities).

Actually, you want the weight to be (or to be proportional) to the
*unused* portion of your resources, so that an entirely unloaded
system has maximum weight and a totally loaded system has zero weight.

But either way, you don't state this in the draft.

And now I realize you sneaked one past me.  The draft refers to RFC
2782 in a number of places, and I didn't recognize it.  You should say
RFC 3263, which is what people who've implemented SIP recognize.  And
the implication is to replace, in the RFC 3263 process, the "weight"
values in the SRV records with the "weight" values fed back from the
servers.

> mentioned with DNS (where we update the weight every
> minute). Otherwise they are algorithms that can be used to improve the
> oscillation in the feedback chain command. We think that since we are
> proposing a default one that works, this should be OK.

I'd like to see some analysis and modeling to show that this
particular updating strategy works in a wide range of operating
situations.  Otherwise ... unpleasant ... surprises may happen in the
field.

> > But I don't see any of the needed analysis in the draft, and I
> > don't see any indication in the draft of a realization that this
> > analysis needs to be done.  Until that is fixed, I predict that a
> > solution will not be found, or worse, a solution that appears to
> > work on paper but does not work in practice.
> 

You mean it works in your one situation.

Dale

---

From: Thierry Bessis <[email-masked]>
Date: Mon, 14 Mar 2011 18:36:52 -0500

Dale,

> draft that you've chosen "minimum".
I agree that we need to clarify.
Memory and CPU are just typical examples of constrain resources. The 
draft should probably say:
<< The availability of the key Resources required to setup and maintain 
SIP sessions are measured in terms of available ratios.
The weight represents the MIN of those ratios. >>
> Also, the particular smoothing chosen is not trivial, and should be
But the issue is that it may depends on the applications and situations 
(which is part of your point below).
For instance, in our case we found that a flat average sliding window 
over the last 10 seconds works well. But would it be always the case ?
May be an ascending weights average sliding window of 20s may work 
better in some other cases ?
> system has maximum weight and a totally loaded system has zero weight.
Yes, absolutely.
> RFC 3263, which is what people who've implemented SIP recognize.
Actually RFC 3263 refers to RFC 2782, so those people should be familiar 
with RFC 3263.
>   And
We can do that, but 3263 says very little about weights. The explanation 
really is in RFC 3263
> situations.  Otherwise ... unpleasant ... surprises may happen in the
Moving forward, we'll try to do some modeling and simulations for 
different types of situations.
Otherwise, what is the "particular" updating strategy you mentioned: The 
fact that we use an existing  SIP message to convey the feedback ?
> You mean it works in your one situation.
Inside one NE, we have a variable set of different types of SIP servers: 
statefull proxies, stateless proxies, Application servers, servers that 
are exposed to the users, etc.. This may help to build some confidence.
It also works between NEs with DNS, since we are updating the DNS 
weights in SRV the same way.

But yes, we' ve not proved that it could work in any situation.

Cordially,

Thierry

Conference information:

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Tue, 15 Mar 2011 16:06:22 -0400

Inside one NE, we have a variable set of different types of SIP servers: statefull proxies, stateless proxies, Application servers, servers that are exposed to the users, etc.. This may help to build some confidence.
_______________________________________________

Can you describe the benefits you have seen in your cluster (Network Element) from load balancing?

Dale

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 15 Mar 2011 16:34:30 -0400

On Mar 14, 2011, at 7:36 PM, Thierry Bessis wrote:

> It also works between NEs with DNS, since we are updating the DNS weights in SRV the same way. 
> 

We've used the same type of solution as well, but it doesn't work in all situations (or at least not the same way).  In particular, when load-balancing a farm of P-CSCF's, S-CSCF's or subscriber-access SBC's, you have to load-balance *subscribers* not SIP messages/transactions/calls.  In other words, the same subscriber/endpoint had better have all of its SIP messages go to the same P-CSCF/S-CSCF/SBC, or all hell breaks loose. (there're a couple vendors who claim in powerpoint their *-CSCF/SBC's don't need this, but in practice you do, if for no other reason than troubleshooting and monitoring)  And since subscriber capacity/utilization is a different resource measurement from CPU utilization, both sides have to be in on the game and know what mode they're in. (not that's that's a big deal)

But on sip-trunks and peering links you can load-balance per session/out-of-dialog-message this way.  It's just not the same thing as session/call constraints, which is I think what SCE is trying to get instead.

-hadriel

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 15 Mar 2011 16:05:20 -0500

Dale,

What do you mean by "from" load balancing ?

Our main benefit is that our resource utilization (essentially CPU and 
Memory as we discussed) is very good. We see that the system effectively 
load the cluster evenly, and returns to its equilibrium even after some 
"disrupting" events occurred. Such event are for example:  temporary 
loss and then return of a node, or a temporary overload of some nodes 
due to administrative work, or even the dynamic installation of some new 
components on existing hosts. Also, we are dealing with remote nodes 
which are NOT getting (or not willing to use)  the weights and may 
concentrate the traffic unevenly. Then, our mechanism will fix the issue 
using the "good behavior" of nodes which are using the weights to 
re-balance the cluster.

Since all our nodes get overloaded at roughly the same time, we know we 
have roughly the best possible throughput. In addition, based on this 
knowledge, we decide to never re-transmit any message in the same 
cluster when we get the "shedding" signal back: This is a huge advantage.

Does that answer your question ?

Cordially,

Thierry

   Corporate IM: tbessis

Engage: https://engage.alcatel-lucent.com/people/tbessis
My Calendar:   www.google.com/calendar/embed?src=new_frs3175%40yahoo.fr
ALTA Hot Line: http://alta.all.alcatel-lucent.com/hotline

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 15 Mar 2011 16:44:50 -0500

Kaplan,

> We've used the same type of solution as well, but it doesn't work in all situations (or at least not the same way).  In particular, when load-balancing a farm of P-CSCF's, S-CSCF's or subscriber-ac…
This is a good point. We are doing the same thing as you describe: we 
have a farm of P-CSCF and the subcriber/endpoint MUST go to the same 
P-CSCF node once it is registered. But we still successfully use the 
same principle. We have multiple types of settings: P-CSCF and other 
CSCF (like I-CSCF, ...  S-CSCF) running on the same host (and sharing 
the resources), or hosts that are used only for P-CSCF.
It works in both cases because we use the Memory also in the weight. The 
memory is sensitive to both the number of simultaneous instances, and 
the number of registrations. In components that maintains registries 
(such as P-CSCF), additional memory will be consumed by the 
registrations and the memory will become the leading factor in the weights.

So basically what happens is that the weight in DNS will reflect the 
current number of subscribers, and less subscribers will register to 
P-CSCF that have more registrations. (I'm talking about DNS here, 
because of course our SIP load balancing feedback technique will not 
work with subscribers, nor would SCE ! ).

Some people have argued that this is not optimal, since you may have lot 
of CPU available and very few memory, so you could process lot of 
transactions but still your weight is low, so you are not utilizing you 
CPU wisely. In practice though it is still optimal: statistically call 
volumes and registrations go together: we are using such big capacity 
numbers that this is verified.

Cordially,

Thierry

Conference information:

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Wed, 16 Mar 2011 18:48:02 +0530

Thierry,
 
Please read inline.
 
Thanks
Partha

From: Thierry Bessis [mailto:[email-masked]] 
Sent: Friday, March 11, 2011 9:36 PM
To: Parthasarathi R (partr)
Cc: Paul E. Jones; [email-masked]
Subject: Re: [dispatch] SIP load balancing

Paul, 

	For instance, with our approach, it is fundamental that the
weight represents a "Relative" occupation, and not an absolute capacity.

	<Partha> The question is how to perform the "relative" operation
without until understanding the capacity of the peer devices. Say Load
balancer has 3 devices: 2 devices with 1x capacity and 1 device with 2x
capacity. Here all devices will be able to provide the "relative" value
without understanding the capacity of the other device. </Partha>
	
	The first reason is that we found that loading continuously all
the servers at the same utilization rate (same percentage) is the
optimal way of loading a cluster. (We can discuss that if not agreed). 
	<Partha> I agree that same utilization rate is the optimal way
but the question is how to achieve it.  

	Round robin algorithm works in case all devices have same
capacity and same amount of load per call.

	In reality, there are multiple varying factor like call
duration, different vareity of platform within the cluster (say
Generation-1 10 devices and Generation-2 5 devies), different resource
usage for the call, rate at which calls generation. </Partha>

Thierry>>  yes ! 

	The second reason is that a relative occupation is very easy to
derive from information already available: any system knows where it is
in terms of CPU or Memory usage.
	 
	<Partha> Converting CPU or Memory to weight is not explained
your draft. It is the real algorithm which needs lot of discussion.
Because preparing the mapping between CPU & Memory to weight will be
mind-blogging activity </Partha>

relative capacity. In our case, this "algorithm" balances the load of
 
<Partha> In case every device calculate its own algorithm for weight,
then weight from one device may not same as other device. Say dev1 takes
Min(CPU, Mem, DSP) then weight will be calculated as Min (70, 40, 30) =
30% and another device dev2 calculates Min (CPU, Mem) then weight will
be Min (70, 40) = 40% even though they are equally loaded. It is very
important to have the common unit to solve this issue </Partha>  

	 In contrast, deriving a left capacity in terms of remaining
sessions can be difficult: Host can be used by different applications
and perform different duties, which can change over the day, all of them
impacting their true capacity at a certain moment. Beside, as already
discussed, different sessions require different resources, so there is a
whole complication of dealing with a multitude of call profiles, that
(again) are not even constant over the day.
	
	Also, we do not want to derive some shedding decision from the
weight.
	 
	<Partha>  In case you specify, IETF SoC does not allow and so we
want another value, I think that we are in same page.

	If not, I could not understand your argument. As you mentioned,
all system will know CPU or memory usage and it is possible to come up
the static or refreshable value saying that load shed has to happen when
85% of CPU utilization or 90% of memory utilization. The different
between our proposal is that there is no need of two value per message
instead 1 value in each message & 1 value is static or in periodic
manner  </Partha>

Thierry>> We use a single value: see the example above.  
<Partha> I'm proposing is to use single real time value with units like
number of dialog and exchange the threshold (max dialog) using some
other static interface. Let overload decides how to reduce or when to
shed the load based on this information. The advantage in this mechanism
is that only one real time value is required between entities.
  
In your proposed solution, "weight" as one value for load balancing and
"oc" as another value for overload shedding. Pls correct me incase I
misunderstand something here </Partha> 

--

-- 
Cordially, 
Thierry Bessis 
  IMS Solution: Architecture & Product Management - ALTA Member
  Organization: ALU > Carrier group > Wireline BU >  IMS Sol Arch, SE,
PdM

Conference information:
2801 2801 (US):+1 800 771 8734 - (F):+33 800 941 674 - Access Code:
9797989
others countries see: http://all.alcatel-lucent.com/audio

---

From: Thierry Bessis <[email-masked]>
Date: Wed, 16 Mar 2011 12:03:19 -0500

Hi Partha,

> servers have a fixed amount of sessions registers (for instance 50 
> 000). In this case, the weight also reflects that. The algorithm we 
> use for weights is very simple: this is the smallest remaining (in 
> percentage) of each resource that is (1) Required to process a call, 
> (2) Scarce  (not infinite). Let's consider the following system:
> The server has 70% of its memory available for calls, an average of 
> 30% CPU usage (average over the last 10s for instance), and 40% of its 
> Then the weight at this moment is MIN (70, 30, 40) = 30%. You then 
> need to normalize this weight to match the weight range (an INT ? )
>
> Now we are not saying that everybody should calculate their weight 
> like that. What is important is that the weight reflects your current 
> _relative_ capacity. In our case, this "algorithm" balances the load 
> of our servers very well. Now if you use a server that has a fixed 
> amount of resources and that is dimensioned to process let's say 50 
> 000 sessions, at a pace of 100 session/s, then you could probably 
> derive a weight that would depend on both your count of sessions left, 
> and your current rate of session/s.
> then weight from one device may not same as other device. Say dev1 
> takes Min(CPU, Mem, DSP) then weight will be calculated as Min (70, 
> 40, 30) = 30% and another device dev2 calculates Min (CPU, Mem) then 
> weight will be Min (70, 40) = 40% even though they are equally loaded. 
> It is very important to have the common unit to solve this issue 
> </Partha>
TB >> I think here you imply that the DSP is used as a finite pool of 
bearer channels resources, and your are probably using the DSP to 
process bearers. May be you are an SBC or so. Then, it is obvious that 
you should also consider this resource as well, since you need it to 
process new sessions. So you can't say that you have 40% resource 
available to process calls if actually one of your resource needed has 
only 30% available left !
Our recommendation is that the occupation (weight) reflects the MIN of 
ANY resource that must be used for call processing and that is constrained.

I think it is on the contrary an advantage to not specify exactly how it 
is calculated, but to specify exactly what it is, or what it means 
instead (The most constrained resource ratio).
>> system will know CPU or memory usage and it is possible to come up 
>> the static or refreshable value saying that load shed has to happen 
>> when 85% of CPU utilization or 90% of memory utilization. The 
>> different between our proposal is that there is no need of two value 
>> per message instead 1 value in each message & 1 value is static or in 
> <Partha> I'm proposing is to use single real time value with units 
> like number of dialog and exchange the threshold (max dialog) using 
> some other static interface.
TB> This proposal is exactly what we tried to stay away from ! I thought 
I gave lot of arguments in this thread to explain  why working with 
dialogs is at the same time more difficult and not the most 
straightforward information to do resource based load sharing.  
Moreover, using and sharing thresholds and trying to do overload control 
as well from there is also operationally very difficult: We did that 
before and it did not work, because (inside the same company !) nobody 
never agreed about what those threshold were and how to use them. This 
is why we decided to separate load distribution and overload shedding 
(OLC provides a rejection rate, and load distribution provides a 
resource ratio). Our OLC mechanism what the starting point for Vijay's 
effort to lead the IETF OLC draft.
> Let overload decides how to reduce or when to shed the load based on 
> this information. The advantage in this mechanism is that only one 
> real time value is required between entities.
See my comment above: we tried that, and it was very difficult to come 
up with a common approach in all of our (IMS related) products.
Beside, having the destination clearly specifies the requested rejection 
rate is much more convenient and determinist than sharing "session 
available" and threshold information and letting the source node figure 
out what it really means in terms of shedding.
> In your proposed solution, "weight" as one value for load balancing 
> and "oc" as another value for overload shedding. Pls correct me incase 
> I misunderstand something here </Partha>
Yes, but the "oc"  valus is the one already proposed in the OLC draft: 
this is not in THIS proposed solution.

Cordially,

Thierry
>
> --
> -- 
> Cordially,
> Thierry Bessis IMS Solution: Architecture & Product Management - ALTA 
> Member

Conference information:

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Wed, 16 Mar 2011 16:13:08 -0400

More importantly, you need to conceptualize all of these items as *choices*
that have *consequences*, and be able to say, "We *know* what the consequences
of these particular choices are."

Dale

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Wed, 16 Mar 2011 16:22:08 -0400

_______________________________________
From: Thierry Bessis [[email-masked]]

Our main benefit is that our resource utilization (essentially CPU and Memory as we discussed) is very good. We see that the system effectively load the cluster evenly, and returns to its equilibrium even after some "disrupting" events occurred. Such event are for example:  temporary loss and then return of a node, or a temporary overload of some nodes due to administrative work, or even the dynamic installation of some new components on existing hosts. Also, we are dealing with remote nodes which are NOT getting (or not willing to use)  the weights and may concentrate the traffic unevenly. Then, our mechanism will fix the issue using the "good behavior" of nodes which are using the weights to re-balance the cluster.

Since all our nodes get overloaded at roughly the same time, we know we have roughly the best possible throughput. In addition, based on this knowledge, we decide to never re-transmit any message in the same cluster when we get the "shedding" signal back: This is a huge advantage.
________________________________________

How does this compare to the situation that obtains when using static weights?

Dale

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Thu, 17 Mar 2011 04:48:14 +0530

Hi Thierry,
 
Please read inline.
 
Thanks
Partha

From: Thierry Bessis [mailto:[email-masked]]
Sent: Wed 3/16/2011 10:33 PM
To: Parthasarathi R (partr)
Cc: [email-masked]
Subject: Re: [dispatch] SIP load balancing

Hi Partha, 

	Thierry>> We could be more explicit, but should we ? (see next paragraph). Memory and CPU are just the two most obvious examples or resources that you need to process sessions. For instance, some of our servers have a fixed amount of sessions registers (for instance 50 000). In this case, the weight also reflects that. The algorithm we use for weights is very simple: this is the smallest remaining (in percentage) of each resource that is (1) Required to process a call, (2) Scarce  (not infinite). Let's consider the following system: 
	The server has 70% of its memory available for calls, an average of 30% CPU usage (average over the last 10s for instance), and 40% of its registers available (remember that those resources may be shared by other components or devices for any other usage: it does not matter). 
	Then the weight at this moment is MIN (70, 30, 40) = 30%. You then need to normalize this weight to match the weight range (an INT ? ) 
	
	Now we are not saying that everybody should calculate their weight like that. What is important is that the weight reflects your current relative capacity. In our case, this "algorithm" balances the load of our servers very well. Now if you use a server that has a fixed amount of resources and that is dimensioned to process let's say 50 000 sessions, at a pace of 100 session/s, then you could probably derive a weight that would depend on both your count of sessions left, and your current rate of session/s.  
	 
	<Partha> In case every device calculate its own algorithm for weight, then weight from one device may not same as other device. Say dev1 takes Min(CPU, Mem, DSP) then weight will be calculated as Min (70, 40, 30) = 30% and another device dev2 calculates Min (CPU, Mem) then weight will be Min (70, 40) = 40% even though they are equally loaded. It is very important to have the common unit to solve this issue </Partha>  

TB >> I think here you imply that the DSP is used as a finite pool of bearer channels resources, and your are probably using the DSP to process bearers. May be you are an SBC or so.
 
<partha> Here is the interesting part, dev1 achieve the specific service using DSP whereas dev2 achieve the same result using multi core/processor architecture or softDSP or ASIC implementation. IMO, weight varies based on the system architecture in case algorithm is not spelt out correctly or generic units are not defined. </Partha>
 
 Then, it is obvious that you should also consider this resource as well, since you need it to process new sessions. So you can't say that you have 40% resource available to process calls if actually one of your resource needed has only 30% available left ! 
Our recommendation is that the occupation (weight) reflects the MIN of ANY resource that must be used for call processing and that is constrained. 

I think it is on the contrary an advantage to not specify exactly how it is calculated, but to specify exactly what it is, or what it means instead (The most constrained resource ratio). 

		If not, I could not understand your argument. As you mentioned, all system will know CPU or memory usage and it is possible to come up the static or refreshable value saying that load shed has to happen when 85% of CPU utilization or 90% of memory utilization. The different between our proposal is that there is no need of two value per message instead 1 value in each message & 1 value is static or in periodic manner  </Partha>

	Thierry>> We use a single value: see the example above.  
	<Partha> I'm proposing is to use single real time value with units like number of dialog and exchange the threshold (max dialog) using some other static interface. 

TB> This proposal is exactly what we tried to stay away from ! I thought I gave lot of arguments in this thread to explain  why working with dialogs is at the same time more difficult and not the most straightforward information to do resource based load sharing.  
<Partha> dialog is only one of the unit. we will able to work out other unit which acceptable to everybody. As Paul Jones mentioned in another mail thread, SCE is working well in terms H.323 network for lot of years already but it is not working in SIP entity like SBC.   </Partha>
 
Moreover, using and sharing thresholds and trying to do overload control as well from there is also operationally very difficult: We did that before and it did not work, because (inside the same company !) nobody never agreed about what those threshold were and how to use them. 
<Partha> I agree that threshold varies from device to device varies within the same company and huge difference will be seen in case of multi vendor scenario.  It may be tough to manage in case it is provisioned statically as the threshold has to be tweaked based on the solutions. It is the main reason, I proposed to pass the threshold value from device while booting and refresh it whenever required.  
 
Hope we both agrees to have that two variables x,y is required to solve this issue. Load balancer (upstream entity) will use f(x,y) based on its own definition to share the load and shed the load towards single device. The open issue is whether both x & y has to be passed every time or x shall be passed in real time and y shall be passed once in a while. 
 
BTW, I have written Media overload control requirement (draft-partha-dispatch-sip-media-overload-control-00) draft to identify the common problem which will be agreed upon lot of vendors and solve it based on the requirement. 

</partha>
This is why we decided to separate load distribution and overload shedding (OLC provides a rejection rate, and load distribution provides a resource ratio). Our OLC mechanism what the starting point for Vijay's effort to lead the IETF OLC draft. 

	Let overload decides how to reduce or when to shed the load based on this information. The advantage in this mechanism is that only one real time value is required between entities.

See my comment above: we tried that, and it was very difficult to come up with a common approach in all of our (IMS related) products. 
Beside, having the destination clearly specifies the requested rejection rate is much more convenient and determinist than sharing "session available" and threshold information and letting the source node figure out what it really means in terms of shedding. 

	In your proposed solution, "weight" as one value for load balancing and "oc" as another value for overload shedding. Pls correct me incase I misunderstand something here </Partha> 

Yes, but the "oc"  valus is the one already proposed in the OLC draft: this is not in THIS proposed solution. 

Cordially, 

Thierry

	--
	-- 
	Cordially, 
	Thierry Bessis   IMS Solution: Architecture & Product Management - ALTA Member
	  Organization: ALU > Carrier group > Wireline BU >  IMS Sol Arch, SE, PdM

	
	Conference information:

 

Conference information:

---

From: Hadriel Kaplan <[email-masked]>
Date: Wed, 16 Mar 2011 20:39:05 -0400

Hey Partha, 
comments inline...

>> <Partha> In case every device calculate its own algorithm for weight, then weight from one device may not same as other device. Say dev1 takes Min(CPU, Mem, DSP) then weight will be calculated as…

HK> By definition if dev1 calculates 30% and dev2 calculates 40%, then they are *not* equally loaded.  Unless you're just saying "well what if they forgot to take a resource bottleneck into account". (in which case all bets are off anyway - it would be a bug)

> <partha> Here is the interesting part, dev1 achieve the specific service using DSP whereas dev2 achieve the same result using multi core/processor architecture or softDSP or ASIC implementation. IM…

HK> But that's the point - we can't ever write a spec to specify all possible inputs into an algorithm for every possible architecture, because each architecture has its own limitations/bottlenecks.  For example you've mentioned DSP's, and Thierry has mentioned CPU and memory; but there're also SRTP encryption resources, media recording resources, DTMF interworking resources, v4-v6 interworking resources, TCP-media resources, and so on.  In some architectures the resources are the same, in some not.  Some SIP devices have no such resources, some have more.  We can't possibly enumerate them all in a spec.

HK> So instead, the spec will just say: "whatever your limiting factor is at any given time, use it for your numbers".  Only the developers of the systems know what their limiting factors are, after all.

> <Partha> dialog is only one of the unit. we will able to work out other unit which acceptable to everybody. As Paul Jones mentioned in another mail thread, SCE is working well in terms H.323 networ…

HK> It worked in H.323 because everything was a call and consumed equal resources, and you were treating it like a PRI with a fixed/configured number of channels, and things rarely changed.  Arguably that's frequently also true for SIP across SIP-trunks.  Unfortunately, that's the *only* case it works well in, afaict.  Inside the service provider and peering between them things aren't that "clean" (or restricted, depending on how you view the world).  It's much harder to have fixed limits of dialogs/channels/calls between a big mesh network of systems, where the members of the mesh (the number of discrete systems/blades) grow and shrink relatively frequently.  And it's gotten even worse with SUBSCRIBE dialogs and other non-call/non-register traffic consuming resources.  That's not to say it isn't possible to do, but it ends up wasting resources because you can't take advantage of statistical multiplexing, and you have to re-configure lot of systems frequently.

-hadriel

---

From: "Paul E. Jones" <[email-masked]>
Date: Thu, 17 Mar 2011 06:19:51 -0400

Hadriel,

SCE did not work for H.323 because H.323 was mostly about call counting 
and using a fixed number of resources.  SCE was designed with the notion 
that calls use different resources, actually.  That's why the SCE value 
does not increase or decrease by one, necessarily.  If the limiting 
resource was DS0s, I'd agree, but that's not the case most of the time.  
That's true of both H.323 and SIP, of course.

Once convergence is reached, SCE values for non-DS0-type calls often 
increase or decrease by one if the device processing the call does not 
handle any part of the media.  The reason is that call state information 
often consumes about the same amount of memory, etc.  This is not always 
the case, of course, so SCE was not designed with the preconceived 
notion that calls did consume an equal amount of resources.  It was 
designed, though, to consider what the average call consumed.

Paul

On 3/16/2011 8:39 PM, Hadriel Kaplan wrote:
>>> <Partha>  In case every device calculate its own algorithm for weight, then weight from one device may not same as other device. Say dev1 takes Min(CPU, Mem, DSP) then weight will be calculated as Min (70, 40, 30) = 30% and another device dev2 calculates Min (CPU, Mem) then weight will be Min (70, 40) = 40% even though they are equally loaded. It is very important to have the common unit to solve this issue</Partha>
>> TB>>  I think here you imply that the DSP is used as a finite pool of bearer channels resources, and your are probably using the DSP to process bearers. May be you are an SBC or so.
>>
>> <partha>  Here is the interesting part, dev1 achieve the specific service using DSP whereas dev2 achieve the same result using multi core/processor architecture or softDSP or ASIC implementation. IMO, weight varies based on the system architecture in case algorithm is not spelt out correctly or generic units are not defined.</Partha>
>
>> <Partha>  dialog is only one of the unit. we will able to work out other unit which acceptable to everybody. As Paul Jones mentioned in another mail thread, SCE is working well in terms H.323 network for lot of years already but it is not working in SIP entity like SBC.</Partha>

---

From: Thierry Bessis <[email-masked]>
Date: Thu, 17 Mar 2011 15:19:17 -0500

Hi Partha,

Thank you for your comments. See my answers..
>> derive a weight that would depend on both your count of sessions 
>> left, and your current rate of session/s.
>> weight will be Min (70, 40) = 40% even though they are equally 
>> loaded. It is very important to have the common unit to solve this 
>> issue </Partha>
> process bearers. May be you are an SBC or so.
> <partha> Here is the interesting part, dev1 achieve the specific 
> service using DSP whereas dev2 achieve the same result using multi 
> core/processor architecture or softDSP or ASIC implementation. IMO, 
> weight varies based on the system architecture in case algorithm is 
> not spelt out correctly or generic units are not defined. </Partha>
TB> It seems to me that this comment just reinforces my point (I hope I 
understand well): We need to let the destination come up its own way 
with its percentage of resource left, because there are so many possible 
implementations that it would be cumbersome and restrictive to impose 
any algorithm. Also just to be sure: The proposed weight is a percentage 
(a ratio): it has no unit. Part of the robustness and universality is 
derived from this characteristic.
>  Then, it is obvious that you should also consider this resource as 
> well, since you need it to process new sessions. So you can't say that 
> you have 40% resource available to process calls if actually one of 
> your resource needed has only 30% available left !
> I think it is on the contrary an advantage to not specify exactly how 
> it is calculated, but to specify exactly what it is, or what it means 
>>> per message instead 1 value in each message & 1 value is static or 
>>> in periodic manner </Partha>
> <Partha> I agree that threshold varies from device to device varies 
> within the same company and huge differencewill be seen in case of 
> multi vendor scenario.  It may be tough to manage in case it is 
> provisioned statically as the threshold has to be tweaked based on the 
> solutions. It is the main reason, I proposed to pass the threshold 
> value from device while booting and refresh it whenever required.
> Hope we both agrees to have that two variables x,y is required to 
> solve this issue.
Not at all ! We don't need any threshold to balance the load. We just 
need a weight: What you want is to balance the weights so that they tend 
to be identical everywhere: why do we need a threshold to do that ?
> Load balancer (upstream entity) will use f(x,y) based on its own 
> definition to share the load and shed the load towards single device.
TB> Again: We are not interested to redefine overload control and load 
shedding in this proposal: The new OLC draft has just be defined to do 
that.  I've explained (and we should probably make it clear in the 
proposal) that both mechanisms work very well together while each has 
its own independent algorithm.
> The open issue is whether both x & y has to be passed every time or x 
> shall be passed in real time and y shall be passed once in a while.
> BTW, I have written Media overload control requirement 
> (draft-partha-dispatch-sip-media-overload-control-00) draft to 
> identify the common problem which will be agreed upon lot of 
> vendors and solve it based on the requirement.
TB> This requirement was an entry for the OLC draft, is that your point ?

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Thu, 17 Mar 2011 15:23:06 -0500

Hi Kaplan,

Thank you for your comment: that's exactly the way I see it ! :-)

Cordially,

Thierry

Conference information:

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Fri, 18 Mar 2011 03:37:12 +0530

Thierry/Hadriel,

My understanding about Thierry proposal from the mail thread is that (downstream) system load is calculated in terms of percentage, percentage is the unit and passed across to load balancer. The normalization to weight is a optional process. 

Please read inline.

Thanks

Partha 

From: Thierry Bessis [mailto:[email-masked]]
Sent: Fri 3/18/2011 1:53 AM
To: Hadriel Kaplan
Cc: Parthasarathi R (partr); [email-masked]
Subject: Re: [dispatch] SIP load balancing

Hi Kaplan, 

Cordially, 

Thierry

	HK> By definition if dev1 calculates 30% and dev2 calculates 40%, then they are *not* equally loaded.  Unless you're just saying "well what if they forgot to take a resource bottleneck into account". (in which case all bets are off anyway - it would be a bug)
	<partha> It is not required to be bug in the system but may be misinterpretation of the unit termed as "weight" between two systems. I'll explain with real time example:

	In case Person X report his weight as 150 then I'll assume that person X is overweight(may be looks like giant) because IMO, weight above 100 for a person is high. Actually, Person X is not overweight in case he mentioned his weight in terms of pounds(lb) but I assume with my standard metric of Kilogram(kg) for weight. Hope you agree here that unit has to be spelt out explicitly or else, it possibly leads to misinterpretation. 

	This may happen in the system implementation as well because of the multiple resource usage. Each implementator will come up with his own unit under the name of weight which confuses load balancer in case metric is not spelt out properly. </Partha> 

	

	
	-hadriel

Conference information:

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 24 Mar 2011 15:54:59 -0400

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Thierry Bessis [[email-masked]]

TB> It seems to me that this comment just reinforces my point (I hope I understand well): We need to let the destination come up its own way with its percentage of resource left, because there are so many possible implementations that it would be cumbersome and restrictive to impose any algorithm. Also just to be sure: The proposed weight is a percentage (a ratio): it has no unit. Part of the robustness and universality is derived from this characteristic.
________________________________________

I haven't done an analysis, but I strongly expect that the overall system behaves better if all systems report their available capacity
in the same amount of traffic ("number of calls") rather than percentage of capacity that is unused.

The reason is that if two servers have significantly different capacity, say by a factor of 10, when both servers report 1% of capacity
free, one server will be able to accept 10 times as much traffic as the other.

Dale

---

From: Thierry Bessis <[email-masked]>
Date: Mon, 28 Mar 2011 16:04:50 -0500

Dale,

This reasoning makes a lot of sense, but the best strategy is described 
in one particular scenario, using one information that we would not 
have, and another that would be approximate at best.

At best you could estimate the number of sessions left (the other 
proposal), but then, we have no ideas of the percentage of resources 
left (Still the other proposal). So if we have an node that has (or 
think it has  ! ...) 10 times more capacity , starting from idle, the 
other proposal will basically not use the smallest node until the 
biggest gets close to overload, which is something that we (and at least 
some operators) do not like.

  Now in the situation that you describe, you are correct that one 
server would have 10 times more capacity, so we are going to an 
unbalanced situation. But this is how you got there in the first place: 
the next weight in the smallest node will be zero, while it would be 
still 1% in the biggest one, so this will (again) tend to balance.

Also, I believe we do not need to be optimal in extreme cases such as 
this one (99% utilization and one node 10 times the capacity of the 
other). This is especially true when you consider that nodes could have 
different views of their "sessions left" so the theoretical "optimal" 
might not be optimal at all.

Cordially,

Thierry

Conference information:

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 28 Mar 2011 17:35:39 -0400

ISTM that it doesn't matter much what algorithm you use until at least 
one node starts to approach full capacity. (And this includes not 
mattering if all are at the same percentage utilization. Is there 
anything wrong with the 10x box getting all the traffic for awhile 
before the 1x box gets any, as long as all the traffic is being served?)

What matters is what happens when you start to approach full capacity.

Even then, maybe it doesn't matter whether you get to the point where 
either all the boxes are at 99% capacity, or all boxes have room for one 
more call of any type.

But it does matter if 1% available capacity is enough for 100 calls, or 
if 1% capacity is insufficient for even one more call.

I'm starting to suspect that an algorithm that works for boxes that 
handle 10k calls may not work so well for boxes that handle 100 calls.

	Thanks,
	Paul

On 3/28/2011 5:04 PM, Thierry Bessis wrote:
> Now in the situation that you describe, you are correct that one server
> would have 10 times more capacity, so we are going to an unbalanced
> situation. But this is how you got there in the first place: the next
> weight in the smallest node will be zero, while it would be still 1% in
> the biggest one, so this will (again) tend to balance.
>
>> From:[email-masked]  [[email-masked]] On Behalf Of Thierry Bessis [[email-masked]]
>>

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 29 Mar 2011 03:37:32 -0400

> free, one server will be able to accept 10 times as much traffic as the other.

I don't think anyone's proposing to advertise a literal percentage of capacity, for obvious reasons. (ie, the device isn't advertising the numbers 0-100 to mean 0-100%)

But anyway, although in theory the algorithm auto-adjusts in such a case regardless - in practice my guess is the operators of the systems would configure the literal starting/max-number based on the capacity relative to other systems, so that you wouldn't really have a case of two systems of very different capacity starting out advertising the same values.

-hadriel

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Tue, 29 Mar 2011 14:37:01 +0530

Hadriel,

In your proposed algorithm, Two information is required for load
balancing:

1) Max or total capacity is exchanged by non-sip mechanism (may be
configuration) one time or frequently in case capacity of the device
varies 
2) Current capacity of the system is passed in each message. 

It is the place where I'm looking for units like "number of calls" or
"number of media packets" to be clearly spelt out in the mechanism
because percentage as a unit will not serve the purpose and finding the
relation between two or more system capacity is really tough (to
configure or manage).

Thanks
Partha 

-----Original Message-----
From: Hadriel Kaplan [mailto:[email-masked]] 
Sent: Tuesday, March 29, 2011 1:08 PM
To: Worley, Dale R (Dale)
Cc: Thierry Bessis; Parthasarathi R (partr); [email-masked]
Subject: Re: [dispatch] SIP load balancing

On Mar 24, 2011, at 3:54 PM, Worley, Dale R (Dale) wrote:

> I haven't done an analysis, but I strongly expect that the overall 
> system behaves better if all systems report their available capacity
in the same amount of traffic ("number of calls") rather than percentage
of capacity that is unused.
> 
> The reason is that if two servers have significantly different 
> capacity, say by a factor of 10, when both servers report 1% of
capacity free, one server will be able to accept 10 times as much
traffic as the other.

I don't think anyone's proposing to advertise a literal percentage of
capacity, for obvious reasons. (ie, the device isn't advertising the
numbers 0-100 to mean 0-100%)

But anyway, although in theory the algorithm auto-adjusts in such a case
regardless - in practice my guess is the operators of the systems would
configure the literal starting/max-number based on the capacity relative
to other systems, so that you wouldn't really have a case of two systems
of very different capacity starting out advertising the same values.

-hadriel

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 29 Mar 2011 05:35:08 -0400

> configuration) one time or frequently in case capacity of the device

Nope - it's not signaled separately.  I was talking about the number used as the baseline/reference/100% number value.  So think of it like the call-capacity: if we signaled a call-capacity, all I'm saying is that the max call-capactiy (the capacity when the system has no current calls) would be configured by the admin to begin with, rather than hardcoded in the system.  But in this case we're not talking about a call-capacity... we're talking about a "resource load" or some such. (which may happen to be a call capacity in someone's system, but doesn't have to be)

> "number of media packets" to be clearly spelt out in the mechanism

And I'm saying there's no way we could ever agree on all the possible set of constraints which affect system load - so we could never send a unit like "number of calls" or "number of media packets", because there are just way too many such units which have different impacts on different system types.

> relation between two or more system capacity is really tough (to

Just to be clear we're on the same page - it's not really a "percentage unit".  A percentage unit would be something like 73.4%.  No one's proposing signaling "73.4%".  

-hadriel

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 29 Mar 2011 07:41:53 -0400

> But anyway, although in theory the algorithm auto-adjusts in such a case regardless - in practice my guess is the operators of the systems would configure the literal starting/max-number based on t…

Its possible to abuse anything. But we're in trouble if we start out 
assuming that this be used contrary to the design intent.

If people scale their use of percent capacity so that the big box maxes 
out at 100 and the smaller box maxes out at 50, then it has been 
transformed into something like the SCE mechanism, that have quite 
different properties. If that is what we expect people to use then we 
should design and analyze it on that basis.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 29 Mar 2011 08:16:44 -0400

inline

> Just to be clear we're on the same page - it's not really a "percentage unit".  A percentage unit would be something like 73.4%.  No one's proposing signaling "73.4%".

Well...

IIUC, Thierry *is* talking about a "percentage unit", but rescaled from 
[0..100) to [0..2^16-1].

I'm sure he'll correct me if I have that wrong.

	Thanks,
	Paul

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 29 Mar 2011 10:16:00 -0500

Kaplan,
> I don't think anyone's proposing to advertise a literal percentage of capacity, for obvious reasons. (ie, the device isn't advertising the numbers 0-100 to mean 0-100%)
Actually we do !  This is what we are doing in our system. (we really 
are using a byte [0..255] but it means the exact same thing).
We are dealing with cases where some nodes may have 2-3 times the 
capacity of others due to hardware generation gap. (but not 10 times ... 
). we are also dealing with growing/degrowing, and temporary partial 
capacity loss: it works well.
> But anyway, although in theory the algorithm auto-adjusts in such a case regardless - in practice my guess is the operators of the systems would configure the literal starting/max-number based on t…
This is also what we are trying to avoid: having the operator need to 
compare nodes capacity (possibly from different vendors), and come up 
with complicated engineering rules to configure the weights (rules that 
additionally would need to change each time something is modfied in the 
network).  All our nodes advertise about 100% availability when they 
start, whatever their capacity (unless they were also used for other tasks)

Cordially,

Thierry

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 29 Mar 2011 10:29:00 -0500

All,
>> Just to be clear we're on the same page - it's not really a 
>> "percentage unit".  A percentage unit would be something like 73.4%.  
>> No one's proposing signaling "73.4%".
>
> Well...
>
> IIUC, Thierry *is* talking about a "percentage unit", but rescaled 
> from [0..100) to [0..2^16-1].
You completely correct Paul. Whatever the scale we decide to use, I 
proposed a percentage. It is the ratio of the number of available 
resources on the total number of resources.
Vijay will make it clear in the next draft, and we will probably propose 
to use directly a percentage [0 ..100] to make things easier.
We think that a direct percentage should be enough in terms of 
precision, because the resource usage precision is also in this order of 
magnitude.

cordially,

Thierry

Conference information:

---

From: Thierry Bessis <[email-masked]>
Date: Tue, 29 Mar 2011 10:42:03 -0500

>> But anyway, although in theory the algorithm auto-adjusts in such a 
>> case regardless - in practice my guess is the operators of the 
>> systems would configure the literal starting/max-number based on the 
>> capacity relative to other systems, so that you wouldn't really have 
>> a case of two systems of very different capacity starting out 
>> advertising the same values.
>
> If people scale their use of percent capacity so that the big box 
> maxes out at 100 and the smaller box maxes out at 50, then it has been 
Yes, I agree with Paul's point.
The whole point of our proposal is two folds: (1) simplicity 
(straightforward weight calculation) and (2) NO dependency (No network  
engineering or topology considerations to setup weights: each node 
provides its weight independently).

Cordially,

Thierry

Conference information:

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 29 Mar 2011 18:58:15 -0400

No you aren't.  It's a scaled number.  That's a big difference.
Why?  Because it allows the operator to do it either way - they can let all devices start at any number and converge using the algorithm dynamically, or they can provision it.

And if they provision it: (1) the number range is large enough that they can provision it once, and (2) even if they get it wrong or not quite right it converges anyway.  By "provision it once", I mean they don't have to change the setting for installed systems when they add new systems of larger capacities, or if their installed systems decrease in capacity as SIP gets more chatty (ie, as a single session/call now takes more SIP messages than it did when they first installed the systems, which is a common problem).

-hadriel

On Mar 29, 2011, at 11:16 AM, Thierry Bessis wrote:

Kaplan,

Actually we do !  This is what we are doing in our system. (we really are using a byte [0..255] but it means the exact same thing).
We are dealing with cases where some nodes may have 2-3 times the capacity of others due to hardware generation gap. (but not 10 times ... ). we are also dealing with growing/degrowing, and temporary partial capacity loss: it works well.

This is also what we are trying to avoid: having the operator need to compare nodes capacity (possibly from different vendors), and come up with complicated engineering rules to configure the weights (rules that additionally would need to change each time something is modfied in the network).  All our nodes advertise about 100% availability when they start, whatever their capacity (unless they were also used for other tasks)

Cordially,

Thierry

-hadriel

_______________________________________________
dispatch mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dispatch

--
--
Cordially,
Thierry Bessis
  IMS Solution: Architecture & Product Management - ALTA Member
  Organization: ALU > Carrier group > Wireline BU >  IMS Sol Arch, SE, PdM
[file:///C:/Documents%20and%20Settings/tbessis/My%20Documents/al_corp_h_3d_16mm.jpg]
  1960 Lucent Lane - Room 6L-517
  Naperville, IL  60563  U S
  Tel/Fax: +1 630 979 7989
  Corporate IM: tbessis

Engage: https://engage.alcatel-lucent.com/people/tbessis
My Calendar:   www.google.com/calendar/embed?src=new_frs3175%40yahoo.fr<http://www.google.com/calendar/embed?src=new_frs3175%40yahoo.fr>
ALTA Hot Line: http://alta.all.alcatel-lucent.com/hotline

Conference information: