Subject: [ippm] statistics techniques

From: "gab.seun jones.ewulomi" <[email-masked]>
Date: Tue, 18 Feb 2003 20:40:36 +0000

Hi Guys,

My apologies on the somewhat easy question

Im doing a active test just using ping(via script). I have the script to 
ping a device(100 counts) then send it to a file.

My main question is what will give a more accurate rtt.

1)taking the mean of all 100 pings
2)median
3)mode

Just want techniques on what others would use and why

regards,
gab

_________________________________________________________________
Surf together with new Shared Browsing 
http://join.msn.com/?page=features/browse&pgmarket=en-gb&XAPID=74&DI=1059

_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: stanislav shalunov <[email-masked]>
Date: 18 Feb 2003 15:46:58 -0500

> 1)taking the mean of all 100 pings

How can you compute the mean when some of the values are close to
infinity (losses)?

Median and minimum make a lot more sense.

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

A fanatic is one who can't change his mind and won't change the
subject.                                   -- Winston Churchill
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: Kavé Salamatian <[email-masked]>
Date: Tue, 18 Feb 2003 23:28:46 +0100

>How can you compute the mean when some of the values are close to
This might not be a problem. There is a lot of methods for dealing with
missing values. However there is a more fundamental question. What do you
expect from the mean or the median or the mode ? Sometimes trivial questions
are the most difficult to answer. For example suppose that you have measure
the rtt for 100 counts, maybe you are searching for a formula f(.) that will
combine the results obtained in the past measurements to predict the rtt in
measurement 101. If this is your goal clearly the mean or mode or median is
not the best estimator. If your problem is to find a parametric
representation of the rtt distribution that have as parameter the mean,
please define initially what a prior parametric structure are you supposing.
If your problem is something else please define it.

Sometime, when I read some papers or draft in Internet measurement it remind
me of the following oriental story:
Somebody had lost his key in a dark yard but he was wandering about the key
in the room. A fellow ask him: you have lost the key in the yard but you are
searching it here. The guy answer: but there is light here and the yard is
dark....

Regards

Kavé Salamatian
Associate Professor, LIP6-UPMC

---

From: stanislav shalunov <[email-masked]>
Date: 18 Feb 2003 18:06:50 -0500

I can not speak for the original poster, but since you appear to be
addressing me while (not?) answering his questions:

1. The values of delay for lost packets are not ``missing''.  They are
   potentially very large unknowable values.  That packet that I sent
   last year and that appeared to be lost might still be delivered one
   day.

2. The necessity for aggregation is more about human consumption of
   data than about using it to predict anything or to compute
   parameters of a distribution (there's no distribution).  Minimum
   (or something close to it) is meaningful in itself: given enough
   measurements, it's a reasonable guess of propagation delay.  Median
   is useful for several reasons:

   a. It reflects the whole sample.

   b. It is well-defined for samples where more than 50% of packets
      arrived during the measurement (thus making it widely usable).

   c. It is robust.

   d. It is easy to understand.

---

From: Sandrine Vaton <[email-masked]>
Date: Wed, 19 Feb 2003 13:23:56 +0100

>    last year and that appeared to be lost might still be delivered one

The word "missing" should be understood in the sense of "unobserved" but
in the sense of "censored". In statistics a model with "censored data" is
a model such that is X>xmax you do not observe X but you anyhow have the
information that X was superior to xmax. This is a classical problem in
the industry for example or in polling. In your case the actual value of
delay is not known when the delay is too big but you know that (delay >
delay_max): this is already some information on the value of the delay and
this should be taken into account in your estimation procedure.

For censored values problems a powerful algorithm is the Expectation
Maximization algorithm. You can also use Baysian techniques to simulate
the value of the delay on the information that the delay is superior to
delay_max. In both cases you suppose a statistical model for your series
of delays and you try to estimate the parameters of that model (mean,
variance, etc...) on the basis of a series of delays.

>       arrived during the measurement (thus making it widely usable).

I do not agree with point b. If only 50% of the packets were delivered and
if you estimate the mean delay as the mean over the packets delivered then
you will strongly underestimate the average delay (you do not take into
account those packets that are not delivered yet but that could be
delivered one day).

>    c. It is robust.

In what sense?

>    d. It is easy to understand.

Sandrine Vaton
http://perso-info.enst-bretagne.fr/~vaton

---

From: stanislav shalunov <[email-masked]>
Date: 19 Feb 2003 22:01:49 -0500

Sandrine Vaton <[email-masked]> writes:

> in the sense of "censored".

And, if you know that the delay was greater than some value and have
no knowledge of the distribution, what do you do?

> >    Median is useful for several reasons:

> I do not agree with point b. If only 50% of the packets were
> delivered and if you estimate the mean delay as the mean over the
> packets delivered

Sure.  That's why I talk about _median_, not mean.

> In what sense?

It is not significantly affected by small amounts of noise.

Sex is the mathematics urge sublimated.                 -- M. C. Reed.
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: stanislav shalunov <[email-masked]>
Date: 20 Feb 2003 12:07:11 -0500

Sandrine Vaton <[email-masked]> writes:

> > > SV -> Hum hum I did not get that point. That looks tricky. But,
> > > unless you are in a model with only one parameter (an exponential
> > > distribution for example) the median will not be sufficient to fully
> > > determine the distribution.
> >
> > As would be the mean.
> >
> > But, I didn't even say I make any assumptions about the model.
> 
> The problem is that most of the time you need more than one parameter to
> characterize a distribution. For example for a Gaussian distribution you need
> the mean and the variance, unless you suppose that there is a relationship
> between mean and variance (for example variance=mean^c with c=...)

How does this make mean relevant?  I am saying that it is a better
practice to report median of delays than attempt to report mean of delays.

> > > SV -> This is true when the noise is additive with a symetric distribution
> > > around the origin.
> >
> > Suppose 1% of measurements is distorted (by a very large margin).
> > Median is unlikely to be affected.  Mean is.
> 
> The median will be affected if you have more than 50% of "censored"
> RTT. I agree with you for saying that with only 1% then the median
> is not affected.

If more than 50% of packets are lost, you ought to report median=infinity.
If any packets are lost, you ought to report mean=infinity.

Which one is better?

> But once again the median is maybe not the only parameter of
> interest. In particular, the distribution of the delays is clearly
> not exponential, is it?

It is unlikely that packet delay is a random quantity from a
statistical point of view.  (There's no probability space, there's a
feedback loop, etc.)  Certainly different measurements (taken close
enough) are not independent.

If one were to model delay as an i.i.d. random quantity, then shifted
exponentially distributed delay (two parameters) might be appropriate
for certain regimes, but polynomial distribution (perhaps
1/(a*(x-b)^3), two parameters again) seems better supported by
evidence.  Others suggested gamma-distribution and a host of
heavy-tail distributions.

There are certainly at least two independent quantities: the
propagation delay and some measure of the queuing delay.  I like
minimum and median to represent these two.

A fool's brain digests philosophy into folly, science into superstition,
and art into pedantry.  Hence University education.       -- G. B. Shaw
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: Jing Shen <[email-masked]>
Date: Fri, 21 Feb 2003 10:43:45 +0800

The RTT value is at least related to two factors: the time of
measurement 
and the status of network. To some of the measurement, RTT reaches
its bigest value at 9-10 o'clock and 15-16 o'clock, and the lowest value
is reached at 3-5 o'clock. So it's self-similar, and from viewpoint of
statics mathematical expectation should be value we need. 

In a 100 sample points measurement, it may be a better choice to 
take the median value if the measurement time is fixed.

-- 
Jing Shen

**********************************************************************
* The SunShine of life is made up of very little beams which is      *
*  bright all the time                                               *
**********************************************************************
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: Sandrine Vaton <[email-masked]>
Date: Fri, 21 Feb 2003 14:15:04 +0100

> practice to report median of delays than attempt to report mean of delays.

SV -> I think that we do not understand each other. What I was saying is that only
*one* parameter is generally not enough to characterize the distribution of the
delays.

> If any packets are lost, you ought to report mean=infinity.

SV -> No. There ARE ways of estimating the finite mean (or median) even if a
majority of packets are lost. And the estimate will not be infinite
(fortunately...). For example the proportion of packets for which you do not have
the rtt gives you a lot of information on the parameters of the distribution of
delays (this proportion is a quartile of the distribution).

> enough) are not independent.

SV -> Here we come to an agreement. The successive measurements should not be
considered as independent. I am completely in favor of tha point of view and I have
been defending that point of view for years!

From my simulations I even found out that most of the time a Hidden Markov Model
(HMM) was a valuable approach. Broadly speaking the channel can be considered as a
finite state random machine (a Markov chain) with 4 or 5 states and the distribution
of the delay depends on the state of the channel at present.

This enriched model should not prevent you from defining a distribution for the
delay. In the case of a HMM this distribution is a weighted mixture of the component
distributions (the weights being defined by the stationary regime of the
"underlying" Markov chain, that is to say the the channel).

See my Web page http://perso-info.enst-bretagne.fr/~vaton for a series of
publications in that sense. I also made available some software that I wrote for
training the parameters of a HMM on a series of packet loss as well as some MCMC
algorithms for various applications in network characterization.

> heavy-tail distributions.

You can choose any law you want! You can even define the distribution by an
analytical formula even if it does not correspond to a well known distribution   (
-x^2 cos(x) + 3 x sin (x)+3800 is not forbidden if you feel that it better fits your
empirical distribution...)

Kind regards,

Sandrine Vaton.