Subject: [lmap] milliseconds in ma-periodic-interval and ma-event-random-spread

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 11 Feb 2016 16:55:59 +0100

Hi,

the timing objects use milliseconds for (a) the periodic timer
interval and (b) for the random spread. Since timing objects are used
to start the execution of schedules (which then trigger tasks) and
they are not used for packet timings (as this happens inside of
measurement tasks), I find the millisecond resolution (i) awkward to
read and configure, (ii) somewhat painful to implement (can't simply
use a time_t internally), and (iii) pretty useless (since there will
be some additional unpredictable delay until a task has been started
and a first packet his a wire).

Note also that timing events have a maximum resolution of
seconds. Hence, my proposal is to change ma-periodic-interval and
ma-event-random-spread from milliseconds to seconds.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 11 Feb 2016 11:41:35 -0500

Hi Juergen,

Brief reply with reservations on your proposal, apologies in advance.

Many active measurements are conducted using periodic packet streams
at integer second intervals, and many use 1 second inter-packet intervals.
Or, when emulating VoIP, 20ms inter-packet intervals.
Integer random spreads don't help these cases.

The intent of the random spread is to assist multiple streams from aligning.
We aren't just talking about 1 sender here. Large Scale.

Even if some implementations have an unpredictable delay after the
task is initiated, others could anticipate the schedule and 
prepare a measurement process so it actually begins sending 
when scheduled.

Perhaps we can work out another solution, but I feel that 
changing to integer random spreads would be a mistake.
http://tools.ietf.org/html/rfc3432#section-3

Al

> -----Original Message-----
> From: lmap [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Thursday, February 11, 2016 10:56 AM
> To: [email-masked]
> Subject: [lmap] milliseconds in ma-periodic-interval and ma-event-
> random-spread
> 
> Hi,
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 15 Feb 2016 12:16:20 +0100

Al,

this is the scheduler starting measurement tasks, it is _not_ the task
internal scheduler that sends packet streams. Some large existing
measurement systems use cron for this purpose and cron does at most
second granularity (at least those that I have seen in my life so
far). This is _not_ about inter-packet intervals etc.

/js

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 15 Feb 2016 07:31:07 -0500

Hi Juergen,
I understand the difference between these schedulers quite well.

If we both send one packet per second, and we both use random
Integer second offsets for our start times, say 12:00:01 and 12:00:07,
then we both send packets at the same time on the integer second
chime (after 6 seconds).  As I said below, we cannot count on random
time to start a test after the schedule says 'go'. 

The random offset is to deliberately avoid synchronization in cases like this.

I get it, this is harder to do.
Al
________________________________________
From: Juergen Schoenwaelder [[email-masked]]
Sent: Monday, February 15, 2016 6:16 AM
To: MORTON, ALFRED C (AL)
Cc: [email-masked]
Subject: Re: [lmap] milliseconds in ma-periodic-interval and ma-event-random-spread

Al,

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 15 Feb 2016 14:42:50 +0100

> time to start a test after the schedule says 'go'.

I do not understand your example. What are 12:00:01 and 12:00:07?
One-off absolute times without ma-event-random-spread? Or your concern
is that both packets will be aligned to the systems' notion of a
'complete' second? Well, not really true, since the event is going to
start a task and the likelihood that the tasks start at exactly the
same time and get their first packet on the way is small. Again, I
question that it is realistic to schedule tasks to start once per
second in order to send one packet.

> The random offset is to deliberately avoid synchronization in cases

The question is whether the granularity needs to be milliseconds. I
did not question to have ma-event-random-spread. Note that regular
crons do not even go to second resolution, they stop a minutes. OK,
this is not a strong argument but perhaps the disconnect here is what
kind of tasks we ultimately assume to be scheduled.

Anyway, my original proposal was primarily about ma-periodic-interval.
If people feel strongly that the ma-event-random-spread should remain
with milliseconds resolution, I can probably live with that. Having
ma-periodic-interval at milliseconds resolution is however more of a
pain to implement.

/js

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 15 Feb 2016 18:37:35 -0500

Hi Juergen, please see [ACM] replies below.
Al

> From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-
> university.de]
> Sent: Monday, February 15, 2016 8:43 AM
> One-off absolute times without ma-event-random-spread? 
[ACM] 
Sorry, I was working on a soft keyboard during the break between appointments.
I meant that 12:00:00 (HH:MM:SS) is the start time of the measurement interval (T0)
and 12:00:01 is a random start time selected from the interval of 
allowed start times, 12:00:00 to 12:00:00 + dT, where dT = 00:00:10 (10 seconds)
and the specific random value is 00:00:01 in this case.

12:00:07 is another start time, where the specific random value is 00:00:07 (7 seconds).

> Or your concern
> same time and get their first packet on the way is small. 
[ACM] 
It is an important point that the Metric Parameters describe when the 
measurement stream starts and stops.  The housekeeping time to start 
a measurement process and arrange for a measurement peer to receive the
stream should take place in advance of the start of the measurement 
interval (when the first measurement packet is sent on the link).
That's what is meant by the time parameters here
https://tools.ietf.org/html/draft-morton-ippm-initial-registry-03#section-8.3.5
(ignore the data format, please).

> Again, I
[ACM] 
I do too, because I don't understand why anyone would do that.
The measurement system is supposed to send one packet per second
in my example -- one sending process would send, wait, send, wait, ...

So, we have two measurement streams, sending 1 packet per second as follows:

Stream 1         12:00:01, :02, :03, :04, :05, :06, :07, :08, :09, ...
Stream 2   <waiting for its random start time> 12:00:07, :08, :09, ...

and the streams synchronize on one second clock ticks.

> Anyway, my original proposal was primarily about ma-periodic-interval.
[ACM] 
I can live with one second resolution on ma-periodic-interval,
as long as there is millisecond resolution on ma-event-random-spread.

> ma-periodic-interval at milliseconds resolution is however more of a
[ACM] 
I understand, but it was possible when we implemented RFC 3432,
back before it was published and in draft stage.

thanks,
Al

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 22 Feb 2016 13:38:16 +0100

> and the streams synchronize on one second clock ticks.

We need to distinguish the start of the task(s) creating streams and
the timing of the streams generated by the tasks.

> back before it was published and in draft stage.

Is RFC 3432 scheduling the invocation of arbitrary tasks or is it
about sending packets from within a measurement task?

The LMAP scheduler is by design a coarse grained scheduler, if you
need precision for your packet trains, you have to implement that fine
grained scheduling in the measurement tasks (to minimize the many
possible sources of extra delays).

t0	 	scheduled invocation as per instruction
t1=t0+ds	invocation of the scheduled action
t2=t1+do	measurement task starts running
t3=t2+dm	time first packet is given to the OS kernel
t4=t3+dk	time packet hits the wire

To achieve reasonable precision, you have to control t3 (t4 is usually
hard without special OS support) from within the measurement task.

ds = delay by the lmap scheduler (which is also depending on the load
     of the OS)

do = delay by the OS to load and start a measurement task (which is
     also depending on the load of the OS)

dm = delay by the measurement task (time to initialize and generate the
     first test packet)

dk = delay by the kernel to move the packet on the wire (may include
     media access specific delays)

My point is that making the LMAP scheduler highly precise is a
non-goal since ds+do+dm may add notable delays. The only way to
achieve precision where it is needed is to not depend on the LMAP
scheduler but to do the fine grained scheduling inside the measurement
task.

/js

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 22 Feb 2016 11:52:28 -0500

Hi Juergen, 
please see reply appended below,
Al

> Sent: Monday, February 22, 2016 7:38 AM
[ACM] 
You've sorted-out the schedule, task, and measurement packet timing 
very nicely, and though it appears we've been talking about different
schedules/processes, this overlap has existed for quite some time.

I don't think it's useful to apply random time offsets to the 
   t0	 	scheduled invocation as per instruction

but we do need to pass t3
   t3=t2+dm	time first packet is given to the OS kernel

and although dm will have a random component
   dm = delay by the measurement task (time to initialize and generate the
        first test packet)

we need to pass a random offset to the measurement task that governs
t3 to the extent possible. If we prefer, we could pass a time range for
the random offsets and the measurement task selects the actual value
(which could be a 10 second range, for example, with millisecond
resolution).

We also need to ensure that t3 is achievable by setting t0 far enough 
  > t2=t1+do	measurement task starts running

The time needed between t2 and t3 may be significant, to account for
a control protocol to arrange the tests with a Measurement Peer,
as may be needed in OWAMP, TWAMP, and others.

Al

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 22 Feb 2016 18:00:17 +0100

>    t0	 	scheduled invocation as per instruction

But this is what ma-event-random-spread is all about. I think it is
useful to have so that you can schedule report tasks randomized at a
course grained level. Again, seconds resolution is what I find
sufficient. The idea is to have the same event configured on 1000s of
devices to trigger reporting but they do not all report at the same
time; the spread may in the order of an hour.
 
> (which could be a 10 second range, for example, with millisecond

This sounds like just like one more parameter to pass to the task, not
something we need to take special care of at the info/data model.

/js