Subject: [lmap] some lmap result report model changes

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 18 Mar 2016 11:54:07 +0100

Hi,

I am starting to implement the result reporting parts of the data and
information model and I think there is room for improvements.

1) I think we report results from action and not from tasks. To make
   this clear, I propose to change ma-report-task-obj to
   ma-report-action-obj and to refer to an action name instead of a
   task name. Perhaps it is good to ship the name of the task the
   action is associated with as part of the ma-report-action-obj.

2) It is almost impossible to implement 'conflict' on a per row
   level. The lmap scheduler can likely track which actions were
   running during the execution of an action but it surely does not
   know the internals of the action and hence it can't know how to
   determine conflicts on a per row level. Likewise, the measurement
   action does not know how to determine what else is going on on the
   system and related to the lmap framework. Hence, the conflict
   should be reported as part of ma-report-action-obj and not as a
   result row property.

3) A measurement action may produce different result 'tables'. The
   assumption that there is a single header followed by a set of rows
   is too limited. We need to allow for a set of result 'tables'
   instead of a single one.

4) The description of ma-report-row-start-time and
   ma-report-row-end-time says that this timestamp is the start and
   the end of the action. If so, they should be reported as part of
   the ma-report-action-obj and not as part of the ma-report-row-obj.

5) I think the two lists of options (ma-report-task-options and
   ma-report-task-action-options) should be collapsed into a single
   combined list ma-report-action-options (since an action receives a
   single combined list of options).

With all of this we get the following structures at the information
model level:

     object {
         datetime              ma-report-date;
        [uuid                  ma-report-agent-id;]
        [string                ma-report-group-id;]
        [string                ma-report-measurement-point;]
        [ma-report-action-obj  ma-report-actions<0..*>;]
     } ma-report-obj;

     object {
         string                  ma-report-action-name;
         string                  ma-report-action-task-name;
        [ma-metric-registry-obj  ma-report-action-metrics<0..*>;]
        [ma-option-obj           ma-report-action-options<0..*>;]
        [string                  ma-report-action-tags<0..*>;]
         datetime                ma-report-action-start-time;
        [datetime                ma-report-action-end-time;]
	[string		         ma-report-action-conflicts<0..*>;]
	[ma-report-table-obj	 ma-report-action-result<0..*>;]
     } ma-report-action-obj;

     object {
        [string                  ma-report-task-column-labels<0..*>;]
        [ma-report-row-obj       ma-report-task-rows<0..*>;]
     } ma-report-table-obj;

     object {
         data                    ma-report-row-value<0..*>;
     } ma-report-row-obj;

This looks kind of implementable I think. I am not sure how efficient
this is; I guess the answer is to simply convert some data into this
format and to see what the result looks like and how big it gets
(compared to the CSV format I am using internally). A possible concern
is the fact that there is quite some redundant data; column headers
hardly change and so do many fields of the ma-report-action-obj. On
the other hand, having the result reports reasonably self-contained
may be a plus (and given a decent secure transport, compression may
take care of some of the noise). Other protocols allow to split the
more static parts from the more dynamic parts but of course this adds
complexity since the receiver needs to keep state in order to put
things back together.

/js

PS: I think there should also be a way to augment in structured
    results in cases where more than a simple table structure is
    needed to report results.

PS: I need to decide how data is passed internally between actions and
    schedules. I am most likely going to use something more compact
    than this on the wire format.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>