Subject: [core] SenML suggestions 3/3: Streamability

From: Christian Amsüss <[email-masked]>
Date: Wed, 20 Jul 2016 15:24:17 +0200

Hello everyone interested in SenML,

(for introduction, see the 1/3 mail).

Third, streamability.

As said in Ari's extended-values thread, streamability is now good but
not yet perfect. (By streamability here I mean that consumers can
process SenML in arbitrarily small or aligned chunks without the need
for seeking back or buffering received data, and still grasp the
document and set their state accordingly). With the -02 draft, a client
would need to buffer roughly the first item (so it can parse the
potentially-first "n" attribute in the context of the potentially-last
"bn" attribute).

Do you think it would be viable to require that a base attribute and its
corresponding individual attribute be mutually exclusive in a record?
That would allow streamability. I'd have them exclusive in a pairwise
fashion; alternatively, an "either there's base or non-base attributes"
would work for that use case just as well.

Sorry for the data rate peak I'm sending here -- I wanted to make sure
1/3 and 2/3 are consistent with each other.

Best regards
Christian

-- 
Christian Amsüss                      | Energy Harvesting Solutions GmbH
founder, system architect             | headquarter:
mailto:[email-masked]  | Arbeitergasse 15, A-4400 Steyr
tel:+43-664-97-90-6-39                | http://www.energyharvesting.at/
                                      | ATU68476614

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 20 Jul 2016 23:04:40 +0200

(moving this into the right thread to not hijack Ari's
allow-absance-of-value proposal)

On Wed, Jul 20, 2016 at 10:54:40PM +0200, Michael Koster wrote:
> So to be clear, does this mean that a base value or a regular value is
> still required in every record?

No, only that they must not occur in the same record.

> Meaning I can't have a simple collection representation like this?
> 
> [
>   {
>     "bn": "/collection/"
>   },
>   {
>     "n": "resource1",
>     "vs": "foo",
>   },
>   {
>     "n": "resource2",
>     "vs" "bar",
>   },
>   {
>     "n": "resource3",
>     "vs": "baz"
>   }
> ]

You can have this. You could even simplify it to this per (what I think
by now are) established semantics:

    [
      {
        "bn": "/collection/resource1",
        "vs": "foo",
      },
      {
        "n": "resource2",
        "vs" "bar"
      },
      {
        "n": "resource3",
        "vs": "baz"
      }
    ]

What you can not do is

    [
      {
        "n": "resource1",
        "vs": "foo",
        "bn": "/collection/"
      },
      {
        "n": "resource2",
        "vs" "bar"
      },
      {
        "n": "resource3",
        "vs": "baz"
      }
    ]

because an application would be delaying interpretation of "resource1"
until the bn occurred, and there may be arbitrary lengths of "foo"
inbetween.

Best regards
Christian

---

From: Michael Koster <[email-masked]>
Date: Wed, 20 Jul 2016 23:09:39 +0200

I would not prefer this, and it's not clear it's a simplification. It means my code has to identify a value to pack in with the base and then split it out at the other end. 

More work for a tiny gain in efficiency on the wire. Plus it's not obvious to many how the URI resolution works.

Best regards,

Michael

> On Jul 20, 2016, at 11:04 PM, Christian Amsüss <[email-masked]> wrote:
>

---

From: Michael Koster <[email-masked]>
Date: Wed, 20 Jul 2016 23:16:29 +0200

The new proposal doesn't seem to allow an absence of value. 

On careful reading, it still requires a value (or a sum), but the value may be one other than that defined in the current draft.

I don't think "bn" qualifies as such a value.

It seems overly restrictive to force this.

Best regards,

Michael

Values are represented using basic data types. This specification
+  defines floating point numbers ("v" field for "Value"), booleans ("vb" for 
+  "Boolean Value"), strings ("vs" for "String Value") and binary data ("vd" for
+  "Data Value"). Exactly one value field MUST appear unless there is Sum field 
+  in which case it is allowed to have no Value field.

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 20 Jul 2016 23:16:49 +0200

On Wed, Jul 20, 2016 at 11:09:39PM +0200, Michael Koster wrote:
> > [{"bn":"/foo/bar",...}, {"n": "baz"}]
>
> I would not prefer this, and it's not clear it's a simplification. It
> means my code has to identify a value to pack in with the base and
> then split it out at the other end. 
> 
> More work for a tiny gain in efficiency on the wire. Plus it's not
> obvious to many how the URI resolution works.

How far this is a simplifcation could be argued, but I'd prefer not to
do this right now. As for it not being obvious to many, I agree, but
this is something we'll need to work on.

I'd like to point out, however, that this is not the point of this
thread. The point is that [{"n":"bar", ...(much)..., "bn":"/foo/"}...]
should be avoided. The example you were worried about not being allowed
any more would still be allowed.

Christian

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 20 Jul 2016 23:23:09 +0200

On Wed, Jul 20, 2016 at 11:16:29PM +0200, Michael Koster wrote:
> On careful reading, it still requires a value (or a sum), but the
> value may be one other than that defined in the current draft.

The change Ari suggested in "SenML values and extensions in SenML
Records" already suggests dropping that restriction, and has received
positive feedback so far.

> It seems overly restrictive to force this.

The context of this statment is ambiguous to me. What do you consider
overly restrictive, the above "there must be a "v*" parameter", or my
"there must not be base and non-base of a kind"?

Best regards
Christian