Subject: [core] SenML suggestions 1/3: Object Values

From: Christian Amsüss <[email-masked]>
Date: Wed, 20 Jul 2016 15:19:35 +0200

Hello everyone interested in SenML,

I'd like to bring up again three suggestions for SenML which all already
been on the list, but were not followed up on. In the interest of thread
followability, I'll send them in separate mails.

As a preamble, I'd like to state that some of this reflects the notion
that SenML can be used to convey not only sensor data, but as a generic
way of transporting representations of multiple resources in a single
request.

First, object values.

SenML extensions like the "l" attribute for links show the need for
having structured data in extensions. The data transported with this
SenML extension can be viewed as a representation of a resource
(in particular the link-format representation of the batch IIRC), and
could thus just as well go into a SenML record's value.

In order to avoid nesting JSON, it would be practical to allow a new
value-like property, "vo", to be an object value, which would be deeper
JSON. (For "sv"/"bv"/"v", the point has been made that some JSON parsing
mechanisms expect every recognized key to have a pre-specified type; if
that's at all an issue here, it might need splitting into "va" (array
value) and "vo" (object value in the sense of a JSON "object"
(dictionary)).

Inside a REST client or server, such a representation would appear
pre-parsed -- it in an API, an object would be passed instead of a
string. If this sounds odd to you, beware that this is already
happening: JSON strings are unicode character strings, so any REST
engine that allows a resource to be handled both as
text/plain;charset=utf-8 and as part of a SenML batch already needs to
handle byte strings and character strings (though that may be invisible
as JSON libraries are commonly dealing with UTF-8 encoded byte strings
for practical reasons).

The same scheme would work for CBOR in a straightforward way. While
something like that would probably be feasible in XML with namespaces
and such, I don't think it's worth the hassle, especially given that the
whole XML/EXI serialization might be dropped in favor of a generic XML
serialization of JSON. Converters that have no dedicated knowledge of
what to do with the "vo" data should just pack the JSON into a string
and put that as a string value. (Which would *not* roundtrip, but
roundtrippability in conversion is something I'm not sure we have anyway
given CBOR's extended capabilities).

I do not currently have use cases for this myself, but I think that it
allows the hypermedia things to mix their results into SenML in a much
more straightforward way.

Best regards
Christian

-- 
Christian Amsüss                      | Energy Harvesting Solutions GmbH
founder, system architect             | headquarter:
mailto:[email-masked]  | Arbeitergasse 15, A-4400 Steyr
tel:+43-664-97-90-6-39                | http://www.energyharvesting.at/
                                      | ATU68476614

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 20 Jul 2016 18:27:21 +0200

> On Jul 20, 2016, at 3:19 PM, Christian Amsüss <[email-masked]> wrote:
> 

I’m strong against adding this. It make high speed parsing really hard. I think l allows you to put this type of extensibility in a separate document and use l to reference it. 

> as JSON libraries are commonly dealing with UTF-8 encoded byte strings

They can also just keep the bytes intact and not process the string in any way. 

> more straightforward way.

I’d like SENML to have the sort of performance of ProtoBuf and I think that is very difficult with support for this type flexibility.  

> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 20 Jul 2016 19:02:38 +0200

On Wed, Jul 20, 2016 at 06:27:21PM +0200, Cullen Jennings wrote:
> I’m strong against adding this. It make high speed parsing really
> hard. I think l allows you to put this type of extensibility in a
> separate document and use l to reference it. 

If this impedes high-speed parsing (which in my impression it does not,
but that's just a weakly founded (see below) gut feeling), that's a very
valid reason, but then we should make it explicit that any nested/object
SenML extensions must only exist in formats that kind of inherit from
SenML, and that they can't just show up in regular SenML.

> > JSON strings are unicode character strings, so any REST
> > handle byte strings and character strings [...]]
> 

This was more to argue why it's not novel on the semantic side.

> I’d like SENML to have the sort of performance of ProtoBuf and I think
> that is very difficult with support for this type flexibility.  

Could you elaborate a little on how the potential presence of a "vo" key
slows down a parser? Granted, the state inside the stream can, when
depth is [{"":""}] at maximum, be represented in one enum and not an
enum and two moderately sized ints. Is this what is sufficient to slow
things down, or are there other considerations I missed?

Best regards
Christian