Subject: Re: [apps-discuss] New Version Notification for draft-shelby-exi-registration-01.txt

From: Yusuke DOI <[email-masked]>
Date: Mon, 02 Apr 2012 16:45:08 +0900

Dear Mark,

I have a following use case.

As EXI is expected to be used in constrained environments, we want to 
make it as short as possible. With application+exi content-type, we can 
make sure the field is under the application's convention; schemaId '1' 
may mean the version 1 of a schema for the application, and so on.

In other words, this kind of content-type definition (application + 
encoding) can imply the format of schemaId field used in the 
application. And this helps 'option/encoding negotiation' of EXI.

Regards,

Yusuke

On 2012年03月31日 19:55, Mark Nottingham wrote:
> What's the use case here? Content negotiation, or something else?
>
> Regards,
>
>
> On 29/03/2012, at 10:52 PM, Zach Shelby wrote:
>
>> I have posted a new version of the +exi registration draft. This version improves the Interoperability considerations of the registration requirements and further clarifies when this suffix may be used, as per the feedback on the list from Carine and others.
>>
>> http://www.ietf.org/id/draft-shelby-exi-registration-01.txt
>>
>> Regards,
>> Zach
>>
>> Begin forwarded message:
>>
>>> From: [email-masked]
>>> Date: March 29, 2012 10:47:32 PM GMT+02:00
>>> To: [email-masked]
>>> Subject: New Version Notification for draft-shelby-exi-registration-01.txt
>>>
>>> A new version of I-D, draft-shelby-exi-registration-01.txt has been successfully submitted by Zach Shelby and posted to the IETF repository.
>>>
>>> Filename:	 draft-shelby-exi-registration
>>> Revision:	 01
>>> Title:		 The +exi Media Type Suffix
>>> Creation date:	 2012-03-29
>>> WG ID:		 Individual Submission
>>> Number of pages: 5
>>>
>>> Abstract:
>>>   Efficient XML Interchange (EXI) is an XML representation technique
>>>   specified by the W3C to provie a binary alternative to the standard
>>>   text XML representation.  This document defines a new Structure
>>>   Syntax Suffix&quot;+exi&quot; for use in a specific class of protocols, where
>>>   &quot;exi&quot; content-type encoding or the generic&quot;application/exi&quot; Media
>>>   Type are not applicable.
>>>
>>>
>>>
>>>
>>> The IETF Secretariat
>>
>> --
>> Zach Shelby, Chief Nerd, Sensinode Ltd.
>> http://www.sensinode.com
>> http://zachshelby.org  - My blog "On the Internet of Things"
>> http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
>> Mobile: +358 40 7796297
>>
>> _______________________________________________
>> apps-discuss mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/apps-discuss
>
> --
> Mark Nottingham
> http://www.mnot.net/
>
>
>
>
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Carine Bournez <[email-masked]>
Date: Mon, 2 Apr 2012 08:45:22 -0400

> http://www.ietf.org/id/draft-shelby-exi-registration-01.txt

Hi,
Thanks for this new version.
I still think that this proposal is too restrictive.
As you say in section 2, you want to "use EXI as a native encoding 
(without the use of XML as an interemediate) in Schema-informed mode, 
and the base Media Type together with the SchemaID indicates to the 
protocol the Schema that was used to produce the EXI grammar as described 
in Section 4." 
I don't think this will be necessarily the case for all the foo+exi
media types that you can think of. Carsten pointed out in the earlier
discussion that your use case is for constrained devices, and I 
understand the choice of schema-informed, strict, and shortest schemaID
possible, but the original discussion about proposing a +exi suffix 
was not restricted to such use cases.

A consequence of relaxing these constraints in section 2 is that 
section 4 would have MAYs everywhere instead of those MUSTs:
<<
 Interoperability considerations:  The registration of a Media Type
      using this suffix MUST describe how to determine the XML Schema
      that is used to encode/decode a payload identified by that Media
      Type.  In particular this description defines how to determine the
      Schema used to encode a payload using the SchemaID option of the
      EXI header.  The format of the identifier to be used in the
      SchemaID, a reference to where the corresponding Schema is
      defined, and a description of how future versions of such Schemas
      will be handled MUST be included.  A default Schema version in the
      absence of the SchemaID field MAY be defined.

Other points than schemaID would also need to be mentioned (I thought 
I had commented on that earlier as well).
It will be fine to specify a number of constraints when you define and
register a given foo+exi media type, but for the +exi suffix generic 
description, I'd just list interop key points without specific restrictions.

-- 
Carine Bournez -+- W3C Europe

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 2 Apr 2012 11:08:25 -0400

Potentially exploding the number of XML media types defined across the Web and in e-mail to save a few bytes in *one* use case seems like a poor trade-off to me. 

On 02/04/2012, at 3:45 AM, Yusuke DOI wrote:

> As EXI is expected to be used in constrained environments, we want to make it as short as possible. With application+exi content-type, we can make sure the field is under the application's convention; schemaId '1' may mean the version 1 of a schema for the application, and so on.
> 
> In other words, this kind of content-type definition (application + encoding) can imply the format of schemaId field used in the application. And this helps 'option/encoding negotiation' of EXI.
> 
> Regards,
> 
> Yusuke
> 
> 

--
Mark Nottingham
http://www.mnot.net/

---

From: "Paul E. Jones" <[email-masked]>
Date: Mon, 2 Apr 2012 11:49:31 -0400

Folks,

I've monitored part of this thread, though admittedly I've not read every message.  So, I apologize if I'm beating a dead horse here.

I've noted that nearly every time that JSON is employed the associated media type is application/json.  In fact, I cannot recall having seen any standards with application/whatever+json.  So, does this suggest that the level of granularity is not needed?

Since EXI is a binary representation of XML, why give it a different media type?  To add to Mark's concern, there are a number of different binary XML formats.  Should we create a different media type (and use the + notation) for all of them?  Then we have nxm different "new" media types.

It was asked before why "Content-Coding" is not used.  I recognize that converting to EXI is different than producing a lossless compression, since the EXI cannot be converted back to ASCII without the schema in hand.  Nonetheless, I would assume a sender would not send EXI unless it knew the recipient could deal with it.

In any case, I am of the mind that all of the +xml definitions is unfortunate and we ought to try to find a solution that does not exasperate the problem.  To that end, we should try to understand why application/json has been sufficient for most uses.

Paul

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Mark Nottingham
> Sent: Monday, April 02, 2012 11:08 AM
> To: Yusuke DOI
> Cc: [email-masked] application-layer protocols
> Subject: Re: [apps-discuss] New Version Notification for draft-shelby-exi-
> registration-01.txt
> 
> Potentially exploding the number of XML media types defined across the Web
> and in e-mail to save a few bytes in *one* use case seems like a poor
> trade-off to me.
> 
> 
> encoding) can imply the format of schemaId field used in the application.
> And this helps 'option/encoding negotiation' of EXI.
> >
> > Regards,
> >
> > Yusuke
> >
> >
> >>> I have posted a new version of the +exi registration draft. This
> version improves the Interoperability considerations of the registration
> requirements and further clarifies when this suffix may be used, as per
> the feedback on the list from Carine and others.
> >>>
> >>>> A new version of I-D, draft-shelby-exi-registration-01.txt has been
> successfully submitted by Zach Shelby and posted to the IETF repository.
> >>>>
> >>>> Syntax Suffix&quot;+exi&quot; for use in a specific class of
> >>>> protocols, where  &quot;exi&quot; content-type encoding or the
> >>>> generic&quot;application/exi&quot; Media  Type are not applicable.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> The IETF Secretariat
> >>>
> >>> --

---

From: Zach Shelby <[email-masked]>
Date: Wed, 11 Apr 2012 09:27:22 +0300

Carine,

> description, I'd just list interop key points without specific restrictions.

This draft is defining what you must define when registering a given foo+exi, thus I think we are in line with making sure constraints are specified. This draft *does not* intend to define a generic +exi suffix. 

Regarding the other points than schemaID, I do agree that other settings for EXI are controlled entirely by the EXI header. This should be obvious, but probably it should be called out somehow.

Zach

---

From: Zach Shelby <[email-masked]>
Date: Wed, 11 Apr 2012 09:31:44 +0300

> What's the use case here? Content negotiation, or something else?

In particular this would be aimed at M2M applications that make use of Schema-informed EXI.

- Content-negotiation when an application also uses other forms of its base media type (+xml, +json etc.)
- Interoperability, this registration would force the schemaID to be defined precisely and thus solve a problem in EXI. This is why the use of this +exi form would only be for Schema informed use of EXI. 
- This use of EXI is *not* content-encoding in my opinion, thus we are providing a better media type model and encouraging media types that mean something semantically. 

Zach

>>> Syntax Suffix &quot;+exi&quot; for use in a specific class of protocols, where
>>> &quot;exi&quot; content-type encoding or the generic &quot;application/exi&quot; Media
>>> Type are not applicable.
>>> 
>>> 
>>> 
>>> 
>>> The IETF Secretariat
>> 
>> --

---

From: Carine Bournez <[email-masked]>
Date: Wed, 11 Apr 2012 03:50:48 -0400

Hi,
> This draft is defining what you must define when registering a given foo+exi, thus I think we are in line with making sure constraints are specified. This draft *does not* intend to define a generi…

The title of the draft is "The +exi Media Type Suffix". Is there some
subtlety that I am missing entirely? I don't think you can override or
define another draft for +exi once one is registered, that's the point
of registering.
Let me clarify my point: I think defining a +exi suffix for schema-informed
mode saying "you must specify the form of schemaID that you are using" would
be a huge mistake. It is (1) not useful, since the EXI 1.0 spec already says
you have to define schemaID for your application, (2) harmful, since it tries
to redefine the mechanism for options that is already defined in EXI 1.0,
at least for the schema-informed mode.
What's wrong with using Content-Encoding, exactly? Semantically, EXI *is* xml,
so having something like foo+xml vs. foo+exi does not make sense, while
*encoding* foo+xml with EXI makes sense.

---

From: [email-masked] (Henry S. Thompson)
Date: Wed, 11 Apr 2012 09:10:49 +0100

> *encoding* foo+xml with EXI makes sense.

+1

This approach seems BAD (Broken As Designed) and would set a very
harmful precedent.

ht
-- 
       Henry S. Thompson, School of Informatics, University of Edinburgh
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: (44) 131 650-4587, e-mail: [email-masked]
                       URL: http://www.ltg.ed.ac.uk/~ht/
 [mail from me _always_ has a .sig like this -- mail without it is forged spam]

---

From: Yusuke DOI <[email-masked]>
Date: Wed, 11 Apr 2012 17:30:05 +0900

Hi,

> at least for the schema-informed mode.

I cannot understand (some of) your point. I think EXI spec does not define any 'mechanism for options' according to schemaId definition. The EXI spec (in schema-informed mode) is useful only if decoders know the running application a priori.

Regards,

Yusuke

---

From: Carine Bournez <[email-masked]>
Date: Wed, 11 Apr 2012 04:59:20 -0400

> I cannot understand (some of) your point. I think EXI spec does not define any 'mechanism for options' according to schemaId definition. The EXI spec (in schema-informed mode) is useful only if dec…

Rephrasing: the EXI 1.0 spec has a mechanism to tell the other end that
schema-informed mode is in use, and the EXI 1.0 spec says that the 
application level has to take care of defining the form of schemaID.

---

From: Zach Shelby <[email-masked]>
Date: Wed, 11 Apr 2012 14:50:23 +0300

> application level has to take care of defining the form of schemaID.

Right. This is what we are trying to fix at the application level, by providing a registry where this information is easily available to anyone on the Internet that runs into that media type. Just waving our hands and hoping that in some specification, somewhere, the interpretation of schemaID has been defined does not work on the Internet. The knowledge alone of application/foo with content-encoding: exi does not tell you sufficient information for Schema-informed mode to actually decode the payload. For encodings like gzip or deflate, that is obviously not a problem. 

Now, if EXI folks think that the way we are approaching this is not the right one, how do you propose to fix the problem? 

One possible way would be for application/foo media types that expect to be used with exi content-encoding (Schema-informed mode) to include the needed schema information in the application/foo media type registration. Not sure how easy it would be to add such information there, or if some instructions would be needed regarding that.  

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 11 Apr 2012 14:04:05 +0200

> application level has to take care of defining the form of schemaID.

Let me try to translate this into language pertaining to the discussion at hand:

EXI can be used as a content-coding for XML documents in "Built-In" mode.
This would be the usage for the "Content-Encoding: EXI" that has been registered.

EXI requires out of band information to use the "Schema-Informed" mode.
In the Internet, This would normally be provided in an Internet media-type definition.
The media-type definition may also want to constrain the allowable values for the numerous options EXI provides, allowing for constrained implementations of EXI.

No existing application/foo+xml media type provides the latter kind of information.
With Content-Encoding: EXI, they are therefore limited to "Built-In" mode.
Any application that needs a tailored, schema-informed version of EXI should therefore register one or more application/foo+exi media types.

The next thing we should do is write down a template for all the information that needs to be nailed down in an application/foo+exi media type registration.

Grüße, Carsten

---

From: [email-masked] (Henry S. Thompson)
Date: Wed, 11 Apr 2012 16:15:12 +0100

Carsten Bormann writes:

> EXI requires out of band information to use the "Schema-Informed"
> mode.  In the Internet, This would normally be provided in an
> Internet media-type definition.  The media-type definition may also
> want to constrain the allowable values for the numerous options EXI
> provides, allowing for constrained implementations of EXI.

Your conclusion (use media type) doesn't follow.  Consider the analogy
of Content-Encoding: gzip -- suppose gzip provided improved
performance based on the decade in which the encoded document was
written, since natural language priors slowly shift over time.  Now
decoding requires OOB information about document creation date.  How shall
we provide it.  I know -- we'll define media types
application/...+gzip for every decade of interest!  I hope it's
obvious that would be a mistake. . .

If decoding requires some parameters, surely the right thing to do is
to transmit them in the obvious place, i.e. in the response message
header.

Either use the Content-Encoding (and Accept-Encoding) header itself,
by registering a new Content-Coding value [1] (foo+exi), or (better)
ask the W3C EXI WG to publish a Note defining an extension-header for
specifying or requesting a schemaID (and more generally, perhaps,
other properties specifiable in the EXI Options docuemtns).

Not only would the second option fit the architecture better, it would
remove the necessity for publishing a new RFC every time you update
the relevant schema!

ht

[1] http://www.iana.org/assignments/http-parameters/http-parameters.xml

---

From: Tony Hansen <[email-masked]>
Date: Wed, 11 Apr 2012 11:25:10 -0400

> I've noted that nearly every time that JSON is employed the associated media type is application/json.  In fact, I cannot recall having seen any standards with application/whatever+json.  So, does…

The following have all been registered:
     vnd.collection+json
     vnd.hal+json
     vnd.oftn.l10n+json

     Tony Hansen

---

From: Julian Reschke <[email-masked]>
Date: Wed, 11 Apr 2012 17:30:42 +0200

> to transmit them in the obvious place, i.e. in the response message

If decoding requires additional information, than IMHO you can't use 
HTTP's Content-Coding or Transfer-Encoding. By all means, if you think 
otherwise then send feedback to the HTTPbis WG.

> remove the necessity for publishing a new RFC every time you update

Adding more request headers here is problematic because they contribute 
to the message size, even when they are never used.

Best regards, Julian

---

From: Ned Freed <[email-masked]>
Date: Wed, 11 Apr 2012 08:57:39 -0700 (PDT)

> otherwise then send feedback to the HTTPbis WG.

Julian is 100% correct here. If EXI requires knowledge of the type itself, it's
not a separable encoding, which is what Content-Coding, Transfer-Encoding, and
Content-Transfer-Encoding all require. And while I suppose we could adopt the
foo+exi approach, having an explosion of encodings is a really bad idea.

And if this is the case, it really isn't suitable for a +exi suffix either.
+exi is supposed to mean the format can at least be parsed,  independent of any
knowledge of the type semantics. This is why I support the idea of having +ber
and +der but not +per - with +per you have to know the ASN.1 schema in order to
parse the material properly.

For better or worse, we don't currently have a place in the architecture for
these entities. We could define some other sort of type naming convention if
there's really a need for it, but adding additional out of band information
that has to be processed in order to interpret the material correctly is rather
obviously highly problematic.

				Ned

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 11 Apr 2012 18:09:22 +0200

Actually, the context in which we (at least some of us) are discussing this does not involve HTTP.  So HTTP header fields definitely are not the right way to handle this.  (But even in HTTP, I'd expect coding parameters to be specified in the payload, just like it is done in RFC1950.)

Our use case is about reigning in the exuberant puzzle of choices.  Flexibility is overrated.  Options often cost more money to implement than the optional functionality, and they hurt interoperability.  We want to nail down as much as possible.  I'd expect us to never send an EXI header, but always define that in the media type.  Yes, we might have a couple more media types in the end.  But the point of standardization is to create markets by reducing choice, so that proliferation will be limited.

Grüße, Carsten

---

From: Tony Hansen <[email-masked]>
Date: Wed, 11 Apr 2012 12:41:00 -0400

Sometimes wikipedia actually has useful information. :-) From the 
wikipedia entry for EXI:

"An advantage of EXI over Fast Infoset is that EXI (optionally) uses 
more constraints from the XML schema. This can make the EXI data more 
compact; for example, if the XML schema specifies that elements named 
'bar' may only exist within elements named 'foo', EXI can assign a 
shorter token to the 'bar' element, knowing that it doesn't have to 
share the same token space as elements that occur elsewhere in the document.

"The main disadvantage is that to take advantage of this 
"schema-informed" compression, not only does the document require a 
schema, but the decoder needs a copy of the same schema that the encoder 
used."

This supports Ned's comment that +exi might not be viable.

     Tony Hansen

---

From: Carine Bournez <[email-masked]>
Date: Wed, 11 Apr 2012 12:44:48 -0400

> EXI requires out of band information to use the "Schema-Informed" mode.

Not necessarily, see below.

> Any application that needs a tailored, schema-informed version of EXI should therefore register one or more application/foo+exi media types.

If you use the content-encoding and the original media type, let's say svg,
then the schema is the svg schema. There is absolutely no issue in that case.
If foo has an application/foo media type for which the schema is known, 
the application readily knows which schema to use when receiving the EXI
stream with the content-encoding header.

> The next thing we should do is write down a template for all the information that needs to be nailed down in an application/foo+exi media type registration.

I don't think that a +exi system will provide a better solution for the 
Web.

---

From: Zach Shelby <[email-masked]>
Date: Wed, 11 Apr 2012 22:26:11 +0300

Ned,

> obviously highly problematic.

I agree with your analysis in general. Luckily the situation with EXI schema-informed mode is not as bad as with PER. We actually do have a schemaID field in the EXI header where some indication about the OOB schema information can be included. The standard itself however doesn't specify how to use that field. It is really a documentation problem we're trying to solve, where the application/foo+exi registration includes the definition of how to interpret the schemaID field of the EXI header. 

Let me use SenML as a practical example. Here draft-jennings-senml-08 defines a schema for use with the EXI serialization (version 1 of the markup). In the application/senml+exi IANA registration, I would indicate that in the absence of the schemaID field the schema is by default version 1 of SenML. For future versions, the integer of the SenML version is to be included in the schemaID field. This way a single senml+exi registration can handle this, and all future versions of the SenML markup.  

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 11 Apr 2012 22:43:07 +0200

> stream with the content-encoding header.

Ah, SVG actually is a great example for why I think there need to be separate +exi registrations.

So, for image/svg+xml (see http://www.w3.org/TR/SVG/mimereg.html), what is "the SVG schema"?

What values of schemaId are defined, and what do they point to?

What about "application/svg+xml"?  “application/vnd.oipf.dae.svg+xml”?

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 12 Apr 2012 01:05:45 +0200

> parse the material properly.

So how would you handle the PERs and (schema-informed) EXIs of this world?
Should it be -exi, not +exi?

(As in application/senml-exi?)

(Again, the media type registration will do a lot more than just tack EXI encoding on existing XML.)

Grüße, Carsten

---

From: Tony Hansen <[email-masked]>
Date: Thu, 12 Apr 2012 14:50:25 -0400

The Structured Syntax Suffixes are appropriate for use when the media 
type identifies the semantics of the protocol payload. That is, knowing 
the semantics of the specific media type provides for more specific 
processing of the content than that afforded by generic processing of 
the underlying representation.

At the same time, using the suffix provides receivers of the media types 
to do generic processing of the underlying representation in cases where

     1) they do not need to handle specially the specific semantics of 
the exact media type, and,

     2) there is no special knowledge needed by such a generic processor 
in order to parse that underlying representation other than what would 
be needed to parse any example of that underlying representation.

Please correct me if the following is wrong:

It is my understanding that the general underlying representation of 
EXI-based types *may* be totally generic. However, each EXI type also 
pre-defines a dictionary specific to that type that the EXI 
representation may use to shorten the output. So if I hand a media type 
to a generic EXI parser, it will do fine until it hits something that is 
only defined in that type-specific dictionary. Unfortunately, the 
type-specific dictionary is not included in the EXI representation, but 
must be gathered out-of-band.

So it seems like +exi *could* be used if the representation stuck to 
only using the generic dictionary and eschewed any type-specific 
out-of-band dictionaries.

     Tony Hansen

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Apr 2012 12:09:34 -0700 (PDT)

> subtlety that I am missing entirely?

If such a subtlety is there, I'm missing it too.

> I don't think you can override or

Exactly.

> at least for the schema-informed mode.

I remain to be convinced that +exi is a good idea or even allowable, but
not for these reasons. The problem as I see it is that the point of + suffixes
is to allow some degree of processing even though you don't know the specifics
of the type. But this doesn't seem to be the case for +exi.

> *encoding* foo+xml with EXI makes sense.

The main problem with encodings is that they are not intended to have this sort
of structure. And more to the point, I doubt very much that existing software
is prepared to deal with taking apart structured encoding names and figuring
out if they have the necessary schema available.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Apr 2012 13:57:00 -0700 (PDT)

> must be gathered out-of-band.

Even this might, and I emphasize might, be barely acceptable if there was
always a SchemaId in the content clearly identifying the necessary schema and
such schemata are always generally available. But it has been clearly stated
that that's not always the case. Indeed, one of the purposes of registering
these types is apparently to specify the SchemaId default when one is not
included.

If that's the case, then this is really not acceptable. Media type
registrations are not intended for this sort of automatic information
extraction. Some other mechanism would have to be used, and we'd have to have a
pretty serious discussion about the advisability of doing that.

> out-of-band dictionaries.

Agreed. That would make the definition of +exi acceptable. Whether or not
that's an acceptable restriction for users of EXI is, of course, another
question entirely.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Apr 2012 14:20:19 -0700 (PDT)

> I agree with your analysis in general. Luckily the situation with EXI
> schema-informed mode is not as bad as with PER.

I'm afraid I'm not seeing any substantive difference. In fact PER might
arguably be better, because if memory serves the OSI presentation layer
included the means of negotiating use or non-use of PER as well as identifyin
the specific ASN.1 schema involved.

> We actually do have a schemaID field in the EXI header where some indication
> about the OOB schema information can be included. The standard itself
> however doesn't specify how to use that field. It is really a documentation
> problem we're trying to solve, where the application/foo+exi registration
> includes the definition of how to interpret the schemaID field of the EXI
> header.

But that's only part of the problem. Consistent interpretation of the field is
of course necessary, but not sufficient.

> Let me use SenML as a practical example. Here draft-jennings-senml-08 defines
> a schema for use with the EXI serialization (version 1 of the markup). In the
> application/senml+exi IANA registration, I would indicate that in the absence
> of the schemaID field the schema is by default version 1 of SenML.

And pray tell how is a generic +exi processor going to be able to know that?
Your specification doesn't define any sort of registry for saying "media type
foo has this schema as a default", nor is there any way to query such a
registry, and it's an open question as to whether or not such a registry is
viable or usable in this context.

> For future versions, the integer of the SenML version is to be included in
> the schemaID field. This way a single senml+exi registration can handle this,
> and all future versions of the SenML markup.

As I said in a previous email, mandating the presence of the SchemaID field
might be an acceptable compromise. Or it might not. This is not up to me, and
to be frank I'm still on the fence about this approach myself.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Apr 2012 14:34:12 -0700 (PDT)

> (As in application/senml-exi?)

That's definitely an option - an informal sort of suffix. Another alternative
would be to define some other formalism (it really cannot be "-" since other
media types use "-" for other purposes). Looking at the ABNF of allowed
characters, carat is allowed and I don't think it has ever been used. OTOH, how
many things like EXI are ever going to show up?

				Ned

---

From: Zach Shelby <[email-masked]>
Date: Fri, 13 Apr 2012 09:27:28 +0300

> that's an acceptable restriction for users of EXI is, of course, another

Using EXI in its normal mode is already fine as a content-encoding. The problem we need to solve here is exactly the Schema-informed mode where an indication of out-of-band dictionary information is needed, and where the use of content-encoding is not really appropriate (as we all seem to agree).

Now I am as much on the fence as Ned it here about this. Seems we are in a sticky situation where serializations that require OOB dictionary information fall between RESTful cracks. Examples of these that I know of include:

-  ASN.1 PER 
- EXI Schema-informed mode
- Google Protocol Buffers 
- Surely more... 

These tend to be applied in M2M and other applications where data efficiency is extremely important and using OOB dictionary information is what gives us that efficiency. 

Zach

---

From: Zach Shelby <[email-masked]>
Date: Fri, 13 Apr 2012 09:39:07 +0300

> many things like EXI are ever going to show up?

This could be the right approach to look at. As I wrote in a previous mail there are a few types of serializations that require OOB dictionary information such as EXI schema-informed mode, ASN.1 PER, and Google Protocol Buffers.     

So application/senml^exi would need to result in some sort of registry entry where information on how to determine the OOB dictionary is easily available to anyone who runs across that media type. It seems to me that the media type registry already contains quite a few fields that could be used for that purpose -- for example "interoperability considerations". 

Zach

---

From: Carine Bournez <[email-masked]>
Date: Fri, 13 Apr 2012 08:15:58 -0400

> So, for image/svg+xml (see http://www.w3.org/TR/SVG/mimereg.html), what is "the SVG schema"?

There's currently no schema defined for use with EXI, for various reasons
(other things in SVG are under discussion for future use with EXI).

> What values of schemaId are defined, and what do they point to?

The EXI 1.0 spec mentions the use of a URI as an example for schemaIds.

> What about "application/svg+xml"?  ?application/vnd.oipf.dae.svg+xml??

The +exi proposal would have the same question: either define the schema, or 
the mechanism to use schemaId, or another out-of-band mechanism.

---

From: Tony Hansen <[email-masked]>
Date: Fri, 13 Apr 2012 11:38:10 -0400

At this point, I'm not seeing what application/senml*exi (for any 
character used for the '*') buys me over application/senml. The key 
point in my mind is that I have to know that it's senml in order to 
parse it. Knowing that it uses exi underneath doesn't buy me anything 
without also knowing that it is senml. That is, I have to know about the 
senml dictionary just in order to process the file. The type's schema is 
useful for understanding the XML, but it's not required in order to 
*parse* the file.

Contrast this with application/anything+xml. Here I can parse it either 
as application/anything or as generic xml without knowing anything about 
the type anything, and get useful information out. The type's schema is 
useful for understanding the XML, but it's not required in order to 
*parse* the file.

In other words, I don't think a *exi extension is going to buy me 
anything for processing the media type. So why use it?

Note: this is a serious question. Is there useful stuff that can be done 
with an application/senml*exi file without knowing that it's senml?

     Tony Hansen

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 13 Apr 2012 17:50:32 +0200

> Note: this is a serious question. Is there useful stuff that can be done with an application/senml*exi file without knowing that it's senml?

(Assuming that this is about the schema-informed EXI version:)

I don't think so.

Calling it application/senml is still not very good because there are also application/senml+json and application/senml+xml.

So I would call the schema-informed version application/senml-exi.
(Actually, I'd call it application/senml-exi-a and elide the EXI header, but that is a different discussion.)

Grüße, Carsten

---

From: [email-masked] (Henry S. Thompson)
Date: Fri, 13 Apr 2012 17:17:25 +0100

> (Actually, I'd call it application/senml-exi-a and elide the EXI header, but that is a different discussion.)

Why not call it application/senml+xml and encode it with EXI?  Doesn't
that tell you everything you need?  Provided you put everything you
need about schemaIDs for senml in the media type registration for
application/senml+xml.

ht

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 18:29:33 +0200

> need about schemaIDs for senml in the media type registration for

You can't use +xml unless it's XML.

Best regards, Julian

---

From: [email-masked] (Henry S. Thompson)
Date: Fri, 13 Apr 2012 17:36:27 +0100

> You can't use +xml unless it's XML.

But it _is_ XML if it's encoded using EXI. . .

Surely the whole point of EXI is that it can be used to efficiently
encode _any_ XML infoset.  Are you telling me I can't use EXI with
e.g. application/rdf+xml?

ht

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 18:47:05 +0200

> But it _is_ XML if it's encoded using EXI. . .

No. We're talking about bits on the wire, not data models.

> e.g. application/rdf+xml?

You can use EXI to compress application/rdf+xml, but once you did that 
it's not application/rdf+xml anymore. The same would be true for 
application/rdf+xml compressed with gzip (which reminds me about an epic 
debate we had last year about ".svgz").

Best regards, Julian

---

From: [email-masked] (Henry S. Thompson)
Date: Fri, 13 Apr 2012 18:04:17 +0100

Julian Reschke writes:

> HST wrote:
> application/rdf+xml compressed with gzip (which reminds me about an
> epic debate we had last year about ".svgz").

What?  Surely Content-Encoding is logically independent from, and
processing-wise transparent wrt Content-Type.

Here's what 2616 says:

  "The Content-Type entity-header field indicates the media type of
   the entity-body"

and

  "The Content-Encoding entity-header field is used as a modifier to
   the media-type. When present, its value indicates what additional
   content codings have been applied to the entity-body, and thus what
   decoding mechanisms must be applied in order to obtain the
   media-type referenced by the Content-Type header field. "

So I take an entity-body which is application/rdf+xml, I encode it
with gzip, I send it marked

 Content-Encoding: gzip
 Content-Type: application/rdf+xml

The gzip-aware UA decodes the entity body with gunzip and hands over
the result as application/rdf+xml.

Same for exi, modulo the fudge about infoset vs. character sequence
which was thrashed through wrt the 'exi' content-code.

ht

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 19:38:15 +0200

On 2012-04-13 19:04, Henry S. Thompson wrote:
> ...

Yes.

> ...

All true, but as far as I can tell, you didn't mention Content-Encoding 
in the message I replied to.

Best regards, Julian

---

From: [email-masked] (Henry S. Thompson)
Date: Fri, 13 Apr 2012 19:09:01 +0100

Julian Reschke writes:

>> ...
>
> All true, but as far as I can tell, you didn't mention
> Content-Encoding in the message I replied to.

Ah, I see the disconnect.  In my original message, I said:

 "Why not call it application/senml+xml and encode it with EXI?
  Doesn't that tell you everything you need?"

I should have been more explicit -- what I meant the above to convey
was

 "Why not call it application/senml+xml and encode it with EXI?
  That is, use Content-Type: application/senml+xml and 
               Content-Encoding: exi .
  Doesn't that tell you everything you need?"

So, reset.  With that clarification, what do you think?

ht

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 20:16:59 +0200

> So, reset.  With that clarification, what do you think?

Yes, absolutely, that works with HTTP.

AFAIR, Core (<http://tools.ietf.org/wg/core/>) doesn't have 
Content-Encoding, though.

Best regards, Julian

---

From: Ned Freed <[email-masked]>
Date: Fri, 13 Apr 2012 11:14:38 -0700 (PDT)

>   Doesn't that tell you everything you need?"

I'm a newcomer to all this myself, but I'm fairly sure the general answer to
that is "no".

This will work if the exi encoding isn't using schema-specific compression. But
what if it is? Then you need to see start of the actual content before you can
be sure you can decode it - it could contain a SchemaId pointing at a schema
you do not have available for whatever reason. Or it could contain no SchemaId
at all and you don't know what the default schema is for this type.

So this would mess up negotiation schemes.

				Ned

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 20:24:39 +0200

> Content-Encoding, though.

...with the constraint Ned mentioned in a parallel message (no 
out-of-band information needed).

Best regards, Julian

---

From: Zach Shelby <[email-masked]>
Date: Fri, 13 Apr 2012 21:24:59 +0300

> Yes, absolutely, that works with HTTP.

Correct, content-encoding is valid for the non-schema informed mode of EXI where there is no OOB dictionary needed. There has never been an argument against that, so all good there.

We are trying to improve the interoperability situation with schema-informed EXI (my +exi draft was an attempt at that). As Ned pointed out though, for schema-informed EXI content-encoding is not really valid, but at the same time +exi is not suitable either. For this reason we are now looking at just using e.g. application/senml-exi informally or possible thinking about a more formal suffix style for encodings that require OOB dictionary information. 

> AFAIR, Core (<http://tools.ietf.org/wg/core/>) doesn't have Content-Encoding, though.

We actually fixed that in the -09 revision of CoAP. It is possible for us to deal with content-encodings, we just have to make multiple code entries in our registry. 

Zach

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 13 Apr 2012 13:25:27 -0500

OK. If you revise the draft, it might be helpful to add some examples to Section 2 to illustrate this (I know it's already there, but examples do help!).

Cheers,

--
Mark Nottingham
http://www.mnot.net/

---

From: Carine Bournez <[email-masked]>
Date: Fri, 13 Apr 2012 16:02:14 -0400

> out-of-band information needed).

I suppose that the registration for application/senml+xml could 
mandate use of a particular schema when encoded with EXI and exclude
any additional schema (i.e. no schemaId allowed).

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 22:16:29 +0200

> any additional schema (i.e. no schemaId allowed).

Nope. Content-Encoding and media type are orthogonal things.

Best regards, Julian

---

From: [email-masked] (Henry S. Thompson)
Date: Fri, 13 Apr 2012 21:28:00 +0100

> Nope. Content-Encoding and media type are orthogonal things.

But she said "when encoded with EXI".  Why can't a media type
registration say that, literally?  That is "When encoding with EXI,
use this schema: [ref] and schemaID: http://...".

ht

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Apr 2012 22:30:58 +0200

> use this schema: [ref] and schemaID: http://...".

Back to our confusion :-)

Yes, you can use EXI, but you can't use EXI-as-a-HTTP-Content-Encoding 
in this case. The HTTP Content Encoding must be fully defined in absence 
of any out-of-band information.

For instance, a recipient (say browser) needs to be able to remove the 
encoding without having to know *anything* about the media type.

Best regards, Julian

---

From: Zach Shelby <[email-masked]>
Date: Fri, 13 Apr 2012 23:38:48 +0300

On Apr 13, 2012, at 11:30 PM, Julian Reschke wrote:

> Yes, you can use EXI, but you can't use EXI-as-a-HTTP-Content-Encoding in this case. The HTTP Content Encoding must be fully defined in absence of any out-of-band information.
> 
> For instance, a recipient (say browser) needs to be able to remove the encoding without having to know *anything* about the media type.

Exactly. So now we are back to registering something like application/senml-exi (in addition to senml+xml and senml+json) where the schema information is included in the media type registration and no content-encoding is used. This would work for the applications I know of right now (SEP2 and SenML). Do we need to write up some kind of information guideline draft for including this kind of information in the media registration when OOB info needs to be defined (not just for EXI, but in general)? 

Zach

---

From: Tony Hansen <[email-masked]>
Date: Fri, 13 Apr 2012 17:05:06 -0400

> Exactly. So now we are back to registering something like application/senml-exi (in addition to senml+xml and senml+json) where the schema information is included in the media type registration and…

I'm thinking that a statement along these lines needs to go into 
draft-ietf-appsawg-media-type-regs (whose WGLC ends today) -- in other 
words, beef up the section on +suffix registration with text that helps 
people better understand when a +suffix should be used, and when it 
should *not* be used.

I'll work with Ned on that.

     Tony

---

From: [email-masked] (Henry S. Thompson)
Date: Fri, 13 Apr 2012 22:17:45 +0100

Zach Shelby writes:

> Do we need to write up some kind of information guideline draft for
> including this kind of information in the media registration when
> OOB info needs to be defined (not just for EXI, but in general)?

This still just feels wrong.  Consider encrypted transmissions.  You
need a key to decrypt them.  That's (of necessity) OOB information.
The idea that the media type of the transmission somehow should be
involved in that whole aspect of things just seems wrong.
Compression, like encryption, is (at least partly) _orthogonal_ to
media type.

We are getting into the delicate, difficult territory I already
mentioned once, to do with the extent to which accepting 'exi' as a
coding-type amounted to bending the rules.  The simple two-layer story
implicit in the original media type architecture (here's a character
sequence, and here's a key to unlock its meaning), later extended to
three (the +... media types) in one direction, or three/four in
another (transfer- and content-encoding) is just not flexible enough
to deal with the complexity of contemporary practice.  Compound
documents, e.g. XHTML+MathML+(SVG-with-XHTML-inside) are one class of
problems.  What does it mean, for example, to say that the meaning of
a fragment identifier is determined by _the_ media type of the
entity-body in such a case?  Encodings of information, rather than
byte-sequences (e.g. EXI) are another.  OOB-dependencies may well be
another.

I'll crawl back into my hole now, and maybe wait to see what the
proposed media type registration for 'senml-exi' looks like, since
I've run out of ideas to try to provide a locally-adequate
workaround. . .

ht

---

From: Ned Freed <[email-masked]>
Date: Fri, 13 Apr 2012 15:19:00 -0700 (PDT)

> I'll work with Ned on that.

After this exercise I have to agree that better text is needed.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 13 Apr 2012 15:40:48 -0700 (PDT)

> any additional schema (i.e. no schemaId allowed).

That's certainly doable from a registration standpoint - we even have
an Encoding Considerations section that could be used for this.

However, keep in mind that this approach could be a bit of a problem if
a new version of senml comes out that includes a schema change.

				Ned

---

From: Carine Bournez <[email-masked]>
Date: Sat, 14 Apr 2012 08:10:40 -0400

> Nope. Content-Encoding and media type are orthogonal things.

It was the case until the pack200 content encoding was registered.
EXI has not set any precedent here.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 14 Apr 2012 14:27:41 +0200

> EXI has not set any precedent here.

If "pack200" violates that principle, it shouldn't have been registered. 
(Does it? Pointers, please)

However, just because it happened once doesn't make it correct to do it 
again.

Best regards, Julian

---

From: Ned Freed <[email-masked]>
Date: Sat, 14 Apr 2012 06:55:26 -0700 (PDT)

> (Does it? Pointers, please)

My understanding is that pack200 is a type-specific compression format. That
is, it only works if the input is a jar file. In this sense there is overlap
with EXI - I believe EXI relies on the input having XML syntax (although it
doesn't necessarily have to be well formed), which means it is only applicable
to a subset of media types.

If that's indeed the case, then yes, pack200 is not orthogonal to the media
type. That said, it's not in the same class as EXI. I'm pretty sure I can
decompress anything that's been compressed with pack200 with nothing but
knowledge of the pack200 algorithm. I don't need to to have access to an
external schema or type-specific rules about which schema is the default.

I also note that pack200 doesn't present the same sort of issues for
negotiation. A server knows the types of the objects it sends out and therefore
knows when pack200 can and cannot be applied. And a client requesting pack200
encoding and getting it back won't ever have any difficulty handling it. The
most that can be said is that it requires special-case handling on the server
when an encoding is selected. (Which is also the case with EXI.)

> However, just because it happened once doesn't make it correct to do it

Quite true. Even if pack200 had exactly the same characteristics as EXI, that
would not provide justification for making the same mistake again.

				Ned

---

From: "Martin J. Dürst" <[email-masked]>
Date: Sun, 15 Apr 2012 15:08:10 +0900

On 2012/04/14 22:55, Ned Freed wrote:

> doesn't necessarily have to be well formed),

If it's not well-formed, it's not XML. Did you mean "not valid"?

Regards,   Martin.

> which means it is only
> applicable

---

From: Ned Freed <[email-masked]>
Date: Sun, 15 Apr 2012 16:24:14 -0700 (PDT)

> If it's not well-formed, it's not XML. Did you mean "not valid"?

Yes, it has to be well formed, but validity isn't required in non-strict mode.

				Ned

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 23 Apr 2012 06:57:00 -0700

So what was the summary of the discussion on this? Other than clarifying draft-ietf-appsawg-media-type-regs, anything decided? Any changes I need to make to senml registrations?

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Apr 2012 22:28:28 +0200

My 5-line summary is:

1) EXI may or may not be a content-encoding in its schemaless form, but we don't care about that, we use EXI schema-informed mode.

2) +exi is considered undesirable for schema-informed EXI.
So we should go to an EXI specific media type, let's call that -exi.

This means three media types for SenML

application/senml+json
application/senml+xml
application/senml-exi

The third media type would in particular nail down the way EXI is used in schema-informed mode.

Grüße, Carsten

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 23 Apr 2012 16:05:30 -0600

> My 5-line summary is:

Thanks for the summary!

> The third media type would in particular nail down the way EXI is used in schema-informed mode.

Although it's not beautiful, I can see why we've ended up at #3 and it
seems like the least-worst solution at this point.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Ned Freed <[email-masked]>
Date: Mon, 23 Apr 2012 18:14:36 -0700 (PDT)

> seems like the least-worst solution at this point.

I have to agree. For better or worse, we didn't think about this case when the
architecture was designed. And as we all know, adding mandatory-to-implement
information fields after the fact doesn't work very well (if at all). So this
is where we end up.

				Ned

---

From: Zach Shelby <[email-masked]>
Date: Tue, 24 Apr 2012 10:53:51 +0300

I agree with Carsten's summary here, and plan on just letting my +exi draft expire. With SenML we can try out this senml-exi form and see how we get the information into the registry.

Zach

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 24 Apr 2012 09:55:09 +0100

Carsten Bormann writes:

> My 5-line summary is:
>
> 1) EXI may or may not be a content-encoding in its schemaless form,
> but we don't care about that, we use EXI schema-informed mode.
>

Works for me.

ht

---

From: Tony Hansen <[email-masked]>
Date: Tue, 24 Apr 2012 06:40:31 -0400

> information fields after the fact doesn't work very well (if at all). So this

Thanks for the summary Carsten. I think you captured the discussion very 
well.

     Tony Hansen

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 24 Apr 2012 08:49:51 -0700

>>> application/senml-exi

I think the idea that first two are OK but the third one needs to have a "-" instead of a "+" is simply baffling. Can someone explain the logic to me ?

Cullen 

I'll rest the urge to insert a rant here about the reason people don't bother to register new thing and just use text/plain for everything …

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Apr 2012 17:56:22 +0200

> I think the idea that first two are OK but the third one needs to have a "-" instead of a "+" is simply baffling. Can someone explain the logic to me ?

Logic = 
-- Least amount of pain
-- Gets the job done

(The detailed technical discussion is in the last 75 messages, which I won't try to repeat.  Yes, this is all logical, even if the result of all that logic appears to incite cognitive dissonance.  We must have spent some 100 hours of engineer time on this already.)

It's just a string, and it is not even sent in the packet!

(Can we spend the next 100 hours debating what to put in the EXI header, please?)

Grüße, Carsten

---

From: Ned Freed <[email-masked]>
Date: Tue, 24 Apr 2012 08:51:04 -0700 (PDT)

> I think the idea that first two are OK but the third one needs to have a "-"
> instead of a "+" is simply baffling. Can someone explain the logic to me ?

+exi implies a syntax that can be processed generically by systems knowing
nothing specific about senml. But in order to decode schema-aware EXI, which is
what is being used here, you have to know the schema for the specific version
of senml that's in use. There is an field in the document itself that can be
used to provide a pointer to the schema, but it's optional. In other words, in
some cases the schema has to be known through out of band information, and that
violates the rules for +suffix.

> I'll rest the urge to insert a rant here about the reason people don't bother
> to register new thing and just use text/plain for everything …

In turn I'll resist the urge to rant about folks who jump into a long
discussion without making any attempt to read the preceeding messages and
expect others to do their work for them.

				Ned

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 24 Apr 2012 09:16:22 -0700

> (Can we spend the next 100 hours debating what to put in the EXI header, please?)

So lets me paraphrase your answer. 

There is no explainable technical reason but the IETF is so unreasonable that I've given up caring after 100s of hours of discussion and just trying to move forward. 

I feel your pain and this is reasonable at some level but it is a serious problem for the IETF - if we can't make it easy for people to register types and know what to do, they just give up and use text/plain or some moral equivalent of that. 

So that said, can someone take stab at explaining the logic behind senml+xml but senml-exi? I tired reading the draft and did not leap out to me why one was + and the other -.

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 24 Apr 2012 09:17:47 -0700

Oops - had not read Ned's response before sending that. I'll read Neds.

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Apr 2012 18:27:03 +0200

> There is no explainable technical reason

That's not what I said -- I said it's in the last 75 messages and I'm not that interested in repeating all this.

> but the IETF is so unreasonable that I've given up caring after 100s of hours of discussion and just trying to move forward. 

It's not the IETF that is unreasonable, it is just that this maze of little specifications that this needs to operate in is more complicated than I would like it to be.

> I feel your pain and this is reasonable at some level but it is a serious problem for the IETF - if we can't make it easy for people to register types and know what to do, they just give up and use…

I think once Tony's draft is done, at least this part of the discussion will be easier.
We may need to work some more on 4.2.8 there, maybe adding this as an example.

(The rest is indeed in Ned's answer.)

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Apr 2012 18:31:28 +0200

On Apr 24, 2012, at 18:27, Carsten Bormann wrote:

> Tony's draft

draft-ietf-appsawg-media-type-regs-05.txt

Sorry, it does have a few more authors.
(It's probably Tony who was in my mind because he was championing it in Paris.)

Grüße, Carsten

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 24 Apr 2012 10:47:55 -0700

> +exi implies a syntax that can be processed generically by systems knowing

Thanks - that answer does help. But what do you mean can be "processed generically"? If we mean can check the syntax is valid, sure, all can do that. If we mean understand the semantic content of the data without the schema information,  it seem pretty unlikely that senml should have the + for json or xml. 

I will point out the senml mandates that data inside has to identify the schema in use for that inside the exi data - now it's not a globally resolvable URI for the schema or something but it does make sense in the context of senml. You can process the senml exi without the schema at least in the context of determining it might be valid exi. 

But really the questions is what's is our definition of "generically processable" ?

> expect others to do their work for them.

I've really had read the whole thread when I sent the previous email. I think you will agree with me that many people find it is hard to register a new type like this. I applaud you doing the draft-ietf-appsawg-media-type-regs draft and the work on draft-hansen-media-type-suffix-regs to try and make this easier.

---

From: Ned Freed <[email-masked]>
Date: Tue, 24 Apr 2012 10:51:28 -0700 (PDT)

> Thanks - that answer does help. But what do you mean can be "processed
> generically"? If we mean can check the syntax is valid, sure, all can do that.

Well, there are these things called XML editors and renderers... And probably
ones for JSON too, although I'm not as familiar with that space.

> If we mean understand the semantic content of the data without the schema
> information,  it seem pretty unlikely that senml should have the + for json or
> xml.

No, it doesn't mean that. If it did there would basically be no use-case at all
for +xml or +json.

> I will point out the senml mandates that data inside has to identify the
> schema in use for that inside the exi data - now it's not a globally resolvable
> URI for the schema or something but it does make sense in the context of senml.
> You can process the senml exi without the schema at least in the context of
> determining it might be valid exi.

First of all, it is by no means clear that even if the schema reference
was mandatory, it would fulfill the requirements of +suffixes. The idea
was that no external information would be needed to understand the syntax
of such types. As best I can tell, there is no consensus to loosen this.

That said, a +exi suffix is necessarily a generic mechanism that needs to be
able to apply to multiple types. If all such usage required explicit schema
specification in the document, or better still, mandated that EXI be operated
in schema-agnostic mode, then +exi might make sense. But it seems people using
EXI aren't willing to make that tradeoff in order to use +exi. Which given the 
target space I completely understand and agree with.

> But really the questions is what's is our definition of "generically

It's pretty simple: Processing at the syntactic but no the semantic level.

> I've really had read the whole thread when I sent the previous email. I think
> you will agree with me that many people find it is hard to register a new type
> like this.

Since on average I approve registration of several types a week, often on the
first submission and rarely requiring more than one revision, I really don't
agree. With senml you're apparently after the additional imprimateur of a
standards body - which is, or at least should be, a separate consideration -
and that necessarily is going to involve a lot more process.

>  I applaud you doing the draft-ietf-appsawg-media-type-regs draft and
> the work on draft-hansen-media-type-suffix-regs to try and make this easier.

Thanks. However, the main intent is to try and streamline the standards tree
process for other standards bodies, and of course to formalize the +suffix
business. The effect on RFCs that happen to define media types is going to be
fairly minimal.

				Ned

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 24 Apr 2012 14:15:27 -0700

> It's pretty simple: Processing at the syntactic but no the semantic level.

Ok, but it seem that is what EXI in this form allows - you can step over the elements, often find their types at the "string" "int" sort of level but can not expand what the actually tags names are without the schema but you tell there is a tag named something, you know if you encounter the same tag again, and so on. You can check it is valid.  I don't use EXI editors but they do exist and they should be able to load it and display it. It would look a lot like XML where all the tag names had been changed to tag1, tag2, tag3 etc but otherwise had the structure and many of the values. 

Cullen

I should mention one thing - I have only participated in this thread because I did not think it is was clear why some things use + and some use -. For the uses of senml-exi that I care about senml-exi will work just as well as senml+exi. But what happens here seem like it will impact other uses of EXI.

---

From: Ned Freed <[email-masked]>
Date: Tue, 24 Apr 2012 17:48:28 -0700 (PDT)

> Ok, but it seem that is what EXI in this form allows - you can step over the
> elements, often find their types at the "string" "int" sort of level but can
> not expand what the actually tags names are without the schema but you tell
> there is a tag named something, you know if you encounter the same tag again,
> and so on. You can check it is valid.  I don't use EXI editors but they do
> exist and they should be able to load it and display it. It would look a lot
> like XML where all the tag names had been changed to tag1, tag2, tag3 etc but
> otherwise had the structure and many of the values.

This boils down to expectations. The expectation is going to be behavior
comparable to XML. That expectation won't be met.

Contrast this with BER and friends, where there is no expectation that you'll
be able to see the "tag names" unless you have the schema.

> I should mention one thing - I have only participated in this thread because
> I did not think it is was clear why some things use + and some use -. For the
> uses of senml-exi that I care about senml-exi will work just as well as
> senml+exi. But what happens here seem like it will impact other uses of EXI.

That's actually rather doubtful, since the criteria for +suffix are, AFAIK,
fairly unique.

No, it's the design choice to have schema dependencies in the absence of a
reliable way to know what the schema is that is going to impact EXI. For
example, schema-aware EXI as content encoding appears to be a nonstarter, and
that's going to be true irrespective of how it is handled as a suffix.

				Ned