Subject: [core] A question on handling of option parameters defined in HTTP

From: Yusuke DOI <[email-masked]>
Date: Thu, 09 Aug 2012 11:54:03 +0900

Dear all,

As I spoke up to mic on vancouver, I have interest on option handling in CoAP.

In SEP2, we had to make 'schema negotiaton' on (schema-informed) EXI communication, and used content type parameters as follows:

 Accept: sep-exi;level=[schema version]

I think current CoAP cannot carry such parameters, and I need it to use schema-informed EXI over CoAP.
(Note: SEP2 is just an example. I cannot say anything on the use of CoAP on future SEP2)

Is there any reasonable way to carry such parameters?

For example, creating 'parameter' option to add parameter on option is acceptable?

Such as:
 Option 'Parameter' (elective, proxy-safe)
 option value: (uint)option index, (string)parameter key, (string)parameter value
 
describes given parameter (key-value) on the option at the given option index on the packet order.

Example:
 CON GET
 Uri-Port: ["/some-url"]
 Accept: ["some-exi"]
 Parameter: [1]["level"]["1"]

The first argument of Parameter option mean this parameter is for 1st (0-origin) option. This request corresponds to HTTP equivalent:

 GET /some-url
 Accept: some-exi;level=1

Please let me know your feelings on my idea.

Regards,

// Yusuke DOI <[email-masked]> Corporate R&D Center, TOSHIBA Corp.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Aug 2012 12:01:15 +0200

> Is there any reasonable way to carry such parameters?

As of today, you would have to register a new content-type number for each value of the parameter.

E.g., for text/plain; charset=utf-8 we registered ct=0.
If we needed any other charset (heaven forbid, no we don't), we'd have to register another ct value.

If there is a strong use case for being able to supply an arbitrary parameter to a media type, then the solution you suggested makes a lot of sense.

So if we were to register audio/rtp-midi (RFC 6295), we might register a ct value for:

audio/rtp-midi; rate=*; j_sec=*

or some such, and the first parameter option would then set the rate, and the second one the j_sec.
Doing this at the full generality, we'd probably arrive at a language not unlike URI templates...

Or we'd simple have a parameter option that has both parameter name and value, and set the first one to "rate=32500", and the second one to "j_sec=recj".

Of course, Accept can be repeated, and it is not quite clear where to put the parameters for each, which might call for an envelope option...

Lots of complexity, so we'd need a really good usecase to motivate this.

So:

Would the basic scheme (registering a separate content-type number for each level you need) work in your use case?

Grüße, Carsten

---

From: DOI Yusuke <[email-masked]>
Date: Sun, 12 Aug 2012 18:10:42 +0900 (JST)

Carsten,

> Would the basic scheme (registering a separate content-type number for each level you need) work in your use case?

I guess this scheme may work for me, as far as anybody can
register/find every content-type number at his/her needs. In this
case, I guess the problem is cost-benefit comparison between 'a new
open registry management about content type number' vs. 'a new option
to add arbitrary parameter with inefficient ASCII/UTF-8/whatever
codings'.

There may be another application (maybe with HTTP-CoAP Proxy) which
want to make use of arbitrary parameters on HTTP option. I'm not sure
CoAP {should|doesn't need to} be prepared to such application.

> Of course, Accept can be repeated, and it is not quite clear where to put the parameters for each, which might call for an envelope option...

My idea is use of option index in packet order. If there are one URI
Path option and two Accept options, index [0][1][2] should point each
option clearly.

// Yusuke DOI <[email-masked]>, TOSHIBA R&D Center

From: Carsten Bormann <[email-masked]>
Subject: Re: [core] A question on handling of option parameters defined in HTTP
Date: Thu, 9 Aug 2012 12:01:15 +0200

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 12 Aug 2012 13:22:00 +0200

Hi Yusuke,

> register/find every content-type number at his/her needs.

coap-11 section 12.3 says (about the CoAP media type registry):

   The IANA
   policy for additions in the range 256-65535 inclusive is "First Come
   First Served" as described in [RFC5226].

So we already have the wide-open policy in place that would enable this.

> In this

Indeed.  Here is my attempt at this comparison:
Registering new numbers for each new level of the XML profile does require (1) a little bit of clerical work and (2) that the number gets inserted into implementations.  But implementations already need to be changed if they want to handle the new profile, so that cost is relatively small.
A new CoAP Option requires new code, new interop testing, and is another source of bugs.

So, based on the information I have, I think I would prefer to handle this specific use case using the wide-open registration policy of CoAP content-type numbers.

> CoAP {should|doesn't need to} be prepared to such application.

I'm not sure either.
For now, I think "YAGNI" applies ("you ain't gonna need it"; don't add complexity for a potential future use case if (1) you don't know whether you will actually ever need it and (2) you will have a chance to add the necessary complexity later, at a time when you know more about the actual properties of the use case -- I think both conditions are satisfied).

> Path option and two Accept options, index [0][1][2] should point each

Ah, I see.  I'd prefer to use some form of nesting to indicate such a relationship.  
But, from the above, it seems we aren't defining this option yet.

Grüße, Carsten

---

From: DOI Yusuke <[email-masked]>
Date: Mon, 13 Aug 2012 14:22:20 +0900 (JST)

Hi Carsten,

Thank you for your insights.

Then, my question is very simple. Is 16 bit space enough or not for
parameterized content types?

> So we already have the wide-open policy in place that would enable this.

For content type, it's enough values. But for parameters, I'm not sure
16 bit (minus 256) is enough or not. 60 thousands seems to be big
enough for now, but I feel anxious if it's enough for every
parameterized content types. My use case needs a new ct number for
every version of XML schema used in EXI communication (including
proprietary ones).

If I'm going to be selfish, I may register (not-yet-existing) 1000
versions of schema ID before it runs out. As we may need them per
application per clients at maximum, 1000 is a large but not
*unrealistic* value we need in my guess. 

Then, the ct value space only serves 60+ companies/groups with similar
requirements.

> A new CoAP Option requires new code, new interop testing, and is another source of bugs.

Yes I agree. If requirement benefit does not outweigh the cost, we
shouldn't do that. On the other hand, I think content-type ID space is
precious (costy) to consume them for every combination of content-type
parameters.

Yusuke

From: Carsten Bormann <[email-masked]>
Subject: Re: [core] A question on handling of option parameters defined in HTTP
Date: Sun, 12 Aug 2012 13:22:00 +0200

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 13 Aug 2012 08:10:42 +0200

On Aug 13, 2012, at 07:22, DOI Yusuke <[email-masked]> wrote:

> Is 16 bit space enough or not for

Good question.
There is no strong reason to stay within 16 bits, so if we foresee a strong demand we could extend this to, say, 3 bytes.
(We also may want to reserve some space to continue the experiment under http://svn.tools.ietf.org/svn/wg/core/mediatypes.txt if people find that useful.)

There is also a question how well IANA will be able to handle such a large registry.
AFAIK, so far the largest registry is http://www.iana.org/assignments/enterprise-numbers with some 40000 entries.
But the media type registry would likely be allocated in chunks, so it might be less onerous.

Grüße, Carsten

---

From: DOI Yusuke <[email-masked]>
Date: Mon, 13 Aug 2012 18:55:45 +0900 (JST)

Carsten,

3 or 4 bytes seems to be enough in this case.  I can agree on your
idea to loosen the 16-bit length limit of content type id.

I guess the assignment could be something like IP address prefix
assignment.  Then ct registry could be something like this:

   ...
- application-example.net-exi 0x00f3.1000-0x00f3.1fff
- application-example.org-exi 0x00f3.2000-0x00f3.2fff
   ...
- application-very.large.example.com-exi 0xf000.0000-0xf000.ffff

In terms of fairness and efficiency, I guess block assignment should
be done on some larger number (not from 256). My idea is to assign
larger block on larger number zone, smaller block on smaller number
zone (as shown above). Selfish bulk assign request should pay cost for
(slightly) larger packet, isn't it?

From: Carsten Bormann <[email-masked]>
Subject: Re: [core] A question on handling of option parameters defined in HTTP
Date: Mon, 13 Aug 2012 08:10:42 +0200

---

From: Bert Greevenbosch <[email-masked]>
Date: Tue, 14 Aug 2012 00:59:58 +0000

Hi Yuseke, Carsten, all,

I can see some value of having a larger content type id space, with a suffix mechanism for application specific parameters. I guess extending the value space is easily done by allowing more bytes in the Content-Type option and adjusting the description of the "ct" attribute.

I think we need to be careful about bulk assignments though, as it should not be easy to occupy large blocks that afterwards remain unused. So if we go this way, we should consider some IANA guidelines and extra requirements for bulk assignments.

Another question is if the client and server can remain oblivious of the distinction between the base content type and the parameters. One of the main usages of the content type is the "accept" option, where the client indicates the acceptable content types to the server. Currently, if the client can accept multiple content types, it will include one option for each content type. But if we allow to split the content type in base and parameters and bulk assignment, do we also need to adjust the semantics of the "accept" option to allow bulk acceptance?

B.T.W. in the current version of CoAP, the content type and media type seem to be the same thing. Is there a reason to use two different terms?

Best regards,
Bert

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of DOI Yusuke
Sent: 13 August 2012 17:56
To: [email-masked]
Cc: [email-masked]
Subject: Re: [core] A question on handling of option parameters defined in HTTP

Carsten,

   ...

> But the media type registry would likely be allocated in chunks, so it might be less onerous.
_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 14 Aug 2012 09:07:08 +0200

> B.T.W. in the current version of CoAP, the content type and media type seem to be the same thing. Is there a reason to use two different terms?

Hi Bert,

let me try to answer this specific item:

In coap-09, we added a column to the registry that specifies the content-encoding.
So a CoAP content-type number stands not just for the Media Type, but for the Content-Coding, too.
(So far that is always the identity encoding, but that might change for additional registrations.)

We probably should change the name of the registry to make this more obvious.

Grüße, Carsten

---

From: Yusuke DOI <[email-masked]>
Date: Thu, 16 Aug 2012 12:13:40 +0900

Dear Bart,

Thanks for the comments. I feel anxous again about 'ct number per content-type and any parameter combination' approach.

> I think we need to be careful about bulk assignments though, as it should not be easy to occupy large blocks that afterwards remain unused. So if we go this way, we should consider some IANA guidel…

As the space is for content-type parameters and combination of parameters (it may corresponds to anything), it may not be observable how many combinations are actually used.

> Another question is if the client and server can remain oblivious of the distinction between the base content type and the parameters. One of the main usages of the content type is the "accept" opt…

Ouch. I don't think this should happen. Because parameters could be anything, I think it's too difficult to define a general behavior to distinct generic ct and specific ct.

If it's not too late, I'll write up some proposal draft(s) for arbtrary parameter option (by number and by string).

Yusuke

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 20 Aug 2012 12:34:11 +0300

Yusuke DOI wrote:
> If it's not too late, I'll write up some proposal draft(s) for arbtrary
> parameter option (by number and by string).

I think it would be a good idea to explore this a bit. Maybe don't go
into details yet, just describe the the problem, give a few motivating
examples, and outline your solution. Doesn't have to be a draft if
it's not too long.

Klaus

---

From: DOI Yusuke <[email-masked]>
Date: Wed, 22 Aug 2012 13:48:19 +0900 (JST)

Hi Klaus, and all,

As a weekend work, I wrote a draft and just submitted as proposal to
solve parameter problem in some use cases.  (including my EXI
content/schema negotiation)

>	Title           : CoAP Option Parameter Option
>	Author(s)       : Yusuke Doi
>	Filename        : draft-doi-core-parameter-option-00.txt
>	Pages           : 7
>	Date            : 2012-08-21

Any comments are appriciated. 

A trick is to make three options with identical functionarity
(Parameter-Option 1, 2, 3). This is because parameter options need
index to another option, and calculation of option index could be
dirty code if the parameter option is before the other option. I think
there was rough agreement to make three partitions of option number in
the last F2F, I made a parameter option per partition.

A concern is to have complex data type (1-octet integer, 2-octet
integer, string). I don't know other options with complex type. Is
there any reason to avoid it?

Regards,

From: Klaus Hartke <[email-masked]>
Subject: Re: [core] A question on handling of option parameters defined in HTTP
Date: Mon, 20 Aug 2012 12:34:11 +0300

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 23 Aug 2012 01:31:19 +0000

Hi Yuseke, all,

I have reviewed your "draft-doi-core-parameter-option" document. I realise that this idea still needs to ripen, so I hope the following questions can help getting a clearer view on the proposed mechanism:

(1) In section 2.2, the server can accept any rate, whereas the client chose the rate 32500 and signals it through the content-type. How did the server indicate the acceptable rates to the client?

(2) In the other direction, how can the client indicate acceptable parameters to the server?

(3) What is the rationale behind the "oid" field? I understand that it is for indexing the option-parameters. Does that mean that a content-type "some/content-type;par1=8;par2=9" would have two "Option-Parameter" options, in which the one for par1 has "oidx=0" and par2 "oix=1"?

(4) For the "value" field, I guess you propose to use a text string with the same content as the "value" field from RFC 2045?

(5) Why do you define 3 "Option-Parameter" options? What is the related implementation complexity, and how does having three separate options resolve it?

(6) To avoid confusion with CoAP options, would it be helpful to rename "Option-Parameter" to "ContentType-Parameter"?

I see value in content type parameters, especially as HTTP already supports a similar mechanism. So I look forward to discuss more about how this could be reflected by CoAP.

Best regards,
Bert

---
Bert Greevenbosch M.Sc.
Senior Research and Standardisation Engineer

email: [email-masked]
telephone: +86-755-28978760

HUAWEI TECHNOLOGIES CO.,LTD.
Building F1 / Floor 8
Huawei Industrial Base
Bantian, Longgang District
Shenzhen  518129
P.R. China

*******************************************************************************
This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it!
******************************************************************************

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of DOI Yusuke
Sent: 22 August 2012 12:48
To: [email-masked]
Cc: [email-masked]
Subject: Re: [core] A question on handling of option parameters defined in HTTP

Hi Klaus, and all,

Regards,

---

From: Yusuke DOI <[email-masked]>
Date: Thu, 23 Aug 2012 16:21:34 +0900

Hi Bert,

Thanks for your review.

> (1) In section 2.2, the server can accept any rate, whereas the client chose the rate 32500 and signals it through the content-type. How did the server indicate the acceptable rates to the client?

This (server-to-client node/resource capability notification) is beyond the scope of this document. In (only) HTTP, I guess no standard way is defined. The client may try to POST rate 32500 anyway to find it is acceptable or not. If an application uses DNS-SD, the TXT RR could be good place to describe server capability. In CoAP, I guess it could be done with the link format. I've heard Kerry is working on the issue (draft-lynn-core-discovery-mapping).

> (2) In the other direction, how can the client indicate acceptable parameters to the server?

Client can send Accept option with appropriate parameteras it sends GET request to the server.

> (3) What is the rationale behind the "oid" field? I understand that it is for indexing the option-parameters. Does that mean that a content-type "some/content-type;par1=8;par2=9" would have two "Op…

It's index of the option in the CoAP message.
A  CoAP message may have the following options:

CON GET
[0]Uri-Host: "example.org"
[1]Uri-Port: 15683
[2]Uri-Path: "example.exi"
[3]Accept: example-exi (is an imagenary content-type for an example application with schema-informed EXI encoding)
[4]Accept: example+json (same as above, with JSON)
[5]Option-Parameter1: 3,9,"5" (aid=9 means "level")
[6]Option-Parameter1: 4,2,"0.8"  (aid=2 means "version")

In the first Option-Parameter1, its oidx=3. This parameter is for the option with index [3]. This is
  Accept: example-exi;level=5
The second one is
  Accept: example+json;version=0.8

Since this index is fragile to option add/remove, this is not a proxy-safe option.

> (4) For the "value" field, I guess you propose to use a text string with the same content as the "value" field from RFC 2045?

I guess UTF-8 should be 'modern way'. I have no strong opinion on it. It could be RFC2045-style, UTF-8 string, and opaque octets.

(let me change the order of your question)
> (6) To avoid confusion with CoAP options, would it be helpful to rename "Option-Parameter" to "ContentType-Parameter"?

I agree.

At first, I have been thinking the parameter is also usable for other options. But while thinking through your review, I noticed it's confusing to have generic option parameters because the same semantics could be expressed in different manner in CoAP options. For example, content preference among multiple Accept-able content type is described by the order of options. It may be harmful to have "q" parameter (the same semantics in HTTP) to CoAP.

> (5) Why do you define 3 "Option-Parameter" options? What is the related implementation complexity, and how does having three separate options resolve it?

This is because I didn't want adding a parameter BEFORE its target.

The oidx of Option-Parameter points another option ('target') by the order in the packet. If I need to put parameter options before the target, I need to wait for the header building process to decide the oidx field.

However, as I agreed on (6), if this option only points to Content-Type and Accept header, a option number larger than Accept option is enough. Or, separate option for parameter of Content-Type and parameter of Accept could be a good idea because Content-Type does not need oidx.

Regards,