Subject: [core] wglc: coap caching and observe, caching underspecified

From: Salvatore Loreto <[email-masked]>
Date: Mon, 16 Apr 2012 15:10:52 +0200

Hi there,

while reviewing the coap 09,
I have notice that the response to an Observe request can of course be 
cached
however *can not* be used to answer simple GET requests.

 From Section 5.6 a cache key consists of
the request method, target URI and all the options used to obtain the 
stored response,
and then based on the second bullet of Section 5.6

o all options match between those in the presented request and those
    of the request used to obtain the stored response (which includes
    the request URI), except that there is no need for a match of the 
Token,
    Max-Age, or ETag request option(s), and

a *simple GET* does not match the Cache Key for a stored response to an 
Observe request.

I want also suggest to add some text to clarify a little better the 
cache model,
even if I implicitly assume that we are using the same of HTTP where

   Each cache entry consists of a cache key and one or more HTTP
   responses corresponding to prior requests that used the same key.

cheers
Salvatore

-- 
Salvatore Loreto, PhD
www.sloreto.com

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 16 Apr 2012 17:06:14 +0200

Salvatore Loreto wrote:
> the request method, target URI and all the options used to obtain the stored
> response,
>    the request URI), except that there is no need for a match of the Token,

I was about to report the same issue, because the same problem arises
with the Block options.

Proposed change:

   all options match between those in the presented request
   and those of the request used to obtain the stored response
   (which includes the request URI), with the exception of those
   options that are recognized by the cache and defined to opt
   out from this rule, and

Addition to section 5.10.1. (Token):

   The Token Option MUST be ignored when deciding
   if a stored response can be used to satisfy a request
   presented to a cache.

The Max-Age option actually can not occur in a request.

The usage of the ETag option in requests should be clear.

Some text needs to be added to -block and -observe for the Block1,
Block2 and Observe options.

Klaus

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 16 Apr 2012 17:22:38 +0200

>     options that are recognized by the cache and defined to opt
Hi Klaus

that does not work
you are  proposing to let the Cache free to decide if and how to respond 
to a request
on behalf of the origin server
this would just open a can of warms from the interoperability prospective.

I can not yet comment about Block as I have not reviewed it however I 
can tell you that
An Observer request is a different request from a plain GET request and 
a sensor
can decide to answer differently to them.

All in all the* Cache behavior definition is not just an editorial issue*,
it is a an important technical issue that needs to be solved in the 
right way.

/Sal

> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 16 Apr 2012 17:29:15 +0200

Salvatore Loreto wrote:
> that does not work
> you are  proposing to let the Cache free to decide if and how to respond to
> a request

What I mean is:

A cache is presented with a request that contains an Observe option
(elective). If it
- does not recognize the option, it ignores the option;
- does recognize the option, -observe overrides the rule that the
Observe option is part of the cache key.

A cache is presented with a request that contains a Block option. If it
- does not recognize the option, it rejects it with a Bad Option response;
- does recognize the option, -block overrides the rule that the Block
option is part of the cache key.

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 16 Apr 2012 18:07:27 +0200

Second attempt:

    all options match between those in the presented request
    and those of the request used to obtain the stored response
    (which includes the request URI), with the exception of those
    options where the option definition says otherwise, and

i.e. it's the option definition that decides if an option is part of
the cache key or not, not the cache implementation.

Klaus

On 16 April 2012 17:22, Salvatore Loreto <[email-masked]> wrote:
>>> stored
> I can not yet comment about Block as I have not reviewed it however I can
> tell you that
> An Observer request is a different request from a plain GET request and a
> sensor
> it is a an important technical issue that needs to be solved in the right
> way.
>
> /Sal
>
>>

---

From: Thomas Fossati <[email-masked]>
Date: Mon, 16 Apr 2012 19:45:05 +0200

Hi Klaus,

> the cache key or not, not the cache implementation.

this simply does not scale.

Options should be as transparent as possible to caches because otherwise each hop is forced to understand the end-to-end semantics to enable the forwarding -- which is clearly not a desirable behavior.

The currently defined mechanism is really heavy, and mimics the an HTTP cache in the hardest scenario, i.e. that with a Vary header with nearly all the headers in it.

What you are proposing would make this already implicit Vary depend on out of band information (i.e. each RFC describing a new option) :-)

Personally, I would like to keep caching as simple as possible: 
- use the URI as the lookup key (only for GET's)
- query the cache and then run through the result set to return the first match depending on the content negotiation options (i.e. Accept, ETag, and If-None-Match)

Very basic, trivial to implement in most scenarios, and guaranteed to work in enough use cases to be useful.

(Well, we still need to think carefully about the interaction of options like Observe, which heavily modify the semantics of the method -- BTW, waka, an HTTP/2.0 proposal by Roy Fielding has an explicit MONITOR method: see http://www.ietf.org/proceedings/83/slides/slides-83-httpbis-5.pdf)

Anyway, starting from a simple design could help us coming with a solution that works with less pain.

Yes, perhaps it'd be worth to stop and think a little bit more about CoAP caches design.

My two cents.

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 16 Apr 2012 20:20:23 +0200

Hi Thomas,

> Options should be as transparent as possible to caches because otherwise each hop is forced to understand the end-to-end semantics to enable the forwarding -- which is clearly not a desirable behav…

That's the current behaviour specified in coap-09. Section 5.4.1.
(Critical/Elective) states:

     o Unrecognized options of class "elective" MUST
        be silently ignored.

     o Unrecognized options of class "critical" that
        occur in a confirmable request MUST cause the
        return of a 4.02 (Bad Option) response

Intermediaries and caches are not exempt from this, i.e. if there is
one cache in the path that doesn't recognize an option, either the
option is filtered out or the request is failed.

This problem has been brought up before, and I agree that it's not a
desirable behaviour. But I'm not sure there is a solution to it:

A "critical" option in a request means that the sender is not
interested in the response if the option is not understood. It is
critical that the recipient understands the option in order to
understand the full request. If a cache doesn't understand a presented
request, how can it satisfy it with a stored response that is the
result of a different request it didn't understand?

Klaus

---

From: [email-masked]
Date: Mon, 16 Apr 2012 16:31:24 -0400

Hi Klaus,

> result of a different request it didn't understand?

there is a significant difference here: a caching proxy is not an
endpoint, it's an intermediary.  As such it doesn't produce any
response, it just try to reply with a matching cached response produced
by an origin server: it's just a dumb node :-)

Actually, it's conceptually wrong to leave an intermediary the ability
to drop traffic just because it does not understand its semantics.
Criticalness (?) should be an end-to-end, not an hop-by-hop attribute.

Keeping with the "Vary" metaphor, we could add the odd-numbered options
automatically to the implicit Vary used by the cache selection
algorithm, and leave the even-numbered out.

If the proxy finds no match response in cache it just forwards what has
been received to the intended endpoint: no drop on the forwarding plane
because of unrecognized criticalness.

This would play smoothly with critical options, while letting a caching
proxy be basically agnostic of the carried semantics.

What do you think ?

---

From: Klaus Hartke <[email-masked]>
Date: Tue, 17 Apr 2012 03:45:55 +0200

Hi Thomas,

> because of unrecognized criticalness.

let me rephrase this to make sure I understand it correctly:

* Recognized critical option - act according to the option definition.
For example, the client's token is not part of the key cache, and an
intermediary generates its own token when it makes a request to the
server to satisfy the client's request.

* Recognized elective option - act according to the option definition.
For example, a cache returns a 2.03 Valid response if any of the etags
in the request matches the etag of a fresh stored response, and an
intermediary includes the etags of its own stored responses rather
than those supplied by the client when it makes a request to the
server to satisfy the client's request.

* Unrecognised critical option - the option value is part of the cache
key, and an intermediary includes the option without knowing what it
means when it makes a request to the server to satisfy the client's
request.

* Unrecognised elective option - option value is *not* part of the
cache key, and an intermediary includes the option without knowing
what it means when it makes a request to the server to satisfy the
client's request.

So for recognized options it's the same as what I proposed: the option
definition decides if an option is part of the cache key (Uri-Host),
is not part of the cache key (Token), or has caching-related semantics
(ETag).

For unrecognised options, your suggestion of simply including the
option without knowing what it means breaks at least the ETag, Accept,
Size, Token, Observe, Block2 and Block1 options. Two examples:

* A proxy with cache does not recognize the critical Block1 option. It
is presented with a request that contains a Block1 option. It finds no
matching response, so it forwards the request to the server. It's
presented with another request that also contains a Block1 option, but
comes from a different client. It finds no matching response, so it
forwards the request to the server. The server cannot see that the two
requests are actually originating from two different clients, and
erroneously combines the two unrelated blocks.

* A cache does not recognize the elective ETag option. It is presented
with a request that contains an ETag option. It finds no matching
stored response, so it forwards the request to the server. The cache
receives a 2.03 Valid response and stores it. It's presented with
another request that contains an ETag option, but with a different
value. Because the ETag Option is not part of the cache key, the
stored 2.03 matches and is returned, erroneously validating the second
etag.

Klaus

---

From: Thomas Fossati <[email-masked]>
Date: Tue, 17 Apr 2012 09:03:50 +0200

Hi Klaus,

On Apr 17, 2012, at 3:45 AM, Klaus Hartke wrote:
> [...]

yes, nearly.  The missing bit is not there because I was not crystal clear (I implied a prior email of mine.)

A cache can't leave out from the resource selection algorithm the content negotiation options (ETag, Accept, If-None-Match).  So these attributes need to be understood.

Further, my concern is more about future options than those defined in the base spec since these are likely to be implemented pervasively.

> erroneously combines the two unrelated blocks.

This happens because of an optimization in Block, not for the segmentation logics per se, right ?

Wouldn't be better if end-to-end options were designed to be cache friendly instead of the other way around ?  If we don't keep this in mind I fear CoAP will not scale smoothly.

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 17 Apr 2012 09:41:28 +0200

Hi there,

I think we should define first in the CoAP draft what is a cache, which 
type of cache we can have in a CoAP network etc.etc.
then decide what is a Cache operation is etc. etc

my guess is that at the end it will be something similar to what we have 
in httpbis-p6

    Proper cache operation preserves the semantics of HTTP transfers
    ([Part2  <http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-19#ref-Part2>]) while eliminating the transfer of information already held
    in the cache.  Although caching is an entirely OPTIONAL feature of
    HTTP, we assume that reusing the cached response is desirable and
    that such reuse is the default behavior when no requirement or
    locally-desired configuration prevents it.  Therefore, HTTP cache
    requirements are focused on preventing a cache from either storing a
    non-reusable response or reusing a stored response inappropriately.

then my problem is with the option that change the semantic of a request 
like Observe.
This is something we have to solve!

cheers
Salvatore

-- 
Salvatore Loreto, PhD
www.sloreto.com

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 20 Apr 2012 06:42:16 -0600

This thread seems to be leading towards a separate bit in the option that indicates if a proxy needs to understand the option or not. Not sure what I think of that idea but it would be one possibility to consider. 

On Apr 17, 2012, at 1:41 AM, Salvatore Loreto wrote:

> Hi there,
> 
> I think we should define first in the CoAP draft what is a cache, which type of cache we can have in a CoAP network etc.etc.
> my guess is that at the end it will be something similar to what we have in httpbis-p6 
> 
>    ([
> Part2
> ]) while eliminating the transfer of information already held
> then my problem is with the option that change the semantic of a request like Observe.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 20 Apr 2012 14:51:18 +0200

> This thread seems to be leading towards a separate bit in the option that indicates if a proxy needs to understand the option or not. Not sure what I think of that idea but it would be one possibil…

I would like to see a real-live example for an option that is neither critical nor elective, first.  We have had this discussion for a long time, but nobody ever came up with an option that would *need* that third/fourth state.

Grüße, Carsten

---

From: Thomas Fossati <[email-masked]>
Date: Fri, 20 Apr 2012 15:23:26 +0200

Carsten,

> I would like to see a real-live example for an option that is neither critical nor elective, first.  We have had this discussion for a long time, but nobody ever came up with an option that would *…

the issue here is that we need a way to distinguish end-to-end from hop-by-hop semantics in options, otherwise proxies can't cooperate in making CoAP networks scale.

I don't mind if we want to do that with an explicit bit, or by some implicit rule -- as I proposed previously in this same thread -- but we need to do that.

The current caching rule set treats an intermediary the same way as an endpoint, forcing a flow to break at a forwarder whenever some end-to-end semantics is not understood.  And this is plain wrong.

We don't know how much CoAP will need to scale, but embedding such a limitation in the core protocol is something that we could regret.

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Fri, 20 Apr 2012 17:07:46 +0200

EXAMPLE 1

Suppose that in the future there will be the need to standardize a
"Date" option, e.g. containing the number of seconds from epoch.

That option will probably be elective, so that a client not
understanding it silently ignores the date.

But why a cache, not knowing this new option,  should ignore it as well?

Shouldn't be better to save that option together with other options
(known) in the cache, and forward them to the client, which is the
only recipient of that option?

EXAMPLE 2

A minimalistic cache implementation could not know the (still
critical) "Content-Type" option, and save its value along with the
representation treating them as "opaque binary string". Handling
"Content-Type" at a cache is not useful, because it is not required to
understand the content of the response, or even if the Content-Type
value makes sense or not (in the future more values will be added
hopefully).

So in this context why should that cache discard that representation?

#######

I am personally in favour of having some kind of bit somewhere to
distinguish between end-to-end and hop-by-hop options, as others like
Cullen and Thomas suggested.

Best,
Angelo

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Fri, 20 Apr 2012 17:09:48 +0200

A failed request, does not ever end in returning "4.02 Bad Option" for a cache.

In my understanding a cache only stores responses, so it can never
emit 4.xx (client error) messages to the server sending the "critical"
option in its response.

---

From: Lishitao <[email-masked]>
Date: Sat, 21 Apr 2012 01:47:47 +0000

Carsten

I think at least the proxy needs to understand the Observe option, so it can distinguish a normal get request and an observer request. 

Other example, like what we did right now for the Condition option (in draft-li-core-conditional-observe-01), the proxy also needs to understanding this option, 

so it will not combine those requests with different conditions but to the same resource together.

The usage of observe mechanism is different from HTTP (HTTP does not have this), so when we define caching for coap, a special consideration or special processing might be needed.

shitao

-----邮件原件-----
发件人: [email-masked] [mailto:[email-masked]] 代表 Carsten Bormann
发送时间: 2012年4月20日 20:51
收件人: Cullen Jennings
抄送: core WG
主题: Re: [core] wglc: coap caching and observe, caching underspecified

Grüße, Carsten

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 21 Apr 2012 03:20:30 +0100

Hi,

> The current caching rule set treats an intermediary the same way as an endpoint, forcing a flow to break at a forwarder whenever some end-to-end semantics is not understood.  And this is plain wron…

So I've no real opinion as to whether this extra bit is
needed/useful or not, but if its added then it'll need
to play well with how DTLS is done.

Be a shame to accidentally muck things up embedding some
hop-by-hop option into something secured end-to-end.

I would guess that any solution here that isn't "don't
use hop-by-hop options if you need (e2e) security" will
be fairly complex.

S

---

From: Klaus Hartke <[email-masked]>
Date: Sat, 21 Apr 2012 04:54:42 +0200

> This thread seems to be leading towards a separate bit in the option that indicates if a proxy needs to understand the option or not. Not sure what I think of that idea but it would be one possibil…

Picking up Thomas Fossati's suggestion of forwarding unrecognised
options, I could imagine a bit that says "this option can be safely
forwarded by an intermediary that doesn't understand it".

With such a bit, specifying out-of-band whether an option is part of
the cache key doesn't work any longer, so I guess we'd need another
bit that says "this option is part of the cache key".

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Sat, 21 Apr 2012 05:02:16 +0200

> emit 4.xx (client error) messages to the server sending the "critical"

Assume you have a client, a proxy with cache, and a server:

If the client makes a request to the proxy with a critical option that
the proxy does not understand, the proxy returns a 4.02 (Bad Option)
response to the client.

If the client makes a request to the proxy that the proxy understands
and the proxy makes a request to the server with a critical option
that the server does not understand, the server returns a 4.02 (Bad
Option) response to the proxy and the proxy in turn returns a 5.02
(Bad Gateway) response to the client.

(This is all defined in section 5.7.)

So you are right that the proxy never sends a 4.02 (Bad Option)
response to the server in reply to a response. The quoted text also
doesn't say that: it's only a confirmable *request* that MUST cause
the return of a 4.02 (Bad Option) response. (Section 5.4.1.)

Klaus

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 21 Apr 2012 12:11:02 +0200

Hi Stephen,

> use hop-by-hop options if you need (e2e) security" will

currently CoAP provides no way to do E2E security in presence of intermediaries.

BTW Angelo, me and Sal contributed a position paper about that (and argued about a CONNECT-like method for CoAP) in the SOS workshop in Paris.

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 21 Apr 2012 17:14:55 +0100

> currently CoAP provides no way to do E2E security in presence of intermediaries.

So I could buy not being able to benefit from
middleboxes if you want e2e security but the
mere existence of a middlebox shouldn't stop a
node from using e2e, or did you mean something
else?

> BTW Angelo, me and Sal contributed a position paper about that (and argued about a CONNECT-like method for CoAP) in the SOS workshop in Paris.

Couldn't make it there unfortunately.

S.

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 21 Apr 2012 18:40:39 +0200

> node from using e2e, or did you mean something

Yep, I meant exactly so.

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 21 Apr 2012 17:57:09 +0100

>> node from using e2e, or did you mean something

Well, with an AD hat on then that's going to be
an issue.

Why exactly is it good that the existence of a
middlebox prevents use of e2e security? (I'm not
saying you personally want that, the question is
to the wg)

S

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 21 Apr 2012 19:01:23 +0200

>> node from using e2e, or did you mean something

I think there is some confusion here.
(I'm not addressing Thomas specifically here, I know that he has thought a lot about the issues.)

CoAP does not address middleboxes in the general sense, at all.
We support intermediaries, which by definition break up an end-to-end relationship to do some processing (often some form of caching).

CoAP "does not support" end-to-end security in the sense that SMTP "does not support" end-to-end security.  Nothing in the architecture stops a CoAP application from employing end-to-end security by e.g. layering object security on top.

Whether a concept like "CONNECT" actually does something remotely useful for improving end-to-end security is up for discussion.  If you need end-to-end comsec, just don't go through an intermediary in the first place.  Use IP instead.  Much better.
Or, if you indeed need the services of an intermediary without trusting it, use object security on top (like SMTP would be doing).

Now we all know that, as of today, we don't have a credible object security story for constrained systems.  The CoRE working group is unlikely to change that.  JOSE might.  Burdening the standardization of CoAP with this can of worms would be insane.

Grüße, Carsten

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 21 Apr 2012 18:31:14 +0100

> I think there is some confusion here.

Quite possible.

> Now we all know that, as of today, we don't have a credible object security story for constrained systems.  The CoRE working group is unlikely to change that.  JOSE might.  Burdening the standardiz…

I agree with that last and am not trying to burden CoAP with
that. (It'd be great if a good object security scheme existed,
but that's just not a pre-condition of any sort.)

However, if e.g. a node that sometimes uses a proxy can't ever
use e2e security, then that'd be a problem. It'd also be a
problem if being "behind" a proxy, in practice meant you could
not turn on e2e security. (When I say "problem" I mean a thing
that needs to be properly justified, not a religious sin type
thing that can never be accepted.)

Its that latter that I took from Thomas' mail (without having
read the drafts, sorry).

If requiring e2e or not is the sender's choice for each session
then its probably fine. (Bringing us back to the start of this,
where setting that putative new bit related to caching would
seem to make no sense in an e2e-secured session.)

S

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 21 Apr 2012 21:27:34 +0200

>> I think there is some confusion here.

Sorry, I read "middlebox" as per RFC 3234, hence including proxies into the category.

Thus the confusion.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 21 Apr 2012 21:50:07 +0200

> Sorry, I read "middlebox" as per RFC 3234, hence including proxies  

A CoAP intermediary is not "on the datagram path" in the sense of 3234.

I think the confusion comes in when people start to think about CoAP  
intermediaries in the same way we tend to think about midleboxes in  
the IPv4 Internet.

In the IPv4 Internet, middleboxes are (or at least have the  
connotation of) "network fog" -- they are in the way, you are stuck  
with them, you can't get around them.
This is not the primary use I see for CoAP intermediaries.

CoAP intermediaries are voluntarily used by CoAP endpoints, typically  
to improve some performance characteristic (battery life, response  
time, throughput, reliability).
In the CoAP applications I'm seeing, CoAP intermediaries are not "put  
in the way" to restrict communication (or to bridge address spaces --  
most work on CoAP is IPv6 based).

Often, CoAP intermediaries mark trust boundaries.
There may be no end-to-end trust relationship at all, so no use for  
end-to-end security -- (CoAP-hop)-by-hop security is actually what is  
desirable in these cases.

However, there are also applications where the performance improvement  
is essential (i.e., the intermediary is "getting in the way" for  
performance reasons) but does not imply any trust.
That's where it would help to have alternate modes of security to the  
ones we already have.
This is also true where the intermediaries are placed mainly to  
perform a protocol transition, e.g. from CoAP to HTTP -- the only  
things end-to-end here are the URIs and the payloads.
It is conceivable to augment this by, say, tunneling DTLS through  
HTTP, so a CONNECT-style function might be made to work (i.e., the  
HTTP-side endpoint would run CoAP over DTLS over HTTP to the  
intermediary, then the DTLS is unpacked and sent on over UDP to the  
CoAP-side endpoint).
Alper Yegin also launched a trial balloon a while ago that had a  
special security protocol transported within CoAP options -- I think  
the WG consensus was we didn't quite know how to make that work.
There is a bit too much invention going on here for my security taste.
Also, all these proposals do not cover securing other aspects such as  
discovery.
For the applications I have looked at that really need to cross  
untrusted intermediaries, I'd rather do object security.

Gruesse, Carsten

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 21 Apr 2012 22:55:43 +0200

> A CoAP intermediary is not "on the datagram path" in the sense of 3234.

Actually, section 2.14 says that proxies fall under the same term "because the actual applications sessions traverse them."  
Ten years have passed since 3234, so the terminology may have shifted a bit in the meanwhile, but since there was no "updated by" nor "obsoleted by" I've just taken it for granted.

> In the CoAP applications I'm seeing, CoAP intermediaries are not "put in the way" to restrict communication (or to bridge address spaces -- most work on CoAP is IPv6 based).

There may be different use cases though, where some kind of policy is enforced (e.g. allow this, deny that, etc.) at an intermediary.

> There may be no end-to-end trust relationship at all, so no use for end-to-end security -- (CoAP-hop)-by-hop security is actually what is desirable in these cases.

Is the requesting client supposed to be aware of the hop-by-hop security in place ?

If yes (i.e. if the client can explicitly decide to request a secured path), we need to specify what a Proxy-Uri with a coaps scheme means in terms of requirements on the intermediaries.

> However, there are also applications where the performance improvement is essential (i.e., the intermediary is "getting in the way" for performance reasons) but does not imply any trust.
> That's where it would help to have alternate modes of security to the ones we already have.

Right.  That's exactly the scenario that we took into consideration for the SOS paper.  I think that Angelo has an use case for this.

> This is also true where the intermediaries are placed mainly to perform a protocol transition, e.g. from CoAP to HTTP -- the only things end-to-end here are the URIs and the payloads.
> It is conceivable to augment this by, say, tunneling DTLS through HTTP, so a CONNECT-style function might be made to work (i.e., the HTTP-side endpoint would run CoAP over DTLS over HTTP to the intermediary, then the DTLS is unpacked and sent on over UDP to the CoAP-side endpoint).

Yes, or use websocket as the tunneling facility (looks simpler.)

> For the applications I have looked at that really need to cross untrusted intermediaries, I'd rather do object security.

Well, at present the only way to go with object security in CoAP seems to roll your own crypto envelope, which is an *extremely* delicate task.  So, basically, we are left with no sensible choice at that layer, and neither we have one at the IP layer [*].  That's why DTLS is, in theory, an appealing way out.  But still we have to figure out how to make it fit into the constrained devices without the application bits being spilled on the desk :-)

[*] there was some work on IPsec at SICS in collaboration with Lancaster University, someone knows more about the current status ?

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 21 Apr 2012 23:38:33 +0200

> If yes (i.e. if the client can explicitly decide to request a secured path), we need to specify what a Proxy-Uri with a coaps scheme means in terms of requirements on the intermediaries.

The client that asks an intermediary to do a "coaps://" request has pretty much the same problem as a local client that asks a local piece of software to do a "coaps://" request.
(Except that, in the latter case, it may or may not be easier to establish some side channels for transferring the security information.  But let's assume for generality it is not.)

So what I'd like to convey with a request, inside or outside the URI, is:

-- what are my requirements on the communications security?
   ("coaps" is just one bit, but doesn't tell me what kind of security I actually want -- what certificates/cert chains do I accept, what cipher suites are acceptable, ...)
-- what kind of (mutual?) authentication do I want to use?  Is that implicit in the comsec setup (as with PSK identities or client certificates)?  Or do I add things like oauth, password-style options like http authentication, cookies, ...
-- what other parameters do I want to supply to enable authorization (say, role selection in a chinese wall scenario)?

etc.

HTTP solves essentially none of that, so there is nothing we can go ahead and copy.

So far, the working hypothesis is that there is some local out-of-band mechanism to get a local DTLS implementation to do most of this for us.  This doesn't transfer to the proxy scenario, where we'd need a way to transport all these data from the client to the proxy so that it can do the right security for us.

Again, having a "coaps" scheme strikes me as pretty non-functional -- it conveys one bit, "hey, I'd like some security with that", but none of the crucial information listed above.  Useless.

Until we are able to put this stuff into or alongside a URI, I see no way to do this.
And it's none of the business of the CoRE working group to invent all this mechanism.

Grüße, Carsten

PS.: http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05026.html
points to another thread around the same basic problem.

---

From: Thomas Fossati <[email-masked]>
Date: Sun, 22 Apr 2012 12:22:22 +0200

> And it's none of the business of the CoRE working group to invent all this mechanism.

I think instead that there is room to provide a decent level of security even in presence of proxies.  Either using a CONNECT-like primitive, or forcing intermediaries to always act on coaps URIs in some pre-arranged way (e.g. CoAP spec says that each proxy in the chain has to provide a security level at least "X" when the coaps scheme is used -- where X could be TLS or DTLS with mutual auth.)

Or, if we don't agree that this is possible, we should state it clearly in the spec: "resources in the coaps scheme cannot be retrieved through proxies, and are therefore not allowed in Proxy-Uri."

> points to another thread around the same basic problem.

Pushing security related meta-info in the URI may close our issue, but then opens another which may be actually bigger, ie. the secure binding of these meta-info to the underlying secure channel.  This all leave us in a situation where in the short/middle term we don't have a practical solution in place.

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 22 Apr 2012 13:56:36 +0200

On Apr 22, 2012, at 12:22, Thomas Fossati wrote:

> I think instead that there is room to provide a decent level of security even in presence of proxies.  

(I think we should clearly differentiate between trusted and untrusted intermediaries.)

> Either using a CONNECT-like primitive,

(This would be the untrusted intermediary?)
I don't know how to do this in a way that is not just tunneling DTLS.
Instead of tunneling DTLS, I'd rather send it over (UDP over) IP.
That already works.

> or forcing intermediaries to always act on coaps URIs in some pre-arranged way (e.g. CoAP spec says that each proxy in the chain has to provide a security level at least "X" when the coaps scheme is used -- where X could be TLS or DTLS with mutual auth.)

(Trusted intermediary?  But not trusted enough to set the outgoing security properly?)
So where do all the security parameters come from that I listed?
(I don't think there is such a thing as a "security level".)

> Or, if we don't agree that this is possible, we should state it clearly in the spec: "resources in the coaps scheme cannot be retrieved through proxies, and are therefore not allowed in Proxy-Uri."

Since the coaps scheme is all but meaningless, it is indeed not hard to "support" it.
The more important question is what are the security objectives that can be met.

> Pushing security related meta-info in the URI

(important point: I said "into or alongside" the URI...  
I don't have an answer on how to do this yet.)

> may close our issue, but then opens another which may be actually bigger, ie. the secure binding of these meta-info to the underlying secure channel.  This all leave us in a situation where in the short/middle term we don't have a practical solution in place.

Can you give an example where such a binding would be needed?

Grüße, Carsten

---

From: Thomas Fossati <[email-masked]>
Date: Sun, 22 Apr 2012 21:00:04 +0200

> (I think we should clearly differentiate between trusted and untrusted intermediaries.)

Sure, this may represent a basic discriminant.  

Anyway it's not always possible for a client to assume it (eg. if multiple proxies may be traversed and trust is not transitive.)  And this is always true whenever the trust relationship has to be composed hop-by-hop and there is no E2E verification. 

> Instead of tunneling DTLS, I'd rather send it over (UDP over) IP.

How does it work if proxy is an ALG but not a BR ?  You seem to assume that the proxy is always a BR.

> I don't have an answer on how to do this yet.)

If it doesn't get its way into the URI, it has to be specified as a CoAP option, or in some other (to be defined) CoAP specific way.

This train is headed straight to an in-CoAP security mechanism (which, given the peculiar nature of CoAP deployments and target hardware seems to me like the less painful approach.)

> Can you give an example where such a binding would be needed?

What I meant is: how do the client gets undeniable evidence that the requested COMSEC services have been enforced on each step of the path to the resource ?

Ie. how the requested security attributes are bound to the response so that the client can verify their application ?

I feel like we are waiting for Godot (pick one of DTLS, IPsec, JOSE, OAuth), while still missing the basic set of security services to operate CoAP networks with a decent level of security.

Perhaps it's time to think seriously about a CoAP specific COMSEC mechanism instead of relying on external blocks that for one reason or another are never completely satisfactory and/or applicable ?

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 22 Apr 2012 21:52:59 +0200

> Anyway it's not always possible for a client to assume it (eg. if multiple proxies may be traversed and trust is not transitive.)  And this is always true whenever the trust relationship has to be…

I have three use-cases in mind for the trusted intermediary:

1) the intermediary is the client's big brother.  The client doesn't really have the security smarts, so the client asks the big brother to please handle all that stuff for me.  That would be one of the trusted intermediary cases.  
2) the intermediary is the server's big brother -- again, the server leaves the handling of complicated things like certificates to its big brother (which probably talks via PSK to it).  While the client does not really "trust" the intermediary, it will have to trust it as much as it trusts the server.

In both cases, the intermediaries are pretty much autonomous to make security-related decisions; there is very little point to try to get something transitive going on.

3) the intermediary embodies a domain boundary, but is not tightly implicated in the security dealings of either the client or the server.  I can blindly trust it with my data, but I can't trust it to know (read: I need to tell it) how to talk to that other guy.  That requires a lot more work, and maybe that is what you have in mind.
For now, I'd try to avoid that case.

For the untrusted intermediary, I'm a bit at a loss, see below.

> How does it work if proxy is an ALG but not a BR ?  You seem to assume that the proxy is always a BR.

If I just want to send packets, I don't need an intermediary.
So I don't care whether the proxy is a router or not: I'm not talking to it in the first place.
I'm sending my packets right to where they have to go.

> If it doesn't get its way into the URI, it has to be specified as a CoAP option, or in some other (to be defined) CoAP specific way.

I was trying to point out that other spaces have similar problems here.
But, sure, we don't have to solve their problems.
However, as soon as we want to interoperate with, say, HTTP, we do.

> This train is headed straight to an in-CoAP security mechanism (which, given the peculiar nature of CoAP deployments and target hardware seems to me like the less painful approach.)

Again, how do we transport this over HTTP.  If the answer is "run CoAP over Websockets", that is easy.  If we actually want to interoperate with the HTTP world, we may need a grander vision.

> What I meant is: how do the client gets undeniable evidence that the requested COMSEC services have been enforced on each step of the path to the resource ?

It doesn't.

> Ie. how the requested security attributes are bound to the response so that the client can verify their application ?

If the server were to certify "yes, this request came in over DTLS with AES-128 based on a DHE handshake authenticated with ECC-mumble", the client still wouldn't know anything about what else happened on the way while the intermediary had unpacked the data.  Maybe I don't get what you want to be able to verify.

> I feel like we are waiting for Godot (pick one of DTLS, IPsec, JOSE, OAuth), while still missing the basic set of security services to operate CoAP networks with a decent level of security.

We know how to do that, and we have zeroed in on the use of DTLS for that.
We just don't know how to get miracles out of intermediaries, yet.
If we can't solve your use-case, we probably need to know more about your use-case.

> Perhaps it's time to think seriously about a CoAP specific COMSEC mechanism instead of relying on external blocks that for one reason or another are never completely satisfactory and/or applicable ?

Well, Alper tried that...
I'm pretty sure DTLS is the right comsec mechanism for us, at least as long as we don't try to use multicast.
But I'm always willing to be surprised...

Grüße, Carsten

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 23 Apr 2012 06:59:25 -0700

> And it's none of the business of the CoRE working group to invent all this mechanism.

It is 100% the job of CoAP to define how to secure CoAP.

---

From: Thomas Fossati <[email-masked]>
Date: Mon, 23 Apr 2012 20:27:18 +0200

Just to recap the base question I've posed, that may have been diluted a bit in the ping-pong with Carsten :-)

Proxy-URI and coaps scheme, currently an obscure corner of the spec.  How shall we treat the combo ?

1) Deprecate because we don't know how to make it work;
2) leave as an implementation/deployment "detail";
3) define a pass-through facility (ie. CONNECT-like method) to allow E2E DTLS;
4) define a comprehensive solution for both trusted and untrusted intermediary scenarios;
5) other?

---

From: Thomas Fossati <[email-masked]>
Date: Mon, 23 Apr 2012 21:07:18 +0200

> 1) the intermediary is the client's big brother.  The client doesn't really have the security smarts, so the client asks the big brother to please handle all that stuff for me.  That would be one o…

Right, let's put some taxonomy in place: this could be the "forward trusted proxy".

> 2) the intermediary is the server's big brother -- again, the server leaves the handling of complicated things like certificates to its big brother (which probably talks via PSK to it).  While the…

This one the "reverse trusted proxy".

> In both cases, the intermediaries are pretty much autonomous to make security-related decisions; there is very little point to try to get something transitive going on.

Well, it depends on how many proxies are in the chain from the client to the server.  Trust transitivity comes in its full splendor when nproxies > 1, i.e. when there is an explicit leap of faith of the client WRT the next, non directly visible, proxy in the chain.

> For now, I'd try to avoid that case.

This would imply also some sort of discovery/publishing, which is a (possibly overlooked) companion theme WRT the Proxy-URI handling.

> I'm sending my packets right to where they have to go.

Ok, so you are not talking about proxies here :-)

> However, as soon as we want to interoperate with, say, HTTP, we do.

Agreed, yet the HTTP-CoAP boundary is already a well defined interface, which provides a natural demarcation point where to handle all the needed translations -- even if these are more complex than a simple non-lossy manipulation of the wire format.

So to say that we may or may not be interested in keeping 1:1 semantic compatibility with whatever comsec mechanism will be (or is) specified in the future of the HTTP world since a CoAP-HTTP translator is already expected to be a well placed, robust and intelligent beast.

> Again, how do we transport this over HTTP.  If the answer is "run CoAP over Websockets", that is easy.  If we actually want to interoperate with the HTTP world, we may need a grander vision.

What are you thinking of specifically, OAuth 2 ?  If so, that could be ok for an unconstrained scenario (say, Repute over CoAP), but asking a constrained endpoint to implement a full blown web protocol (as you rightly say, "Grander") seems quite out of scope for their tiny brains.

>> What I meant is: how do the client gets undeniable evidence that the requested COMSEC services have been enforced on each step of the path to the resource ?

Well, lack of verification implies full trust.  And if this is to be assumed anyway, I think it is not even interesting to supply parameters from the client to the proxy through some possibly complex protocol (which should e.g. take into consideration that other clients on the same link are not as trustworthy as the proxy).  Just let the big brother do what it deems better for our good...

> If the server were to certify "yes, this request came in over DTLS with AES-128 based on a DHE handshake authenticated with ECC-mumble", the client still wouldn't know anything about what else happ…

I'd like to give the evidence to the client that the security services expected from the proxy have been enforced E2E.  Otherwise, if the proxy can basically do whatever it likes, I don't see the point in supplying the sec attributes in the first place.

> We know how to do that, and we have zeroed in on the use of DTLS for that.

Ok, and in fact we've taken this advice so seriously that we are actually trying to push DTLS on class 1 devices.  But this seems to be a much more difficult task than expected.

The shortfall in pushing the MTI comsec primitives on (a certain class of) end nodes must be acknowledged because it raises a very different model for (constrained) CoAP networks, where a "plaintext" island needs to establish trust relationships with another island through computationally robust intermediaries that can handle the delegated security -- i.e. where the proxy role gains more and more weight than initially expected.

DTLS is perfect if it can flow E2E, but if no end node is able to use it (and they currently can't, both because it is difficult to push DTLS on a class 1 node without leaving basically no space for the application, and because - even if DTLS were cheaply available -- proxies are still too thick to let the E2E flow pass through.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Apr 2012 21:15:52 +0200

> 1) Deprecate because we don't know how to make it work;

Wo know how to make it work, if:

1) the intermediary is trusted with the confidentiality and integrity of the data
2) the intermediary has enough context to translate the coaps:// into the security parameters needed.

The problem is that coaps://, like https://, relies on a lot of context.
For HTTP, this context was pretty much fixed for a decade (use TLS 1.0 without client authentication, authenticate the server's DNS name and verify its cert based on the 200 "known-good" root CAs etc.), so people often did not notice the problems until they started to use more complex setups such as OpenID.

> 2) leave as an implementation/deployment "detail";

That's what we have with https:// (and what won't work too well with coaps://).

> 3) define a pass-through facility (ie. CONNECT-like method) to allow E2E DTLS;

Not needed in the use-cases I know -- just use end-to-end DTLS.
What would be your use-case where you can't do that?
Why is it the job of the CoAP intermediary to forward your packets?

> 4) define a comprehensive solution for both trusted and untrusted intermediary scenarios;

Does that exist?

> 5) other?

Be my guest.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Apr 2012 21:35:00 +0200

Thomas,

good thoughts, thank you.

> What are you thinking of specifically, OAuth 2 ?  If so, that could be ok for an unconstrained scenario (say, Repute over CoAP), but asking a constrained endpoint to implement a full blown web prot…

I haven't looked at the complexity of doing a protocol that could interwork with OAuth.
Yes, OAuth itself may be something that must be left to the big brother intermediaries.

> I'd like to give the evidence to the client that the security services expected from the proxy have been enforced E2E.  Otherwise, if the proxy can basically do whatever it likes, I don't see the p…

Do you have a mechanism in mind to do that?

> Ok, and in fact we've taken this advice so seriously that we are actually trying to push DTLS on class 1 devices.  But this seems to be a much more difficult task than expected.

Well, there are implementations now that are tailored to class-1 environments (e.g., tinydtls).
It is probably going to be the most complex software component on the device, but we never expected otherwise for end-to-end security.

> The shortfall in pushing the MTI comsec primitives on (a certain class of) end nodes must be acknowledged because it raises a very different model for (constrained) CoAP networks, where a "plaintex…

That is another, quite valid way to do a deployment.
It should not be the only one.

> DTLS is perfect if it can flow E2E, but if no end node is able to use it (and they currently can't, both because it is difficult to push DTLS on a class 1 node without leaving basically no space for the application,

Olaf's number in Paris was < 13KiB with PSK and hardware AES (I think he has optimized it even a bit further since).  That is not a beautiful number, but within the capabilities of at least some class-1 devices.

> and because - even if DTLS were cheaply available -- proxies are still too thick to let the E2E flow pass through.

Again, if their job is to just pass through DTLS, they are IP routers, not CoAP intermediaries.

Grüße, Carsten

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 23 Apr 2012 23:46:10 +0100

Hi Carsten,

> Again, if their job is to just pass through DTLS, they are IP routers, not CoAP intermediaries.

Is there text somewhere saying something like: "don't setup a
gateway device that's a CoAP proxy but turns off IP routing
since that'd prevent devices "behind" that proxy using e2e
DTLS"?

I'm sure that'd need a good bit of wordsmithing and I'm not
at all sure where it'd go (being more of a deployment
consideration).

I do have to wonder though if that approach will work out.
Won't people be just too likely to turn off IP routing on
many gateway devices so they get that happy firewalled
feeling? (I'm not saying they're right, I'm worried it'll
happen regardless.)

If so, that'd mean lots of cases where e2e DTLS would fail
in practice, which seems like a bad outcome.

(Related question: is there any CoAP-level routing happening
in proxies that'd mean that the e2e DTLS packets really do
need to go to the CoAP process on the middlebox?)

S

---

From: Thomas Fossati <[email-masked]>
Date: Tue, 24 Apr 2012 08:47:40 +0200

> Do you have a mechanism in mind to do that?

No I've not really thought about it.  My guess is that we'd need a new option with something like:
(a) code representing the requested E2E security services
(b) a random transaction id
that is acknowledged (HMAC'd or PK signed) by the receiving party using an E2E credential (usual PSK or public key should work).

> Olaf's number in Paris was < 13KiB with PSK and hardware AES (I think he has optimized it even a bit further since).  That is not a beautiful number, but within the capabilities of at least some cl…

13K (+ 1K-2K if AES is done in software) is an interesting number indeed (last time I've tried TinyDTLS the example client was ~70KB, I'll give it another look, definitely!).

Yet we may still face the need to answer the hard question: who am I throwing out of the window, RPL or DTLS ?

And no, as you know very well, go and buy some other RAM/ROM is not an option :-)

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Apr 2012 11:20:29 +0200

> since that'd prevent devices "behind" that proxy using e2e

That would be a charter item of the end-to-end protocol police, no?

I don't think "saying" that will make any difference...

> feeling? (I'm not saying they're right, I'm worried it'll

As long as the above protocol police isn't invested with universal powers, you are probably right.

> in practice, which seems like a bad outcome.

Another potential outcome might be that people stop designing their networks around mandatory application layer gateways where they really need end-to-end security.
But maybe I'm fantasizing...

> need to go to the CoAP process on the middle box?)

One could imagine a constrained client using the hostname lookup (e.g., DNS) in an intermediary to talk to a server.  A bit like a SOCKS5 for CoAP and DTLS.

I went ahead and put a strawman CONNECT in coap-misc:

	http://tools.ietf.org/html/draft-bormann-coap-misc-15#section-5

I feel a bit dirty, I'm going to take a shower now...
I still hope that this lands in Appendix B of that document (the cemetery), but at least we now have something specific to discuss.

Grüße, Carsten

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 24 Apr 2012 10:48:26 +0100

> I feel a bit dirty, I'm going to take a shower now...

I feel your pain:-)

Having a strawman for this is a good approach I'd say
so folks get to figure it out.

Only other thing - some of Thomas' ideas sound like a
recent discussion on httpbis about making TLS always-on
for HTTP/2.0. I'm a bit skeptical that it can be done
well, but if it were to be attempted, there would I
guess be benefits in having similar approaches in core
and httpbis. (Note: I don't mean the same on the wire
format, so I should probably use the word architecture
but then I'm never quite sure what that means;-)

Some of the TLS discussion starts at [1]. Probably best
not to re-ignite the thread over there unless/until
there's a concrete proposal over here though.

S.

[1] http://lists.w3.org/Archives/Public/ietf-http-wg/2012AprJun/0085.html

---

From: Alper Yegin <[email-masked]>
Date: Tue, 24 Apr 2012 17:16:36 +0300

> Well, Alper tried that…

Yep, I did. draft-yegin-coap-security-options.

Folks found missing pieces to make it fully workable. Things that are easily fixable.
But more importantly, main objection I received was "we have TLS, we don't need anything else". 

So discussions like the one in this thread, and points like

> But I'm always willing to be surprised…
> 

and

"For the applications I have looked at that really need to cross untrusted intermediaries, I'd rather do object security."

are worth noting in this context.

Alper

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 Apr 2012 10:39:59 +0200

> so folks get to figure it out.

It also appears to be a good way to terminate a thread :-)

Back to the actual subject of this thread:
Note that CONNECT gets around the need to (as opposed to solves how to) specify security parameters.  We still have the need in coaps:// -- I'm not seeing anything equivalent being discussed for https://.

Note that, for https://, the need for discussion of "breaking TLS" is about retroactively opening up (without consent of the parties) the existing assumption of a homogeneous security semantics of https://.  For coaps://, we could do a lot more to capture intent/consent, so much of the discussion in that thread simply does not apply. 

Let me try to summarize that rather verbose discussion anyway:

-- Some site/network administrators require visibility into traffic
   -- whether for bogus legal reasons, legitimate malware protection
      or DLP requirements or whatever is pretty much irrelevant
-- Some end-points need to deny that visibility
   -- banking, paypal, ... but is that really all?
-- HTTPS has no way of expressing any consent or denial
   -- so the policy has to be cooked up out of thin air
   -- clients interpret "let's have some security" based on their
      random local policy (which may or may not include untrustable
      root CAs and MITM certs)
   -- intermediaries have policies that are second-guessing the
      end-points' policies (e.g., exception tables for banking)
   -- servers have no say (except flat out denying port 80), which is
      a pity as their administrators are often in a better position to
      provide informed input than the clients' end-users
   -- intermediaries can't really negotiate their policies in such a
      way that the end-points can live with the result
   -- users have no visibility about the actual outcome
-- (we can't even say whether HTTPS is specified to achieve
   confidentiality or just integrity protection)
-- Summary: HTTPS is horribly broken
   -- extending its security model to the entire Web (and not just the
      part that agreed to suffer by using the https:// scheme) would
      expose this breakage in unacceptable ways (this is what people
      mean when they say "all-or-nothing security").
   -- on the other hand "using encryption sparingly" is not really
      going to help HTTPS either, because there *are* many valid uses
      for security that require different policies than paypal, and
      the system already is bursting at its seams as is.

Note that I said HTTPS is broken, not TLS -- it is the combination of
TLS with a single implied one-size-fits-all security policy that is
causing the problem.

In the HTTP web, we are now building tables that map server domain
names to security policy information (e.g., firewall exception tables
for banking, client tables for HSTS, ...).  Hosts.txt for security
clearly does not scale beyond an elite club (which doesn't mean it
won't work well for Paypal).  Maybe linked data can help, but I'm not
sure its proponents have thought much about liability issues.

Other observations:
1) Conveying security policy towards the client in or alongside a URI
only provides part of the picture, as this information could, too, be
compromised by a MITM-style intermediary (Roberto Peon seems to have some
ideas here, promised a draft); there often needs to be some mutual
checking.
2) This checking is, however, fundamentally limited by clients that
collude with what would be an adversary from the point of view of the
server.
3) Ceterum censeo: Trying to use the CoRE WG to invent all the
required mechanism to solve this area of problems would be insane.

So let's try to get some focus on the issue what security parameters would have to come with a coaps:// scheme.

Grüße, Carsten

---

From: "Dijk, Esko" <[email-masked]>
Date: Thu, 26 Apr 2012 13:28:47 +0000

> We know how to make it work, if:
>

That was exactly the case I thought was missing from Thomas' list. There's no E2E security then. But that's how I interpret the current spec and it seems clear how the combo Proxy-URI/coaps is treated. The proxy simply acts in its role of a CoAP client who MUST use "coaps" to execute a request. Security parameters are the ones predefined for the "coaps" scheme. (This process is "clear" except for the mentioned lack of details in what a 'coaps' request means - but that's a more general issue not specific to this combo.)

But maybe others will see this as option 2. Anyhow, that's what the spec currently seems to define.

Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Carsten Bormann
Sent: Monday 23 April 2012 21:16
To: Thomas Fossati
Cc: core WG
Subject: Re: [core] Proxy-URI and coaps scheme

Does that exist?

Be my guest.

Grüße, Carsten

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 26 Apr 2012 15:30:47 -0600

> 4) define a comprehensive solution for both trusted and untrusted intermediary scenarios;

I don't want 2 and don't want 3.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 27 Apr 2012 08:25:05 +0200

On Apr 26, 2012, at 23:30, Cullen Jennings wrote:

> I don't want 2

well, obviously.

> and don't want 3. 

Do you mean you don't want a 3 at all (I proposed one straw man way of doing this in
http://tools.ietf.org/html/draft-bormann-coap-misc-15#section-5) or that you don't want this to be the only outcome?

I'm leaning towards:
-- 1 for now, and
-- start a separate work item on 4 (with some limits on "comprehensive"), and
-- maybe also further investigate 3.

Grüße, Carsten

---

From: Zach Shelby <[email-masked]>
Date: Fri, 27 Apr 2012 09:37:04 +0300

> http://tools.ietf.org/html/draft-bormann-coap-misc-15#section-5) or that you don't want this to be the only outcome?

At least personally I don't find 3 to be useful. There would be good reasons for enabling end-to-end TLS security work across a CoAP-HTTP proxy, but I think that falls under 4 in Thomas's list.

> -- 1 for now, and

Agreed.

> -- start a separate work item on 4 (with some limits on "comprehensive"), and

Smells like someone needs to start such a draft.

> -- maybe also further investigate 3.

If we really see a need.

Zach

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://www.sensinode.com
http://zachshelby.org  - My blog "On the Internet of Things"
http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

---

From: Thomas Fossati <[email-masked]>
Date: Fri, 27 Apr 2012 08:53:50 +0200

> Smells like someone needs to start such a draft.

Well, I have this embryonic idea about pushing a challenge-response scheme into the base request-response messaging model of CoAP.  Unfortunately I have yet to find the time to sketch a proposal.

Anyone who finds it as a sound approach, and would like to do some brainstorming, is definitely welcome.

---

From: "Dijk, Esko" <[email-masked]>
Date: Fri, 27 Apr 2012 08:10:25 +0000

Zach, Carsten, Thomas,

I'm curious what are the reasons you have to now favor deprecation (option 1)...?
Two arguments I have seen so far are "we know how to make it work" and "we don't know how to make it work".

If the argument is "we can't make E2E security work" then it implies we also must deprecate these cases:
- CoAP request with "coaps://" Proxy-URI
- CoAP request with "https://" Proxy-URI
- CoAPS request with "https://" Proxy-URI
- HTTP request with "coaps://" Request-Line
- HTTPS request with "coaps://" Request-Line

These are not obscure corners of the spec IMO, but clearly present throughout in -coap "Proxying" and "HTTP mapping".

Esko

PS The following cases should be ok as it's the client's own choice to *not* request E2E security:
- CoAPS request with "coap://" Proxy-URI
- CoAPS request with "http://" Proxy-URI
- HTTPS request with "coap://" Request-Line

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Zach Shelby
Sent: Friday 27 April 2012 8:37
To: Carsten Bormann
Cc: core WG
Subject: Re: [core] Proxy-URI and coaps scheme

Agreed.

If we really see a need.

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 27 Apr 2012 12:03:23 +0200

> I'm curious what are the reasons you have to now favor deprecation (option 1)...?

Well, deprecation is the wrong word (we use that when existing functionality gets out of fashion).
Saying "this doesn't mean very much until we define what it means" is more like it.

> - HTTPS request with "coaps://" Request-Line

There is no new problem with https:// -- the security parameters that are expected here are relatively well understood, even if the certainty in this space is quickly eroding.  Obviously, the URI does not give us a way to trigger other HTTP-based security features such as sending OAuth tokens in "Authorization:" headers (well, there is a lively debate on how to enable equivalent URI functionality right now; see also http://tools.ietf.org/html/draft-ietf-oauth-v2-bearer-19#section-2.3).  At some point it would also be nice for a client to be able to influence which cipher suites, root CAs etc. an intermediary accepts on the way forward.  But I think everybody understands that the ways to specify any such additional security parameters will be the subject of future work, outside of the CoRE WG, and nobody expects to be able to use e.g. PSK for HTTPS.

There is a problem with every use of coaps:// -- independent of whether that is interpreted in an intermediary or right in a client.  For use within a client, one could make the argument that the client is likely to be able to map aspects of the URI such as the authority to security parameters, e.g., it has a table that indicates that it tends to speak to the water meter using a PSK it has stored for this specific purpose.  There is a bit of a problem with HATEOAS here, as the server really should be able to indicate its preferences when supplying the next URI to use, but that's just reflecting the state of the art.

For intermediaries, the client component of the intermediary has to determine the security parameters for its outgoing DTLS connection.  The same argument as above can be made for *some* trusted intermediary scenarios.  Where the forward proxy chosen by a client is its "big brother", it may actually know more about the right way to set up the DTLS connection than the client itself.  Similarly, a border reverse proxy may know the best way to talk to the nodes in "its" 6LoWPAN.  The vague "coaps://" is appropriate here and need not be recommended against as long as we clearly describe the boundaries of this narrow application.

To me, option 1 means just that: clearly delimiting the area where coaps:// already "works" (and supplying a bit more detail where needed).
Of course, we want to expand that area, and that expansion is the work of option 4.

Again, the problem is not the proxying, but the fact that the coaps;// scheme is necessarily underspecified ("CoAP with *some* security" :-).

Grüße, Carsten

---

From: "Dijk, Esko" <[email-masked]>
Date: Fri, 27 Apr 2012 11:09:11 +0000

Thanks Carsten,

> To me, option 1 means just that: clearly delimiting the area where coaps:// already "works" (and supplying a bit more detail where needed).

That's an option 1 I can agree to!

Now besides the technical details: this means for this limited scope use case that the proxy is "allowed" to forward a response, obtained securely (coaps or https), containing sensitive data, via an unprotected channel to the client. I just point this out to check if anyone sees a problem with that. (The client at least seems not bothered; the server has no say here.)

regards,
Esko

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]]
Sent: Friday 27 April 2012 12:03
To: Dijk, Esko
Cc: Zach Shelby; Thomas Fossati; core WG
Subject: Re: [core] Proxy-URI and coaps scheme

Grüße, Carsten