Subject: [core] pre-draft: Request-Tag

From: Christian Amsüss <[email-masked]>
Date: Fri, 3 Mar 2017 18:32:29 +0100

Hello working group, hello OSCOAP people,

discussion from the OSCOAP draft has at different layers hit the problem
of blockwise requests being linked together only via the endpoint
address[1].

Encouraged by Göran, I'd like to suggest a new option to CoAP that would
primarily solve OSCOAP's problem, but also allow faster responses from
proxies by simultaneous processing of block1 requests.

Below is an early version of what I'd like to submit as a draft. I would
appreciate feedback on it, especially on the approach of packing this
into the endpoint definition instead of just defining a blockwise
extension. Note that the actual definition of the change is more concise
now than in my previous sketch, because many things I spellt out ther
explicitly fall into place automatically because of the endpoint
discrimination. (For example, the "atomic vs non-atomic" considerations
are now merely a reference to what blockwise already noted in its
security context).

Best regards
Christian

PS. what's the expected level of maturity for an individual submission
upload into the data tracker?

[1]: http://www.ietf.org/mail-archive/web/core/current/msg08299.html

============================
Request-Discriminator option
============================

Abstract
========

This memo describes an optional extension to the Constrained Application
Protocol (CoAP, RFC7252) that allows extending the endpoint context of a
request with an additional discriminator. This allows processing
concurrent requests that would otherwise be serialized by proxies in
presence of blockwise transfer, and linking blockwise requests on
transports that do provide authentication but no ordering guarantees.

Introduction
============

Blockwise transfer [RFC7959] in the presence of Block1 ties an exchange
to a combination of resource and remote endpoint, keeping any two
exchanges where those parameters match from happening simultaneously.
Wherever either of those parameters get conflated (eg. because a proxy
acts as a single endpoint towards a server when forwarding operations on
the same resource, or in Object Security of CoAP (OSCOAP) where all a
client's request are targetted at the same resource from an outside
point of view), the client needs to queue up requests to keep the
transfers from overriding each other.

(@@@ who else references 'same Endpoint'? other documents could be
affected of profit)

The OSCOAP security layer provides message integrity and replay
protection, but with blockwise, the security context (which augments the
endpoint address) alone is insufficient protection because it does not
(and does not indent to) forbid out-of-order delivery.

Both situations can be mitigated by allowing the client to add an
additional discriminator (the Request-Disciminator option), which must
be treated as a part of the (security context of an) endpoint as defined
in RFC7252 section 1.2.

Terminology
===========

The terms Endpoint, byte, @@@ are used as in RFC7252.

An exchange is @@@ the thing with tokens and not MIDs -- should check
with OSCOAP, i never really considered it there, and it didn't happen in
the tests

Request Discriminator: An opaque identifier chosen by the client. It can
  be absent or a string of up to 8 bytes, including the empty string.
  The phrasing as two words always indicates such a value, while
  Request-Discriminator (with dash) is the CoAP option in which it is
  transferred.

The Request-Discriminator option
================================

A new option is defined for all request methods:

    +-----+---+---+---+---+-----------------------+--------+--------+---------+
    | No. | C | U | N | R | Name                  | Format | Length | Default |
    +-----+---+---+---+---+-----------------------+--------+--------+---------+
    | TBD | x | x | - |   | Request-Discriminator | opaque |    0-8 | (none)  |
    +-----+---+---+---+---+-----------------------+--------+--------+---------+
    
    C=Critical, U=Unsafe, N=NoCacheKey, R=Repeatable

It is critical (because mechanisms may rely on endpoint identities not
to be conflated), unsafe (because the channels implied by endpoint
identities terminate at proxies), not repeatable. (@@@ unsafe makes
nocachekey irrelevant, but should we or not set it to ease proxy
implementations?)

A client may set a Request-Discriminator option to indicate that the
message's Endpoint is to be augmented with the opaque option value. The
server MUST NOT treat the message's endpoint as equal to any other
message's endpoint that does not have the identical
Request-Discriminator. The absence of the option and a zero-length
option are distinct. (@@@ i wouldn't insist on that -- especially with
the inclusion in the AAD it might be better to have absence and h""
equal, so we don't have variable types in the AAD)

The option is not used in responses. Response messages implicitly bear
their corresponding (via the token @@@ or better refer to Exchange
definition?) request's discriminator.

If a request that uses Request-Discriminator is rejected with 4.02 Bad
Option, the client SHOULD retry the request without it, but only if and
when

* no limitations on simultaneous operations on the same endpoint are in
  place, and

* the application does not use distinct Request Discriminators to
  preclude out-of-order delivery on the same endpoint.

For inclusion in OSCOAP
=======================

@@@ if this stays a document of its own, oscoap should make a normative
reference to it and state something like

Whenever the Block1 option is used as inner option, it is associated
with a Request Discriminator. The same Request Discriminator MUST NOT be
reused within a security context unless every single block sent has
elicted a protected response (and its sequence number is consequently
marked used). (@@@ if we're cautious, we could s/unless.*/at all/, but i
don't see any harm in allowing it.) OSCOAP requires that out-of-order
delivery of blockwise transfers is caught by the Request-Discriminator
option, so a client MUST NOT fall back to not using the
Request-Discriminator option if it encounters a 4.02 Bad Option
response.

Note that this does not require the use of the Request-Discriminator in
many cases. An application implementing OSCOAP MUST use that option at
least when a Block1 request has not been losslessly concluded[*], and a
server MUST NOT respond with 4.02 due to the presence of a
Request-Discriminator option. (@@@ or they could declare the complete
context invalid, but i don't think we want contexts to be that volatile)

[*]: Simple package loss will not make a discriminator unusable, because
there is still CoAP retransmission in place. Only when the block
transfer is aborted, or when the same block gets sent with a different
sequence number (@@@ may that be at all?), the discriminator is unusable
for any further blockwise transfer.

A Request-Discriminator option MAY be used when initializing a Block2
transfer as well. A server SHOULD, inside a Request-Discriminator, only
send response messages with matching ETags if they can be expected to
assemble into a consistent representation again. (@@@ imo that's a
requirement that there already is from the ETags themselves. jim, do you
think this is a starting point for your concerns about validating the
whole message?)

@@@ this is more of a verbal patch than something actually includable

The associated Request Discriminator will need to be included in the
AAD. Whether we include that in the request too to keep the AAD the
same both for request and response, or have it in responses only, I
don't care.

The Request-Discriminator option is added to the "E*" category and
like and listed together with Block1/2.

Notes on applications
=====================

@@@ spell out example exchanges: proxy forwarding multiple requests,
oscoap application. include a case with non-piggybacked reponses.

>         Client              Foe         Server
> 
> POST "incarcerate"(1/2) --->  --->
>                        <---  <---   2.31 CONTINUE
> POST "valjean"(2/2)     --->@
>                        <---RST
> 
> (Client: Odd, but let's go on and promote Javert)
> 
> POST "promote"(1/2)    --->  --->
>                       <---  <---    2.31 CONTINUE
>                            @ --->
>                             <---    2.04 Valjean Promoted!
> 
> (The @ indicates a maliciously delayed / wormholed package as used in

this can't happen in oscoap now any more because the incomplete first
POST permanently poisons the first request's request discriminator, and
the second request will have a different one, for which the server will
not accept the delayed "valjean" any more.

Rationale
=========

This part is informative and serves to illustrate why this option is
necessary, and how it is different from similar concepts.

Why not use...

* another port: Proxy implementations can work around the simultaneous
  transfer restrictions by using different ports as a client. This is
  not possible with some constrained implementations (which typically
  get their one static socket from the operating system). Moreover, for
  the OSOCAP inner-bockwise application, the best equivalent would be
  starting another context, which is application dependent and very
  costly.

  Moreover, some transports do not have any such variability (@@@
  over-serial, or is there something more complete that has the same
  limitation?)

* extend the blockwise mechanism with another option: That would not
  make things easier in the author's opinion.

* put a discriminator into OSCOAP: That would have the same effect for
  the OSCOAP inner-blockwise case. It would still not allow different
  OSCOAP requests to happen concurrently on a device (especially in the
  presence of proxying).

How is this identifier different from the...

* message ID and token: Both are defined on the same level as the
  Request Discriminator (that is, from endpoint to endpoint), but the
  discriminator has an even longer lifetime than the token in that it
  at least spans all the requests transferred within a blockwise
  transfer.

* OSCOAP's sequence number: As the above, the sequence number is only
  used for a single exchange, and does not span a blockwise transfer.
  Implementations might, however, opt to use the sequence number of the
  first package of a blockwise transfer as the Request Discriminator for
  the whole transfer.

* ETag: While the Request Discriminator option serves a similar purpose
  in blockwise transfers as the ETag (ETag allows the client to filter
  non-matching Block2 responses, Request-Discriminator allows the server
  to filter non-matching Block1 requests), the ETag is stably determined
  by the server (and can thus be used for caching), while the Request
  Discriminator is an ephemeral label used exclusively during the
  transfer.

The naming of options
=====================

@@@ this section will obviously be removed over time.

The naming of options is a difficult matter -- especially here where the
use to the application (describing which requests of a blockwise bunch
belong together) deviates from what it by its definition (and in
implementation) does, that is, provide a lightweight sub-channel inside
the channel described by the endpoint address.

Alternatives under current consideration are:

* Request-Tag: More on the Block1 application side, in parallel to the
  ETag that the client uses to make sure the responses to its Block2
  request match up.

  Could lead to confusion when used in any context that relies on
  endpoints but is not blockwise.

* Endpoint-Discriminator: On the other end of the spectrum. Technically
  correct, but its use in a blockwise transfer is not immediately
  obvious.

* Request-Discriminator: Some middle ground. I'd understand it not as
  much as something that discriminates between requests, but a
  discriminator introduced by the requester.

Standard hygiene
================

@@@ if this stays in at all, it'll be shortened into some kind of
'additional considerations' -- depending on how much discussion there is
on it.

This draft does something that might be considered dirty in terms of RFC
interaction: It defines, when implemented, additional semantics into the
term "same endpoint" -- one could say that it hijacks the term to be an
extension point. This is done right now because:

* It keeps RFC interdependencies low.

* It is compatible in the sense that whoever does not implement this
  option (and consequently responds 4.02 Bad Option to its use) do
  trivially implement the changed semantics by just not allowing the
  Request-Discriminator option to take any other value than none.

* It allows later or concurrent drafts to use the "same endpoint"
  semantics and optionally utilize this extension without mandating it
  as a dependency.

* RFC7252 already defines "Endpoint" to include the security
  association. Given this protects blockwise transfers against a very
  small range of attacks (those where the attacker can't modify the
  message, but delay it), this can be seen as a security mode and then
  plug into the extension point described in RFC7252 4.1.

Security Considerations
=======================

The Request Discriminator is limited in its used between a pair of end
points. If used conservatively (ie. only when necessary, and with
as-small-as-possible random discriminators), it only indicates that (and
roughly how many) operations that require distinct endpoints are in
simultaneous use or have previously been unsuccessful.

When used to preclude out-of-order situations between endpoints (as for
example in OSCOAP), it is essential that implementations store the
usable state of a discriminator for as long as required (eg. in parallel
with sequence numbers). Failure to do so leads to reuse of a
discriminator, and thus opens up the possibility of replays.

Servers that rely on consistent states set by clients must be aware that
the out-of-order guarantees added by this mechanism only cover
operations that are required by RFC7959 to originate from the same
endpoint (or security association). Block1 requests should therefore be
limited to atomic operations as outlined in that document's security
considerations.

IANA Considerations
===================

@@@ have a number assigned and the option known

References
==========

CoAP

@@@ i think i can get away with only having coap as a normative
reference here (plus 2119 if keywords are used), though blockwise might
be required too due to it updating coap.

Informative References
----------------------

OSCOAP
blockwise
coap-over-serial if referenced

-- 
Christian Amsüss                      | Energy Harvesting Solutions GmbH
founder, system architect             | headquarter:
mailto:[email-masked]  | Arbeitergasse 15, A-4400 Steyr
tel:+43-664-97-90-6-39                | http://www.energyharvesting.at/
                                      | ATU68476614

---

From: Göran Selander <[email-masked]>
Date: Sat, 4 Mar 2017 23:45:21 +0000

Thanks for compiling this.

I think there is sufficient material here for a draft.

If I understand right, you propose an augmentation to the endpoint
identifier. I was thinking of the solution as a  "body-identifier” (using
RFC7959’s distinction between "body" and “payload”), not limited to server
response as ETag but also applicable to client request (and proxy
request/response).

What would be the cons of instead identifying the body?

Thanks
Göran

---

From: Jim Schaad <[email-masked]>
Date: Sun, 5 Mar 2017 21:47:03 -0800

Christian,

I am sorry, but I do not believe that this is the correct approach to take.

1.  I want a value that can be modified in transit.  For that reason, it
should not be authenticated.  The ability for a proxy to change means that
it always has the ability to do the disambiguation if two different clients
come in apparently asking for the same thing, but with a different and
masked disambiguation.

2.  It is my belief that the cue to use the new option is not correct at
all.  The new option is not needed for any OSCOAP internal blocking.  It is
needed solely to deal with outer blocking.  This means that it is more
likely to be needed if an inner Block1 is absent than if it is present.  The
assumption if it is present is that it SHOULD be sufficient to deal with the
problem.  

As I read the problem that you have outlined below, it does not appear to
match the same problem that I think needs to be solved.  I don't know if
your problem is real or not, I don't have a sufficient understanding of your
description and should go look at the mail message referenced to see if that
helps, but I have been having a problem getting to things this weekend.

In regarding to states of drafts to publish.  I have done everything from a
sketch of a draft to a fully written draft as a first document published.
Don't let that be a barrier to publication.  The only question is do you
think you have enough to give a brief overview of the problem.  Often times
it is easier to see ASCII art in drafts since they are a fixed width font.

Jim

> -----Original Message-----
> From: core [mailto:[email-masked]] On Behalf Of Christian Amsüss
> Sent: Friday, March 3, 2017 9:32 AM
> To: [email-masked]
> Subject: [core] pre-draft: Request-Tag
> 
of
> blockwise requests being linked together only via the endpoint address[1].
> 
> appreciate feedback on it, especially on the approach of packing this into
the
> endpoint definition instead of just defining a blockwise extension. Note
that
> the actual definition of the change is more concise now than in my
previous
> sketch, because many things I spellt out ther explicitly fall into place
> automatically because of the endpoint discrimination. (For example, the
> "atomic vs non-atomic" considerations are now merely a reference to what
> blockwise already noted in its security context).
> 
> Best regards
> Christian
> 
upload
> into the data tracker?
concurrent
> requests that would otherwise be serialized by proxies in presence of
> blockwise transfer, and linking blockwise requests on transports that do
> provide authentication but no ordering guarantees.
> 
> Introduction
> ============
> 
> Blockwise transfer [RFC7959] in the presence of Block1 ties an exchange to
a
> combination of resource and remote endpoint, keeping any two exchanges
> where those parameters match from happening simultaneously.
acts
> as a single endpoint towards a server when forwarding operations on the
> same resource, or in Object Security of CoAP (OSCOAP) where all a client's
> request are targetted at the same resource from an outside point of view),
> the client needs to queue up requests to keep the transfers from
overriding
> each other.
protection,
> but with blockwise, the security context (which augments the endpoint
> address) alone is insufficient protection because it does not (and does
not
> indent to) forbid out-of-order delivery.
> 
additional
> discriminator (the Request-Disciminator option), which must be treated as
a
> part of the (security context of an) endpoint as defined in RFC7252
section
> 1.2.
the
> tests
> It is critical (because mechanisms may rely on endpoint identities not to
be
> conflated), unsafe (because the channels implied by endpoint identities
> terminate at proxies), not repeatable. (@@@ unsafe makes nocachekey
> irrelevant, but should we or not set it to ease proxy
> implementations?)
> 
> message's endpoint that does not have the identical Request-Discriminator.
> The absence of the option and a zero-length option are distinct. (@@@ i
> wouldn't insist on that -- especially with the inclusion in the AAD it
might be
> better to have absence and h""
their
> corresponding (via the token @@@ or better refer to Exchange
Option,
> the client SHOULD retry the request without it, but only if and when
> 
> Whenever the Block1 option is used as inner option, it is associated with
a
> Request Discriminator. The same Request Discriminator MUST NOT be
elicted a
> protected response (and its sequence number is consequently marked
> used). (@@@ if we're cautious, we could s/unless.*/at all/, but i don't
see
> any harm in allowing it.) OSCOAP requires that out-of-order delivery of
> blockwise transfers is caught by the Request-Discriminator option, so a
client
> MUST NOT fall back to not using the Request-Discriminator option if it
> encounters a 4.02 Bad Option response.
> 
many
> cases. An application implementing OSCOAP MUST use that option at least
> when a Block1 request has not been losslessly concluded[*], and a server
> MUST NOT respond with 4.02 due to the presence of a Request-
> Discriminator option. (@@@ or they could declare the complete context
> invalid, but i don't think we want contexts to be that volatile)
> 
> there is still CoAP retransmission in place. Only when the block transfer
is
> aborted, or when the same block gets sent with a different sequence
> number (@@@ may that be at all?), the discriminator is unusable for any
> further blockwise transfer.
> 
send
> response messages with matching ETags if they can be expected to assemble
> into a consistent representation again. (@@@ imo that's a requirement that
> there already is from the ETags themselves. jim, do you think this is a
starting
> point for your concerns about validating the whole message?)
> 
> 
> The associated Request Discriminator will need to be included in the AAD.
> Whether we include that in the request too to keep the AAD the same both
> for request and response, or have it in responses only, I don't care.
> 
> The Request-Discriminator option is added to the "E*" category and like
and
> listed together with Block1/2.
> 
> 
> Notes on applications
> =====================
> 
> this can't happen in oscoap now any more because the incomplete first POST
> permanently poisons the first request's request discriminator, and the
> second request will have a different one, for which the server will not
accept
> the delayed "valjean" any more.
> 
> Rationale
> =========
> 
necessary,
> and how it is different from similar concepts.
> 
> Why not use...
> 
use to
> the application (describing which requests of a blockwise bunch belong
> together) deviates from what it by its definition (and in
the
> channel described by the endpoint address.
> 
'additional
> considerations' -- depending on how much discussion there is on it.
> 
points.
> If used conservatively (ie. only when necessary, and with as-small-as-
> possible random discriminators), it only indicates that (and roughly how
> many) operations that require distinct endpoints are in simultaneous use
or
> have previously been unsuccessful.
> 
> example in OSCOAP), it is essential that implementations store the usable
> state of a discriminator for as long as required (eg. in parallel with
sequence
> numbers). Failure to do so leads to reuse of a discriminator, and thus
opens
> up the possibility of replays.
> 
the
> out-of-order guarantees added by this mechanism only cover operations
> that are required by RFC7959 to originate from the same endpoint (or
> security association). Block1 requests should therefore be limited to
atomic
> operations as outlined in that document's security considerations.
> 
> IANA Considerations
> ===================
> 
> @@@ i think i can get away with only having coap as a normative reference
> here (plus 2119 if keywords are used), though blockwise might be required
> too due to it updating coap.
> 
> Informative References
> ----------------------
> 
> OSCOAP
> blockwise

---

From: 'Christian Amsüss' <[email-masked]>
Date: Mon, 6 Mar 2017 10:01:08 +0100

Hello Jim,

> match the same problem that I think needs to be solved.

from the points you've brought up it appears to me that I failed to keep
the two uses I see for the described option:

> 1.  I want a value that can be modified in transit.

This is what the Request-Discriminator option would be used for in outer
messages. It's unencrypted, unauthenticated, and its scope is between
endpoints on CoAP level, so typically between a proxy and a server.

That option must not be reflected in the AAD.

> assumption if it is present is that it SHOULD be sufficient to deal with the

The OSCOAP application of this option is with what I'd hope to address
the issues of inner-blockwise transfers falling victim to out-of-order
attacks, or what you described in earlier mails as protecting the
integrity of the complete message (or as I see it the body).

The draft-ietf-core-object-security-01 draft had the tag-previous-block
fields in the external AAD. That has been removed in the latest editors
copy (git b14b3c7d9bd), so we are again missing protection against block
2 of a request being blackholed by a MITM (with the whole block transfer
then failing) and replayed as block 2 of a different request.

This is where the Request-Discriminator option comes into play as an
option in the inner message (together with inner block options).

> Often times it is easier to see ASCII art in drafts since they are a
> fixed width font.

Good point, didn't think of non-fixedwidth mail clients.

Thanks
Christian

-- 
You don't use science to show that you're right, you use science to become right.
  -- Randall Munroe

---

From: Christian Amsüss <[email-masked]>
Date: Mon, 6 Mar 2017 13:01:29 +0100

Hello Göran,

> What would be the cons of instead identifying the body?

I'm aiming for the endpoint identifier because

* it works better with implementations (libraries) where the application
  (resource handler) can opt in to do block handling itself (which is
  especially relevant in all scenarios where the application can't give
  its underlying library a memmapped view of the body.

  In those cases, if we did touched the blockwise transfer, the resource
  implementation would need to become aware that it needs to do
  additional work for the underlying OSCOAP layer, while with this, the
  OSCOAP layer would merely need to detect that there is a Block1 option
  present, and add an appropriate Request-Tag.

* I think it can be expressed more compactly this way. The existing (and
  probably necessary) complexity of block handling is not involved.

* It may (or, as always with this phrase, may not) help differentiate
  between requests that should be linked together (like the blockwise
  requests do) in other CoAP extensions. I did not find any when quickly
  thinking through the current specs, because Observe responses can all
  be linked to their single request.

But basically, yes, I could reshape the draft in a way that it becomes a
pure Block1-Tag. It would achieve the same thing and could (just like
the Request-Discriminator) draft be applicable both to CoAP messages
(and solve the outer "all requests hit /" problem, as well as help
proxies) and to OSCOAP messages (where it would solve the "promote
valjean" replay-of-blackholed-request problem).

If there develops an informed mood of "it's better as an additional
block option", I'll happily write up such a version.

Best regards
Christian

-- 
You don't use science to show that you're right, you use science to
become right.
  -- Randall Munroe

---

From: Christian Amsüss <[email-masked]>
Date: Mon, 6 Mar 2017 16:06:22 +0100

Hello core list,
and especially hello people versed in CoAP over DTLS,

based on on-list and personal / relayed feedback, I'd like to add the
following clarification, and a question:

* Clarify expected uses:

  This draft will predominantly be useful in OSCOAP's outer messages,
  where all requests are on first sight targeted at the `/` resource, or
  in OSCOAP's inner messages.

  Without OSCOAP, this is probably a corner case (for who else has
  several PUT/POST requests transmitted over a proxy delivered to the
  same resource simultaneously, and expects a non-4.08 outcome?), that
  by conforming proxies is solved by queueing up requests for sequential
  processing.

  Nevertheless it is a concern to be discussed with the CoAP community
  as a whole because this corner case gets triggered by OSCOAP in
  proxies that "we" will use even though they do not implement object
  security and can keep proxies blocked by a single client.

* Alternative approaches still under consideration:

  - Introducing a tag that is common to all parts of a single body would
    be a viable alternative, and largely equivalent to the proposal,
    just different to implement in text and code.

  - It might be worth considering to plainly demand of non-initial
    blocks that if they are to belong to a first block, they need to
    have larger sequence numbers if the security layer used provides
    any. That'd be an update (maybe even Erratum) for RFC7959, and would
    solve the OSCOAP inner-blockwise issues, and leave the proxy+block1
    issue to another (maybe even slimmer) solution.

* CoAP over DTLS question: How does CoAP over DTLS deal with delayed
  PUTs? As I see it (but not having implemented CoAP over DTLS myself
  yet), the same attack as described on OSCOAP can be carried out on
  DTLS too, as long as it happens inside the replay window. Worse yet,
  as DTLS sequence numbers vary between retransmits of the same message
  ID (and thus block request), the application might not even notice
  that a package was "blackholed".

  (To spare you the lookup, the attack sequence is:

  Client POSTs "incarcerate" (block 0, more to come).
  Server returns 2.31 Continue.
  Client POSTs "valjean" (block 1, end of request), but
  MITM blackholes the request (and subsequent retransmits) for further use.
  Later, Client POSTs "promote" (block 0, more to come).
  Server returns 2.31 Continue.
  MITM injects the previously stored "valjean" package.
  Server duefully promotes Valjean.

  )

Best regards
Christian

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Olaf Bergmann <[email-masked]>
Date: Fri, 10 Mar 2017 11:02:23 +0100

Hi Christian,

(I have not been following this discussion, so please be patient if I am
opening a can or worms here that has alreading been discussed in
length...)

>   that a package was "blackholed".

I guess by "blackholed" you mean an artifically created 100 % packet
loss? (If not you can stop reading here.)

>   Server duefully promotes Valjean.

I do not see any problem here where securing CoAP as a transfer protocol
could be of any use. And DTLS doesn't. To understand why this is not a
problem, take a step back and look what happens in your scenario:

1. Client sends a request to change the Server state to X.
   The server's response tells Client that the server is now in state X.
2. Client sends a request to change the Server state to Y.
   Client never gets a response and thus does not know whether Server
   is in state Y (or X' because something else has happened since step 1.)
3. At some time, Client stops waiting for a response and discards its
   transaction state.
4. Even later, Client sends a request to change the Server state to Z.
   The server's response tells the Client that the server is now in
   state Z.
5. The server receives a request to change its state to Y.

In absence of an attacker, the very same scenario would occur if, for
some reason, the request in 2 would have taken a very long time and thus
would have been delivered out of order.

Both, Client and Server must be aware that this might happen. A robust
application layer protocol thus would have to be designed in a way that
you could detect (and possibly resolve) this situation. CoAP as a
REST-based transfer protocol does not help here. (What you are
suggesting is to invent a protocol element with application layer
semantics that helps.)

And by the way: Proper authorization could help mitigating the effects
of this scenario/attack. That is the reason why the DCAF proposal had
made extensive use of timestamps.

Grüße
Olaf

---

From: Christian Amsüss <[email-masked]>
Date: Fri, 10 Mar 2017 11:36:44 +0100

Hello Olaf,

thanks for your input.

> loss? (If not you can stop reading here.)

Stored for further use, and then artificially lost along with all
retransmission attempts.

> 1. Client sends a request to change the Server state to X.

This would be acceptable if we were talking about a PUT (especially when
processing is not atomic, ie. does not create a 2.13 responses).

In a POST request with atomic processing, the "but there is a valid
request to put part 2 the resource in state Y" doesn't hold.

> would have been delivered out of order.

Yes, it would; but it would need to lose all retransmission attempts
that either have the same sequence number (in the OSCOAP case) or the
same message ID (in the DTLS or unsecured case).

> suggesting is to invent a protocol element with application layer

CoAP with blockwise advertises the possibility to process
larger-than-MTU payloads (ie. bodies) as a whole. At least in OSCOAP,
there is the expectation to obtain a blockwise'd body in a way that it
is cryptographically secured to originate from the given peer.

Best regards
Christian

---

From: Olaf Bergmann <[email-masked]>
Date: Fri, 10 Mar 2017 16:54:20 +0100

Hi Christian,

> request to put part 2 the resource in state Y" doesn't hold.

No, you are putting to much application semantics into this
reasoning. First of all, the initial request (PUT or POST) is a REST
operation that changes state in the server. There is no "but we want do
something else in the future" on that layer. 

> is cryptographically secured to originate from the given peer.

Which it does as you have described.  My personal opinion is that the
freshness issue in your scenario is not something that must be solved in
the transfer protocol.

Grüße
Olaf

---

From: Christian Amsüss <[email-masked]>
Date: Sun, 12 Mar 2017 14:04:58 +0100

Hello OSCOAP people,

>   PUTs?

I've received word back on and off list from DTLS implementors (Olaf and
Matthias), and their take on this sounds to me rather that blocks being
misassignable between blockwise requests is more of an application issue
than something blockwise and crypto should protect us from.

On the other hand, the last published draft version did include
preparations for chaining inner-blockwise requests into a
cryptographically secured total.

Before we push further the "how", is there agreement among the OSCOAP
authors that we want a way to protect the body of a blockwise request,
even in the light of Olaf's comments?

(My personal take on this is that there are protection-worthy REST
transactions atop the blockwise layer, and that blockwise mode itself
knowingly sacrifices complete RESTfulness in favor of slimness (that's
how the "same endpoint" language gets in there, but it avoids additional
round trips and allocations for creation of a per-transaction resource).
Thus, I think that it'd be within the scope of the OSCOAP layer to link
those messages cryptographically -- but then again, I'm not too firm
with the theoretical background of REST.)

Best regards
Christian

-- 
I shouldn't have written all those tank programs.
  -- Kevin Flynn

---

From: Göran Selander <[email-masked]>
Date: Sun, 12 Mar 2017 14:38:29 +0000

Hi Christian,

I’m in favour of specifying how to disambiguate multiple concurrent block
fragmentations, and your proposal seems efficient from the point of view
of fitting into existing implementations.

So I think you should submit that, and let us have a discussion where this
feature may also be useful (outside OSCOAP, where it is definitely
useful). Even if the issue is up to the application to handle, given that
this feature is specified it is easy for applications to just use without
having to re-invent it.

Göran

---

From: Jim Schaad <[email-masked]>
Date: Mon, 13 Mar 2017 00:19:35 -0700

1.  I still don't understand what exactly your problem is, but I think that
if one expects things to be changed together, then that would be an
application problem rather than a CoAP problem.  I think that I agree with
Olaf and Matthias on that point.

2.  There is a different endpoint problem which needs to be solved for
OSCOAP because of the removal of all URI-Path elements from the query which
means that endpoints collapse in a way that they do not with DTLS only
solutions.

3. I think that there needs to be a solution for dealing with ensuring that
the correct content has been acquired after doing a series of bitwise
transactions.  This could be done either at the CoAP layer or at the
application layer.  Both are probably equally doable.  The only thing that
might needs to be done in CoAP is the signaling of how to get the secure
content checking structure.

4.  My understanding of REST is that the idea of dividing a content across
multiple messages would not occur to them.  One can obtain the entire HTTP
content over a TLS/DTLS connection in a "single" transaction and thus the
entire message content would be adequately protected.  The fact that CoAP
breaks up the entire message content into multiple messages means that this
feature has been lost.  I think that is probably too bad.

> From: Christian Amsüss [mailto:[email-masked]]
> Sent: Sunday, March 12, 2017 6:05 AM
> To: Göran Selander <[email-masked]>; Jim Schaad
> <[email-masked]>; Francesca Palombini
> <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [core] pre-draft: Request-Tag
> 
> Hello OSCOAP people,
> 
preparations
> for chaining inner-blockwise requests into a cryptographically secured
total.
> authors that we want a way to protect the body of a blockwise request,
even
> in the light of Olaf's comments?
> 
> knowingly sacrifices complete RESTfulness in favor of slimness (that's how
> the "same endpoint" language gets in there, but it avoids additional round
> trips and allocations for creation of a per-transaction resource).
those
> messages cryptographically -- but then again, I'm not too firm with the
> theoretical background of REST.)
> 
> Best regards
> Christian
> 
> --

---

From: Christian Amsüss <[email-masked]>
Date: Thu, 16 Mar 2017 10:22:29 +0100

Hello Jim,

> means that endpoints collapse in a way that they do not with DTLS only

For the purpose of setting the goals on what we protect in blockwise,
I'd set aside the issue of outer blockwise requests hitting the same
endpoint. My original proposal for a solution happens to cover that too,
but first I'd know what we're trying so solve.

> Olaf and Matthias on that point.

My problem (or actually, our problem, as this developed from the
protection of the complete message in a blockwise transfer you asked
about) is that when a user sends inner-blockwise requests, neither can
the server nor the client be sure that they belong to the same request.

If that's something where we decide not to offer protection, that's fine
with me (but then blockwise needs stronger cautionary wording in
OSCOAP), but I opine that we can provide that. (The draft I've proposed
might easily not even be the best way to go about that; first, we should
set a goal).

> multiple messages would not occur to them. [...]

Protection of full messags on application level is completely doable
without protected blockwise. (Göran suggested checksum resources, it'd
just depend on what exactly the application is).

Then again, so would have been blockwising in the first place; we could
have (instead of the blockwise we have now) defined a more
application-level, more RESTful blockwise [2], but what we have is a
little less RESTful (by referring to "further blocks from the same
endpoint"), but much slimmer and more efficient. I think we can protect
these slim blocks without making things too complicated.

As for me not having made my problem statement fully clear, please allow
me to try again, and to serve an enhanced example:

Problem statement
=================

An application can not gain the same security it has about the integrity
of its messages and the correlation between request and response that it
gets from OSCOAP in non-blocked or outer-blockwise mode in
inner-blockwise mode.

In particular, the integrity of the body of a Block1 request is not
guaranteed.

Example
=======

My previous example[1] only showed how the server can be tricked into
processing a wrongly assembled request and act upon it; this new example
creates a request/response mismatch and gives the client a wrong answer:

Client: POST /judge, payload "Homeless stole apples. Wh"...
Evil proxy intercepts and stores the request.
Client times out.

Client: POST /judge, payload "Hitman killed someone. Wh"...
Evil proxy passes on the request
Server: 2.31 continue
(Note that were this request to pass, it would respond "Hang him.")
Evil proxy, before passing on the response, inserts:
  POST /judge, payload "Homeless stole apples. Wh"...
Server: 2.31 continue (by the rules of blockwise, the previous
  transaction is aborted)
Evil proxy swallows the new response, and sends back the first 2.31.

Evil proxy now innocently passes on what ensues:

Client: POST /judge, payload ..."at shall we do with him?"
Server: 2.05 "Set him free."
Client sets the hitman free.

Best regards, and see you later
Christian

[1]: http://www.ietf.org/mail-archive/web/core/current/msg08412.html
[2]: along the lines of POSTing to a /block resource, retrieving a
     transaction Location, PUTting the request blocks in there, GETting
     response blocks out of it and DELETEing it in the end)

-- 
The detailed semantics of CoAP methods are "almost, but not entirely
unlike" [HHGTTG] those of HTTP methods.
[HHGTTG]: Adams, D., "The Hitchhiker's Guide to the Galaxy", October 1979.
  -- Shelby, et al., Internet-Draft Constrained Application Protocol (CoAP)

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 16 Mar 2017 12:47:22 +0100

On 13 Mar 2017, at 08:19, Jim Schaad <[email-masked]> wrote:
> multiple messages would not occur to them.  

It is orthogonal to REST.

> One can obtain the entire HTTP
> entire message content would be adequately protected.  

But that’s not how it works.  The message is broken up into packets and then reassembled on the receiver by TCP; independent of this, swathes of content are made into TLS records, each of which is properly protected by the transport layer security.

All this cannot be ignored when it is necessary to discuss the security properties of content while it is transferred.  For instance, TCP connections can easily broken by an attacker, so the receiver of partial content (e.g., for streaming) needs to be able to resume the transfer if that attack isn’t supposed to be a fatal threat.

Please also be aware of application layer protocols such as HTTP streaming that do the equivalent of block-wise on top of REST, for many of the same reasons.

> The fact that CoAP

It never was a feature, just a (rather leaky) abstraction that allowed us to ignore more if the inner workings.

The difference between CoAP and HTTP is that HTTP creates state in the form of the TCP connection (and the TLS connection bound to that), so it can bind all of the messages that build one body together (sequence preservation also helps with that).  We have a weak form of that binding in CoAP for Block2 (ETag), but not for Block1, where we assumed only having one transaction in flight for each resource would be good enough.   Now that all the secure CoAP transactions of one endpoint are funneled through what looks like one resource at the surface, we may have to go beyond that.

Grüße, Carsten

---

From: Christian Amsüss <[email-masked]>
Date: Thu, 16 Mar 2017 16:05:00 +0100

Hello working group,

to sum up things after a pleasent call with Jim, Francesca and Göran
(thanks to all involved): I'll update the draft with an enhanced problem
statement and discussion of the various block scenarios as Request-Tag
at [1], moving torwards the "tag the request body" rather than the
"discriminate the connection" terminology. Consideration will be given
to not increasing message sizes, but later when the individual block
transfer scenarios are understood better.

Discussion on details will move to the draft's issue tracker, while
we'll come back to the list when in need for guidance on RESTfullness
and usability for other purposes.

Best regards
Christian

[1]: https://github.com/chrysn/request-tag