Subject: [core] OSCOAP's inner-blockwise

From: Christian Amsüss <[email-masked]>
Date: Tue, 17 Jan 2017 13:42:03 +0100

Hello OSCOAP authors and implementors,

I'm taking a discussion from [oscoap:#49] to the list because it exceeds
the topic of the issue, and maybe could use feedback from poeple not
directly involved with the detail of the issue.

I think that the protection currently drafted for the inner blockwise
option (that is, using the MAC of the previous exchanged block as part
of the AAD in securing the next block, thus creating a chain of
authenticated messages), is rather intrusive: It requires retaining
additional state, and requires the message protection part to interact
with the blockwise transfer part). What I am advocating is a slimmer
course of action, like determining resource consistency using an ETag,
to achieve the same protection. (If an ETag is insufficient, maybe a
Secure-ETag that's required to change under these or those conditions
is.)

I suggest thinking about the block2 case first, as if it turns out that
tag chaining is absolutely required there, things won't be any easier
for the block1 case.

Jim Schaad wrote in [oscoap:#49]:
> the reason for using the crypto tag is to have the ability to
> establish that the entire message has been integrity protected rather
> than just the independent pieces

But does chaining the tags actually achieve that? All they establish is
that both client and server agree that there was a continuous exchange
of messages that led to the last block transferred. A client could have
skipped a block during the transfer (eg. because it already has the
block), finish a securely chained transfer, and then would then still
work with a partially secured resource.[1]

We would need to require block transfers to be strictly iterating / not
use cached ETags, but IMO that would lose so much of blockwise that it
feels more like its own fragmentation scheme.

Best regards
Christian

[oscoap:#49]: https://github.com/core-wg/oscoap/issues/49

[1]: Following the letter of the draft, the resource could even change
during the transfer (provided no ETags are used), and the server could
sign off the stream. That can be fixed easily in the draft, but requires
implementations to transfer additional information between components.

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Jim Schaad <[email-masked]>
Date: Wed, 18 Jan 2017 00:00:14 -0800

Going back to first principles.  My goals are:

1.  The ability to prevent eavesdroppers from knowing what is being
transmitted.
2.  The ability to have end-to-end integrity on the entirety of the content
being sent.
3.  The ability to do something about making traffic analysis harder.

The first can be address with encryption.
The second can be addressed with encryption as long as blocking does not
rear its ugly head.
The last I am ignoring at present.

Doing the integrity protection on large items is simple if one only does
outer blocking of the data.  Simply encrypt the entire message and then do
the necessary blocking of the message.  This breaks down if one wants to do
random access of the large block of data.  Göran has some additional
problems with this approach as if a single block is corrupted, but passes
the checksum, then the entire set of block transfers needs to be done again
as it is impossible to know which of them was corrupted.  The introduction
of inner blocking allowed for a simple way to address this issue, but with
the loss of the end-to-end integrity on the entirety of the message.

At the time, I was not aware of the abilities to do random access of
blocking.  It is difficult to understand a couple of things about what this
means without knowing what the expected use cases of random access is
supposed to be doing.

I am not sure at the moment that I have any understanding of the expected
scenarios where random access would be used.  The ones that I can think of
are:

1.  I receive a message that provides a set of instructions with what has
changed and I can get incremental updates. 
2.  It is something that is large, but I might only care about pieces of the
data rather than the whole thing - such as a log file.

In the first case, the integrity is probably better covered by the contents
of the set of instructions.  In the second case I only care about the
integrity of what has been transferred rather than the entire content.

What other scenarios are there that people can think of where random access
is useful and what type of integrity checking of the data would be
required/needed?

Jim

---

From: Göran Selander <[email-masked]>
Date: Mon, 23 Jan 2017 13:26:40 +0000

hi Christian,

Thanks for bringing this to the list.

We have been re-iterating the argumentation for having a binding between
the blocks.  It was introduced to protect against interchange of blocks
from different representations of the same resource exchanged between the
same client and server. On second thought such a protection a) should not
be mandatory to as this is not prohibited RFC7959, and b) can instead be
achieved using ETag, as you propose, which is protected by OSCOAP and thus
can be used to verify that blocks belong to the same representation.

With this in mind, I don’t see a reason to bind the integrity of the
different blocks to each other and we should remove the tag-previous-block
from the external_aad.

Unless there is an objection, it will be removed in the next version.

Göran

---

From: Jim Schaad <[email-masked]>
Date: Mon, 23 Jan 2017 15:24:44 -0800

I do not believe that this would make me happy.  I do not believe that it would be possible to get an integrity check over the entire content in this manner.  It is possible that the creating of some type of Secure ETag - either at the OSCOAP or CoAP layer might do what I would be looking for.

Jim

> -----Original Message-----
> From: Göran Selander [mailto:[email-masked]]
> Sent: 23 January 2017 05:27
> To: Christian Amsüss <[email-masked]>; draft-ietf-core-
> [email-masked]; [email-masked]; Jim Schaad
> <[email-masked]>; Francesca Palombini
> <[email-masked]>
> Subject: Re: OSCOAP's inner-blockwise
> 
> hi Christian,
> 
> the blocks.  It was introduced to protect against interchange of blocks from
> different representations of the same resource exchanged between the
> With this in mind, I don’t see a reason to bind the integrity of the different
> blocks to each other and we should remove the tag-previous-block from the
> external_aad.
> 
> >I'm taking a discussion from [oscoap:#49] to the list because it
> >exceeds the topic of the issue, and maybe could use feedback from
> >poeple not directly involved with the detail of the issue.
> >
> >
> >sign off the stream. That can be fixed easily in the draft, but
> >requires implementations to transfer additional information between
> components.
> >
> >--

---

From: Göran Selander <[email-masked]>
Date: Tue, 24 Jan 2017 09:59:15 +0000

Hi Jim,

I’m not sure I understand the issue. If you verify all blocks of a
particular representation (using ETAG or otherwise knowing that the
representation´is the same for different blocks) then you have also
verified the entire representation, since the block options are integrity
protected and contain information about the number of the individual block
and if it is the last block.

If you want a separate integrity verification of the entire representation
it is possible to define a new resource containing the hash of a
representation:

/hashof/r1 for resource /r1

Requesting this resource with GET

GET /hashof/r1

returns the the SHA-256 of the representation of /r1. Specific
representations can be requested, e.g. with ETAG in the Uri-Query:

GET /hashof/r1?r1-ETAG
where r1-ETAG is the ETAG of the representation of r1 for which the hash
is requested.

(Of course, since this this resource contains the plain hash, a security
protocol such as DTLS or OSCOAP must be used when accessing the hash in
order to verify the integrity of the representation.)

Would anything like this address your concern?

Göran

On 2017-01-24 00:24, "Jim Schaad" <[email-masked]> wrote:

>I do not believe that this would make me happy.  I do not believe that it
>would be possible to get an integrity check over the entire content in
>this manner.  It is possible that the creating of some type of Secure
>ETag - either at the OSCOAP or CoAP layer might do what I would be
>looking for.
>
>Jim
>
>
>>from
>>different

---

From: chrysn <[email-masked]>
Date: Tue, 24 Jan 2017 11:10:05 +0100

Hello Jim,

On Mon, Jan 23, 2017 at 03:24:44PM -0800, Jim Schaad wrote:
> I do not believe that this would make me happy.  I do not believe that
> it would be possible to get an integrity check over the entire content
> in this manner.  It is possible that the creating of some type of
> Secure ETag - either at the OSCOAP or CoAP layer might do what I would
> be looking for.

well, let's work that out; I'm confident we can come up with something
that gives both the flexibility of inner blockwise and a way to trust
the assembled message.

(Again, I'll focus on the block2 use here, because that's the one where
constrained servers need to deal with the biggest increase of complexity
due to current OSCOAP. Later we can see how to apply those findings to
block1. If you want to bring up block1 now for whatever reasons, feel
free to -- this is just so nobody expect everything below to hold for
block1 too without adaptions).

On Wed, Jan 18, 2017 at 12:00:14AM -0800, Jim Schaad wrote:
> [... Outer-blockwise is safe but inflexible/impractical, and ...]
> The introduction of inner blocking allowed for a simple way to address
> this issue, but with the loss of the end-to-end integrity on the
> entirety of the message.

What I think makes this topic difficult is that none of the documents
involved really describe the composite message. RFC7959 ("blockwise")
does describe the relationship between the payload (the individual
message's contents) and the body (the concatenation of the payloads).

The aiocoap and txThings implementations (and possibly others) do have a
concept of "the whole message", because it is convenient for a user to
call something like `request(code=GET, path="/firmware", accept=42)` and
get back a `Message(code=2.05, etag=..., payload=some-64k-blob)` object
-- but in my understanding of CoAP, this whole message deliberately does
not exist, not only due to resource constraints, but also for
architectural reasons (state transfer vs. message passing).

We still want to be be sure of the integrity of the composite message,
though.

As we won't get protection of such a composite message, we'll need to
verify the that it was composed of matching parts. The -01 draft
provides a way to describe a sequence of matching parts (the
tag-previous-block), but does not describe criteria by which to decide
what matches. (For example, may a server sign off a 4th-block response
as the successor of a 2nd-block response?).

To avoid writing tons of irrelevant details: The case you're worried
about, is it about mismatching payloads, mismatching options, or both?

> data rather than the whole thing - such as a log file.

I can't relate to 1, but 2 is something I do regularly, and occasionally
even writing.

There is one other I think of as relevant:

3.  A large resource changes mid-blockwise:

    GET /radar block=1 -- CONTENT, ETag=0xbeef
    GET /radar block=2 -- CONTENT, ETag=0xbeef
    GET /radar block=3 -- CONTENT, ETag=0xcafe

    (oups, we'll need to rewind)

    GET /radar block=1 -- CONTENT, ETag=0xcafe
    GET /radar block=2 -- CONTENT, ETag=0xcafe
    (skipping 3, because 0xcafe is cached)
    GET /radar block=4 -- CONTENT, ETag=0xcafe, last block

Best regards
Christian

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 24 Jan 2017 12:18:50 +0100

Hello Göran,

> different blocks to each other and we should remove the tag-previous-block

Pending Jim's objections, I'd tentatively move forward to the Block1
case; dropping tag-previous-block from external_aad_req would again make
us open to

        Client              Foe         Server

POST "incarcerate"(1/2) --->  --->
                       <---  <---   2.31 CONTINUE
POST "valjean"(2/2)     --->@
                       <---RST

(Client: Odd, but let's go on and promote Javert)

POST "promote"(1/2)    --->  --->
                      <---  <---    2.31 CONTINUE
                           @ --->
                            <---    2.04 Valjean Promoted!

(The @ indicates a maliciously delayed / wormholed package as used in
draft-mattsson-core-coap-actuators-02).

To mitigate that without linking messages again, I propose the following
addition:

---------

(in section 4.3 "CoAP options", figure 4)

Option Request-Tag (details as below) is E:* as are the block options.

(in section 4.3.1.3 "The Block Options")

The Block options (Block1, Block2, Size1 and Size2 *as well as
Request-Tag*) MAY be either...

(new section 6.6) Integrity protection for `Block1` requests

When a client uses inner block options to split up the payload of a
request (that is, whenever the Block1 option is used as a class E
option), the client MUST add a Request-Tag option with the same value to
all the blocks of a single request, and the server MUST verify that the
Request-Tag values match before acting atomically on several requests.

[Author's note: I'm only limiting the server w/rt atomic requests, as
the different messages from a non-atomic operation don't "meet" in the
server before taking effect on the resource. To mitigate what those
would do in a delayed-message scenario like
draft-mattsson-core-coap-actuators-02 figure 4, that could be covered
by If-Match and fluctuating ETags, but not on blockwise layer.]

The Request-Tag value must be unique per participant within the security
context. The client may, for example, use the partial IV of the first
block it sends as the request's tag.

The role of the Request-Tag is similar to the ETag of responses. Like
the ETag, it allows the recipient of blocks to verify that those blocks
belong to the same state of a resource. Unlike the ETag, they can not be
used for caching, because there is no single authority to assign them to
a body. (Two clients would not arrive at the same Request-Tag for the
same body, while the origin server is the authority on ETags for a given
resource).

The Request-Tag option can be used outside of OSCOAP protected messages
to separate Block1 requests from a single endpoint to the same resource.
As using it would caus mixups of blocks on a server that does not
understand it, it is marked as Critical, safe-to-forward, and part of
the cache key. Proxies that understand the Block1 and Request-Tag
options MAY give it special treatment and use the full body of the
request as the cache key instead, as they already may with Block
options.

       +-----+---+---+---+---+-------------+--------+--------+---------+
       | No. | C | U | N | R | Name        | Format | Length | Default |
       +-----+---+---+---+---+-------------+--------+--------+---------+
       | TBD | x |   |   |   | Request-Tag | opaque |    0-8 | (none)  |
       +-----+---+---+---+---+-------------+--------+--------+---------+

[Author's note on length: Allowing length 0 so people can use the
partial IV unmodified as option value. That's still a
one-time-per-context situation.]

(for 7 "Security Considerations", add)

Applications that accept Block1 transfers in a non-atomic fashion need
to be aware that OSCOAP does not provide sequence guarantees for the
individual blocks, and that updates may arrive maliciously delayed
within the window size. When this is a concern, the application might
want to switch to atomic mode, where the integrity of the whole request
body is provided.

(for 9.1 "CoAP Option Numbers Registry")

Number: TDB, Name: Request-Tag, Reference: this-document

---------

If you want to accept this into the draft, I'd be happy to phrase it as
a pull request.

Does this sound viable to you?

Best regards
Christian

PS. sorry for any confusion caused by my last message on the same
thread, my mail client mixed up the hats, and it was sent from my Free
Software developer hat address <[email-masked]> instead of my work
address.

-- 
Christian Amsüss                      | Energy Harvesting Solutions GmbH
founder, system architect             | headquarter:
mailto:[email-masked]  | Arbeitergasse 15, A-4400 Steyr
tel:+43-664-97-90-6-39                | http://www.energyharvesting.at/
                                      | ATU68476614

---

From: Göran Selander <[email-masked]>
Date: Wed, 25 Jan 2017 08:33:03 +0000

Hi Christian, Jim and all,

I make an attempt to summarise (and simplify) the discussion of
inner-blockwise. Please correct me if I’m wrong or missing something.

In addition to the integrity of each block (which is already covered in
-01) there are two (related) things that must be possible to verify:

1. the integrity of a representation partitioned into blocks
2. that different blocks belong to the same representation

Both these requirements can be addressed by using an identifier of the
representation, unique for this constellation of involved nodes (e.g.
unique within the set of messages protected with given keys).

This identifier must be present and integrity protected in the blocks
transferred, and may be used as reference for verifying integrity of the
whole representation (which also can be verified by verifying all blocks).
With this identifier, the chaining of block integrity can be removed thus
reducing state and simplifying the procedure as described in -01.

For GET etc. ETag (“ETag2") does the job, but for PUT etc. there is a need
for a Request-Tag (“ETag1") option as proposed by Christian below.

Questions: 

- Is such an option something that is of any other use besides protecting
block-transfer?

- Is this a separate draft or something that should to be integrated into
OSCOAP?

Other comments?

Göran

---

From: Jim Schaad <[email-masked]>
Date: Wed, 25 Jan 2017 11:12:08 -0800

Is this something to be done at the COSE or at the CoAP layer?

> Sent: 25 January 2017 00:33
> To: Christian Amsüss <[email-masked]>; Jim Schaad
> <[email-masked]>
> Cc: [email-masked]; [email-masked]; Francesca
> Palombini <[email-masked]>
> Subject: Re: [core] OSCOAP's inner-blockwise
> 
> I make an attempt to summarise (and simplify) the discussion of inner-
> blockwise. Please correct me if I’m wrong or missing something.
> 
> 1. the integrity of a representation partitioned into blocks 2. that different
> blocks belong to the same representation
> 
> 
> For GET etc. ETag (“ETag2") does the job, but for PUT etc. there is a need for
> a Request-Tag (“ETag1") option as proposed by Christian below.
> 
> 
> Questions:
> 
> >>different blocks to each other and we should remove the
> >>tag-previous-block  from the external_aad.
> >
> >case; dropping tag-previous-block from external_aad_req would again
> >make us open to
> >
> >        Client              Foe         Server
> >
> >option), the client MUST add a Request-Tag option with the same value
> >to all the blocks of a single request, and the server MUST verify that
> >the Request-Tag values match before acting atomically on several requests.
> >
> >The Request-Tag value must be unique per participant within the
> >security context. The client may, for example, use the partial IV of
> >the first block it sends as the request's tag.
> >
> >belong to the same state of a resource. Unlike the ETag, they can not
> >be used for caching, because there is no single authority to assign
> >them to a body. (Two clients would not arrive at the same Request-Tag
> >for the same body, while the origin server is the authority on ETags
> >for a given resource).
> >