Subject: [core] draft-ietf-core-block-08 - Block ordering

From: Klaus Hartke <[email-masked]>
Date: Wed, 11 Apr 2012 06:32:05 +0200

Is a client allowed to up/download multiple blocks of the same
representation concurrently? Or does it have to wait for the response
before it up/downloads the next block?

Do blocks have to be up/downloaded with strictly increasing block numbers?

If blocks do not need to be uploaded with strictly increasing block
numbers, should the client set the M in the request with the highest
block number or in the last request it sends?

Use case:
Step 1. The client uploads the first block to check if the server
supports block-wise transfers.
Step 2. The client uploads the last block to check if the server is
willing to accept a representation of this size.
Step 3. The client concurrently uploads all blocks in between, with
the M bit unset in the second-to-last block.

Klaus

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 11 Apr 2012 11:09:35 +0200

> before it up/downloads the next block?

There is nothing in block-08 that would allow out-of-order accesses.
There is also nothing in a stateless implementation that would disallow this.
(A stateless server simply has no idea about whether access are in-order.)
But a client can't rely on a server implementation being stateless.

> Do blocks have to be up/downloaded with strictly increasing block numbers?

It is not so much that the block numbers have to increase, but the blocks need to fit together (see examples).

> block number or in the last request it sends?

(Clear the M bit, you mean.)
That is an interesting question for a future extension we might want to come up with.

> supports block-wise transfers.

Yes.

> willing to accept a representation of this size.

Well, we have the size option for that, now.

> the M bit unset in the second-to-last block.

Makes a lot of sense.
In particular, you could use the shared state between all the block transfers to do better congestion control than lock-step.

So, for me the question is:

1 Should we require servers to support this out-of-order behavior?  No.
2 Should we require servers to actively detect and thwart this behavior?  Certainly not.
3 Should clients expect this behavior to work?  No (since we don't require it from servers).
4 Should clients expect this behavior to have the intended effect if it does work (i.e., elicits 2.xx responses)?  I think so.

1 to 3 just mean the protocol does not make any promises.
4 may possibly require new text, although to me it is kind of obvious that, when a server gets a request that it can't process, that it does not send an affirmative response.

Grüße, Carsten

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 11 Apr 2012 20:27:38 +0200

> There is nothing in block-08 that would allow out-of-order accesses.

Section 2.1 gives detailed instructions on how to construct requests
with Block options. Section 2.2 does not say a lot about what requests
that could theoretically be constructed are allowed or not; I'd assume
that everything that is syntactically possible is allowed unless it's
explicitly forbidden.

There does not seem to be any statement in the draft that forbids
starting an up/download with a block number other than 0, or that
forbids up/downloading blocks out of order.

Klaus

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 11 Apr 2012 23:35:23 +0200

> that everything that is syntactically possible is allowed unless it's

That is probably a fair assumption.
It is also a fair assumption that a server will not implement all conceivable combinations.
(This is what I meant by "allow", sorry for being imprecise here.)
So a client that wants to maximize interoperability will probably operate in sequential fashion.
But I think it is OK for a client to try out of order access; servers will need to make sure they only respond with a success code if that actually is implemented.

Note that there is already some license for out of order access in the text, e.g., 

   If the ETag Options do not match in a GET transfer, the
   requester has the option of attempting to retrieve fresh values for
   the blocks it retrieved first.  

> forbids up/downloading blocks out of order.

Correct, and it would be a mistake to explicitly forbid this.
(It would be an even worse mistake to require servers to check this.)
But a server that, say, has to convert line-endings or text encoding on the fly, may only have reasonable means to support sequential access.
So it would also be a mistake to require all servers to allow out-of-order access.
(Again, the word "allow" can have two meanings here:
-- not disallow
-- require implementation support
)

Grüße, Carsten

---

From: "Dijk, Esko" <[email-masked]>
Date: Thu, 12 Apr 2012 09:20:23 +0000

Just an editorial note on the block-08 text and the concepts of ordering and first/last block:
the term "final block" should probably be changed to "last block" to get this term consistent throughout. "Last block" is clearly defined in relation to the More Flag while the term "final block" is not.

Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Carsten Bormann
Sent: Wednesday 11 April 2012 23:35
To: Klaus Hartke
Cc: [email-masked]
Subject: Re: [core] draft-ietf-core-block-08 - Block ordering

Grüße, Carsten

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: "Dijk, Esko" <[email-masked]>
Date: Thu, 12 Apr 2012 09:36:30 +0000

Carsten,

> So a client that wants to maximize interoperability will probably operate in sequential fashion.
Ok. If this constitutes the minimum level of interoperability for core-block, should we mandate it as such?

Option 1) e.g. specify a CoAP end-point MUST be prepared to receive block numbers in sequential fashion.
Option 2) e.g. specify a CoAP end-point MAY use block numbers in sequential fashion.
              [this automatically implies an end-point MUST be prepared to receive sequential block numbers. At least an implementer MUST
               consider this possibility.]

Though, the sequential case is so obvious from the I-D that it is already a de-facto minimum interoperability (though not formally stated).

Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Carsten Bormann
Sent: Wednesday 11 April 2012 23:35
To: Klaus Hartke
Cc: [email-masked]
Subject: Re: [core] draft-ietf-core-block-08 - Block ordering

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 12 Apr 2012 12:28:04 +0200

> Option 1) e.g. specify a CoAP end-point MUST be prepared to receive block numbers in sequential fashion.

Well, it is hard to say what such a MUST ("be prepared to receive...") really constrains.
A server can still always go belly-up, stop serving a resource, ...

I think the point is indeed that there is a larger expectation of successful completion when a client steps sequentially through the blocks of a resource.

>               consider this possibility.]

But it also MAY use out-of order accesses... just with a slightly lower level of expectation that the server will be able to successfully respond.

> Though, the sequential case is so obvious from the I-D that it is already a de-facto minimum interoperability (though not formally stated).

That was indeed my reasoning so far.

To me it seems we could make all this more apparent by stating two things:

1) the protocol does not fundamentally disallow out-of-order access (and, many servers will be able to provide out-of-order access (e.g., because they are stateless))
2) still, there is an expectation that more servers will be able to cope with sequential access than with out-of-order access.  Servers that implement one of the Block options SHOULD enable sequential access, and MAY enable out-of-order access.

Grüße, Carsten

---

From: "Dijk, Esko" <[email-masked]>
Date: Thu, 12 Apr 2012 13:17:14 +0000

> 2) still, there is an expectation that more servers will be able to cope with sequential access than with out-of-order access.  Servers that implement
> one of the Block options SHOULD enable sequential access, and MAY enable out-of-order access.

That sounds ok for me!

thanks
Esko

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]]
Sent: Thursday 12 April 2012 12:28
To: Dijk, Esko
Cc: Klaus Hartke; [email-masked]
Subject: Re: [core] draft-ietf-core-block-08 - Block ordering

Grüße, Carsten

---

From: Bert Greevenbosch <[email-masked]>
Date: Mon, 16 Apr 2012 08:22:13 +0000

Hi all,

A related issue: if the blocks are not sent in sequential order, how to ensure that an attacker cannot send absurd block numbers?

For example, an implementation that just appends the total data it has received until now with zeros until the start of the latest received block, would create a very large resource when a received block number is very high. This can be especially nasty as the attacker does not send much data to create this resource.

For example:
To setup the block transfer, an attacker first sends PUT with block no. 0, block size 1024 and the more flag set to 0. The next block he sends has block number 200000. This corresponds to the block with byte positions 204800000..204801023 in the original resource. The naive implementation could therefore create a file of 204801023 bytes, i.e. 195 MB!

A topic for the "security considerations" section?

Best regards,
Bert

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Dijk, Esko
Sent: 12 April 2012 21:17
To: Carsten Bormann
Cc: [email-masked]; Klaus Hartke
Subject: Re: [core] draft-ietf-core-block-08 - Block ordering

That sounds ok for me!

thanks
Esko

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]]
Sent: Thursday 12 April 2012 12:28
To: Dijk, Esko
Cc: Klaus Hartke; [email-masked]
Subject: Re: [core] draft-ietf-core-block-08 - Block ordering

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Apr 2012 10:39:58 +0200

> A topic for the "security considerations" section?

Definitely worth a mention.

Now #209.

I'm sure we can find a few more things that might be obvious to experienced implementers but would still be a good checklist to tick off while implementing.

Grüße, Carsten

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Mon, 16 Apr 2012 16:11:19 +0000

Following this discussion I fear that blockwise transfer is becoming something even more complex than TCP.
To me it now sounds like "everything is allowed, do whatever you want." A protocol should, however, give clear rules on how to communicate, shouldn't it?

What happened to the simple stop-and-wait idea?
I think we should make this the default and then allow a special use-case for Block2 to have this random access (aligned to the block sizes) for resources. This means Block1 MUST always start with block 0 and monotonically go up to the last block. This massively simplifies the implementations and provides actual protocol rules.

For crazy transfer schemes, people can still use the query option for instance.

Ciao
Matthias

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Apr 2012 18:40:37 +0200

> Following this discussion I fear that blockwise transfer is becoming something even more complex than TCP.

Then you haven't implemented TCP yet :-)

> To me it now sounds like "everything is allowed, do whatever you want."

A protocol does two things:

-- assign meaning to certain sequences of packets ("MAY")
-- constrain implementations to actually implement that meaning for certain sequences ("MUST implement")

It is quite OK to have more sequences that have well defined meanings than sequences that are "MUST implement".

> A protocol should, however, give clear rules on how to communicate, shouldn't it?

The MUST implement is clearly sequential, and I don't think anyone proposes to change that.
Please do look at Figure 4 though, why "sequential" may not always be in terms of block numbers.

> What happened to the simple stop-and-wait idea?

That is the MUST implement.
(Well, if you implement Block at all, that is.)

> I think we should make this the default and then allow a special use-case for Block2 to have this random access (aligned to the block sizes) for resources. This means Block1 MUST always start with…

I don't see a difference between stateless Block2 and stateless Block1.
(Clearly, the latter has different use cases.)
You cannot *not* implement random access when you run stateless.

Oh, and you can't really specify "MUST always go up to the last block" even for atomic Block1, because the client might lose power in the middle.  You still have to say what happens when it gets power again a minute later and maybe tries the same sequence of transfers again, before the state/context has timed out at the server.  That's what I meant by saying "overwrite state/context at the server".

Grüße, Carsten

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Mon, 16 Apr 2012 17:48:28 +0000

> The MUST implement is clearly sequential, and I don't think anyone proposes

Good, but is this actually said anywhere in the draft?
I re-read block-08 and could only find implicit statements such as "sequence of blocks" (nothing about the order) and the examples.

> (Clearly, the latter has different use cases.) You cannot *not* implement
> random access when you run stateless.

I thought that with some restrictions for the client, we could minimize the probability to break resources or make it easier for the server to check if the received representation can be valid. But in the end this does not matter, as the client always has to play along to not break the resources.

> Oh, and you can't really specify "MUST always go up to the last block" even
> for atomic Block1, because the client might lose power in the middle.  You still
> have to say what happens when it gets power again a minute later and
> maybe tries the same sequence of transfers again, before the state/context
> has timed out at the server.  That's what I meant by saying "overwrite
> state/context at the server".

Yes, that was a problematic textual shortcut I took.

---

From: "Dijk, Esko" <[email-masked]>
Date: Wed, 18 Apr 2012 07:54:18 +0000

The assumption of sequential block ordering seems to be present in various places in the draft. This gives some confusion whether random access and non-sequential transfers are truly supported, and if so, to what degree exactly.

What could help to clarify is

 1) A definitions section defining relevant terms (first block, last block, final block, sequence, sequence of blocks, sequence of requests/transfers, transfer sequence, ...)

 2) A section on the topic of ordering.

Some examples where the draft could be more clear on ordering/non-sequential transfer cases:

* the general term "sequence" seems used both for
  (1) the sequence of blocks describing the resource body; and
  (2) the sequence of blocks in a blockwise transfer.
For (1), block 0 is the first block of the sequence and block 'Nmax' is the last block which has a size between 1-BlockSize. The last block is the one with the M bit unset according to the definition on page 9.
For (2), with non-sequential ordering or random access, the first block in the sequence could be any block 0...Nmax and the last block could also be any block 0...Nmax. The M bit is *set* on the last block if it's not block Nmax. (Would an end-point expect more blocks then since the More bit is 1?)

* in other words: there's a concept of "last block" or "final block" which refers to  [page 9 definition]
   (1) the last block of the body; but seems *also*
   (2) whether there are one or more blocks available (does this mean in the sequence of blocks being transferred?)

* " The block size given (SZX) suggests a block size (in the case
         of block number 0) or repeats the block size of previous blocks
         received (in the case of block numbers other than 0)."
Does this constrain a block transfer sequence/order to always start with block 0? If not intended that way: Block number 0 here should probably be "the first requested block in a transfer sequence"; "Block numbers other than 0" should be "subsequent blocks in a transfer sequence".

* "Note that any reduction in the block size may mean that the second request
   starts with a block number larger than one,"
There seems an assumption here that block transfer is linear.

* "In a blockwise transfer of a request payload (e.g., a PUT or POST)
   that is intended to be implemented in an atomic fashion at the
   server, the actual creation/replacement takes place at the time the
   final block, i.e. a block with the M bit unset in the Block1 Option,
   is received."
This constrains the last block in a sequence to be the last block of the body?

* "last block" and "final block" -> maybe harmonize terms

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 18 Apr 2012 09:04:15 -0600

>               consider this possibility.]

Saying a sever MAY do something is more or less equivalent to saying the client MUST NOT bother to use this. I'm not a fan of anything where the client will not know if three code will work with the server or not.

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 18 Apr 2012 09:05:25 -0600

To make your questions a little more pointed .... lets assume that the resource is the type that changes rapidly and where the server has to provide take a snapshot of the whole resource to keep it consistent. So say a resource is 7 blocks long and the client requests block 3 causing the server to make a snapshot of it with the assumption that the client will request the other blocks. Then the client asks for block 5. If the client never asks for another block, when does the server discard the snapshot ?

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 18 Apr 2012 19:12:17 +0200

On Apr 18, 2012, at 17:05, Cullen Jennings wrote:

> So say a resource is 7 blocks long and the client requests block 3 causing the server to make a snapshot of it with the assumption that the client will request the other blocks. Then the client asks for block 5. If the client never asks for another block, when does the server discard the snapshot ?

The Block2 server cache timeout issue is unrelated to the block ordering issue -- you can make up an equivalent scenario with the numbers 0, 1, and 2.

To me this sounds like a quality of implementation issue, though.  A server that wants to provide this kind of cache will have to do this within the space it has.  So some will be able to keep it up for hundreds of seconds, while others will have to discard the cache earlier.  What can we standardize here, except for the clients to plan for occasional failure?

It is probably worth pointing out in the draft that "admitting" each further client by giving it a new cache (and discarding even relatively young older caches in the process) is not a winning strategy.  Sharing the cache is better, but limits the rate in which updates can be made available, to that of the slowest requester.  That is only a complete strategy as long as there is enough space for one cache for each resource...  An so on.  Lots of room for policy evolution, no need for mechanism.

Except maybe another error code ("would like to, but can't build cache for you"). 5.00 might work, though.

Grüße, Carsten