Subject: [core] WGLC comments on draft-ietf-core-block-08

From: Cullen Jennings <[email-masked]>
Date: Tue, 17 Apr 2012 22:58:38 -0600

This draft is fairly hard to understand. I suspect that is because the level complexity in it has evolved over time and I think the terminology could be re factored to make it clearer. 

I think we have three different semantic messages. One is a asking for a given block - Let me call this GetBlock. One is transferring a specific block, let me call this TransferBlock, and one is acknowledge receipt of a given block, let me call that AckBlock. Further more, we have these block transfer protocol going in the direction of the CoAP request, and the transfer going in the direction of the CoAP Response. 

I propose we reroute this draft to have 6 instead of 2 options and the options are called 

The first three are ReqGetBlock, ReqTransferBlock, ReqAckBlock, and second three are RespGetBlock, RespTransferBlock, RespAckBlock. Now the blocks of three might share the same option code in which case the code is the same as it is today or they might just allocate different option codes in which case the code is probably simpler. But the key thing here even if they have just two option codes, is they will make it much easier to rewrite the spec in a way that is easy to understand. The draft can talk about when you get a TransferBlock, you send an AckBlock for without having to try and figure out all the cases that happen for different method codes. I think we should also define BlockServer being the thing sending the TransferBlock and BlockClient being the thing receiving. 

It makes things clear like the size in a GetBlock is just a request the other size and the server will set the size it wants in the TransferBlock. 

On the size negotiation, I think the GetBlock should have a size which is the request size. The server is allowed to choose a size smaller than this but not one larger. 

I think we need to get more presses about the atomic nature block transfer semantics. Specifically the requirements around if someone client A is doing a put on resource while client B is going a get, but with block transfers. I'm Ok with the idea that a server might know how to incrementally update a resource in an atopic way but I'm not seeing how a client would understand some bizarrely fragmented result of a get. A lot more detail is needed on this. 

Some people will want to use this for streaming of infinitely long resources. It seems like the basic machismo will support that with no changes but probably needs some details explaining how to do that. Particularly when mapping to HTTP. 

It's unclear to me how a client discovers the server support blocks. Does it just put a GetBlock request on every GET it does? Similarly, how the server know if the client supports it. I assume the idea is since it is critical, you try and if you get an error on that destination, then you retry without it. That seems a bit painful. One advantage of different option codes for the GetBlock from TransferBlock is that GetBlock could be elective and TransferBlock Critical resulting in much easier negotiation of support.  I'll note that ETAGS are usually relatively big. 

I'm not seeing need use the same size block for all the transfers for a given request as long as the combination of num and sz accurately indicate where the block fits in the overall resource. This becomes particularly relevant in not delaying streams. 

I wonder if the size option should be moved into it's own spec.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 18 Apr 2012 11:02:53 +0200

> This draft is fairly hard to understand. I suspect that is because the level complexity in it has evolved over time and I think the terminology could be re factored to make it clearer. 

Klaus last week gave me a great set of proposals for further editorial improvement that I haven't had time to properly act on.  I'll combine them with your editorial points and make a ticket in the next couple of days.

> I propose we reroute this draft to have 6 instead of 2 options and the options are called 

I think it is a great idea to multiply out the existing options and the kinds of uses (what I called "descriptive" and "control") and give them explicit names for the exposition in the spec.
I think actually giving them different option numbers borders on gratuitous change at this point.

> 
> The first three are ReqGetBlock, ReqTransferBlock, ReqAckBlock, and second three are RespGetBlock, RespTransferBlock, RespAckBlock.

I don't think there is a "ReqGetBlock"; similarly, RespTransferBlock and RespAckBlock are the same.

> Now the blocks of three might share the same option code in which case the code is the same as it is today or they might just allocate different option codes in which case the code is probably simpler. But the key thing here even if they have just two option codes, is they will make it much easier to rewrite the spec in a way that is easy to understand. The draft can talk about when you get a TransferBlock, you send an AckBlock for without having to try and figure out all the cases that happen for different method codes. I think we should also define BlockServer being the thing sending the TransferBlock and BlockClient being the thing receiving. 
> 

Much of this comes down to the specific application policies that a resource requires.  Block gives you mechanism, but it doesn't replace thinking about policy.
Trying to nail down all that policy would be a mistake.

> Some people will want to use this for streaming of infinitely long resources. It seems like the basic machismo will support that with no changes but probably needs some details explaining how to do…

One disadvantage Block has here is that it only supports a small number of rigid block sizes.  So if I have a video frame of 563 bytes, this is hard to cut down into the right number of blocks.

> It's unclear to me how a client discovers the server support blocks.

It usually doesn't have to.

For Block2, just request without it -- the server will switch to block-wise if needed.
For Block1, if the resource to be sent is large, you just don't win with a server that doesn't to blocks.

> Does it just put a GetBlock request on every GET it does?

If it needs to influence the block size, that is the only thing it can do.
If it trusts the server to do something sensible, there is not need to send a Block2.

> Similarly, how the server know if the client supports it.

Again, it doesn't have normally have to.
Even if the client does not send a block option, the server will have little recourse but to send blocks if the resource is large.

The design works best with smart objects the resources of which exhibit a bimodal distribution, with frequently transferred small resources, and few much larger ones (that typically are operated on during setup and management, only).

> I assume the idea is since it is critical, you try and if you get an error on that destination, then you retry without it. That seems a bit painful. One advantage of different option codes for the GetBlock from TransferBlock is that GetBlock could be elective and TransferBlock Critical resulting in much easier negotiation of support.  

Yes, we considered something like that early in the design, but didn't think this separation gave you much.  Let's face it: Only the most minimal implementations won't know anything about Block.

(Note that only the preferred block size indication field is somewhat elective; the block number field is critical.  The preferred block size indication field also is not supposed to be ignored once the server does act on the Block2 option.)

So if this is really desirable, we could add a separate elective option just about preferred block sizes.

> I'll note that ETAGS are usually relatively big. 

(We limited them to 8 bytes, so they are big, but not that big.)
Yes, they will inflate the Block2 responses for a changing resource by maybe 5 to 20 %.
With the above assumptions about bimodality, that appears quite bearable.

> I'm not seeing need use the same size block for all the transfers for a given request as long as the combination of num and sz accurately indicate where the block fits in the overall resource. This…

I agree on all points, except for the stream argument -- we didn't make Block that great for streams (see above).

> I wonder if the size option should be moved into it's own spec. 

Size is mainly useful in conjunction with Block.  We also want to alert Block implementers that it is a good idea to implement Size along with Block.  Together, Block and Size constitute a "module" that you normally either implement together or don't.

Grüße, Carsten

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 18 Apr 2012 08:54:06 -0600

> I don't think there is a "ReqGetBlock"; similarly, RespTransferBlock and RespAckBlock are the same.

perhaps we need to have a WG phone call to sort this out but I think we are talking past each other. I agree they can reuse the same packet layout and even code point, but semantically the message with the data and message with the ack that you got the data are two different semantic meanings. 

> Trying to nail down all that policy would be a mistake.

hmm, here is the problem. The client has to know what the server will and will not do or you have interoperability problems. 

> For Block2, just request without it -- the server will switch to block-wise if needed.

and if the the client does not support it? 

> For Block1, if the resource to be sent is large, you just don't win with a server that doesn't to blocks.

This sounds like everyone has to do it. That's not really what we are looking for. This needs to be an optional mechanism - blocks trades off some application complexity of implementing blocks vs the the loss of reliability that is provided by just using IP level fragmentation. 

> Even if the client does not send a block option, the server will have little recourse but to send blocks if the resource is large.

again - I disagree. If I am on a network with a MTU of 128 and I need to send a 300 byte body, both block and non block could work. I think it would be bad if the server implemented an algorithm like "only use block if the body is 5 times lager than MTU". That would mean in cases where the body was only 3 times MTU, and both the client and server supported block, it would not get used. 

I think the draft needs to explain when the server should use block and when it should not. 

> The design works best with smart objects the resources of which exhibit a bimodal distribution, with frequently transferred small resources, and few much larger ones (that typically are operated on…

Ok - that may explain some of the disconnect. I want to use this in a case where all the resources in the system include a public key signature. This results in them all being larger than one MTU but smaller than 5 MTU. I think of small as less than one MTU, large as bigger than you could possible use IP fragmentation, say 10 MTU, and medium as in between the two. 

> Yes, we considered something like that early in the design, but didn't think this separation gave you much.  Let's face it: Only the most minimal implementations won't know anything about Block.

right - here again I disagree. I think you are premising this on everyone will implement this. I think lots of devices will not implement it for a variety of reasons - I'm not trying to convince you. It's very hard to predict the future and either of us could be right. However I am trying to convince you to design a protocol that works for a wide range of possible futures. I'd call that flexibility. Obviously there is a trade off between flexibility and simplicity, but in this case I see no problem with using 6 option codes instead of 2 - particularly if that greatly simplifies people understanding of this protocol. 

> So if this is really desirable, we could add a separate elective option just about preferred block sizes.

I'm thinking that if the GetBlock has a num of 3 and size indicating 1024, that means the clients wants 1024 bytes of data starting at location 3*1024. If the server only wants to hand back 512 bytes of data, it returns a transfer with the num=6 and size indicating 512. The client looks at the 6 and 512 in the transfer and knows that this data is 512 bytes of data that goes at location 6*512. In the clients next request it can ask for whatever it wants. Of course the logical thing to ask for is the next byte it need which would be at location 7*512. After it gets that, instead of asking for 8*512, it might ask for 4*1024 again if it was willing to receive that. Something like that reduces all of the implementation complexity that comes from statements around what happened on a previous request change what happens in a future request. It's also very complicated to  test code where previous packets impact how the code path for the current packet. 

> (We limited them to 8 bytes, so they are big, but not that big.)

Thanks for reminding me - I had forgotten they were limited to 8. 

> I agree on all points, except for the stream argument -- we didn't make Block that great for streams (see above).

I guess this is what I would ask about stream. Think about it a bit and see if the current mechanisms can be used for streaming with very little or no change. If yes, explain how int he draft, if not explain why in the draft. If stream complicated block in a significant way, I would not want to add it as a requirement but if it is easy to do, there are many situations that can take advantage of streams. 

> Size is mainly useful in conjunction with Block.  We also want to alert Block implementers that it is a good idea to implement Size along with Block.  Together, Block and Size constitute a "module"…

fair enough - that's fine with me

> 
> Grüße, Carsten
> 
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core