Subject: [core] Consensus call on BERT draft-bormann-core-block-bert-01

From: Jaime Jiménez <[email-masked]>
Date: Wed, 15 Jun 2016 07:57:48 +0000

Hi all,

the authors of TCP/TLS think that the blockwise transfer document for CoAP for reliable transport should be merged with the main TCP/TLS draft. I would like to ask the WG for their opinion on it. 

https://tools.ietf.org/html/draft-bormann-core-block-bert-01 

As there hasn’t been much discussion on the mailing list about it, I think one week time for reading and commenting is sufficient, so let’s check back 22-06 on this one.

Ciao!
- - Jaime Jimenez

---

From: peter van der Stok <[email-masked]>
Date: Wed, 15 Jun 2016 17:11:51 +0200

Hi all,

I can imagine that people will implement coap TCP/TLS RFC without 
implementing block RFC.
So a separation seems reasonable.

Peter

Jaime Jiménez schreef op 2016-06-15 09:57:
> Hi all,
> 
> the authors of TCP/TLS think that the blockwise transfer document for
> CoAP for reliable transport should be merged with the main TCP/TLS
> draft. I would like to ask the WG for their opinion on it.
> 
> As there hasn’t been much discussion on the mailing list about it, I
> think one week time for reading and commenting is sufficient, so let’s
> check back 22-06 on this one.
> 
> Ciao!
> - - Jaime Jimenez
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 15 Jun 2016 17:22:09 +0200

> So a separation seems reasonable.

Interesting point.  We currently have no way to indicate block-wise
capability, we just assume everybody (except for the most basic devices)
will do that.  I would expect that any device that has resources for TCP
has resources for block-wise.

But then, with larger messages sizes (via the Max-Message-Size
capability indication) available in TCP, there may be no need to
implement block-wise on a TCP node (assuming the device's application
does not care about head-of-line blocking).

But is there a point in a TCP device that has block-wise but not BERT?

Grüße, Carsten

---

From: "Kovatsch, Matthias" <[email-masked]>
Date: Mon, 20 Jun 2016 10:16:22 +0000

Dear group

In general I liked the BERT proposal from the beginning due to its necessity. Thus, I am for adoption as working group item. I am not fully sure where, yet.

> But then, with larger messages sizes (via the Max-Message-Size capability
> indication) available in TCP, there may be no need to implement block-wise on a
> TCP node (assuming the device's application does not care about head-of-line
> blocking).

I agree. I can imagine a constrained node with a custom TCP implementation, rather low Max-Message-Size, and hence no need for Block.

> But is there a point in a TCP device that has block-wise but not BERT?

My best guess is blockwise TCP always supports BERT.

The reason why I am not fully convinced yet that BERT must be part of the TCP draft is BERT over UDP. Does it make sense there? The receiver would not be able to negotiate the size. The sender has no way to learn the supported maximum message size. Like the path MTU, it would need trial and error to figure it out. Does anyone have a good use case for BERT over UDP? Otherwise, I tend for integration of BERT into the TCP draft.

Ciao
Matthias

---

From: Jaime Jiménez <[email-masked]>
Date: Tue, 21 Jun 2016 13:49:37 +0000

Hi,

a gentle reminder that the consensus call ends tomorrow. 

Ciao!
- - Jaime Jimenez

> On 20 Jun 2016, at 13:16, Kovatsch, Matthias <[email-masked]> wrote:
> 
> Dear group
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 21 Jun 2016 16:01:20 +0200

Hi Matthias,

> The reason why I am not fully convinced yet that BERT must be part of the TCP draft is BERT over UDP. Does it make sense there? The receiver would not be able to negotiate the size. The sender has…

Right now we are doing BERT for TCP/TLS/Websockets only (that's what the
RT in BERT is about -- Reliable Transport).

The reason is that making use of BERT is tied to increasing the maximum
message size (which is currently limited to 1152 by Section 4.6 of RFC
7252).

I currently don't know a good way to provide increased message size on
UDP -- while in theory the MTU could be larger than the usual Internet
MTU of around 1500 bytes, networks that can do this in practice are rare.
(Yes, there may be special networks that have larger MTUs.  We still
have a detection problem.  And no, you don't want to use fragmentation
except maybe in very controlled environments.)

Do you have a specific use case in mind that would benefit from BERT
over UDP?

(If we do want to do BERT over UDP, it is not clear to me that defining
the extension in the TTW draft is detrimental -- another document could
easily pick this extension up and define the missing pieces for UDP/DTLS.)

Grüße, Carsten

---

From: Michel Veillette <[email-masked]>
Date: Tue, 21 Jun 2016 14:32:53 +0000

Hi Carsten

Section 4.6 of RFC 7252 provides a recommended maximum message size, but doesn't preclude the use of larger messages.
With the adoption of the "L3" format, don't we extend this maximum CoAP message size to 4 GB?
In this context, why do we need BERT?

I'm asking because we already have many fragmentation options (6lowPAN, IP fragmentation, CoAP block, TCP) and in general more options means less interoperability.

Regards,
Michel

-----Original Message-----
From: core [mailto:[email-masked]] On Behalf Of Carsten Bormann
Sent: Tuesday, June 21, 2016 10:01 AM
To: Kovatsch, Matthias <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Consensus call on BERT draft-bormann-core-block-bert-01

Hi Matthias,

> The reason why I am not fully convinced yet that BERT must be part of the TCP draft is BERT over UDP. Does it make sense there? The receiver would not be able to negotiate the size. The sender has…

Right now we are doing BERT for TCP/TLS/Websockets only (that's what the RT in BERT is about -- Reliable Transport).

The reason is that making use of BERT is tied to increasing the maximum message size (which is currently limited to 1152 by Section 4.6 of RFC 7252).

I currently don't know a good way to provide increased message size on UDP -- while in theory the MTU could be larger than the usual Internet MTU of around 1500 bytes, networks that can do this in practice are rare.
(Yes, there may be special networks that have larger MTUs.  We still have a detection problem.  And no, you don't want to use fragmentation except maybe in very controlled environments.)

Do you have a specific use case in mind that would benefit from BERT over UDP?

(If we do want to do BERT over UDP, it is not clear to me that defining the extension in the TTW draft is detrimental -- another document could easily pick this extension up and define the missing pieces for UDP/DTLS.)

Grüße, Carsten

---

From: "Kovatsch, Matthias" <[email-masked]>
Date: Tue, 21 Jun 2016 15:34:22 +0000

> Do you have a specific use case in mind that would benefit from BERT over UDP?

Nope, that's why I asked if anyone has one.
That I could send around images of a few KiB on the ETH network is not really a use case, just an indicator that it is possible and convenient in some rare cases...

Since there has been no response since my last, I am now in favor of:

- BERT becomes part of the TCP draft where it defines how to use Blockwise over reliable/streambased channels.
- Either TCP-based devices have Block with BERT or they do not have Block support at all and make use of Max-Message-Size.

Ciao
Matthias

---

From: "Kovatsch, Matthias" <[email-masked]>
Date: Tue, 21 Jun 2016 15:44:01 +0000

Hi Michel

> Section 4.6 of RFC 7252 provides a recommended maximum message size, but
> doesn't preclude the use of larger messages.

To make it clear again: we now created an alternate CoAP universe where shipping around tons of data seems to be normal. However, it is not able to contact the constrained RESTful universe that we actually targeted... (Unless people are aware of this and design systems with reasonable message sizes and expectations.)

> In this context, why do we need BERT?

We need Blockwise Transfers to avoid head-of-line blocking: When you start sending your 100 MiB file, you will not be able to interact with the device for a long time until the large message was transferred. You might want to interleave other messages intro the transfer.

We need BERT because the available small Block sizes introduce a lot of overhead and notably degrade the performance in the alternate CoAP universe running TCP.

Ciao
Matthias

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 21 Jun 2016 17:56:47 +0200

> Section 4.6 of RFC 7252 provides a recommended maximum message size, but doesn't preclude the use of larger messages.

That is correct.  Still, the recommendation hold.

> With the adoption of the "L3" format, don't we extend this maximum CoAP message size to 4 GB?

The theoretical max is now slightly over 4 GiB, but will implementations
be able to handle that?
One of the purposes of Max-Message-Size in the signaling draft is to
enable capability indication.

> In this context, why do we need BERT?

Assuming that something like Max-Message-Size is used to indicate a
larger size, there are two reasons why you might still want to use BERT:

-- Max-Message-Size may be smaller than the payload that you actually
want to transfer.
-- You may want to avoid head-of-line blocking by very large messages.

Of course, you can open additional TCP connections to circumvent
head-of-line blocking, but that also requires additional resources.

> I'm asking because we already have many fragmentation options (6lowPAN, IP fragmentation, CoAP block, TCP) and in general more options means less interoperability.

* In TCP it is indeed natural to just send the data over the byte stream
that you need to send (unless you need to avoid HOL blocking).
TCP's segmentation is much more efficient than the other items on your list.
* You never want to use IP fragmentation; that's why TCP implementations
actively work to avoid that using Path MTU Discovery.
* 6LoWPAN (adaptation layer) fragmentation is aided by the 802.15.4 ACK,
so it is not quite as bad as IP fragmentation.  Still you may want to
minimize it, e.g. using the TCP MSS option.

So, in summary, for TCP/TLS/Websockets, there is indeed a choice between
just sending the large messages (and using TCP segmentation) and using
Block (possibly with BERT).  Block is less efficient, but also less
aggressive.  And, given head-of-line blocking, there may still be good
reasons to use it even over TCP.  But it is indeed not a requirement
(which it is, for most practical applications, on UDP).

Should we have a capability indication for blockwise transfers in the
signaling protocol?

Grüße, Carsten

---

From: Michel Veillette <[email-masked]>
Date: Tue, 21 Jun 2016 16:53:53 +0000

Hi Carsten and thanks for the answer

For networks for which TCP is considered too "aggressive" and for devices for which additional TCP connections is considered too much "resources", don't we already have CoAP block over UDP. The only remaining reason seem to be middlebox traversal. For this use case, some implementers seen to want a TCP solution which behave like CoAP block.

Support of this scenario shall not preclude the proper use of TCP with multiple packets window and multiple connections.
To your question " Should we have a capability indication for blockwise transfers in the signaling protocol?"
I assume that the answer is yes.

Regards,
Michel

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]] 
Sent: Tuesday, June 21, 2016 11:57 AM
To: Michel Veillette <[email-masked]>
Cc: Kovatsch, Matthias <[email-masked]>; [email-masked]
Subject: Re: [core] Consensus call on BERT draft-bormann-core-block-bert-01

> With the adoption of the "L3" format, don't we extend this maximum CoAP message size to 4 GB?

The theoretical max is now slightly over 4 GiB, but will implementations be able to handle that?
One of the purposes of Max-Message-Size in the signaling draft is to enable capability indication.

> In this context, why do we need BERT?

Assuming that something like Max-Message-Size is used to indicate a larger size, there are two reasons why you might still want to use BERT:

-- Max-Message-Size may be smaller than the payload that you actually want to transfer.
-- You may want to avoid head-of-line blocking by very large messages.

Of course, you can open additional TCP connections to circumvent head-of-line blocking, but that also requires additional resources.

> I'm asking because we already have many fragmentation options (6lowPAN, IP fragmentation, CoAP block, TCP) and in general more options means less interoperability.

* In TCP it is indeed natural to just send the data over the byte stream that you need to send (unless you need to avoid HOL blocking).
TCP's segmentation is much more efficient than the other items on your list.
* You never want to use IP fragmentation; that's why TCP implementations actively work to avoid that using Path MTU Discovery.
* 6LoWPAN (adaptation layer) fragmentation is aided by the 802.15.4 ACK, so it is not quite as bad as IP fragmentation.  Still you may want to minimize it, e.g. using the TCP MSS option.

So, in summary, for TCP/TLS/Websockets, there is indeed a choice between just sending the large messages (and using TCP segmentation) and using Block (possibly with BERT).  Block is less efficient, but also less aggressive.  And, given head-of-line blocking, there may still be good reasons to use it even over TCP.  But it is indeed not a requirement (which it is, for most practical applications, on UDP).

Should we have a capability indication for blockwise transfers in the signaling protocol?

Grüße, Carsten

---

From: Christian Groves <[email-masked]>
Date: Wed, 22 Jun 2016 15:55:26 +1000

Hello Carsten,

I've been working on a draft for CoAP over WebRTC datachannels (DC) 
(e.g. SCTP/DTLS/UDP(or TCP). One of the scenarios that I've been looking 
at is that the WebRTC endpoint provides interworking from CoAP over UDP 
or CoAP over TCP to CoAP over WebRTC DCs. Whilst SCTP doesn't suffer 
from head of line blocking and allows larger MTU sizes (minimising the 
need for block/bert) it may be advantageous to signal the use of 
blockwise transfers so that the WebRTC endpint point doesn't have to do 
block/bert interworking.

Regards, Christian

---

From: Jaime Jiménez <[email-masked]>
Date: Thu, 23 Jun 2016 13:59:00 +0000

Hi all,

to try to get back the discussion on focus, we are now discussing whether it is good to have BERT in the TCP document. Somehow the discussion has shifted to whether we want to have Blockwise transfers on TCP.

I think that it could be good to give few days more for the discussion to continue. From Matthias’ comments I get that the points of discussion are:

1. TCP draft should define how -block works.
2. Some TCP devices will do not have -block.
3. Some TCP devices will have -block. In those cases I assume they should support BERT, right?
4. A capability indication for blockwise transfers seems necessary.

So, in order to move forward with this part it is (3) that I would like to confirm for now. Specially with Michel, since I am not sure about your opinion on it from previous’ emails.

Thanks!
- - Jaime Jimenez

On 22 Jun 2016, at 08:55, Christian Groves <[email-masked]<mailto:[email-masked]>> wrote:

Hello Carsten,

I've been working on a draft for CoAP over WebRTC datachannels (DC) (e.g. SCTP/DTLS/UDP(or TCP). One of the scenarios that I've been looking at is that the WebRTC endpoint provides interworking from CoAP over UDP or CoAP over TCP to CoAP over WebRTC DCs. Whilst SCTP doesn't suffer from head of line blocking and allows larger MTU sizes (minimising the need for block/bert) it may be advantageous to signal the use of blockwise transfers so that the WebRTC endpint point doesn't have to do block/bert interworking.

Regards, Christian

Regards,
Michel

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]]
Sent: Tuesday, June 21, 2016 11:57 AM
To: Michel Veillette <[email-masked]<mailto:[email-masked]>>
Cc: Kovatsch, Matthias <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [core] Consensus call on BERT draft-bormann-core-block-bert-01

Michel Veillette wrote:
Hi Carsten

Grüße, Carsten
_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 23 Jun 2016 16:10:45 +0200

Hi Jaime,

thanks for summarizing the ongoing discussion.
Indeed, we are not discussing the technical substance of BERT, which
seems fine so far with everyone who reviewed it, but:

1) Whether TCP should use block-wise

> Hi all,
> 
> to try to get back the discussion on focus, we are now discussing
> whether it is good to have BERT in the TCP document. Somehow the
> discussion has shifted to whether we want to have Blockwise transfers on
> TCP. 
> 
> I think that it could be good to give few days more for the discussion
> to continue. From Matthias’ comments I get that the points of discussion
> are:
> 

The -block draft does that (no change caused by TCP).

> 2. Some TCP devices will do not have -block.

That seems to be a valid point of view, so 4. below.

> should support BERT, right?

BERT is useful with Max-Message-Size > 2048.  Maybe we can say that BERT
support is assumed when block-wise support is available and
Max-Message-Size > 2048?

> 4. A capability indication for blockwise transfers seems necessary. 

Right.  To be included in next version of -sig.

Grüße, Carsten

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Thu, 23 Jun 2016 18:40:31 +0000

Hi

> The -block draft does that (no change caused by TCP).

With my sloppy comment for Jaime meant is that the TCP draft has to say how Blockwise Transfers apply to CoAP-over-TCP (if the upcoming RFC applies directly or if something needs to be taken into account---like for Observe).

> support is assumed when block-wise support is available and

2048 because it is nice and round?
Technically it would be something around 1152, which would allow for blocks larger than 1024, right?

> Right.  To be included in next version of -sig.

+1

Matthias

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 23 Jun 2016 20:45:33 +0200

> 2048 because it is nice and round?

Because BERT operates in multiples of 1024, and 2048 is the first
multiple of 1024 that actually requires BERT.
(You need a couple more bytes than 2048 of course, for the message
header and options.)

(Technically, sending a 1024-byte block and then a final block of 1025
bytes also requires BERT, so pedantically speaking you are right...)

Grüße, Carsten

---

From: Christian Groves <[email-masked]>
Date: Fri, 24 Jun 2016 11:04:05 +1000

+1 on 4 also.
Christian

---

From: "weigengyu" <[email-masked]>
Date: Fri, 24 Jun 2016 10:36:31 +0800

Hi all,

When a CoAP client is tied with a CoAP server only by a TCP connection,
it seems to be OK.

But, whether the senario of a CoAP client---Gateway---a CoAP server needs 
concerning?
Note: CoAP-over-TCP is beween the CoAP client and the gateway, and 
CoAP-over-UDP is beween the gateway and the CoAP server,

Is a CoAP session is Hop-by-Hop, or End-to-End?

Regards,

Gengyu WEI
Network Technology Center
School of Computer
Beijing University of Posts and Telecommunications
-----原始邮件----- 
From: Christian Groves
Sent: Friday, June 24, 2016 9:04 AM
To: Kovatsch Matthias ; Carsten Bormann ; Jaime Jiménez
Cc: [email-masked] WG
Subject: Re: [core] Consensus call on BERT draft-bormann-core-block-bert-01

+1 on 4 also.
Christian

On 24/06/2016 4:40 AM, Kovatsch Matthias wrote:
> With my sloppy comment for Jaime meant is that the TCP draft has to say 
> how Blockwise Transfers apply to CoAP-over-TCP (if the upcoming RFC 
> applies directly or if something needs to be taken into account---like for 
> Observe).
>

---

From: Abhijan Bhattacharyya <[email-masked]>
Date: Fri, 24 Jun 2016 12:33:36 +0530

> Is a CoAP session is Hop-by-Hop, or End-to-End?

Probably a draft on transport layer proxy for CoAP (CoAP as application 
layer ; fwd:UDP -> TCP, reverse: TCP -> UDP ) would have captured all such 
issues? 

Regards
Abhijan Bhattacharyya
Associate Consultant
Scientist, Innovation Lab, Kolkata, India
Tata Consultancy Services
Mailto: [email-masked]
Website: http://www.tcs.com
____________________________________________
Experience certainty.   IT Services
                        Business Solutions
                        Consulting
____________________________________________

"core" <[email-masked]> wrote on 06/24/2016 08:06:31 AM:

> From: "weigengyu" <[email-masked]>
> To: "Kovatsch Matthias" <[email-masked]>, "Carsten Bormann" 
> <[email-masked]>, Jaime Jiménez <[email-masked]>, 
> "Christian Groves" <[email-masked]>
> Cc: [email-masked]
> Date: 06/24/2016 08:06 AM
> Sent by: "core" <[email-masked]>
> 
> Hi all,
> 

=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

---

From: Jaime Jiménez <[email-masked]>
Date: Fri, 24 Jun 2016 15:42:28 +0000

Hi all,

I have to close the consensus call on BERT. To summarise the discussion so far:

1. We need to indicate whether the TCP-device supports -block, thus some "blockwise availability” signalling should be added to -sig.
2. When the TCP device supports -block, they should support BERT too. Thus, we should add BERT to the main TCP draft. The technical discussion on the Max-Message-Size can continue there.
3. Any further issues -as the one raised by Wei Geng-Yu - can be discussed in the context of the merged draft.

As next steps, Brian Raymor has stepped forward as Editor of the TCP-TLS document. I would like to thank him for taking time to merge the contributions. The authors should continue helping him as they have been doing already.

Therefore, if that’s OK for everyone, let’s close the Consensus Call. Thanks everyone for the input!

Ciao!
- - Jaime Jimenez

> Is a CoAP session is Hop-by-Hop, or End-to-End?

Probably a draft on transport layer proxy for CoAP (CoAP as application layer ; fwd:UDP -> TCP, reverse: TCP -> UDP ) would have captured all such issues?

Regards
Abhijan Bhattacharyya
Associate Consultant
Scientist, Innovation Lab, Kolkata, India
Tata Consultancy Services
Mailto: [email-masked]<mailto:[email-masked]>
Website: http://www.tcs.com<http://www.tcs.com/>
____________________________________________
Experience certainty.        IT Services
                       Business Solutions
                       Consulting
____________________________________________

"core" <[email-masked]<mailto:[email-masked]>> wrote on 06/24/2016 08:06:31 AM:

> From: "weigengyu" <[email-masked]<mailto:[email-masked]>>
> To: "Kovatsch Matthias" <[email-masked]<mailto:[email-masked]>>, "Carsten Bormann"
> <[email-masked]<mailto:[email-masked]>>, Jaime Jiménez <[email-masked]<mailto:[email-masked]>>,
> "Christian Groves" <[email-masked]<mailto:[email-masked]>>
> Cc: [email-masked]<mailto:[email-masked]>
> Sent by: "core" <[email-masked]<mailto:[email-masked]>>
>
> Hi all,
>
> Cc: [email-masked]<mailto:[email-masked]> WG

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 27 Jun 2016 17:57:36 +0200

Jaime Jiménez wrote:
> 1. We need to indicate whether the TCP-device supports -block, thus some
> "blockwise availability” signalling should be added to -sig. 
> 2. When the TCP device supports -block, they should support BERT too.

Both items are now in the editor's draft for -sig:

https://core-wg.github.io/coap-sig/#rfc.section.4.3

Barring any ideas how this should be done differently, I'll submit that
as -02 later today.

Grüße, Carsten