Subject: Re: [core] draft-ietf-core-coap-09: retransmission window

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 2 Apr 2012 12:59:27 +0200

Dear WG:

A gentle reminder about retransmission window calculation from one year
ago.

It is probably a good practice that CoAP is TCP friendly and adaptive to
the environment changes.
ISA100.11a found that RFC 2988 was a fair method for achieving both
objectives. 
ISA100.11a also checked for a good respect of recommendation is RFC
5405.

Is there any reason why CoAP goes for a different approach? 
Do we have a clear idea how a large set of CoAP devices will share a
common network with TCP applications, as well as UDP applications that
would comply with the above RFCs?

Cheers,

Pascal

Subject: retransmission timer in CoAP
From: "Pascal Thubert (pthubert)" <[email-masked]>
To: "core WG" <[email-masked]>
Date: Sat, 26 Mar 2011 18:36:16 +0200

During a discussion around CON/ACK, the method for computing the retransmission timers came up.

For information, ISA100.11a settled for following RFC 2988. The main discussion, in fact, is what the initial value should be.

The point is that in a large deployment, the latency can grow very high and if the default is small, that means that many devices have to be reconfigured.

OTOH, in a small deployment, a too large default can  more easily be fixed because that represents few devices.

I understand that this comes from experience, that the recommendation is to be conservative; and that a few seconds is in fact, very optimistic.

What do you think?

Pascal Thubert
IPv6 Engineering
Product Development
 <mailto:[email-masked]> [email-masked]
Phone: +33 4 9723 2634
Mobile: +33 6 1998 2985

Cisco Systems France
Village d'Entreprises Green Side
400 Avenue de Roumanille 
Batiment T 3 
06410 
BIOT - SOPHIA ANTIPOLIS
France
 <http://www.cisco.com/global/FR/> Cisco.com

Description: Think before you print.Think before you print.

Cisco Systems France, Société à responsabiité limitée, Rue Camille Desmoulins – Imm Atlantis Zac Forum Seine Ilot 7 92130 Issy les Moulineaux, Au capital de 91.470 €, 349 166 561 RCS Nanterre, Directeur de la publication: Jean-Luc Michel Givone, For corporate legal information go to:
 <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: "Dijk, Esko" <[email-masked]>
Date: Thu, 12 Apr 2012 10:04:41 +0000

Hi Pascal,

did you have a specific proposal in mind for changing the retransmission window calculation?
Or could we comply to both RFCs already by simply choosing different Section 9 protocol constants?
(I haven't read the mentioned RFCs, but it appears relevant).

Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: Monday 2 April 2012 12:59
To: [email-masked]
Subject: Re: [core] draft-ietf-core-coap-09: retransmission window

Dear WG:

A gentle reminder about retransmission window calculation from one year ago.

It is probably a good practice that CoAP is TCP friendly and adaptive to the environment changes.
ISA100.11a found that RFC 2988 was a fair method for achieving both objectives.
ISA100.11a also checked for a good respect of recommendation is RFC 5405.

Is there any reason why CoAP goes for a different approach?
Do we have a clear idea how a large set of CoAP devices will share a common network with TCP applications, as well as UDP applications that would comply with the above RFCs?

Cheers,

Pascal

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 12 Apr 2012 12:14:09 +0200

Hello Esko:

The whole point is that a constant does not work satisfactorily. In fact
it will work even less in a WSN than it does on the broad Internet
because of the latencies that we can experience there.
You need a smoothed estimate of the round-trip time, and an estimate of
its variation to compute your retransmission timeout. The algorithm is
quite simple but you really need to read it in section 2 of
http://tools.ietf.org/html/rfc6298 .

ISA100.11a devices have incorporate that algorithm at app layer. The
only question is really what's the initial value.

Quote:
"
   The Internet, to a considerable degree, relies on the correct
   implementation of the RTO algorithm (as well as those described in
   RFC 5681) in order to preserve network stability and avoid congestion
   collapse.
"

Cheers,

Pascal

-----Original Message-----
From: Dijk, Esko [mailto:[email-masked]] 
Sent: jeudi 12 avril 2012 12:05
To: Pascal Thubert (pthubert); [email-masked]
Subject: RE: [core] draft-ietf-core-coap-09: retransmission window

Hi Pascal,

did you have a specific proposal in mind for changing the retransmission
window calculation?
Or could we comply to both RFCs already by simply choosing different
Section 9 protocol constants?
(I haven't read the mentioned RFCs, but it appears relevant).

Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Pascal Thubert (pthubert)
Sent: Monday 2 April 2012 12:59
To: [email-masked]
Subject: Re: [core] draft-ietf-core-coap-09: retransmission window

Dear WG:

Cheers,

Pascal

________________________________
The information contained in this message may be confidential and
legally protected under applicable law. The message is intended solely
for the addressee(s). If you are not the intended recipient, you are
hereby notified that any use, forwarding, dissemination, or reproduction
of this message is strictly prohibited and may be unlawful. If you are
not the intended recipient, please contact the sender by return e-mail
and destroy all copies of the original message.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 12 Apr 2012 12:27:13 +0200

On Apr 12, 2012, at 12:04, Dijk, Esko wrote:

> comply to both RFCs

I don't think we can "comply with" RFC 2988.
(Or with RFC 6298, which obsoletes RFC 2988.)

RFC 6298 is defined in the context of TCP; we simply don't have the connection state that we could base the same calculations on.

However, a future specification could use RFC 6298 as an inspiration.
That could even recommend keeping RFC-6298-style state, e.g., around DTLS connections, or in conjunction with observation state or cache entries.
Coap-09 certainly is open to other ways of deriving these numbers.

I believe any update here should be done based on relevant deployment experience.
ISA100.11a experience may or may not be "relevant", as the communication patterns in industrial process control may be much more regular than I'd expect in many CoAP applications.

For me the one open question in this thread (see subject) is whether we should give guidance for a default retransmission window.  Most implementations will probably work well with a default on the order of 256 s.  Should we give that guidance?

Grüße, Carsten

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 12 Apr 2012 13:16:56 +0200

Hi Carsten

 > RFC 6298 is defined in the context of TCP; we simply don't have the connection state that we could base the same calculations on.

[Pascal] there are ways. The question is whether all applications need to make the effort or not. If CoAP is used over a local/dedicated network, then it can be kept very simple and some RTO value can be engineered and hard coded. If CoAP operates over the Internet and becomes successful (and success here is a LARGE number of devices) then CoAP applications must be good internet citizens.

[Pascal] These are ways... 

[Pascal]  I believe section 3.1 of RFC 5405 derives from such experience. Section 3.1.2 has:
" Similar to the
   recommendation in [RFC1536], an application SHOULD maintain an
   estimate of the RTT for any destination with which it communicates.
   Applications SHOULD implement the algorithm specified in [RFC2988] to
   compute a smoothed RTT (SRTT) estimate.  They SHOULD also detect
   packet loss and exponentially back-off their retransmission timer
   when a loss event occurs.
"

 > ISA100.11a experience may or may not be "relevant", as the communication patterns in industrial process control may be much more regular than I'd expect in many CoAP applications.

[Pascal] I had it the other way around, like are we designing this protocol in a way that could be used by ISA as a replacement for their specific ways. That was the sense of the questions I asked when I came to the mike and so far the answer was satisfactory. Yes, probably ISA100 could in some future migrate to CoAP. That's still doable as long as CoAP allows for an RTO computation that can make it Internet friendly.

> For me the one open question in this thread (see subject) is whether we should give guidance for a default retransmission window.  Most implementations will probably work well with a default on the…

[Pascal] I certainly disagree. If CoAP does not play by Internet rules, it has to stay confined at the edge. Is that what we want? Can we document how this protocol, though disrespectful of RFC 5405, can still be innocuous to the Internet at large when the number of devices grows to the billions?

Grüße too,

Pascal

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 12 Apr 2012 14:04:21 +0200

Pascal,

thanks -- pointing to RFC 5405 out of the last paragraph of section 9 (or of 4.6?) is certainly a good idea.

I still don't think it is a slam-dunk how to apply 6298 (and the more general considerations of 5405) to the different kinds of CoAP environments.  Sure, for an ISA100.11a type environment, there are indications that 6298 works as is, but in the more general case, we'd need to spend some time understanding what that even means in the first place.

> [Pascal]  I believe section 3.1 of RFC 5405 derives from such experience.

No, I don't think there is much experience with CoAP-like traffic in the Internet yet.

> Section 3.1.2 has:

Yes, but it also says:

> Some applications cannot maintain a reliable RTT estimate for a
>    destination.  The first case is that of applications that exchange
>    too few UDP datagrams with a peer to establish a statistically
>    accurate RTT estimate.  Such applications MAY use a predetermined
>    transmission interval that is exponentially backed-off when packets
>    are lost.  TCP uses an initial value of 3 seconds [RFC2988], which is
>    also RECOMMENDED as an initial value for UDP applications.  SIP
>    [RFC3261] and GIST [GIST] use an interval of 500 ms, and shorter
>    values are likely problematic in many cases.  As in the previous
>    case, note that the initial timeout is not the maximum possible
>    timeout.

Well, the initial timeout of TCP has since moved down [RFC6298].
Otherwise we are right there already.

Grüße, Carsten

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Wed, 18 Apr 2012 09:16:51 +0200

On Wed, Mar 28, 2012 at 12:36, Carsten Bormann <[email-masked]> wrote:
>> Question 2: Should be specified that "the recipient MUST be prepared to receive the same confirmable message *within the potential retransmission window*" as well?
>
> Now ticket #201.

What about NON messages? How much time the recipient MUST be prepared
to receive the same non-confirmable message? (to eliminate duplication
caused by the network)

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 18 Apr 2012 09:38:30 +0200

> to receive the same non-confirmable message? (to eliminate duplication

Indeed, duplicate detection is not only for confirmable messages.

So #201 needs to talk about CON and NON.  
The main point of #201 is that we clarify the text to use the term "retransmission window" as much as possible.

For NON, there is no retransmission, so the window could be smaller than for confirmable messages.
However, for most implementations, it will be easier to simply have a single timeout value for both cases.
This just needs to be large enough, and "retransmission window" will do it for NON, too.

I added this discussion to #201.

Grüße, Carsten

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Wed, 18 Apr 2012 09:49:43 +0200

Can we define how much smaller can be the timeout for NONs?

On class-I devices, I will have only a very limited set of slots for
deduplication, depending upon available resources, and at some point I
cannot receive any more messages since the deduplication slots have
exhausted... So having a smaller timeout is a real benefit for me.

When deduplication cannot be assured anymore, since the deduplication
slots in my buffer are all busy, new messages should be ignored?

I think that this is probably the best solution. Moreover if the
message is a CON I will have another opportunity to receive it on
retransmission. I will prefer avoiding sending a  RST in this
situation.

But let's imagine that I have some observation in place, and the
servers are sending to me NON notifications at an aggregate rate
higher than the maximum one that I can receive.

Suppose that my bottleneck is the deduplication buffer, which has N
slots. The maximum rate R at which I can receive messages, if the
timeout is T will be:
R = N / T

Probably in this case sending some RST will be benificial.

Best,
Angelo

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 18 Apr 2012 09:13:00 -0600

On Mar 28, 2012, at 3:00 AM, Angelo P. Castellani wrote:

> Section 4.1 tells:
> 
>    The same Message ID MUST NOT be re-used (per Message
>    ID variable) within the potential retransmission window, calculated
>    as RESPONSE_TIMEOUT * RESPONSE_RANDOM_FACTOR * (2 ^ MAX_RETRANSMIT -
>    1) plus the expected maximum round trip time.
> 
> 
> Question 1: Given that these parameters are not mandated by the standard, does the spec assume that will be available some way to synchronize those parameters across different implementations to correctly avoid reusing a MID in the retransmission window?

The IESG will want to know that this protocol is TCP Friendly. Setting these values very low would cause this protocol to not be TCP Friendly. We will need to meet the appropriate BCP on congestion control for this spec to be approved. I would fall over backwards to see the IESG approve this spec with the spec saying you can set these however you want. 

I know that it seems to the WG like, hey, I should be able to set these however I want for my network and that is a good thing. But I don't think you will like this. The end result will be huge pressure on vendor B to set their default values to be more aggressive than vendor A. The arms race to aggression will result in setting that are too aggressive for many networks. 

The setting of these values is a classic case where we want to avoid tragedy of the commons. What the vendors need is standards body to set a bound on the level of aggression and that is what people will use as the if they want to say their device implements the RFC. Sure, the vendor might allow you to configure the device in a way that is not compliant with the spec - but hopefully it won't default to non compliant they want to put "Supports RFC XXXX" in their product sheet.

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Thu, 19 Apr 2012 09:06:42 +0200

On Wed, Apr 18, 2012 at 17:13, Cullen Jennings <[email-masked]> wrote:
> The setting of these values is a classic case where we want to avoid tragedy of the commons. What the vendors need is standards body to set a bound on the level of aggression and that is what people will use as the if they want to say their device implements the RFC.

+1