Subject: Re: [core] 🔔 WGLC draft-ietf-core-coap-tcp-tls

From: Robert Quattlebaum <[email-masked]>
Date: Mon, 3 Apr 2017 15:08:32 -0700

Hello Everyone,

I had an observation about the implications to observation that the
current text of draft-ietf-core-coap-tcp-tls appendix A seems to have.
I fully realize that I am way too late in this observation, as WGLC
has already come and gone, but I figure it can't hurt to bring it up
and see what others think.

I've got several smart light switches in my house. They all speak CoAP
and can be configured to observe the state of each other. This allows
for a "two-way" switch (called "three-way" in the US) to be
implemented without changing wiring, which I've found to be rather
convenient.

A slightly different scenario I have implemented is where I have one
switch (Switch-A) that, when turned on or off, will turn on (or off)
several other lights (Switch-B) which are all on dimmers. I can then
walk over to those lights and control them individually (perhaps
turning them off, or by adjusting the dimmer) to suit my needs at the
time. When leaving the room I can turn them all off by controlling the
light switch at the entrance to the room.

To implement this, I have Switch-B observe the value of Switch-A.
However, to allow local changes, Switch-B keeps track of the observe
counter to determine if the value has been updated or not.

Now, you might say that I should use ETags for this. But Etags only
tell me if the value is the same as the last time I retrieved (or
observed) a resource, it doesn't necessarily tell me if the resource
had changed in the mean time. A resource could have changed to a
different value and then changed back. Also, ETags are optional. If
you are observing a resource, Observe counters are (or at least were)
required.

My own (ab)use of the Observe counter aside, the current language in
section A.1 of draft-ietf-core-coap-tcp-tls also makes the life of a
CoAP non-caching proxy/bridge more difficult by requiring additional
state to keep track of. Imagine a CoAP UDP-only node (Node-A) which is
observing a resource on another device (Node-B, TLS-only) via a CoAP
proxy. If the proxy connects to Node-B using TLS or WebSockets, then
Node-B may opt to leave the value on the Observe option empty. That
means the CoAP proxy must now keep track of what observations are
occurring, regenerating an index and populating the observe header
before sending the observe packet to Node-A. Even if Node-B did
include an Observe count, the current language says the proxy MUST
ignore it.

Section A.3 also seems to have similar problems in the proxy case. If
Node-B is trying to observe a resource on Node-A and it follows the
guidance in section A.3, it is going to be checking for the health of
the connection to the proxy rather than sending individual GET
requests. This means that the proxy is going to have to send its own
GET requests to Node-A to ensure liveliness.

Again, I fully realize that I missed the boat when it comes to this
sort of feedback, but nonetheless I wanted to go ahead and put this
out there to see what others thought.

- -\- RQ

---

From: Michael Koster <[email-masked]>
Date: Tue, 4 Apr 2017 07:05:38 -0700

Hi,

It seems that we have inadvertently removed some signaling functionality that is not solely related to the persistence of the TLS connection.

The observe option is more than simply a sequence number. It also a signal used by the server to indicate to the client that it is unwilling to send or to continue to send notifications. Making it optional and requiring it to be ignored removes the ability for the server to gracefully end or limit observe/notify operations.

   When included in a response, the Observe Option identifies the
   message as a notification.  This implies that a matching entry exists
   in the list of observers and that the server will notify the client
   of changes to the resource state. 

   A server that is unable or unwilling to add a new entry to the list
   of observers of a resource MAY silently ignore the registration
   request and process the GET request as usual.  The resulting response
   MUST NOT include an Observe Option, the absence of which signals to
   the client that it will not be notified of changes to the resource
   and, e.g., needs to poll the resource for its state instead.

Best regards,

Michael

> On Apr 3, 2017, at 3:08 PM, Robert Quattlebaum <[email-masked]> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Hello Everyone,
> 
> -----BEGIN PGP SIGNATURE-----
> 
> iQEcBAEBCAAGBQJY4seZAAoJEE6OLsxF2Ko6s+QH/jhkaA19Mp9Ub02yfyUzkKa3
> xaWtEIhAUxHFe0TXv9sbwJUi8V4q4Qb+VZRYWJqZB/u6SdLayTledcKKXYHId6ZB
> EqcKyFb29k3oFZs1hvtZxBtowekAlpV7efLxLNB+DFkioOl8AYEhibmAeQunJTZP
> KTpPhZiImsrZVScOSGT12wZ5/ogzjYTHOv9kM6DN4B/hrqt2ylxW/FS/7oxfr/My
> 7iRYof+dJvoD1Yz1xtJoNphQk1wja4eDyNuwAYkFlhdCPxpokG1BekLYmjGIQxHc
> h0k8buaj4uXdFE9RiAIDNCUx5zu+1Ur1Q2my37zbHzznPC8NgwCRSw7oMxt/ukY=
> =G/G9
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Michael Koster <[email-masked]>
Date: Tue, 4 Apr 2017 07:11:51 -0700

Hi,

My reading of how Etags work in both HTTP and CoAP implies that the Etag changes value every time a resource state update occurs. In a practical sense, if a resource changes value and then back to some previously held value, the Etag should change to a new unique value. I don't imagine an implementation trying to re-use an Etag when the resource value matches some previous value. But I could be missing something.

Is this a reasonable interpretation?

Best regards,

Michael

---

From: Klaus Hartke <[email-masked]>
Date: Tue, 4 Apr 2017 16:20:37 +0200

Michael Koster wrote:
> The observe option is more than simply a sequence number. It also a signal
> used by the server to indicate to the client that it is unwilling to send or
> to continue to send notifications. Making it optional and requiring it to be
> ignored removes the ability for the server to gracefully end or limit
> observe/notify operations.

draft-ietf-core-coap-tcp-tls doesn't make the Observe option optional.
The presence of the option still signals if the client is in the list
of observers or not. But since TCP already deals with reordering, the
sequence number inside the Observe in notifications is not needed any
more.

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Tue, 4 Apr 2017 16:23:31 +0200

Michael Koster wrote:
> My reading of how Etags work in both HTTP and CoAP implies that the Etag
> changes value every time a resource state update occurs. In a practical
> sense, if a resource changes value and then back to some previously held
> value, the Etag should change to a new unique value. I don't imagine an
> implementation trying to re-use an Etag when the resource value matches some
> previous value. But I could be missing something.

The point of caches is to reuse earlier responses as often as
possible. So, if a resource changes back to some earlier state, it
would be beneficial if the response describing that state could be
reused. Using a different Etag for the same response would be
counterproductive.

Klaus

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 4 Apr 2017 16:28:12 +0200

On 4. Apr 2017, at 16:11, Michael Koster <[email-masked]> wrote:
> 
>  I don't imagine an implementation trying to re-use an Etag when the resource value matches some previous value. 

In many implementations the ETag is simply a hash over the representation.

(There is a lot of discussion about ETags on the Web; e.g., see https://www.mnot.net/blog/2007/08/07/etags .)

Grüße, Carsten

---

From: Michael Koster <[email-masked]>
Date: Tue, 4 Apr 2017 07:30:35 -0700

I see it now; the observe option must be included but may be empty. The value must be ignored.

Thanks!

Michael

> On Apr 4, 2017, at 7:20 AM, Klaus Hartke <[email-masked]> wrote:
> 
> Michael Koster wrote:

---

From: Michael Koster <[email-masked]>
Date: Tue, 4 Apr 2017 07:31:52 -0700

OK, thanks for the clarifications. That makes sense, especially for enumerations.

Michael

> On Apr 4, 2017, at 7:28 AM, Carsten Bormann <[email-masked]> wrote:
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 4 Apr 2017 16:35:05 +0200

> I see it now; the observe option must be included but may be empty. The value must be ignored.

Add “in a 2.xx response that indicates an ongoing observation interest”, and you are almost right (for an option of uint format, being “empty” just means the number is 0; maybe we should have said that in A.1).

Grüße, Carsten

---

From: Michael Koster <[email-masked]>
Date: Tue, 4 Apr 2017 07:38:25 -0700

Ah, of course, the value will still be relevant when it indicates cancellation.

I think a little more text or a little more thought would have helped me.

Best regards,

Michael

> On Apr 4, 2017, at 7:35 AM, Carsten Bormann <[email-masked]> wrote:
>

---

From: Jim Schaad <[email-masked]>
Date: Tue, 4 Apr 2017 09:06:33 -0700

This would indicate to me that there needs to be some text in the document about dealing with transitions from TCP->UDP and the other direction.  If the value is "unneeded" and "ignored" then you might end up with problems if a proxy going between these two different transports does not do the right thing in terms of putting in "good" values for the observe option.

Are there other places where the fact that TCP has reliable deliver that are going to cause problems for proxies?  For example, should a UDP->TCP proxy always ack a request since it is going from unreliable to reliable delivery?

Jim

> -----Original Message-----
> From: core [mailto:[email-masked]] On Behalf Of Klaus Hartke
> Sent: Tuesday, April 4, 2017 7:21 AM
> To: Michael Koster <[email-masked]>
> Cc: core <[email-masked]>
> Subject: Re: [core] 🔔 WGLC draft-ietf-core-coap-tcp-tls
> 
> Michael Koster wrote:
> > The observe option is more than simply a sequence number. It also a
> > signal used by the server to indicate to the client that it is
> > unwilling to send or to continue to send notifications. Making it
> > optional and requiring it to be ignored removes the ability for the
> > server to gracefully end or limit observe/notify operations.
> 
> The presence of the option still signals if the client is in the list of observers or
> not. But since TCP already deals with reordering, the sequence number inside
> the Observe in notifications is not needed any more.
> 
> Klaus
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 4 Apr 2017 18:30:13 +0200

> On Apr 4, 2017, at 18:06, Jim Schaad <[email-masked]> wrote:
> 

The Observe value in a response is not meant to be end-to-end (it cannot be, as observations are hop-by-hop).

RFC 7641 may be slightly opaque here, but Section 5 does say:

   Each hop MUST generate
   its own values for the Observe Option in notifications

See also Figure 8 in RFC 7641 for an example of how that looks like.

> Are there other places where the fact that TCP has reliable deliver that are going to cause problems for proxies?  For example, should a UDP->TCP proxy always ack a request since it is going from u…

Any UDP CoAP in a proxy will send ACKs right away (maybe with a little piggyback timeout), independent of whether the request then goes out via TCP or UDP.  So this is not a new thing.

Grüße, Carsten

---

From: Klaus Hartke <[email-masked]>
Date: Tue, 4 Apr 2017 18:31:09 +0200

> This would indicate to me that there needs to be some text in the document about dealing with transitions from TCP->UDP and the other direction.  If the value is "unneeded" and "ignored" then you m…

Section 5 of RFC 7641 [1] contains some text that explains how
intermediaries work in the context of resource observation. In short,
resource observation is a hop-by-hop feature: In a
client--proxy--server chain, the Proxy generates the notifications and
sequence numbers for the client; and the server generates the
notifications and sequence numbers for the proxy. A TCP<->UDP
transition does not change that.

> Are there other places where the fact that TCP has reliable deliver that are going to cause problems for proxies?  For example, should a UDP->TCP proxy always ack a request since it is going from u…

Acknowledgements in CoAP-over-UDP are a hop-by-hop feature as well: In
a client--proxy--server chain, the proxy acknowledges the confirmable
messages from the client; and the server acknowledges the confirmable
messages from the proxy. An acknowledgement in CoAP has no meaning
other than "I have received your confirmable message; you can stop
retransmitting it now." Over TCP, TCP does the retransmission and
acknowledgement, so the messaging layer of CoAP-over-UDP is not
needed.

I can see some problems if someone tries to implement a CoAP proxy by
rewriting UDP datagrams into TCP segments. But that's not how CoAP
proxies are intended to work.

Klaus

[1] https://tools.ietf.org/html/rfc7641#section-5

---

From: Robert Quattlebaum <[email-masked]>
Date: Tue, 4 Apr 2017 10:49:13 -0700

That ACKs are hop-by-hop makes total intuitive sense to me. That CoAP
observations are hop-by-hop makes considerably less intuitive sense to
me. That being said, it's hard to argue with the text from RFC7641
that Carsten pointed out:

> Each hop MUST generate its own values for the Observe Option in
> notifications

This seems baffling to me, as it requires each "hop" to be aware of
observations when it wouldn't necessarily otherwise be required for
the feature to work¹. Even when the proxy is aggregating observations
(as it ideally should be), reusing the same numbers from the origin
seems like it would have been a good idea, were it not for this text.
But RFC7641 is cooked, so it is what it is. This text from RFC7641
does appear to make section A.1 from draft-ietf-core-coap-tcp-tls more
consistent.

An unfortunate consequence of this design is that it implies momentary
changes in the value of an observed resource cannot be reliably
detected across a "hop", but in the grand scheme of things I suppose
it isn't that big of a deal.

- -\- RQ

¹ I realize that this is technically incorrect, because for that to
  work resets would need to be propagated across "hops", too. I think
  it is rather unfortunate that resets are not propagated across
  hops, since it would have allowed for relatively dumb proxies. But
  seeing that apparently well-established consensus is that resets
  are not intended to propagate across hops I won't press the issue
  further.

> On Apr 4, 2017, at 9:31 AM, Klaus Hartke <[email-masked]> wrote:
> 
> Jim Schaad wrote:

---

From: Jim Schaad <[email-masked]>
Date: Tue, 4 Apr 2017 11:05:40 -0700

That ACKs are hop-by-hop does not make total intuitive sense to me, because the proxies that I have dealt with are much more stateless and therefore are far more likely to let the retransmission be largely invisible to them.  This is the model that I am far more used to from protocols such as RADIUS where we are trying to deal with the questions of how re-tries are done when moving from UDP to TCP and back.  If you ACK back from the proxy, then the client may never know that the server is down or is not going to respond.  Does a proxy know that it needs to fail some way if it later times out?  Really lightweight proxies are not going to do this.

Jim

> From: Robert Quattlebaum [mailto:[email-masked]]
> Sent: Tuesday, April 4, 2017 10:49 AM
> To: Klaus Hartke <[email-masked]>; Carsten Bormann <[email-masked]>
> Cc: Jim Schaad <[email-masked]>; core <[email-masked]>
> observations are hop-by-hop makes considerably less intuitive sense to me.
> That being said, it's hard to argue with the text from RFC7641 that Carsten
> pointed out:
> 
> observations when it wouldn't necessarily otherwise be required for the
> feature to work¹. Even when the proxy is aggregating observations (as it ideally
> should be), reusing the same numbers from the origin seems like it would
> have been a good idea, were it not for this text.
> But RFC7641 is cooked, so it is what it is. This text from RFC7641 does appear
> to make section A.1 from draft-ietf-core-coap-tcp-tls more consistent.
> 
> changes in the value of an observed resource cannot be reliably detected
> across a "hop", but in the grand scheme of things I suppose it isn't that big of a
> deal.
> 
> - -\- RQ
> 
> iQEcBAEBCAAGBQJY49ulAAoJEE6OLsxF2Ko6dBUH+QEga+qDxbg0jgXZOx/bE8U
> o
> WJtatgpXZIXKmmyePG2hNf+DZWBWnoo4n4N/NumLM3GD+vHSdDuWjNaWm
> CFSLnxx
> keKfuMpuppyV8k3ZIAeCJVsJw6+irysOzUb1t/izRwmvu+AZdmtH4VbnKy1WgOP
> f
> kTl55I74FEG/duy0jzL7rpOSJiUKrgazWHM4xTSgRZvQqj4/yOtOvx2AZAhNdXbn
> ENy4yVYZUsvJgDMfHXWHx9comRUTMaff202MjBompFiWyiyfB73kyjTsStLjYlDb
> D/6P0lWXh35lA2VIsANptr8Dl4f29bo9SwSrPwhcrMRn4VquRizFd6wnl8nVynY=
> =T675
> >> This would indicate to me that there needs to be some text in the
> document about dealing with transitions from TCP->UDP and the other
> direction.  If the value is "unneeded" and "ignored" then you might end up
> with problems if a proxy going between these two different transports does
> not do the right thing in terms of putting in "good" values for the observe
> option.
> >
> >> Are there other places where the fact that TCP has reliable deliver that are
> going to cause problems for proxies?  For example, should a UDP->TCP proxy
> always ack a request since it is going from unreliable to reliable delivery?
> >