Subject: [core] Moving Observe forward

From: Zach Shelby <[email-masked]>
Date: Wed, 4 Jan 2012 11:45:21 +0200

I have been going through the IETF82 minutes (http://tools.ietf.org/wg/core/minutes) and updating tickets etc. The discussion on the new Max-OFE option that was added to observe-03 is interesting reading for those of you that were not there.

That discussion left me feeling like we were premature in adding this option to Observe, as it clearly has nothing near WG consensus. I am not convinced that we even need it at this point.

Observe otherwise is in really good shape, and the WG seemed to think in Taipei that it would be ready for WGLC if this Max-OFE problem would be taken care of. 

My proposal would be to dump Max-OFE into a separate personal draft for future discussion and work, it could be standardized in that or some other form later. That would leave the Observe WG document ready for WGLC, but allow continued work on the idea.

What do others think?

Zach

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://www.sensinode.com
http://zachshelby.org  - My blog "On the Internet of Things"
http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

---

From: Likepeng <[email-masked]>
Date: Thu, 05 Jan 2012 00:32:56 +0000

> My proposal would be to dump Max-OFE into a separate personal draft for future discussion and work, it could be standardized in that or some other form later. That would leave the Observe WG docume…

+1.

Maybe it can be moved to misc draft.

Kind Regards
Kepeng
-----邮件原件-----
发件人: [email-masked] [mailto:[email-masked]] 代表 Zach Shelby
发送时间: 2012年1月4日 17:45
收件人: [email-masked] WG
主题: [core] Moving Observe forward

What do others think?

Zach

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: "Dijk, Esko" <[email-masked]>
Date: Fri, 13 Jan 2012 09:57:58 +0000

Hello Zach,

The discussion in the minutes shows a tension between two design views for this option:
1) "extend freshness" view as in RFC 5861; allow to serve stale content ; which is currently the view of Max-OFE
2) defining Max-Age and "refresh after time T" options (with T < Max-Age) ; stale content is never served.

Both views seem to amount to the same thing, once implemented? In both cases it gives a cache some well-defined time period to try and re-establish an observe relation before the content in the cache finally "really" expires.

Without a Max-OFE option or similar, a cache can still  implement such behaviour but has to estimate the right time (when to try re-establish an observe relation) itself. If this is correct I'm ok with removing the option for the time being.

regards
Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Zach Shelby
Sent: Wednesday 4 January 2012 10:45
To: [email-masked] WG
Subject: [core] Moving Observe forward

What do others think?

Zach

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: Zach Shelby <[email-masked]>
Date: Fri, 13 Jan 2012 13:39:59 +0200

Hi,

> 2) defining Max-Age and "refresh after time T" options (with T < Max-Age) ; stale content is never served.

Except 2) isn't really useful in the Observe relationship. Someone just threw out that idea during the meeting as a possible Max-OFE alternative. The main tension was just that people really didn't agree with adding the Max-OFE option in the first place. But then again we were told at a previous meeting (Quebec?) to go solve this problem... 

> Without a Max-OFE option or similar, a cache can still  implement such behaviour but has to estimate the right time (when to try re-establish an observe relation) itself. If this is correct I'm ok…

Correct. 

Now, Carsten brings up an interesting idea in re-opening Ticket #174, that these things could be decoupled from each other. Let's explore if that makes any sense. 

Zach

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 16 Jan 2012 10:45:25 +0200

Hi Zach,

I agree with you analysis that the Max-OFE option has been added 
prematurely to the draft
without any discussion in the mailing list before doing it.
Sorry, bu I am not convinced yet of the need for a second parameter; but 
happy and ready to change my mind in the case.
Carsten replied me that it is necessary because both only the server 
will have something to base a statistical calculation on
(e.g. related to the updates frequency) and also is the server that 
intends to act on the observation relationship;
but I don't understand then because all of those can not be reflected in 
the Max-Age value

About the idea present in the reopened ticket #174
an option sent by the server to the client indicating the amount of time 
the time the server
promise not to give up the observation relationship
looks to me quite similar to the "Lifetime" option, even if in this 
proposal is the server to decide the duration
and not anymore the client to require it.

cheers
Salvatore

On 1/13/12 1:39 PM, Zach Shelby wrote:
>> 2) defining Max-Age and "refresh after time T" options (with T<  Max-Age) ; stale content is never served.

-- 
Salvatore Loreto
www.sloreto.com

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Jan 2012 10:19:58 +0100

On Jan 16, 2012, at 09:45, Salvatore Loreto wrote:

> an option sent by the server to the client indicating the amount of time the time the server
> looks to me quite similar to the "Lifetime" option, even if in this proposal is the server to decide the duration

The difference you mentioned plus the difference that a lifetime simple runs out and requires re-establishing the observation relationship while this "promise" can be renewed by the server in its every vital sign.

Grüße, Carsten

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 16 Jan 2012 13:32:11 +0200

Hi Carsten,

yes I see the differences,
  but your proposal still implies some extra effort on the server
even if minor that the one required when we had "Lifetime" option.

However I don't get the need for an explicit signal.

If I am and Observer and I receive back from the server an update with a 
Max-Age
I assume that the server has set the Max-Age to that value, not 
randomly, but because it knows (based on some internal statistics)
that after that that resource value will be different or because it want 
to force me to check with it anyway;
so in the case I, as observer, don't receive anything I can resonable 
assume that I am not subscribe any more to the Server
or something bad has happened to the server.

of course a server can send a notification that does not contain Max-Age 
but that is another problem.

-- 
Salvatore Loreto
www.sloreto.com

---

From: TianLinyi <[email-masked]>
Date: Mon, 16 Jan 2012 11:48:18 +0000

Hi, All

I am wondering why the client wants to extend the period rather than re-subscribe the relationship if it really needs. What is the lost if the client doesn't do so? 

Cheers,
Linyi

________________________________________
发件人: [email-masked] [[email-masked]] 代表 Salvatore Loreto [[email-masked]]
发送时间: 2012年1月16日 16:45
到: [email-masked]
主题: Re: [core] Moving Observe forward

Hi Zach,

cheers
Salvatore

--
Salvatore Loreto
www.sloreto.com

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Jan 2012 13:33:41 +0100

> even if minor that the one required when we had "Lifetime" option.

Exactly -- announcing a value for this option is a promise by there server to go an extra mile.

> If I am and Observer and I receive back from the server an update with a Max-Age
> I assume that the server has set the Max-Age to that value, not randomly, but because it knows (based on some internal statistics)
> that after that that resource value will be different or because it want to force me to check with it anyway;
> so in the case I, as observer, don't receive anything I can resonable assume that I am not subscribe any more to the Server

This is one of those "If all you have is a hammer, everything looks like a nail" situations.

Max-Age tells a client how long the value is valid (and how long a proxy can relay it, with a counted down max-age).
Now, say, we are in an observation relationship.
If no new value comes in at the end of max-age, there may be five reasons:

1) the server is no longer operating
2) the server has rebooted and lost the state
3) the server has been trying to reach the client and is still unsuccessful, as the network is down
4) the server tried to reach the client for a while and has given up
5) the value has not changed.

We can do very little about 1 (if the server no longer is there, the resource no longer is there, so this is unrepairable).  (It would still be nice to have a way to control the time needed for reaching this diagnosis.)

1 is very hard to distinguish from 3.  So let's treat 3 like 1 for the moment.

2 and 4 are the cases that can be repaired by re-establishing the observation relationship.

5 is an important optimization.  Without any of this, the server must send a new value when max-age runs out.  This would be fine in a pull model, but in a push model we can do better: If the observation relationship still exists, the client can continue to use the old value, because it would have received a new one if it indeed had changed.

If we want to pursue the efficiencies of 5, we need to tell the client how long it can defer action for 2/4.
For this, it needs to know how impatient the server is going to be.  For a server that puts observation relationships in stable storage and never gives up, 2 actually cannot happen (but a server might still go into extended exponential back off for extended lengths of 4).

So, for a server, the new proposed hold-off value means "after max-age runs out, I promise not to give up on you for at least this duration".  The client can postpone observation relationship recovery to after that value, because any gap is a relatively reliable indication of 1, 3 (unrepairable), or 5 (no change).  Avoiding 4 is exactly what hold-off is about (Ticket 174 is about "Robust observation relationships").  2 is a place where we can maybe fudge a bit: A server should not give a hold-off if it plans to reboot and forget everything frequently; if that is infrequent (as in "battery change every 3 years"), it might benefit from a bit of a hold-off…  (I'm not sure how to solve this better and still be conservative with packets.)

The remaining issue is, if the client is an intermediary, what does it tell *its* clients.  If the intermediary has an observation relationship with its client, it can provide an appropriate (independent) value for hold-off -- there is no need for its client to become active for observation relationship recovery just because the origin server has a hiccup.  If the ultimate client is just sending GET requests without Observe, things become a bit more thorny.  There is no business for hold-off here.  So this is where your fudge factors ("statistical calculations") might come in again.  Setting max-age to the remaining hold-off obviously is wrong, as a new notification can come into the intermediary every moment.  Setting it to a very small value leads to a lot of polling.  So this may be the place of educated guessing.

Let's go through the two examples from Taipei:

1) The ECB currency exchange reference rate.  This is published by the origin server every work day at 14:15 and carries a max-age of 86400 (or a multiple of that to cope with non-trading days).  This is a big server, so it might well have some patience, but on the other hand wants to get rid of dead clients relatively quickly.  If the server manages to push out all updates within 10 seconds, it should set the hold-off to 10+max-retransmit-delay+network-latency, so, say, 60 seconds.  An intermediary that observes the origin server but does not get an update should not continue to send out the old reference rate to its clients as "current" for much longer.

2) a temperature sensor.  This publishes a new temperature every minute, *if* it has changed more than a threshold from the previous value (noise suppression).  It is also not very likely that the temperature will change dramatically, still, max-age should stay at 60 seconds, as clients polling from an intermediary should get fresh data.  Hold-Off could be something like 10 minutes, which would require the temp sensor to send a value every ~ ten minutes as a keepalive even if the temperature is completely stable and/or cope with eager clients trying to recover the observation relationship.

Makes sense?

> of course a server can send a notification that does not contain Max-Age but that is another problem.

We have set the default value for Max-Age to 60 seconds, so the server would have to expend extra effort to set it to 0.

Grüße, Carsten