Subject: [Netconf] New draft: draft-mm-netconf-time-capability-00

From: Tal Mizrahi <[email-masked]>
Date: Mon, 08 Jul 2013 10:48:18 +0300

Hi,

We have submitted a new draft titled “Time Capability in NETCONF”.
http://tools.ietf.org/html/draft-mm-netconf-time-capability-00

Any comments will be welcome.
Thanks,
Tal Mizrahi.

---

From: Jonathan Hansford <[email-masked]>
Date: Mon, 08 Jul 2013 11:23:11 +0100

Hi Tal, 

Like the idea of this capability. A couple of points on
first reading: 

 	* p3, scheduled time: this is defined as "the time at
which the RPC should be completed", however scheduling (p4) states "the
server ... executes the RPC at the scheduled time". Though many (most?)
RPCs may complete in a short period of time, wouldn't it be more
appropriate to define the scheduled time as "the time at which the RPC
should be executed"? As you state later, there are a number of factors
that can impact on the time to complete. Though some may be predictable,
allowing the server to estimate when to start execution, others will not
be.
 	* p9, 5.2: s/5.1. ,/5.1,
 	* How would you see this working when
supporting the configuration of a set of network elements in a robust
and transaction-oriented way, where the operation should complete on all
devices or be fully reversed?

Jonathan 

On 2013-07-08 08:48, Tal
Mizrahi wrote: 

> Hi,
> 
> We have submitted a new draft titled "Time
Capability in NETCONF".
> Tal Mizrahi.
_______________________________________________
> Netconf mailing list
[email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Mon, 8 Jul 2013 07:27:45 -0700

Hi,

I do not support a general time-scheduled protocol operation mechanism for
all NETCONF operations.  There is one interesting use-case mentioned:
synchronized <commit> (or <edit-config> on :writable-running) operations
across multiple servers.

The proposal doesn't actually work within NETCONF because <rpc> requests
within a single session are serialized.  This solution seems to send 1
<rpc-reply>
when the operation is finally executed.  How does the client know the operation
was scheduled successfully?  IMO, a better solution would be an immediate
<rpc-reply> (scheduled OK) and the execution results sent in a <notification>.

How do clients monitor what operations are pending?

What if the NACM rules change while the operation is pending?
Does access control get checked twice?  Clients should not be able
to schedule operations they are not permitted to execute.

What if a session is lost or closed before its scheduled operation is started?

What if the server reboots while operations are pending?

IMO, YANG date-and-time is better time parameter than 'seconds since 1970'.
It is already supported by NETCONF servers.

How does a client cancel an operation?
Can client A cancel operations for client B,
assuming client A is allowed to invoke <kill-session>?

Andy

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 09 Jul 2013 00:24:24 +0300

Hi Jonathan,

Thanks for your prompt comments.

> allowing the server to estimate when to start execution, others will not

As you point out, while some RPCs complete in a short time, others may  
take a while. So if the RPC simply writes the value of a single leaf,  
its execution time may be negligible compared to the execution  
accuracy. However, if the RPC is, for example a <commit> operation, it  
may take a while. Hence, we tried  to generally be clear and accurate  
in what we mean by “executes the RPC at the scheduled time”. I take  
your note that we need to make sure we are consistent in our phrasing.

> devices or be fully reversed?

The time capability we propose should work well in networks with a  
high degree of reliability, where you can benefit from a coordinated  
update at the (small) cost of (rare) failures. But your point is well  
taken, and similar points have also been raised by Andy in his  
comments from today, so we will consider it further.

Thanks,
Tal.

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 09 Jul 2013 00:30:47 +0300

Hi Andy,

We appreciate your comments.
Most of your comments (although they end with a question mark :) ) are  
issues that have not been addressed in the current draft, and we take  
them as inputs to the next draft.

> IMO, YANG date-and-time is better time parameter than 'seconds since 1970'.

Unfortunately, the IETF does not have a single standard way to  
represent time. The date-and-time format you refer to (RFC 3339) is  
one way to go, but it only allows a 10th of a second resolution  
(correct me if I am missing something), and we are hoping to allow  
significantly finer-grained coordination. The IETF also has other  
methods of representing time; RFC 6374 emphasizes this confusion by  
allowing two different ways to represent time: the NTP time format and  
the PTP time format.
In this draft we propose the PTP time format.

Thanks,
Tal.

---

From: Andy Bierman <[email-masked]>
Date: Mon, 8 Jul 2013 15:38:01 -0700

Hi,

Can you explain why the server returns the execution time for operations
since that doesn't seem to have anything to do with the problem statement
of starting operations at a coordinated time?

Andy

On Mon, Jul 8, 2013 at 2:24 PM, Tal Mizrahi <[email-masked]> wrote:
> As you point out, while some RPCs complete in a short time, others may take
> a while. So if the RPC simply writes the value of a single leaf, its
> execution time may be negligible compared to the execution accuracy.
> However, if the RPC is, for example a <commit> operation, it may take a
> while. Hence, we tried  to generally be clear and accurate in what we mean
> by “executes the RPC at the scheduled time”. I take your note that we need
> to make sure we are consistent in our phrasing.
>
>
> The time capability we propose should work well in networks with a high
> degree of reliability, where you can benefit from a coordinated update at
> the (small) cost of (rare) failures. But your point is well taken, and
> similar points have also been raised by Andy in his comments from today, so
> we will consider it further.
>
>
> Thanks,
> Tal.
>
>
>>> Any
>> comments will be welcome.
>>>
>>> Thanks,
>>> Tal Mizrahi.
>>>
>>>

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 09 Jul 2013 10:06:35 +0300

Hi Andy,

> of starting operations at a coordinated time?

The <execution-time> can provide the client with feedback about the  
actual execution time compared to the scheduled time sent by the  
client (see Figure 3 in the draft). For example, if the server was  
heavily utilized with other tasks and consequently performed the RPC  
half a second later than scheduled, this information may be important  
to the client.
Another benefit of the <execution-time> is that it allows to attach a  
timestamp to the information the server sends to the client (Figure 2  
in the draft). For example, when the client sends a request for  
statistics, it may be useful to attach a timestamp specifying when  
these statistics were captured.
As you point out, the latter benefit has nothing to do with the  
problem statement.

Thanks,
Tal.

---

From: Tal Mizrahi <[email-masked]>
Date: Sun, 14 Jul 2013 16:43:12 +0300

Hello,

We plan to present this draft in Berlin.
Any further comments/questions will be welcome - we will try to  
address the major issues in our presentation.

Thanks,
Tal.

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 18 Jul 2013 14:35:27 +0200

Hello,
- The idea that scheduled time should be the required completion time 
for an operation is unfortunate. That assumes a device can estimate how 
long a complex configuration operation will take. That is already a a 
risky assumption. Even worse some operations might depend on external 
factors like traffic load, I/O related delays, interaction with external 
nodes. It is impossible to estimate these delays.
- because of the above IMO nanosecond precision for timing is not 
realistic. The YANG time types are good enough. Anyway in many protocols 
we have timers on the scale of seconds, so coordinating configuration 
changes between nodes with milli/nanosecond precision does not look like 
a sensible goal.

- there must be a way to check pending scheduled operations. This has 
some interesting access control issues as well. Can I see other peoples 
pending operations? Even if I can read them, what if I do not have 
rights to a managed object included in a pending operation? I should not 
be able to read it.
- there must be a way to remove pending scheduled operations.

- often a configuration session will consist of multiple operations: 
(lock, edit-config, unlock), (lock, discard-changes, edit-config, 
commit, unlock). If I can only schedule single operations does that mean 
I can not use lock? If I want to schedule commit, should I keep the 
running and the candidate configurations locked to ensure I commit, what 
I really want? What if my session is closed e.g. by a network timeout?

- if a management user is removed/undefined will his pending operations 
also be removed, or will they stand there and fail at the scheduled 
execution time? The latter case will confuse people that do not know the 
user was removed.

- generally timed operations work nicely if no one else configures the 
node while my changes are pending. What is the proposed method to ensure 
this?

regards Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
System Manager
ECN: 831 7320                        Tel: +36-1-437-7320
Mobile: +36-70-330-7909              email: [email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Jul 2013 06:56:48 -0700

Hi,

There are lots of operational problems with delayed operations.
I like the solution in draft-kwatsen-conditional-enablement-00
better than this one because it covers more use-cases and
seems more tractable.

For example:
  * in order to stage time-triggered-config,
    the user and access control can be checked once at the time
    the request is made.
  * the normal RPC mechanism is used
    (1 request and 1 immediate reply)
  * the staged config can be viewed or deleted with standard
    protocol operations, using standard access control
    [Not clear what filters would need to be added.]
  * conditional-enablement easily handles periodic config to
    support maintenance windows, URL blocking, etc.
  * time-shift applies to config activation, which is much less
     complicated than delaying arbitrary protocol operations.

Andy

On Thu, Jul 18, 2013 at 5:35 AM, Balazs Lengyel
<[email-masked]> wrote:
> Hello,
> - The idea that scheduled time should be the required completion time for an
> operation is unfortunate. That assumes a device can estimate how long a
> complex configuration operation will take. That is already a a risky
> assumption. Even worse some operations might depend on external factors like
> traffic load, I/O related delays, interaction with external nodes. It is
> impossible to estimate these delays.
> - because of the above IMO nanosecond precision for timing is not realistic.
> The YANG time types are good enough. Anyway in many protocols we have timers
> on the scale of seconds, so coordinating configuration changes between nodes
> with milli/nanosecond precision does not look like a sensible goal.
>
> - there must be a way to check pending scheduled operations. This has some
> interesting access control issues as well. Can I see other peoples pending
> operations? Even if I can read them, what if I do not have rights to a
> managed object included in a pending operation? I should not be able to read
> it.
> - often a configuration session will consist of multiple operations: (lock,
> edit-config, unlock), (lock, discard-changes, edit-config, commit, unlock).
> If I can only schedule single operations does that mean I can not use lock?
> If I want to schedule commit, should I keep the running and the candidate
> configurations locked to ensure I commit, what I really want? What if my
> session is closed e.g. by a network timeout?
>
> - if a management user is removed/undefined will his pending operations also
> be removed, or will they stand there and fail at the scheduled execution
> time? The latter case will confuse people that do not know the user was
> removed.
>
> - generally timed operations work nicely if no one else configures the node
> while my changes are pending. What is the proposed method to ensure this?
>
> regards Balazs
>

---

From: Tal Mizrahi <[email-masked]>
Date: Thu, 18 Jul 2013 17:31:11 +0300

Hi Balazs,

Thanks for reviewing the draft.
I think most of your questions/comments (as well as Andy's) are valid  
questions that we plan to address in the next draft, e.g., mutual  
exclusion issues, managing pending operations, lost/closed session, etc.

> - The idea that scheduled time should be the required completion  
> time for an operation is unfortunate. That assumes a device can  
> estimate how long a complex configuration operation will take. That  
> is already a a risky assumption. Even worse some operations might  
> depend on external factors like traffic load, I/O related delays,  
> interaction with external nodes. It is impossible to estimate these  
> delays.
> realistic. The YANG time types are good enough. Anyway in many  
> protocols we have timers on the scale of seconds, so coordinating  
> configuration changes between nodes with milli/nanosecond precision  
> does not look like a sensible goal.

I agree that in some cases it may be harder to predict the completion  
time of an operation than others. Indeed, a commit operation may be an  
example where your scheduling time will have a lower accuracy.
However, consider a simpler case where you want to do a scheduled  
<edit-config> of a single leaf. In this case you can schedule the  
update fairly accurately.
We are trying to provide a tool that will allow accurate scheduling  
where possible, which is why we are targeting a higher resolution than  
tenth-of-a-second. I agree that we do not need a nanosecond  
resolution, however, it would be best if we chose one of the existing  
time formats already used in the IETF rather than invent a new one,  
and if we indeed agree that we need to target a better resolution than  
a tenth-of-a-second, the PTP/NTP time formats both seem like good  
candidates.

> commit, unlock). If I can only schedule single operations does that  
> mean I can not use lock?

For example, when you invoke a (lock, edit-config, unlock) procedure,  
the <lock> and <unlock> will typically be performed according to the  
'normal' paradigm, i.e., without using the time element, whereas the  
<edit-config> will include the time element, allowing the edit-config  
to be scheduled in a coordinated way.

Thanks,
Tal.

Quoting Balazs Lengyel <[email-masked]>:

> Hello,
> - The idea that scheduled time should be the required completion  
> time for an operation is unfortunate. That assumes a device can  
> estimate how long a complex configuration operation will take. That  
> is already a a risky assumption. Even worse some operations might  
> depend on external factors like traffic load, I/O related delays,  
> interaction with external nodes. It is impossible to estimate these  
> delays.
> realistic. The YANG time types are good enough. Anyway in many  
> protocols we have timers on the scale of seconds, so coordinating  
> configuration changes between nodes with milli/nanosecond precision  
> does not look like a sensible goal.
>
> - there must be a way to check pending scheduled operations. This  
> has some interesting access control issues as well. Can I see other  
> peoples pending operations? Even if I can read them, what if I do  
> not have rights to a managed object included in a pending operation?  
> I should not be able to read it.
> commit, unlock). If I can only schedule single operations does that  
> mean I can not use lock? If I want to schedule commit, should I keep  
> the running and the candidate configurations locked to ensure I  
> commit, what I really want? What if my session is closed e.g. by a  
> network timeout?
>
> - if a management user is removed/undefined will his pending  
> operations also be removed, or will they stand there and fail at the  
> scheduled execution time? The latter case will confuse people that  
> do not know the user was removed.
>
> - generally timed operations work nicely if no one else configures  
> the node while my changes are pending. What is the proposed method  
> to ensure this?
>
> regards Balazs
>

---

From: Tal Mizrahi <[email-masked]>
Date: Thu, 18 Jul 2013 17:48:14 +0300

Hi Andy,

> better than this one because it covers more use-cases and

The concept in our draft is to suggest a general solution, allowing  
servers to coordinate any type of operation, including commit,  
edit-config, or any future operations that may come up. We are also  
targeting a higher resolution than hours.

Your points below are well taken.

Thanks,
Tal.

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Jul 2013 07:59:22 -0700

On Thu, Jul 18, 2013 at 7:48 AM, Tal Mizrahi <[email-masked]> wrote:
> The concept in our draft is to suggest a general solution, allowing servers
> to coordinate any type of operation, including commit, edit-config, or any
> future operations that may come up. We are also targeting a higher
> resolution than hours.
>

I think your solution has use-cases outside of configuration.

  - synchronized actions across multiple servers (e.g., <reset-foo>)
  - synchronized <get> across multiple servers

Andy

>>> realistic.
>>> some
>>> (lock,
>>> also
>>> node

---

From: Andy Bierman <[email-masked]>
Date: Sat, 20 Jul 2013 17:57:18 -0700

> impossible to estimate these delays.

I agree it is better to synch the start-times, not the finish-times.
That is too difficult to predict.

IMO, 1 micro-second resolution is enough, not nano-seconds.
There can be a monitoring leaf specifying the start-time-granularity
supported by the server.

> while my changes are pending. What is the proposed method to ensure this?

How about other timed events like a confirmed-commit timeout?

In general, what if the config being patched via a timed operation assumes
a different running config and system state than what actually exists?

(This is why NETCONF editing operations need to support ETags).

IMO, conditional config is too hard to accomplish via staged edit operations.
It is better to just stage the config itself.

Andy

---

From: Tal Mizrahi <[email-masked]>
Date: Mon, 22 Jul 2013 17:44:52 +0300

Hi Andy,

> There can be a monitoring leaf specifying the start-time-granularity  

I don't disagree. I believe a micro-second resolution is probably  
granular enough.
I am just saying there is an advantage to use an existing time format  
(PTP/NTP) rather than to invent a new one.

> How about other timed events like a confirmed-commit timeout?

Please note the " Confirmed Commit Capability" section in the draft:
    When the time capability is supported, and a confirmed <commit>
    operation is used with the <scheduled-time> element, the confirmation
    timeout MUST be counted from the scheduled time, i.e., the client
    begins the timeout measurement starting at the scheduled time.

Thanks,
Tal.

---

From: Radek Krejčí <[email-masked]>
Date: Mon, 22 Jul 2013 16:53:40 +0200

> That is too difficult to predict.

+1

Even if you do <edit-config> on a single leaf, it can represent a
complex operation with a hardly predictable duration.

Radek

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 23 Jul 2013 09:23:17 +0200

> (PTP/NTP) rather than to invent a new one.
IMHO anything better then millisec precision is unrealistic, unneeded or 
rather just wishfull thinking.
- Have anyone ever seen a configuration operations that were specified 
to this precision?
- It would be extremely fragile to depend on that level of precision. 
AFAIK even in the most tightly controlled networks network delays can 
fluctuate with milliseconds, so requiring microsecond precision seems to 
be unrealistic.
- Do you foresee any real use cases needing that precision?
- How well are the timers of different nodes syncronised today? Do we 
really have 1 microsecond  syncronisation(or will have in the next 10 
years)?
regards Balazs

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 23 Jul 2013 10:57:51 +0300

Hi Balazs,

> IMHO anything better then millisec precision is unrealistic,  
> unneeded or rather just wishfull thinking.

It is not necessarily bad to provision for a higher resolution than is  
currently needed.
(Honestly, does anyone really need 128-bit IP addresses? :) )

> - How well are the timers of different nodes syncronised today? Do  
> we really have 1 microsecond  syncronisation(or will have in the  
> next 10 years)?

NTP allows an accuracy in the order of milliseconds. PTP typically  
allows an accuracy in the order of 1 microsecond, and the accuracy  
requirements of time-aware applications are becoming increasingly  
stringent. So in the next 10 years it is very likely to see network  
devices (switches, routers) that maintain a clock with an accuracy in  
the order of tens of nanoseconds.
Since this level of accuracy is available, it makes sense to provision  
for a high-resolution time field, even if we know *current*  
applications only require a coarser-grained resolution.

Regards,
Tal.

Quoting Balazs Lengyel <[email-masked]>:

>>> There can be a monitoring leaf specifying the  
>>> start-time-granularity supported by the server.
>>
>> I am just saying there is an advantage to use an existing time  
>> format (PTP/NTP) rather than to invent a new one.
>>
> IMHO anything better then millisec precision is unrealistic,  
> unneeded or rather just wishfull thinking.
> - Have anyone ever seen a configuration operations that were  
> specified to this precision?
> - It would be extremely fragile to depend on that level of  
> precision. AFAIK even in the most tightly controlled networks  
> network delays can fluctuate with milliseconds, so requiring  
> microsecond precision seems to be unrealistic.
> - How well are the timers of different nodes syncronised today? Do  
> we really have 1 microsecond  syncronisation(or will have in the  
> next 10 years)?
> regards Balazs
>
> --

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 23 Jul 2013 10:40:45 +0200

On Tue, Jul 23, 2013 at 10:57:51AM +0300, Tal Mizrahi wrote:

> Since this level of accuracy is available, it makes sense to
> provision for a high-resolution time field, even if we know
> *current* applications only require a coarser-grained resolution.

If nobody seriously can implement a certain precision, implementations
will simply cheat and the consequence of that in the long run is often
worse than having a less fine grained precision that is implementable
and thus meaningful.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 23 Jul 2013 12:06:13 +0200

Hello Tal,
Can you give us some examples of applications today that use coordinated 
configuration? What precision is used today?

(I don't seriously object to adding a few extra characters/bits for 
precision. However I still feel it would be a fragile design to depend 
on such high precision coordinated configuration.)
regards Balazs

On 2013-07-23 09:57, Tal Mizrahi wrote:
> Hi Balazs,
>
>> IMHO anything better then millisec precision is unrealistic, unneeded 
>> or rather just wishfull thinking.
>
>> IMHO anything better then millisec precision is unrealistic, unneeded 
>> or rather just wishfull thinking.
>> fluctuate with milliseconds, so requiring microsecond precision seems 
>> to be unrealistic.

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 23 Jul 2013 14:04:48 +0300

Hi Balazs,

> Can you give us some examples of applications today that use  
> coordinated configuration? What precision is used today?

It is a fair question (I am a bit confused about the word *today* in  
your question; since we are proposing a new feature, it is not used  
today, but existing applications can benefit from it in the future).

We plan to present a few use-cases in next week's meeting in Berlin.
In general the time capability addresses two main scenarios:
- Coordinated updates, which allow you to update multiple servers at  
the same time, minimizing the transient effect of any anomalies or  
inconsistencies that may be caused by the transition to the new  
configuration.
- Sequential updates, where the client can schedule a different update  
time at each server, allowing an ordered sequence of updates.

We have also proposed an extension to the OpenFlow protocol that is  
very similar to the proposed time capability in NETCONF. The following  
technical report provides some more details as well as a few  
time-triggered update use-cases:  
http://tx.technion.ac.il/~dew/OFTimeTR.pdf

Regards,
Tal.

Quoting Balazs Lengyel <[email-masked]>:

> Hello Tal,
> Can you give us some examples of applications today that use  
> coordinated configuration? What precision is used today?
>
> precision. However I still feel it would be a fragile design to  
> depend on such high precision coordinated configuration.)
> regards Balazs
>
>> It is not necessarily bad to provision for a higher resolution than  
>> is currently needed.
>> devices (switches, routers) that maintain a clock with an accuracy  
>> in the order of tens of nanoseconds.

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 23 Jul 2013 23:27:09 +0200 (CEST)

Hi,

Tal Mizrahi <[email-masked]> wrote:
> (correct me if I am missing something)

date-and-time (and RFC 3339) allow finer granularity than a 10th of a
second.  In fact, they allow any number of fraction digits of the
second:

   time-secfrac    = "." 1*DIGIT

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 23 Jul 2013 15:38:29 -0700

Hi,

I don't think it will be clear which RFC is the current one.
The problem is that RFC YYYY updates XXXX and does not
obsolete XXXX.  Only the main module is obsolete, but
the submodules may in fact still be current.  (i.e., copy
old include-stmt into new main module).

The WG overhead is not that significant either way.
The deciding factor should be how confusing this will
be to readers over time.

Andy