Subject: Re: [IPTEL] supporting CPL versions

From: Jonathan Rosenberg <[email-masked]>
Date: Fri, 01 Sep 2000 03:29:06 -0400

I want to come to closure ASAP on this time-switch issue.

James Undery wrote:
> 
> Jonathan Rosenberg wrote:
> 
> > James Undery wrote:
> > >
> > > > I understand the desire to be compatible with iCal, but I fear that no
> > > > one will implement this thing completely, and even fewer folks will
> > > > implement it correctly. Is there any way to simplify and retain some
> > > > level of interoperability with ical? What if we keep the time and
> > > > duration formats, and allow for repition, but eliminate the "by*"
> > > > parameters which is most of the nightmare of this thing?
> > > >
> > >
> > > >From a quick first glance this would appear to be a good solution, however, I am
> > > less convinced about simplifying recurrence this way, principally because I like
> > > the idea of specifying the last concepts like "the last friday of the month". I
> > > also understand that compatibility with iCalendar may be nice from a philosophical
> > > viewpoint, but, as a CPL implementor it is too complex for my needs and I would
> > > prefer to return to something closer to the syntax of draft 01 for time nodes.
> >
> > I agree, although I'll note that this syntax does not, AFAIK, allow you
> > to say things like "the last Friday of every month".
> 
> >From what I can recall there was a suggestion in the "implementation notes" about using
> negative numbers in a similar way to the "by*" parameters in version 02.
> 
> >
> > We could always add support for iCal style time switches in the next
> > revision of CPL, which could then support both.
> 
> This would be nice as I'd like first CPL RFC to be a simple as it need be and then
> problems with backward compatibility in future versions would be minimised. (When CPL is
> in use and we have a better idea of how it is to be used in practise.)

So, is there consensus around this? We keep the time-switch from the
older drafts, and then think about defining the iCal one for the next
CPL rev once we understand its usage better? Any objections?

-Jonathan R.

-- 
Jonathan D. Rosenberg                       72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.cs.columbia.edu/~jdrosen         PHONE: (732) 741-7244
http://www.dynamicsoft.com

_______________________________________________
IPTEL mailing list
[email-masked]
http://lists.bell-labs.com/mailman/listinfo/iptel

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 01 Sep 2000 09:11:58 -0400

> CPL rev once we understand its usage better? Any objections?

I'm *very* uncomfortable with this. At the very least, we need to define
how CPL can be reasonably extended in the future. Also, if we go that
route, I would suggest going for real simplicity, i.e., fixed time range
only, as a proper subset of the iCal specification. That will make for a
much cleaner transition strategy rather than mixing two models. That
way, we have an overall implementation that's simpler.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Sun, 03 Sep 2000 01:23:16 -0400

> way, we have an overall implementation that's simpler.

I have no problem using a subset of iCal; namely the start and stop
times, duration, and repitition stuff. The complexity is all in the byX
attributes. I wouldn't be so concerned about doing these if someone
could show me some pseudocode about how these might be implemented in a
reasonable way. If the pseudocode for this is many pages, then I
definitely need to question whether its worth it...

Regarding extensibility:

Ignoreing unknown attributes and tags (but parsing the content of
unknown tags) seems a reasonable thing for extensibility, but has the
unfortunate side effect of eliminating the possibility of DTD
validation. Is there any way to build extensibility in without ruling
out validation of a script against the DTD in the baseline CPL
specification? 

-Jonathan R.

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 03 Sep 2000 09:07:39 -0400

> definitely need to question whether its worth it...

I think a subset makes sense for now.

> out validation of a script against the DTD in the baseline CPL

I don't like just ignoring attributes. That's why I suggested that, in
addition, CPL itself has the ability to switch on extensions, so that
one can implement default behavior if a particular feature is not
supported. 

Outside of making extensions just parameter values, it will be hard to
anticipate DTDs. I'm also not sure it matters. If the creator can
generate new features, it presumably can validate them. If the server
doesn't understand the feature, it doesn't much matter whether it is
syntactically correct. The only borderline case is that a new feature is
really just a mis-spelling of an old one. Having the switch around the
feature would tell the server whether it should go checking, but
unfortunately, I suspect that no XML checker would be smart enough for
that. Namespaces?

---

From: Jiri Kuthan <[email-masked]>
Date: Sun, 03 Sep 2000 20:40:30 +0200

Henning Schulzrinne wrote:
> At the very least, we need to define
> how CPL can be reasonably extended in the future. 

I second this. Extensibility is needed since both signaling
and signaling processing are likely to evolve further.
CPL's extensibility concept is not documented right now.
(There is only a recommendation in Section 12.1 of RFC2824
to inform users of unknown features included in 
scripts.)

In particular, we need to document how CPL interpreters 
deal with unknown elements (UE) in CPL scripts. Two possible 
(RFC-2824-conformant) strategies I see are: 
1)  conservative
    '- on installation: reject scripts that include unknown elements (UE)'
2) liberal
    '- on installation: accept scripts with UEs, issue warnings;
     - in run-time: if UEs encountered, skip them and pass control
       to UE handlers embedded in the CPL scripts'

My preference is 2.

Jiri

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 04 Sep 2000 00:47:43 -0400

> I think a subset makes sense for now.

OK, so the trick is to define exactly which subset. My initial proposal
is:

Parameters:  dtstart      Start of interval (RFC 2445 DATE-TIME)
                dtend        End of interval (RFC 2445 DATE-TIME)
                duration     Length of interval (RFC 2445 DURATION)
                freq         Frequency of recurrence (one of "secondly",
                             "minutely", "hourly", "daily", "weekly",
                             "monthly", or "yearly")
                interval     How often the recurrence repeats
                until        Bound of recurrence (RFC 2445 DATE-TIME)
                count        Number of occurences of recurrence

I wouldn't even object to adding the byX attributes under the condition
of "contraction", that is, the unit in the by component is always a
greater interval than the freq component. These cases are easier to
handle, it seems. You compute the start time of the next interval as if
there were bo byX parameter, and then check if this start time is within
the interval indicated by the byX parameter.

What do people think?

> unfortunately, I suspect that no XML checker would be smart enough for

THe problem isn't restricted to mis-spellings. Here is what I am talking
about. If I have some extension foo, and I use your approach of having a
tag which indicates what to do if the extension isn't supported:

<if understand="foo">
 <foo bar="baz"/>
</if>

This little CPL won't be accepted by baseline CPLs, since they won't
know about foo in the DTD and thus the script is immediately rejected.
The only way I know of to turn off parsing for subcompnents is via
CDATA:

<if understand="foo">
 <![CDATA[
  <foo bar="baz"/>
 ]]>
</if>

Parsers that do understand foo could conceivably extract the content of
the CDATA and then re-validate. A bit of a pain, but it would work.
Nesting of if tags is a bit annoying, since the XML parser will pick up
the first ]] and not the last, as you would want. Not sure what to do
about that...

-Jonathan R.

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 04 Sep 2000 12:45:25 -0400

Jonathan Rosenberg wrote:

> the interval indicated by the byX parameter.

Seems like a good idea. We should probably check whether this covers the
common recurrence cases such as

- Tu and Th, some time range
- first Monday of the month (as in Labor Day...)

> the first ]] and not the last, as you would want. Not sure what to do

This seems ugly. Would namespaces help?

-- 
Henning Schulzrinne   http://www.cs.columbia.edu/~hgs

---

From: James Undery <[email-masked]>
Date: Tue, 05 Sep 2000 11:16:20 +0100

> > greater interval than the freq component. These cases are easier to

This is a bit late in the day to have a change of heart, but I've just
actually thought about implementing this efficiently, and I can't see what the
fuss was.

Firstly check all the byX's are satisfied.
Secondly find if the if the time lies in a repetition of the time period.

Jonathan, correctly state if there is repetition the second step is harder,
however, this can be calculated in constant time with the correct time
representation (depends on the freq, yearly and monthly RFC 2445 DATE-TIME is
good, all other seconds are fine), then normalise your timeframe on dtstart
and use modulo arithmetic to check if you are in a repetition. I assume
calculating the bounds of occurences (if they exist) cause no problem for
anyone.

There is a mistake in the spec though "No parameters other than dtstart, dtend
and duration SHOULD be specified unless freq is present." The byX parameters
need not be excluded as they make perfect sense in a single period. (Also I
removed a surplus comma after dtend)

> - Tu and Th, some time range

<time dtstart="20000805T090000" duration="P5W" byday="TU,TH">

> - first Monday of the month (as in Labor Day...)

<time dtstart="20000805T090000" duration="P52W" byday="1MO">

James Undery

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 05 Sep 2000 11:13:04 -0400

> Secondly find if the if the time lies in a repetition of the time period.

This is true for byXs that represent contraction, that is, elimination
of some of the 
allowed repittion periods. Thats why I am proposing this.

> calculating the bounds of occurences (if they exist) cause no problem for

This doesn't cover the expansion cases, whereby you can have occurrences
more frequently than the recurrence interval.

Also, you can't use real modulo arithmetic, because date addition is too
convoluted. The only way to do this, as I see it, is to start with
dtstart, and add the recurrence interval using calendar addition
functions (the java Calendar class seems to do most of this for you,
thank goodness). THen, add the duration to that, and check if the
current time is between the two. If the current time is after the end of
the event, add another recurrence interval (keeping track of bounds of
recurrences). Separately, before doing this, you would check to see if
the byX relations are satisfied.

> removed a surplus comma after dtend)

I don't think so. byX only make sense for repitions. It would be
pointless to define a single event (via dtstart, dtend, and duration),
and then eliminate it with a byX.

> <time dtstart="20000805T090000" duration="P5W" byday="TU,TH">

Ahh. Now I see why you wanted to allow byX without freq. You want to
express an event as *really* long, and then eliminate blocks of its time
with byX. Thats not how it works. byX modifies when the *beginning* of
an event occurs, it does not block out certain times during the event.
So, the correct way to express this is:

<time dtstart="20000805T090000" duration="P1H" freq="daily"
byday="TU,TH" count="104">

This would specify a one hour window from 9am to 10am, every tuesday and
thursday, for a year. 

> <time dtstart="20000805T090000" duration="P52W" byday="1MO">

Same as above. The correct way is:

<time dtstart="20000805T090000" duration="P1H" freq="monthly"
byday="1MO" count="100">

This, unfortunately, represents the expansion case, and its one of the
things I can't figure out how to handle. The problem is that you can't
really use the month repitition any more. The rule specifies the first
monday of the month, 9am-10am. What confuses me is that the monthly
repititons on the dtstart may not even fall on Monday; so, in this case,
do you ignore those repititions? Its like byX is specifying an alternate
repition mechanism, and I can't figure out how it interacts with the
regular freq mechanism.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 05 Sep 2000 11:18:08 -0400

> This seems ugly. Would namespaces help?

I don't think so. Unfortunately, I believe they suffer from the same DTD
validation problems - i.e., I don't know how to do DTD validation on a
document with elements from external namespaces. This may just be my
ignorance. Any XML namespace experts care to comment?

-Jonathan R.

---

From: Jiri Kuthan <[email-masked]>
Date: Tue, 05 Sep 2000 22:46:55 +0200

> know about foo in the DTD and thus the script is immediately rejected.

Jonathan,

is insisting on DTD conformance so useful if it prohibits extensibility?
How do you plan to deal with extensions?

Jiri

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 05 Sep 2000 18:05:21 -0400

> How do you plan to deal with extensions?

IMHO, XML without DTD validation is useless. DTD validation ensures that
all other kinds of things are correct with CPL, things I would have to
check manually without validation, effectively writing my own validating
parser.

-Jonathan R.

---

From: Jonathan Lennox <[email-masked]>
Date: Tue, 5 Sep 2000 19:10:53 -0400 (EDT)

> ignorance. Any XML namespace experts care to comment?

The nature of DTD validation is that a DTD defines a fixed content model,
and nothing more.   It doesn't (as far as I can tell) have room for
extensibility within this framework.

As such, I think we have three conflicting goals:

* Extensibility
* Backward compatibility for extensions
* DTD validation

It's not clear all of these can be satisfied at once (at least not without
doing something like the CDATA suggestion, which I think is truly ugly).

With some minor effort, it would be possible, I think, to define a wrapper
around DTDs in order to support only *those namespaces you understand* --
for example, it seems like this would be possible for SAX applications by
using the Namespace SAX Filter <http://home.ccil.org/~cowan/XML/>.  However,
this wouldn't allow us to do, for example:

<extension ns="http://www.example.com/cpl/our-first-weird-extension">
   <supported>
      <color-switch xmlns="http://www.example.com/cpl/our-first-weird-extension">
        <color is="blue">
            ... 
        </color>
        <color is="green">
            ...
        </color>
        <otherwise>
            ...
        </otherwise>
      </color-switch>
   </supported>
   <unsupported>
      <mail url="mailto:[email-masked]?subject=Server%20is%20colorblind!" />
   </unsupported>
</extension>

Is this needed?  Obviously, if this sort of thing is present, no
syntactic-level DTD validation is going to be possible.  You're going to
need to use a non-validating XML parser, and then do your validation
entirely at the semantic level.  It's not entirely clear to me how much this
is useful, though.

As a point of reference, the Sieve people considered this sort of
extensibility mechanism, but decided instead to use a "you must declare all
extensions you are using.  A server rejects all scripts requesting unknown
extensions."  This was the justification given for this (Ned Freed
<[email-masked]>, 25 Jan 1998):

        The problem lies in how "support" affects the way the language is
        used. People will use the presence of "support" to justify all sorts
        of extensions. These extensions will then be used willy-nilly, and
        they will be used *without* support tests being made. We will then
        end up with massive interoperability problems.

        You are recreating the PostScript situation here. Nothing more and
        nothing less. 

As such, I would suggest that CPL extensions use XML namespaces, and we
apply the rule "if the script requests an unknown namespace, it must be
rejected".

Given this, I *think* that it would be possible to define a custom DTD given
something like a namespace filter which would allow you to validate all
the CPL extensions you undertand.  I'm almost positive it would be
possible to define an XML schema which would do this, since XML schemas have
been defined post-namespaces.  (Retrofitting namespaces onto DTDs is ugly at
best, and impossible at worst.)

Useful information about XML namespaces is available at:
<http://www.informatik.tu-darmstadt.de/DVS1/staff/bourret/xml/NamespacesFAQ.htm>

-- 
Jonathan Lennox
[email-masked]

---

From: James Undery <[email-masked]>
Date: Wed, 06 Sep 2000 08:38:30 +0100

> allowed repittion periods. Thats why I am proposing this.

Yes I've lost the plot here, I just read what I thought should be there. i.e. the
old cron style definitions.

> repition mechanism, and I can't figure out how it interacts with the

Having actually read the spec rather than getting bored after the parameters are
defined, I have to reassert my previous suggestion of ditching iCalender and
reverting to the previous spec (assuming I read that one too). They were much more
readable, had no less power and didn't have the complexity. Provided a sensible
method for versioning CPL I can see no reason for including iCalendar until a need
and use is demonstrated.

However, the versioning issue really comes from the use of XML rather than defining
a true scripting language. I'd be tempted to retain the syntax, yet not specify the
language as XML specifically, so versioning can be introduced as a more natural
rule.

James Undery

---

From: James Undery <[email-masked]>
Date: Wed, 06 Sep 2000 08:46:54 +0100

> check manually without validation, effectively writing my own validating

Writting a parser for CPL is a trival problem and most of the checks suggested
and required by the draft mean a fair quantity of checking needs to be done
manually. The real complexity in an XML parser is checking conformity to a
schema, yet with CPL its the semantics of the script that are important not
the syntax of the schema. You can check the script conforms to a schema but
that doesn't mean you can do anything with it or that the script really makes
sense.

James Undery

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 05 Sep 2000 20:19:15 -0400

Jonathan Lennox wrote:

> entirely at the semantic level.  It's not entirely clear to me how much this

The problem with not having the "if-supported" switches is that the
client effectively has to keep, in an extreme case, 2^N versions of the
CPL script, if N is the number of extensions it would like to use. This
becomes unmanageable pretty quickly unless the client has some kind of
internal meta-representation that allows it to dynamically create a
version of the script that only uses the desired extensions. This may
not be so bad, as in 

#ifdef feature17
<feature17>
  fancy stuff here
</feature17>
#elseif
basic call handling here
#endif

I suppose one could even let the server do this "preprocessing" if one
were sufficiently aesthetically challenged, by defining a new media type
"CPL-with-C-preprocessor".

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 07 Sep 2000 01:47:52 -0400

> that doesn't mean you can do anything with it or that the script really makes

Its not just me not wanting to do the work; there are lots of xml tools
getting written and being distributed, many of which are strongly based
on usage of the DTD. For example, XML SPy is a nice tool for editing
XML. If you have a DTD, it does "tab style" completions of tag names and
attributes as you type in the document. Very nice for quickly building
valid CPL scripts. I'd like to be able to usefully use such tools with
CPL.

-Jonathan R.

---

From: James Undery <[email-masked]>
Date: Thu, 07 Sep 2000 11:36:50 +0100

> valid CPL scripts. I'd like to be able to usefully use such tools with

If we are going to continue with DTDs (and XML schemas) I'd have a look at this
chapter from "Learning XML" which seems to suggest grim choices when it comes to
conditionals in XML.

Section 5.2.4.4
http://world.std.com/~eray/book/ch05_02.htm

Any I agree whole heartedly writting CPL in emacs using the compeletions is a
pleasure compared to handcoding a script, my testscripts are generated
programatically ready for a nice GUI where drag and drop creation of the scripts
means users will never need to see the syntax of their scripts. Having looked at the
source of loads of webpages, it's obvious few people handcode, the closest they may
get is fine tuning pre-generated scripts. The point I am trying to make is by
limiting ourselves to the strict bounds of XML we are making life difficult for
future expansion. I am not suggesting ditching the current syntax just defining it
using a different method, so we don't require preprocessors or SGML hacks to provide
extensibility.

James Undery