Subject: OPES Rules Language

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Wed, 8 Jun 2005 14:03:32 -0600

I'm going to give the rules language a shot by trying to author
a draft that merges the best of P and Sieve.  Simple semantics
and rich matching, with control points tailored to the OPES
architecture.  Send your wish lists.

Hilarie

---

From: Markus Hofmann <[email-masked]>
Date: Wed, 08 Jun 2005 17:54:40 -0400

Awesom, Hilarie!

Folks, please submit comments and thoughts to Hilarie - and don't 
forget to copy the mailing list for an open discussion.

Thanks,
   Markus

---

From: "Alex Rousskov" <[email-masked]>
Date: Thu, 09 Jun 2005 08:41:16 -0600

On Wed, 2005/06/08 (MDT), <[email-masked]> wrote:

> and rich matching,

Will you extend Sieve? In other words, will your language be a
Sieve extenstion, following Sieve extension requirements?

Will you make your language specific to HTTP adaptations?
Or will it have modules/libraries/extensions/whatever for
each application protocol being adapted (HTTP, SMTP, etc.)?

> with control points tailored to the OPES architecture.

Do you mean points where adaptation is triggered? For
example pre/post-cache and request/response path in
caching HTTP proxies? If not, please ignore the paragraph
below.

In my limited experience with ICAP configurations, and based
on the "application agnostic core" principle, there may be
no need for explicit control points. The control point can
(should?) be an external to the language matter. The language
does not have to depend on the message processing "point"
where the adaptation is triggered. And IIRC, OPES architecture
only provides examples of possible control points.

> Send your wish lists.

My wish list can be easily derived from the Sieve versus P
comparison I have posted some time ago. I want all the good
features that made the basis for that comparison :-).

Thank you,

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 09 Jun 2005 18:00:59 +0200

General agreement with Alex except on his restriction and added need below.

>only provides examples of possible control points.

Some difficulty to understand how you trigger the adaptation then?
Or we do not speak of the same thing?

>features that made the basis for that comparison :-).

Idem.

Due to the interest of this I would like to use the language to see how to 
amalgamate it in a Context Reference Center command system. This means that 
I will replace every word of the language by a numeric value permitting to 
translate it in any language. The call out server will be a CRC permiting 
to call on common contextual references to document or translate the 
content of a text/page/mail.

So, I am interested that nothing in the language depends on the 
value/structure of the used terms.

I currently use a moke-up in ASCII using Perl like variable. FYI the intent 
is to document the solution for second week of August to support a language 
description ISO (ISO 639-4) standard proposition. Thank you for your help.
jfc

---

From: "Alex Rousskov" <[email-masked]>
Date: Thu, 09 Jun 2005 10:55:41 -0600

> Some difficulty to understand how you trigger the adaptation then?

The proxy/processor configuration will have a mechanism to specify
what rules/code to apply and where to apply them. Different processors
will have diffierent invocation points and different specification
mechanisms (e.g.,access control lists or hard-coded triggers).

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 09 Jun 2005 19:03:08 +0200

>mechanisms (e.g.,access control lists or hard-coded triggers).

This does not make the language universal.
Objection to have both?
Obviously I can add them but that would be bad if the language ends to be 
in an ISO standard what would be great.
jfc

---

From: "Alex Rousskov" <[email-masked]>
Date: Thu, 09 Jun 2005 11:33:24 -0600

> This does not make the language universal.

No language can be universal. Think about it. Somewhere the language
scope ends and the language environment scope begins. For example,
C++ standard does not specify when C++ programs are executed while
shell scripts do not care what language the programs they run were
written in.

The reason to remove invocation points from the rules language
is simple: all existing proxy implementations already have their
configuration language that determines invocation points. Usually,
it comes in a form of an ACL of some sort. Apache, Cisco, NetApp,
sendmail, etc. all have that. Trying to change or replace that
language is fruitless, IMHO.

On the other hand, providing a universal language to describe what
(if anything) happens at the selected invocation point does have a
[slim] chance of being deployed because no popular implementations
(related to HTTP and ICAP) have any good knobs for that. SMTP world
has Sieve and Milter.

> Objection to have both?
> Obviously I can add them but that would be bad if the language ends to  
> be in an ISO standard what would be great.

Sorry, I do not understand this part. Perhaps you can give a specific
ISO requirement you are trying to satisfy?

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 09 Jun 2005 22:24:19 +0200

>(related to HTTP and ICAP) have any good knobs for that. SMTP world

Let me think over that and see the implementation.

>ISO requirement you are trying to satisfy?

We (Internet and the world at large) have a problem with language 
identification. At IETF, W3C and ISO there are roughly three approaches 
which are fiercely disputed because the people supporting the first two 
have not (yet) a network vision and there is a lof of money in the first one.

- one is from publishers point of view (books and programs). They start 
from characters (Unicode), to computer (locales), pages (XML, HTML), to a 
IANA language registration stewardship giving the registrant a commercial 
advantage. That approach uses concepts ("English", "Latin Script", ccTLD 
for country/market, etc.). This is OK to classify items in a cupboard or in 
a directory. This is operational. It should extend from 400 to 7500 
languages. This should be ISO 639-3 and okayed in August.

- another is ISO standard consistent. ISO 12620 defines the data elements, 
ISO 11179 defines the registries. We are talking of precise rules. A script 
is defined by its charset, a language tested by statistics on recurrent 
words, etc. We are no more working on concepts but on values which can be 
used by computers (and OPES to test the language of an unknown 
message/page, and massage it - translating, entering notes, classifying it, 
etc.). The base used in that project includes 20.000 languages. This should 
be ISO 639-6 and big work ahead.

Both use ASCII string IDs for languages (2 or 3 for the first approach, 4 
for the second one). None of them can support multilingualism. The first 
one is deliberately ASCII and English oriented (the table) and wants to 
replace RFC 3066. The second is fully open to multilingualism but work on 
multilingualism has not been carried yet in ISO 11179. It is supported by 
Govs, R&D, etc.

- the last one (my team) starts from the smart-user-in-a-network's point of 
view. We cannot accept the first one only. The second one is OK but with 
some conceptual addition and a lot of wording simplification - this is 
complex if we want to stay fully compatible with metamodels we could have 
to hook. We will give all the language IDs a number (an additional column) 
and we index twice. So we can come from the ID number, or from a string. We 
make the numeric language ID an IPv6 Interface ID and the string language 
ID a domain name. Access is therefore very fast and proven. When we call a 
language (by IPv6 or by name) we call the registry of that language. The 
response we get give us all the registered details of the language (it can 
be an XML page, an ASN.1 structure, or we can add the sub-address of one 
element to get only one word). Addresses can retain the information or to 
be recursive and point to another address if the information is common or 
in case of update. This concept should be documented hopefully in ISO 639-4 
this year or after having been demonstrated next year.

My idea is to say the Sieve/P language is a language to parse a text. Let 
stabilize its commands as a metamodel (XML fields) conforming to the ISO 
11179 standard (it is OK no problem - but careful wording). This means that 
we will have documented a universal way for a user in every language to 
enter a script to work with his texts. This kind of thing is developed in 
lexicons already so the concepts and tools are there. It may take time to 
fill all the entries (actually we need 100 entries for each script and then 
to refine - wikipedia could help a lot): this would be the first 
multilingual computer language.

This means that any user, on any keyboard, could enter an save a script to 
have an OPES working on his language, this being documented in ISO standards.

Now you realise that the OPES will use a CRC (context reference center, the 
system where we will locate the registry) as a call-out server. This CRC 
can belong to a mailing list for example. It can document a vernacular 
version of the language being used (dictionary, ontology, syntax, grammar, 
etc.). For example each time we would copy someone outside of the OPES 
mailing list and we use "OPES", the system could add in the language of the 
e-mail a foot note explaining the destinee what an OPES is and possibly 
correct my Franglish in readable English.

Was I clear enough?

For that to work, I think we need to have a simple enough language neutral 
to ASCII strings (commands and processed text). This is also why at this 
stage I would be happy to keep triggers. There are enough tricky things 
which may happen in language logic (bidi for example) to feel more 
confortable if you have more control.

Obviously if it does not work I will drop the request ... but it would be 
great if it did: for the service provided, for the fun, for the first attempt.
jfc

---

From: "Alex Rousskov" <[email-masked]>
Date: Thu, 09 Jun 2005 14:52:27 -0600

On Thu, 2005/06/09 (MDT), <[email-masked]> wrote:

> For that to work, I think we need to have a simple enough language  
> neutral to ASCII strings (commands and processed text).

If I understand your intent correctly, there are two rather different
problems here: programmer-language (keywords) neutrality and  
content-language
(data on the wire) neutrality. P/Sieve must be neutral to the language of
the data on the wire, of course.

I doubt this group has enough energy and motivation to support the former;
P/Sieve programmers will most likely have to use English keywords. Said
that, it would be a trivial task (from technical point of view) to provide
alternative keyword codings so that non-english programmers feel more
confortable. For some reason, popular programming language standards do not
offer that kind of flexibility (AFAIK).

Alex.

---

From: jfcm <[email-masked]>
Date: Fri, 10 Jun 2005 03:12:02 +0200

At 22:52 09/06/2005, Alex Rousskov wrote:
>content-language (data on the wire) neutrality. P/Sieve must be neutral to 
>the language of

Yeap. This is the interest.

>offer that kind of flexibility (AFAIK).

Some do as C (you define functions, etc.) for ASCII version. But you have 
Arabic, Korean, Chinese development environments. Here the interest is to 
possibly support 20.000 languages, if there is someone interested in each 
and a way to write it :-) (far far less - but there is no reason why this 
does not increase a lot with the current trend).

But the requirement is simple: keywords must be neutral to each other. 
What, I think, is no problem. This only means no fancy idea like a keyword 
being designed to match the begining of another keyword.

The way it should work is as follows:

(a) P/Sieve ASCII keyword ---> (b) numeric-universal-tag (unitag) ---> (c) 
P/Sieve Cyrillic Keyword.

(a) is normal effort with the small provision above and an optional trigger 
keyword.
(b) is a simple incrementation in the unitag system (will be documented in 
a Draft soon) with the ASCII keyword description in English and French.
(c) made by Russian developers. togetgher with the Russian description and 
How To.

jfc

---

From: Tony Finch <[email-masked]>
Date: Fri, 10 Jun 2005 11:52:58 +0100

> offer that kind of flexibility (AFAIK).

Natural language translations of programming language keywords are
generally a bad idea. It bloats the programming language specification and
reduces the ability of programmers from different countries to understand
each others' work. It's better to treat programming languages like
mathematical notation, with a formal intrinsic meaning which is only
suggested by the choice of keyword. Programmers are used to this, so it
isn't a problem.

The question of names defined by programmers (as opposed to keywords
defined by the language specification) is much simpler. If the language
source character set is specified to be unicode, then programmers can
use whatever natural language they like when choosing names. This is
standard practice these days.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: WEST 5 OR 6 BECOMING VARIABLE 3 OR 4. SHOWERS AT FIRST. MODERATE OR
GOOD.

---

From: "Alex Rousskov" <[email-masked]>
Date: Fri, 10 Jun 2005 08:53:23 -0600

On Fri, 2005/06/10 (MDT), <[email-masked]> wrote:

> each others' work.

I doubt those are the reasons. Specification bloat is a minor technical
problem because the core spec remains the same and only more keyword
tables are added as appendices of sorts. The understanding is not
a big problem for keywords because each programmer can view the program
in the language of their choice (automated translation is easy).

> It's better to treat programming languages like

I agree, but many people are annoyed by the fact that the notation is
using a foreign (to them) language, especially if it is a language of
the country they tend to hate. It is not a _technical_ problem these
days (it used to be when some EU folks could not type certain symbols
like "{" and "}" because of 7 bit encoding conflicts).

In true math, it is a lesser problem because notation is mostly symbolic
or using letters from alphabets that do not belong to current
"world-dominating" cultures. Also, mathematicians tend to be more
reasonable than script kiddies :-).

> standard practice these days.

This practice _does_ make it difficult for programmers to understand
"foreign" programs though (and no automated seamless translation is
possible for non-keywords). Said that, I think that P/Sieve should be
Unicode-based. I do not remember if Sieve allows Unicode (it was not
a huge problem for them because they did not have variables in the
core of the spec; they will now).

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Fri, 10 Jun 2005 10:00:50 -0600

This discussion seems silly.  For most languages based on a phonetic
character set, it is a straightforward, almost mechanical task to
change the lexical analyzer to accept a different character set and to
recognize different keywords.  It won't make it much easier to
understand a program written in a foreign country, though, because
even if you could backtranslate "Perl in French" to "Perl in Italian",
the variable names, when transliterated, wouldn't make much sense (and
there's the difficult problem of definining "transliteration" across
all character sets).  I don't know if all non-alphabetic human
languages can support programming languages easily; it is an
interesting question, but far out of scope.

So, my take on this is that it has nothing to do with OPES.  OPES
rules will need to support variables and matching from generalized
character sets, but that's as much of a requirement as makes sense
at this stage.

Hilarie