Subject: RE: some (well a lot ) review remarks
From: Anton Okmianski <[email-masked]>
Date: Thu, 1 Apr 2004 14:30:23 -0500

Albert, Rainer, WG:

I have not thought through all the details, but...

I share Albert's concern about defining message length limits and
mandating partitioning in syslog-protocol given that it is supposed to
be transport independent.  I'd prefer if syslog-protocol was just
about the format and encoding of the message payload. Payload being a
complete syslog message without any transport-specific parts.

As it stands now, syslog-transport is really about transferring syslog
message segments and not complete syslog messages.  If
syslog-transport does all segmentation, then it is easier to specify
how to avoid issues such as double-segmentation (syslog-protocol level
and then IP fragmentation).

The relay provisions should probably also be specified only as far as
payload is concerned. For example in DHCP, when message goes through
relay, relay agent is allowed to insert an option specifying its
address as well as giaddr (the network from which request for IP
originated from).  So, it just specifies additional provisions for
payload for cases with relay agents and that's it. I think it will be
easiest if we allowed relay agent to function like any
syslog-transport client and re-assemble the whole message before doing
anything with it. In other words, it should work with syslog messages,
not syslog message segments.

Just my 2 cents.

Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of
> ons-huis.net!ALbert
> Sent: Tuesday, March 30, 2004 11:59 AM
> To: [email-masked]; [email-masked]
> Subject: some (well a lot ) review remarks
>
>
> Hello Rainer, Anton, WG
>
> Last weeks I have reviewed some of the current draft
> documents. I had planned to spend more time on it, coming
> month. But, plans are changed. It will be very busy, which is
> good .. But also means less tome for syslog.
>
>
> I have one important "issue": I think it would be a great
> improvement if some parts of Rainer's document would move to
> Anton's. I think the protocol document should NOT specify how
> to break messages in parts. It should assume the transport
> layer can "transport" 'long messages'. The transport layer,
> should when a long message can't be send in once, spilt in is
> several parts.
>
> I think it should be possible to write another 'transport'
> document, which describes how 'long, new, better' messages
> can be transported by rfc-3164 syslog messages.
> Note: I'm NOT saying we should make an RFC for this. Only
> that, if we could write it, the separation between transport
> and  "upper" layers are better. Note 2: That document should
> describe how to 'shrink' the longer headers into the old
> headers, etc. Not by putting the complete message into the payload.
>
> --------------
> Aside from the point above, I will include a raider long list of
> (personal) notes 'asis'.
>
> Rainer, please use those notes when relevant, Skip the
> others. I'm sorry, I don't have more time to rewrite my notes
> to a good review. It is either me to throw away them, of give
> another that option.:-)
>
> Hope it helps a bit.
>
> ===========================================================
>
> --
> Groetjes,
> --ALbert Mietus
> Private mail to:           albert at ons-huis dot net
> Business mail to:     albert dot mietus at PTS dot nl
> Spam:   Just don't do it! Thrust me, I will not order!
>
> Hello WG,
>
> This is my comment on Rainer's syslog-protcol draft 04. I
> have been very passive on following this mailinglist; (to
> busy:-) But on receiving a request to review I decided to do
> so. But I apologise if I raise issues that are discussed
> already. I didn't read them. This week I started, after a log
> time, to study the current draft document(s).
>
> Before continuing, let me say I like the idea of splitting
> syslog in several layers. I say so, in the hope the
> individual sub-specifications will become short. My comment
> is split into 3 parts, of which you are reading part 1 now.
> Part 1 is about the "idea" (the design). In a separate mail I
> will comment on the text of the document; in the hope I will
> become clearer, cleaner and shorter. The 3rd mail will
> contain some details and bit that didn't fit in the others.
>
> Possible, I will send more comment in futures mail; it will
> depend on the amount of time I have now, and then.
>
> -------------------------------------------------------
>
> This review is becoming (very) long, I really hope all
> comments are worth reading. I have tried to express my
> thoughts about is a well as possible, given the time I have...
>
> General Comment on the idea/design of syslog-protocol
> =====================================================
>
> * This idea is GOOD!
>
>
>
> H2 Architecture
> ================
>
> Traditional, syslog knows Devices, Collectors and Relays.
> I would like to add two 'things'. One I would like to call a
> *Generator* , the other an *Runner*.
>
> *Generator*
> As we can see in e.g. most Unix implementations, it is the
> application that knows WHAT to log, transmit that to the
> system (the log device, the syslogdaemon) that know HOW to
> log it. Also on embedded systems, this can be seen.
> Historically, the combination of (a part of the) application
> and the "system" form the log-device. I would like to split
> this, now this opportunity exits. The part that is build-in
> in the application, (in C: the lines syslog(..."hai there");)
> can be called  the (LOG-)Generator. The communication between
> generator and Device is system/platform depended. On Unix
> systems usually the log-device, on embedded systems a
> function-call, and on windows log-events can be used.
>
> By introducing this Generator, we (can) make clear this
> private/dedicated communication exist; and is allowed. We
> also make clear the Generator is syslog-protocol INDEPENDED.
>
> The function of the Device, know becomes clear: it get
> log-message (-events) and transport them. It also does some
> bookkeeping, like timestamping, adding crypto (for -sign), etc.
>
> *Runner*
> A LOG-runner is a other kind of syslog-thing, which is
> frequently used. Without a proper place in the architecture.
> Whereas a relay (should) forwards syslog-messages without
> knowledge of the semantics of the message, the Runner does.
> The most simple life-form of a runner is a filter. It
> "relays" messages, but only when the are important. On Unix
> there a several of these in perl, grep-scripts  etc.
> Formally, the are knot relays (I think). A more complex
> runner, is a "program" that receives log-messages, CHANGES
> them, and send (or stores) the result. Examples: statically
> analyses, Intrusion detection, etc.
>
> Both kind of Runners are useful, frequently used, but the not
> part of the architecture. And as we try to make syslog
> "better", we better add them and make sure out standards can
> "deal" with them. Otherwise, non-RFC compliance log-programs
> will be standard.
>
>
> H4 Syslog format
> ================
>
> 412 enterpriseID
> -----------------
>
> I don't see any reason to include an enterpriseID; not into
> the header. (When needed, it can be used in the structured MSG part)
>
> Currently, it is just a number. It will be unused, misused or
> will lead to a lot of (operational) management. I'm afraid
> for the latter, as in H4.1.3 is suggested  that the semantic
> of the Facility can be enterpriceID depended.
>
> Also, it is required to use the "IANA assigned vendor"
> number. This implies open-source/free/non-commercial are
> 'ruled out' as the often will not to so.
>
> Last, should the number of the Generator-vendor, the
> system-vendor of the device-vendor be used? (See above about
> generator/device) This is not clear to me. And whatever one
> is chosen, it will be hard to implement. Not using the
> defacto (Unix) logging api's!
>
> 413 Facility
> -------------
>
> Although, at first sight, liked the idea of "a terrible lot"
> of facilities. The current <used a number idea> is wrong, as
> I see it. Aside from the problems mentioned above, more then
> a million facilities will mean relays can't be managed! The
> set of facilities, which will be seen in a (major) network,
> expressed as numbers, will be are more or less at random.
> Which implies very long complex and unmanageable configfiles
> (or MIB's) for each LOG-router!
>
> As we heave learn form routing IPv4, hierarchical structuring
> is needed.
>
> I think, extending the set of facilities is good. But I can't
> imagine more the say 1000 are ever needed.
>
> So, my counter-proposition is:
>         *  Make  facilities (as a number) structured
>         *  Limit the number of facilities to a manageable number
>         *  Keep the format such that extending the allowed
> numbers is possible
>
> A Facility then is still a number, at least 3 (or 4) digits
> long. A longer number means the it is an extended facility.
> They have to be assign by IANA. Facilities of length 3 (4)
> MUST have the format '(K)KLM', where 'K' (or 'KK') indicated
> the kind of facility; 'L' give a sub indication and 'M' is
> *SITE* configurable (so, by the local sysadmin, see example
> below). The  'K/KK' is based on the RFC3164 facilities, clean
> up and extended. Those numbers can be IANA assigned. L can be
> chosen by the (generator) vendor, and `M' by the admin. 'M'
> defaults to 0 (zero), and  applications/vendors MAY give the
> possibility to set that digit.
>
> Example: For mail, there will be an K specified, let say 1.
> Then all mail-log will have the format '1XY', which is easily
> routable. It will do for small sites. Some vendors, like
> "sendmail" (only 1 process) will probably use only one value
> for L e.g. '0'; others, like "postfix" (several processes)
> can use multiple values, like '1', '2' and '3'. When
> supported by both sendmail and postfix, the local sysadmin
> can add (change) the M-digit, such that mail-systems on the
> border, and internal ones use another facility.
>
> In all cases, the local sysadmin can either use simple
> routing-rules, like 1** (for all mail), or 10* for  sendmail
> and 1[1-2]*, 13* for postfix, or even more complex. Now, the
> sysadmin has a choice, and can keep it manageable.
>
> Note: the 0 for sendmail and 1-4 for postfix are "by
> example". However, we can add a "rule" that '0' shall be used
> when only 1 L-value is used, and when several values are
> used, zero should be skipped. Also, I would like to
> prescribe/reserve "9" for local additions. (on all digits).
>
> The (K)KLM idea is used a suggestion to improve, IF this idea
> is accepted, THEN we can discuss variations like 3 or 4
> digits, where to save mappings (IANA of this rfc), etc.
>
> 5 Structured data
> ==================
>
> In short: I thing having the option of structured data is a
> nice option. But lets keep it simple.
>
> The current one is to complex, it has to be as we need 4
> pages to describe it. Also, I find those pages hard to study
> (given the time I had :-). Also it can lead to not
> implementing it. Programmers, especially there bosses, don't
> have a lot of time!
>
> More positive: The main reason why structured data is complex
> (currently) comes down into 1 problems:
> 1) It isn't part of the main-design (the ABNF on page 9)
> 2) The "structure" can start anywhere in MSG
>
> Both are easy to solve:
> Ad 2) Specify that the structured part ALWAYS START directly
> after the header. Ad 1) We need to introduce it where it
> belongs. In an optional field on page 9
>
>
> Let give it a try ( I also use the "improvement" on the ABNF
> of my other mail; it saves typing) (Also I "forget" the SP
> parts, for now. Just the idea)
>
> SYSLOG-MSG  = HEADER DATA
> HEADER      = VERSIONING PRIO ID         // See other mail
> DATA        = [ *STR-DATA ] MSG
> STR-DATA    = see below
> MSG         = free format
>
> Given this ABNF, the structured data ALWAYS comes (in RFC3164
> notation) at the start of the MSG-part (of in new ABNF:
> before the free format MSG).
>
> This implies receivers always have (as last resort) the
> option to see everything after the header as free-format. And
> just store/forward it. It implies the start of STR-DATA is
> simple to find: Its starts directly after the header, or
> directly after another STR-DATA
>
> This implies to, we have the option to start STR-DATA with
> '<' which is more usable and XML-alike. The complex long
> "[@#" cookie isn't needed anymore. However, we free to use
> it. My personal vote is for the (XML) < > style.
>
> See also my other posting about details of structured-data.
>
> 6 Multi-Part Messages
> =====================
>
> There are some mistakes in the this part, but I like the
> general idea. However, I fell spliting/reassembly is done in
> other protocols too. Maybe we can use/reference a (de facto)
> standard? I don't know an RFC which we can use, but I'm sure
> there must be one!
>
> Second, it to complex, and to long (to read). I have studied
> it, but I'm not sure I do understand it.  Some details about
> which I find hard/wrong/dislike
>
> MP-timestamp
> ------------
> I do not like having several messages having the same TIMESTAMP
>
>
>
> 62 SD-ID receiving an optional STR-DATA
> =======================================
> This must be a mistake!
> In the 3rd paragraph is stated the a receiver sometimes MUST
> NOT parse a STR-DATA of a log-message that is received.
> However, when the option Multi-part is not implemented, is
> doesn't now this!
>
>
>
> Hello WG,
>
> This is part 2 (of 3) of my comment on the 3th
> draft-syslog-protocol. See the introduction at my posting
> 1@3. This one contains comment on the text of the document;
> to help to clarify, and shorten the document. It does NOT
> contain comment on the "idea" (design); se posting 1@3.
>
> -------------------------------------------------------
>
>
>
> Implementation hints
> ====================
>
> The current RFC contains a lot of valuable hints for
> programmers, like the one about time-secfrac (Yes I
> introduced it, at least the bug:-)!
>
> Currently the are scattered around the document, making the
> document long and more complex to read for non-programmers.
>
> I would suggest to move all of them to a new chapter, at the
> end of the document (after the current chapter 9).
>
>
> ABNF (Chapter 4)
> ================
>
> I think we can clarify the syntax, by "unflattening" RGC-3164
> syslog uses some field and subfields which are nice when
> introducing syslog to others. The "understand" names as priority.
>
> So, keep it structured. I give it a try (please correct the
> syntax of the ABNF, as I not writing it daily anymore)
>
> SYSLOG-MSG  = HEADER DATA
> HEADER      = VERSIONING PRIO ID
> DATA        = [ STR-DATA ] MSG
> VERSIONING  = "V" VERSION
> VERSION     = 1*3 DIGIT
> PRIO        = '<' FACILITY '.' SEVERITY '>'      // See 2@3
> for notation change
> ID          = TIMESTAMP SP HOSTNAME SP TAG
> STR-DATA    = see elsewhere
> MSG         = free format
> (etc)
>
> Now we have meaningful field, that can be used to. E.g the ID
> field (see other mail) make each message unique
>
> 5.1 Format (typo?)
> ==================
> On page 20, the paragraph starting with "The structured data
> element MUST ..." is confusing. The 2nd last line say _no
> space_ is allowed, the last one says one or more space are.
> Is this a typo? Or it is unclear (at least to me)
>
> Structures data, ID-length
> ==========================
>
> I don't see why we should limit the several field to 64
> positions. I agree this will normally suffice. But so will
> 32, or 16, of any other number. 64 is "to big" to use as a
> fix-sized ("reserved") space for programmer's, database
> fields, etc. (to big == spoil to much bytes on huge logs). So
> dynamic field need to be used anyhow. Then there is no need
> for a trivial maximum. Note, there is a maximum anyhow, given
> by the size of a single log-message. That will do for "short
> term allocation".
>
> Removing this limit, make the rdc cleaner and smaller.
>
> Structured data, spaces
> =======================
>
> I would like to have all line about SP (spaces) in chapter 5
> removed. The point about 0,1 or more spaces is not relevant.
> In general, syslog uses SP to separate field (when needed).
> And allows them in MSG-part. The syntax and semantics of
> STR-DATA is does not depend on the amount of space. Nor is it
> harder to read. Implementing receivers even become easier
> when spaces (in STR-DATA) can be skipped (while { if 'sp'
> then skip } ) instead of checking the correct number, and
> doing something if wrong !
>
> Proposal: Allow spaces anywhere, but inside SD-ID (see note)
> and SD-PARAM. SP in SD-VALUE is allowed (already), but not a
> separator. Prescribe (at least 1) SP between each param-value pair.
>
> *Note: as SP between '[#@'(or '<') and the SP-ID itself is
> probably not a good idea, but not a problem. We can fix is,
> by moving the fixed string in the ABNF:
> STR-DATA    = STR-START ... STR-END
> STR-START   = "[#@" SD-ID   ; or '<'
> STR-END     = "]"           ; or '>'
> ...         = as before, SP are allowed.
>
> Note2: Doing so allows for format line for human reading,
> which is handy
> Example     <x-gam-example  doYoe="like this"  or     = "This
> one?"   >
>             <z-gam-more     Yes  = "I do"      find   = "it
> readable!" >
>
> This example is simple to parse, both for humans and
> computers. This change will make chapter-5 shorter, I think!
>
> Last, I think any whitespace should be allowed instead of SP (eg.
TAB)
>
> Chapter-5, MSG
> ==============
>
> I suggest, but only as a detail, the text of chapter-5 should
> be part of 4.2
>
>
>
> Hello WG,
>
> This is part 3 (of 3) of my comment on the 3th
> draft-syslog-protocol. It only contains short remark on
> details, and bit that didn't fit in part 1 (idea/design) or
> part 2 (the document itself). See posting 1@3 for more introduction
>
> -------------------------------------------------------
>
> 413/314 FACILITY/SEVERITY
> ==========================
>
> In this draft, both facility and severity are numbers. Even
> with my suggestion, the are 'just numbers'. And numbers are
> hard to read for humans. Especially when the are a lot of
> them. Most people will forget which column contains which number.
>
> Therefore, I think it is better to use the (verbose)notation
> used by most syslog implementations: "'<' facility '.'
> severity '>'" Both facility and severity are numbers (at
> least in the wire). Collectors (viewers) can translate those
> numbers into there names. But still use this format. And Even
> without them, it is easier to read the first (new) then the
> second (current) line: V1 0 <888.4> 2003-010-11T22:14:13.003Z
> new.Formated. ... V1 0 888 4 2003-010-11T22:14:13.003Z
> old.Formated. ...
>
> Note: I agree, we should not the tricky "8 times F plus S"
> notation. I'm not suggesting that! Just insert a dot and the angles.
>
>
> 4151 timestamp, without time
> ============================
>
> There are 'devices' as meant in this section which  haven't
> an idea of TIME. So it is a good idea this section.
>
> Often, those devices can store ("know") a few bit of
> information. Therefore, I would like to change this fixed
> TIMESTAMP, to the same one, but with a sequence number
> attached; the factional-seconds field can be used for it.
>
> Then a timestamp becomes 2001-01-01T00:00:60.<seq>Z.
> As in the current draft, this time doesn't exist. But al
> least, collectors can (more or less) sort a set of
> logmessages form 1 device
>
> Note: the latter is needed for e.g. syslog-sign
>
>
> 417 TAG
> =======
>
> I think we need to make the TAG stuctured! All current syslog
> receivers (collectors, relays) use __PARTS of__ the RFC3164
> TAG to route messages. In RFC3164 the TAG is simple and
> short, so it is quite simple to use it for routing. Note: not
> the complete TAG is used, only the 'program name', never the PID
part.
>
> With the new TAG, with an static ID an a dynamic part,
> similar routing should be possible. At least, the RFC should
> be clear on it. So, by demanding the static part is "fixed",
> and make sure that static part can be found.
>
> Given the current practice, routing is based (mainly) on the
> program-name, it would be wise to (at least suggest) how
> (where) that part can be found.
>
> Proposal:  Forget native support for VMS/Windows/DOS and even
> Unix pathnames. And introduce an URI (URL)-alike schema.
> Where only '/' (not the one form Unix, but from URL's) is
> used to "path separation" and ':' and '//' are major separators.
>
> In this case, the ABNF is simple; only the semantics become a
> little more complex. Also, it become simple for
> web-applications to log. The have an URL already. All "old
> fashioned:-)" application have an URL:
> ''file://path/to/appl'' already. This is valid on any system.
>
> Note: the dynamic part has to be added
> Note2: for web-applictions, which include a 'hostpart' in the
> URL: that hostname is NOT the same as the HOSTPART in the
> header. Frequently (a web-farm) several systems share the
> same URL, but not the hostname. Then the sysadmin can decide
> which one to use for routing.
>
>
> Message ID
> ==========
>
> Given the architecture of syslog-networks, messages can be
> duplicated. But, sometimes messages are related (e.g. with
> the signatures of syslog-sign). Both the RFC3164 and the
> current -protocol draft do not have possibilities to unique
> identify a message.
>
> In practice, messages are unique by there hostname, TAG and
> timestamp. But, we can't trust on this, as it isn't required.
>
> I would like to introduce this requirement. It is simple to
> add to the RFC, and simple to implement. Given the HOSTNAME
> and TAG, all the implementers have to do is never send a
> message with the same timestamp. Given the microsecond
> resolution, this doable. (It does imply some systems have to
> fake the last digits; I don't see a problem with this.
> Otherwise we can add a "." SequenceNo to the timestamp
>
> Structured data, tokens
> =======================
>
> Given the current draft, of my counter proposal of it, of
> structured data, the are (only) 2 kinds of tokens. The IANA
> controlled ones and the experimental ones; the latter
> starting with "x-"
>
> I think, we can safely add an third: "X-" for
> private/local/vendor specific tokens. As we can see my e.g.
> mail, this kind of field will be used a lot. Now we have an
> option to allow the, without giving the a status of "testing".
>
>
> STR-DATA, can we use it for syslog-sign (or similar)?
> =====================================================
>
> Just an idea: in a protocol as syslog-sign (here just as
> example), where messages reference to other messages (now
> implicit), we could use the STR-DATA to do so?
>
> I verified the syntax/semantics of this, and YES, we could do
> so. This means, I like STR-DATA a lot more :-) It is great.
> Even when -sign doesn't use it, I (we) can use the same
> format to present it to the user!
>
> 64 MultiPart examples
> =====================
>
> All examples use rfc3164 headers, shouldn't -protocol headers be
used?
>
>

------------------------------

Date: Fri, 2 Apr 2004 10:28:42 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: some (well a lot ) review remarks

Anton,

I have only minor concerns with moving this into the transport. But if
we move it into the transport, then we must do it right, which in my
point of view means NO size restriction (no, 64K is handy for UDP
fragmentation, but I would also like to be able to send 1 MB if there is
really need for this...). My concern is when we move this into the
transport, the upper layer will probably need to be able to handle
full-size messages, which could easily be turned into a DoS - on the
other hand, we are able to work around this (like specify a way to
access frames instead of full messages).

If we do this right, I see indeed clear advantages of pushing this into
the transport. for the UDP transport, I would still propose to use what
is currently specified in -protocol.

Any more comments?
Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Thursday, April 01, 2004 9:30 PM
> To: 'ons-huis.net!ALbert'; [email-masked]; Rainer Gerhards
> Subject: RE: some (well a lot ) review remarks
>
> Albert, Rainer, WG:
>
> I have not thought through all the details, but...
>
> I share Albert's concern about defining message length limits and
> mandating partitioning in syslog-protocol given that it is supposed to
> be transport independent.  I'd prefer if syslog-protocol was just
> about the format and encoding of the message payload. Payload being a
> complete syslog message without any transport-specific parts.
>
> As it stands now, syslog-transport is really about transferring syslog
> message segments and not complete syslog messages.  If
> syslog-transport does all segmentation, then it is easier to specify
> how to avoid issues such as double-segmentation (syslog-protocol level
> and then IP fragmentation).
>
> The relay provisions should probably also be specified only as far as
> payload is concerned. For example in DHCP, when message goes through
> relay, relay agent is allowed to insert an option specifying its
> address as well as giaddr (the network from which request for IP
> originated from).  So, it just specifies additional provisions for
> payload for cases with relay agents and that's it. I think it will be
> easiest if we allowed relay agent to function like any
> syslog-transport client and re-assemble the whole message before doing
> anything with it. In other words, it should work with syslog messages,
> not syslog message segments.
>
> Just my 2 cents.
>
> Anton.
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of
> > ons-huis.net!ALbert
> > Sent: Tuesday, March 30, 2004 11:59 AM
> > To: [email-masked]; [email-masked]
> > Subject: some (well a lot ) review remarks
> >
> >
> > Hello Rainer, Anton, WG
> >
> > Last weeks I have reviewed some of the current draft
> > documents. I had planned to spend more time on it, coming
> > month. But, plans are changed. It will be very busy, which is
> > good .. But also means less tome for syslog.
> >
> >
> > I have one important "issue": I think it would be a great
> > improvement if some parts of Rainer's document would move to
> > Anton's. I think the protocol document should NOT specify how
> > to break messages in parts. It should assume the transport
> > layer can "transport" 'long messages'. The transport layer,
> > should when a long message can't be send in once, spilt in is
> > several parts.
> >
> > I think it should be possible to write another 'transport'
> > document, which describes how 'long, new, better' messages
> > can be transported by rfc-3164 syslog messages.
> > Note: I'm NOT saying we should make an RFC for this. Only
> > that, if we could write it, the separation between transport
> > and  "upper" layers are better. Note 2: That document should
> > describe how to 'shrink' the longer headers into the old
> > headers, etc. Not by putting the complete message into the payload.
> >
> > --------------
> > Aside from the point above, I will include a raider long list of
> > (personal) notes 'asis'.
> >
> > Rainer, please use those notes when relevant, Skip the
> > others. I'm sorry, I don't have more time to rewrite my notes
> > to a good review. It is either me to throw away them, of give
> > another that option.:-)
> >
> > Hope it helps a bit.
> >
> > ===========================================================
> >
> > --
> > Groetjes,
> > --ALbert Mietus
> > Private mail to:           albert at ons-huis dot net
> > Business mail to:     albert dot mietus at PTS dot nl
> > Spam:   Just don't do it! Thrust me, I will not order!
> >
> > Hello WG,
> >
> > This is my comment on Rainer's syslog-protcol draft 04. I
> > have been very passive on following this mailinglist; (to
> > busy:-) But on receiving a request to review I decided to do
> > so. But I apologise if I raise issues that are discussed
> > already. I didn't read them. This week I started, after a log
> > time, to study the current draft document(s).
> >
> > Before continuing, let me say I like the idea of splitting
> > syslog in several layers. I say so, in the hope the
> > individual sub-specifications will become short. My comment
> > is split into 3 parts, of which you are reading part 1 now.
> > Part 1 is about the "idea" (the design). In a separate mail I
> > will comment on the text of the document; in the hope I will
> > become clearer, cleaner and shorter. The 3rd mail will
> > contain some details and bit that didn't fit in the others.
> >
> > Possible, I will send more comment in futures mail; it will
> > depend on the amount of time I have now, and then.
> >
> > -------------------------------------------------------
> >
> > This review is becoming (very) long, I really hope all
> > comments are worth reading. I have tried to express my
> > thoughts about is a well as possible, given the time I have...
> >
> > General Comment on the idea/design of syslog-protocol
> > =====================================================
> >
> > * This idea is GOOD!
> >
> >
> >
> > H2 Architecture
> > ================
> >
> > Traditional, syslog knows Devices, Collectors and Relays.
> > I would like to add two 'things'. One I would like to call a
> > *Generator* , the other an *Runner*.
> >
> > *Generator*
> > As we can see in e.g. most Unix implementations, it is the
> > application that knows WHAT to log, transmit that to the
> > system (the log device, the syslogdaemon) that know HOW to
> > log it. Also on embedded systems, this can be seen.
> > Historically, the combination of (a part of the) application
> > and the "system" form the log-device. I would like to split
> > this, now this opportunity exits. The part that is build-in
> > in the application, (in C: the lines syslog(..."hai there");)
> > can be called  the (LOG-)Generator. The communication between
> > generator and Device is system/platform depended. On Unix
> > systems usually the log-device, on embedded systems a
> > function-call, and on windows log-events can be used.
> >
> > By introducing this Generator, we (can) make clear this
> > private/dedicated communication exist; and is allowed. We
> > also make clear the Generator is syslog-protocol INDEPENDED.
> >
> > The function of the Device, know becomes clear: it get
> > log-message (-events) and transport them. It also does some
> > bookkeeping, like timestamping, adding crypto (for -sign), etc.
> >
> > *Runner*
> > A LOG-runner is a other kind of syslog-thing, which is
> > frequently used. Without a proper place in the architecture.
> > Whereas a relay (should) forwards syslog-messages without
> > knowledge of the semantics of the message, the Runner does.
> > The most simple life-form of a runner is a filter. It
> > "relays" messages, but only when the are important. On Unix
> > there a several of these in perl, grep-scripts  etc.
> > Formally, the are knot relays (I think). A more complex
> > runner, is a "program" that receives log-messages, CHANGES
> > them, and send (or stores) the result. Examples: statically
> > analyses, Intrusion detection, etc.
> >
> > Both kind of Runners are useful, frequently used, but the not
> > part of the architecture. And as we try to make syslog
> > "better", we better add them and make sure out standards can
> > "deal" with them. Otherwise, non-RFC compliance log-programs
> > will be standard.
> >
> >
> > H4 Syslog format
> > ================
> >
> > 412 enterpriseID
> > -----------------
> >
> > I don't see any reason to include an enterpriseID; not into
> > the header. (When needed, it can be used in the structured MSG part)
> >
> > Currently, it is just a number. It will be unused, misused or
> > will lead to a lot of (operational) management. I'm afraid
> > for the latter, as in H4.1.3 is suggested  that the semantic
> > of the Facility can be enterpriceID depended.
> >
> > Also, it is required to use the "IANA assigned vendor"
> > number. This implies open-source/free/non-commercial are
> > 'ruled out' as the often will not to so.
> >
> > Last, should the number of the Generator-vendor, the
> > system-vendor of the device-vendor be used? (See above about
> > generator/device) This is not clear to me. And whatever one
> > is chosen, it will be hard to implement. Not using the
> > defacto (Unix) logging api's!
> >
> > 413 Facility
> > -------------
> >
> > Although, at first sight, liked the idea of "a terrible lot"
> > of facilities. The current <used a number idea> is wrong, as
> > I see it. Aside from the problems mentioned above, more then
> > a million facilities will mean relays can't be managed! The
> > set of facilities, which will be seen in a (major) network,
> > expressed as numbers, will be are more or less at random.
> > Which implies very long complex and unmanageable configfiles
> > (or MIB's) for each LOG-router!
> >
> > As we heave learn form routing IPv4, hierarchical structuring
> > is needed.
> >
> > I think, extending the set of facilities is good. But I can't
> > imagine more the say 1000 are ever needed.
> >
> > So, my counter-proposition is:
> >         *  Make  facilities (as a number) structured
> >         *  Limit the number of facilities to a manageable number
> >         *  Keep the format such that extending the allowed
> > numbers is possible
> >
> > A Facility then is still a number, at least 3 (or 4) digits
> > long. A longer number means the it is an extended facility.
> > They have to be assign by IANA. Facilities of length 3 (4)
> > MUST have the format '(K)KLM', where 'K' (or 'KK') indicated
> > the kind of facility; 'L' give a sub indication and 'M' is
> > *SITE* configurable (so, by the local sysadmin, see example
> > below). The  'K/KK' is based on the RFC3164 facilities, clean
> > up and extended. Those numbers can be IANA assigned. L can be
> > chosen by the (generator) vendor, and `M' by the admin. 'M'
> > defaults to 0 (zero), and  applications/vendors MAY give the
> > possibility to set that digit.
> >
> > Example: For mail, there will be an K specified, let say 1.
> > Then all mail-log will have the format '1XY', which is easily
> > routable. It will do for small sites. Some vendors, like
> > "sendmail" (only 1 process) will probably use only one value
> > for L e.g. '0'; others, like "postfix" (several processes)
> > can use multiple values, like '1', '2' and '3'. When
> > supported by both sendmail and postfix, the local sysadmin
> > can add (change) the M-digit, such that mail-systems on the
> > border, and internal ones use another facility.
> >
> > In all cases, the local sysadmin can either use simple
> > routing-rules, like 1** (for all mail), or 10* for  sendmail
> > and 1[1-2]*, 13* for postfix, or even more complex. Now, the
> > sysadmin has a choice, and can keep it manageable.
> >
> > Note: the 0 for sendmail and 1-4 for postfix are "by
> > example". However, we can add a "rule" that '0' shall be used
> > when only 1 L-value is used, and when several values are
> > used, zero should be skipped. Also, I would like to
> > prescribe/reserve "9" for local additions. (on all digits).
> >
> > The (K)KLM idea is used a suggestion to improve, IF this idea
> > is accepted, THEN we can discuss variations like 3 or 4
> > digits, where to save mappings (IANA of this rfc), etc.
> >
> > 5 Structured data
> > ==================
> >
> > In short: I thing having the option of structured data is a
> > nice option. But lets keep it simple.
> >
> > The current one is to complex, it has to be as we need 4
> > pages to describe it. Also, I find those pages hard to study
> > (given the time I had :-). Also it can lead to not
> > implementing it. Programmers, especially there bosses, don't
> > have a lot of time!
> >
> > More positive: The main reason why structured data is complex
> > (currently) comes down into 1 problems:
> > 1) It isn't part of the main-design (the ABNF on page 9)
> > 2) The "structure" can start anywhere in MSG
> >
> > Both are easy to solve:
> > Ad 2) Specify that the structured part ALWAYS START directly
> > after the header. Ad 1) We need to introduce it where it
> > belongs. In an optional field on page 9
> >
> >
> > Let give it a try ( I also use the "improvement" on the ABNF
> > of my other mail; it saves typing) (Also I "forget" the SP
> > parts, for now. Just the idea)
> >
> > SYSLOG-MSG  = HEADER DATA
> > HEADER      = VERSIONING PRIO ID         // See other mail
> > DATA        = [ *STR-DATA ] MSG
> > STR-DATA    = see below
> > MSG         = free format
> >
> > Given this ABNF, the structured data ALWAYS comes (in RFC3164
> > notation) at the start of the MSG-part (of in new ABNF:
> > before the free format MSG).
> >
> > This implies receivers always have (as last resort) the
> > option to see everything after the header as free-format. And
> > just store/forward it. It implies the start of STR-DATA is
> > simple to find: Its starts directly after the header, or
> > directly after another STR-DATA
> >
> > This implies to, we have the option to start STR-DATA with
> > '<' which is more usable and XML-alike. The complex long
> > "[@#" cookie isn't needed anymore. However, we free to use
> > it. My personal vote is for the (XML) < > style.
> >
> > See also my other posting about details of structured-data.
> >
> > 6 Multi-Part Messages
> > =====================
> >
> > There are some mistakes in the this part, but I like the
> > general idea. However, I fell spliting/reassembly is done in
> > other protocols too. Maybe we can use/reference a (de facto)
> > standard? I don't know an RFC which we can use, but I'm sure
> > there must be one!
> >
> > Second, it to complex, and to long (to read). I have studied
> > it, but I'm not sure I do understand it.  Some details about
> > which I find hard/wrong/dislike
> >
> > MP-timestamp
> > ------------
> > I do not like having several messages having the same TIMESTAMP
> >
> >
> >
> > 62 SD-ID receiving an optional STR-DATA
> > =======================================
> > This must be a mistake!
> > In the 3rd paragraph is stated the a receiver sometimes MUST
> > NOT parse a STR-DATA of a log-message that is received.
> > However, when the option Multi-part is not implemented, is
> > doesn't now this!
> >
> >
> >
> > Hello WG,
> >
> > This is part 2 (of 3) of my comment on the 3th
> > draft-syslog-protocol. See the introduction at my posting
> > 1@3. This one contains comment on the text of the document;
> > to help to clarify, and shorten the document. It does NOT
> > contain comment on the "idea" (design); se posting 1@3.
> >
> > -------------------------------------------------------
> >
> >
> >
> > Implementation hints
> > ====================
> >
> > The current RFC contains a lot of valuable hints for
> > programmers, like the one about time-secfrac (Yes I
> > introduced it, at least the bug:-)!
> >
> > Currently the are scattered around the document, making the
> > document long and more complex to read for non-programmers.
> >
> > I would suggest to move all of them to a new chapter, at the
> > end of the document (after the current chapter 9).
> >
> >
> > ABNF (Chapter 4)
> > ================
> >
> > I think we can clarify the syntax, by "unflattening" RGC-3164
> > syslog uses some field and subfields which are nice when
> > introducing syslog to others. The "understand" names as priority.
> >
> > So, keep it structured. I give it a try (please correct the
> > syntax of the ABNF, as I not writing it daily anymore)
> >
> > SYSLOG-MSG  = HEADER DATA
> > HEADER      = VERSIONING PRIO ID
> > DATA        = [ STR-DATA ] MSG
> > VERSIONING  = "V" VERSION
> > VERSION     = 1*3 DIGIT
> > PRIO        = '<' FACILITY '.' SEVERITY '>'      // See 2@3
> > for notation change
> > ID          = TIMESTAMP SP HOSTNAME SP TAG
> > STR-DATA    = see elsewhere
> > MSG         = free format
> > (etc)
> >
> > Now we have meaningful field, that can be used to. E.g the ID
> > field (see other mail) make each message unique
> >
> > 5.1 Format (typo?)
> > ==================
> > On page 20, the paragraph starting with "The structured data
> > element MUST ..." is confusing. The 2nd last line say _no
> > space_ is allowed, the last one says one or more space are.
> > Is this a typo? Or it is unclear (at least to me)
> >
> > Structures data, ID-length
> > ==========================
> >
> > I don't see why we should limit the several field to 64
> > positions. I agree this will normally suffice. But so will
> > 32, or 16, of any other number. 64 is "to big" to use as a
> > fix-sized ("reserved") space for programmer's, database
> > fields, etc. (to big == spoil to much bytes on huge logs). So
> > dynamic field need to be used anyhow. Then there is no need
> > for a trivial maximum. Note, there is a maximum anyhow, given
> > by the size of a single log-message. That will do for "short
> > term allocation".
> >
> > Removing this limit, make the rdc cleaner and smaller.
> >
> > Structured data, spaces
> > =======================
> >
> > I would like to have all line about SP (spaces) in chapter 5
> > removed. The point about 0,1 or more spaces is not relevant.
> > In general, syslog uses SP to separate field (when needed).
> > And allows them in MSG-part. The syntax and semantics of
> > STR-DATA is does not depend on the amount of space. Nor is it
> > harder to read. Implementing receivers even become easier
> > when spaces (in STR-DATA) can be skipped (while { if 'sp'
> > then skip } ) instead of checking the correct number, and
> > doing something if wrong !
> >
> > Proposal: Allow spaces anywhere, but inside SD-ID (see note)
> > and SD-PARAM. SP in SD-VALUE is allowed (already), but not a
> > separator. Prescribe (at least 1) SP between each param-value pair.
> >
> > *Note: as SP between '[#@'(or '<') and the SP-ID itself is
> > probably not a good idea, but not a problem. We can fix is,
> > by moving the fixed string in the ABNF:
> > STR-DATA    = STR-START ... STR-END
> > STR-START   = "[#@" SD-ID   ; or '<'
> > STR-END     = "]"           ; or '>'
> > ...         = as before, SP are allowed.
> >
> > Note2: Doing so allows for format line for human reading,
> > which is handy
> > Example     <x-gam-example  doYoe="like this"  or     = "This
> > one?"   >
> >             <z-gam-more     Yes  = "I do"      find   = "it
> > readable!" >
> >
> > This example is simple to parse, both for humans and
> > computers. This change will make chapter-5 shorter, I think!
> >
> > Last, I think any whitespace should be allowed instead of SP (eg.
> TAB)
> >
> > Chapter-5, MSG
> > ==============
> >
> > I suggest, but only as a detail, the text of chapter-5 should
> > be part of 4.2
> >
> >
> >
> > Hello WG,
> >
> > This is part 3 (of 3) of my comment on the 3th
> > draft-syslog-protocol. It only contains short remark on
> > details, and bit that didn't fit in part 1 (idea/design) or
> > part 2 (the document itself). See posting 1@3 for more introduction
> >
> > -------------------------------------------------------
> >
> > 413/314 FACILITY/SEVERITY
> > ==========================
> >
> > In this draft, both facility and severity are numbers. Even
> > with my suggestion, the are 'just numbers'. And numbers are
> > hard to read for humans. Especially when the are a lot of
> > them. Most people will forget which column contains which number.
> >
> > Therefore, I think it is better to use the (verbose)notation
> > used by most syslog implementations: "'<' facility '.'
> > severity '>'" Both facility and severity are numbers (at
> > least in the wire). Collectors (viewers) can translate those
> > numbers into there names. But still use this format. And Even
> > without them, it is easier to read the first (new) then the
> > second (current) line: V1 0 <888.4> 2003-010-11T22:14:13.003Z
> > new.Formated. ... V1 0 888 4 2003-010-11T22:14:13.003Z
> > old.Formated. ...
> >
> > Note: I agree, we should not the tricky "8 times F plus S"
> > notation. I'm not suggesting that! Just insert a dot and the angles.
> >
> >
> > 4151 timestamp, without time
> > ============================
> >
> > There are 'devices' as meant in this section which  haven't
> > an idea of TIME. So it is a good idea this section.
> >
> > Often, those devices can store ("know") a few bit of
> > information. Therefore, I would like to change this fixed
> > TIMESTAMP, to the same one, but with a sequence number
> > attached; the factional-seconds field can be used for it.
> >
> > Then a timestamp becomes 2001-01-01T00:00:60.<seq>Z.
> > As in the current draft, this time doesn't exist. But al
> > least, collectors can (more or less) sort a set of
> > logmessages form 1 device
> >
> > Note: the latter is needed for e.g. syslog-sign
> >
> >
> > 417 TAG
> > =======
> >
> > I think we need to make the TAG stuctured! All current syslog
> > receivers (collectors, relays) use __PARTS of__ the RFC3164
> > TAG to route messages. In RFC3164 the TAG is simple and
> > short, so it is quite simple to use it for routing. Note: not
> > the complete TAG is used, only the 'program name', never the PID
> part.
> >
> > With the new TAG, with an static ID an a dynamic part,
> > similar routing should be possible. At least, the RFC should
> > be clear on it. So, by demanding the static part is "fixed",
> > and make sure that static part can be found.
> >
> > Given the current practice, routing is based (mainly) on the
> > program-name, it would be wise to (at least suggest) how
> > (where) that part can be found.
> >
> > Proposal:  Forget native support for VMS/Windows/DOS and even
> > Unix pathnames. And introduce an URI (URL)-alike schema.
> > Where only '/' (not the one form Unix, but from URL's) is
> > used to "path separation" and ':' and '//' are major separators.
> >
> > In this case, the ABNF is simple; only the semantics become a
> > little more complex. Also, it become simple for
> > web-applications to log. The have an URL already. All "old
> > fashioned:-)" application have an URL:
> > ''file://path/to/appl'' already. This is valid on any system.
> >
> > Note: the dynamic part has to be added
> > Note2: for web-applictions, which include a 'hostpart' in the
> > URL: that hostname is NOT the same as the HOSTPART in the
> > header. Frequently (a web-farm) several systems share the
> > same URL, but not the hostname. Then the sysadmin can decide
> > which one to use for routing.
> >
> >
> > Message ID
> > ==========
> >
> > Given the architecture of syslog-networks, messages can be
> > duplicated. But, sometimes messages are related (e.g. with
> > the signatures of syslog-sign). Both the RFC3164 and the
> > current -protocol draft do not have possibilities to unique
> > identify a message.
> >
> > In practice, messages are unique by there hostname, TAG and
> > timestamp. But, we can't trust on this, as it isn't required.
> >
> > I would like to introduce this requirement. It is simple to
> > add to the RFC, and simple to implement. Given the HOSTNAME
> > and TAG, all the implementers have to do is never send a
> > message with the same timestamp. Given the microsecond
> > resolution, this doable. (It does imply some systems have to
> > fake the last digits; I don't see a problem with this.
> > Otherwise we can add a "." SequenceNo to the timestamp
> >
> > Structured data, tokens
> > =======================
> >
> > Given the current draft, of my counter proposal of it, of
> > structured data, the are (only) 2 kinds of tokens. The IANA
> > controlled ones and the experimental ones; the latter
> > starting with "x-"
> >
> > I think, we can safely add an third: "X-" for
> > private/local/vendor specific tokens. As we can see my e.g.
> > mail, this kind of field will be used a lot. Now we have an
> > option to allow the, without giving the a status of "testing".
> >
> >
> > STR-DATA, can we use it for syslog-sign (or similar)?
> > =====================================================
> >
> > Just an idea: in a protocol as syslog-sign (here just as
> > example), where messages reference to other messages (now
> > implicit), we could use the STR-DATA to do so?
> >
> > I verified the syntax/semantics of this, and YES, we could do
> > so. This means, I like STR-DATA a lot more :-) It is great.
> > Even when -sign doesn't use it, I (we) can use the same
> > format to present it to the user!
> >
> > 64 MultiPart examples
> > =====================
> >
> > All examples use rfc3164 headers, shouldn't -protocol headers be
> used?
> >
> >
>
>
>
>

------------------------------

Date: Tue, 13 Apr 2004 12:47:37 -0400
From: Anton Okmianski <[email-masked]>
Subject: fragmentation in transport

Rainer & all:

I started thinking about the best way of specifying message
fragmentation in the UDP transport layer. I don't think our present
scheme is ideal for good protocol layer isolation because it would
require transport layer to be able to parse protocol message in order
to read/modify structured content tag "msgpart".  I think it is better
if transport could deal with syslog messages as generic payload.

I propose that we stick to a definition of fragmentation in the
transport layer that is more agnostic to syslog-protocol format and
only manipulates the header.  Specifically, I propose we add a header
that mimics fragmentation fields of IP (RFC791) follows, except with
bigger field sizes:

- - multi-part-flag (1 byte)
- - msg-id (2 bytes)
- - fragment-length (32 bytes)
- - fragment-offset (32 bytes)
- - more-fragments-flag (1 byte)

All in all this will be an additional 67 byte header and would enable
transferring payload of up to 4GB.  If multi-part-flag is 0, we can
omit the rest of the header.

The trickiest part is the msg-id field. I propose that we specify that
fragment reassembly is done using source IP, source port and msg-id
field. This means that we will place a restriction that each
application must send all messages using a single random exclusive
source port (no SO_REUSEADDR flag), while changing msg-id for every
message. So, msg-id only has to be unique within a given process
because each process will use a different source port.  To avoid a
problem with mix up due to port re-use with short-lived processes, I
propose that we recommend that apps start msg-id at a random number
and then increment and cycle it. This would reduce the hole to mostly
theoretical domain. The idea of using ports to identify applications
is consistent with what they were designed to accomplish in the first
place.

IPv6 includes some a separate RFC2675 for jumbo datagrams, which could
be used to transfer large packets.  But it seems they restricted it to
the networks with large MTU sizes only. I am also worried that
adoption of this standard will lag the other IPv6 standards and it
would be risky for a syslog protocol to rely on intermediary entities
supporting it. So, we probably won't be able to use it and my proposal
is to use the same mechanism for UDP/IPv4 and UDP/IPv6.

Please let me know what you think before I write it up into the draft.

Thanks,
Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Friday, April 02, 2004 3:29 AM
> To: Anton Okmianski; ons-huis.net!ALbert; [email-masked]
> Subject: RE: some (well a lot ) review remarks
>
>
> Anton,
>
> I have only minor concerns with moving this into the
> transport. But if we move it into the transport, then we must
> do it right, which in my point of view means NO size
> restriction (no, 64K is handy for UDP fragmentation, but I
> would also like to be able to send 1 MB if there is really
> need for this...). My concern is when we move this into the
> transport, the upper layer will probably need to be able to
> handle full-size messages, which could easily be turned into
> a DoS - on the other hand, we are able to work around this
> (like specify a way to access frames instead of full messages).
>
> If we do this right, I see indeed clear advantages of pushing
> this into the transport. for the UDP transport, I would still
> propose to use what is currently specified in -protocol.
>
> Any more comments?
> Rainer
>
> > -----Original Message-----
> > From: Anton Okmianski [mailto:[email-masked]]
> > Sent: Thursday, April 01, 2004 9:30 PM
> > To: 'ons-huis.net!ALbert'; [email-masked]; Rainer
Gerhards
> > Subject: RE: some (well a lot ) review remarks
> >
> > Albert, Rainer, WG:
> >
> > I have not thought through all the details, but...
> >
> > I share Albert's concern about defining message length limits and
> > mandating partitioning in syslog-protocol given that it is
> supposed to
> > be transport independent.  I'd prefer if syslog-protocol was just
> > about the format and encoding of the message payload.
> Payload being a
> > complete syslog message without any transport-specific parts.
> >
> > As it stands now, syslog-transport is really about
> transferring syslog
> > message segments and not complete syslog messages.  If
> > syslog-transport does all segmentation, then it is easier
> to specify
> > how to avoid issues such as double-segmentation
> (syslog-protocol level
> > and then IP fragmentation).
> >
> > The relay provisions should probably also be specified only
> as far as
> > payload is concerned. For example in DHCP, when message
> goes through
> > relay, relay agent is allowed to insert an option specifying its
> > address as well as giaddr (the network from which request for IP
> > originated from).  So, it just specifies additional provisions for

> > payload for cases with relay agents and that's it. I think
> it will be
> > easiest if we allowed relay agent to function like any
> > syslog-transport client and re-assemble the whole message
> before doing
> > anything with it. In other words, it should work with
> syslog messages,
> > not syslog message segments.
> >
> > Just my 2 cents.
> >
> > Anton.
> >
> > > -----Original Message-----
> > > From: [email-masked]
> > > [mailto:[email-masked]] On Behalf Of
> > > ons-huis.net!ALbert
> > > Sent: Tuesday, March 30, 2004 11:59 AM
> > > To: [email-masked]; [email-masked]
> > > Subject: some (well a lot ) review remarks
> > >
> > >
> > > Hello Rainer, Anton, WG
> > >
> > > Last weeks I have reviewed some of the current draft documents.
I
> > > had planned to spend more time on it, coming month. But,
> plans are
> > > changed. It will be very busy, which is good .. But also
> means less
> > > tome for syslog.
> > >
> > >
> > > I have one important "issue": I think it would be a great
> > > improvement if some parts of Rainer's document would move to
> > > Anton's. I think the protocol document should NOT specify how to

> > > break messages in parts. It should assume the transport layer
can
> > > "transport" 'long messages'. The transport layer, should
> when a long
> > > message can't be send in once, spilt in is several parts.
> > >
> > > I think it should be possible to write another
> 'transport' document,
> > > which describes how 'long, new, better' messages can be
> transported
> > > by rfc-3164 syslog messages.
> > > Note: I'm NOT saying we should make an RFC for this. Only
> that, if
> > > we could write it, the separation between transport and  "upper"
> > > layers are better. Note 2: That document should describe how to
> > > 'shrink' the longer headers into the old headers, etc. Not by
> > > putting the complete message into the payload.
> > >
> > > --------------
> > > Aside from the point above, I will include a raider long list of
> > > (personal) notes 'asis'.
> > >
> > > Rainer, please use those notes when relevant, Skip the
> others. I'm
> > > sorry, I don't have more time to rewrite my notes to a
> good review.
> > > It is either me to throw away them, of give another that
> option.:-)
> > >
> > > Hope it helps a bit.
> > >
> > > ===========================================================
> > >
> > > --
> > > Groetjes,
> > > --ALbert Mietus
> > > Private mail to:           albert at ons-huis dot net
> > > Business mail to:     albert dot mietus at PTS dot nl
> > > Spam:   Just don't do it! Thrust me, I will not order!
> > >
> > > Hello WG,
> > >
> > > This is my comment on Rainer's syslog-protcol draft 04. I
> have been
> > > very passive on following this mailinglist; (to
> > > busy:-) But on receiving a request to review I decided to
> do so. But
> > > I apologise if I raise issues that are discussed already.
> I didn't
> > > read them. This week I started, after a log time, to study the
> > > current draft document(s).
> > >
> > > Before continuing, let me say I like the idea of
> splitting syslog in
> > > several layers. I say so, in the hope the individual
> > > sub-specifications will become short. My comment is split into 3

> > > parts, of which you are reading part 1 now. Part 1 is about the
> > > "idea" (the design). In a separate mail I will comment on
> the text
> > > of the document; in the hope I will become clearer, cleaner and
> > > shorter. The 3rd mail will contain some details and bit
> that didn't
> > > fit in the others.
> > >
> > > Possible, I will send more comment in futures mail; it
> will depend
> > > on the amount of time I have now, and then.
> > >
> > > -------------------------------------------------------
> > >
> > > This review is becoming (very) long, I really hope all
> comments are
> > > worth reading. I have tried to express my thoughts about
> is a well
> > > as possible, given the time I have...
> > >
> > > General Comment on the idea/design of syslog-protocol
> > > =====================================================
> > >
> > > * This idea is GOOD!
> > >
> > >
> > >
> > > H2 Architecture
> > > ================
> > >
> > > Traditional, syslog knows Devices, Collectors and Relays. I
would
> > > like to add two 'things'. One I would like to call a
> > > *Generator* , the other an *Runner*.
> > >
> > > *Generator*
> > > As we can see in e.g. most Unix implementations, it is the
> > > application that knows WHAT to log, transmit that to the
> system (the
> > > log device, the syslogdaemon) that know HOW to log it. Also on
> > > embedded systems, this can be seen. Historically, the
> combination of
> > > (a part of the) application and the "system" form the
> log-device. I
> > > would like to split this, now this opportunity exits. The
> part that
> > > is build-in in the application, (in C: the lines syslog(..."hai
> > > there");) can be called  the (LOG-)Generator. The communication
> > > between generator and Device is system/platform depended. On
Unix
> > > systems usually the log-device, on embedded systems a
> > > function-call, and on windows log-events can be used.
> > >
> > > By introducing this Generator, we (can) make clear this
> > > private/dedicated communication exist; and is allowed. We
> also make
> > > clear the Generator is syslog-protocol INDEPENDED.
> > >
> > > The function of the Device, know becomes clear: it get
> log-message
> > > (-events) and transport them. It also does some bookkeeping,
like
> > > timestamping, adding crypto (for -sign), etc.
> > >
> > > *Runner*
> > > A LOG-runner is a other kind of syslog-thing, which is
frequently
> > > used. Without a proper place in the architecture. Whereas a
relay
> > > (should) forwards syslog-messages without knowledge of
> the semantics
> > > of the message, the Runner does. The most simple life-form of a
> > > runner is a filter. It "relays" messages, but only when the are
> > > important. On Unix there a several of these in perl,
> grep-scripts
> > > etc. Formally, the are knot relays (I think). A more complex
> > > runner, is a "program" that receives log-messages, CHANGES them,

> > > and send (or stores) the result. Examples: statically analyses,
> > > Intrusion detection, etc.
> > >
> > > Both kind of Runners are useful, frequently used, but the
> not part
> > > of the architecture. And as we try to make syslog "better", we
> > > better add them and make sure out standards can "deal" with
them.
> > > Otherwise, non-RFC compliance log-programs will be standard.
> > >
> > >
> > > H4 Syslog format
> > > ================
> > >
> > > 412 enterpriseID
> > > -----------------
> > >
> > > I don't see any reason to include an enterpriseID; not into the
> > > header. (When needed, it can be used in the structured MSG part)
> > >
> > > Currently, it is just a number. It will be unused,
> misused or will
> > > lead to a lot of (operational) management. I'm afraid for the
> > > latter, as in H4.1.3 is suggested  that the semantic of
> the Facility
> > > can be enterpriceID depended.
> > >
> > > Also, it is required to use the "IANA assigned vendor"
> number. This
> > > implies open-source/free/non-commercial are 'ruled out'
> as the often
> > > will not to so.
> > >
> > > Last, should the number of the Generator-vendor, the
> system-vendor
> > > of the device-vendor be used? (See above about
> > > generator/device) This is not clear to me. And whatever one is
> > > chosen, it will be hard to implement. Not using the
> defacto (Unix)
> > > logging api's!
> > >
> > > 413 Facility
> > > -------------
> > >
> > > Although, at first sight, liked the idea of "a terrible lot" of
> > > facilities. The current <used a number idea> is wrong, as
> I see it.
> > > Aside from the problems mentioned above, more then a million
> > > facilities will mean relays can't be managed! The set of
> facilities,
> > > which will be seen in a (major) network, expressed as
> numbers, will
> > > be are more or less at random. Which implies very long
> complex and
> > > unmanageable configfiles (or MIB's) for each LOG-router!
> > >
> > > As we heave learn form routing IPv4, hierarchical structuring is
> > > needed.
> > >
> > > I think, extending the set of facilities is good. But I can't
> > > imagine more the say 1000 are ever needed.
> > >
> > > So, my counter-proposition is:
> > >         *  Make  facilities (as a number) structured
> > >         *  Limit the number of facilities to a manageable number
> > >         *  Keep the format such that extending the
> allowed numbers
> > > is possible
> > >
> > > A Facility then is still a number, at least 3 (or 4)
> digits long. A
> > > longer number means the it is an extended facility. They
> have to be
> > > assign by IANA. Facilities of length 3 (4) MUST have the format
> > > '(K)KLM', where 'K' (or 'KK') indicated the kind of facility;
'L'
> > > give a sub indication and 'M' is
> > > *SITE* configurable (so, by the local sysadmin, see
> example below).
> > > The  'K/KK' is based on the RFC3164 facilities, clean up and
> > > extended. Those numbers can be IANA assigned. L can be
> chosen by the
> > > (generator) vendor, and `M' by the admin. 'M' defaults to
> 0 (zero),
> > > and  applications/vendors MAY give the possibility to set that
> > > digit.
> > >
> > > Example: For mail, there will be an K specified, let say
> 1. Then all
> > > mail-log will have the format '1XY', which is easily routable.
It
> > > will do for small sites. Some vendors, like "sendmail" (only 1
> > > process) will probably use only one value for L e.g. '0';
others,
> > > like "postfix" (several processes) can use multiple values, like

> > > '1', '2' and '3'. When supported by both sendmail and
> postfix, the
> > > local sysadmin can add (change) the M-digit, such that
> mail-systems
> > > on the border, and internal ones use another facility.
> > >
> > > In all cases, the local sysadmin can either use simple
> > > routing-rules, like 1** (for all mail), or 10* for  sendmail and

> > > 1[1-2]*, 13* for postfix, or even more complex. Now, the
sysadmin
> > > has a choice, and can keep it manageable.
> > >
> > > Note: the 0 for sendmail and 1-4 for postfix are "by example".
> > > However, we can add a "rule" that '0' shall be used when only 1
> > > L-value is used, and when several values are used, zero should
be
> > > skipped. Also, I would like to prescribe/reserve "9" for local
> > > additions. (on all digits).
> > >
> > > The (K)KLM idea is used a suggestion to improve, IF this idea is
> > > accepted, THEN we can discuss variations like 3 or 4
> digits, where
> > > to save mappings (IANA of this rfc), etc.
> > >
> > > 5 Structured data
> > > ==================
> > >
> > > In short: I thing having the option of structured data is a nice
> > > option. But lets keep it simple.
> > >
> > > The current one is to complex, it has to be as we need 4 pages
to
> > > describe it. Also, I find those pages hard to study
> (given the time
> > > I had :-). Also it can lead to not implementing it. Programmers,
> > > especially there bosses, don't have a lot of time!
> > >
> > > More positive: The main reason why structured data is complex
> > > (currently) comes down into 1 problems:
> > > 1) It isn't part of the main-design (the ABNF on page 9)
> > > 2) The "structure" can start anywhere in MSG
> > >
> > > Both are easy to solve:
> > > Ad 2) Specify that the structured part ALWAYS START
> directly after
> > > the header. Ad 1) We need to introduce it where it belongs. In
an
> > > optional field on page 9
> > >
> > >
> > > Let give it a try ( I also use the "improvement" on the
> ABNF of my
> > > other mail; it saves typing) (Also I "forget" the SP
> parts, for now.
> > > Just the idea)
> > >
> > > SYSLOG-MSG  = HEADER DATA
> > > HEADER      = VERSIONING PRIO ID         // See other mail
> > > DATA        = [ *STR-DATA ] MSG
> > > STR-DATA    = see below
> > > MSG         = free format
> > >
> > > Given this ABNF, the structured data ALWAYS comes (in RFC3164
> > > notation) at the start of the MSG-part (of in new ABNF:
> before the
> > > free format MSG).
> > >
> > > This implies receivers always have (as last resort) the option
to
> > > see everything after the header as free-format. And just
> > > store/forward it. It implies the start of STR-DATA is simple to
> > > find: Its starts directly after the header, or directly after
> > > another STR-DATA
> > >
> > > This implies to, we have the option to start STR-DATA
> with '<' which
> > > is more usable and XML-alike. The complex long "[@#" cookie
isn't
> > > needed anymore. However, we free to use it. My personal
> vote is for
> > > the (XML) < > style.
> > >
> > > See also my other posting about details of structured-data.
> > >
> > > 6 Multi-Part Messages
> > > =====================
> > >
> > > There are some mistakes in the this part, but I like the general
> > > idea. However, I fell spliting/reassembly is done in
> other protocols
> > > too. Maybe we can use/reference a (de facto) standard? I
> don't know
> > > an RFC which we can use, but I'm sure there must be one!
> > >
> > > Second, it to complex, and to long (to read). I have
> studied it, but
> > > I'm not sure I do understand it.  Some details about which I
find
> > > hard/wrong/dislike
> > >
> > > MP-timestamp
> > > ------------
> > > I do not like having several messages having the same TIMESTAMP
> > >
> > >
> > >
> > > 62 SD-ID receiving an optional STR-DATA
> > > =======================================
> > > This must be a mistake!
> > > In the 3rd paragraph is stated the a receiver sometimes MUST NOT

> > > parse a STR-DATA of a log-message that is received. However,
when
> > > the option Multi-part is not implemented, is doesn't now this!
> > >
> > >
> > >
> > > Hello WG,
> > >
> > > This is part 2 (of 3) of my comment on the 3th
> > > draft-syslog-protocol. See the introduction at my posting
> 1@3. This
> > > one contains comment on the text of the document; to help to
> > > clarify, and shorten the document. It does NOT contain comment
on
> > > the "idea" (design); se posting 1@3.
> > >
> > > -------------------------------------------------------
> > >
> > >
> > >
> > > Implementation hints
> > > ====================
> > >
> > > The current RFC contains a lot of valuable hints for
programmers,
> > > like the one about time-secfrac (Yes I introduced it, at
> least the
> > > bug:-)!
> > >
> > > Currently the are scattered around the document, making
> the document
> > > long and more complex to read for non-programmers.
> > >
> > > I would suggest to move all of them to a new chapter, at
> the end of
> > > the document (after the current chapter 9).
> > >
> > >
> > > ABNF (Chapter 4)
> > > ================
> > >
> > > I think we can clarify the syntax, by "unflattening"
> RGC-3164 syslog
> > > uses some field and subfields which are nice when
> introducing syslog
> > > to others. The "understand" names as priority.
> > >
> > > So, keep it structured. I give it a try (please correct
> the syntax
> > > of the ABNF, as I not writing it daily anymore)
> > >
> > > SYSLOG-MSG  = HEADER DATA
> > > HEADER      = VERSIONING PRIO ID
> > > DATA        = [ STR-DATA ] MSG
> > > VERSIONING  = "V" VERSION
> > > VERSION     = 1*3 DIGIT
> > > PRIO        = '<' FACILITY '.' SEVERITY '>'      // See 2@3
> > > for notation change
> > > ID          = TIMESTAMP SP HOSTNAME SP TAG
> > > STR-DATA    = see elsewhere
> > > MSG         = free format
> > > (etc)
> > >
> > > Now we have meaningful field, that can be used to. E.g
> the ID field
> > > (see other mail) make each message unique
> > >
> > > 5.1 Format (typo?)
> > > ==================
> > > On page 20, the paragraph starting with "The structured
> data element
> > > MUST ..." is confusing. The 2nd last line say _no space_
> is allowed,
> > > the last one says one or more space are. Is this a typo? Or it
is
> > > unclear (at least to me)
> > >
> > > Structures data, ID-length
> > > ==========================
> > >
> > > I don't see why we should limit the several field to 64
> positions. I
> > > agree this will normally suffice. But so will 32, or 16, of any
> > > other number. 64 is "to big" to use as a fix-sized ("reserved")
> > > space for programmer's, database fields, etc. (to big == spoil
to
> > > much bytes on huge logs). So dynamic field need to be
> used anyhow.
> > > Then there is no need for a trivial maximum. Note, there is a
> > > maximum anyhow, given by the size of a single
> log-message. That will
> > > do for "short term allocation".
> > >
> > > Removing this limit, make the rdc cleaner and smaller.
> > >
> > > Structured data, spaces
> > > =======================
> > >
> > > I would like to have all line about SP (spaces) in chapter 5
> > > removed. The point about 0,1 or more spaces is not relevant. In
> > > general, syslog uses SP to separate field (when needed).
> And allows
> > > them in MSG-part. The syntax and semantics of STR-DATA is
> does not
> > > depend on the amount of space. Nor is it harder to read.
> > > Implementing receivers even become easier when spaces (in
> STR-DATA)
> > > can be skipped (while { if 'sp' then skip } ) instead of
checking
> > > the correct number, and doing something if wrong !
> > >
> > > Proposal: Allow spaces anywhere, but inside SD-ID (see note) and
> > > SD-PARAM. SP in SD-VALUE is allowed (already), but not a
> separator.
> > > Prescribe (at least 1) SP between each param-value pair.
> > >
> > > *Note: as SP between '[#@'(or '<') and the SP-ID itself
> is probably
> > > not a good idea, but not a problem. We can fix is, by moving the
> > > fixed string in the ABNF:
> > > STR-DATA    = STR-START ... STR-END
> > > STR-START   = "[#@" SD-ID   ; or '<'
> > > STR-END     = "]"           ; or '>'
> > > ...         = as before, SP are allowed.
> > >
> > > Note2: Doing so allows for format line for human reading,
> which is
> > > handy
> > > Example     <x-gam-example  doYoe="like this"  or     = "This
> > > one?"   >
> > >             <z-gam-more     Yes  = "I do"      find   = "it
> > > readable!" >
> > >
> > > This example is simple to parse, both for humans and
> computers. This
> > > change will make chapter-5 shorter, I think!
> > >
> > > Last, I think any whitespace should be allowed instead of SP
(eg.
> > TAB)
> > >
> > > Chapter-5, MSG
> > > ==============
> > >
> > > I suggest, but only as a detail, the text of chapter-5 should be
> > > part of 4.2
> > >
> > >
> > >
> > > Hello WG,
> > >
> > > This is part 3 (of 3) of my comment on the 3th
> > > draft-syslog-protocol. It only contains short remark on
> details, and
> > > bit that didn't fit in part 1 (idea/design) or part 2
> (the document
> > > itself). See posting 1@3 for more introduction
> > >
> > > -------------------------------------------------------
> > >
> > > 413/314 FACILITY/SEVERITY
> > > ==========================
> > >
> > > In this draft, both facility and severity are numbers.
> Even with my
> > > suggestion, the are 'just numbers'. And numbers are hard
> to read for
> > > humans. Especially when the are a lot of them. Most people will
> > > forget which column contains which number.
> > >
> > > Therefore, I think it is better to use the
> (verbose)notation used by
> > > most syslog implementations: "'<' facility '.' severity '>'"
Both
> > > facility and severity are numbers (at least in the wire).
> Collectors
> > > (viewers) can translate those numbers into there names. But
still
> > > use this format. And Even without them, it is easier to read the

> > > first (new) then the second (current) line: V1 0 <888.4>
> > > 2003-010-11T22:14:13.003Z new.Formated. ... V1 0 888 4
> > > 2003-010-11T22:14:13.003Z old.Formated. ...
> > >
> > > Note: I agree, we should not the tricky "8 times F plus
> S" notation.
> > > I'm not suggesting that! Just insert a dot and the angles.
> > >
> > >
> > > 4151 timestamp, without time
> > > ============================
> > >
> > > There are 'devices' as meant in this section which
> haven't an idea
> > > of TIME. So it is a good idea this section.
> > >
> > > Often, those devices can store ("know") a few bit of
information.
> > > Therefore, I would like to change this fixed TIMESTAMP,
> to the same
> > > one, but with a sequence number attached; the factional-seconds
> > > field can be used for it.
> > >
> > > Then a timestamp becomes 2001-01-01T00:00:60.<seq>Z.
> > > As in the current draft, this time doesn't exist. But al least,
> > > collectors can (more or less) sort a set of logmessages form 1
> > > device
> > >
> > > Note: the latter is needed for e.g. syslog-sign
> > >
> > >
> > > 417 TAG
> > > =======
> > >
> > > I think we need to make the TAG stuctured! All current syslog
> > > receivers (collectors, relays) use __PARTS of__ the
> RFC3164 TAG to
> > > route messages. In RFC3164 the TAG is simple and short, so it is
> > > quite simple to use it for routing. Note: not the complete TAG
is
> > > used, only the 'program name', never the PID
> > part.
> > >
> > > With the new TAG, with an static ID an a dynamic part, similar
> > > routing should be possible. At least, the RFC should be
> clear on it.
> > > So, by demanding the static part is "fixed", and make sure that
> > > static part can be found.
> > >
> > > Given the current practice, routing is based (mainly) on the
> > > program-name, it would be wise to (at least suggest) how
> > > (where) that part can be found.
> > >
> > > Proposal:  Forget native support for VMS/Windows/DOS and
> even Unix
> > > pathnames. And introduce an URI (URL)-alike schema. Where
> only '/'
> > > (not the one form Unix, but from URL's) is used to "path
> separation"
> > > and ':' and '//' are major separators.
> > >
> > > In this case, the ABNF is simple; only the semantics
> become a little
> > > more complex. Also, it become simple for web-applications to
log.
> > > The have an URL already. All "old fashioned:-)"
> application have an
> > > URL: ''file://path/to/appl'' already. This is valid on any
system.
> > >
> > > Note: the dynamic part has to be added
> > > Note2: for web-applictions, which include a 'hostpart' in the
> > > URL: that hostname is NOT the same as the HOSTPART in the
header.
> > > Frequently (a web-farm) several systems share the same
> URL, but not
> > > the hostname. Then the sysadmin can decide which one to use for
> > > routing.
> > >
> > >
> > > Message ID
> > > ==========
> > >
> > > Given the architecture of syslog-networks, messages can be
> > > duplicated. But, sometimes messages are related (e.g. with the
> > > signatures of syslog-sign). Both the RFC3164 and the current
> > > -protocol draft do not have possibilities to unique identify a
> > > message.
> > >
> > > In practice, messages are unique by there hostname, TAG and
> > > timestamp. But, we can't trust on this, as it isn't required.
> > >
> > > I would like to introduce this requirement. It is simple
> to add to
> > > the RFC, and simple to implement. Given the HOSTNAME and TAG,
all
> > > the implementers have to do is never send a message with the
same
> > > timestamp. Given the microsecond resolution, this doable.
> (It does
> > > imply some systems have to fake the last digits; I don't see a
> > > problem with this. Otherwise we can add a "." SequenceNo to the
> > > timestamp
> > >
> > > Structured data, tokens
> > > =======================
> > >
> > > Given the current draft, of my counter proposal of it, of
> structured
> > > data, the are (only) 2 kinds of tokens. The IANA
> controlled ones and
> > > the experimental ones; the latter starting with "x-"
> > >
> > > I think, we can safely add an third: "X-" for
> private/local/vendor
> > > specific tokens. As we can see my e.g. mail, this kind of
> field will
> > > be used a lot. Now we have an option to allow the, without
giving
> > > the a status of "testing".
> > >
> > >
> > > STR-DATA, can we use it for syslog-sign (or similar)?
> > > =====================================================
> > >
> > > Just an idea: in a protocol as syslog-sign (here just as
> example),
> > > where messages reference to other messages (now
> implicit), we could
> > > use the STR-DATA to do so?
> > >
> > > I verified the syntax/semantics of this, and YES, we could do
so.
> > > This means, I like STR-DATA a lot more :-) It is great. Even
when
> > > -sign doesn't use it, I (we) can use the same format to
> present it
> > > to the user!
> > >
> > > 64 MultiPart examples
> > > =====================
> > >
> > > All examples use rfc3164 headers, shouldn't -protocol headers be
> > used?
> > >
> > >
> >
> >
> >
> >
>
>

------------------------------

Date: Tue, 13 Apr 2004 20:57:09 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: fragmentation in transport

This is a not-well-thought out response, but I thought it is better to
provide it than to wait for 2 more days (Which otherwise might happen).
I think if we move multi-part messages into the transport, we MUST NOT
modify the header. Instead, each transport should add its own header,
where it requires. If we specify the header in -protocol, we are back at
where we are currently, that is that -protocol specifies the multi-part
messaging. If so, I'd prefer that we do it as it is specified so far,
because multi-part messages have low overhead ... Because we assummed
that they would be needed only very infrequently.

I have to admit that while I like support for large size message, I
think we should not try to make this feature mandatory... 99,999% of the
messages will never need it.

Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Tuesday, April 13, 2004 6:48 PM
> To: [email-masked]
> Subject: fragmentation in transport
>
> Rainer & all:
>
> I started thinking about the best way of specifying message
> fragmentation in the UDP transport layer. I don't think our present
> scheme is ideal for good protocol layer isolation because it would
> require transport layer to be able to parse protocol message in order
> to read/modify structured content tag "msgpart".  I think it is better
> if transport could deal with syslog messages as generic payload.
>
> I propose that we stick to a definition of fragmentation in the
> transport layer that is more agnostic to syslog-protocol format and
> only manipulates the header.  Specifically, I propose we add a header
> that mimics fragmentation fields of IP (RFC791) follows, except with
> bigger field sizes:
>
> - multi-part-flag (1 byte)
> - msg-id (2 bytes)
> - fragment-length (32 bytes)
> - fragment-offset (32 bytes)
> - more-fragments-flag (1 byte)
>
> All in all this will be an additional 67 byte header and would enable
> transferring payload of up to 4GB.  If multi-part-flag is 0, we can
> omit the rest of the header.
>
> The trickiest part is the msg-id field. I propose that we specify that
> fragment reassembly is done using source IP, source port and msg-id
> field. This means that we will place a restriction that each
> application must send all messages using a single random exclusive
> source port (no SO_REUSEADDR flag), while changing msg-id for every
> message. So, msg-id only has to be unique within a given process
> because each process will use a different source port.  To avoid a
> problem with mix up due to port re-use with short-lived processes, I
> propose that we recommend that apps start msg-id at a random number
> and then increment and cycle it. This would reduce the hole to mostly
> theoretical domain. The idea of using ports to identify applications
> is consistent with what they were designed to accomplish in the first
> place.
>
> IPv6 includes some a separate RFC2675 for jumbo datagrams, which could
> be used to transfer large packets.  But it seems they restricted it to
> the networks with large MTU sizes only. I am also worried that
> adoption of this standard will lag the other IPv6 standards and it
> would be risky for a syslog protocol to rely on intermediary entities
> supporting it. So, we probably won't be able to use it and my proposal
> is to use the same mechanism for UDP/IPv4 and UDP/IPv6.
>
> Please let me know what you think before I write it up into the draft.
>
> Thanks,
> Anton.
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> > Sent: Friday, April 02, 2004 3:29 AM
> > To: Anton Okmianski; ons-huis.net!ALbert; [email-masked]
> > Subject: RE: some (well a lot ) review remarks
> >
> >
> > Anton,
> >
> > I have only minor concerns with moving this into the
> > transport. But if we move it into the transport, then we must
> > do it right, which in my point of view means NO size
> > restriction (no, 64K is handy for UDP fragmentation, but I
> > would also like to be able to send 1 MB if there is really
> > need for this...). My concern is when we move this into the
> > transport, the upper layer will probably need to be able to
> > handle full-size messages, which could easily be turned into
> > a DoS - on the other hand, we are able to work around this
> > (like specify a way to access frames instead of full messages).
> >
> > If we do this right, I see indeed clear advantages of pushing
> > this into the transport. for the UDP transport, I would still
> > propose to use what is currently specified in -protocol.
> >
> > Any more comments?
> > Rainer
> >
> > > -----Original Message-----
> > > From: Anton Okmianski [mailto:[email-masked]]
> > > Sent: Thursday, April 01, 2004 9:30 PM
> > > To: 'ons-huis.net!ALbert'; [email-masked]; Rainer
> Gerhards
> > > Subject: RE: some (well a lot ) review remarks
> > >
> > > Albert, Rainer, WG:
> > >
> > > I have not thought through all the details, but...
> > >
> > > I share Albert's concern about defining message length limits and
> > > mandating partitioning in syslog-protocol given that it is
> > supposed to
> > > be transport independent.  I'd prefer if syslog-protocol was just
> > > about the format and encoding of the message payload.
> > Payload being a
> > > complete syslog message without any transport-specific parts.
> > >
> > > As it stands now, syslog-transport is really about
> > transferring syslog
> > > message segments and not complete syslog messages.  If
> > > syslog-transport does all segmentation, then it is easier
> > to specify
> > > how to avoid issues such as double-segmentation
> > (syslog-protocol level
> > > and then IP fragmentation).
> > >
> > > The relay provisions should probably also be specified only
> > as far as
> > > payload is concerned. For example in DHCP, when message
> > goes through
> > > relay, relay agent is allowed to insert an option specifying its
> > > address as well as giaddr (the network from which request for IP
> > > originated from).  So, it just specifies additional provisions for
>
> > > payload for cases with relay agents and that's it. I think
> > it will be
> > > easiest if we allowed relay agent to function like any
> > > syslog-transport client and re-assemble the whole message
> > before doing
> > > anything with it. In other words, it should work with
> > syslog messages,
> > > not syslog message segments.
> > >
> > > Just my 2 cents.
> > >
> > > Anton.
> > >
> > > > -----Original Message-----
> > > > From: [email-masked]
> > > > [mailto:[email-masked]] On Behalf Of
> > > > ons-huis.net!ALbert
> > > > Sent: Tuesday, March 30, 2004 11:59 AM
> > > > To: [email-masked]; [email-masked]
> > > > Subject: some (well a lot ) review remarks
> > > >
> > > >
> > > > Hello Rainer, Anton, WG
> > > >
> > > > Last weeks I have reviewed some of the current draft documents.
> I
> > > > had planned to spend more time on it, coming month. But,
> > plans are
> > > > changed. It will be very busy, which is good .. But also
> > means less
> > > > tome for syslog.
> > > >
> > > >
> > > > I have one important "issue": I think it would be a great
> > > > improvement if some parts of Rainer's document would move to
> > > > Anton's. I think the protocol document should NOT specify how to
>
> > > > break messages in parts. It should assume the transport layer
> can
> > > > "transport" 'long messages'. The transport layer, should
> > when a long
> > > > message can't be send in once, spilt in is several parts.
> > > >
> > > > I think it should be possible to write another
> > 'transport' document,
> > > > which describes how 'long, new, better' messages can be
> > transported
> > > > by rfc-3164 syslog messages.
> > > > Note: I'm NOT saying we should make an RFC for this. Only
> > that, if
> > > > we could write it, the separation between transport and  "upper"
> > > > layers are better. Note 2: That document should describe how to
> > > > 'shrink' the longer headers into the old headers, etc. Not by
> > > > putting the complete message into the payload.
> > > >
> > > > --------------
> > > > Aside from the point above, I will include a raider long list of
> > > > (personal) notes 'asis'.
> > > >
> > > > Rainer, please use those notes when relevant, Skip the
> > others. I'm
> > > > sorry, I don't have more time to rewrite my notes to a
> > good review.
> > > > It is either me to throw away them, of give another that
> > option.:-)
> > > >
> > > > Hope it helps a bit.
> > > >
> > > > ===========================================================
> > > >
> > > > --
> > > > Groetjes,
> > > > --ALbert Mietus
> > > > Private mail to:           albert at ons-huis dot net
> > > > Business mail to:     albert dot mietus at PTS dot nl
> > > > Spam:   Just don't do it! Thrust me, I will not order!
> > > >
> > > > Hello WG,
> > > >
> > > > This is my comment on Rainer's syslog-protcol draft 04. I
> > have been
> > > > very passive on following this mailinglist; (to
> > > > busy:-) But on receiving a request to review I decided to
> > do so. But
> > > > I apologise if I raise issues that are discussed already.
> > I didn't
> > > > read them. This week I started, after a log time, to study the
> > > > current draft document(s).
> > > >
> > > > Before continuing, let me say I like the idea of
> > splitting syslog in
> > > > several layers. I say so, in the hope the individual
> > > > sub-specifications will become short. My comment is split into 3
>
> > > > parts, of which you are reading part 1 now. Part 1 is about the
> > > > "idea" (the design). In a separate mail I will comment on
> > the text
> > > > of the document; in the hope I will become clearer, cleaner and
> > > > shorter. The 3rd mail will contain some details and bit
> > that didn't
> > > > fit in the others.
> > > >
> > > > Possible, I will send more comment in futures mail; it
> > will depend
> > > > on the amount of time I have now, and then.
> > > >
> > > > -------------------------------------------------------
> > > >
> > > > This review is becoming (very) long, I really hope all
> > comments are
> > > > worth reading. I have tried to express my thoughts about
> > is a well
> > > > as possible, given the time I have...
> > > >
> > > > General Comment on the idea/design of syslog-protocol
> > > > =====================================================
> > > >
> > > > * This idea is GOOD!
> > > >
> > > >
> > > >
> > > > H2 Architecture
> > > > ================
> > > >
> > > > Traditional, syslog knows Devices, Collectors and Relays. I
> would
> > > > like to add two 'things'. One I would like to call a
> > > > *Generator* , the other an *Runner*.
> > > >
> > > > *Generator*
> > > > As we can see in e.g. most Unix implementations, it is the
> > > > application that knows WHAT to log, transmit that to the
> > system (the
> > > > log device, the syslogdaemon) that know HOW to log it. Also on
> > > > embedded systems, this can be seen. Historically, the
> > combination of
> > > > (a part of the) application and the "system" form the
> > log-device. I
> > > > would like to split this, now this opportunity exits. The
> > part that
> > > > is build-in in the application, (in C: the lines syslog(..."hai
> > > > there");) can be called  the (LOG-)Generator. The communication
> > > > between generator and Device is system/platform depended. On
> Unix
> > > > systems usually the log-device, on embedded systems a
> > > > function-call, and on windows log-events can be used.
> > > >
> > > > By introducing this Generator, we (can) make clear this
> > > > private/dedicated communication exist; and is allowed. We
> > also make
> > > > clear the Generator is syslog-protocol INDEPENDED.
> > > >
> > > > The function of the Device, know becomes clear: it get
> > log-message
> > > > (-events) and transport them. It also does some bookkeeping,
> like
> > > > timestamping, adding crypto (for -sign), etc.
> > > >
> > > > *Runner*
> > > > A LOG-runner is a other kind of syslog-thing, which is
> frequently
> > > > used. Without a proper place in the architecture. Whereas a
> relay
> > > > (should) forwards syslog-messages without knowledge of
> > the semantics
> > > > of the message, the Runner does. The most simple life-form of a
> > > > runner is a filter. It "relays" messages, but only when the are
> > > > important. On Unix there a several of these in perl,
> > grep-scripts
> > > > etc. Formally, the are knot relays (I think). A more complex
> > > > runner, is a "program" that receives log-messages, CHANGES them,
>
> > > > and send (or stores) the result. Examples: statically analyses,
> > > > Intrusion detection, etc.
> > > >
> > > > Both kind of Runners are useful, frequently used, but the
> > not part
> > > > of the architecture. And as we try to make syslog "better", we
> > > > better add them and make sure out standards can "deal" with
> them.
> > > > Otherwise, non-RFC compliance log-programs will be standard.
> > > >
> > > >
> > > > H4 Syslog format
> > > > ================
> > > >
> > > > 412 enterpriseID
> > > > -----------------
> > > >
> > > > I don't see any reason to include an enterpriseID; not into the
> > > > header. (When needed, it can be used in the structured MSG part)
> > > >
> > > > Currently, it is just a number. It will be unused,
> > misused or will
> > > > lead to a lot of (operational) management. I'm afraid for the
> > > > latter, as in H4.1.3 is suggested  that the semantic of
> > the Facility
> > > > can be enterpriceID depended.
> > > >
> > > > Also, it is required to use the "IANA assigned vendor"
> > number. This
> > > > implies open-source/free/non-commercial are 'ruled out'
> > as the often
> > > > will not to so.
> > > >
> > > > Last, should the number of the Generator-vendor, the
> > system-vendor
> > > > of the device-vendor be used? (See above about
> > > > generator/device) This is not clear to me. And whatever one is
> > > > chosen, it will be hard to implement. Not using the
> > defacto (Unix)
> > > > logging api's!
> > > >
> > > > 413 Facility
> > > > -------------
> > > >
> > > > Although, at first sight, liked the idea of "a terrible lot" of
> > > > facilities. The current <used a number idea> is wrong, as
> > I see it.
> > > > Aside from the problems mentioned above, more then a million
> > > > facilities will mean relays can't be managed! The set of
> > facilities,
> > > > which will be seen in a (major) network, expressed as
> > numbers, will
> > > > be are more or less at random. Which implies very long
> > complex and
> > > > unmanageable configfiles (or MIB's) for each LOG-router!
> > > >
> > > > As we heave learn form routing IPv4, hierarchical structuring is
> > > > needed.
> > > >
> > > > I think, extending the set of facilities is good. But I can't
> > > > imagine more the say 1000 are ever needed.
> > > >
> > > > So, my counter-proposition is:
> > > >         *  Make  facilities (as a number) structured
> > > >         *  Limit the number of facilities to a manageable number
> > > >         *  Keep the format such that extending the
> > allowed numbers
> > > > is possible
> > > >
> > > > A Facility then is still a number, at least 3 (or 4)
> > digits long. A
> > > > longer number means the it is an extended facility. They
> > have to be
> > > > assign by IANA. Facilities of length 3 (4) MUST have the format
> > > > '(K)KLM', where 'K' (or 'KK') indicated the kind of facility;
> 'L'
> > > > give a sub indication and 'M' is
> > > > *SITE* configurable (so, by the local sysadmin, see
> > example below).
> > > > The  'K/KK' is based on the RFC3164 facilities, clean up and
> > > > extended. Those numbers can be IANA assigned. L can be
> > chosen by the
> > > > (generator) vendor, and `M' by the admin. 'M' defaults to
> > 0 (zero),
> > > > and  applications/vendors MAY give the possibility to set that
> > > > digit.
> > > >
> > > > Example: For mail, there will be an K specified, let say
> > 1. Then all
> > > > mail-log will have the format '1XY', which is easily routable.
> It
> > > > will do for small sites. Some vendors, like "sendmail" (only 1
> > > > process) will probably use only one value for L e.g. '0';
> others,
> > > > like "postfix" (several processes) can use multiple values, like
>
> > > > '1', '2' and '3'. When supported by both sendmail and
> > postfix, the
> > > > local sysadmin can add (change) the M-digit, such that
> > mail-systems
> > > > on the border, and internal ones use another facility.
> > > >
> > > > In all cases, the local sysadmin can either use simple
> > > > routing-rules, like 1** (for all mail), or 10* for  sendmail and
>
> > > > 1[1-2]*, 13* for postfix, or even more complex. Now, the
> sysadmin
> > > > has a choice, and can keep it manageable.
> > > >
> > > > Note: the 0 for sendmail and 1-4 for postfix are "by example".
> > > > However, we can add a "rule" that '0' shall be used when only 1
> > > > L-value is used, and when several values are used, zero should
> be
> > > > skipped. Also, I would like to prescribe/reserve "9" for local
> > > > additions. (on all digits).
> > > >
> > > > The (K)KLM idea is used a suggestion to improve, IF this idea is
> > > > accepted, THEN we can discuss variations like 3 or 4
> > digits, where
> > > > to save mappings (IANA of this rfc), etc.
> > > >
> > > > 5 Structured data
> > > > ==================
> > > >
> > > > In short: I thing having the option of structured data is a nice
> > > > option. But lets keep it simple.
> > > >
> > > > The current one is to complex, it has to be as we need 4 pages
> to
> > > > describe it. Also, I find those pages hard to study
> > (given the time
> > > > I had :-). Also it can lead to not implementing it. Programmers,
> > > > especially there bosses, don't have a lot of time!
> > > >
> > > > More positive: The main reason why structured data is complex
> > > > (currently) comes down into 1 problems:
> > > > 1) It isn't part of the main-design (the ABNF on page 9)
> > > > 2) The "structure" can start anywhere in MSG
> > > >
> > > > Both are easy to solve:
> > > > Ad 2) Specify that the structured part ALWAYS START
> > directly after
> > > > the header. Ad 1) We need to introduce it where it belongs. In
> an
> > > > optional field on page 9
> > > >
> > > >
> > > > Let give it a try ( I also use the "improvement" on the
> > ABNF of my
> > > > other mail; it saves typing) (Also I "forget" the SP
> > parts, for now.
> > > > Just the idea)
> > > >
> > > > SYSLOG-MSG  = HEADER DATA
> > > > HEADER      = VERSIONING PRIO ID         // See other mail
> > > > DATA        = [ *STR-DATA ] MSG
> > > > STR-DATA    = see below
> > > > MSG         = free format
> > > >
> > > > Given this ABNF, the structured data ALWAYS comes (in RFC3164
> > > > notation) at the start of the MSG-part (of in new ABNF:
> > before the
> > > > free format MSG).
> > > >
> > > > This implies receivers always have (as last resort) the option
> to
> > > > see everything after the header as free-format. And just
> > > > store/forward it. It implies the start of STR-DATA is simple to
> > > > find: Its starts directly after the header, or directly after
> > > > another STR-DATA
> > > >
> > > > This implies to, we have the option to start STR-DATA
> > with '<' which
> > > > is more usable and XML-alike. The complex long "[@#" cookie
> isn't
> > > > needed anymore. However, we free to use it. My personal
> > vote is for
> > > > the (XML) < > style.
> > > >
> > > > See also my other posting about details of structured-data.
> > > >
> > > > 6 Multi-Part Messages
> > > > =====================
> > > >
> > > > There are some mistakes in the this part, but I like the general
> > > > idea. However, I fell spliting/reassembly is done in
> > other protocols
> > > > too. Maybe we can use/reference a (de facto) standard? I
> > don't know
> > > > an RFC which we can use, but I'm sure there must be one!
> > > >
> > > > Second, it to complex, and to long (to read). I have
> > studied it, but
> > > > I'm not sure I do understand it.  Some details about which I
> find
> > > > hard/wrong/dislike
> > > >
> > > > MP-timestamp
> > > > ------------
> > > > I do not like having several messages having the same TIMESTAMP
> > > >
> > > >
> > > >
> > > > 62 SD-ID receiving an optional STR-DATA
> > > > =======================================
> > > > This must be a mistake!
> > > > In the 3rd paragraph is stated the a receiver sometimes MUST NOT
>
> > > > parse a STR-DATA of a log-message that is received. However,
> when
> > > > the option Multi-part is not implemented, is doesn't now this!
> > > >
> > > >
> > > >
> > > > Hello WG,
> > > >
> > > > This is part 2 (of 3) of my comment on the 3th
> > > > draft-syslog-protocol. See the introduction at my posting
> > 1@3. This
> > > > one contains comment on the text of the document; to help to
> > > > clarify, and shorten the document. It does NOT contain comment
> on
> > > > the "idea" (design); se posting 1@3.
> > > >
> > > > -------------------------------------------------------
> > > >
> > > >
> > > >
> > > > Implementation hints
> > > > ====================
> > > >
> > > > The current RFC contains a lot of valuable hints for
> programmers,
> > > > like the one about time-secfrac (Yes I introduced it, at
> > least the
> > > > bug:-)!
> > > >
> > > > Currently the are scattered around the document, making
> > the document
> > > > long and more complex to read for non-programmers.
> > > >
> > > > I would suggest to move all of them to a new chapter, at
> > the end of
> > > > the document (after the current chapter 9).
> > > >
> > > >
> > > > ABNF (Chapter 4)
> > > > ================
> > > >
> > > > I think we can clarify the syntax, by "unflattening"
> > RGC-3164 syslog
> > > > uses some field and subfields which are nice when
> > introducing syslog
> > > > to others. The "understand" names as priority.
> > > >
> > > > So, keep it structured. I give it a try (please correct
> > the syntax
> > > > of the ABNF, as I not writing it daily anymore)
> > > >
> > > > SYSLOG-MSG  = HEADER DATA
> > > > HEADER      = VERSIONING PRIO ID
> > > > DATA        = [ STR-DATA ] MSG
> > > > VERSIONING  = "V" VERSION
> > > > VERSION     = 1*3 DIGIT
> > > > PRIO        = '<' FACILITY '.' SEVERITY '>'      // See 2@3
> > > > for notation change
> > > > ID          = TIMESTAMP SP HOSTNAME SP TAG
> > > > STR-DATA    = see elsewhere
> > > > MSG         = free format
> > > > (etc)
> > > >
> > > > Now we have meaningful field, that can be used to. E.g
> > the ID field
> > > > (see other mail) make each message unique
> > > >
> > > > 5.1 Format (typo?)
> > > > ==================
> > > > On page 20, the paragraph starting with "The structured
> > data element
> > > > MUST ..." is confusing. The 2nd last line say _no space_
> > is allowed,
> > > > the last one says one or more space are. Is this a typo? Or it
> is
> > > > unclear (at least to me)
> > > >
> > > > Structures data, ID-length
> > > > ==========================
> > > >
> > > > I don't see why we should limit the several field to 64
> > positions. I
> > > > agree this will normally suffice. But so will 32, or 16, of any
> > > > other number. 64 is "to big" to use as a fix-sized ("reserved")
> > > > space for programmer's, database fields, etc. (to big == spoil
> to
> > > > much bytes on huge logs). So dynamic field need to be
> > used anyhow.
> > > > Then there is no need for a trivial maximum. Note, there is a
> > > > maximum anyhow, given by the size of a single
> > log-message. That will
> > > > do for "short term allocation".
> > > >
> > > > Removing this limit, make the rdc cleaner and smaller.
> > > >
> > > > Structured data, spaces
> > > > =======================
> > > >
> > > > I would like to have all line about SP (spaces) in chapter 5
> > > > removed. The point about 0,1 or more spaces is not relevant. In
> > > > general, syslog uses SP to separate field (when needed).
> > And allows
> > > > them in MSG-part. The syntax and semantics of STR-DATA is
> > does not
> > > > depend on the amount of space. Nor is it harder to read.
> > > > Implementing receivers even become easier when spaces (in
> > STR-DATA)
> > > > can be skipped (while { if 'sp' then skip } ) instead of
> checking
> > > > the correct number, and doing something if wrong !
> > > >
> > > > Proposal: Allow spaces anywhere, but inside SD-ID (see note) and
> > > > SD-PARAM. SP in SD-VALUE is allowed (already), but not a
> > separator.
> > > > Prescribe (at least 1) SP between each param-value pair.
> > > >
> > > > *Note: as SP between '[#@'(or '<') and the SP-ID itself
> > is probably
> > > > not a good idea, but not a problem. We can fix is, by moving the
> > > > fixed string in the ABNF:
> > > > STR-DATA    = STR-START ... STR-END
> > > > STR-START   = "[#@" SD-ID   ; or '<'
> > > > STR-END     = "]"           ; or '>'
> > > > ...         = as before, SP are allowed.
> > > >
> > > > Note2: Doing so allows for format line for human reading,
> > which is
> > > > handy
> > > > Example     <x-gam-example  doYoe="like this"  or     = "This
> > > > one?"   >
> > > >             <z-gam-more     Yes  = "I do"      find   = "it
> > > > readable!" >
> > > >
> > > > This example is simple to parse, both for humans and
> > computers. This
> > > > change will make chapter-5 shorter, I think!
> > > >
> > > > Last, I think any whitespace should be allowed instead of SP
> (eg.
> > > TAB)
> > > >
> > > > Chapter-5, MSG
> > > > ==============
> > > >
> > > > I suggest, but only as a detail, the text of chapter-5 should be
> > > > part of 4.2
> > > >
> > > >
> > > >
> > > > Hello WG,
> > > >
> > > > This is part 3 (of 3) of my comment on the 3th
> > > > draft-syslog-protocol. It only contains short remark on
> > details, and
> > > > bit that didn't fit in part 1 (idea/design) or part 2
> > (the document
> > > > itself). See posting 1@3 for more introduction
> > > >
> > > > -------------------------------------------------------
> > > >
> > > > 413/314 FACILITY/SEVERITY
> > > > ==========================
> > > >
> > > > In this draft, both facility and severity are numbers.
> > Even with my
> > > > suggestion, the are 'just numbers'. And numbers are hard
> > to read for
> > > > humans. Especially when the are a lot of them. Most people will
> > > > forget which column contains which number.
> > > >
> > > > Therefore, I think it is better to use the
> > (verbose)notation used by
> > > > most syslog implementations: "'<' facility '.' severity '>'"
> Both
> > > > facility and severity are numbers (at least in the wire).
> > Collectors
> > > > (viewers) can translate those numbers into there names. But
> still
> > > > use this format. And Even without them, it is easier to read the
>
> > > > first (new) then the second (current) line: V1 0 <888.4>
> > > > 2003-010-11T22:14:13.003Z new.Formated. ... V1 0 888 4
> > > > 2003-010-11T22:14:13.003Z old.Formated. ...
> > > >
> > > > Note: I agree, we should not the tricky "8 times F plus
> > S" notation.
> > > > I'm not suggesting that! Just insert a dot and the angles.
> > > >
> > > >
> > > > 4151 timestamp, without time
> > > > ============================
> > > >
> > > > There are 'devices' as meant in this section which
> > haven't an idea
> > > > of TIME. So it is a good idea this section.
> > > >
> > > > Often, those devices can store ("know") a few bit of
> information.
> > > > Therefore, I would like to change this fixed TIMESTAMP,
> > to the same
> > > > one, but with a sequence number attached; the factional-seconds
> > > > field can be used for it.
> > > >
> > > > Then a timestamp becomes 2001-01-01T00:00:60.<seq>Z.
> > > > As in the current draft, this time doesn't exist. But al least,
> > > > collectors can (more or less) sort a set of logmessages form 1
> > > > device
> > > >
> > > > Note: the latter is needed for e.g. syslog-sign
> > > >
> > > >
> > > > 417 TAG
> > > > =======
> > > >
> > > > I think we need to make the TAG stuctured! All current syslog
> > > > receivers (collectors, relays) use __PARTS of__ the
> > RFC3164 TAG to
> > > > route messages. In RFC3164 the TAG is simple and short, so it is
> > > > quite simple to use it for routing. Note: not the complete TAG
> is
> > > > used, only the 'program name', never the PID
> > > part.
> > > >
> > > > With the new TAG, with an static ID an a dynamic part, similar
> > > > routing should be possible. At least, the RFC should be
> > clear on it.
> > > > So, by demanding the static part is "fixed", and make sure that
> > > > static part can be found.
> > > >
> > > > Given the current practice, routing is based (mainly) on the
> > > > program-name, it would be wise to (at least suggest) how
> > > > (where) that part can be found.
> > > >
> > > > Proposal:  Forget native support for VMS/Windows/DOS and
> > even Unix
> > > > pathnames. And introduce an URI (URL)-alike schema. Where
> > only '/'
> > > > (not the one form Unix, but from URL's) is used to "path
> > separation"
> > > > and ':' and '//' are major separators.
> > > >
> > > > In this case, the ABNF is simple; only the semantics
> > become a little
> > > > more complex. Also, it become simple for web-applications to
> log.
> > > > The have an URL already. All "old fashioned:-)"
> > application have an
> > > > URL: ''file://path/to/appl'' already. This is valid on any
> system.
> > > >
> > > > Note: the dynamic part has to be added
> > > > Note2: for web-applictions, which include a 'hostpart' in the
> > > > URL: that hostname is NOT the same as the HOSTPART in the
> header.
> > > > Frequently (a web-farm) several systems share the same
> > URL, but not
> > > > the hostname. Then the sysadmin can decide which one to use for
> > > > routing.
> > > >
> > > >
> > > > Message ID
> > > > ==========
> > > >
> > > > Given the architecture of syslog-networks, messages can be
> > > > duplicated. But, sometimes messages are related (e.g. with the
> > > > signatures of syslog-sign). Both the RFC3164 and the current
> > > > -protocol draft do not have possibilities to unique identify a
> > > > message.
> > > >
> > > > In practice, messages are unique by there hostname, TAG and
> > > > timestamp. But, we can't trust on this, as it isn't required.
> > > >
> > > > I would like to introduce this requirement. It is simple
> > to add to
> > > > the RFC, and simple to implement. Given the HOSTNAME and TAG,
> all
> > > > the implementers have to do is never send a message with the
> same
> > > > timestamp. Given the microsecond resolution, this doable.
> > (It does
> > > > imply some systems have to fake the last digits; I don't see a
> > > > problem with this. Otherwise we can add a "." SequenceNo to the
> > > > timestamp
> > > >
> > > > Structured data, tokens
> > > > =======================
> > > >
> > > > Given the current draft, of my counter proposal of it, of
> > structured
> > > > data, the are (only) 2 kinds of tokens. The IANA
> > controlled ones and
> > > > the experimental ones; the latter starting with "x-"
> > > >
> > > > I think, we can safely add an third: "X-" for
> > private/local/vendor
> > > > specific tokens. As we can see my e.g. mail, this kind of
> > field will
> > > > be used a lot. Now we have an option to allow the, without
> giving
> > > > the a status of "testing".
> > > >
> > > >
> > > > STR-DATA, can we use it for syslog-sign (or similar)?
> > > > =====================================================
> > > >
> > > > Just an idea: in a protocol as syslog-sign (here just as
> > example),
> > > > where messages reference to other messages (now
> > implicit), we could
> > > > use the STR-DATA to do so?
> > > >
> > > > I verified the syntax/semantics of this, and YES, we could do
> so.
> > > > This means, I like STR-DATA a lot more :-) It is great. Even
> when
> > > > -sign doesn't use it, I (we) can use the same format to
> > present it
> > > > to the user!
> > > >
> > > > 64 MultiPart examples
> > > > =====================
> > > >
> > > > All examples use rfc3164 headers, shouldn't -protocol headers be
> > > used?
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
>
>
>
>

------------------------------

Date: Tue, 13 Apr 2004 16:09:16 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: fragmentation in transport

Rainer, my intention was definitely to specify the extra header in the
transport layer, not syslog-protocol. Other transports don't have to
use this header, and there is no reliance on syslog-protocol at all in
transport.  Sorry, if I did not make it clear.

> I have to admit that while I like support for large size
> message, I think we should not try to make this feature
> mandatory... 99,999% of the messages will never need it.

I am not sure if it is great for a standard if some things are
optional to implement. I think it adds to ambiguity. Somebody who says
they support RFC-transport would have to qualify it with the specific
things they support or don't. Then half of implementations will not
support it and we are back to square one where we can't reliably use
large messages and hope for vendor-neutral tools.

But I do agree that implementation should not impose large overhead on
small messages just because it needs to support large ones which
happen once in a blue moon. In my proposal the overhead is just one
byte for small messages - hardly significant. We are wasting much more
by specifying things like facility or IP in ASCII instead of a binary.

BTW, in either case, in syslog-transport we have to specify or
recommend the message size limit for non-fragmented messages. I think
the limit for non-fragmented messages should be recommended to be
pretty low. Probably the size of minimum MTU that must be supported
(~560 bytes minus headers for IPv4 and ~1280 for IPv6).  Large UDP
messages do not work reliably in many implementations.  In fact, I
think in Stevens book he made tests with various IP implementations
and most were not working with messages bigger than 30k. TFTP uses 512
byte UDP packets, for example.

Thanks,
Anton.

> -----Original Message-----
> From: Rainer Gerhards [mailto:[email-masked]]
> Sent: Tuesday, April 13, 2004 2:57 PM
> To: Anton Okmianski; [email-masked]
> Subject: RE: fragmentation in transport
>
>
> This is a not-well-thought out response, but I thought it is
> better to provide it than to wait for 2 more days (Which
> otherwise might happen). I think if we move multi-part
> messages into the transport, we MUST NOT modify the header.
> Instead, each transport should add its own header, where it
> requires. If we specify the header in -protocol, we are back
> at where we are currently, that is that -protocol specifies
> the multi-part messaging. If so, I'd prefer that we do it as
> it is specified so far, because multi-part messages have low
> overhead ... Because we assummed that they would be needed
> only very infrequently.
>
> I have to admit that while I like support for large size
> message, I think we should not try to make this feature
> mandatory... 99,999% of the messages will never need it.
>
> Rainer
>
> > -----Original Message-----
> > From: Anton Okmianski [mailto:[email-masked]]
> > Sent: Tuesday, April 13, 2004 6:48 PM
> > To: [email-masked]
> > Subject: fragmentation in transport
> >
> > Rainer & all:
> >
> > I started thinking about the best way of specifying message
> > fragmentation in the UDP transport layer. I don't think our
present
> > scheme is ideal for good protocol layer isolation because it would

> > require transport layer to be able to parse protocol
> message in order
> > to read/modify structured content tag "msgpart".  I think
> it is better
> > if transport could deal with syslog messages as generic payload.
> >
> > I propose that we stick to a definition of fragmentation in the
> > transport layer that is more agnostic to syslog-protocol format
and
> > only manipulates the header.  Specifically, I propose we
> add a header
> > that mimics fragmentation fields of IP (RFC791) follows,
> except with
> > bigger field sizes:
> >
> > - multi-part-flag (1 byte)
> > - msg-id (2 bytes)
> > - fragment-length (32 bytes)
> > - fragment-offset (32 bytes)
> > - more-fragments-flag (1 byte)
> >
> > All in all this will be an additional 67 byte header and
> would enable
> > transferring payload of up to 4GB.  If multi-part-flag is 0, we
can
> > omit the rest of the header.
> >
> > The trickiest part is the msg-id field. I propose that we
> specify that
> > fragment reassembly is done using source IP, source port and
msg-id
> > field. This means that we will place a restriction that each
> > application must send all messages using a single random exclusive

> > source port (no SO_REUSEADDR flag), while changing msg-id for
every
> > message. So, msg-id only has to be unique within a given process
> > because each process will use a different source port.  To avoid a

> > problem with mix up due to port re-use with short-lived
> processes, I
> > propose that we recommend that apps start msg-id at a random
number
> > and then increment and cycle it. This would reduce the hole
> to mostly
> > theoretical domain. The idea of using ports to identify
> applications
> > is consistent with what they were designed to accomplish in
> the first
> > place.
> >
> > IPv6 includes some a separate RFC2675 for jumbo datagrams,
> which could
> > be used to transfer large packets.  But it seems they
> restricted it to
> > the networks with large MTU sizes only. I am also worried that
> > adoption of this standard will lag the other IPv6 standards and it

> > would be risky for a syslog protocol to rely on
> intermediary entities
> > supporting it. So, we probably won't be able to use it and
> my proposal
> > is to use the same mechanism for UDP/IPv4 and UDP/IPv6.
> >
> > Please let me know what you think before I write it up into
> the draft.
> >
> > Thanks,
> > Anton.
> >
> > > -----Original Message-----
> > > From: [email-masked]
> > > [mailto:[email-masked]] On Behalf Of
> Rainer Gerhards
> > > Sent: Friday, April 02, 2004 3:29 AM
> > > To: Anton Okmianski; ons-huis.net!ALbert;
[email-masked]
> > > Subject: RE: some (well a lot ) review remarks
> > >
> > >
> > > Anton,
> > >
> > > I have only minor concerns with moving this into the
> transport. But
> > > if we move it into the transport, then we must do it
> right, which in
> > > my point of view means NO size restriction (no, 64K is
> handy for UDP
> > > fragmentation, but I would also like to be able to send 1 MB if
> > > there is really need for this...). My concern is when we
> move this
> > > into the transport, the upper layer will probably need to
> be able to
> > > handle full-size messages, which could easily be turned into
> > > a DoS - on the other hand, we are able to work around this
> > > (like specify a way to access frames instead of full messages).
> > >
> > > If we do this right, I see indeed clear advantages of
> pushing this
> > > into the transport. for the UDP transport, I would still
> propose to
> > > use what is currently specified in -protocol.
> > >
> > > Any more comments?
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: Anton Okmianski [mailto:[email-masked]]
> > > > Sent: Thursday, April 01, 2004 9:30 PM
> > > > To: 'ons-huis.net!ALbert'; [email-masked]; Rainer
> > Gerhards
> > > > Subject: RE: some (well a lot ) review remarks
> > > >
> > > > Albert, Rainer, WG:
> > > >
> > > > I have not thought through all the details, but...
> > > >
> > > > I share Albert's concern about defining message length
> limits and
> > > > mandating partitioning in syslog-protocol given that it is
> > > supposed to
> > > > be transport independent.  I'd prefer if
> syslog-protocol was just
> > > > about the format and encoding of the message payload.
> > > Payload being a
> > > > complete syslog message without any transport-specific parts.
> > > >
> > > > As it stands now, syslog-transport is really about
> > > transferring syslog
> > > > message segments and not complete syslog messages.  If
> > > > syslog-transport does all segmentation, then it is easier
> > > to specify
> > > > how to avoid issues such as double-segmentation
> > > (syslog-protocol level
> > > > and then IP fragmentation).
> > > >
> > > > The relay provisions should probably also be specified only
> > > as far as
> > > > payload is concerned. For example in DHCP, when message
> > > goes through
> > > > relay, relay agent is allowed to insert an option
> specifying its
> > > > address as well as giaddr (the network from which
> request for IP
> > > > originated from).  So, it just specifies additional
> provisions for
> >
> > > > payload for cases with relay agents and that's it. I think
> > > it will be
> > > > easiest if we allowed relay agent to function like any
> > > > syslog-transport client and re-assemble the whole message
> > > before doing
> > > > anything with it. In other words, it should work with
> > > syslog messages,
> > > > not syslog message segments.
> > > >
> > > > Just my 2 cents.
> > > >
> > > > Anton.
> > > >
> > > > > -----Original Message-----
> > > > > From: [email-masked]
> > > > > [mailto:[email-masked]] On Behalf Of
> > > > > ons-huis.net!ALbert
> > > > > Sent: Tuesday, March 30, 2004 11:59 AM
> > > > > To: [email-masked]; [email-masked]
> > > > > Subject: some (well a lot ) review remarks
> > > > >
> > > > >
> > > > > Hello Rainer, Anton, WG
> > > > >
> > > > > Last weeks I have reviewed some of the current draft
> documents.
> > I
> > > > > had planned to spend more time on it, coming month. But,
> > > plans are
> > > > > changed. It will be very busy, which is good .. But also
> > > means less
> > > > > tome for syslog.
> > > > >
> > > > >
> > > > > I have one important "issue": I think it would be a great
> > > > > improvement if some parts of Rainer's document would move to

> > > > > Anton's. I think the protocol document should NOT
> specify how to
> >
> > > > > break messages in parts. It should assume the transport
layer
> > can
> > > > > "transport" 'long messages'. The transport layer, should
> > > when a long
> > > > > message can't be send in once, spilt in is several parts.
> > > > >
> > > > > I think it should be possible to write another
> > > 'transport' document,
> > > > > which describes how 'long, new, better' messages can be
> > > transported
> > > > > by rfc-3164 syslog messages.
> > > > > Note: I'm NOT saying we should make an RFC for this. Only
> > > that, if
> > > > > we could write it, the separation between transport
> and  "upper"
> > > > > layers are better. Note 2: That document should
> describe how to
> > > > > 'shrink' the longer headers into the old headers, etc. Not
by
> > > > > putting the complete message into the payload.
> > > > >
> > > > > --------------
> > > > > Aside from the point above, I will include a raider
> long list of
> > > > > (personal) notes 'asis'.
> > > > >
> > > > > Rainer, please use those notes when relevant, Skip the
> > > others. I'm
> > > > > sorry, I don't have more time to rewrite my notes to a
> > > good review.
> > > > > It is either me to throw away them, of give another that
> > > option.:-)
> > > > >
> > > > > Hope it helps a bit.
> > > > >
> > > > > ===========================================================
> > > > >
> > > > > --
> > > > > Groetjes,
> > > > > --ALbert Mietus
> > > > > Private mail to:           albert at ons-huis dot net
> > > > > Business mail to:     albert dot mietus at PTS dot nl
> > > > > Spam:   Just don't do it! Thrust me, I will not order!
> > > > >
> > > > > Hello WG,
> > > > >
> > > > > This is my comment on Rainer's syslog-protcol draft 04. I
> > > have been
> > > > > very passive on following this mailinglist; (to
> > > > > busy:-) But on receiving a request to review I decided to
> > > do so. But
> > > > > I apologise if I raise issues that are discussed already.
> > > I didn't
> > > > > read them. This week I started, after a log time, to
> study the
> > > > > current draft document(s).
> > > > >
> > > > > Before continuing, let me say I like the idea of
> > > splitting syslog in
> > > > > several layers. I say so, in the hope the individual
> > > > > sub-specifications will become short. My comment is
> split into 3
> >
> > > > > parts, of which you are reading part 1 now. Part 1 is
> about the
> > > > > "idea" (the design). In a separate mail I will comment on
> > > the text
> > > > > of the document; in the hope I will become clearer,
> cleaner and
> > > > > shorter. The 3rd mail will contain some details and bit
> > > that didn't
> > > > > fit in the others.
> > > > >
> > > > > Possible, I will send more comment in futures mail; it
> > > will depend
> > > > > on the amount of time I have now, and then.
> > > > >
> > > > > -------------------------------------------------------
> > > > >
> > > > > This review is becoming (very) long, I really hope all
> > > comments are
> > > > > worth reading. I have tried to express my thoughts about
> > > is a well
> > > > > as possible, given the time I have...
> > > > >
> > > > > General Comment on the idea/design of syslog-protocol
> > > > > =====================================================
> > > > >
> > > > > * This idea is GOOD!
> > > > >
> > > > >
> > > > >
> > > > > H2 Architecture
> > > > > ================
> > > > >
> > > > > Traditional, syslog knows Devices, Collectors and Relays. I
> > would
> > > > > like to add two 'things'. One I would like to call a
> > > > > *Generator* , the other an *Runner*.
> > > > >
> > > > > *Generator*
> > > > > As we can see in e.g. most Unix implementations, it is the
> > > > > application that knows WHAT to log, transmit that to the
> > > system (the
> > > > > log device, the syslogdaemon) that know HOW to log
> it. Also on
> > > > > embedded systems, this can be seen. Historically, the
> > > combination of
> > > > > (a part of the) application and the "system" form the
> > > log-device. I
> > > > > would like to split this, now this opportunity exits. The
> > > part that
> > > > > is build-in in the application, (in C: the lines
> syslog(..."hai
> > > > > there");) can be called  the (LOG-)Generator. The
> communication
> > > > > between generator and Device is system/platform depended. On
> > Unix
> > > > > systems usually the log-device, on embedded systems a
> > > > > function-call, and on windows log-events can be used.
> > > > >
> > > > > By introducing this Generator, we (can) make clear this
> > > > > private/dedicated communication exist; and is allowed. We
> > > also make
> > > > > clear the Generator is syslog-protocol INDEPENDED.
> > > > >
> > > > > The function of the Device, know becomes clear: it get
> > > log-message
> > > > > (-events) and transport them. It also does some bookkeeping,
> > like
> > > > > timestamping, adding crypto (for -sign), etc.
> > > > >
> > > > > *Runner*
> > > > > A LOG-runner is a other kind of syslog-thing, which is
> > frequently
> > > > > used. Without a proper place in the architecture. Whereas a
> > relay
> > > > > (should) forwards syslog-messages without knowledge of
> > > the semantics
> > > > > of the message, the Runner does. The most simple
> life-form of a
> > > > > runner is a filter. It "relays" messages, but only
> when the are
> > > > > important. On Unix there a several of these in perl,
> > > grep-scripts
> > > > > etc. Formally, the are knot relays (I think). A more complex

> > > > > runner, is a "program" that receives log-messages,
> CHANGES them,
> >
> > > > > and send (or stores) the result. Examples: statically
> analyses,
> > > > > Intrusion detection, etc.
> > > > >
> > > > > Both kind of Runners are useful, frequently used, but the
> > > not part
> > > > > of the architecture. And as we try to make syslog
> "better", we
> > > > > better add them and make sure out standards can "deal" with
> > them.
> > > > > Otherwise, non-RFC compliance log-programs will be standard.
> > > > >
> > > > >
> > > > > H4 Syslog format
> > > > > ================
> > > > >
> > > > > 412 enterpriseID
> > > > > -----------------
> > > > >
> > > > > I don't see any reason to include an enterpriseID;
> not into the
> > > > > header. (When needed, it can be used in the
> structured MSG part)
> > > > >
> > > > > Currently, it is just a number. It will be unused,
> > > misused or will
> > > > > lead to a lot of (operational) management. I'm afraid for
the
> > > > > latter, as in H4.1.3 is suggested  that the semantic of
> > > the Facility
> > > > > can be enterpriceID depended.
> > > > >
> > > > > Also, it is required to use the "IANA assigned vendor"
> > > number. This
> > > > > implies open-source/free/non-commercial are 'ruled out'
> > > as the often
> > > > > will not to so.
> > > > >
> > > > > Last, should the number of the Generator-vendor, the
> > > system-vendor
> > > > > of the device-vendor be used? (See above about
> > > > > generator/device) This is not clear to me. And
> whatever one is
> > > > > chosen, it will be hard to implement. Not using the
> > > defacto (Unix)
> > > > > logging api's!
> > > > >
> > > > > 413 Facility
> > > > > -------------
> > > > >
> > > > > Although, at first sight, liked the idea of "a
> terrible lot" of
> > > > > facilities. The current <used a number idea> is wrong, as
> > > I see it.
> > > > > Aside from the problems mentioned above, more then a million

> > > > > facilities will mean relays can't be managed! The set of
> > > facilities,
> > > > > which will be seen in a (major) network, expressed as
> > > numbers, will
> > > > > be are more or less at random. Which implies very long
> > > complex and
> > > > > unmanageable configfiles (or MIB's) for each LOG-router!
> > > > >
> > > > > As we heave learn form routing IPv4, hierarchical
> structuring is
> > > > > needed.
> > > > >
> > > > > I think, extending the set of facilities is good. But I
can't
> > > > > imagine more the say 1000 are ever needed.
> > > > >
> > > > > So, my counter-proposition is:
> > > > >         *  Make  facilities (as a number) structured
> > > > >         *  Limit the number of facilities to a
> manageable number
> > > > >         *  Keep the format such that extending the
> > > allowed numbers
> > > > > is possible
> > > > >
> > > > > A Facility then is still a number, at least 3 (or 4)
> > > digits long. A
> > > > > longer number means the it is an extended facility. They
> > > have to be
> > > > > assign by IANA. Facilities of length 3 (4) MUST have
> the format
> > > > > '(K)KLM', where 'K' (or 'KK') indicated the kind of
facility;
> > 'L'
> > > > > give a sub indication and 'M' is
> > > > > *SITE* configurable (so, by the local sysadmin, see
> > > example below).
> > > > > The  'K/KK' is based on the RFC3164 facilities, clean up and

> > > > > extended. Those numbers can be IANA assigned. L can be
> > > chosen by the
> > > > > (generator) vendor, and `M' by the admin. 'M' defaults to
> > > 0 (zero),
> > > > > and  applications/vendors MAY give the possibility to
> set that
> > > > > digit.
> > > > >
> > > > > Example: For mail, there will be an K specified, let say
> > > 1. Then all
> > > > > mail-log will have the format '1XY', which is easily
routable.
> > It
> > > > > will do for small sites. Some vendors, like "sendmail" (only
1
> > > > > process) will probably use only one value for L e.g. '0';
> > others,
> > > > > like "postfix" (several processes) can use multiple
> values, like
> >
> > > > > '1', '2' and '3'. When supported by both sendmail and
> > > postfix, the
> > > > > local sysadmin can add (change) the M-digit, such that
> > > mail-systems
> > > > > on the border, and internal ones use another facility.
> > > > >
> > > > > In all cases, the local sysadmin can either use simple
> > > > > routing-rules, like 1** (for all mail), or 10* for
> sendmail and
> >
> > > > > 1[1-2]*, 13* for postfix, or even more complex. Now, the
> > sysadmin
> > > > > has a choice, and can keep it manageable.
> > > > >
> > > > > Note: the 0 for sendmail and 1-4 for postfix are "by
> example".
> > > > > However, we can add a "rule" that '0' shall be used
> when only 1
> > > > > L-value is used, and when several values are used, zero
should
> > be
> > > > > skipped. Also, I would like to prescribe/reserve "9"
> for local
> > > > > additions. (on all digits).
> > > > >
> > > > > The (K)KLM idea is used a suggestion to improve, IF
> this idea is
> > > > > accepted, THEN we can discuss variations like 3 or 4
> > > digits, where
> > > > > to save mappings (IANA of this rfc), etc.
> > > > >
> > > > > 5 Structured data
> > > > > ==================
> > > > >
> > > > > In short: I thing having the option of structured
> data is a nice
> > > > > option. But lets keep it simple.
> > > > >
> > > > > The current one is to complex, it has to be as we need 4
pages
> > to
> > > > > describe it. Also, I find those pages hard to study
> > > (given the time
> > > > > I had :-). Also it can lead to not implementing it.
> Programmers,
> > > > > especially there bosses, don't have a lot of time!
> > > > >
> > > > > More positive: The main reason why structured data is
complex
> > > > > (currently) comes down into 1 problems:
> > > > > 1) It isn't part of the main-design (the ABNF on page 9)
> > > > > 2) The "structure" can start anywhere in MSG
> > > > >
> > > > > Both are easy to solve:
> > > > > Ad 2) Specify that the structured part ALWAYS START
> > > directly after
> > > > > the header. Ad 1) We need to introduce it where it belongs.
In
> > an
> > > > > optional field on page 9
> > > > >
> > > > >
> > > > > Let give it a try ( I also use the "improvement" on the
> > > ABNF of my
> > > > > other mail; it saves typing) (Also I "forget" the SP
> > > parts, for now.
> > > > > Just the idea)
> > > > >
> > > > > SYSLOG-MSG  = HEADER DATA
> > > > > HEADER      = VERSIONING PRIO ID         // See other mail
> > > > > DATA        = [ *STR-DATA ] MSG
> > > > > STR-DATA    = see below
> > > > > MSG         = free format
> > > > >
> > > > > Given this ABNF, the structured data ALWAYS comes (in
RFC3164
> > > > > notation) at the start of the MSG-part (of in new ABNF:
> > > before the
> > > > > free format MSG).
> > > > >
> > > > > This implies receivers always have (as last resort) the
option
> > to
> > > > > see everything after the header as free-format. And just
> > > > > store/forward it. It implies the start of STR-DATA is
> simple to
> > > > > find: Its starts directly after the header, or directly
after
> > > > > another STR-DATA
> > > > >
> > > > > This implies to, we have the option to start STR-DATA
> > > with '<' which
> > > > > is more usable and XML-alike. The complex long "[@#" cookie
> > isn't
> > > > > needed anymore. However, we free to use it. My personal
> > > vote is for
> > > > > the (XML) < > style.
> > > > >
> > > > > See also my other posting about details of structured-data.
> > > > >
> > > > > 6 Multi-Part Messages
> > > > > =====================
> > > > >
> > > > > There are some mistakes in the this part, but I like
> the general
> > > > > idea. However, I fell spliting/reassembly is done in
> > > other protocols
> > > > > too. Maybe we can use/reference a (de facto) standard? I
> > > don't know
> > > > > an RFC which we can use, but I'm sure there must be one!
> > > > >
> > > > > Second, it to complex, and to long (to read). I have
> > > studied it, but
> > > > > I'm not sure I do understand it.  Some details about which I
> > find
> > > > > hard/wrong/dislike
> > > > >
> > > > > MP-timestamp
> > > > > ------------
> > > > > I do not like having several messages having the same
> TIMESTAMP
> > > > >
> > > > >
> > > > >
> > > > > 62 SD-ID receiving an optional STR-DATA
> > > > > =======================================
> > > > > This must be a mistake!
> > > > > In the 3rd paragraph is stated the a receiver
> sometimes MUST NOT
> >
> > > > > parse a STR-DATA of a log-message that is received. However,
> > when
> > > > > the option Multi-part is not implemented, is doesn't now
this!
> > > > >
> > > > >
> > > > >
> > > > > Hello WG,
> > > > >
> > > > > This is part 2 (of 3) of my comment on the 3th
> > > > > draft-syslog-protocol. See the introduction at my posting
> > > 1@3. This
> > > > > one contains comment on the text of the document; to help to

> > > > > clarify, and shorten the document. It does NOT contain
comment
> > on
> > > > > the "idea" (design); se posting 1@3.
> > > > >
> > > > > -------------------------------------------------------
> > > > >
> > > > >
> > > > >
> > > > > Implementation hints
> > > > > ====================
> > > > >
> > > > > The current RFC contains a lot of valuable hints for
> > programmers,
> > > > > like the one about time-secfrac (Yes I introduced it, at
> > > least the
> > > > > bug:-)!
> > > > >
> > > > > Currently the are scattered around the document, making
> > > the document
> > > > > long and more complex to read for non-programmers.
> > > > >
> > > > > I would suggest to move all of them to a new chapter, at
> > > the end of
> > > > > the document (after the current chapter 9).
> > > > >
> > > > >
> > > > > ABNF (Chapter 4)
> > > > > ================
> > > > >
> > > > > I think we can clarify the syntax, by "unflattening"
> > > RGC-3164 syslog
> > > > > uses some field and subfields which are nice when
> > > introducing syslog
> > > > > to others. The "understand" names as priority.
> > > > >
> > > > > So, keep it structured. I give it a try (please correct
> > > the syntax
> > > > > of the ABNF, as I not writing it daily anymore)
> > > > >
> > > > > SYSLOG-MSG  = HEADER DATA
> > > > > HEADER      = VERSIONING PRIO ID
> > > > > DATA        = [ STR-DATA ] MSG
> > > > > VERSIONING  = "V" VERSION
> > > > > VERSION     = 1*3 DIGIT
> > > > > PRIO        = '<' FACILITY '.' SEVERITY '>'      // See 2@3
> > > > > for notation change
> > > > > ID          = TIMESTAMP SP HOSTNAME SP TAG
> > > > > STR-DATA    = see elsewhere
> > > > > MSG         = free format
> > > > > (etc)
> > > > >
> > > > > Now we have meaningful field, that can be used to. E.g
> > > the ID field
> > > > > (see other mail) make each message unique
> > > > >
> > > > > 5.1 Format (typo?)
> > > > > ==================
> > > > > On page 20, the paragraph starting with "The structured
> > > data element
> > > > > MUST ..." is confusing. The 2nd last line say _no space_
> > > is allowed,
> > > > > the last one says one or more space are. Is this a typo? Or
it
> > is
> > > > > unclear (at least to me)
> > > > >
> > > > > Structures data, ID-length
> > > > > ==========================
> > > > >
> > > > > I don't see why we should limit the several field to 64
> > > positions. I
> > > > > agree this will normally suffice. But so will 32, or
> 16, of any
> > > > > other number. 64 is "to big" to use as a fix-sized
> ("reserved")
> > > > > space for programmer's, database fields, etc. (to big ==
spoil
> > to
> > > > > much bytes on huge logs). So dynamic field need to be
> > > used anyhow.
> > > > > Then there is no need for a trivial maximum. Note, there is
a
> > > > > maximum anyhow, given by the size of a single
> > > log-message. That will
> > > > > do for "short term allocation".
> > > > >
> > > > > Removing this limit, make the rdc cleaner and smaller.
> > > > >
> > > > > Structured data, spaces
> > > > > =======================
> > > > >
> > > > > I would like to have all line about SP (spaces) in chapter 5

> > > > > removed. The point about 0,1 or more spaces is not
> relevant. In
> > > > > general, syslog uses SP to separate field (when needed).
> > > And allows
> > > > > them in MSG-part. The syntax and semantics of STR-DATA is
> > > does not
> > > > > depend on the amount of space. Nor is it harder to read.
> > > > > Implementing receivers even become easier when spaces (in
> > > STR-DATA)
> > > > > can be skipped (while { if 'sp' then skip } ) instead of
> > checking
> > > > > the correct number, and doing something if wrong !
> > > > >
> > > > > Proposal: Allow spaces anywhere, but inside SD-ID
> (see note) and
> > > > > SD-PARAM. SP in SD-VALUE is allowed (already), but not a
> > > separator.
> > > > > Prescribe (at least 1) SP between each param-value pair.
> > > > >
> > > > > *Note: as SP between '[#@'(or '<') and the SP-ID itself
> > > is probably
> > > > > not a good idea, but not a problem. We can fix is, by
> moving the
> > > > > fixed string in the ABNF:
> > > > > STR-DATA    = STR-START ... STR-END
> > > > > STR-START   = "[#@" SD-ID   ; or '<'
> > > > > STR-END     = "]"           ; or '>'
> > > > > ...         = as before, SP are allowed.
> > > > >
> > > > > Note2: Doing so allows for format line for human reading,
> > > which is
> > > > > handy
> > > > > Example     <x-gam-example  doYoe="like this"  or     =
"This
> > > > > one?"   >
> > > > >             <z-gam-more     Yes  = "I do"      find   = "it
> > > > > readable!" >
> > > > >
> > > > > This example is simple to parse, both for humans and
> > > computers. This
> > > > > change will make chapter-5 shorter, I think!
> > > > >
> > > > > Last, I think any whitespace should be allowed instead of SP
> > (eg.
> > > > TAB)
> > > > >
> > > > > Chapter-5, MSG
> > > > > ==============
> > > > >
> > > > > I suggest, but only as a detail, the text of
> chapter-5 should be
> > > > > part of 4.2
> > > > >
> > > > >
> > > > >
> > > > > Hello WG,
> > > > >
> > > > > This is part 3 (of 3) of my comment on the 3th
> > > > > draft-syslog-protocol. It only contains short remark on
> > > details, and
> > > > > bit that didn't fit in part 1 (idea/design) or part 2
> > > (the document
> > > > > itself). See posting 1@3 for more introduction
> > > > >
> > > > > -------------------------------------------------------
> > > > >
> > > > > 413/314 FACILITY/SEVERITY
> > > > > ==========================
> > > > >
> > > > > In this draft, both facility and severity are numbers.
> > > Even with my
> > > > > suggestion, the are 'just numbers'. And numbers are hard
> > > to read for
> > > > > humans. Especially when the are a lot of them. Most
> people will
> > > > > forget which column contains which number.
> > > > >
> > > > > Therefore, I think it is better to use the
> > > (verbose)notation used by
> > > > > most syslog implementations: "'<' facility '.' severity '>'"
> > Both
> > > > > facility and severity are numbers (at least in the wire).
> > > Collectors
> > > > > (viewers) can translate those numbers into there names. But
> > still
> > > > > use this format. And Even without them, it is easier
> to read the
> >
> > > > > first (new) then the second (current) line: V1 0 <888.4>
> > > > > 2003-010-11T22:14:13.003Z new.Formated. ... V1 0 888 4
> > > > > 2003-010-11T22:14:13.003Z old.Formated. ...
> > > > >
> > > > > Note: I agree, we should not the tricky "8 times F plus
> > > S" notation.
> > > > > I'm not suggesting that! Just insert a dot and the angles.
> > > > >
> > > > >
> > > > > 4151 timestamp, without time ============================
> > > > >
> > > > > There are 'devices' as meant in this section which
> > > haven't an idea
> > > > > of TIME. So it is a good idea this section.
> > > > >
> > > > > Often, those devices can store ("know") a few bit of
> > information.
> > > > > Therefore, I would like to change this fixed TIMESTAMP,
> > > to the same
> > > > > one, but with a sequence number attached; the
> factional-seconds
> > > > > field can be used for it.
> > > > >
> > > > > Then a timestamp becomes 2001-01-01T00:00:60.<seq>Z.
> > > > > As in the current draft, this time doesn't exist. But
> al least,
> > > > > collectors can (more or less) sort a set of
> logmessages form 1
> > > > > device
> > > > >
> > > > > Note: the latter is needed for e.g. syslog-sign
> > > > >
> > > > >
> > > > > 417 TAG
> > > > > =======
> > > > >
> > > > > I think we need to make the TAG stuctured! All current
syslog
> > > > > receivers (collectors, relays) use __PARTS of__ the
> > > RFC3164 TAG to
> > > > > route messages. In RFC3164 the TAG is simple and
> short, so it is
> > > > > quite simple to use it for routing. Note: not the complete
TAG
> > is
> > > > > used, only the 'program name', never the PID
> > > > part.
> > > > >
> > > > > With the new TAG, with an static ID an a dynamic
> part, similar
> > > > > routing should be possible. At least, the RFC should be
> > > clear on it.
> > > > > So, by demanding the static part is "fixed", and make
> sure that
> > > > > static part can be found.
> > > > >
> > > > > Given the current practice, routing is based (mainly) on the

> > > > > program-name, it would be wise to (at least suggest) how
> > > > > (where) that part can be found.
> > > > >
> > > > > Proposal:  Forget native support for VMS/Windows/DOS and
> > > even Unix
> > > > > pathnames. And introduce an URI (URL)-alike schema. Where
> > > only '/'
> > > > > (not the one form Unix, but from URL's) is used to "path
> > > separation"
> > > > > and ':' and '//' are major separators.
> > > > >
> > > > > In this case, the ABNF is simple; only the semantics
> > > become a little
> > > > > more complex. Also, it become simple for web-applications to
> > log.
> > > > > The have an URL already. All "old fashioned:-)"
> > > application have an
> > > > > URL: ''file://path/to/appl'' already. This is valid on any
> > system.
> > > > >
> > > > > Note: the dynamic part has to be added
> > > > > Note2: for web-applictions, which include a 'hostpart' in
the
> > > > > URL: that hostname is NOT the same as the HOSTPART in the
> > header.
> > > > > Frequently (a web-farm) several systems share the same
> > > URL, but not
> > > > > the hostname. Then the sysadmin can decide which one
> to use for
> > > > > routing.
> > > > >
> > > > >
> > > > > Message ID
> > > > > ==========
> > > > >
> > > > > Given the architecture of syslog-networks, messages can be
> > > > > duplicated. But, sometimes messages are related (e.g.
> with the
> > > > > signatures of syslog-sign). Both the RFC3164 and the current

> > > > > -protocol draft do not have possibilities to unique
> identify a
> > > > > message.
> > > > >
> > > > > In practice, messages are unique by there hostname, TAG and
> > > > > timestamp. But, we can't trust on this, as it isn't
required.
> > > > >
> > > > > I would like to introduce this requirement. It is simple
> > > to add to
> > > > > the RFC, and simple to implement. Given the HOSTNAME and
TAG,
> > all
> > > > > the implementers have to do is never send a message with the
> > same
> > > > > timestamp. Given the microsecond resolution, this doable.
> > > (It does
> > > > > imply some systems have to fake the last digits; I
> don't see a
> > > > > problem with this. Otherwise we can add a "."
> SequenceNo to the
> > > > > timestamp
> > > > >
> > > > > Structured data, tokens
> > > > > =======================
> > > > >
> > > > > Given the current draft, of my counter proposal of it, of
> > > structured
> > > > > data, the are (only) 2 kinds of tokens. The IANA
> > > controlled ones and
> > > > > the experimental ones; the latter starting with "x-"
> > > > >
> > > > > I think, we can safely add an third: "X-" for
> > > private/local/vendor
> > > > > specific tokens. As we can see my e.g. mail, this kind of
> > > field will
> > > > > be used a lot. Now we have an option to allow the, without
> > giving
> > > > > the a status of "testing".
> > > > >
> > > > >
> > > > > STR-DATA, can we use it for syslog-sign (or similar)?
> > > > > =====================================================
> > > > >
> > > > > Just an idea: in a protocol as syslog-sign (here just as
> > > example),
> > > > > where messages reference to other messages (now
> > > implicit), we could
> > > > > use the STR-DATA to do so?
> > > > >
> > > > > I verified the syntax/semantics of this, and YES, we could
do
> > so.
> > > > > This means, I like STR-DATA a lot more :-) It is great. Even
> > when
> > > > > -sign doesn't use it, I (we) can use the same format to
> > > present it
> > > > > to the user!
> > > > >
> > > > > 64 MultiPart examples
> > > > > =====================
> > > > >
> > > > > All examples use rfc3164 headers, shouldn't -protocol
> headers be
> > > > used?
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
> >
> >
>

------------------------------

Date: Thu, 15 Apr 2004 17:34:58 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -protocol issue 15: Describe Relay Mode?

Hi WG,

I would like to once again drag your attention to the question if we
should define syslog relaying in -protocol. This is tracked as issue 15
at

http://www.syslog.cc/ietf/protocol/issue15.html

I think we should describe relay mode in a different document. As I
received no other response to my previous question on this topic, I
think there is agreement that this should be done in a separate
document. Thus, I will specify it as such if there is no further
objection (I am about to do the next edit).

Rainer

------------------------------

Date: Thu, 15 Apr 2004 14:42:29 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: -protocol issue 15: Describe Relay Mode?

Rainer:

No objections. But I would consider not wasting time on a separate
document either.

I think describing relay *operations* may not be necessary at all.
Relay can be though of as just a different receiver and, as far as
syslog-protocol is concerned, it is just part of the transport to the
eventual receiver. The focus of syslog-protocol should be on the
message format and semantics of the fields only.  We just have to make
sure that we provide for enough fields (including structured content)
to remove any dependence on using information from the lower layers
(like IP packet information). If this done, we don't care about what
relays, filters, aggregators, correlators, etc do.  At least, if
somebody wants to relay the message end-to-end intact, they would be
able to do so without confusing the receiver due to modified transport
layer data such as source IP/port of relay in the packet.

I think we need to state the above assumption that the receiver MUST
NOT rely on the information from the transport layer for anything else
other than receiving a complete syslog-protocol message.  For example,
receiver should not assume that the IP address in the IP header of the
packet is the IP address of the original sender of the message.  It
MUST NOT be used as an identifier for message source (hence the need
for a standard source IP structured content tag in addition to
HOSTNAME field). If that's the case, then you can insert as many
relays as you want as far as syslog-protocol is concerned.

What do you think about stating this?

Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Thursday, April 15, 2004 11:35 AM
> To: [email-masked]
> Subject: -protocol issue 15: Describe Relay Mode?
>
>
> Hi WG,
>
> I would like to once again drag your attention to the
> question if we should define syslog relaying in -protocol.
> This is tracked as issue 15 at
>
http://www.syslog.cc/ietf/protocol/issue15.html

I think we should describe relay mode in a different document. As I
received no other response to my previous question on this topic, I
think there is agreement that this should be done in a separate
document. Thus, I will specify it as such if there is no further
objection (I am about to do the next edit).

Rainer

------------------------------

Date: Fri, 16 Apr 2004 23:47:29 -0700
From: Pradeep Kumar <[email-masked]>
Subject: How to contribute/participate

Hello

A good part of my time on and off work goes in unserstanding the Syslog
solutions, issues in the real world.

I would like to participate in this working group.

Pradeep Kumar
Systems Engineer

Protego Networks Inc.

Desk             +1-(408)-262-5220 ext 250

Mobile :         +1-(408)-802-5639

Mobile Email: [email-masked]
Fax                +1-408-262-5639

Yahoo IM       protegopk

Protego <http://protegonetworks.com>  MARS - Integration, Insight and
Control

"The only solution to leverage your network and deliver Security Threat
Mitigation in real-time."

------------------------------

Date: Mon, 19 Apr 2004 06:50:40 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Re: How to contribute/participate

Hi Pradeep,

I appreciate your interest in the subject.  The way to participate in any
IETF Working Group is by joining the mailing list and sending in comments
on the active documents.  The information about both is contain in our
Charter.
  http://www.ietf.org/html.charters/syslog-charter.html

Thanks,
Chris

On Fri, 16 Apr 2004, Pradeep Kumar wrote:

> Hello
>
>
>
> A good part of my time on and off work goes in unserstanding the Syslog
> solutions, issues in the real world.
>
> I would like to participate in this working group.
>
>
>
> Pradeep Kumar
> Systems Engineer
>
> Protego Networks Inc.
>
>
>
> Desk             +1-(408)-262-5220 ext 250
>
> Mobile :         +1-(408)-802-5639
>
> Mobile Email: [email-masked]
> Fax                +1-408-262-5639
>
> Yahoo IM       protegopk
>
>
> Protego <http://protegonetworks.com>  MARS - Integration, Insight and
> Control
>
> "The only solution to leverage your network and deliver Security Threat
> Mitigation in real-time."
>
>
>
>
>
>

------------------------------

Date: Mon, 19 Apr 2004 17:13:58 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Issue 14: allow unqualified hostname

Hi WG,

this is in regard to issue 14, which talks about allowing the
unqualified hostname. Based on previous feedback, I think this is
concensus in the WG (see http://www.syslog.cc/ietf/protocol/issue14.html
for a short list).

If nobody objects, I will go ahead and edit it in the following way:

Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare" hostname
MAY be used.

Rainer

------------------------------

Date: Mon, 19 Apr 2004 17:42:58 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -protocol issue 12: time zone info required?

Hi WG,

I am cleaning up bullet points for the edit.

http://www.syslog.cc/ietf/protocol/issue12.html

Talks about time zone infos, as it looks this has been sufficiently
solved in -protocol-03. So I am going to set this to "solved" (with no
edit in -04). Please let me know if you think otherwise.

Rainer

------------------------------

Date: Mon, 19 Apr 2004 17:13:58 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Issue 14: allow unqualified hostname

Hi WG,

this is in regard to issue 14, which talks about allowing the
unqualified hostname. Based on previous feedback, I think this is
concensus in the WG (see http://www.syslog.cc/ietf/protocol/issue14.html
for a short list).

If nobody objects, I will go ahead and edit it in the following way:

Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare" hostname
MAY be used.

Rainer

------------------------------

Date: Mon, 19 Apr 2004 17:56:54 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -protocol issue 8: structure data anywhere in MSG

Hi WG,

sorry, many postings from me today. I promise a new ID will follow
shortly :)

This is in regard to structured data elements. Currently, they are
allowed everywhere. It was suggested that they should be allowed only in
a specific field. I am tracking this here:

http://www.syslog.cc/ietf/protocol/issue8.html

Currently missing on this page are Albert's well thought-out comments:

http://www.syslog.cc/ietf/autoarc/msg01176.html

After thinking about the full issue, I, too, would recommend to move the
structured data elements to a specific header element (much in the way
Albert describes). If nobody objects, I will do so with the next edit. I
think the text will become much cleaner, also implementation errors are
probably less likely.

Rainer

------------------------------

Date: Mon, 19 Apr 2004 18:32:06 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -protocol issue 7: enterprise ID

WG,

I am going to move the enterprise ID to the origin SD-ID (and move it
OUT of the required header). I think this is the best concensus that can
be seen from previous discussion:

http://www.syslog.cc/ietf/protocol/issue7.html

Again, please make yourself heard if you have some good arguments why it
should stay in the header.

Rainer

------------------------------

Date: Mon, 19 Apr 2004 16:19:04 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: Issue 14: allow unqualified hostname

Rainer:

I like Devin's suggestion of recommending a specific preference order:

FQDN
Static IPv4/IPv6
Hostname
Dynamic IPv4/IPv6
"127.0.0.1" (when everything is unknown)

Maybe the language should be a bit more restrictive than just SHOULDs
and MAYs here.  Maybe: "MUST provide FQDN if it is known. If unknown -
static IP. If unknown -- hostname. If unknown - dynamic IP.  If
unknown -- (a) can't use syslog or (b) we explain what they should
use."

I don't know if we decided on the last one. If syslog is to be used
for remote logging only, then requiring knowledge of at least an IP is
acceptable.  If, however, we expect it to be used in host-local
scenarios as well, then we need to clarify what they should there when
nothing I known. Devin suggested 127.0.0.1. I like it. Maybe also
allow an IPv6 equivalent of that as well if it exists.

Thanks,
Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Monday, April 19, 2004 11:14 AM
> To: [email-masked]
> Subject: Issue 14: allow unqualified hostname
>
>
> Hi WG,
>
> this is in regard to issue 14, which talks about allowing the
> unqualified hostname. Based on previous feedback, I think
> this is concensus in the WG (see
> http://www.syslog.cc/ietf/protocol/issue14.htm> l
> for a short
> list).
>
> If nobody objects, I will go ahead and
> edit it in the following way:
>
> Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare"
> hostname MAY be used.
>
> Rainer
>
>

------------------------------

Date: Tue, 20 Apr 2004 11:09:56 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 14: allow unqualified hostname

Anton:

thanks for the reminder. I agree with you. I will edit some wording in
this spirit. We can fine-polish it in the next revision if there is
need.

Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Monday, April 19, 2004 10:19 PM
> To: Rainer Gerhards
> Cc: [email-masked]
> Subject: RE: Issue 14: allow unqualified hostname
>
> Rainer:
>
> I like Devin's suggestion of recommending a specific preference order:
>
> FQDN
> Static IPv4/IPv6
> Hostname
> Dynamic IPv4/IPv6
> "127.0.0.1" (when everything is unknown)
>
> Maybe the language should be a bit more restrictive than just SHOULDs
> and MAYs here.  Maybe: "MUST provide FQDN if it is known. If unknown -
> static IP. If unknown -- hostname. If unknown - dynamic IP.  If
> unknown -- (a) can't use syslog or (b) we explain what they should
> use."
>
> I don't know if we decided on the last one. If syslog is to be used
> for remote logging only, then requiring knowledge of at least an IP is
> acceptable.  If, however, we expect it to be used in host-local
> scenarios as well, then we need to clarify what they should there when
> nothing I known. Devin suggested 127.0.0.1. I like it. Maybe also
> allow an IPv6 equivalent of that as well if it exists.
>
> Thanks,
> Anton.
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> > Sent: Monday, April 19, 2004 11:14 AM
> > To: [email-masked]
> > Subject: Issue 14: allow unqualified hostname
> >
> >
> > Hi WG,
> >
> > this is in regard to issue 14, which talks about allowing the
> > unqualified hostname. Based on previous feedback, I think
> > this is concensus in the WG (see
> > http://www.syslog.cc/ietf/protocol/issue14.htm> l
> > for a short
> > list).
> >
> > If nobody objects, I will go ahead and
> > edit it in the following way:
> >
> > Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare"
> > hostname MAY be used.
> >
> > Rainer
> >
> >
>
>
>
>

------------------------------

Date: Tue, 20 Apr 2004 18:53:00 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: Message size limit

Rainer:

Are you going to specify a message size limit in -protocol?

I think we do need some limit.  For fragmentation in -protocol I need
to know how many bytes to dedicate to the fragment offset and
potentially message length.

I suggest we limit it to 4GB in size (32 bit size) or 16MB (24 bit
size). We can also recommend that implementations of receivers provide
a configuration option for a maximum accepted message size.  Keeping
even a single 4GB message in memory becomes quite a challenge. I think
we should also specify a minimum message size that all syslog
receivers MUST support in any configuration. This will be kind of like
the minimum MTU that hosts on the internet must support.

Thanks,
Anton.

------------------------------

Date: Tue, 20 Apr 2004 09:12:41 -0700
From: Devin Kowatch <[email-masked]>
Subject: Re: Issue 14: allow unqualified hostname

On Mon, Apr 19, 2004 at 05:13:58PM +0200, Rainer Gerhards wrote:
> Hi WG,
>
> this is in regard to issue 14, which talks about allowing the
> unqualified hostname. Based on previous feedback, I think this is
> concensus in the WG (see http://www.syslog.cc/ietf/protocol/issue14.html
> for a short list).
>
> If nobody objects, I will go ahead and edit it in the following way:
>
> Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare" hostname
> MAY be used.

I thought the FQDN included the hostname?  If so the 'Hostname & FQDN'
wording is confusing.  No other objections.

- -- 
Devin Kowatch
[email-masked]

------------------------------

Date: Wed, 21 Apr 2004 11:12:38 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Message size limit

Anton:

I advocated for no limit, but I sea the reasoning ;) I'd go for 24 bits,
that should be more than far enough for not only the forseeable future.
One may argue this is to much, but this has been argued so often in the
past... and prooved to be wrong. So I'd like to waste some more
bandwidth. I suggest that you use variable-length fields for your header
(ASCII digits terminated by SP) as seldomly someone will need these big
numbers.

Regarding the minimum, I also agree. I think it should be 512 (as in
DNS). We should just ensure that - if your header is added - it still
fits into the minimum MTU (which is 576 out of my head... but I may be
wrong here).

I also suggest that -protocol should say.

"Messages SHOULD be less than 1024 bytes, but MAY be as large as
16777000 bytes". I have reduced the 24 bit max by 215 bytes to take care
for your header.

I think pointing out that a the max message size should be less than 1K
could probably save us from too many "large-message-implementations".
I'd like to have people think about what they do. But maybe doing it in
the normative part is not a good idea...

Comments on this suggestion?

Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Wednesday, April 21, 2004 12:53 AM
> To: Rainer Gerhards
> Cc: [email-masked]
> Subject: RE: Message size limit
>
> Rainer:
>
> Are you going to specify a message size limit in -protocol?
>
> I think we do need some limit.  For fragmentation in -protocol I need
> to know how many bytes to dedicate to the fragment offset and
> potentially message length.
>
> I suggest we limit it to 4GB in size (32 bit size) or 16MB (24 bit
> size). We can also recommend that implementations of receivers provide
> a configuration option for a maximum accepted message size.  Keeping
> even a single 4GB message in memory becomes quite a challenge. I think
> we should also specify a minimum message size that all syslog
> receivers MUST support in any configuration. This will be kind of like
> the minimum MTU that hosts on the internet must support.
>
> Thanks,
> Anton.
>
>
>
>

------------------------------

Date: Thu, 22 Apr 2004 01:38:56 +1200
From: Andrew Ross <[email-masked]>
Subject: RE: Message size limit

I'd agree with Rainer that ASCII digits for length would be best. This
would be more in line with the way numbers are represented in BEEP
exchanges. 16MB is a LOT of message. If anyone wants to send more, point
them to the FTP RFC :-)

Cheers

Andrew

- -----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Rainer Gerhards
Sent: Wednesday, 21 April 2004 9:13 p.m.
To: Anton Okmianski
Cc: [email-masked]
Subject: RE: Message size limit

Anton:

I advocated for no limit, but I sea the reasoning ;) I'd go for 24 bits,
that should be more than far enough for not only the forseeable future.
One may argue this is to much, but this has been argued so often in the
past... and prooved to be wrong. So I'd like to waste some more
bandwidth. I suggest that you use variable-length fields for your header
(ASCII digits terminated by SP) as seldomly someone will need these big
numbers.

Regarding the minimum, I also agree. I think it should be 512 (as in
DNS). We should just ensure that - if your header is added - it still
fits into the minimum MTU (which is 576 out of my head... but I may be
wrong here).

I also suggest that -protocol should say.

"Messages SHOULD be less than 1024 bytes, but MAY be as large as
16777000 bytes". I have reduced the 24 bit max by 215 bytes to take care
for your header.

I think pointing out that a the max message size should be less than 1K
could probably save us from too many "large-message-implementations".
I'd like to have people think about what they do. But maybe doing it in
the normative part is not a good idea...

Comments on this suggestion?

Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Wednesday, April 21, 2004 12:53 AM
> To: Rainer Gerhards
> Cc: [email-masked]
> Subject: RE: Message size limit
>
> Rainer:
>
> Are you going to specify a message size limit in -protocol?
>
> I think we do need some limit.  For fragmentation in -protocol I need
> to know how many bytes to dedicate to the fragment offset and
> potentially message length.
>
> I suggest we limit it to 4GB in size (32 bit size) or 16MB (24 bit
> size). We can also recommend that implementations of receivers provide
> a configuration option for a maximum accepted message size.  Keeping
> even a single 4GB message in memory becomes quite a challenge. I think
> we should also specify a minimum message size that all syslog
> receivers MUST support in any configuration. This will be kind of like
> the minimum MTU that hosts on the internet must support.
>
> Thanks,
> Anton.
>
>
>
>

------------------------------

Date: Wed, 21 Apr 2004 20:09:22 +0200
From: "ons-huis.net!ALbert" <[email-masked]>
Subject: Re: Message size limit

 >> I think we do need some limit.  For fragmentation in -protocol I need
 >> to know how many bytes to dedicate to the fragment offset and
 >> potentially message length.

> I advocated for no limit, but I sea the reasoning ;) I'd go for 24 bits,

At 'trick' we can do is limit to 23 bits. And specify that whenever the
highest (24th) bit is 1/true/set another schema is valid, one with a
longer field for the length

ASN.1/BER uses a format like this one. (But more extended)
It works: For now, we can use upto "very long" messages. For tomorrow
(or a little later:-) it is simple to extend. Ofcoarse we can also bump
the syslog-version number then

For the min length: use 1024! Or a little less, like in rfc3164
implementations.

- -- 
Groetjes,
- --ALbert Mietus
Private mail to:           albert at ons-huis dot net
Business mail to:     albert dot mietus at PTS dot nl
Spam:   Just don't do it! Thrust me, I will not order!

------------------------------

Date: Wed, 21 Apr 2004 11:14:06 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: Message size limit

Andrew, Rainer & all:

Why ASCII?  I was planning on using binary for transport
specification.  Encoding even a length of 1024 with space takes 5
octets in ASCII.  In binary, I dedicate 4 bytes to length, which can
cover up to 4bln. Note: this does not mean that -protocol should allow
that much.

Here is a transport packet format I am working on:

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Flags     |        Message Identifier (3 octets)          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Length (4 octets)                      |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   Fragment Offset (4 octets)                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.                                                               .
.                   Message (variable length)         +-+-+-+-+-+
.                                                     | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Note: The first bit in Flags indicates if this is a multi-part
message. Second - if there is more fragments.  For single-part
messages, there is two options: (a) use the same header and declare
certain fields insignificant or (b) use a different header which only
contains Flags field. I am not yet clear which approach is better.  I
am checking precedents.

The other issue is that message Length is redundant in all but the
first packet of a multi-part message. This is because length of each
segment can be inferred from the length of the datagram which is in
the IP/UDP headers. We only need Length field to indicate total
message length of a fragmented message in the first datagram.

Also, I am not sure about the Padding.  Seems like many protocols try
to make packet fields align to 32-bit boundaries. I am not sure why
this is necessary - I will explore.

Any comments?

Thanks,
Anton.

> -----Original Message-----
> From: Andrew Ross [mailto:[email-masked]]
> Sent: Wednesday, April 21, 2004 9:39 AM
> To: 'Rainer Gerhards'; 'Anton Okmianski'
> Cc: [email-masked]
> Subject: RE: Message size limit
>
>
>
> I'd agree with Rainer that ASCII digits for length would be
> best. This would be more in line with the way numbers are
> represented in BEEP exchanges. 16MB is a LOT of message. If
> anyone wants to send more, point them to the FTP RFC :-)
>
> Cheers
>
> Andrew
>
>
>
> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Wednesday, 21 April 2004 9:13 p.m.
> To: Anton Okmianski
> Cc: [email-masked]
> Subject: RE: Message size limit
>
>
> Anton:
>
> I advocated for no limit, but I sea the reasoning ;) I'd go
> for 24 bits, that should be more than far enough for not only
> the forseeable future. One may argue this is to much, but
> this has been argued so often in the past... and prooved to
> be wrong. So I'd like to waste some more bandwidth. I suggest
> that you use variable-length fields for your header (ASCII
> digits terminated by SP) as seldomly someone will need these
> big numbers.
>
> Regarding the minimum, I also agree. I think it should be 512
> (as in DNS). We should just ensure that - if your header is
> added - it still fits into the minimum MTU (which is 576 out
> of my head... but I may be wrong here).
>
> I also suggest that -protocol should say.
>
> "Messages SHOULD be less than 1024 bytes, but MAY be as large
> as 16777000 bytes". I have reduced the 24 bit max by 215
> bytes to take care for your header.
>
> I think pointing out that a the max message size should be
> less than 1K could probably save us from too many
> "large-message-implementations". I'd like to have people
> think about what they do. But maybe doing it in the normative
> part is not a good idea...
>
> Comments on this suggestion?
>
> Rainer
>
> > -----Original Message-----
> > From: Anton Okmianski [mailto:[email-masked]]
> > Sent: Wednesday, April 21, 2004 12:53 AM
> > To: Rainer Gerhards
> > Cc: [email-masked]
> > Subject: RE: Message size limit
> >
> > Rainer:
> >
> > Are you going to specify a message size limit in -protocol?
> >
> > I think we do need some limit.  For fragmentation in
> -protocol I need
> > to know how many bytes to dedicate to the fragment offset and
> > potentially message length.
> >
> > I suggest we limit it to 4GB in size (32 bit size) or 16MB (24 bit

> > size). We can also recommend that implementations of
> receivers provide
> > a configuration option for a maximum accepted message size.
>  Keeping
> > even a single 4GB message in memory becomes quite a
> challenge. I think
> > we should also specify a minimum message size that all syslog
> > receivers MUST support in any configuration. This will be
> kind of like
> > the minimum MTU that hosts on the internet must support.
> >
> > Thanks,
> > Anton.
> >
> >
> >
> >
>
>
>
>

------------------------------

Date: Thu, 22 Apr 2004 14:10:14 +1200
From: Andrew Ross <[email-masked]>
Subject: RE: Message size limit

Hi Anton,

I hadn't been following your previous posts close enough to realise you
wanted to use a binary format. My comment to use ASCII (0-9) was meant
instead of a HEX type format (0-F).

Binary would work for me too, but you MUST pad it out to the byte
boundary. It is nightmare to code otherwise. BER is very efficient, but
yuck to code against. I vote for simplicity over efficiency in this
case.

Cheers

Andrew

- -----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Anton Okmianski
Sent: Thursday, 22 April 2004 3:14 a.m.
To: [email-masked]; 'Rainer Gerhards'
Cc: [email-masked]
Subject: RE: Message size limit

Andrew, Rainer & all:

Why ASCII?  I was planning on using binary for transport
specification.  Encoding even a length of 1024 with space takes 5
octets in ASCII.  In binary, I dedicate 4 bytes to length, which can
cover up to 4bln. Note: this does not mean that -protocol should allow
that much.

Here is a transport packet format I am working on:

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Flags     |        Message Identifier (3 octets)          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Length (4 octets)                      |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   Fragment Offset (4 octets)                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.                                                               .
.                   Message (variable length)         +-+-+-+-+-+
.                                                     | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Note: The first bit in Flags indicates if this is a multi-part
message. Second - if there is more fragments.  For single-part
messages, there is two options: (a) use the same header and declare
certain fields insignificant or (b) use a different header which only
contains Flags field. I am not yet clear which approach is better.  I
am checking precedents.

The other issue is that message Length is redundant in all but the
first packet of a multi-part message. This is because length of each
segment can be inferred from the length of the datagram which is in
the IP/UDP headers. We only need Length field to indicate total
message length of a fragmented message in the first datagram.

Also, I am not sure about the Padding.  Seems like many protocols try
to make packet fields align to 32-bit boundaries. I am not sure why
this is necessary - I will explore.

Any comments?

Thanks,
Anton.

> -----Original Message-----
> From: Andrew Ross [mailto:[email-masked]]
> Sent: Wednesday, April 21, 2004 9:39 AM
> To: 'Rainer Gerhards'; 'Anton Okmianski'
> Cc: [email-masked]
> Subject: RE: Message size limit
>
>
>
> I'd agree with Rainer that ASCII digits for length would be
> best. This would be more in line with the way numbers are
> represented in BEEP exchanges. 16MB is a LOT of message. If
> anyone wants to send more, point them to the FTP RFC :-)
>
> Cheers
>
> Andrew
>
>
>
> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Wednesday, 21 April 2004 9:13 p.m.
> To: Anton Okmianski
> Cc: [email-masked]
> Subject: RE: Message size limit
>
>
> Anton:
>
> I advocated for no limit, but I sea the reasoning ;) I'd go
> for 24 bits, that should be more than far enough for not only
> the forseeable future. One may argue this is to much, but
> this has been argued so often in the past... and prooved to
> be wrong. So I'd like to waste some more bandwidth. I suggest
> that you use variable-length fields for your header (ASCII
> digits terminated by SP) as seldomly someone will need these
> big numbers.
>
> Regarding the minimum, I also agree. I think it should be 512
> (as in DNS). We should just ensure that - if your header is
> added - it still fits into the minimum MTU (which is 576 out
> of my head... but I may be wrong here).
>
> I also suggest that -protocol should say.
>
> "Messages SHOULD be less than 1024 bytes, but MAY be as large
> as 16777000 bytes". I have reduced the 24 bit max by 215
> bytes to take care for your header.
>
> I think pointing out that a the max message size should be
> less than 1K could probably save us from too many
> "large-message-implementations". I'd like to have people
> think about what they do. But maybe doing it in the normative
> part is not a good idea...
>
> Comments on this suggestion?
>
> Rainer
>
> > -----Original Message-----
> > From: Anton Okmianski [mailto:[email-masked]]
> > Sent: Wednesday, April 21, 2004 12:53 AM
> > To: Rainer Gerhards
> > Cc: [email-masked]
> > Subject: RE: Message size limit
> >
> > Rainer:
> >
> > Are you going to specify a message size limit in -protocol?
> >
> > I think we do need some limit.  For fragmentation in
> -protocol I need
> > to know how many bytes to dedicate to the fragment offset and
> > potentially message length.
> >
> > I suggest we limit it to 4GB in size (32 bit size) or 16MB (24 bit

> > size). We can also recommend that implementations of
> receivers provide
> > a configuration option for a maximum accepted message size.
>  Keeping
> > even a single 4GB message in memory becomes quite a
> challenge. I think
> > we should also specify a minimum message size that all syslog
> > receivers MUST support in any configuration. This will be
> kind of like
> > the minimum MTU that hosts on the internet must support.
> >
> > Thanks,
> > Anton.
> >
> >
> >
> >
>
>
>
>

------------------------------

Date: Fri, 23 Apr 2004 14:10:45 +0200
From: Rainer Gerhards <[email-masked]>
Subject: binary fields in syslog

Hi WG,

this is not only in reply to Anton's posts, but a general thing. So I
thought I start with a new subject.

Anton suggest the use of a binary header in -transport (eg
http://www.syslog.cc/ietf/autoarc/msg01197.html). Previously, all
numbers in syslog were expressed in ASCII digits terminated by spaces.

I can understand Anton's efficiency concerns. However, if we go for a
binary header in -transport, we could/should also reconsider that issue
in -protocol and all other syslog IDs/RFCs. Not just to keep them
consistent, but also to be as efficient as possible.

HOWEVER, I do NOT like the idea of binary numbers. I think there is good
reasoning that many modern protocols have gone away from binary and
sacrified some efficiency to do things somewhat easier. The plus in
ascii digits is:

- - esay human interaction - not just reading but also (telnet) crafting
messages
  (I couldn't envision troubleshooting SMTP configs without the ability
to
  telnet SMTP commands)
- - it's instantly interoperable - no need for network bytes ordering
- - I think it reduces implementation errors

Ffinally, in the context of syslog, we are already very far from RFC
3164. I am already very concerned if our changes will be accepted by the
implementor community. If we now move even one more step ahead and
create a fully binary protocol... I fear we would finally loose them.
And to gain some more efficiency, we could also BER-encode the
structured data elements ;)

So I short: I *strongly, strongly* advocate to keep things in ASCII. But
if we *really* go ahead and define transport to use binary data (for
efficiency), we should also go ahead and change -protocol to be binary,
too.

Rainer

------------------------------

Date: Fri, 23 Apr 2004 15:52:23 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Issue 13: precise or lax

Hi WG,

I am tracking this at:

http://www.syslog.cc/ietf/protocol/issue13.html

during the Seoul meeting, it was suggested that the too-pricese part
should be moved into a non-normative appendix. I like this idea, as it
serves all needs best. I am now about to move text. I would appreciate
suggestions on what should be moved to the non-normative part.

Rainer

------------------------------

Date: Fri, 23 Apr 2004 15:01:21 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 14: allow unqualified hostname

Anton & all,

in IPv6, we have the "unspecified address", which I think is exactly
what we should use in the case an device does actually know nothing
about itself (last case in Anton's messsage below) it is
"0:0:0:0:0:0:0:0".

Some links:
http://www.iana.org/assignments/ipv6-address-space
http://www.cisco.com/en/US/products/sw/iosswrel/ios_abcs_ios_the_abcs_ip
_version_60900aecd800c111d.html

According to RFC 3330 (http://www.rfc-editor.org/rfc/rfc3330.txt), I
think we can also use "0.0.0.0" for IPv4 addressing in this case.

Comments are highly appreciated.

Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Monday, April 19, 2004 10:19 PM
> To: Rainer Gerhards
> Cc: [email-masked]
> Subject: RE: Issue 14: allow unqualified hostname
>
> Rainer:
>
> I like Devin's suggestion of recommending a specific preference order:
>
> FQDN
> Static IPv4/IPv6
> Hostname
> Dynamic IPv4/IPv6
> "127.0.0.1" (when everything is unknown)
>
> Maybe the language should be a bit more restrictive than just SHOULDs
> and MAYs here.  Maybe: "MUST provide FQDN if it is known. If unknown -
> static IP. If unknown -- hostname. If unknown - dynamic IP.  If
> unknown -- (a) can't use syslog or (b) we explain what they should
> use."
>
> I don't know if we decided on the last one. If syslog is to be used
> for remote logging only, then requiring knowledge of at least an IP is
> acceptable.  If, however, we expect it to be used in host-local
> scenarios as well, then we need to clarify what they should there when
> nothing I known. Devin suggested 127.0.0.1. I like it. Maybe also
> allow an IPv6 equivalent of that as well if it exists.
>
> Thanks,
> Anton.
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> > Sent: Monday, April 19, 2004 11:14 AM
> > To: [email-masked]
> > Subject: Issue 14: allow unqualified hostname
> >
> >
> > Hi WG,
> >
> > this is in regard to issue 14, which talks about allowing the
> > unqualified hostname. Based on previous feedback, I think
> > this is concensus in the WG (see
> > http://www.syslog.cc/ietf/protocol/issue14.htm> l
> > for a short
> > list).
> >
> > If nobody objects, I will go ahead and
> > edit it in the following way:
> >
> > Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare"
> > hostname MAY be used.
> >
> > Rainer
> >
> >
>
>

------------------------------

Date: Fri, 23 Apr 2004 10:36:12 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: Issue 14: allow unqualified hostname

Rainer:

If syslog client sends a message over TCP/UDP/IP, it by definition has
an IP address.  If it send the message locally, then I think 127.0.0.1
(local IP) is more appropriate.

But I am assuming that we won't use syslog-protocol through say local
UNIX pipes like syslog does now locally on say Solaris.  If we do want
to support that, than we need the case of an unspecified IP.  But if
syslog is used over TCP/IP, "unspecified IP" is an oxymoron.

Anton.

> -----Original Message-----
> From: Rainer Gerhards [mailto:[email-masked]]
> Sent: Friday, April 23, 2004 9:01 AM
> To: [email-masked]
> Cc: Anton Okmianski
> Subject: RE: Issue 14: allow unqualified hostname
>
>
> Anton & all,
>
> in IPv6, we have the "unspecified address", which I think is
> exactly what we should use in the case an device does
> actually know nothing about itself (last case in Anton's
> messsage below) it is "0:0:0:0:0:0:0:0".
>
> Some links:
> http://www.iana.org/assignments/ipv6-address-space
> http://www.cisco.com/en/US/products/sw/iosswrel/ios_abcs_ios_t
> he_abcs_ip
> _version_60900aecd800c111d.html
>
> According to RFC 3330
> (http://www.rfc-editor.org/rfc/rfc3330.txt), I think we can
> also use "0.0.0.0" for IPv4 addressing in this case.
>
> Comments are highly appreciated.
>
> Rainer
>
> > -----Original Message-----
> > From: Anton Okmianski [mailto:[email-masked]]
> > Sent: Monday, April 19, 2004 10:19 PM
> > To: Rainer Gerhards
> > Cc: [email-masked]
> > Subject: RE: Issue 14: allow unqualified hostname
> >
> > Rainer:
> >
> > I like Devin's suggestion of recommending a specific
> preference order:
> >
> > FQDN
> > Static IPv4/IPv6
> > Hostname
> > Dynamic IPv4/IPv6
> > "127.0.0.1" (when everything is unknown)
> >
> > Maybe the language should be a bit more restrictive than
> just SHOULDs
> > and MAYs here.  Maybe: "MUST provide FQDN if it is known.
> If unknown -
> > static IP. If unknown -- hostname. If unknown - dynamic IP.  If
> > unknown -- (a) can't use syslog or (b) we explain what they should

> > use."
> >
> > I don't know if we decided on the last one. If syslog is to be
used
> > for remote logging only, then requiring knowledge of at
> least an IP is
> > acceptable.  If, however, we expect it to be used in host-local
> > scenarios as well, then we need to clarify what they should
> there when
> > nothing I known. Devin suggested 127.0.0.1. I like it. Maybe also
> > allow an IPv6 equivalent of that as well if it exists.
> >
> > Thanks,
> > Anton.
> >
> > > -----Original Message-----
> > > From: [email-masked]
> > > [mailto:[email-masked]] On Behalf Of
> Rainer Gerhards
> > > Sent: Monday, April 19, 2004 11:14 AM
> > > To: [email-masked]
> > > Subject: Issue 14: allow unqualified hostname
> > >
> > >
> > > Hi WG,
> > >
> > > this is in regard to issue 14, which talks about allowing the
> > > unqualified hostname. Based on previous feedback, I think
> > > this is concensus in the WG (see
> > > http://www.syslog.cc/ietf/protocol/issue14.htm> l
> > > for a short
> > > list).
> > >
> > > If nobody objects, I will go ahead and
> > > edit it in the following way:
> > >
> > > Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare"
> > > hostname MAY be used.
> > >
> > > Rainer
> > >
> > >
> >
> >
>

------------------------------

Date: Fri, 23 Apr 2004 16:52:48 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 14: allow unqualified hostname

> If syslog client sends a message over TCP/UDP/IP, it by definition has
> an IP address.  If it send the message locally, then I think 127.0.0.1
> (local IP) is more appropriate.

I think the case we are describing is a very unusual one. It only
happens when the device (better said: software running) has no way of
*obtaining knowledge* of its IP address. I am not sure if we will find
this in reality at all. So it does not matter (at least in my view) that
it has an IP address - it can not get hold of it). So I am using a
special value to say "I'm clueless".

On the other hand, 127.0.0.1 is a valid value - it may be used when a
sender talks to a receiver on the same machine.

I would like to differentiate these two cases.

> But I am assuming that we won't use syslog-protocol through say local
> UNIX pipes like syslog does now locally on say Solaris.  If we do want
> to support that, than we need the case of an unspecified IP.  But if
> syslog is used over TCP/IP, "unspecified IP" is an oxymoron.

As I said... depends. I am talking about a device *having* an IP address
but the software not knowing it. If we all agree this is so unlikely
that we should not support it, then I think we should fully drop this
"last resort rule" - because we are saying it can never apply ;)

Rainer
>
> Anton.
>
> > -----Original Message-----
> > From: Rainer Gerhards [mailto:[email-masked]]
> > Sent: Friday, April 23, 2004 9:01 AM
> > To: [email-masked]
> > Cc: Anton Okmianski
> > Subject: RE: Issue 14: allow unqualified hostname
> >
> >
> > Anton & all,
> >
> > in IPv6, we have the "unspecified address", which I think is
> > exactly what we should use in the case an device does
> > actually know nothing about itself (last case in Anton's
> > messsage below) it is "0:0:0:0:0:0:0:0".
> >
> > Some links:
> > http://www.iana.org/assignments/ipv6-address-space
> > http://www.cisco.com/en/US/products/sw/iosswrel/ios_abcs_ios_t
> > he_abcs_ip
> > _version_60900aecd800c111d.html
> >
> > According to RFC 3330
> > (http://www.rfc-editor.org/rfc/rfc3330.txt), I think we can
> > also use "0.0.0.0" for IPv4 addressing in this case.
> >
> > Comments are highly appreciated.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: Anton Okmianski [mailto:[email-masked]]
> > > Sent: Monday, April 19, 2004 10:19 PM
> > > To: Rainer Gerhards
> > > Cc: [email-masked]
> > > Subject: RE: Issue 14: allow unqualified hostname
> > >
> > > Rainer:
> > >
> > > I like Devin's suggestion of recommending a specific
> > preference order:
> > >
> > > FQDN
> > > Static IPv4/IPv6
> > > Hostname
> > > Dynamic IPv4/IPv6
> > > "127.0.0.1" (when everything is unknown)
> > >
> > > Maybe the language should be a bit more restrictive than
> > just SHOULDs
> > > and MAYs here.  Maybe: "MUST provide FQDN if it is known.
> > If unknown -
> > > static IP. If unknown -- hostname. If unknown - dynamic IP.  If
> > > unknown -- (a) can't use syslog or (b) we explain what they should
>
> > > use."
> > >
> > > I don't know if we decided on the last one. If syslog is to be
> used
> > > for remote logging only, then requiring knowledge of at
> > least an IP is
> > > acceptable.  If, however, we expect it to be used in host-local
> > > scenarios as well, then we need to clarify what they should
> > there when
> > > nothing I known. Devin suggested 127.0.0.1. I like it. Maybe also
> > > allow an IPv6 equivalent of that as well if it exists.
> > >
> > > Thanks,
> > > Anton.
> > >
> > > > -----Original Message-----
> > > > From: [email-masked]
> > > > [mailto:[email-masked]] On Behalf Of
> > Rainer Gerhards
> > > > Sent: Monday, April 19, 2004 11:14 AM
> > > > To: [email-masked]
> > > > Subject: Issue 14: allow unqualified hostname
> > > >
> > > >
> > > > Hi WG,
> > > >
> > > > this is in regard to issue 14, which talks about allowing the
> > > > unqualified hostname. Based on previous feedback, I think
> > > > this is concensus in the WG (see
> > > > http://www.syslog.cc/ietf/protocol/issue14.htm> l
> > > > for a short
> > > > list).
> > > >
> > > > If nobody objects, I will go ahead and
> > > > edit it in the following way:
> > > >
> > > > Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare"
> > > > hostname MAY be used.
> > > >
> > > > Rainer
> > > >
> > > >
> > >
> > >
> >
>
>

------------------------------

Date: Fri, 23 Apr 2004 11:30:34 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: binary fields in syslog

Rainer & all:

I see your concerns, but I don't think -protocol and -transport have
to either both do a complete binary or neither. For one, you can think
of them as protocols which operate at different layers of the OSI
model.

Keep in mind that we already do "binary" in -protocol because we
support entire Unicode. I actually think ultimately it would have been
great to separate semantics/schema of -protocol (defined with say with
ASN.1) and its encoding (BER, DER, ASCII+Unicode, XML, etc). This
would have allowed for binary encoding for high-performance scenarios
and XML encoding for those who can't live without it. It would also
make it much easier to deal with optional fields or fields that may
have multiple values. Trust me, people will re-invent syslog in XML
because ASCII formats are "too inflexible" (google for "IBM CBE"). I
think the current specification strikes a good balance between binary
and ASCII and has a good scope for this iteration.

Syslog UDP (SUDP for now) transport adds a very limited function on
top of UDP - fragmenting of messages.  As we have agreed, it is not
even part of -protocol because it is basically part of the transport
layer as opposed to application data.  There are precedents for
application data being in whatever format. So, if we want most of it
to be in straight ASCII, ZIP or whatever it is fine.  But for the
transport layer, TCP/UDP/IP follow their own conventions and they use
binary. It is more consistent to just follow them and use binary for
syslog UDP transport header. It would be consistent with other
transport encapsulation layers of UDP/IP and may even make SUDP
applicable in other domains.  So, I think the distinction between
transport and application data is key here.

I can't speak for how eagerly implementors will adopt one format
versus the other. I hope a shear use of binary vs ASCII in the simple
transport header will no change that. In fact, I think it could make
it easier to program when you have a fixed-size header. In SUDP case,
it could be one of two fixed size headers: 1 or 12 bytes. Still better
than a variable field size for every field.

Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Friday, April 23, 2004 8:11 AM
> To: [email-masked]
> Subject: binary fields in syslog
>
>
> Hi WG,
>
> this is not only in reply to Anton's posts, but a general
> thing. So I thought I start with a new subject.
>
> Anton suggest the use of a binary header in -transport (eg
> http://www.syslog.cc/ietf/autoarc/msg01197.htm> l). Previously,
> all numbers in syslog were expressed in
> ASCII digits terminated by spaces.
>
> I can understand Anton's efficiency concerns. However, if we
> go for a binary header in -transport, we could/should also
> reconsider that issue in -protocol and all other syslog
> IDs/RFCs. Not just to keep them consistent, but also to be as
> efficient as possible.
>
> HOWEVER, I do NOT like the idea of binary numbers. I think
> there is good reasoning that many modern protocols have gone
> away from binary and sacrified some efficiency to do things
> somewhat easier. The plus in ascii digits is:
>
> - esay human interaction - not just reading but also (telnet)
> crafting messages
>   (I couldn't envision troubleshooting SMTP configs without
> the ability to
>   telnet SMTP commands)
> - it's instantly interoperable - no need for network bytes ordering
> - I think it reduces implementation errors
>
> Ffinally, in the context of syslog, we are already very far
> from RFC 3164. I am already very concerned if our changes
> will be accepted by the implementor community. If we now move
> even one more step ahead and create a fully binary
> protocol... I fear we would finally loose them. And to gain
> some more efficiency, we could also BER-encode the structured
> data elements ;)
>
> So I short: I *strongly, strongly* advocate to keep things in
> ASCII. But if we *really* go ahead and define transport to
> use binary data (for efficiency), we should also go ahead and
> change -protocol to be binary, too.
>
> Rainer
>
>

------------------------------

Date: Fri, 23 Apr 2004 18:03:58 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: binary fields in syslog

Anton,

actually, I am not convinced. But that's only my opinion. I would
*really* appreciate any more feedback from the group. I see that you
have very valid reasoning, but, again, I am a bit in fear what this will
cause in regard to momentum.

Anyhow, that'll be my last message before we get more comments from
others ;)

Rainer

> -----Original Message-----
> From: Anton Okmianski [mailto:[email-masked]]
> Sent: Friday, April 23, 2004 5:31 PM
> To: Rainer Gerhards; [email-masked]
> Subject: RE: binary fields in syslog
>
> Rainer & all:
>
> I see your concerns, but I don't think -protocol and -transport have
> to either both do a complete binary or neither. For one, you can think
> of them as protocols which operate at different layers of the OSI
> model.
>
> Keep in mind that we already do "binary" in -protocol because we
> support entire Unicode. I actually think ultimately it would have been
> great to separate semantics/schema of -protocol (defined with say with
> ASN.1) and its encoding (BER, DER, ASCII+Unicode, XML, etc). This
> would have allowed for binary encoding for high-performance scenarios
> and XML encoding for those who can't live without it. It would also
> make it much easier to deal with optional fields or fields that may
> have multiple values. Trust me, people will re-invent syslog in XML
> because ASCII formats are "too inflexible" (google for "IBM CBE"). I
> think the current specification strikes a good balance between binary
> and ASCII and has a good scope for this iteration.
>
> Syslog UDP (SUDP for now) transport adds a very limited function on
> top of UDP - fragmenting of messages.  As we have agreed, it is not
> even part of -protocol because it is basically part of the transport
> layer as opposed to application data.  There are precedents for
> application data being in whatever format. So, if we want most of it
> to be in straight ASCII, ZIP or whatever it is fine.  But for the
> transport layer, TCP/UDP/IP follow their own conventions and they use
> binary. It is more consistent to just follow them and use binary for
> syslog UDP transport header. It would be consistent with other
> transport encapsulation layers of UDP/IP and may even make SUDP
> applicable in other domains.  So, I think the distinction between
> transport and application data is key here.
>
> I can't speak for how eagerly implementors will adopt one format
> versus the other. I hope a shear use of binary vs ASCII in the simple
> transport header will no change that. In fact, I think it could make
> it easier to program when you have a fixed-size header. In SUDP case,
> it could be one of two fixed size headers: 1 or 12 bytes. Still better
> than a variable field size for every field.
>
> Anton.
>
>
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> > Sent: Friday, April 23, 2004 8:11 AM
> > To: [email-masked]
> > Subject: binary fields in syslog
> >
> >
> > Hi WG,
> >
> > this is not only in reply to Anton's posts, but a general
> > thing. So I thought I start with a new subject.
> >
> > Anton suggest the use of a binary header in -transport (eg
> > http://www.syslog.cc/ietf/autoarc/msg01197.htm> l). Previously,
> > all numbers in syslog were expressed in
> > ASCII digits terminated by spaces.
> >
> > I can understand Anton's efficiency concerns. However, if we
> > go for a binary header in -transport, we could/should also
> > reconsider that issue in -protocol and all other syslog
> > IDs/RFCs. Not just to keep them consistent, but also to be as
> > efficient as possible.
> >
> > HOWEVER, I do NOT like the idea of binary numbers. I think
> > there is good reasoning that many modern protocols have gone
> > away from binary and sacrified some efficiency to do things
> > somewhat easier. The plus in ascii digits is:
> >
> > - esay human interaction - not just reading but also (telnet)
> > crafting messages
> >   (I couldn't envision troubleshooting SMTP configs without
> > the ability to
> >   telnet SMTP commands)
> > - it's instantly interoperable - no need for network bytes ordering
> > - I think it reduces implementation errors
> >
> > Ffinally, in the context of syslog, we are already very far
> > from RFC 3164. I am already very concerned if our changes
> > will be accepted by the implementor community. If we now move
> > even one more step ahead and create a fully binary
> > protocol... I fear we would finally loose them. And to gain
> > some more efficiency, we could also BER-encode the structured
> > data elements ;)
> >
> > So I short: I *strongly, strongly* advocate to keep things in
> > ASCII. But if we *really* go ahead and define transport to
> > use binary data (for efficiency), we should also go ahead and
> > change -protocol to be binary, too.
> >
> > Rainer
> >
> >
>
>

------------------------------

Date: Fri, 23 Apr 2004 12:18:13 -0400
From: "Harrington, David" <[email-masked]>
Subject: RE: Issue 13: precise or lax

Hi,

I've around the IETF for quite a while, and I think it is important to
understand the goals of the IETF.

RFC2026 (The Internet Standards Process) says:
"These procedures are explicitly aimed at recognizing and adopting
      generally-accepted practices.  Thus, a candidate specification
      must be implemented and tested for correct operation and
      interoperability by multiple independent parties and utilized in
      increasingly demanding environments, before it can be adopted as
      an Internet Standard."

Are these too-precise parts generally-accepted practices?
Are they implemented by more then one independent implementation?

The goal is to reach agreement on a standard way to do things. If they
describe the way one implementation has done it, then I question whether
it belongs in the document at all. If they document multiple existing
practices that are incompatible, then I question whether it belongs in
the document at all.

The members of this WG should reach agreement on the specification of a
standardized approach, and be willing to implement and test that
specification. Those aspects of the technology for which the members of
this WG are unwilling to agree on a standard approach, and modify their
implementations as needed to comply with that standard approach, should
be removed from the document.

If the behaviors described are not agreed upon by this WG to become the
standard approach, then let's not document them in a standards-track
RFC.

dbh

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Friday, April 23, 2004 9:52 AM
> To: [email-masked]
> Subject: Issue 13: precise or lax
>
> Hi WG,
>
> I am tracking this at:
>
> http://www.syslog.cc/ietf/protocol/issue13.html
>
> during the Seoul meeting, it was suggested that the too-pricese part
> should be moved into a non-normative appendix. I like this idea, as it
> serves all needs best. I am now about to move text. I would appreciate
> suggestions on what should be moved to the non-normative part.
>
> Rainer
>
>
>
>

------------------------------

Date: Fri, 23 Apr 2004 12:48:21 -0400
From: "Harrington, David" <[email-masked]>
Subject: RE: binary fields in syslog

Hi

You might want to read RFC3535, the Overview of the 2002 IAB Network
Management Workshop and draft-jones-opsec-05.txt.

The focus of the workshop was largely on configuration but many of the
observations would also apply to a protocol like syslog.

In particular during the workshop, in the opsec draft, and in
now-expired I-Ds that preceded the workshop, operators made it very
clear that binary management protocols make management harder. One of
the strengths of syslog as compared to SNMP is the ability for a human
to understand the message without an application required to translate
the information.

dbh

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Friday, April 23, 2004 8:11 AM
> To: [email-masked]
> Subject: binary fields in syslog
>
> Hi WG,
>
> this is not only in reply to Anton's posts, but a general thing. So I
> thought I start with a new subject.
>
> Anton suggest the use of a binary header in -transport (eg
> http://www.syslog.cc/ietf/autoarc/msg01197.html). Previously, all
> numbers in syslog were expressed in ASCII digits terminated by spaces.
>
> I can understand Anton's efficiency concerns. However, if we go for a
> binary header in -transport, we could/should also reconsider
> that issue
> in -protocol and all other syslog IDs/RFCs. Not just to keep them
> consistent, but also to be as efficient as possible.
>
> HOWEVER, I do NOT like the idea of binary numbers. I think
> there is good
> reasoning that many modern protocols have gone away from binary and
> sacrified some efficiency to do things somewhat easier. The plus in
> ascii digits is:
>
> - esay human interaction - not just reading but also (telnet) crafting
> messages
>   (I couldn't envision troubleshooting SMTP configs without
> the ability
> to
>   telnet SMTP commands)
> - it's instantly interoperable - no need for network bytes ordering
> - I think it reduces implementation errors
>
> Ffinally, in the context of syslog, we are already very far from RFC
> 3164. I am already very concerned if our changes will be
> accepted by the
> implementor community. If we now move even one more step ahead and
> create a fully binary protocol... I fear we would finally loose them.
> And to gain some more efficiency, we could also BER-encode the
> structured data elements ;)
>
> So I short: I *strongly, strongly* advocate to keep things in
> ASCII. But
> if we *really* go ahead and define transport to use binary data (for
> efficiency), we should also go ahead and change -protocol to
> be binary,
> too.
>
> Rainer
>
>
>
>

------------------------------

Date: Fri, 23 Apr 2004 14:36:23 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: binary fields in syslog

Rainer & all:

Let me comment on something...

> > > - esay human interaction - not just reading but also (telnet)
> > > crafting messages
> > >   (I couldn't envision troubleshooting SMTP configs without
> > > the ability to
> > >   telnet SMTP commands)

This is analogy is not quite kosher IMO.  SMTP uses TCP/IP and you
don't get to troubleshoot TCP segmentation, flow control or IP
fragmentation when simulating SMTP via Telnet.  I guess you are
implying that syslog transport is part of the application layer. I
think, in the present form, it is not because it has pretty much
nothing in the protocol that is syslog-specific. It is just a generic
extension to the UDP protocol to support fragmentation.

> > > - it's instantly interoperable - no need for network
> bytes ordering

It is a small issue, agree. But so many IETF protocols use network
byte ordering (including TCP/UDP/IP) that it is pretty much a defacto
standard.

> > > - I think it reduces implementation errors

Maybe. But maybe there are additional errors in code that looks for
ASCII field delimiters, creates temporary strings, checks for valid
characters and performs conversion to system types with bounds
checking. Either way the code is simple IMO.

Data compactness and performance are not my primary concerns here.
Rather consistency with other protocols on the same layer and a
minimalist approach.

Here is an example of how ASCII transport header could look if we go
that way:

TransportHeaderVersion
[space]
MultiPartFalg
[space]
MessageID
[space]
TotalLength
[space]
Offset
[space]

Example of transport headers of two segments:

"1 1 12990 1024 0 "
"1 1 12990 1024 512 "

The single part message header could be:

"1 0 0 0 0 "

Or, with special rules, just:

"1 0 "

If people insist on this approach, I can go in that direction and
define limits for all of the fields. But IMO this would make for a
somewhat strange-looking transport encapsulation and probably reduce
applicability of this protocol to syslog.

I also think it is a slippery slope toward somebody asking that these
transport fields be encoded in XML because it makes it even more
easier to troubleshoot the protocol when you have field descriptors
right there... I think one needs to read a spec if he is
troubleshooting the protocol anyway & use a tool like Ethereal.

Anyway, I will proceed with WG consensus. Please voice your opinions
or preferences as I'd like to publish the next draft soon.

Thanks,
Anton.

------------------------------

Date: Fri, 23 Apr 2004 12:50:51 -0400
From: "Harrington, David" <[email-masked]>
Subject: RE: Issue 14: allow unqualified hostname

Hi

I believe there are instances when a software application may not know
its IP address and syslog should address that possibility. I favor the
"0.0.0.0" and "0:0:0:0:0:0:0:0" approach.

dbh

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Friday, April 23, 2004 10:53 AM
> To: Anton Okmianski
> Cc: [email-masked]
> Subject: RE: Issue 14: allow unqualified hostname
>
> > If syslog client sends a message over TCP/UDP/IP, it by
> definition has
> > an IP address.  If it send the message locally, then I
> think 127.0.0.1
> > (local IP) is more appropriate.
>
> I think the case we are describing is a very unusual one. It only
> happens when the device (better said: software running) has no way of
> *obtaining knowledge* of its IP address. I am not sure if we will find
> this in reality at all. So it does not matter (at least in my
> view) that
> it has an IP address - it can not get hold of it). So I am using a
> special value to say "I'm clueless".
>
> On the other hand, 127.0.0.1 is a valid value - it may be used when a
> sender talks to a receiver on the same machine.
>
> I would like to differentiate these two cases.
>
> > But I am assuming that we won't use syslog-protocol through
> say local
> > UNIX pipes like syslog does now locally on say Solaris.  If
> we do want
> > to support that, than we need the case of an unspecified IP.  But if
> > syslog is used over TCP/IP, "unspecified IP" is an oxymoron.
>
> As I said... depends. I am talking about a device *having* an
> IP address
> but the software not knowing it. If we all agree this is so unlikely
> that we should not support it, then I think we should fully drop this
> "last resort rule" - because we are saying it can never apply ;)
>
> Rainer
> >
> > Anton.
> >
> > > -----Original Message-----
> > > From: Rainer Gerhards [mailto:[email-masked]]
> > > Sent: Friday, April 23, 2004 9:01 AM
> > > To: [email-masked]
> > > Cc: Anton Okmianski
> > > Subject: RE: Issue 14: allow unqualified hostname
> > >
> > >
> > > Anton & all,
> > >
> > > in IPv6, we have the "unspecified address", which I think is
> > > exactly what we should use in the case an device does
> > > actually know nothing about itself (last case in Anton's
> > > messsage below) it is "0:0:0:0:0:0:0:0".
> > >
> > > Some links:
> > > http://www.iana.org/assignments/ipv6-address-space
> > > http://www.cisco.com/en/US/products/sw/iosswrel/ios_abcs_ios_t
> > > he_abcs_ip
> > > _version_60900aecd800c111d.html
> > >
> > > According to RFC 3330
> > > (http://www.rfc-editor.org/rfc/rfc3330.txt), I think we can
> > > also use "0.0.0.0" for IPv4 addressing in this case.
> > >
> > > Comments are highly appreciated.
> > >
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: Anton Okmianski [mailto:[email-masked]]
> > > > Sent: Monday, April 19, 2004 10:19 PM
> > > > To: Rainer Gerhards
> > > > Cc: [email-masked]
> > > > Subject: RE: Issue 14: allow unqualified hostname
> > > >
> > > > Rainer:
> > > >
> > > > I like Devin's suggestion of recommending a specific
> > > preference order:
> > > >
> > > > FQDN
> > > > Static IPv4/IPv6
> > > > Hostname
> > > > Dynamic IPv4/IPv6
> > > > "127.0.0.1" (when everything is unknown)
> > > >
> > > > Maybe the language should be a bit more restrictive than
> > > just SHOULDs
> > > > and MAYs here.  Maybe: "MUST provide FQDN if it is known.
> > > If unknown -
> > > > static IP. If unknown -- hostname. If unknown - dynamic IP.  If
> > > > unknown -- (a) can't use syslog or (b) we explain what
> they should
> >
> > > > use."
> > > >
> > > > I don't know if we decided on the last one. If syslog is to be
> > used
> > > > for remote logging only, then requiring knowledge of at
> > > least an IP is
> > > > acceptable.  If, however, we expect it to be used in host-local
> > > > scenarios as well, then we need to clarify what they should
> > > there when
> > > > nothing I known. Devin suggested 127.0.0.1. I like it.
> Maybe also
> > > > allow an IPv6 equivalent of that as well if it exists.
> > > >
> > > > Thanks,
> > > > Anton.
> > > >
> > > > > -----Original Message-----
> > > > > From: [email-masked]
> > > > > [mailto:[email-masked]] On Behalf Of
> > > Rainer Gerhards
> > > > > Sent: Monday, April 19, 2004 11:14 AM
> > > > > To: [email-masked]
> > > > > Subject: Issue 14: allow unqualified hostname
> > > > >
> > > > >
> > > > > Hi WG,
> > > > >
> > > > > this is in regard to issue 14, which talks about allowing the
> > > > > unqualified hostname. Based on previous feedback, I think
> > > > > this is concensus in the WG (see
> > > > > http://www.syslog.cc/ietf/protocol/issue14.htm> l
> > > > > for a short
> > > > > list).
> > > > >
> > > > > If nobody objects, I will go ahead and
> > > > > edit it in the following way:
> > > > >
> > > > > Hostname & FQDN SHOULD  be used, IP (v4/6) address or "bare"
> > > > > hostname MAY be used.
> > > > >
> > > > > Rainer
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
>
>
>
>

------------------------------

Date: Tue, 27 Apr 2004 08:26:45 -0700 (PDT)
From: Marshall Glen <[email-masked]>
Subject: draft-marshall-security-audit-09.txt

Chris,

Thanks.  Here's a quick overview ...

This document specifies an XML schema for a security audit message payload
to meet healthcare application requirements.  It was motivated by the
healthcare privacy laws, e.g., HIPAA in the US.  And it represents a
consensus view of three key healthcare standards organizations: HL7, DICOM ,
and ASTM E31.  Because of the array of possible transport mechanisms favored
by SDOs, this document is purposely transport-neutral.

Although this is an informative document, it is being used as the basis for
a normative supplement (#95) now under review by DICOM.  The supplement goes
further and specifies RFC 3195 Cooked message format.

HL7 may also incorporate this message into its normative work.  Their
favored application message transports are now ebXML and WSDL/SOAP, with
specifications as draft standards for trial use.

Additional work now underway by the Integrating the Healthcare Enterprise
(IHE) initiative will provide an implementation specification, leading to a
live multi-vendor integration test and demo at the HIMSS conference in
February, 2005.  The favored transport is RFC 3195 cooked format, but RFC
3164 is allowed.

Best,
Glen

- -----Original Message-----
From: Chris Lonvick [mailto:[email-masked]]
Sent: Tuesday, April 27, 2004 8:45 AM
To: Marshall Glen
Subject: I-D ACTION:draft-marshall-security-audit-09.txt (fwd)

Hi Glen,

I've been watching you keep up this document.  It looks like you're
getting ready to submit it to the IESG.  Please feel free to send a note
to the syslog WG mailing list ([email-masked]) if you'd like to
offer it for review or discussion.

Regards,
Chris

- ---------- Forwarded message ----------
Date: Fri, 23 Apr 2004 15:36:24 -0400
From: [email-masked]
To: [email-masked]
Subject: I-D ACTION:draft-marshall-security-audit-09.txt

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: Security Audit and Access Accountability Message
XML Data Definitions for Healthcare Applications
	Author(s)	: G. Marshall
	Filename	: draft-marshall-security-audit-09.txt
	Pages		: 45
	Date		: 2004-4-23

This document defines the format of data to be collected, and minimum
set of attributes that need to be captured, for security auditing in
healthcare application systems.  The format is defined as an XML
schema, which is intended as a reference for healthcare standards
developers and application designers.  It consolidates several
previous documents on security auditing of healthcare data.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-marshall-security-audit-09.txt

To remove yourself from the I-D Announcement list, send a message to
[email-masked] with the word unsubscribe in the body of the
message.
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-marshall-security-audit-09.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-marshall-security-audit-09.txt".

NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

- -------------------------------------------------------------------------------
This message and any included attachments are from Siemens Medical Solutions
USA, Inc. and are intended only for the addressee(s).
The information contained herein may include trade secrets or privileged or
otherwise confidential information.  Unauthorized review, forwarding, printing,
copying, distributing, or using such information is strictly prohibited and may
be unlawful.  If you received this message in error, or have reason to believe
you are not authorized to receive it, please promptly delete this message and
notify the sender by e-mail with a copy to [email-masked]

Thank you

------------------------------

Date: Thu, 29 Apr 2004 14:44:29 +0200
From: "Rainer Gerhards" <[email-masked]>
Subject: RE: binary fields in syslog

Anton & all:

comments to just to those things that I think I need to comment on - the
others are snipped out (basically agreement from my side).

> > > > - esay human interaction - not just reading but also (telnet)
> > > > crafting messages
> > > >   (I couldn't envision troubleshooting SMTP configs without
> > > > the ability to
> > > >   telnet SMTP commands)
>
> This is analogy is not quite kosher IMO.  SMTP uses TCP/IP and you
> don't get to troubleshoot TCP segmentation, flow control or IP
> fragmentation when simulating SMTP via Telnet.  I guess you are
> implying that syslog transport is part of the application layer. I
> think, in the present form, it is not because it has pretty much
> nothing in the protocol that is syslog-specific. It is just a generic
> extension to the UDP protocol to support fragmentation.

I think the analogy is right in this case. Because even though I do not
get all these things with telnet/netcat, I can use it for
troubleshooting. I can't use it for troubleshooting the UDP layer,
that's right - but that's not the thing I was asking for.

I think whenever you add something binary on top of UDP (or IP), you can
not use all those tools that work on generic packets, but you need a
specialised tool - which probably is much where the network operator's
position that David quoted comes from...

....

> I also think it is a slippery slope toward somebody asking that these
> transport fields be encoded in XML because it makes it even more
> easier to troubleshoot the protocol when you have field descriptors
> right there...

I think, this, too, is a separate issue. My point is that I would like
to use generic tools, not special ones. If we use ASCII on top of
UDP/TCP I can use generic tools. If we go binary, I can't. The ability
to use generic tools does not necessarily mean it needs to be overly
simple... at least this is my position.

> I think one needs to read a spec if he is
> troubleshooting the protocol anyway & use a tool like Ethereal.

yes, ethereal is among them. But that's a "watch-only" thing. You also
need to use tools to craft packages, and to the best of my (limited;))
knowledge this becomes hard if you go binary. I mean things that you can
find right out of the box on most systems...

Also please let me re-iterate my concerns about implementor adoption of
our work. We are already far away from traditional syslog. If we now
even go binary, I think we will loose a lot of momentum.

> Anyway, I will proceed with WG consensus. Please voice your opinions
> or preferences as I'd like to publish the next draft soon.

My choice is definitely towards an USASCII header.

Rainer

------------------------------

Date: Thu, 29 Apr 2004 15:53:02 -0400
From: [email-masked]
Subject: I-D ACTION:draft-ietf-syslog-protocol-04.txt

- --NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Security Issues in Network Event Logging Working Group of the IETF.

	Title		: The syslog Protocol
	Author(s)	: R. Gerhards
	Filename	: draft-ietf-syslog-protocol-04.txt
	Pages		: 40
	Date		: 2004-4-29

This document describes the syslog protocol. The syslog protocol has
   been used throughout the years to convey event notifications. This
   documents describes a layered architecture for an easily extensible
   syslog protocol. It also describes the basic message format and
   structured elements used to provide meta-information about the
   message.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-syslog-protocol-04.txt

To remove yourself from the I-D Announcement list, send a message to
[email-masked] with the word unsubscribe in the body of the message.
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-syslog-protocol-04.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-syslog-protocol-04.txt".

NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

- --NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

- --OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-syslog-protocol-04.txt

- --OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-syslog-protocol-04.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

- --OtherAccess--

- --NextPart--

------------------------------