
From HKaplan@acmepacket.com  Fri Jun 12 11:50:26 2009
Return-Path: <HKaplan@acmepacket.com>
X-Original-To: sip-clf@core3.amsl.com
Delivered-To: sip-clf@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AFF623A6AB9 for <sip-clf@core3.amsl.com>; Fri, 12 Jun 2009 11:50:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.299
X-Spam-Level: 
X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, J_CHICKENPOX_43=0.6]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YvLwn+h8Mlle for <sip-clf@core3.amsl.com>; Fri, 12 Jun 2009 11:50:25 -0700 (PDT)
Received: from etmail.acmepacket.com (etmail.acmepacket.com [216.41.24.6]) by core3.amsl.com (Postfix) with ESMTP id DFBE13A6C07 for <sip-clf@ietf.org>; Fri, 12 Jun 2009 11:50:24 -0700 (PDT)
Received: from mail.acmepacket.com (216.41.24.7) by etmail.acmepacket.com (216.41.24.6) with Microsoft SMTP Server (TLS) id 8.1.340.0; Fri, 12 Jun 2009 14:50:30 -0400
Received: from mail.acmepacket.com ([127.0.0.1]) by mail ([127.0.0.1]) with mapi; Fri, 12 Jun 2009 14:50:30 -0400
From: Hadriel Kaplan <HKaplan@acmepacket.com>
To: Adam Roach <adam@nostrum.com>, "sip-clf@ietf.org" <sip-clf@ietf.org>
Date: Fri, 12 Jun 2009 14:50:28 -0400
Thread-Topic: [sip-clf] New CLF Syntax draft (text with index)
Thread-Index: AcnPRY2QTmstE+HIQiewPQ2dwFq5SAcQPVAQ
Message-ID: <E6C2E8958BA59A4FB960963D475F7AC31940E573DB@mail>
References: <4A032ED7.7030504@nostrum.com>
In-Reply-To: <4A032ED7.7030504@nostrum.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [sip-clf] New CLF Syntax draft (text with index)
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: SIP Common Log File format discussion list <sip-clf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/sip-clf>, <mailto:sip-clf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sip-clf>
List-Post: <mailto:sip-clf@ietf.org>
List-Help: <mailto:sip-clf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sip-clf>, <mailto:sip-clf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Jun 2009 18:50:26 -0000

Howdy,
Sorry for being late in draft comments.  Ingenious concept to encode it in =
ascii, and all, but I do wonder about the overhead in all this.  Personally=
 I don't care that much about how much work it will be for the decoder/tool=
 to read a CLF file, as much as I care how much overhead it is for a SIP de=
vice to generate the CLF.  A few extra instructions overall to help the rea=
der-tool out is ok, but extra instructions/work per field is pushing it.  S=
o my comments will probably seem obvious:

1) Encoding the field contents with terminating TAB characters is not only =
an extra instruction per field content, but also involves an entire scan of=
 each field content to replace real TAB's with spaces/whatever.  At least i=
f the goal of the encoding is to make it usable by text-parsing tools, with=
out taking the length values into account. (is that the goal?)  This was ex=
actly the type of thing we were trying to avoid by using binary encoding to=
 begin with, I thought: not having to search the received on-the-wire heade=
r content for the "special" character we use for separation, to escape/repl=
ace it.

2) Encoding the pointers/lengths in hex-ascii is an extra itoa(val,16) type=
 of function per length/ptr field.  I have no idea how much real work that =
is, but it's work that I don't have to do.  It also makes the file larger, =
since a 2-byte value is taking 4-bytes to encode.

3) This is sorta a nit, but the entire record length can be ~280 Terabytes,=
 if I'm doing the math right.  Obviously it's up to the logger if it ever w=
ants to record that much, but I have a hard time really imagining the need =
for anything beyond 4GB (ie, a 32-bit value).  So if we can encode binary a=
s binary, the record length field could reasonably be 32-bits instead of th=
is 48-bit thing.

IANAE, but I think most systems these days are memory-constrained for perfo=
rmance, rather than instruction-contrained (i.e., accessing memory is the b=
ottleneck, not instruction cycles).  So I don't know how much of the work a=
bove is really noticeable work for systems, except for the scanning of cont=
ent issue.

-hadriel


> -----Original Message-----
> From: sip-clf-bounces@ietf.org [mailto:sip-clf-bounces@ietf.org] On Behal=
f
> Of Adam Roach
> Sent: Thursday, May 07, 2009 2:56 PM
> To: sip-clf@ietf.org
> Subject: [sip-clf] New CLF Syntax draft (text with index)
>=20
> This version defines a text format in which each record is composed of
> two lines in a log file. The first line is primarily pointers into the
> second line. The second line contains the actual logged fields,
> separated by tab characters.
>=20
> This approach retains the fast-search capabilities that I've been
> advocating, while allowing the use of simple, text-based tools, as Vijay
> has been promoting. This hybrid approach does come with a slight
> increase in log file size; for example, writing out the same 100,000 log
> entries in each of the three formats proposed so far:
>=20
>    - Text:   25 Mb
>    - Binary: 29 Mb
>    - Hybrid: 37 Mb
>=20
> This is about a 20% premium over the binary format, and a 48% premium
> over the text format. Speed of generation should be on the same order of
> speed as the other two versions, and speed of processing should be
> approximately the same as processing binary.
>=20
> The new version of the document can be found here:
>=20
> http://tools.ietf.org/html/draft-roach-sipping-clf-syntax-01
>=20
> /a
> _______________________________________________
> sip-clf mailing list
> sip-clf@ietf.org
> https://www.ietf.org/mailman/listinfo/sip-clf

From HKaplan@acmepacket.com  Wed Jun 17 11:29:48 2009
Return-Path: <HKaplan@acmepacket.com>
X-Original-To: sip-clf@core3.amsl.com
Delivered-To: sip-clf@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A26C228C1B2; Wed, 17 Jun 2009 11:29:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.412
X-Spam-Level: 
X-Spam-Status: No, score=-2.412 tagged_above=-999 required=5 tests=[AWL=0.188,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0vrP-Vd6vO5V; Wed, 17 Jun 2009 11:29:47 -0700 (PDT)
Received: from etmail.acmepacket.com (etmail.acmepacket.com [216.41.24.6]) by core3.amsl.com (Postfix) with ESMTP id ADCD028C278; Wed, 17 Jun 2009 11:29:47 -0700 (PDT)
Received: from mail.acmepacket.com (216.41.24.7) by etmail.acmepacket.com (216.41.24.6) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 17 Jun 2009 14:29:58 -0400
Received: from mail.acmepacket.com ([127.0.0.1]) by mail ([127.0.0.1]) with mapi; Wed, 17 Jun 2009 14:29:58 -0400
From: Hadriel Kaplan <HKaplan@acmepacket.com>
To: "sip-clf@ietf.org" <sip-clf@ietf.org>
Date: Wed, 17 Jun 2009 14:29:47 -0400
Thread-Topic: I-D Action:draft-kaplan-sipping-clf-pcap-00.txt 
Thread-Index: AcnvADVbRjylz/iUR3uTiC6eLPbhbQAd2SIg
Message-ID: <E6C2E8958BA59A4FB960963D475F7AC31941848484@mail>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: multipart/mixed; boundary="_002_E6C2E8958BA59A4FB960963D475F7AC31941848484mail_"
MIME-Version: 1.0
Cc: "dispatch@ietf.org" <dispatch@ietf.org>
Subject: [sip-clf] FW: I-D Action:draft-kaplan-sipping-clf-pcap-00.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: SIP Common Log File format discussion list <sip-clf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/sip-clf>, <mailto:sip-clf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sip-clf>
List-Post: <mailto:sip-clf@ietf.org>
List-Help: <mailto:sip-clf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sip-clf>, <mailto:sip-clf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Jun 2009 18:29:48 -0000

--_002_E6C2E8958BA59A4FB960963D475F7AC31941848484mail_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Howdy,
I have submitted an I-D for Yet Another SIP-CLF Format.  This one is pure b=
inary, and follows the libpcap/PCAP file format, encoding the fields into a=
 format which just happens to be decode-able as RADIUS messages by PCAP dec=
oders.  One of the advantages to this format is off-line decoders such as W=
ireshark can decode it right now, with only a dictionary for the VSA used f=
or SIP-CLF.  For wireshark, for example, this can be done without re-compil=
ing/upgrading wireshark.  Just a 30-second procedure and restarting Wiresha=
rk enables one to view, filter, and search through the SIP-CLF fields as is=
.

The I-D's link is below, and there's a sample CLF file and dictionary file =
and Wireshark instructions (and even screenshot!) available on:
https://sourceforge.net/projects/sip-clf/

There's also an open-source C-code library implementation available there.

-hadriel

> -----Original Message-----
> From: i-d-announce-bounces@ietf.org [mailto:i-d-announce-bounces@ietf.org=
]
> On Behalf Of Internet-Drafts@ietf.org
>=20
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>=20
> 	Title           : PCAP-compatible Binary Syntax for SIP Common Log
> File Format
> 	Author(s)       : H. Kaplan
> 	Filename        : draft-kaplan-sipping-clf-pcap-00.txt
> 	Pages           : 20
> 	Date            : 2009-06-16
>=20
> This document proposes a libpcap/PCAP-compatible binary syntax for
> the SIP common log format (CLF).  It does not cover semantic
> issues, and is meant to be evaluated in the context of the other
> efforts discussing SIP CLF.
>=20
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-kaplan-sipping-clf-pcap-00.txt



--_002_E6C2E8958BA59A4FB960963D475F7AC31941848484mail_
Content-Type: text/plain; name="draft-kaplan-sipping-clf-pcap-00.url.txt"
Content-Description: draft-kaplan-sipping-clf-pcap-00.url.txt
Content-Disposition: attachment;
	filename="draft-kaplan-sipping-clf-pcap-00.url.txt"; size=97;
	creation-date="Wed, 17 Jun 2009 00:00:53 GMT";
	modification-date="Wed, 17 Jun 2009 00:00:53 GMT"
Content-Transfer-Encoding: base64

VGhpcyBhdHRhY2htZW50IHdhcyByZW1vdmVkLg==

--_002_E6C2E8958BA59A4FB960963D475F7AC31941848484mail_--
