From internet-drafts@ietf.org Thu Oct 4 18:56:21 2012 Return-Path: X-Original-To: sip-clf@ietfa.amsl.com Delivered-To: sip-clf@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A27A21F8525; Thu, 4 Oct 2012 18:56:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.493 X-Spam-Level: X-Spam-Status: No, score=-102.493 tagged_above=-999 required=5 tests=[AWL=0.106, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HVGPpOBizrqc; Thu, 4 Oct 2012 18:56:20 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 82A9921F8549; Thu, 4 Oct 2012 18:56:20 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.34 Message-ID: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> Date: Thu, 04 Oct 2012 18:56:20 -0700 Cc: sip-clf@ietf.org Subject: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt X-BeenThere: sip-clf@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: SIP Common Log File format discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 01:56:21 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the SIP Common Log Format Working Group of th= e IETF. Title : Format for the Session Initiation Protocol (SIP) Common = Log Format (CLF) Author(s) : Gonzalo Salgueiro Vijay Gurbani Adam Roach Filename : draft-ietf-sipclf-format-07.txt Pages : 30 Date : 2012-10-04 Abstract: The SIPCLF Workgroup has defined a common log format framework for Session Initiation Protocol (SIP) servers. This common log format mimics the successful event logging format found in well-known web servers like Apache and web proxies like Squid. This document proposes an indexed text encoding format for the SIP Common Log Format (CLF) that retains the key advantages of a text-based format, while significantly increasing processing performance over a purely text-based implementation. This file format adheres to the SIP CLF data model and provides an effective encoding scheme for all mandatory and optional fields that appear in a SIP CLF record. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-sipclf-format There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-sipclf-format-07 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-sipclf-format-07 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From gsalguei@cisco.com Thu Oct 4 19:13:28 2012 Return-Path: X-Original-To: sip-clf@ietfa.amsl.com Delivered-To: sip-clf@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1896D21F84D7 for ; Thu, 4 Oct 2012 19:13:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.513 X-Spam-Level: X-Spam-Status: No, score=-10.513 tagged_above=-999 required=5 tests=[AWL=0.086, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eU6VUyldwhSK for ; Thu, 4 Oct 2012 19:13:27 -0700 (PDT) Received: from av-tac-rtp.cisco.com (av-tac-rtp.cisco.com [64.102.19.209]) by ietfa.amsl.com (Postfix) with ESMTP id 3DEFE21F84CD for ; Thu, 4 Oct 2012 19:13:27 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from chook.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-rtp.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q952DQJL029228 for ; Thu, 4 Oct 2012 22:13:26 -0400 (EDT) Received: from rtp-gsalguei-8914.cisco.com (rtp-gsalguei-8914.cisco.com [10.116.132.53]) by chook.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q952DMK1006968; Thu, 4 Oct 2012 22:13:22 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Gonzalo Salgueiro In-Reply-To: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> Date: Thu, 4 Oct 2012 22:13:22 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> To: "sip-clf@ietf.org Mailing" X-Mailer: Apple Mail (2.1283) Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt X-BeenThere: sip-clf@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: SIP Common Log File format discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 02:13:28 -0000 Folks: I have posted the -07 version of the SIP CLF Format Draft and it is now = in the archives at [1]. This draft was meant to address the few remaining AD review items that = were raised. The diffs between the -06 and the -07 versions are at [2] The principal changes include: - Introduced Base64 Encoded Byte (BEB) to deal with optional logging of = non-UTF-8 message bodies - Added an example using BEB for logging a binary body - Clarified and expanded the text dealing with encoding, character = escaping, and usage of SIP CLF and put it all into its own section - Addressed several minor issues and nits =20 [1] http://tools.ietf.org/html/draft-ietf-sipclf-format [2] http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-sipclf-format-07 Regards, Gonzalo On Mar 12, 2012, at 3:13 PM, Internet-Drafts@ietf.org wrote: >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. This draft is a work item of the SIP Common Log Format = Working Group of the IETF. >=20 > Title : Format for the Session Initiation Protocol = (SIP) Common Log Format (CLF) > Author(s) : Gonzalo Salgueiro > Vijay Gurbani > Adam Roach > Filename : draft-ietf-sipclf-format-06.txt > Pages : 28 > Date : 2012-03-12 >=20 > The SIPCLF Workgroup has defined a common log format framework for > Session Initiation Protocol (SIP) servers. This common log format > mimics the successful event logging format found in well-known web > servers like Apache and web proxies like Squid. This document > proposes an indexed text encoding format for the SIP Common Log > Format (CLF) that retains the key advantages of a text-based format, > while significantly increasing processing performance over a purely > text-based implementation. This file format adheres to the SIP CLF > data model and provides an effective encoding scheme for all > mandatory and optional fields that appear in a SIP CLF record. >=20 >=20 > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > This Internet-Draft can be retrieved at: > ftp://ftp.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt >=20 > _______________________________________________ > sip-clf mailing list > sip-clf@ietf.org > https://www.ietf.org/mailman/listinfo/sip-clf >=20 On Oct 4, 2012, at 9:56 PM, Internet-Drafts@ietf.org wrote: >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. > This draft is a work item of the SIP Common Log Format Working Group = of the IETF. >=20 > Title : Format for the Session Initiation Protocol = (SIP) Common Log Format (CLF) > Author(s) : Gonzalo Salgueiro > Vijay Gurbani > Adam Roach > Filename : draft-ietf-sipclf-format-07.txt > Pages : 30 > Date : 2012-10-04 >=20 > Abstract: > The SIPCLF Workgroup has defined a common log format framework for > Session Initiation Protocol (SIP) servers. This common log format > mimics the successful event logging format found in well-known web > servers like Apache and web proxies like Squid. This document > proposes an indexed text encoding format for the SIP Common Log > Format (CLF) that retains the key advantages of a text-based format, > while significantly increasing processing performance over a purely > text-based implementation. This file format adheres to the SIP CLF > data model and provides an effective encoding scheme for all > mandatory and optional fields that appear in a SIP CLF record. >=20 >=20 > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-sipclf-format >=20 > There's also a htmlized version available at: > http://tools.ietf.org/html/draft-ietf-sipclf-format-07 >=20 > A diff from the previous version is available at: > http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-sipclf-format-07 >=20 >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > _______________________________________________ > I-D-Announce mailing list > I-D-Announce@ietf.org > https://www.ietf.org/mailman/listinfo/i-d-announce > Internet-Draft directories: http://www.ietf.org/shadow.html > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt >=20 From rjsparks@nostrum.com Tue Oct 23 14:07:18 2012 Return-Path: X-Original-To: sip-clf@ietfa.amsl.com Delivered-To: sip-clf@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C3DF1F0424 for ; Tue, 23 Oct 2012 14:07:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E5cN3P7fJ8z3 for ; Tue, 23 Oct 2012 14:07:17 -0700 (PDT) Received: from shaman.nostrum.com (nostrum-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:267::2]) by ietfa.amsl.com (Postfix) with ESMTP id 0537011E80BF for ; Tue, 23 Oct 2012 14:07:14 -0700 (PDT) Received: from unnumerable.local (pool-173-57-93-208.dllstx.fios.verizon.net [173.57.93.208]) (authenticated bits=0) by shaman.nostrum.com (8.14.3/8.14.3) with ESMTP id q9NL750i044578 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 23 Oct 2012 16:07:06 -0500 (CDT) (envelope-from rjsparks@nostrum.com) Message-ID: <508706F9.1020507@nostrum.com> Date: Tue, 23 Oct 2012 16:07:05 -0500 From: Robert Sparks User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Gonzalo Salgueiro References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> In-Reply-To: <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> Content-Type: multipart/alternative; boundary="------------040002060907000702050306" Received-SPF: pass (nostrum.com: 173.57.93.208 is authenticated by a trusted mechanism) Cc: "sip-clf@ietf.org Mailing" Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt X-BeenThere: sip-clf@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: SIP Common Log File format discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 21:07:18 -0000 This is a multi-part message in MIME format. --------------040002060907000702050306 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit The list has been silent on this. It needs review and comment please. There is one part of the new text that is not right yet. " The Value field for the optional element being logged MUST be Base64 encoded if it has any characters outside of the UTF-8 character set." doesn't make sense. This should be saying something like "if it contains any octets with values between 0 and 31 inclusive, or 127". RjS On 10/4/12 9:13 PM, Gonzalo Salgueiro wrote: > Folks: > > I have posted the -07 version of the SIP CLF Format Draft and it is now in the archives at [1]. > > This draft was meant to address the few remaining AD review items that were raised. The diffs between the -06 and the -07 versions are at [2] > > The principal changes include: > > - Introduced Base64 Encoded Byte (BEB) to deal with optional logging of non-UTF-8 message bodies > - Added an example using BEB for logging a binary body > - Clarified and expanded the text dealing with encoding, character escaping, and usage of SIP CLF and put it all into its own section > - Addressed several minor issues and nits > > [1] http://tools.ietf.org/html/draft-ietf-sipclf-format > [2] http://www.ietf.org/rfcdiff?url2=draft-ietf-sipclf-format-07 > > Regards, > > Gonzalo > > On Mar 12, 2012, at 3:13 PM, Internet-Drafts@ietf.org wrote: > >> A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the SIP Common Log Format Working Group of the IETF. >> >> Title : Format for the Session Initiation Protocol (SIP) Common Log Format (CLF) >> Author(s) : Gonzalo Salgueiro >> Vijay Gurbani >> Adam Roach >> Filename : draft-ietf-sipclf-format-06.txt >> Pages : 28 >> Date : 2012-03-12 >> >> The SIPCLF Workgroup has defined a common log format framework for >> Session Initiation Protocol (SIP) servers. This common log format >> mimics the successful event logging format found in well-known web >> servers like Apache and web proxies like Squid. This document >> proposes an indexed text encoding format for the SIP Common Log >> Format (CLF) that retains the key advantages of a text-based format, >> while significantly increasing processing performance over a purely >> text-based implementation. This file format adheres to the SIP CLF >> data model and provides an effective encoding scheme for all >> mandatory and optional fields that appear in a SIP CLF record. >> >> >> A URL for this Internet-Draft is: >> http://www.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt >> >> Internet-Drafts are also available by anonymous FTP at: >> ftp://ftp.ietf.org/internet-drafts/ >> >> This Internet-Draft can be retrieved at: >> ftp://ftp.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt >> >> _______________________________________________ >> sip-clf mailing list >> sip-clf@ietf.org >> https://www.ietf.org/mailman/listinfo/sip-clf >> > > On Oct 4, 2012, at 9:56 PM, Internet-Drafts@ietf.org wrote: > >> A New Internet-Draft is available from the on-line Internet-Drafts directories. >> This draft is a work item of the SIP Common Log Format Working Group of the IETF. >> >> Title : Format for the Session Initiation Protocol (SIP) Common Log Format (CLF) >> Author(s) : Gonzalo Salgueiro >> Vijay Gurbani >> Adam Roach >> Filename : draft-ietf-sipclf-format-07.txt >> Pages : 30 >> Date : 2012-10-04 >> >> Abstract: >> The SIPCLF Workgroup has defined a common log format framework for >> Session Initiation Protocol (SIP) servers. This common log format >> mimics the successful event logging format found in well-known web >> servers like Apache and web proxies like Squid. This document >> proposes an indexed text encoding format for the SIP Common Log >> Format (CLF) that retains the key advantages of a text-based format, >> while significantly increasing processing performance over a purely >> text-based implementation. This file format adheres to the SIP CLF >> data model and provides an effective encoding scheme for all >> mandatory and optional fields that appear in a SIP CLF record. >> >> >> The IETF datatracker status page for this draft is: >> https://datatracker.ietf.org/doc/draft-ietf-sipclf-format >> >> There's also a htmlized version available at: >> http://tools.ietf.org/html/draft-ietf-sipclf-format-07 >> >> A diff from the previous version is available at: >> http://www.ietf.org/rfcdiff?url2=draft-ietf-sipclf-format-07 >> >> >> Internet-Drafts are also available by anonymous FTP at: >> ftp://ftp.ietf.org/internet-drafts/ >> >> _______________________________________________ >> I-D-Announce mailing list >> I-D-Announce@ietf.org >> https://www.ietf.org/mailman/listinfo/i-d-announce >> Internet-Draft directories: http://www.ietf.org/shadow.html >> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt >> --------------040002060907000702050306 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
The list has been silent on this. It needs review and comment please.

There is one part of the new text that is not right yet.

" The Value field for the optional element being logged MUST be Base64
   encoded if it has any characters outside of the UTF-8 character set."
doesn't make sense.

This should be saying something like  "if it contains any octets with values between 0 and 31 inclusive, or 127".

RjS

On 10/4/12 9:13 PM, Gonzalo Salgueiro wrote:
Folks:

I have posted the -07 version of the SIP CLF Format Draft and it is now in the archives at [1].

This draft was meant to address the few remaining AD review items that were raised. The diffs between the -06 and the -07 versions are at [2]

The principal changes include:

- Introduced Base64 Encoded Byte (BEB) to deal with optional logging of non-UTF-8 message bodies
- Added an example using BEB for logging a binary body
- Clarified and expanded the text dealing with encoding, character escaping, and usage of SIP CLF and put it all into its own section
- Addressed several minor issues and nits
 
[1] http://tools.ietf.org/html/draft-ietf-sipclf-format
[2] http://www.ietf.org/rfcdiff?url2=draft-ietf-sipclf-format-07

Regards,

Gonzalo

On Mar 12, 2012, at 3:13 PM, Internet-Drafts@ietf.org wrote:


A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the SIP Common Log Format Working Group of the IETF.

	Title           : Format for the Session Initiation Protocol (SIP) Common Log Format (CLF)
	Author(s)       : Gonzalo Salgueiro
                         Vijay Gurbani
                         Adam Roach
	Filename        : draft-ietf-sipclf-format-06.txt
	Pages           : 28
	Date            : 2012-03-12

  The SIPCLF Workgroup has defined a common log format framework for
  Session Initiation Protocol (SIP) servers.  This common log format
  mimics the successful event logging format found in well-known web
  servers like Apache and web proxies like Squid.  This document
  proposes an indexed text encoding format for the SIP Common Log
  Format (CLF) that retains the key advantages of a text-based format,
  while significantly increasing processing performance over a purely
  text-based implementation.  This file format adheres to the SIP CLF
  data model and provides an effective encoding scheme for all
  mandatory and optional fields that appear in a SIP CLF record.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt

_______________________________________________
sip-clf mailing list
sip-clf@ietf.org
https://www.ietf.org/mailman/listinfo/sip-clf



On Oct 4, 2012, at 9:56 PM, Internet-Drafts@ietf.org wrote:


A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the SIP Common Log Format Working Group of the IETF.

	Title           : Format for the Session Initiation Protocol (SIP) Common Log Format (CLF)
	Author(s)       : Gonzalo Salgueiro
                         Vijay Gurbani
                         Adam Roach
	Filename        : draft-ietf-sipclf-format-07.txt
	Pages           : 30
	Date            : 2012-10-04

Abstract:
  The SIPCLF Workgroup has defined a common log format framework for
  Session Initiation Protocol (SIP) servers.  This common log format
  mimics the successful event logging format found in well-known web
  servers like Apache and web proxies like Squid.  This document
  proposes an indexed text encoding format for the SIP Common Log
  Format (CLF) that retains the key advantages of a text-based format,
  while significantly increasing processing performance over a purely
  text-based implementation.  This file format adheres to the SIP CLF
  data model and provides an effective encoding scheme for all
  mandatory and optional fields that appear in a SIP CLF record.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-sipclf-format

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-sipclf-format-07

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-sipclf-format-07


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
I-D-Announce@ietf.org
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt



    

    

  


--------------040002060907000702050306--

From gsalguei@cisco.com  Tue Oct 23 14:28:33 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5094C1F0C95 for ; Tue, 23 Oct 2012 14:28:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.465
X-Spam-Level: 
X-Spam-Status: No, score=-10.465 tagged_above=-999 required=5 tests=[AWL=0.134, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MmOZ-JcOCvOm for ; Tue, 23 Oct 2012 14:28:32 -0700 (PDT)
Received: from av-tac-rtp.cisco.com (av-tac-rtp.cisco.com [64.102.19.209]) by ietfa.amsl.com (Postfix) with ESMTP id 829751F0C91 for ; Tue, 23 Oct 2012 14:28:32 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from chook.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-rtp.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9NLSVsJ020389 for ; Tue, 23 Oct 2012 17:28:31 -0400 (EDT)
Received: from dhcp-10-150-54-114.cisco.com (dhcp-10-150-54-114.cisco.com [10.150.54.114]) by chook.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9NLSQur012569; Tue, 23 Oct 2012 17:28:27 -0400 (EDT)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Gonzalo Salgueiro 
In-Reply-To: <508706F9.1020507@nostrum.com>
Date: Tue, 23 Oct 2012 17:28:26 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <03DA5392-9085-4713-B605-2F986B3C02AE@cisco.com>
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <508706F9.1020507@nostrum.com>
To: Robert Sparks 
X-Mailer: Apple Mail (2.1283)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 23 Oct 2012 21:28:33 -0000

On Oct 23, 2012, at 5:07 PM, Robert Sparks wrote:

> The list has been silent on this. It needs review and comment please.
>=20
> There is one part of the new text that is not right yet.
>=20
> " The Value field for the optional element being logged MUST be Base64
>    encoded if it has any characters outside of the UTF-8 character =
set."
> doesn't make sense.
>=20
> This should be saying something like  "if it contains any octets with =
values between 0 and 31 inclusive, or 127".

Yep, this was a stupid oversight by me on the non-printable characters =
within UTF-8. I'll replace the above quoted text with the following:

The Value field for the optional element being logged MUST be Base64 =
encoded if it has any non-printable UTF-8 characters or characters =
outside of the UTF-8 character set.

Does that sound fine?

Gonzalo




>=20
> RjS
>=20
> On 10/4/12 9:13 PM, Gonzalo Salgueiro wrote:
>> Folks:
>>=20
>> I have posted the -07 version of the SIP CLF Format Draft and it is =
now in the archives at [1].
>>=20
>> This draft was meant to address the few remaining AD review items =
that were raised. The diffs between the -06 and the -07 versions are at =
[2]
>>=20
>> The principal changes include:
>>=20
>> - Introduced Base64 Encoded Byte (BEB) to deal with optional logging =
of non-UTF-8 message bodies
>> - Added an example using BEB for logging a binary body
>> - Clarified and expanded the text dealing with encoding, character =
escaping, and usage of SIP CLF and put it all into its own section
>> - Addressed several minor issues and nits
>> =20
>> [1]=20
>> http://tools.ietf.org/html/draft-ietf-sipclf-format
>>=20
>> [2]=20
>> http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-sipclf-format-07
>>=20
>>=20
>> Regards,
>>=20
>> Gonzalo
>>=20
>> On Mar 12, 2012, at 3:13 PM,=20
>> Internet-Drafts@ietf.org
>>  wrote:
>>=20
>>=20
>>> A New Internet-Draft is available from the on-line Internet-Drafts =
directories. This draft is a work item of the SIP Common Log Format =
Working Group of the IETF.
>>>=20
>>> 	Title           : Format for the Session Initiation Protocol =
(SIP) Common Log Format (CLF)
>>> 	Author(s)       : Gonzalo Salgueiro
>>>                          Vijay Gurbani
>>>                          Adam Roach
>>> 	Filename        : draft-ietf-sipclf-format-06.txt
>>> 	Pages           : 28
>>> 	Date            : 2012-03-12
>>>=20
>>>   The SIPCLF Workgroup has defined a common log format framework for
>>>   Session Initiation Protocol (SIP) servers.  This common log format
>>>   mimics the successful event logging format found in well-known web
>>>   servers like Apache and web proxies like Squid.  This document
>>>   proposes an indexed text encoding format for the SIP Common Log
>>>   Format (CLF) that retains the key advantages of a text-based =
format,
>>>   while significantly increasing processing performance over a =
purely
>>>   text-based implementation.  This file format adheres to the SIP =
CLF
>>>   data model and provides an effective encoding scheme for all
>>>   mandatory and optional fields that appear in a SIP CLF record.
>>>=20
>>>=20
>>> A URL for this Internet-Draft is:
>>>=20
>>> http://www.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt
>>>=20
>>>=20
>>> Internet-Drafts are also available by anonymous FTP at:
>>>=20
>>> ftp://ftp.ietf.org/internet-drafts/
>>>=20
>>>=20
>>> This Internet-Draft can be retrieved at:
>>>=20
>>> ftp://ftp.ietf.org/internet-drafts/draft-ietf-sipclf-format-06.txt
>>>=20
>>>=20
>>> _______________________________________________
>>> sip-clf mailing list
>>>=20
>>> sip-clf@ietf.org
>>> https://www.ietf.org/mailman/listinfo/sip-clf
>>>=20
>>>=20
>>>=20
>>=20
>> On Oct 4, 2012, at 9:56 PM,=20
>> Internet-Drafts@ietf.org
>>  wrote:
>>=20
>>=20
>>> A New Internet-Draft is available from the on-line Internet-Drafts =
directories.
>>> This draft is a work item of the SIP Common Log Format Working Group =
of the IETF.
>>>=20
>>> 	Title           : Format for the Session Initiation Protocol =
(SIP) Common Log Format (CLF)
>>> 	Author(s)       : Gonzalo Salgueiro
>>>                          Vijay Gurbani
>>>                          Adam Roach
>>> 	Filename        : draft-ietf-sipclf-format-07.txt
>>> 	Pages           : 30
>>> 	Date            : 2012-10-04
>>>=20
>>> Abstract:
>>>   The SIPCLF Workgroup has defined a common log format framework for
>>>   Session Initiation Protocol (SIP) servers.  This common log format
>>>   mimics the successful event logging format found in well-known web
>>>   servers like Apache and web proxies like Squid.  This document
>>>   proposes an indexed text encoding format for the SIP Common Log
>>>   Format (CLF) that retains the key advantages of a text-based =
format,
>>>   while significantly increasing processing performance over a =
purely
>>>   text-based implementation.  This file format adheres to the SIP =
CLF
>>>   data model and provides an effective encoding scheme for all
>>>   mandatory and optional fields that appear in a SIP CLF record.
>>>=20
>>>=20
>>> The IETF datatracker status page for this draft is:
>>>=20
>>> https://datatracker.ietf.org/doc/draft-ietf-sipclf-format
>>>=20
>>>=20
>>> There's also a htmlized version available at:
>>>=20
>>> http://tools.ietf.org/html/draft-ietf-sipclf-format-07
>>>=20
>>>=20
>>> A diff from the previous version is available at:
>>>=20
>>> http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-sipclf-format-07
>>>=20
>>>=20
>>>=20
>>> Internet-Drafts are also available by anonymous FTP at:
>>>=20
>>> ftp://ftp.ietf.org/internet-drafts/
>>>=20
>>>=20
>>> _______________________________________________
>>> I-D-Announce mailing list
>>>=20
>>> I-D-Announce@ietf.org
>>> https://www.ietf.org/mailman/listinfo/i-d-announce
>>>=20
>>> Internet-Draft directories:=20
>>> http://www.ietf.org/shadow.html
>>>=20
>>> or=20
>>> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>>>=20
>>>=20
>>>=20
>=20


From adam@nostrum.com  Tue Oct 23 14:31:45 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 070531F0C96 for ; Tue, 23 Oct 2012 14:31:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.194
X-Spam-Level: 
X-Spam-Status: No, score=-102.194 tagged_above=-999 required=5 tests=[AWL=0.406, BAYES_00=-2.599, HTML_MESSAGE=0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GFaJDeuXaVJd for ; Tue, 23 Oct 2012 14:31:44 -0700 (PDT)
Received: from shaman.nostrum.com (nostrum-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:267::2]) by ietfa.amsl.com (Postfix) with ESMTP id E498A1F0C95 for ; Tue, 23 Oct 2012 14:31:42 -0700 (PDT)
Received: from hydra-en0.roach.at (99-152-144-32.lightspeed.dllstx.sbcglobal.net [99.152.144.32]) (authenticated bits=0) by shaman.nostrum.com (8.14.3/8.14.3) with ESMTP id q9NLVawX047236 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 23 Oct 2012 16:31:37 -0500 (CDT) (envelope-from adam@nostrum.com)
Message-ID: <50870CB8.40908@nostrum.com>
Date: Tue, 23 Oct 2012 16:31:36 -0500
From: Adam Roach 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:14.0) Gecko/20120713 Thunderbird/14.0
MIME-Version: 1.0
To: Gonzalo Salgueiro 
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com>
In-Reply-To: <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com>
Content-Type: multipart/alternative; boundary="------------080205060305070604080501"
Received-SPF: pass (nostrum.com: 99.152.144.32 is authenticated by a trusted mechanism)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 23 Oct 2012 21:31:45 -0000

This is a multi-part message in MIME format.
--------------080205060305070604080501
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 10/4/12 21:13, Oct 4, Gonzalo Salgueiro wrote:
> Folks:
>
> I have posted the -07 version of the SIP CLF Format Draft and it is now in the archives at [1].
>
> This draft was meant to address the few remaining AD review items that were raised. The diffs between the -06 and the -07 versions are at [2]
>
> The principal changes include:
>
> - Introduced Base64 Encoded Byte (BEB) to deal with optional logging of non-UTF-8 message bodies

I think the text is a bit confused on what UTF-8 means. The phrase "any 
characters outside the UTF-8 character set" appears many times, and I'm 
afraid it doesn't really make any sense.

What you probably want to do is define a term somewhere in the document 
to formally specify "Those Things That Must Be Base-64 Encoded". I would 
recommend something like:

"For the purposes of this document, we define 'unprintable' to mean a 
string of octets that: (a) contains an octet with a value in the range 
of 0 to 31, inclusive; (b) contains an octet with a value of 127; or (c) 
contains any series of octets greater than or equal to 128 which do not 
form a valid UTF-8 sequence, as specified by [UNICODE]."

Then, your statements become far simpler; for example: "The Value field 
for the optional element being logged MUST be Base64 encoded if it is 
unprintable." And: "For an optionally logged message body (Tag=01) the 
BEB is set according to whether the message body is unprintable." And: 
"When optionally logging an entire SIP message (Tag=02) the BEB is set 
according to whether the message body portion is unprintable."

You get the idea.

/a


P.S. The above definition would require adding a reference for 
[UNICODE], like so:

    [UNICODE] The Unicode Consortium. The Unicode Standard,
              Version 6.2.0, (Mountain View, CA: The Unicode
              Consortium, 2012. ISBN 978-1-936213-07-8)
              


--------------080205060305070604080501
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    
On 10/4/12 21:13, Oct 4, Gonzalo
      Salgueiro wrote:

    

    

      
Folks:

I have posted the -07 version of the SIP CLF Format Draft and it is now in the archives at [1].

This draft was meant to address the few remaining AD review items that were raised. The diffs between the -06 and the -07 versions are at [2]

The principal changes include:

- Introduced Base64 Encoded Byte (BEB) to deal with optional logging of non-UTF-8 message bodies

    

    

    I think the text is a bit confused on what UTF-8 means. The phrase
    "any characters outside the UTF-8 character set" appears many times,
    and I'm afraid it doesn't really make any sense.

    

    What you probably want to do is define a term somewhere in the
    document to formally specify "Those Things That Must Be Base-64
    Encoded". I would recommend something like:

    

    "For the purposes of this document, we define 'unprintable' to mean
    a string of octets that: (a) contains an octet with a value in the
    range of 0 to 31, inclusive; (b) contains an octet with a value of
    127; or (c) contains any series of octets greater than or equal to
    128 which do not form a valid UTF-8 sequence, as specified by
    [UNICODE]."

    

    Then, your statements become far simpler; for example: "The Value
    field for the optional element being logged MUST be Base64 encoded
    if it is unprintable." And: "For an optionally logged message body
    (Tag=01) the BEB is set according to whether the message body is
    unprintable." And: "When optionally logging an entire SIP message
    (Tag=02) the BEB is set according to whether the message body
    portion is unprintable."

    

    You get the idea.

    

    /a

    

    

    P.S. The above definition would require adding a reference for
    [UNICODE], like so:

    

    
    
   [UNICODE] The Unicode Consortium. The Unicode Standard,
             Version 6.2.0, (Mountain View, CA: The Unicode
             Consortium, 2012. ISBN 978-1-936213-07-8)
             <http://www.unicode.org/versions/Unicode6.2.0/>



  


--------------080205060305070604080501--

From gsalguei@cisco.com  Tue Oct 23 14:56:27 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16E5211E8105 for ; Tue, 23 Oct 2012 14:56:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.475
X-Spam-Level: 
X-Spam-Status: No, score=-10.475 tagged_above=-999 required=5 tests=[AWL=0.124, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tNM2g8bmkLM9 for ; Tue, 23 Oct 2012 14:56:26 -0700 (PDT)
Received: from av-tac-rtp.cisco.com (av-tac-rtp.cisco.com [64.102.19.209]) by ietfa.amsl.com (Postfix) with ESMTP id ACE4911E80E0 for ; Tue, 23 Oct 2012 14:56:25 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from chook.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-rtp.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9NLuOL8023021 for ; Tue, 23 Oct 2012 17:56:24 -0400 (EDT)
Received: from dhcp-10-150-54-114.cisco.com (dhcp-10-150-54-114.cisco.com [10.150.54.114]) by chook.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9NLuIxQ009650; Tue, 23 Oct 2012 17:56:21 -0400 (EDT)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Gonzalo Salgueiro 
In-Reply-To: <50870CB8.40908@nostrum.com>
Date: Tue, 23 Oct 2012 17:56:18 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com>
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com>
To: Adam Roach 
X-Mailer: Apple Mail (2.1283)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 23 Oct 2012 21:56:27 -0000

Thanks Adam. The approach seems reasonable.  I'll make the suggested =
change. =20

Just a few comments inline...




> I think the text is a bit confused on what UTF-8 means. The phrase =
"any characters outside the UTF-8 character set" appears many times, and =
I'm afraid it doesn't really make any sense.
>=20
> What you probably want to do is define a term somewhere in the =
document to formally specify "Those Things That Must Be Base-64 =
Encoded". I would recommend something like:
>=20
> "For the purposes of this document, we define 'unprintable' to mean a =
string of octets that: (a) contains an octet with a value in the range =
of 0 to 31, inclusive; (b) contains an octet with a value of 127; or (c) =
contains any series of octets greater than or equal to 128 which do not =
form a valid UTF-8 sequence, as specified by [UNICODE]."

I'll have to expand the definition a bit since there are octets greater =
than 128 that are non-printable but still valid UTF-8 control =
characters.  I can go with something like:

"For the purposes of this document, we define 'unprintable' to mean a =
string of octets that: (a) contains an octet with a value in the range =
of 0 to 31, inclusive; (b) contains an octet with a value of 127, (c) =
contains any octet greater than or equal to 128 which is a formatting or =
control character (such as 128 to 159) within the UTF-8 character set; =
or (d) falls outside the UTF-8 character range, as specified by =
[UNICODE]."

Does that sound ok?

> Then, your statements become far simpler; for example: "The Value =
field for the optional element being logged MUST be Base64 encoded if it =
is unprintable." And: "For an optionally logged message body (Tag=3D01) =
the BEB is set according to whether the message body is unprintable." =
And: "When optionally logging an entire SIP message (Tag=3D02) the BEB =
is set according to whether the message body portion is unprintable."

I'll make the appropriate replacements once we agree on the definition.
>=20
> You get the idea.

I do. Thanks.

Cheers,

Gonzalo

>=20
> /a
>=20
>=20
> P.S. The above definition would require adding a reference for =
[UNICODE], like so:
>=20
>    [UNICODE] The Unicode Consortium. The Unicode Standard,
>              Version 6.2.0, (Mountain View, CA: The Unicode
>              Consortium, 2012. ISBN 978-1-936213-07-8)
>             =20
> 
>=20
>=20
>=20


From adam@nostrum.com  Tue Oct 23 15:27:02 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E65651F0CA6 for ; Tue, 23 Oct 2012 15:27:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.329
X-Spam-Level: 
X-Spam-Status: No, score=-102.329 tagged_above=-999 required=5 tests=[AWL=0.271, BAYES_00=-2.599, SPF_PASS=-0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6DBUuTmlN7bA for ; Tue, 23 Oct 2012 15:26:59 -0700 (PDT)
Received: from shaman.nostrum.com (nostrum-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:267::2]) by ietfa.amsl.com (Postfix) with ESMTP id A3A4D1F0C8E for ; Tue, 23 Oct 2012 15:26:58 -0700 (PDT)
Received: from hydra-en0.roach.at (99-152-144-32.lightspeed.dllstx.sbcglobal.net [99.152.144.32]) (authenticated bits=0) by shaman.nostrum.com (8.14.3/8.14.3) with ESMTP id q9NMQv3a053272 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 23 Oct 2012 17:26:57 -0500 (CDT) (envelope-from adam@nostrum.com)
Message-ID: <508719B1.4090108@nostrum.com>
Date: Tue, 23 Oct 2012 17:26:57 -0500
From: Adam Roach 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:14.0) Gecko/20120713 Thunderbird/14.0
MIME-Version: 1.0
To: Gonzalo Salgueiro 
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com> <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com>
In-Reply-To: <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: pass (nostrum.com: 99.152.144.32 is authenticated by a trusted mechanism)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 23 Oct 2012 22:27:02 -0000

On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:
> "For the purposes of this document, we define 'unprintable' to mean a string of octets that: (a) contains an octet with a value in the range of 0 to 31, inclusive; (b) contains an octet with a value of 127, (c) contains any octet greater than or equal to 128 which is a formatting or control character (such as 128 to 159) within the UTF-8 character set; or (d) falls outside the UTF-8 character range, as specified by [UNICODE]."
>
> Does that sound ok?

I think we're still talking past each other here.

"Outside the UTF-8 character range" simply isn't a sensible thing to 
say. What we're talking about putting into a log record is a series of 
*octets*, not a series of *characters*. UTF-8 is an encoding that 
defines how octets are put together to make characters.

Once you start talking about the octets as if they *are* characters, 
you're conflating two very different things. So, for example, you can't 
talk about "a string of octets that... falls outside the UTF-8 character 
range."

You can talk about a string of bytes that does not form a valid UTF-8 
sequence, and that's almost certainly what you want to say here.

I'm also getting a bit lost in what you mean when you say "which is a 
formatting or control character (such as 128 to 159)." Keep in mind that 
we're still talking about *octets* here, not characters. In UTF-8, 
there's nothing special about an octet with a value of 128. There's 
nothing special about an octet with a value of 159. Both can appear as 
the second octet in a two-octet character. Or the second or third octet 
in a three-octet character. And so on. The same goes for everything 
between 128 and 191.

Now, octet values of 192, 193, and 245-255 won't appear in valid UTF-8. 
If we wanted to be abundantly careful, we could call those out as being 
invalid. But I think we catch those just fine if we talk about octets 
that form valid UTF-8 sequences.

Or are you meaning to call out UTF-8 code points like U+0080 (the 
Latin-1 padding character)? Because that has nothing to do with an 
*octet* with a value of 128. It would be encoded as a two-octet sequence 
starting with 194. However, if we're intending to go down the rabbit 
hole of making decisions about whether to Base-64 encode based on which 
UTF-8 codepoints we want to consider "printable," then we've got years 
of draft refinement ahead of us (I can already imagine the right-to-left 
mark arguments). That way lies madness.

All of which is a very long winded way to say: octets are not characters 
and characters are not octets; and you need to write the text in a way 
that does not mix them with each other.

/a

From gsalguei@cisco.com  Tue Oct 23 23:30:53 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 112AC21F8C3B for ; Tue, 23 Oct 2012 23:30:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level: 
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CZFX+rt7mFOw for ; Tue, 23 Oct 2012 23:30:52 -0700 (PDT)
Received: from av-tac-sj.cisco.com (av-tac-sj.cisco.com [171.68.227.119]) by ietfa.amsl.com (Postfix) with ESMTP id 4169621F8BBC for ; Tue, 23 Oct 2012 23:30:52 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from chook.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-sj.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9O6Upxv026541 for ; Tue, 23 Oct 2012 23:30:51 -0700 (PDT)
Received: from rtp-gsalguei-8914.cisco.com (rtp-gsalguei-8914.cisco.com [10.116.132.53]) by chook.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9O6Uha0020800; Wed, 24 Oct 2012 02:30:44 -0400 (EDT)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Gonzalo Salgueiro 
In-Reply-To: <508719B1.4090108@nostrum.com>
Date: Wed, 24 Oct 2012 02:30:42 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com>
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com> <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com> <508719B1.4090108@nostrum.com>
To: Adam Roach 
X-Mailer: Apple Mail (2.1283)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 06:30:53 -0000

Adam -

Let me first state that I'm certainly no expert in UTF-8 and will =
happily defer to you on this. I certainly get your point about octets =
versus characters, especially since UTF-8 allows for multi-byte =
characters.  When I made the statement "outside the UTF-8 character =
range" I was referring to the discrete set of characters from U+0000 to =
U+7FFFFFFF  (or U+10FFFF if using the restricted RFC 3629 definition). I =
assumed this was a clearly bounded set.

To be clear, my only intent was to expand your original definition of =
'unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) to =
also include the well known C1 control codes (U+0080 to U+009F). To =
avoid confusion, I should have used the Unicode code point notation.  I =
certainly don't want to try and go down the road of specifying what =
characters are 'printable', but I thought C1 control codes were worth =
stating explicitly in the same way you did C0 control codes. Your =
original definition seemed unnecessarily restrictive since the C1 =
control codes, for example, are not printable but are certainly a valid =
UTF-8 sequence of octets greater than or equal to 128. Does that make =
sense?

If you think your original definition of 'unprintable' is broad enough, =
then I will use that.

Cheers,

Gonzalo

On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:

> On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:
>> "For the purposes of this document, we define 'unprintable' to mean a =
string of octets that: (a) contains an octet with a value in the range =
of 0 to 31, inclusive; (b) contains an octet with a value of 127, (c) =
contains any octet greater than or equal to 128 which is a formatting or =
control character (such as 128 to 159) within the UTF-8 character set; =
or (d) falls outside the UTF-8 character range, as specified by =
[UNICODE]."
>>=20
>> Does that sound ok?
>=20
> I think we're still talking past each other here.
>=20
> "Outside the UTF-8 character range" simply isn't a sensible thing to =
say. What we're talking about putting into a log record is a series of =
*octets*, not a series of *characters*. UTF-8 is an encoding that =
defines how octets are put together to make characters.
>=20
> Once you start talking about the octets as if they *are* characters, =
you're conflating two very different things. So, for example, you can't =
talk about "a string of octets that... falls outside the UTF-8 character =
range."
>=20
> You can talk about a string of bytes that does not form a valid UTF-8 =
sequence, and that's almost certainly what you want to say here.
>=20
> I'm also getting a bit lost in what you mean when you say "which is a =
formatting or control character (such as 128 to 159)." Keep in mind that =
we're still talking about *octets* here, not characters. In UTF-8, =
there's nothing special about an octet with a value of 128. There's =
nothing special about an octet with a value of 159. Both can appear as =
the second octet in a two-octet character. Or the second or third octet =
in a three-octet character. And so on. The same goes for everything =
between 128 and 191.
>=20
> Now, octet values of 192, 193, and 245-255 won't appear in valid =
UTF-8. If we wanted to be abundantly careful, we could call those out as =
being invalid. But I think we catch those just fine if we talk about =
octets that form valid UTF-8 sequences.
>=20
> Or are you meaning to call out UTF-8 code points like U+0080 (the =
Latin-1 padding character)? Because that has nothing to do with an =
*octet* with a value of 128. It would be encoded as a two-octet sequence =
starting with 194. However, if we're intending to go down the rabbit =
hole of making decisions about whether to Base-64 encode based on which =
UTF-8 codepoints we want to consider "printable," then we've got years =
of draft refinement ahead of us (I can already imagine the right-to-left =
mark arguments). That way lies madness.
>=20
> All of which is a very long winded way to say: octets are not =
characters and characters are not octets; and you need to write the text =
in a way that does not mix them with each other.
>=20
> /a
>=20


From musgravepj@gmail.com  Wed Oct 24 10:01:12 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8762721F85DA for ; Wed, 24 Oct 2012 10:01:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.598
X-Spam-Level: 
X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TzDWISMg23I5 for ; Wed, 24 Oct 2012 10:01:11 -0700 (PDT)
Received: from mail-vb0-f44.google.com (mail-vb0-f44.google.com [209.85.212.44]) by ietfa.amsl.com (Postfix) with ESMTP id 4F3DD21F8498 for ; Wed, 24 Oct 2012 10:01:11 -0700 (PDT)
Received: by mail-vb0-f44.google.com with SMTP id fc26so858551vbb.31 for ; Wed, 24 Oct 2012 10:01:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0zr+OIEkOW1qbuLXrjzDjW1oHKu5IUM2d6JbAmOdlJU=; b=FD5Smr8esuHvX1Xw/HIXAgqll/PPi59e3Mk58i9s7HbCoQMCsS6lKHIMdeIOj00hZP O61gYSJA84temO34DwjJ2/T067C+NLCTByiN+W2Dx9D3YSu1wwS4MhnK96QG6XuEpegk 6hsD+gFw8/K+mRLaCS5nP8j4zRTXyzM0pFR84tkoOahgDsWgbOnfirlGpAixwF8+hZFp Tnu8wgcA4xoI0hiyfeklhYcmx1oUQf8xbL/TD/HNVqwzWTm/51vgTH8XvL2q/j8CTy0G nNPj2oLW2K7KffzI47fAxZ8iwUk9AAwasn1/j6N/PxSE5OVIhc9CbsjDEKYI6FGgd/AT 3N0Q==
MIME-Version: 1.0
Received: by 10.220.205.200 with SMTP id fr8mr8996511vcb.34.1351098070698; Wed, 24 Oct 2012 10:01:10 -0700 (PDT)
Received: by 10.58.216.72 with HTTP; Wed, 24 Oct 2012 10:01:10 -0700 (PDT)
In-Reply-To: <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com>
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com> <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com> <508719B1.4090108@nostrum.com> <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com>
Date: Wed, 24 Oct 2012 13:01:10 -0400
Message-ID: 
From: Peter Musgrave 
To: Gonzalo Salgueiro 
Content-Type: multipart/alternative; boundary=14dae9ccd4b68f8b8904ccd108d7
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 17:01:12 -0000

--14dae9ccd4b68f8b8904ccd108d7
Content-Type: text/plain; charset=ISO-8859-1

Sorry to be so absent.

Great that we're gonna get this puppy done.

I will read through front to back this weekend.

Can I ask others to look at the diffs and comment by Monday Oct 29th?

Then it seems we'll need a minor fix up to -08 and we're good to write this
up again.

If anyone wants more time, please let me know.

Regards,

Peter Musgrave (as WG chair)


On Wed, Oct 24, 2012 at 2:30 AM, Gonzalo Salgueiro wrote:

> Adam -
>
> Let me first state that I'm certainly no expert in UTF-8 and will happily
> defer to you on this. I certainly get your point about octets versus
> characters, especially since UTF-8 allows for multi-byte characters.  When
> I made the statement "outside the UTF-8 character range" I was referring to
> the discrete set of characters from U+0000 to U+7FFFFFFF  (or U+10FFFF if
> using the restricted RFC 3629 definition). I assumed this was a clearly
> bounded set.
>
> To be clear, my only intent was to expand your original definition of
> 'unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) to
> also include the well known C1 control codes (U+0080 to U+009F). To avoid
> confusion, I should have used the Unicode code point notation.  I certainly
> don't want to try and go down the road of specifying what characters are
> 'printable', but I thought C1 control codes were worth stating explicitly
> in the same way you did C0 control codes. Your original definition seemed
> unnecessarily restrictive since the C1 control codes, for example, are not
> printable but are certainly a valid UTF-8 sequence of octets greater than
> or equal to 128. Does that make sense?
>
> If you think your original definition of 'unprintable' is broad enough,
> then I will use that.
>
> Cheers,
>
> Gonzalo
>
> On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:
>
> > On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:
> >> "For the purposes of this document, we define 'unprintable' to mean a
> string of octets that: (a) contains an octet with a value in the range of 0
> to 31, inclusive; (b) contains an octet with a value of 127, (c) contains
> any octet greater than or equal to 128 which is a formatting or control
> character (such as 128 to 159) within the UTF-8 character set; or (d) falls
> outside the UTF-8 character range, as specified by [UNICODE]."
> >>
> >> Does that sound ok?
> >
> > I think we're still talking past each other here.
> >
> > "Outside the UTF-8 character range" simply isn't a sensible thing to
> say. What we're talking about putting into a log record is a series of
> *octets*, not a series of *characters*. UTF-8 is an encoding that defines
> how octets are put together to make characters.
> >
> > Once you start talking about the octets as if they *are* characters,
> you're conflating two very different things. So, for example, you can't
> talk about "a string of octets that... falls outside the UTF-8 character
> range."
> >
> > You can talk about a string of bytes that does not form a valid UTF-8
> sequence, and that's almost certainly what you want to say here.
> >
> > I'm also getting a bit lost in what you mean when you say "which is a
> formatting or control character (such as 128 to 159)." Keep in mind that
> we're still talking about *octets* here, not characters. In UTF-8, there's
> nothing special about an octet with a value of 128. There's nothing special
> about an octet with a value of 159. Both can appear as the second octet in
> a two-octet character. Or the second or third octet in a three-octet
> character. And so on. The same goes for everything between 128 and 191.
> >
> > Now, octet values of 192, 193, and 245-255 won't appear in valid UTF-8.
> If we wanted to be abundantly careful, we could call those out as being
> invalid. But I think we catch those just fine if we talk about octets that
> form valid UTF-8 sequences.
> >
> > Or are you meaning to call out UTF-8 code points like U+0080 (the
> Latin-1 padding character)? Because that has nothing to do with an *octet*
> with a value of 128. It would be encoded as a two-octet sequence starting
> with 194. However, if we're intending to go down the rabbit hole of making
> decisions about whether to Base-64 encode based on which UTF-8 codepoints
> we want to consider "printable," then we've got years of draft refinement
> ahead of us (I can already imagine the right-to-left mark arguments). That
> way lies madness.
> >
> > All of which is a very long winded way to say: octets are not characters
> and characters are not octets; and you need to write the text in a way that
> does not mix them with each other.
> >
> > /a
> >
>
> _______________________________________________
> sip-clf mailing list
> sip-clf@ietf.org
> https://www.ietf.org/mailman/listinfo/sip-clf
>

--14dae9ccd4b68f8b8904ccd108d7
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Sorry to be so absent.=A0

Great that we're gonna get=
 this puppy done.=A0

I will read through front to =
back this weekend.=A0

Can I ask others to look at =
the diffs and comment by Monday Oct 29th?



Then it seems we'll need a minor fix up to -08 and =
we're good to write this up again.=A0

If anyon=
e wants more time, please let me know.=A0

Regards,=
=A0



Peter Musgrave (as WG chair)


On Wed, Oct 24, 2012 at 2:30 AM, Gonzalo Salgueiro <gs=
alguei@cisco.com> wrote:


Adam -



Let me first state that I'm certainly no expert in UTF-8 and will happi=
ly defer to you on this. I certainly get your point about octets versus cha=
racters, especially since UTF-8 allows for multi-byte characters. =A0When I=
 made the statement "outside the UTF-8 character range" I was ref=
erring to the discrete set of characters from U+0000 to U+7FFFFFFF =A0(or U=
+10FFFF if using the restricted RFC 3629 definition). I assumed this was a =
clearly bounded set.




To be clear, my only intent was to expand your original definition of '=
unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) to =
also include the well known C1 control codes (U+0080 to U+009F). To avoid c=
onfusion, I should have used the Unicode code point notation. =A0I certainl=
y don't want to try and go down the road of specifying what characters =
are 'printable', but I thought C1 control codes were worth stating =
explicitly in the same way you did C0 control codes. Your original definiti=
on seemed unnecessarily restrictive since the C1 control codes, for example=
, are not printable but are certainly a valid UTF-8 sequence of octets grea=
ter than or equal to 128. Does that make sense?




If you think your original definition of 'unprintable' is broad eno=
ugh, then I will use that.



Cheers,



Gonzalo




On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:



> On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:

>> "For the purposes of this document, we define 'unprintabl=
e' to mean a string of octets that: (a) contains an octet with a value =
in the range of 0 to 31, inclusive; (b) contains an octet with a value of 1=
27, (c) contains any octet greater than or equal to 128 which is a formatti=
ng or control character (such as 128 to 159) within the UTF-8 character set=
; or (d) falls outside the UTF-8 character range, as specified by [UNICODE]=
."


>>

>> Does that sound ok?

>

> I think we're still talking past each other here.

>

> "Outside the UTF-8 character range" simply isn't a sensi=
ble thing to say. What we're talking about putting into a log record is=
 a series of *octets*, not a series of *characters*. UTF-8 is an encoding t=
hat defines how octets are put together to make characters.


>

> Once you start talking about the octets as if they *are* characters, y=
ou're conflating two very different things. So, for example, you can=
9;t talk about "a string of octets that... falls outside the UTF-8 cha=
racter range."


>

> You can talk about a string of bytes that does not form a valid UTF-8 =
sequence, and that's almost certainly what you want to say here.

>

> I'm also getting a bit lost in what you mean when you say "wh=
ich is a formatting or control character (such as 128 to 159)." Keep i=
n mind that we're still talking about *octets* here, not characters. In=
 UTF-8, there's nothing special about an octet with a value of 128. The=
re's nothing special about an octet with a value of 159. Both can appea=
r as the second octet in a two-octet character. Or the second or third octe=
t in a three-octet character. And so on. The same goes for everything betwe=
en 128 and 191.


>

> Now, octet values of 192, 193, and 245-255 won't appear in valid U=
TF-8. If we wanted to be abundantly careful, we could call those out as bei=
ng invalid. But I think we catch those just fine if we talk about octets th=
at form valid UTF-8 sequences.


>

> Or are you meaning to call out UTF-8 code points like U+0080 (the Lati=
n-1 padding character)? Because that has nothing to do with an *octet* with=
 a value of 128. It would be encoded as a two-octet sequence starting with =
194. However, if we're intending to go down the rabbit hole of making d=
ecisions about whether to Base-64 encode based on which UTF-8 codepoints we=
 want to consider "printable," then we've got years of draft =
refinement ahead of us (I can already imagine the right-to-left mark argume=
nts). That way lies madness.


>

> All of which is a very long winded way to say: octets are not characte=
rs and characters are not octets; and you need to write the text in a way t=
hat does not mix them with each other.

>

> /a

>



_______________________________________________

sip-clf mailing list

sip-clf@ietf.org

https://www.ietf.org/mailman/listinfo/sip-clf





--14dae9ccd4b68f8b8904ccd108d7--

From adam@nostrum.com  Wed Oct 24 12:04:14 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2068021F8540 for ; Wed, 24 Oct 2012 12:04:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.437
X-Spam-Level: 
X-Spam-Status: No, score=-102.437 tagged_above=-999 required=5 tests=[AWL=0.162, BAYES_00=-2.599, HTML_MESSAGE=0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4VurZMqgNhuc for ; Wed, 24 Oct 2012 12:04:13 -0700 (PDT)
Received: from shaman.nostrum.com (nostrum-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:267::2]) by ietfa.amsl.com (Postfix) with ESMTP id AB5A621F8551 for ; Wed, 24 Oct 2012 12:04:12 -0700 (PDT)
Received: from Svantevit.local ([4.30.77.1]) (authenticated bits=0) by shaman.nostrum.com (8.14.3/8.14.3) with ESMTP id q9OJ4B1Z089313 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 24 Oct 2012 14:04:11 -0500 (CDT) (envelope-from adam@nostrum.com)
Message-ID: <50883BAB.9070501@nostrum.com>
Date: Wed, 24 Oct 2012 14:04:11 -0500
From: Adam Roach 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:13.0) Gecko/20120614 Thunderbird/13.0.1
MIME-Version: 1.0
To: Gonzalo Salgueiro 
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com> <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com> <508719B1.4090108@nostrum.com> <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com>
In-Reply-To: <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com>
Content-Type: multipart/alternative; boundary="------------000500000403070408000202"
Received-SPF: pass (nostrum.com: 4.30.77.1 is authenticated by a trusted mechanism)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 19:04:14 -0000

This is a multi-part message in MIME format.
--------------000500000403070408000202
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

So, I think I understand your point. (And, to be clear, I don't intend 
to hold myself out as a Unicode expert either). The rationale behind 
what I'm proposing has two main assumptions:

  * The whole point of choosing a non-binary record format is allowing
    the use of commandline tools to work with records. Legacy
    commandline tools will frequently make assumptions about what
    octetes that correspond to ASCII control characters mean (e.g. 10
    and 13), and take certain actions based on them. By and large, I
    wouldn't expect the same kind of reaction to octetes that represent
    extended control characters, like those in the C1 set. I would
    expect normal commandline tools to simply treat octets in the range
    of 128-255 as (relatively) non-special, and process them normally.
    Cursory experimentation with the commandline tools I have easily
    available leads me to believe that such is the case.

  * I presume that we do not want to force Base-64 encoding due to the
    simple presence of an accented Latin character. Similarly, we
    probably don't want a small number of characters from the
    CJK/Cyrillic/Arabic/etc codepages to trigger Base-64 either.

If I'm wrong about either of these assumptions, then we need to re-visit 
what I'm proposing.

/a


On 10/24/12 1:30 AM, Gonzalo Salgueiro wrote:
> Adam -
>
> Let me first state that I'm certainly no expert in UTF-8 and will happily defer to you on this. I certainly get your point about octets versus characters, especially since UTF-8 allows for multi-byte characters.  When I made the statement "outside the UTF-8 character range" I was referring to the discrete set of characters from U+0000 to U+7FFFFFFF  (or U+10FFFF if using the restricted RFC 3629 definition). I assumed this was a clearly bounded set.
>
> To be clear, my only intent was to expand your original definition of 'unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) to also include the well known C1 control codes (U+0080 to U+009F). To avoid confusion, I should have used the Unicode code point notation.  I certainly don't want to try and go down the road of specifying what characters are 'printable', but I thought C1 control codes were worth stating explicitly in the same way you did C0 control codes. Your original definition seemed unnecessarily restrictive since the C1 control codes, for example, are not printable but are certainly a valid UTF-8 sequence of octets greater than or equal to 128. Does that make sense?
>
> If you think your original definition of 'unprintable' is broad enough, then I will use that.
>
> Cheers,
>
> Gonzalo
>
> On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:
>
>> On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:
>>> "For the purposes of this document, we define 'unprintable' to mean a string of octets that: (a) contains an octet with a value in the range of 0 to 31, inclusive; (b) contains an octet with a value of 127, (c) contains any octet greater than or equal to 128 which is a formatting or control character (such as 128 to 159) within the UTF-8 character set; or (d) falls outside the UTF-8 character range, as specified by [UNICODE]."
>>>
>>> Does that sound ok?
>> I think we're still talking past each other here.
>>
>> "Outside the UTF-8 character range" simply isn't a sensible thing to say. What we're talking about putting into a log record is a series of *octets*, not a series of *characters*. UTF-8 is an encoding that defines how octets are put together to make characters.
>>
>> Once you start talking about the octets as if they *are* characters, you're conflating two very different things. So, for example, you can't talk about "a string of octets that... falls outside the UTF-8 character range."
>>
>> You can talk about a string of bytes that does not form a valid UTF-8 sequence, and that's almost certainly what you want to say here.
>>
>> I'm also getting a bit lost in what you mean when you say "which is a formatting or control character (such as 128 to 159)." Keep in mind that we're still talking about *octets* here, not characters. In UTF-8, there's nothing special about an octet with a value of 128. There's nothing special about an octet with a value of 159. Both can appear as the second octet in a two-octet character. Or the second or third octet in a three-octet character. And so on. The same goes for everything between 128 and 191.
>>
>> Now, octet values of 192, 193, and 245-255 won't appear in valid UTF-8. If we wanted to be abundantly careful, we could call those out as being invalid. But I think we catch those just fine if we talk about octets that form valid UTF-8 sequences.
>>
>> Or are you meaning to call out UTF-8 code points like U+0080 (the Latin-1 padding character)? Because that has nothing to do with an *octet* with a value of 128. It would be encoded as a two-octet sequence starting with 194. However, if we're intending to go down the rabbit hole of making decisions about whether to Base-64 encode based on which UTF-8 codepoints we want to consider "printable," then we've got years of draft refinement ahead of us (I can already imagine the right-to-left mark arguments). That way lies madness.
>>
>> All of which is a very long winded way to say: octets are not characters and characters are not octets; and you need to write the text in a way that does not mix them with each other.
>>
>> /a
>>



--------------000500000403070408000202
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    
So, I think I understand your point.
      (And, to be clear, I don't intend to hold myself out as a Unicode
      expert either). The rationale behind what I'm proposing has two
      main assumptions:

      
    
        
  • The whole point of choosing a non-binary record format is
          allowing the use of commandline tools to work with records.
          Legacy commandline tools will frequently make assumptions
          about what octetes that correspond to ASCII control characters
          mean (e.g. 10 and 13), and take certain actions based on them.
          By and large, I wouldn't expect the same kind of reaction to
          octetes that represent extended control characters, like those
          in the C1 set. I would expect normal commandline tools to
          simply treat octets in the range of 128-255 as (relatively)
          non-special, and process them normally. Cursory
          experimentation with the commandline tools I have easily
          available leads me to believe that such is the case.
    
          
    
        
    • 
        
  • I presume that we do not want to force Base-64 encoding due
          to the simple presence of an accented Latin character.
          Similarly, we probably don't want a small number of characters
          from the CJK/Cyrillic/Arabic/etc codepages to trigger Base-64
          either. 
    
        
    • 
      

      If I'm wrong about either of these assumptions, then we need to
      re-visit what I'm proposing.

      

      /a

      

      

      On 10/24/12 1:30 AM, Gonzalo Salgueiro wrote:

    

    

      
Adam -

Let me first state that I'm certainly no expert in UTF-8 and will happily defer to you on this. I certainly get your point about octets versus characters, especially since UTF-8 allows for multi-byte characters.  When I made the statement "outside the UTF-8 character range" I was referring to the discrete set of characters from U+0000 to U+7FFFFFFF  (or U+10FFFF if using the restricted RFC 3629 definition). I assumed this was a clearly bounded set.

To be clear, my only intent was to expand your original definition of 'unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) to also include the well known C1 control codes (U+0080 to U+009F). To avoid confusion, I should have used the Unicode code point notation.  I certainly don't want to try and go down the road of specifying what characters are 'printable', but I thought C1 control codes were worth stating explicitly in the same way you did C0 control codes. Your original definition seemed unnecessarily restrictive since the C1 control codes, for example, are not printable but are certainly a valid UTF-8 sequence of octets greater than or equal to 128. Does that make sense?

If you think your original definition of 'unprintable' is broad enough, then I will use that.

Cheers,

Gonzalo

On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:



      

        
On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:


        

          
"For the purposes of this document, we define 'unprintable' to mean a string of octets that: (a) contains an octet with a value in the range of 0 to 31, inclusive; (b) contains an octet with a value of 127, (c) contains any octet greater than or equal to 128 which is a formatting or control character (such as 128 to 159) within the UTF-8 character set; or (d) falls outside the UTF-8 character range, as specified by [UNICODE]."

Does that sound ok?


        

        
I think we're still talking past each other here.

"Outside the UTF-8 character range" simply isn't a sensible thing to say. What we're talking about putting into a log record is a series of *octets*, not a series of *characters*. UTF-8 is an encoding that defines how octets are put together to make characters.

Once you start talking about the octets as if they *are* characters, you're conflating two very different things. So, for example, you can't talk about "a string of octets that... falls outside the UTF-8 character range."

You can talk about a string of bytes that does not form a valid UTF-8 sequence, and that's almost certainly what you want to say here.

I'm also getting a bit lost in what you mean when you say "which is a formatting or control character (such as 128 to 159)." Keep in mind that we're still talking about *octets* here, not characters. In UTF-8, there's nothing special about an octet with a value of 128. There's nothing special about an octet with a value of 159. Both can appear as the second octet in a two-octet character. Or the second or third octet in a three-octet character. And so on. The same goes for everything between 128 and 191.

Now, octet values of 192, 193, and 245-255 won't appear in valid UTF-8. If we wanted to be abundantly careful, we could call those out as being invalid. But I think we catch those just fine if we talk about octets that form valid UTF-8 sequences.

Or are you meaning to call out UTF-8 code points like U+0080 (the Latin-1 padding character)? Because that has nothing to do with an *octet* with a value of 128. It would be encoded as a two-octet sequence starting with 194. However, if we're intending to go down the rabbit hole of making decisions about whether to Base-64 encode based on which UTF-8 codepoints we want to consider "printable," then we've got years of draft refinement ahead of us (I can already imagine the right-to-left mark arguments). That way lies madness.

All of which is a very long winded way to say: octets are not characters and characters are not octets; and you need to write the text in a way that does not mix them with each other.

/a



      

      

    

    

    

  


--------------000500000403070408000202--

From gsalguei@cisco.com  Wed Oct 24 16:41:21 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 86B9F1F0C5F for ; Wed, 24 Oct 2012 16:41:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.492
X-Spam-Level: 
X-Spam-Status: No, score=-10.492 tagged_above=-999 required=5 tests=[AWL=0.107, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C6evBRHmihHQ for ; Wed, 24 Oct 2012 16:41:20 -0700 (PDT)
Received: from av-tac-rtp.cisco.com (av-tac-rtp.cisco.com [64.102.19.209]) by ietfa.amsl.com (Postfix) with ESMTP id 1DFCE1F0C4C for ; Wed, 24 Oct 2012 16:41:20 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from chook.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-rtp.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9ONfIYx022232 for ; Wed, 24 Oct 2012 19:41:19 -0400 (EDT)
Received: from dhcp-10-150-54-114.cisco.com (dhcp-10-150-54-114.cisco.com [10.150.54.114]) by chook.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9ONfInG009731; Wed, 24 Oct 2012 19:41:18 -0400 (EDT)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=windows-1252
From: Gonzalo Salgueiro 
In-Reply-To: <50883BAB.9070501@nostrum.com>
Date: Wed, 24 Oct 2012 19:41:17 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <5885B8A1-2BB7-4A0D-879A-4739D41EB356@cisco.com>
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com> <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com> <508719B1.4090108@nostrum.com> <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com> <50883BAB.9070501@nostrum.com>
To: Adam Roach 
X-Mailer: Apple Mail (2.1283)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 23:41:21 -0000

Thanks, Adam.  I agree with both your assumptions. I think where we =
diverged is whether C1 was on an equal footing with C0 as far as their =
'unprintability'.  If C1 is handled nicely by text processing tools then =
we can omit them. We all have a vested interest in minimizing the amount =
we Base64 encode these optionally logged bodies and whole messages =
within SIP CLF since it drastically reduces its usefulness and can =
introduce information loss. Thus, I absolutely agree that we don't want =
an accented Latin character or other valid UTF-8 non-control characters =
(CJK/Cyrillic/Arabic/etc) to trigger a Base64 encoding.  At this point I =
think I will go with your original text for defining 'unprintable' =
within the context of this document.

Cheers,

Gonzalo


On Oct 24, 2012, at 3:04 PM, Adam Roach wrote:

> So, I think I understand your point. (And, to be clear, I don't intend =
to hold myself out as a Unicode expert either). The rationale behind =
what I'm proposing has two main assumptions:
> 	=95 The whole point of choosing a non-binary record format is =
allowing the use of commandline tools to work with records. Legacy =
commandline tools will frequently make assumptions about what octetes =
that correspond to ASCII control characters mean (e.g. 10 and 13), and =
take certain actions based on them. By and large, I wouldn't expect the =
same kind of reaction to octetes that represent extended control =
characters, like those in the C1 set. I would expect normal commandline =
tools to simply treat octets in the range of 128-255 as (relatively) =
non-special, and process them normally. Cursory experimentation with the =
commandline tools I have easily available leads me to believe that such =
is the case.
>=20
> 	=95 I presume that we do not want to force Base-64 encoding due =
to the simple presence of an accented Latin character. Similarly, we =
probably don't want a small number of characters from the =
CJK/Cyrillic/Arabic/etc codepages to trigger Base-64 either.=20
> If I'm wrong about either of these assumptions, then we need to =
re-visit what I'm proposing.
>=20
> /a
>=20
>=20
> On 10/24/12 1:30 AM, Gonzalo Salgueiro wrote:
>> Adam -
>>=20
>> Let me first state that I'm certainly no expert in UTF-8 and will =
happily defer to you on this. I certainly get your point about octets =
versus characters, especially since UTF-8 allows for multi-byte =
characters.  When I made the statement "outside the UTF-8 character =
range" I was referring to the discrete set of characters from U+0000 to =
U+7FFFFFFF  (or U+10FFFF if using the restricted RFC 3629 definition). I =
assumed this was a clearly bounded set.
>>=20
>> To be clear, my only intent was to expand your original definition of =
'unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) to =
also include the well known C1 control codes (U+0080 to U+009F). To =
avoid confusion, I should have used the Unicode code point notation.  I =
certainly don't want to try and go down the road of specifying what =
characters are 'printable', but I thought C1 control codes were worth =
stating explicitly in the same way you did C0 control codes. Your =
original definition seemed unnecessarily restrictive since the C1 =
control codes, for example, are not printable but are certainly a valid =
UTF-8 sequence of octets greater than or equal to 128. Does that make =
sense?
>>=20
>> If you think your original definition of 'unprintable' is broad =
enough, then I will use that.
>>=20
>> Cheers,
>>=20
>> Gonzalo
>>=20
>> On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:
>>=20
>>=20
>>> On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:
>>>=20
>>>> "For the purposes of this document, we define 'unprintable' to mean =
a string of octets that: (a) contains an octet with a value in the range =
of 0 to 31, inclusive; (b) contains an octet with a value of 127, (c) =
contains any octet greater than or equal to 128 which is a formatting or =
control character (such as 128 to 159) within the UTF-8 character set; =
or (d) falls outside the UTF-8 character range, as specified by =
[UNICODE]."
>>>>=20
>>>> Does that sound ok?
>>>>=20
>>> I think we're still talking past each other here.
>>>=20
>>> "Outside the UTF-8 character range" simply isn't a sensible thing to =
say. What we're talking about putting into a log record is a series of =
*octets*, not a series of *characters*. UTF-8 is an encoding that =
defines how octets are put together to make characters.
>>>=20
>>> Once you start talking about the octets as if they *are* characters, =
you're conflating two very different things. So, for example, you can't =
talk about "a string of octets that... falls outside the UTF-8 character =
range."
>>>=20
>>> You can talk about a string of bytes that does not form a valid =
UTF-8 sequence, and that's almost certainly what you want to say here.
>>>=20
>>> I'm also getting a bit lost in what you mean when you say "which is =
a formatting or control character (such as 128 to 159)." Keep in mind =
that we're still talking about *octets* here, not characters. In UTF-8, =
there's nothing special about an octet with a value of 128. There's =
nothing special about an octet with a value of 159. Both can appear as =
the second octet in a two-octet character. Or the second or third octet =
in a three-octet character. And so on. The same goes for everything =
between 128 and 191.
>>>=20
>>> Now, octet values of 192, 193, and 245-255 won't appear in valid =
UTF-8. If we wanted to be abundantly careful, we could call those out as =
being invalid. But I think we catch those just fine if we talk about =
octets that form valid UTF-8 sequences.
>>>=20
>>> Or are you meaning to call out UTF-8 code points like U+0080 (the =
Latin-1 padding character)? Because that has nothing to do with an =
*octet* with a value of 128. It would be encoded as a two-octet sequence =
starting with 194. However, if we're intending to go down the rabbit =
hole of making decisions about whether to Base-64 encode based on which =
UTF-8 codepoints we want to consider "printable," then we've got years =
of draft refinement ahead of us (I can already imagine the right-to-left =
mark arguments). That way lies madness.
>>>=20
>>> All of which is a very long winded way to say: octets are not =
characters and characters are not octets; and you need to write the text =
in a way that does not mix them with each other.
>>>=20
>>> /a
>>>=20
>>>=20
>=20
>=20


From musgravepj@gmail.com  Sun Oct 28 15:29:36 2012
Return-Path: 
X-Original-To: sip-clf@ietfa.amsl.com
Delivered-To: sip-clf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4BF3B21F8648 for ; Sun, 28 Oct 2012 15:29:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.999
X-Spam-Level: 
X-Spam-Status: No, score=-0.999 tagged_above=-999 required=5 tests=[BAYES_50=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zvnf3lmi36Cf for ; Sun, 28 Oct 2012 15:29:35 -0700 (PDT)
Received: from mail-ie0-f172.google.com (mail-ie0-f172.google.com [209.85.223.172]) by ietfa.amsl.com (Postfix) with ESMTP id 60A8221F8640 for ; Sun, 28 Oct 2012 15:29:32 -0700 (PDT)
Received: by mail-ie0-f172.google.com with SMTP id 9so6619814iec.31 for ; Sun, 28 Oct 2012 15:29:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=LMCSuSDeWRWf5bmtyTgTx6eLDgweLUhPlk3H33TETKo=; b=nYQywlDNAfm/WwIj423iLD53RFl3457HRdm6Y1EueRHBNIOwqJRe4I0OqbrwehuUA7 ZImNhUV5FW6fpowDhYaeeGsFYXFOKfbYSkya441MY+Juj2r80pzwBBuAjNs5wb5/e0qR kuqYWJevXw5TOjdBtmUG1QO3jsAADpLdPjilqI6ZqWrLqUepI7QPz7XbUdAEYvjixw2d q2g06K2PyCj9VZjr7vqGh9d7M3YzMiOiAkUUANNQHvBnL2EDTVmhdwaz21/KO3Yi4vk0 vNaUykFF3BcZmxN0pJm7rKrbF6T6uhJpjUUb7G+K2Za3CCFRALG6rEHqP4tti9uUxDkV oCZA==
Received: by 10.42.210.12 with SMTP id gi12mr24527014icb.22.1351463371929; Sun, 28 Oct 2012 15:29:31 -0700 (PDT)
Received: from [192.168.1.108] ([204.237.33.112]) by mx.google.com with ESMTPS id u4sm4634013igw.6.2012.10.28.15.29.30 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 28 Oct 2012 15:29:30 -0700 (PDT)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\))
From: Peter Musgrave 
In-Reply-To: <5885B8A1-2BB7-4A0D-879A-4739D41EB356@cisco.com>
Date: Sun, 28 Oct 2012 18:29:28 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <677D295A-BF44-490A-86C1-48C0472E6F58@gmail.com>
References: <20121005015620.22856.1399.idtracker@ietfa.amsl.com> <869FCF91-1032-4411-A7D5-85CEE6F120E5@cisco.com> <50870CB8.40908@nostrum.com> <5A63A1D1-5D2A-4EA8-9E7A-CDA3C9668DE5@cisco.com> <508719B1.4090108@nostrum.com> <2771A52D-9AE5-460F-A896-888AB334153C@cisco.com> <50883BAB.9070501@nostrum.com> <5885B8A1-2BB7-4A0D-879A-4739D41EB356@cisco.com>
To: Gonzalo Salgueiro 
X-Mailer: Apple Mail (2.1498)
Cc: "sip-clf@ietf.org Mailing" 
Subject: Re: [sip-clf] I-D Action: draft-ietf-sipclf-format-07.txt
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIP Common Log File format discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 28 Oct 2012 22:29:36 -0000


Ok - I know less about UTF-8 than either Adam or Gonzalo, but the =
rationale of being able to use text base tools is one I support.=20

I read through the diffs and did not find anything else I wanted to =
bring up.=20



Once we agree on "printable" is there anything else people want to see =
altered?


Thanks,=20

Peter

One nit:=20
4.3 s/macine/machine/


On 2012-10-24, at 7:41 PM, Gonzalo Salgueiro  wrote:

> Thanks, Adam.  I agree with both your assumptions. I think where we =
diverged is whether C1 was on an equal footing with C0 as far as their =
'unprintability'.  If C1 is handled nicely by text processing tools then =
we can omit them. We all have a vested interest in minimizing the amount =
we Base64 encode these optionally logged bodies and whole messages =
within SIP CLF since it drastically reduces its usefulness and can =
introduce information loss. Thus, I absolutely agree that we don't want =
an accented Latin character or other valid UTF-8 non-control characters =
(CJK/Cyrillic/Arabic/etc) to trigger a Base64 encoding.  At this point I =
think I will go with your original text for defining 'unprintable' =
within the context of this document.
>=20
> Cheers,
>=20
> Gonzalo
>=20
>=20
> On Oct 24, 2012, at 3:04 PM, Adam Roach wrote:
>=20
>> So, I think I understand your point. (And, to be clear, I don't =
intend to hold myself out as a Unicode expert either). The rationale =
behind what I'm proposing has two main assumptions:
>> 	=95 The whole point of choosing a non-binary record format is =
allowing the use of commandline tools to work with records. Legacy =
commandline tools will frequently make assumptions about what octetes =
that correspond to ASCII control characters mean (e.g. 10 and 13), and =
take certain actions based on them. By and large, I wouldn't expect the =
same kind of reaction to octetes that represent extended control =
characters, like those in the C1 set. I would expect normal commandline =
tools to simply treat octets in the range of 128-255 as (relatively) =
non-special, and process them normally. Cursory experimentation with the =
commandline tools I have easily available leads me to believe that such =
is the case.
>>=20
>> 	=95 I presume that we do not want to force Base-64 encoding due =
to the simple presence of an accented Latin character. Similarly, we =
probably don't want a small number of characters from the =
CJK/Cyrillic/Arabic/etc codepages to trigger Base-64 either.=20
>> If I'm wrong about either of these assumptions, then we need to =
re-visit what I'm proposing.
>>=20
>> /a
>>=20
>>=20
>> On 10/24/12 1:30 AM, Gonzalo Salgueiro wrote:
>>> Adam -
>>>=20
>>> Let me first state that I'm certainly no expert in UTF-8 and will =
happily defer to you on this. I certainly get your point about octets =
versus characters, especially since UTF-8 allows for multi-byte =
characters.  When I made the statement "outside the UTF-8 character =
range" I was referring to the discrete set of characters from U+0000 to =
U+7FFFFFFF  (or U+10FFFF if using the restricted RFC 3629 definition). I =
assumed this was a clearly bounded set.
>>>=20
>>> To be clear, my only intent was to expand your original definition =
of 'unprintable' beyond the C0 control codes (U+0000 to U+001F, U+007F) =
to also include the well known C1 control codes (U+0080 to U+009F). To =
avoid confusion, I should have used the Unicode code point notation.  I =
certainly don't want to try and go down the road of specifying what =
characters are 'printable', but I thought C1 control codes were worth =
stating explicitly in the same way you did C0 control codes. Your =
original definition seemed unnecessarily restrictive since the C1 =
control codes, for example, are not printable but are certainly a valid =
UTF-8 sequence of octets greater than or equal to 128. Does that make =
sense?
>>>=20
>>> If you think your original definition of 'unprintable' is broad =
enough, then I will use that.
>>>=20
>>> Cheers,
>>>=20
>>> Gonzalo
>>>=20
>>> On Oct 23, 2012, at 6:26 PM, Adam Roach wrote:
>>>=20
>>>=20
>>>> On 10/23/12 16:56, Oct 23, Gonzalo Salgueiro wrote:
>>>>=20
>>>>> "For the purposes of this document, we define 'unprintable' to =
mean a string of octets that: (a) contains an octet with a value in the =
range of 0 to 31, inclusive; (b) contains an octet with a value of 127, =
(c) contains any octet greater than or equal to 128 which is a =
formatting or control character (such as 128 to 159) within the UTF-8 =
character set; or (d) falls outside the UTF-8 character range, as =
specified by [UNICODE]."
>>>>>=20
>>>>> Does that sound ok?
>>>>>=20
>>>> I think we're still talking past each other here.
>>>>=20
>>>> "Outside the UTF-8 character range" simply isn't a sensible thing =
to say. What we're talking about putting into a log record is a series =
of *octets*, not a series of *characters*. UTF-8 is an encoding that =
defines how octets are put together to make characters.
>>>>=20
>>>> Once you start talking about the octets as if they *are* =
characters, you're conflating two very different things. So, for =
example, you can't talk about "a string of octets that... falls outside =
the UTF-8 character range."
>>>>=20
>>>> You can talk about a string of bytes that does not form a valid =
UTF-8 sequence, and that's almost certainly what you want to say here.
>>>>=20
>>>> I'm also getting a bit lost in what you mean when you say "which is =
a formatting or control character (such as 128 to 159)." Keep in mind =
that we're still talking about *octets* here, not characters. In UTF-8, =
there's nothing special about an octet with a value of 128. There's =
nothing special about an octet with a value of 159. Both can appear as =
the second octet in a two-octet character. Or the second or third octet =
in a three-octet character. And so on. The same goes for everything =
between 128 and 191.
>>>>=20
>>>> Now, octet values of 192, 193, and 245-255 won't appear in valid =
UTF-8. If we wanted to be abundantly careful, we could call those out as =
being invalid. But I think we catch those just fine if we talk about =
octets that form valid UTF-8 sequences.
>>>>=20
>>>> Or are you meaning to call out UTF-8 code points like U+0080 (the =
Latin-1 padding character)? Because that has nothing to do with an =
*octet* with a value of 128. It would be encoded as a two-octet sequence =
starting with 194. However, if we're intending to go down the rabbit =
hole of making decisions about whether to Base-64 encode based on which =
UTF-8 codepoints we want to consider "printable," then we've got years =
of draft refinement ahead of us (I can already imagine the right-to-left =
mark arguments). That way lies madness.
>>>>=20
>>>> All of which is a very long winded way to say: octets are not =
characters and characters are not octets; and you need to write the text =
in a way that does not mix them with each other.
>>>>=20
>>>> /a
>>>>=20
>>>>=20
>>=20
>>=20
>=20
> _______________________________________________
> sip-clf mailing list
> sip-clf@ietf.org
> https://www.ietf.org/mailman/listinfo/sip-clf