Subject: Re: [xml2rfc] #198: Misleading "premature end of data" indication

From: "xml2rfc issue tracker" <[email-masked]>
Date: Wed, 22 Jan 2014 10:04:19 -0000

#198: Misleading "premature end of data" indication

Comment (by [email-masked]):

 Sorry, Henrik's comment got lost in the noise.  I hacked up a relatively
 small file that illustrates the problem.  With the current version
 (apparently 2.4.5) and output set to "Use frames to show", I get the
 rather annoying "The file is not there -- check for error messages or you
 may have clicked reload" message with no [other] error messages.   With
 output set to file, I get no messages at all but an empty file,
 reminiscent of the issue just reported in #223.

-- 
--------------------------------+----------------------------------
  Reporter:  [email-masked]  |      Owner:  [email-masked]
      Type:  defect             |     Status:  new
  Priority:  medium             |  Milestone:
 Component:  Version 2 cli      |    Version:  2.4.x
Resolution:                     |   Keywords:
--------------------------------+----------------------------------

Ticket URL: <http://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/198#comment:3>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>

---

From: "xml2rfc issue tracker" <[email-masked]>
Date: Thu, 23 Jan 2014 15:59:44 -0000

Comment (by [email-masked]):

 Thanks for the attached file.

 The problem is with the indicated encoding in the <?xml?> PI, which was
 set to US-ASCII when the encoding was UTF-8 (with no BOM).  Changing the
 encoding to UTF-8 lets xml2rfc process the file.  After correcting some
 minor things (like a missing definition of &rfc2119;) xml2rfc 2.4.5
 processes the file correctly, producing a text file, with some appropriate
 warnings:
 {{{
 henrik@tannat $ xml2rfc show-bogus-eof.xml
 Parsing file show-bogus-eof.xml
 WARNING: Illegal character replaced in string: &#151;
 WARNING: Illegal character replaced in string: &#24858;
 WARNING: Illegal character replaced in string: &#34850;
 WARNING: Illegal character replaced in string: &#1075;
 WARNING: Illegal character replaced in string: &#1083;
 WARNING: Illegal character replaced in string: &#1091;
 WARNING: Illegal character replaced in string: &#1087;
 WARNING: Illegal character replaced in string: &#1099;
 WARNING: Illegal character replaced in string: &#1081;
 Created file show-bogus-eof.txt

 }}}

 The section which had unicode characters embedded is rendered with
 the appropriate xml entities written out, which I believe is as
 desired with the current specification:

 {{{
 2.  Introduction

    Hi, Jim.  The dashes on this line &#151; setting off this phrase
    &#151; are Unicode Em Dash, U+2014.

    And the problem, if it still exists, is rather &#24858;&#34850; or,
    if you prefer, &#1075;&#1083;&#1091;&#1087;&#1099;&#1081;.

    Given Heather's plans, both SHOULD probably generate warnings and
    either be rendered properly or create clear error messages.
 }}}

 The remaining issue is then to identify input files which are declared as
 xml in encoding "US-ASCII" or "ASCII" which contains non-ascii characters.

 At the moment, it unfortunately seems like this has to happen before the
 file is read by the xml parser, which means another pass over the file
 with
 a simple parser which can read the <?xml?> PI, extract the encoding, and
 flag violations, but that should be doable.

Ticket URL: <http://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/198#comment:4>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>