Subject: [xml2rfc] #392 (Version 2 cli): xml2rfc no longer works on Mac, apparently related to SVG processing

From: "xml2rfc issue tracker" <[email-masked]>
Date: Mon, 04 Feb 2019 20:41:18 -0000

#392: xml2rfc no longer works on Mac, apparently related to SVG processing

 Macs configured for standard weeks tend to have an ugly locale such as:

 {{{
  LANG="en_US@calendar=iso8601.UTF-8"
 }}}

 (Spare me the words about Apple hiding this necessary, should-be-default
 configuration in the locale.)

 xml2rfc now croaks with

 {{{
 Traceback (most recent call last):
   File "/Users/cabo/bin/xml2rfc", line 7, in <module>
     from xml2rfc.run import main
   File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/__init__.py", line
 14, in <module>
     from xml2rfc.parser import  XmlRfcError, CachingResolver,
 XmlRfcParser, XmlRfc
   File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/parser.py", line
 17, in <module>
     from xml2rfc.writers import base
   File "/Users/cabo/lib/python3.7/site-
 packages/xml2rfc/writers/__init__.py", line 14, in <module>
     from xml2rfc.writers.pdf import PdfWriter
   File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/writers/pdf.py",
 line 15, in <module>
     import weasyprint
   File "/Users/cabo/lib/python3.7/site-packages/weasyprint/__init__.py",
 line 393, in <module>
     from .css import preprocess_stylesheet  # noqa
   File "/Users/cabo/lib/python3.7/site-
 packages/weasyprint/css/__init__.py", line 25, in <module>
     from . import computed_values
   File "/Users/cabo/lib/python3.7/site-
 packages/weasyprint/css/computed_values.py", line 21, in <module>
     from .utils import (
   File "/Users/cabo/lib/python3.7/site-packages/weasyprint/css/utils.py",
 line 20, in <module>
     from ..images import LinearGradient, RadialGradient
   File "/Users/cabo/lib/python3.7/site-packages/weasyprint/images.py",
 line 17, in <module>
     import cairosvg.parser
   File "/Users/cabo/lib/python3.7/site-packages/cairosvg/__init__.py",
 line 41, in <module>
     from . import surface  # noqa
   File "/Users/cabo/lib/python3.7/site-packages/cairosvg/surface.py", line
 27, in <module>
     from .defs import (
   File "/Users/cabo/lib/python3.7/site-packages/cairosvg/defs.py", line
 24, in <module>
     from .bounding_box import calculate_bounding_box,
 is_non_empty_bounding_box
   File "/Users/cabo/lib/python3.7/site-packages/cairosvg/bounding_box.py",
 line 26, in <module>
     from .features import match_features
   File "/Users/cabo/lib/python3.7/site-packages/cairosvg/features.py",
 line 25, in <module>
     LOCALE = locale.getdefaultlocale()[0] or ''
   File
 "/Users/cabo/.homebrew/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/locale.py",
 line 568, in getdefaultlocale
     return _parse_localename(localename)
   File
 "/Users/cabo/.homebrew/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/locale.py",
 line 495, in _parse_localename
     raise ValueError('unknown locale: %s' % localename)
 ValueError: unknown locale: UTF-8

 *** xml2rfc failed, status 1 (possibly try with -r)
 }}}

-- 
---------------------------+----------------------------------
 Reporter:  [email-masked]   |      Owner:  [email-masked]
     Type:  defect         |     Status:  new
 Priority:  blocker        |  Milestone:
Component:  Version 2 cli  |    Version:  2.10.x
 Keywords:                 |
---------------------------+----------------------------------

Ticket URL: <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/392>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>

---

From: "xml2rfc issue tracker" <[email-masked]>
Date: Tue, 05 Feb 2019 18:18:00 -0000

Comment (by [email-masked]):

 Ugh.  This is down in the python 3.7 libs, so it's a problem not only for
 {{{xml2rfc}}}.

 Nevertheless, it would be best to find a decent workaround for
 {{{xml2rfc}}} specifically.

 I've been trying to reproduce this here, but haven't had success with just
 setting the {{{LANG}}} environment variable as you have it; could you
 provide the source file which triggers this, and the command line switches
 used, please?

-- 
----------------------------+----------------------------------
Resolution:                 |   Keywords:
----------------------------+----------------------------------

Ticket URL: <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/392#comment:1>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 5 Feb 2019 19:55:22 +0100

It doesn’t get to the source file; I can reproduce this with

LANG="en_US@calendar=iso8601.UTF-8” xml2rfc foo.bar

er, and with

LANG=“C” xml2rfc foo.bar

Hmm.  This is getting interesting.

Setting LC_ALL to a valid locale (including the above) fixes this.

Maybe I don’t understand where the locale comes from if LC_ALL=“”

More fun:

{{{
$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=
$ xml2rfc foo.bar
No such file: foo.bar
$ LANG=C xml2rfc foo.bar
Traceback (most recent call last):
  File "/Users/cabo/bin/xml2rfc", line 7, in <module>
    from xml2rfc.run import main
  File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/__init__.py", line 14, in <module>
    from xml2rfc.parser import  XmlRfcError, CachingResolver, XmlRfcParser, XmlRfc
  File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/parser.py", line 17, in <module>
    from xml2rfc.writers import base
  File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/writers/__init__.py", line 14, in <module>
    from xml2rfc.writers.pdf import PdfWriter
  File "/Users/cabo/lib/python3.7/site-packages/xml2rfc/writers/pdf.py", line 15, in <module>
    import weasyprint
  File "/Users/cabo/lib/python3.7/site-packages/weasyprint/__init__.py", line 393, in <module>
    from .css import preprocess_stylesheet  # noqa
  File "/Users/cabo/lib/python3.7/site-packages/weasyprint/css/__init__.py", line 25, in <module>
    from . import computed_values
  File "/Users/cabo/lib/python3.7/site-packages/weasyprint/css/computed_values.py", line 21, in <module>
    from .utils import (
  File "/Users/cabo/lib/python3.7/site-packages/weasyprint/css/utils.py", line 20, in <module>
    from ..images import LinearGradient, RadialGradient
  File "/Users/cabo/lib/python3.7/site-packages/weasyprint/images.py", line 17, in <module>
    import cairosvg.parser
  File "/Users/cabo/lib/python3.7/site-packages/cairosvg/__init__.py", line 41, in <module>
    from . import surface  # noqa
  File "/Users/cabo/lib/python3.7/site-packages/cairosvg/surface.py", line 27, in <module>
    from .defs import (
  File "/Users/cabo/lib/python3.7/site-packages/cairosvg/defs.py", line 24, in <module>
    from .bounding_box import calculate_bounding_box, is_non_empty_bounding_box
  File "/Users/cabo/lib/python3.7/site-packages/cairosvg/bounding_box.py", line 26, in <module>
    from .features import match_features
  File "/Users/cabo/lib/python3.7/site-packages/cairosvg/features.py", line 25, in <module>
    LOCALE = locale.getdefaultlocale()[0] or ''
  File "/Users/cabo/.homebrew/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/locale.py", line 568, in getdefaultlocale
    return _parse_localename(localename)
  File "/Users/cabo/.homebrew/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/locale.py", line 495, in _parse_localename
    raise ValueError('unknown locale: %s' % localename)
ValueError: unknown locale: UTF-8
}}}

Unfortunately, the logic in locale.py is pure spaghetti.
getdefaultlocale seems to pull “UTF-8" out of envvars that are explicitly being overridden.

Grüße, Carsten

> On Feb 4, 2019, at 21:41, xml2rfc issue tracker <[email-masked]> wrote:
>

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 5 Feb 2019 20:16:07 +0100

> LANG="en_US@calendar=iso8601.UTF-8” xml2rfc foo.bar

Ugh. Ok.

> Maybe I don’t understand where the locale comes from if LC_ALL=“”

Oh, dear.

I still cannot reproduce here, so there's some odd combination of unset
and set enviroment variables at play.  However, as a workaround I'll
add ValueError to the exception catch on line 17 of xml2rfc/writers/pdf.py,
in the next release.

Best regards,

	Henrik