Subject: [xml2rfc] url

From: Randy Bush <[email-masked]>
Date: Sun, 07 Jul 2019 06:38:40 -0700

xml2rfc 2.23.0

is this still a proper reference?

    <?rfc include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml-doi/reference.DOI.10.1145/2975159.xml?anchor=JUPITER"?>

randy, who got a python traceback and is suspicious of this as cause

---

From: Randy Bush <[email-masked]>
Date: Sun, 07 Jul 2019 10:29:14 -0700

xml2rfc 2.23.0

this diagnostic is a bit obscrure

$ xml2rfc draft-ietf-lsvr-l3dl.xml --html --text
Parsing file draft-ietf-lsvr-l3dl.xml
Traceback (most recent call last):
  File "/usr/local/bin/xml2rfc", line 10, in <module>
    sys.exit(main())
  File "/Library/Python/2.7/site-packages/xml2rfc/run.py", line 389, in main
    xmlrfc = parser.parse(remove_pis=options.remove_pis, normalize=True)
  File "/Library/Python/2.7/site-packages/xml2rfc/parser.py", line 618, in parse
    include=True, line_no=getattr(element, 'sourceline', 0))
  File "/Library/Python/2.7/site-packages/xml2rfc/parser.py", line 240, in getReferenceRequest
    result = self.cache(path)
  File "/Library/Python/2.7/site-packages/xml2rfc/parser.py", line 347, in cache
    hash = '-'+base64.urlsafe_b64encode(hashlib.sha1(query)) if query else ''
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py", line 101, in urlsafe_b64encode
    return b64encode(s, '-_')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py", line 53, in b64encode
    encoded = binascii.b2a_base64(s)[:-1]
TypeError: must be string or buffer, not _hashlib.HASH
make: *** [all] Error 1

source is at
https://git.rg.net/randy/draft-l3dl/src/master/draft-ietf-lsvr-l3dl.xml

randy

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 7 Jul 2019 20:57:09 +0200

>    hash = '-'+base64.urlsafe_b64encode(hashlib.sha1(query)) if query else ''

This must be a problem with the recent caching fixes in xml2rfc, it tries to base64-encode the query, but that isn’t a string…
I don’t have a problem with xml2rfc 2.22.2.

(The JUPITER bibxml generated from the DOI by xml2rfc.tools works.)

Grüße, Carsten

PS.: I still get the occasional 
requests.exceptions.ConnectionError: (‘Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))
Then I just retry…

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 7 Jul 2019 21:05:28 +0200

> This must be a problem with the recent caching fixes in xml2rfc, it tries to base64-encode the query, but that isn’t a string…

Bug report: I think this should be something like

>>   hash = ‘-‘+base64.urlsafe_b64encode(hashlib.sha1(query).digest()) if query else ‘’

Sorry for not tracking down where the trac is…

Grüße, Carsten

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 7 Jul 2019 21:32:33 +0200

Hi Randy,

Thanks for the traceback.  Will debug, fix, and let you know about
possible workarounds.

	Henrik

On 2019-07-07 19:29, Randy Bush wrote:
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 7 Jul 2019 21:33:31 +0200

>>>  hash = ‘-‘+base64.urlsafe_b64encode(hashlib.sha1(query).digest()) if query else ‘’

With 2.23.0, I actually get

    hash = '-'+base64.urlsafe_b64encode(hashlib.sha1(query)) if query else ''
TypeError: Unicode-objects must be encoded before hashing

(Python 3.7.3)

So this needs more than the change I was proposing (don’t laugh):

hash = '-' + base64.urlsafe_b64encode(hashlib.sha1(query.encode()).digest()).decode() if query else ''

(Python 2 compatibility left as an exercise to the reader.)

Grüße, Carsten

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 8 Jul 2019 00:43:06 +0200

> hash = '-' + base64.urlsafe_b64encode(hashlib.sha1(query.encode()).digest()).decode() if query else ''

That's what I ended up with, too.  And it's fine for both 2.7 and 3.x

I've released an update that should fix this, 2.23.1.

	Henrik