Subject: [Tools-discuss] Why does xml2rfc download over HTTP?

From: Eric Rescorla <[email-masked]>
Date: Fri, 1 Feb 2019 14:07:51 -0800

Hi folks,

I recently went to build a draft and got a pile of errors about how the
references didn't parse. Digging into things a bit, I discovered that the
reference was the page from a captive portal on one of my flights. I'm not
sure how this happened, but looking at xml2rfc, it appears that it tries
HTTPS URIs and then HTTP URIs, which seems like it's going to have the
wrong outcome in the situation where the captive portal blocks HTTPS and
intercepts HTTP.

Is there a good reason to continue to allow HTTP URIs in the tool?

-Ekr

---

From: Richard Barnes <[email-masked]>
Date: Fri, 1 Feb 2019 17:40:54 -0500

In particular, this seems to be the patch necessary to resolve the issue:

Index: trunk/cli/xml2rfc/parser.py
===================================================================
--- trunk/cli/xml2rfc/parser.py (revision 2965)
+++ trunk/cli/xml2rfc/parser.py (working copy)
@@ -53,8 +53,6 @@
                  no_network=None, network_locs= [
                      'https://xml2rfc.ietf.org/public/rfc/',
                      'https://xml2rfc.tools.ietf.org/public/rfc/',
-                     'http://xml2rfc.ietf.org/public/rfc/',
-                     'http://xml2rfc.tools.ietf.org/public/rfc/',
                  ],
                  rfc_number=None, options=base.default_options):
         self.quiet = quiet if quiet != None else options.quiet

On Fri, Feb 1, 2019 at 5:08 PM Eric Rescorla <[email-masked]> wrote:

> ___________________________________________________________
> Tools-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-discuss
>
> Please report datatracker.ietf.org and mailarchive.ietf.org
> bugs at http://tools.ietf.org/tools/ietfdb
> or send email to [email-masked]
>
> Please report tools.ietf.org bugs at
> http://tools.ietf.org/tools/issues
> or send email to [email-masked]
>

---

From: Julian Reschke <[email-masked]>
Date: Sat, 2 Feb 2019 06:40:52 +0100

On 2019-02-01 23:07, Eric Rescorla wrote:
> references didn't parse. Digging into things a bit, I discovered that 
> the reference was the page from a captive portal on one of my flights. 
> I'm not sure how this happened, but looking at xml2rfc, it appears that 
> it tries HTTPS URIs and then HTTP URIs, which seems like it's going to 
> have the wrong outcome in the situation where the captive portal blocks 
> HTTPS and intercepts HTTP.
> 

The easiest fix is to always specify full HTTPs URIs for references or 
to inline them in the first place.

Best regards, Julian

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 2 Feb 2019 05:28:51 -0800

> to inline them in the first place.

That seems like it's kind of working in the wrong direction. People
shouldn't
need to provide the URLs for IETF citations at all, just like the way they
don't when using markdown.

-Ekr

---

From: Julian Reschke <[email-masked]>
Date: Sat, 2 Feb 2019 14:32:16 +0100

On 2019-02-02 14:28, Eric Rescorla wrote:
> ...
> ...

Well - sounds like you should do a suggestion for the V3 vocabulary 
then... :-)

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 2 Feb 2019 14:37:56 +0100

Right.  However, who ever fixes this should keep in mind that software installed on a laptop does not always have access to a good set of root certificates (the browser is likely to, but Python software may not have access to the browser set), so it would be useful to have an escape hatch of some kind (preferably through environment variables, so complex Makefiles don’t have to be touched).

Maybe adding a little bit of robustness to the download process (e.g., don’t cache it if it isn’t valid XML) would also help.

Grüße, Carsten

> On Feb 1, 2019, at 23:40, Richard Barnes <[email-masked]> wrote:
> 
> I recently went to build a draft and got a pile of errors about how the references didn't parse. Digging into things a bit, I discovered that the reference was the page from a captive portal on one of my flights. I'm not sure how this happened, but looking at xml2rfc, it appears that it tries HTTPS URIs and then HTTP URIs, which seems like it's going to have the wrong outcome in the situation where the captive portal blocks HTTPS and intercepts HTTP.
>

---

From: Michael Richardson <[email-masked]>
Date: Sat, 02 Feb 2019 12:32:22 -0500

Carsten Bormann <[email-masked]> wrote:
    > Right.  However, who ever fixes this should keep in mind that software
    > installed on a laptop does not always have access to a good set of root
    > certificates (the browser is likely to, but Python software may not
    > have access to the browser set), so it would be useful to have an
    > escape hatch of some kind (preferably through environment variables, so
    > complex Makefiles don’t have to be touched).

+1

    > Maybe adding a little bit of robustness to the download process (e.g.,
    > don’t cache it if it isn’t valid XML) would also help.

That would have also solved the problem, which I've also experienced.

-- 
]               Never tell me the odds!                 | ipv6 mesh networks [ 
]   Michael Richardson, Sandelman Software Works        | network architect  [ 
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 5 Feb 2019 20:17:56 +0100

Hi,

On 2019-02-02 14:37, Carsten Bormann wrote:
> installed on a laptop does not always have access to a good set of
> root certificates (the browser is likely to, but Python software may
> not have access to the browser set), so it would be useful to have an
> escape hatch of some kind (preferably through environment variables,
> so complex Makefiles don’t have to be touched).
> 
> Maybe adding a little bit of robustness to the download process
> (e.g., don’t cache it if it isn’t valid XML) would also help.

Yes, that makes sense.  Added in my sources, will be part of the next
release.

Thanks, and best regards,

	Henrik