Subject: Re: [Ietf-and-github] FYI: W3C and GitHub tooling

From: "Salz, Rich" <[email-masked]>
Date: Tue, 1 Oct 2019 13:27:44 +0000

I think I've mentioned this before:  I agreed to prepare a datatracker patch that generalizes the "associated url's" so that an WG or document could have an extensible table of URL's.  I want to get that coded up before the next sprint.  Maybe that will help this.

On 10/1/19, 1:17 AM, "Mark Nottingham" <[email-masked]> wrote:

    Thanks. Inspired by the `w3c.json` file, I've started noodling on an `ietf.json` format to collect data from disparate IETF-related repos. See:
      https://github.com/mnot/ietf-repo-data
    
    My initial purpose is to use this to allow people to self-configure the GitHub weekly reports I'm sending out for a few groups (e.g., HTTP, QUIC, TLS). After that, I might see if I can auto-tweet new Internet-Drafts published by the group, etc.
    
    Cheers,

    > On 1 Oct 2019, at 12:01 am, Christopher Wood <[email-masked]> wrote:
    > 
    > Hi Mark,
    > 
    > Thanks for writing this up! I find their tool suite a thing of beauty. Some of the tools -- especially first one -- seems like it would enable lots of interesting features, especially as something for the datatracker (or something else) to hook onto.
    > 
    > Best,
    > Chris
    > 
    > On Tue, Sep 17, 2019, at 1:51 AM, Mark Nottingham wrote:
    >> I spent some time at W3C TPAC (their yearly plenary event) talking to 
    >> folks about the tooling that they use on GitHub, to see if there was 
    >> anything we could leverage (ideas or code), or even collaborate on 
    >> (everything referenced is Open Source, and they're very amenable to 
    >> collaboration where it makes sense). This is a bit of a brain dump, but 
    >> there's lots of interesting potential here, I think; I'll explore what 
    >> we might do in a followup.
    >> 
    >> At a high level, their philosophy is that data should be public 
    >> wherever feasible, and maintained as close as possible to its source. 
    >> Since they use GitHub for all of their work, that means keeping state 
    >> in checked-in files wherever possible.
    >> 
    >> For example, they have a `w3c.json` file that WGs are required to check 
    >> into their repos; see documentation at:
    >> https://w3c.github.io/w3c.json.html
    >> 
    >> This, in turn, is used by several of their tools to track and manage 
    >> their process. The validate-repos tool:
    >> https://github.com/w3c/validate-repos
    >> spiders their repos regularly to extract metadata and assure that they 
    >> are conforming to best practices; see report at:
    >> https://w3c.github.io/validate-repos/report.html
    >> 
    >> They also have a repository manager:
    >> https://github.com/w3c/ash-nazg/
    >> ... that helps them create and interact with repos:
    >> https://labs.w3.org/repo-manager/repos
    >> 
    >> For example, it validates that a contributor has signed the IPR policy 
    >> for the appropriate WG, as seen in the 'checks' for this PR:
    >> https://github.com/w3c/webvtt/pull/470
    >> 
    >> See attached screenshot for a creation view.
    >> 
    >> They also have a beta of an exploration API up:
    >> https://github.com/w3c/gargantua
    >> For example:
    >> https://www.w3.org/PM/Groups/browse.html?gid=107714
    >> Note that this is pulling all of the data (some of it aggregated into 
    >> static files) and presenting it in the client; there's no back-end 
    >> database, per se, for much of it. 
    >> 
    >> Where there is a backend db, there are also APIs, documented at:
    >> https://api.w3.org/doc
    >> And
    >> https://github.com/w3c/w3c-api
    >> 
    >> They manage spec transitions (roughly, the IESG discussion/approval 
    >> process) using a dedicated Github repo:
    >> https://github.com/w3c/transitions
    >> ... (see the issues list), and are talking about developing some 
    >> GitHub-centric tooling to improve that.
    >> 
    >> Some groups also manage their review process (roughly analogous to our 
    >> directorates) using tools built on top of GitHub; e.g.,
    >> http://w3c.github.io/i18n-activity/reviews/
    >> http://w3c.github.io/i18n-activity/projects/
    >> 
    >> Repos are spidered for lots of document checks, e.g., for reference 
    >> quality and statistics:
    >> https://github.com/tidoust/reffy-service
    >> 
    >> Repos can also trigger e-mails for notifications of things like 
    >> document state transitions:
    >> https://github.com/plehegar/transition-notifier
    >> 
    >> For more general info, see:
    >> https://w3c.github.io
    >> ... and their labs page:
    >> https://www.w3.org/2015/labs/
    >> 
    >> I asked how concerned they were about taking such a dependency upon 
    >> GitHub; I'd characterise their position as accepting that risk 
    >> (re-tooling on something like GitLab would incur a significant delay) 
    >> in return for the capabilities, reach, scalability and reliability they 
    >> get from GitHub. They do take backups, etc. I didn't ask about whether 
    >> they're looking at GitHub Actions, but that seems like an interesting 
    >> area to explore.
    >> 
    >> Cheers,
    >> 
    >> 
    >> --
    >> Mark Nottingham https://www.mnot.net/
    >> 
    >> _______________________________________________
    >> Ietf-and-github mailing list
    >> [email-masked]
    >> https://www.ietf.org/mailman/listinfo/ietf-and-github
    >> 
    >> Attachments:
    >> * create-repository.png
    > 
    > _______________________________________________
    > Ietf-and-github mailing list
    > [email-masked]
    > https://www.ietf.org/mailman/listinfo/ietf-and-github
    
    --
    Mark Nottingham   https://www.mnot.net/
    
    _______________________________________________
    Ietf-and-github mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/ietf-and-github