Subject: [xml2rfc] github?

From: Leif Johansson <[email-masked]>
Date: Tue, 08 Oct 2013 15:54:24 +0200

Could we could host the xml2rfc v2 code on github so that contributions
could be organized as pull requests?

            Cheers Leif

---

From: Nico Williams <[email-masked]>
Date: Tue, 8 Oct 2013 10:48:21 -0500

> could be organized as pull requests?

I've asked for the IETF tools team to switch to git or provide git
gateways, but there's no interest.  I have used a great many VCSes over the
past two+ decades: none holds a candle to git, but evidently others
disagree, and will continue to for a while longer.  git is so superior that
soon enough the resistance will fade away.

That said, I'm not sure that the IETF should be relying on free services
from a commercial VCS services purveyor, not without some sort of contract:
we don't want to start depending on the github issue tracker, wiki, PRs,
... only to be cut-off someday.  But yeah, it'd be so nice.

Nico
--

---

From: "Leif Johansson" <[email-masked]>
Date: Tue, 8 Oct 2013 17:58:39 +0200

> 8 okt 2013 kl. 17:48 skrev "Nico Williams" <[email-masked]>:
> 
> I've asked for the IETF tools team to switch to git or provide git gateways, but there's no interest.  I have used a great many VCSes over the past two+ decades: none holds a candle to git, but evidently others disagree, and will continue to for a while longer.  git is so superior that soon enough the resistance will fade away.

+1

> 
> That said, I'm not sure that the IETF should be relying on free services from a commercial VCS services purveyor, not without some sort of contract: we don't want to start depending on the github issue tracker, wiki, PRs, ... only to be cut-off someday.  But yeah, it'd be so nice.

I don't thing xml2rfc-the-tool is annointed (yet) in any way...

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 08 Oct 2013 10:44:12 -0600

+1

/psa

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 08 Oct 2013 20:18:55 +0200

Hi Nico,

On 2013-10-08 17:48 Nico Williams said the following:
> gateways, but there's no interest.

I think that if you go back and look at what I said the previous time this
came up, you'll see that the impression you put forward above doesn't match
what was said.  Please read on below.

>  I have used a great many VCSes over the

There are a number of use-cases for VCSes, and some of them are better handled
by git than by others.  I've very recently run into a situation with a contractor
which has shown me that there are use-cases where git's distributed nature and
local commits is exactly what is *not* desired.

That said, I'd like to tell you that about 3 weeks ago, I *did* set up git
repositories for all WGs on the tools servers, paralleling the current SVN
repositories, and requiring the same kind of authenticated access to push
commits as the current SVN repositories requires to commit.  I did this as a
first step towards providing git support across all the groups which today
have SVN support on the tools.ietf.org servers.  The issue I had with
providing git support the last time this came up, as far as I can remember
the situation, was that I wasn't able to easily provide front-end support
-- issue tracker, repository browser, etc., as Trac didn't have it in released
form at that time, and didn't have another option researched and ready.

This time around, that support *is* available in Trac, so it's more a matter
of upgrading Trac than evaluating and setting up new front-end infrastructure.

However, I've spent about 40 hours during evenings and weekends, outside
my regular work hours during the last 2 weeks, getting a particularly recalcitrant
code merge for the datatracker in shape to be released, and finding and fixing
bugs in the new code, so I haven't had time to test and then hopefully transition
the configuration and deploy the new Trac release.

Can you bear with me for another few weeks to complete what I've already
started, without yelling too loudly that 'the tools team has no interest
in providing git support'?

	Henrik

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 8 Oct 2013 12:29:36 -0700

Just to be clear on a point that I needed a clue-by-four across the head to get: git != github. In specific, Leif asked for GitHub so he could do pull requests. Those are not (I believe; would love to be wrong) part of git, and they are a highly-sought-after feature of git.

Git without GitHub gets you a very valuable feature: you have the whole repo on your disconnected box. GitHub's pull request feature is another level of good, allowing disparate people to contribute snippet-level changes easily.

--Paul Hoffman

---

From: Leif Johansson <[email-masked]>
Date: Tue, 08 Oct 2013 21:40:27 +0200

> Just to be clear on a point that I needed a clue-by-four across the head to get: git != github. In specific, Leif asked for GitHub so he could do pull requests. Those are not (I believe; would love…
yep
> Git without GitHub gets you a very valuable feature: you have the whole repo on your disconnected box. GitHub's pull request feature is another level of good, allowing disparate people to contribut…
yep

one is good, both is better

and I wasn't asking for the whole of the IETF to be moved into github,
given away and shredded by the ghosts of a thousand hipsters in the cloud

I was just asking about xml2rfc the tool

            Cheers Leif

---

From: Nico Williams <[email-masked]>
Date: Tue, 8 Oct 2013 15:02:14 -0500

> local commits is exactly what is *not* desired.

I've only ever seen one use case for centralized VCS: proprietary
codebases.  And even then, if you want to preclude exfiltration then
centralizing your VCS is hardly even enough to get started, and by the time
you're done then git should be OK: because you simply don't allow
exfiltration even of one version.

> form at that time, and didn't have another option researched and ready.

This is great news!  Thanks!

> of upgrading Trac than evaluating and setting up new front-end

Awesome!

> in providing git support'?

Absolutely.  I used weasel words to avoid blaming you or anyone in
particular.  The impression i'd been left with last time this came up was
that there was no interest in migrating from a VCS that was working fine;
that might just be me misremembering.

Nico
--

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Tue, 08 Oct 2013 13:02:38 -0700

On 10/08/2013 12:29 PM, Paul Hoffman wrote:
> Just to be clear on a point that I needed a clue-by-four across the head to
> get: git != github. In specific, Leif asked for GitHub so he could do pull
> requests. Those are not (I believe; would love to be wrong) part of git,
> and they are a highly-sought-after feature of git.
> 
> Git without GitHub gets you a very valuable feature: you have the whole
> repo on your disconnected box. GitHub's pull request feature is another
> level of good, allowing disparate people to contribute snippet-level
> changes easily.
> 

Note that you can use git with a svn repository:

git svn clone http://svn.tools.ietf.org/svn/tools/xml2rfc/

Github does not manage correctly code rebase, which is, IMO, the whole point
of git (i.e. you want a repository containing a clean history, not a catalog
of your mistakes).  A better choice for the IETF would be Gerrit, with the
additional advantage that the storage would be under the control of the IETF.

- -- 
Marc Petit-Huguenin
Email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org
Profile: http://www.linkedin.com/in/petithug

---

From: Nico Williams <[email-masked]>
Date: Tue, 8 Oct 2013 15:07:44 -0500

On Tuesday, October 8, 2013, Marc Petit-Huguenin wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>

I use git with github.  I use got for rebase and cherry-pick operations.  I
use github as a public repo, as a backup, as an issues db, for code review
(but Gerrit is, indeed, better), as a social medium (PRs), ...  I would
expect users to submit rebased commits -- that is more than fair enough
IMO.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 08 Oct 2013 20:10:10 -0400

Paul Hoffman <[email-masked]> wrote:

    paul> Just to be clear on a point that I needed a clue-by-four across the
    paul> head to get: git != github. In specific, Leif asked for GitHub so he
    paul> could do pull requests. Those are not (I believe; would love to be wrong)
    paul> part of git, and they are a highly-sought-after feature of git.

github just puts a web interface and a bunch of friendly automation
(travis-cl is very nice...) on top of pull operations (which now become
"requests")

github is not the only solution: https://gitorious.org/ exists,
redmine/chiliproject, savannah and sf.net all have systems that interface on
top of git, but only gitorious comes close to the ease of forking and making
pull requests that github championed.

pull operations are an integral part of *git*.

Anyone with a web server that speaks http: can arrange to publish code using
git, although the git:// protocol is apparently more efficient.

tcpdump.org maintains it's own git repo at bpf.tcpdump.org, but that repo is
pushed to github nightly.  This gets us all of the benefits of github, but
they don't hold the keys to the castle.

--
Michael Richardson <[email-masked]>, Sandelman Software Works

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 09 Oct 2013 03:49:21 +0200

Hi Leif,

(I did see your patch earlier, but unfortunately was so involved in getting the
latest datatracker release out that I didn't manage to act on it at the time.
Sorry.  Regarding pull requests, below:)

> one is good, both is better

Eh?  I think I understand, but I don't _get it_.  Let me give you my background
so you can provide the information I could be missing:  My main interaction with
github as a developer till now has been as one of the two active contributors to
a project hosted on github (https://github.com/dotse/PacketQ/).  I work on some
code, do commits, push it to my own repository on github, issue a pull request
when I think stuff is ready, and when the people responsible for the primary
repository has the time, they do a pull and merge, and if needed conflict
resolution.

To me, a pull request is just a button that provides the equivalent function of
me manually putting together an email saying 'please pull from my repository at
<URL>, commit <ID>'.

In the case of github, I don't think it actually even creates an email, it just
creates an annotation on the repository you've sent the pull request to, and if
the owners of that don't visit it, the pull request can sit there for ages,
without being acted on.

If that shortcut is sufficiently valuable, there is nothing that prevents us from
having it as an add-on to the Trac front-end I expect to put in place, if it
isn't already there.

The pull action itself, as has already been well explained by Michael, is built
into git itself, so we don't have to do anything extra to get that.

Recapitulating:  If you want the 'pull request' functionality, I'll add the
needed buttons to the web front-end (if not already there), and I'll try to
go one better: I'll try to see to it there's an actual email going out as
a notification.

> I was just asking about xml2rfc the tool

Understood, I got that; now I'm trying to understand what it is I'm missing
here.

Best regards,

	Henrik

---

From: Nico Williams <[email-masked]>
Date: Wed, 9 Oct 2013 00:11:15 -0500

I dunno about Leif, but it's not the pull req button i care about, it's
git, just git; github is just an option anyone can use the moment you use
git.  All of git: commit hashes (history as a tree of Merkle hash chains),
branches, tags, distributed development (clone at will, disconnected
operations, push/pull/format-patch/mail) rebase powers, cherry-picking, the
_index_ (and the power of git add -p), ssh and http and filesystem
protocols, and so on.

The pull req thing is just sugar -nice yummy social networking sugar- but
just sugar.  Sure, PRs are a critical aspect of git's rise to king of the
hill status, but that's not the reason i love git.

The only other VCS that comes close is Fossil, but it lacks a powerful
rebase, and also the index, so for me it's not good enough.  SVN is
like torture.  git has become the standard for open source devlopment.  CVS
is almost dead, svn is dying, mercurial is an also-ran...  It's not like
git is an awful choice imposed by its popularity, it really is awesome,
with very few weak points (limited file rename functionality is my biggest
pet peeve with git; i hear NetBSD had trouble with the number of files they
have to stuff into git, but that's not a problem here, and anyways, Illumos
is in git and it's huge, ditto. linux, glibc, ...).

Nico
--

---

From: Randy Bush <[email-masked]>
Date: Wed, 09 Oct 2013 14:50:32 +0900

if we're gonna seriously repaint the bikeshed, and i do not advocate
doing so, then mercurial whomps git.  but then again, i like emacs.

randy

---

From: Nico Williams <[email-masked]>
Date: Wed, 9 Oct 2013 01:24:44 -0500

> doing so, then mercurial whomps git.  but then again, i like emacs.

Some of this is bikeshedding.  Some of this is social (the avalanche
effect happened).  Some of this is features, and no, Mercurial (which
I used for years because Solaris' [and later OpenSolaris'] core OS/Net
and several other consolidations used it) does not beat git when it
comes to features.  Nor architecture.  Is this the right thread to
discuss the non-bikeshed aspects of this?

Architecture:

 - disconnected repos & tree of Merkle hash trees (with symbolic names
in the form of branches and tags) wins hands down over anything else
for VCS.  Understanding this actually has deep impact on what you
might call bikeshed details (see below).

Features:

 - rebase (including rebase -i and --onto) (I know, there's two
extensions for this in Mercurial, one for rebase, and one for editing
history[*])

 - index (git add, particularly git add -p) -- if you don't know what
this is and you do find out you'll never go back

 - disconnected operations (e.g., diff against upstream, which
Mercurial required extensions to do well back when i had to use it,
and which did not work when disconnected from the upstream).  This is
actually architectural because we can't always be connected to
infrastructure; being unable to perform important *local* operations
while disconnected is a serious shortcoming!

Just those two should be enough.  But there's more!

Bikeshed stuff:

 - naming versions by commits since day 1 (as Mercurial does) is
utterly useless once you have tens of thousands of commits -- useless
and dangerous as it doesn't yield a canonical commit naming form.
Naming commits exclusively by their hashes is not a bikeshed detail,
actually, but a direct consequence of scaling to very large projects
with very large history + using Merkle hash chains.

 - relative commit naming, relative to current "HEAD" is incredibly
useful when reviewing recent history, rebasing, ....  This too is
really a UI issue, not aesthetics: relative-to-HEAD aliasing of
commits is much easier to use than having to constantly lookup commits
to refer to.

 - basic functionality not included in Mercurial (see above; see also
use of PAGER, ...);

 - command-line options stuff (git suffers a bit in this department);

 - documentation (I know git haters hate git for its docs, but
Mercurial's rarely help me as quickly as git's);

 - ...

None of this matters though.  For a while we all had to use CVS.  Then
we had to use Mercurial or SVN.  Now we have to use git.  It's what
you want if you want mind share, because of social network effects.
Or build something to replace git with, just like git replaced the
others.

[*] Some people practically scream that history should never be
edited.  git users agree!  but only w.r.t. history in *public*
proclaimed-stable repos.  Rebase is incredibly powerful and useful
when developing commits for contribution to an upstream.

Nico
--

---

From: Randy Bush <[email-masked]>
Date: Wed, 09 Oct 2013 15:37:54 +0900

jeezus!  is there a *real* problem we're trying to solve?

randy

---

From: Nico Williams <[email-masked]>
Date: Wed, 9 Oct 2013 01:45:29 -0500

> jeezus!  is there a *real* problem we're trying to solve?

Yes, actually: popularity and relevance (laugh all you want, but it
matters).  We [should] want contribution to IETF and related tools to
be easier.  Easier, today, means: use git.  It's easier because of all
the things that make git better and easier *and* because of github and
gitorious.  The latter add social networking effects that really make
this a no-brainer.  Just look at their growth, at their stats --
nothing else comes close; it's a great ecosystem that we are NOT a
part of.

---

From: Randy Bush <[email-masked]>
Date: Wed, 09 Oct 2013 15:47:58 +0900

> matters).

i get it.  xml2rfc need to be on twitter and facebook!

---

From: Nico Williams <[email-masked]>
Date: Wed, 9 Oct 2013 01:50:52 -0500

> i get it.  xml2rfc need to be on twitter and facebook!

Not quite, no: just on the sites that are twitter and fb for
*developers*.  A smaller audience, but still *much* larger than the
usual suspects who attend IETF meetings.

There's no need for xml2rfc to be on github or gitorious either.  Just
having these tools in any public git repo *instantly* enables the use
of github/gitorious too.

---

From: Leif Johansson <[email-masked]>
Date: Wed, 09 Oct 2013 09:21:14 +0200

> me manually putting together an email saying 'please pull from my repository at

Yep basically. In the same way that an email is just a fancy-schmanzy way
to send a letter.

---

From: Leif Johansson <[email-masked]>
Date: Wed, 09 Oct 2013 09:21:53 +0200

On 10/09/2013 07:50 AM, Randy Bush wrote:
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
and I use vi but please lets not paint that

---

From: Martin Sustrik <[email-masked]>
Date: Wed, 09 Oct 2013 09:36:45 +0200

> Yep basically. In the same way that an email is just a fancy-schmanzy 

Except that pull requests can change over time. You believe that pull 
request X points to commit Y, but an hour later you find out it points 
to commit Z. When pushing 'merge' button there's always a probability of 
pulling some commits that weren't reviewed. The design is kind of 
strange.

Martin

---

From: Leif Johansson <[email-masked]>
Date: Wed, 09 Oct 2013 09:43:38 +0200

On 10/09/2013 09:36 AM, Martin Sustrik wrote:
> to commit Z. When pushing 'merge' button there's always a probability
> of pulling some commits that weren't reviewed. The design is kind of
> strange.
>
thats why you use a CI to run your testcases before you commit :-)

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 09 Oct 2013 16:51:41 +0900

> codebases.

I have at least two others, but they don't apply to IETF repositories:

1) Huge agglomerations of data where it's difficult to divide things up 
into separate projects but nobody really wants to download the whole 
history (e.g. huge website).

2) Educational situations where the concept of version management is 
foreign to the students in the first place. Having "commit" mean "commit 
to THE repository" can make things simpler to learn and understand, as a 
first step.

Regards,   Martin.

---

From: Martin Sustrik <[email-masked]>
Date: Wed, 09 Oct 2013 09:57:41 +0200

> thats why you use a CI to run your testcases before you commit :-)

That only works if you are both the author of the patch and maintainer 
of the project. If the patch comes from 3rd party, there's no guarantee 
that it was properly tested. So you test the patched version. Then you 
press 'merge' button. And voila! there's a new untested commit merged 
into mainline that was added by the 3rd party while you were doing the 
testing. Race condition in the process :)

Martin

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 09 Oct 2013 10:12:35 +0200

> exfiltration even of one version.

The use case when working with external contractors is that using a central
repository for commits lets you follow the work as it comes along.  I've had
several cases where a contractor (not one we're currently working with) has
gone down a wrong path, and I've been able to catch this within a day by getting
emailed copies of the commit messages, and gone and looked at the committed code
when something unexpected has triggered me.  Working with local commits and much
larger and less frequent updates to a repository I can see takes me into a situation
where the size of the updates are large enough that inspection requires substantial
chunks of time instead of seconds, and thus just don't happen easily.

Best regards,

	Henrik

---

From: Michael Richardson <[email-masked]>
Date: Wed, 09 Oct 2013 10:26:41 -0400

> That only works if you are both the author of the patch and maintainer of the
    > project. If the patch comes from 3rd party, there's no guarantee that
    it was

That's why you have the CI tied to github:

See, for instance:
https://travis-ci.org/the-tcpdump-group/tcpdump/builds/11890369

referenced at:
https://github.com/the-tcpdump-group/tcpdump/pull/323

"See all is well"

and also, later on:

"We can’t automatically merge this pull request."

I'm not actually saying that xml2rfc should be on github.  It's enough that
it be maintained in git, because then it's really a social question about it
being on github, or any other service *as well*

---

From: Nico Williams <[email-masked]>
Date: Wed, 9 Oct 2013 08:22:04 -0500

> chunks of time instead of seconds, and thus just don't happen easily.

git does not preclude this: just ask the dev to push often to a repo you
can look at.  I often keep pre-rebase branches around just to be able to
show that work.

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 09 Oct 2013 09:03:41 -0700

On 10/09/2013 06:22 AM, Nico Williams wrote:
> 
> On Oct 9, 2013 5:12 AM, "Henrik Levkowetz" <[email-masked] 
> <mailto:[email-masked]>> wrote:
>> The use case when working with external contractors is that using a
>> central repository for commits lets you follow the work as it comes
>> along.  I've had several cases where a contractor (not one we're
>> currently working with) has gone down a wrong path, and I've been able to
>> catch this within a day by getting emailed copies of the commit messages,
>> and gone and looked at the committed code when something unexpected has
>> triggered me.  Working with local commits and much larger and less
>> frequent updates to a repository I can see takes me into a
> situation

My main concern with scm like svn (even when using git svn) is the "commit
until it builds" mentality, which makes the repository unsuitable for
bisection.  The git way, which is "send a patchset for review until it is
perfect" (sent by email [e.g lkml], pushed in Gerrit [e.g. Android] or in a
temporary repository), will make things better in the long term.  And with
Gerrit you can configure Jenkins to automatically verify a patchset, i.e. each
patchset pushed is immediately built, automatically tested and rejected if
that fails.  I think that this feature alone compensates for the increase in
size of code to review prior to merging.

There is currently a Gerrit sandbox for Wireshark, if people wants to try
Gerrit (unfortunately no Jenkins integration so far):

http://www.wireshark.org/lists/wireshark-dev/201309/msg00188.html

I wrote some quickstart instructions here:

http://www.wireshark.org/lists/wireshark-dev/201309/msg00191.html
http://www.wireshark.org/lists/wireshark-dev/201309/msg00197.html

- -- 
Marc Petit-Huguenin

---

From: Doug Royer <[email-masked]>
Date: Wed, 09 Oct 2013 10:56:10 -0600

Years ago, who would have thought that SIMTEL would have gone away?

Personally I do not care about GIT, SVN, CVS, SCCS, RCS, or whatever.

However the other point made (by Nico) was that the IETF needs to take 
responsibility to ensure the hosting and web services does not 
disappear. I think because other sites, fads, tools, and sponsors over 
time have gone away. I did not see any direct reply to that point. A 
quick web search for 'web interface for git' resulted in some hits.

The pull request is just another form of branch - correct?
GITHUB does the access control work, not GIT - correct?

As to GITHUB vs GIT vs the current system:

Is it that GITHUB is desired? Or GIT?

If GITHUB:

		(1) What does it have that is NEEDED?
		(2) What does it have that is DESIRED?

If  GIT, exactly what features of the current system are:
		(3) Broken
		(4) Undesirable

I have used many opens source and proprietary source code repository 
programs over the decades. Other than 'it would be nice if...', I 
personally do not see that moving to GIT will solve any problems. ALL of 
the systems were designed to allow multiple collaborators, and yes with 
different processes to control that access and different methods of 
merging and resolving conflicts and branching. So, exactly what can not 
be done? Or exactly what needs to be changed? And why?

Doug Royer - (K7DMR.us / DougRoyer.com)
[email-masked]
714-989-6135

---

From: Nico Williams <[email-masked]>
Date: Thu, 10 Oct 2013 06:34:40 -0500

> The pull request is just another form of branch - correct?

It's a social thing.

> GITHUB does the access control work, not GIT - correct?

Git can be used without github, with authorization that works just fine.

> Is it that GITHUB is desired? Or GIT?

Yes.

It is git that we want.  Because using git enables the use of github (and
others).  And it is also github (and gitorious that we want.  Also because
git is awesome.

>                 (1) What does it have that is NEEDED?

People.  Developers in particular.  Lots and lots if then.  Just look at
stats for sourceforge vs github.  (Sourceforge simply didn't foster
community; that's basically all github did.)

>                 (2) What does it have that is DESIRED?

Same answer.

> If  GIT, exactly what features of the current system are:

git has won; not using it repels a lot of desirable participants.  git has
a number of features that other VCSes don't; I've already listed some of
them.

programs over the decades. Other than 'it would be nice if...', I
personally do not see that moving to GIT will solve any problems. ALL of
the systems were designed to allow multiple collaborators, and yes with
different processes to control that access and different methods of merging
and resolving conflicts and branching. So, exactly what can not be done? Or
exactly what needs to be changed? And why?

To my knowledge no VCS has anything like git's "index".  That's enough in
my world to doom them all.  More importantly, when faced with a wildly
popular technology one could: ignore it, adopt it, or innovate so as to
replace it.  Sometimes a popular technology is just so good as to make
replacement of it unlikely/difficult (e.g., AC electric current), altering
innovation mostly only at the margins.  Not adopting such a technology is
risky.  git is such a technology.

Nico
--