git-subtree

Hey all,

Avery Pennarun has suckered me into^W^W^Wasked me to submit his git-subtree tool for inclusion into mainline. Apparently there was some discussion about this at GitTogether.

I have a patch ready. It takes git-subtree from the current GitHub master, fixes the tests to use the standard git test harness and updates the build to recognize git-subtree.

How does the git community want the patch presented? Right now it's one monolithic thing. I understand that isn't ideal but I don't think incorporating the entire GitHub master history is necessarily the best idea either.

So I'm looking for some guidance.

Thanks!

                            -Dave

Hi David,

David Greene wrote:

Please read and follow the guidelines listed in Documentation/SubmittingPatches. The TL;DR version is: break it up into logical reviewable commits based on the current `master` and use git format-patch/ git send-email to send those commits to this mailing list.

Cheers!

-- Ram

[1]: https://raw.github.com/git/git/master/Documentation/SubmittingPatches

Ramkumar Ramachandra <artagnon@gmail.com> writes:

I've read that document. The issue is that I didn't develop the code, Avery did. This is a completely new tool for git and I don't have the first idea of what "logical" chunks would look like. I assume, for example, that we'd want the first "chunk" to actually work and do something interesting. I can go spend a bunch of time to see if I can grok enough to create these chunks but I wanted to check first and make sure that would be absolutely necessary. It's a lot of time to learn a completely new codebase. I was hoping to submit something soon and then learn the codebase gradually during maintenance/further development.

How have completely new tools be introduced into the git mainline in the past?

Thanks!

                              -Dave

Hi again,

[+CC: Junio Hamano, our maintainer]

David A. Greene wrote:

Not an issue as long as you have Avery's signoff.

We certainly don't want badly reviewed code that nobody understands floating around in the codebase- so, I'd suggest sending out whatever you think is appropriate for the first round of reviews, and see how things shape up from there.

Yes. For an example of something I was involved with but didn't author, see vcs-svn/.

-- Ram

Ramkumar Ramachandra <artagnon@gmail.com> writes:

As in a signed-off-by log entry on the commit? I did a commit -s to add my own signed-off-by tag and added a "From:" line in accordance with the SubmittingPatches document:

  "If you are forwarding a patch from somebody else, optionally, at the
   beginning of the e-mail message just before the commit message starts,
   you can put a "From: " line to name that person."

I have not used signoffs before in my day-to-day git flow. How do I go about getting one from Avery and incorporating it into the history in an autheticated way? I'm assuming you don't want me to forge his sign-off. :)

Certainly, I'm not trying to avoid review, just trying to figure out the most efficient mechanics.

Fair enough. I think I will take Jeff's suggested route and see where that goes.

Ok, I'll look into that.  Thanks for the pointer.

                             -Dave

greened@obbligato.org (David A. Greene) writes:

Argh. Sorry about all the duplicate replies. I got delivery bounces on all of them. Must have been a stuck queue or something. So again, apologies for the noise.

                              -Dave

Ramkumar Ramachandra <artagnon@gmail.com> writes:

As in a signed-off-by log entry on the commit? I did a commit -s to add my own signed-off-by tag and added a "From:" line in accordance with the SubmittingPatches document:

  "If you are forwarding a patch from somebody else, optionally, at the
   beginning of the e-mail message just before the commit message starts,
   you can put a "From: " line to name that person."

I have not used signoffs before in my day-to-day git flow. How do I go about getting one from Avery and incorporating it into the history in an autheticated way? I'm assuming you don't want me to forge his sign-off. :)

Certainly, I'm not trying to avoid review, just trying to figure out the most efficient mechanics.

Fair enough. I think I will take Jeff's suggested route and see where that goes.

Ok, I'll look into that.  Thanks for the pointer.

                             -Dave

On Thu, Jan 05, 2012 at 09:03:38AM -0600, David A. Greene wrote:

I think this is also somewhat different in that git-subtree has a multi-year history in git that we may want to keep. So it is more analogous to something like gitweb or git-gui, which we have brought in (using subtree merges, no less).

The biggest decision is whether or not to import the existing history. If we do, then we have to decide whether it becomes a sub-component like gitweb (e.g., it gets pulled into a "subtree" directory, and we have make recurse into it), or whether it gets overlaid into the main directory (i.e., we clean and munge the subtree repo a bit, then just "git merge" the history in).

If we want to throw away the existing history, then I think you end up doing the same munging as the latter option above, and then just make a single patch out of it instead of a merge.

I don't use git-subtree, but just glancing over the repo, it looks like that munging is mostly:

  1. git-subtree.sh stays, and gets added to git.git's top-level Makefile

  2. the test.sh script gets adapted into t/tXXXX-subtree.sh

  3. git-subtree.txt goes into Documentation/

  4. The rest of the files are infrastructure that can go away, as they
     are a subset of what git.git already contains.

I'd favor keeping the history and doing the munge-overlay thing. Although part of me wants to join the histories in a subtree so that we can use "git subtree" to do it (which would just be cool), I think the resulting code layout doesn't make much sense unless git-subtree is going to be maintained separately.

-Peff

Jeff King <peff@peff.net> writes:

I agree there may be some value in preserving this history.

I agree. I will leave that decision to the more experienced git developers. I'm happy to work either way.

Right. That's the approach I've taken for now but it's easy to switch. There aren't that many changes.

Done.

Done.

Done.

Done.

I have a patch that does all of the above but it is one monolithic blob. Like I said, the changes aren't extensive so it's easy for me to change strategies.

Ok, that sounds fine to me. I'll do that in a private branch. What should I send as patches to the mailing list? I'm assuming we don't want [PATCH 235/12342], etc. sent to the list chronicling the entire history. :)

Heh.  I thought about that too.  :)

Yeah, I agree.

                                -Dave

greened@obbligato.org (David A. Greene) writes:

I actually did end up doing a subtree merge via git subtree. It was more convenient to put it in contrib/ like that as almost everthing there is in its own subdirectory.

I'm cleaning things up there to remove redundancy, rewrite tests (using earlier work), etc. What number should I use for git-subtree tests? Here are some logical candidates:

        5 - the pull and exporting commands
        6 - the revision tree commands (even e.g. merge-base)
        7 - the porcelainish commands concerning the working tree
        9 - the git tools

git-subtree can pull and export. It also affects revision trees (it merges, for example) and is a porcelainish command that affects the working tree. It is also a "git tool" of a sort.

I originally put them under t97XX but now that is taken, as is everything up to and including t99XX.

Anyone have a strong opinion?

Thanks!

                           -Dave

greened@obbligato.org (David A. Greene) writes:

Oops, I lied. I originally put them under t79XX and I've kept that for now. Sound good?

                             -Dave

Jeff King <peff@peff.net> writes:

I agree there may be some value in preserving this history.

I agree. I will leave that decision to the more experienced git developers. I'm happy to work either way.

Right. That's the approach I've taken for now but it's easy to switch. There aren't that many changes.

Done.

Done.

Done.

Done.

I have a patch that does all of the above but it is one monolithic blob. Like I said, the changes aren't extensive so it's easy for me to change strategies.

Ok, that sounds fine to me. I'll do that in a private branch. What should I send as patches to the mailing list? I'm assuming we don't want [PATCH 235/12342], etc. sent to the list chronicling the entire history. :)

Heh.  I thought about that too.  :)

Yeah, I agree.

                                -Dave

David Greene <dag@cray.com> writes:

It depends on the longer term vision of how the result of this submission will evolve and more importantly, where you fit in the piture.

One possible answer you could give us might go like this:

    The longer term vision is for "git subtree" to become, and be
    developed further as, an integral part of the core git suite.

    I have been an active contributor to the "git subtree" project for
    quite some time, and am very familiar with the code. Avery has been
    too busy to properly take care of the maintenance of "git subtree",
    and expected to be so for the foreseeable future. I will address any
    issue raised during the initial review and will be taking over its
    maintenance and further development.

    My plan is to put this first to contrib/ area, keep it there for a few
    release cycles while ironing out remaining kinks in the code, and
    eventually make it one of the "git" subcommands. Avery's external tree
    will cease to exist as future development will happen in-tree in the
    git repository.

Your answer might differ, of course, but the point is that we would need to weigh pros and cons between inclusion of it in the git repository and keeping it in Avery's repository and have him and his contributors maintain, enhance and distribute it from there, and it largely depends on the nature of the submission. Is it a "throw it over the wall" dump of a large code of unknown quality that we need to clean up first without knowing the vision of how "git subtree" should evolve by original author and/or people who have been actively developing it?

Junio C Hamano <gitster@pobox.com> writes:

This is a very fair question. I'll try to answer it as best I can. I think it mostly jibes with your suggested possible answer.

I've been using git-subtree for about six months now and as an enthusiastic user who wants to introduce this too into my daily corporate work environment, I'd like to see it incorporated as an officially-supported git tool to make that introduction easier.

So my intention is to make git-subtree an integral part of the core git suite and take on further maintenance and development along with Avery and the other git-subtree developers.

I have not previously been a contributor to git-subtree and don't know the code at all but I am a quick learner. The actual git-subtree code itself is not overwhelmingly large and strikes me as a tractable learning project.

I approached Avery about submitting git-subtree to become part of the core git suite. He responded positively but indicated he does not have the cycles to do it at this time. He asked whether I could take on the job and I agreed.

He mentioned that he'd talked to some developers at GitTogether and got a positive reponse there. I don't know whether you were part of those discussions. My impression is that the GitTogether discussions went well and there was general agreement that git-subtree would be a valuable addition to the core git suite.

I am perfectly happy to put this in contrib/ first if it eases the introduction. I would like to move it to the subcommand area after getting everything in tip-top shape. What I don't want is for it to languish forever in contrib/. That means I'll need some guideline of the changes/standards necessary to qualify it for transition from contrib/ to an official subcommand. I expect we'll develop that as we go along but I hope the git community has some institutional knowledge gathered from previous experience.

I have asked Avery how he wants to do maintenance going forward. I haven't heard back from him yet so I can't speak to whether the existing GitHub project will continue or not. I'll pass along his thoughts when I get them.

I certainly don't want this to be an "over the wall" operation. I intend to participate in maintenance of git-subtree in the official git repository.

So I'll go ahead and work on adding this to contrib/. Once I get a response from Avery about his long-term vision I'll pass that along and we can have further discussion. I may start sending patches to the mailing list for review before hearing back from him, however.

Sound good?

                             -Dave

Junio C Hamano <gitster@pobox.com> writes:

This is a very fair question. I'll try to answer it as best I can. I think it mostly jibes with your suggested possible answer.

I've been using git-subtree for about six months now and as an enthusiastic user who wants to introduce this too into my daily corporate work environment, I'd like to see it incorporated as an officially-supported git tool to make that introduction easier.

So my intention is to make git-subtree an integral part of the core git suite and take on further maintenance and development along with Avery and the other git-subtree developers.

I have not previously been a contributor to git-subtree and don't know the code at all but I am a quick learner. The actual git-subtree code itself is not overwhelmingly large and strikes me as a tractable learning project.

I approached Avery about submitting git-subtree to become part of the core git suite. He responded positively but indicated he does not have the cycles to do it at this time. He asked whether I could take on the job and I agreed.

He mentioned that he'd talked to some developers at GitTogether and got a positive reponse there. I don't know whether you were part of those discussions. My impression is that the GitTogether discussions went well and there was general agreement that git-subtree would be a valuable addition to the core git suite.

I am perfectly happy to put this in contrib/ first if it eases the introduction. I would like to move it to the subcommand area after getting everything in tip-top shape. What I don't want is for it to languish forever in contrib/. That means I'll need some guideline of the changes/standards necessary to qualify it for transition from contrib/ to an official subcommand. I expect we'll develop that as we go along but I hope the git community has some institutional knowledge gathered from previous experience.

I have asked Avery how he wants to do maintenance going forward. I haven't heard back from him yet so I can't speak to whether the existing GitHub project will continue or not. I'll pass along his thoughts when I get them.

I certainly don't want this to be an "over the wall" operation. I intend to participate in maintenance of git-subtree in the official git repository.

So I'll go ahead and work on adding this to contrib/. Once I get a response from Avery about his long-term vision I'll pass that along and we can have further discussion. I may start sending patches to the mailing list for review before hearing back from him, however.

Sound good?

                             -Dave