{"thread":{"id":"19607","subject":"[git subtree] documentation and new --squash mode","startedAt":"2009-05-30T18:36:18Z","lastAt":"2009-06-01T01:09:56Z","messageCount":4,"participants":["Avery Pennarun","Sitaram Chamarty"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"115094","messageId":"32541b130905301136i2d9ec05ew9ba54c6ca746af15@mail.gmail.com","threadId":"19607","inReplyTo":null,"subject":"[git subtree] documentation and new --squash mode","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-30T18:36:18Z","receivedAt":"2009-05-30T18:36:18Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"Hi all,\n\nI've finally had some time to write an asciidoc man page for git\nsubtree, which is included below for your convenience.\n\nI added a new \"--squash\" option to the subtree merge command, which\nlets you do cool stuff like switch your subtree branch back and forth\nbetween upstream versions without actually losing any of your local\ncustomizations.  It also avoids the problem of polluting your\nproject's history: it pulls in only individual subproject trees (by\ncreating a synthetic commit for the desired tree, with its parent\nbeing the *previous* synthetic commit).  This saves repository size\n(you don't slurp in all the intermediate trees), plus you don't get\nall the commit messages from the subproject, which are probably\nirrelevant to you anyway.\n\nAs always, comments welcome.  Quite a few people have emailed me about\ngit subtree lately; I think the --squash mode will probably resolve\nthe problems for almost everyone who had them.  Assuming it works\ncorrectly, which is of course 100% guaranteed or your money back.\n\nYou can always find the latest git-subtree at:\ngit://github.com/apenwarr/git-subtree\n\nDocs below.  Check out the EXAMPLES section for a quick start.\n\nHave fun,\n\nAvery\n\n\ngit-subtree(1)\n==============\n\nNAME\n----\ngit-subtree - add, merge, and split subprojects stored in subtrees\n\n\nSYNOPSIS\n--------\n[verse]\n'git subtree' add   --prefix=<prefix> <commit>\n'git subtree' merge --prefix=<prefix> <commit>\n'git subtree' pull  --prefix=<prefix> <repository> <refspec...>\n'git subtree' split --prefix=<prefix> <commit...>\n\n\nDESCRIPTION\n-----------\ngit subtree allows you to include an subproject in your\nown repository as a subdirectory, optionally including the\nsubproject's entire history. For example, you could\ninclude the source code for a library as a subdirectory of your\napplication.\n\nYou can also extract the entire history of a subdirectory from\nyour project and make it into a standalone project.  For\nexample, if a library you made for one application ends up being\nuseful elsewhere, you can extract its entire history and publish\nthat as its own git repository, without accidentally\nintermingling the history of your application project.\n\nMost importantly, you can alternate back and forth between these\ntwo operations.  If the standalone library gets updated, you can\nautomatically merge the changes into your project; if you\nupdate the library inside your project, you can \"split\" the\nchanges back out again and merge them back into the library\nproject.\n\nUnlike the 'git submodule' command, git subtree doesn't produce\nany special constructions (like .gitmodule files or gitlinks) in\nyour repository, and doesn't require end-users of your\nrepository to do anything special or to understand how subtrees\nwork.  A subtree is just another subdirectory and can be\ncommitted to, branched, and merged along with your project in\nany way you want.\n\nIn order to keep your commit messages clean, we recommend that\npeople split their commits between the subtrees and the main\nproject as much as possible.  That is, if you make a change that\naffects both the library and the main application, commit it in\ntwo pieces.  That way, when you split the library commits out\nlater, their descriptions will still make sense.  But if this\nisn't important to you, it's not *necessary*.  git subtree will\nsimply leave out the non-library-related parts of the commit\nwhen it splits it out into the subproject later.\n\n\nCOMMANDS\n--------\nadd::\n\tCreate the <prefix> subtree by importing its contents\n\tfrom the given commit.  A new commit is created\n\tautomatically, joining the imported project's history\n\twith your own.  With '--squash', imports only a single\n\tcommit from the subproject, rather than its entire\n\thistory.\n\nmerge::\n\tMerge recent changes up to <commit> into the <prefix>\n\tsubtree.  As with normal 'git merge', this doesn't\n\tremove your own local changes; it just merges those\n\tchanges into the latest <commit>.  With '--squash',\n\tcreates only one commit that contains all the changes,\n\trather than merging in the entire history.\n\n\tIf you use '--squash', the merge direction doesn't\n\talways have to be forward; you can use this command to\n\tgo back in time from v2.5 to v2.4, for example.  If your\n\tmerge introduces a conflict, you can resolve it in the\n\tusual ways.\n\t\npull::\n\tExactly like 'merge', but parallels 'git pull' in that\n\tit fetches the given commit from the specified remote\n\trepository.\n\nsplit::\n\tExtract a new, synthetic project history from the\n\thistory of the <prefix> subtree.  The new history\n\tincludes only the commits (including merges) that\n\taffected <prefix>, and each of those commits now has the\n\tcontents of <prefix> at the root of the project instead\n\tof in a subdirectory.  Thus, the newly created history\n\tis suitable for export as a separate git repository.\n\t\n\tAfter splitting successfully, a single commit id is\n\tprinted to stdout.  This corresponds to the HEAD of the\n\tnewly created tree, which you can manipulate however you\n\twant.\n\t\n\tRepeated splits of exactly the same history are\n\tguaranteed to be identical (ie. to produce the same\n\tcommit ids).  Because of this, if you add new commits\n\tand then re-split, the new commits will be attached as\n\tcommits on top of the history you generated last time,\n\tso 'git merge' and friends will work as expected.\n\t\n\tNote that if you use '--squash' when you merge, you\n\tshould usually not just '--rejoin' when you split.\n\n\nOPTIONS\n-------\n-q::\n--quiet::\n\tSuppress unnecessary output messages on stderr.\n\n-d::\n--debug::\n\tProduce even more unnecessary output messages on stderr.\n\n--prefix=<prefix>::\n\tSpecify the path in the repository to the subtree you\n\twant to manipulate.  This option is currently mandatory\n\tfor all commands.\n\n\nOPTIONS FOR add, merge, AND pull\n--------------------------------\n--squash::\n\tInstead of merging the entire history from the subtree\n\tproject, produce only a single commit that contains all\n\tthe differences you want to merge, and then merge that\n\tnew commit into your project.\n\n\tUsing this option helps to reduce log clutter. People\n\trarely want to see every change that happened between\n\tv1.0 and v1.1 of the library they're using, since none of the\n\tinterim versions were ever included in their application.\n\t\n\tUsing '--squash' also helps avoid problems when the same\n\tsubproject is included multiple times in the same\n\tproject, or is removed and then re-added.  In such a\n\tcase, it doesn't make sense to combine the histories\n\tanyway, since it's unclear which part of the history\n\tbelongs to which subtree.\n\t\n\tFurthermore, with '--squash', you can switch back and\n\tforth between different versions of a subtree, rather\n\tthan strictly forward.  'git subtree merge --squash'\n\talways adjusts the subtree to match the exactly\n\tspecified commit, even if getting to that commit would\n\trequire undoing some changes that were added earlier.\n\t\n\tWhether or not you use '--squash', changes made in your\n\tlocal repository remain intact and can be later split\n\tand send upstream to the subproject.\n\n\nOPTIONS FOR split\n-----------------\n--annotate=<annotation>::\n\tWhen generating synthetic history, add <annotation> as a\n\tprefix to each commit message.  Since we're creating new\n\tcommits with the same commit message, but possibly\n\tdifferent content, from the original commits, this can help\n\tto differentiate them and avoid confusion.\n\t\n\tWhenever you split, you need to use the same\n\t<annotation>, or else you don't have a guarantee that\n\tthe new re-created history will be identical to the old\n\tone.  That will prevent merging from working correctly.\n\tgit subtree tries to make it work anyway, particularly\n\tif you use --rejoin, but it may not always be effective.\n\n-b <branch>::\n--branch=<branch>::\n\tAfter generating the synthetic history, create a new\n\tbranch called <branch> that contains the new history.\n\tThis is suitable for immediate pushing upstream.\n\t<branch> must not already exist.\n\n--ignore-joins::\n\tIf you use '--rejoin', git subtree attempts to optimize\n\tits history reconstruction to generate only the new\n\tcommits since the last '--rejoin'.  '--ignore-join'\n\tdisables this behaviour, forcing it to regenerate the\n\tentire history.  In a large project, this can take a\n\tlong time.\n\n--onto=<onto>::\n\tIf your subtree was originally imported using something\n\tother than git subtree, its history may not match what\n\tgit subtree is expecting.  In that case, you can specify\n\tthe commit id <onto> that corresponds to the first\n\trevision of the subproject's history that was imported\n\tinto your project, and git subtree will attempt to build\n\tits history from there.\n\t\n\tIf you used 'git subtree add', you should never need\n\tthis option.\n\n--rejoin::\n\tAfter splitting, merge the newly created synthetic\n\thistory back into your main project.  That way, future\n\tsplits can search only the part of history that has\n\tbeen added since the most recent --rejoin.\n\t\n\tIf your split commits end up merged into the upstream\n\tsubproject, and then you want to get the latest upstream\n\tversion, this will allow git's merge algorithm to more\n\tintelligently avoid conflicts (since it knows these\n\tsynthetic commits are already part of the upstream\n\trepository).\n\t\n\tUnfortunately, using this option results in 'git log'\n\tshowing an extra copy of every new commit that was\n\tcreated (the original, and the synthetic one).\n\t\n\tIf you do all your merges with '--squash', don't use\n\t'--rejoin' when you split, because you don't want the\n\tsubproject's history to be part of your project anyway.\n\n\nEXAMPLES\n--------\nLet's use the repository for the git source code as an example.\nFirst, get your own copy of the git.git repository:\n\n\t$ git clone git://git.kernel.org/pub/scm/git/git.git test-git\n\t$ cd test-git\n\ngitweb (commit 1130ef3) was merged into git as of commit\n0a8f4f0, after which it was no longer maintained separately.\nBut imagine it had been maintained separately, and we wanted to\nextract git's changes to gitweb since that time, to share with\nthe upstream.  You could do this:\n\n\t$ git subtree split --prefix=gitweb --annotate='(split) ' \\\n        \t0a8f4f0^.. --onto=1130ef3 --rejoin \\\n        \t--branch gitweb-latest\n        $ gitk gitweb-latest\n        $ git push git@github.com:whatever/gitweb gitweb-latest:master\n\n(We use '0a8f4f0^..' because that means \"all the changes from\n0a8f4f0 to the current version, including 0a8f4f0 itself.\")\n\nIf gitweb had originally been merged using 'git subtree add' (or\na previous split had already been done with --rejoin specified)\nthen you can do all your splits without having to remember any\nweird commit ids:\n\n\t$ git subtree split --prefix=gitweb --annotate='(split) ' --rejoin \\\n\t\t--branch gitweb-latest2\n\nAnd you can merge changes back in from the upstream project just\nas easily:\n\n\t$ git subtree pull --prefix=gitweb \\\n\t\tgit@github.com:whatever/gitweb gitweb-latest:master\n\nOr, using '--squash', you can actually rewind to an earlier\nversion of gitweb:\n\n\t$ git subtree merge --prefix=gitweb --squash gitweb-latest~10\n\nThen make some changes:\n\n\t$ date >gitweb/myfile\n\t$ git add gitweb/myfile\n\t$ git commit -m 'created myfile'\n\nAnd fast forward again:\n\n\t$ git subtree merge --prefix=gitweb --squash gitweb-latest\n\nAnd notice that your change is still intact:\n\t\n\t$ ls -l gitweb/myfile\n\nAnd you can split it out and look at your changes versus\nthe standard gitweb:\n\n\tgit log gitweb-latest..$(git subtree split --prefix=gitweb)\n\n\nAUTHOR\n------\nWritten by Avery Pennarun <apenwarr@gmail.com>\n\n\nGIT\n---\nPart of the linkgit:git[1] suite\n"},{"id":"115116","messageId":"slrnh24bff.n9t.sitaramc@sitaramc.homelinux.net","threadId":"19607","inReplyTo":"32541b130905301136i2d9ec05ew9ba54c6ca746af15@mail.gmail.com","subject":"Re: [git subtree] documentation and new --squash mode","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2009-05-31T07:12:47Z","receivedAt":"2009-05-31T07:12:47Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 2009-05-30 18:36:18, Avery Pennarun <apenwarr@gmail.com> wrote:\n\n> I've finally had some time to write an asciidoc man page for git\n> subtree, which is included below for your convenience.\n\nGreat!  Thanks; I've been sort of holding off on trying this\nfor a while and this ought to push me.\n\nI have a very high level question.  My situation is that I\nam evangelising git at work, and the bulk of the audience\nare SVN, with some VSS and CVS here and there.\n\nAs such, one of the main things I have to tell them is \"you\ncan't just take a subdirectory of the main project and check\nit out/work/commit, etc\", and that \"if you can plan it\nproperly you can do it using submodules\".\n\nThe flaw in the above is that I myself have not yet used\nsubmodules other than doing the toy example in the docs, and\nam not confident of being able to solve arbitrary problems\nthat may be presented to me by folks.  The need to force\nthem to think and plan this in advance is also sometimes an\nimpediment.  [If you're surprised, you need to put \"the\ndaily wtf\" on your RSS feed ;-)]\n\nAs I understand the documentation (and past emails, snippets\nof IRC traffic, etc) git subtree will help me do precisely\nwhat I want -- a simple way to deal with this all-too-common\nSVN-ism.  If not \"simple\", at least\n\n  - simpler than submodules, and\n\n  - no need for advance planning [1] about what parts will\n    be subprojects.\n\nIs my understanding correct?\n\nThanks,\n\nSitaram\n\n[1] I do understand filter-branch quite well, and I know that\neven without advance planning it is possible to separate a\nsubdirectory into its own repo using filter-branch.  I don't\nfancy teaching *them* filter-branch, so that makes me\npersonally central to their projects, which is not scalable\nat all.\n"},{"id":"115153","messageId":"32541b130905310942x3ca98b0ibbed8598f4ba6e56@mail.gmail.com","threadId":"19607","inReplyTo":"slrnh24bff.n9t.sitaramc@sitaramc.homelinux.net","subject":"Re: [git subtree] documentation and new --squash mode","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2009-05-31T16:42:43Z","receivedAt":"2009-05-31T16:42:43Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Sun, May 31, 2009 at 3:12 AM, Sitaram Chamarty <sitaramc@gmail.com> wrote:\n> As I understand the documentation (and past emails, snippets\n> of IRC traffic, etc) git subtree will help me do precisely\n> what I want -- a simple way to deal with this all-too-common\n> SVN-ism.  If not \"simple\", at least\n>\n>  - simpler than submodules, and\n>\n>  - no need for advance planning [1] about what parts will\n>    be subprojects.\n\nWell, I don't know that I'd go so far as to advise against advance\nplanning :)  But git subtree does make it easy for you to change your\nmind later, that much is true.\n\nYou might still have some troubles if you rename subdirectories a lot;\nperhaps I don't understand it correctly, but the so-called file\nmove/rename detection in git doesn't seem to be implemented\neverywhere. If you extract the history of a subdirectory that was\npreviously renamed, you'll get only the history since it was renamed.\n(Which might be fine with you.  You can certainly still merge it back\nin afterwards.)\n\nAlso, git-subtree might have bugs yet.  Watch out for that :)\n\n> [1] I do understand filter-branch quite well, and I know that\n> even without advance planning it is possible to separate a\n> subdirectory into its own repo using filter-branch.  I don't\n> fancy teaching *them* filter-branch, so that makes me\n> personally central to their projects, which is not scalable\n> at all.\n\ngit-subtree is certainly easier to use than filter-branch, IMHO.  Plus\nit helps you merge things back in afterwards, which filter-branch\ndoesn't particularly.\n\nHave fun,\n\nAvery\n"},{"id":"115170","messageId":"2e24e5b90905311809h751eff3en7f5a87f1e5a6ab03@mail.gmail.com","threadId":"19607","inReplyTo":"32541b130905310942x3ca98b0ibbed8598f4ba6e56@mail.gmail.com","subject":"Re: [git subtree] documentation and new --squash mode","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2009-06-01T01:09:56Z","receivedAt":"2009-06-01T01:09:56Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sun, May 31, 2009 at 10:12 PM, Avery Pennarun <apenwarr@gmail.com> wrote:\n>\n> On Sun, May 31, 2009 at 3:12 AM, Sitaram Chamarty <sitaramc@gmail.com> wrote:\n> > As I understand the documentation (and past emails, snippets\n> > of IRC traffic, etc) git subtree will help me do precisely\n> > what I want -- a simple way to deal with this all-too-common\n> > SVN-ism.  If not \"simple\", at least\n> >\n> >  - simpler than submodules, and\n> >\n> >  - no need for advance planning [1] about what parts will\n> >    be subprojects.\n>\n> Well, I don't know that I'd go so far as to advise against advance\n> planning :)  But git subtree does make it easy for you to change your\n> mind later, that much is true.\n\nAgreed, and that's good enough for me.\n\n> You might still have some troubles if you rename subdirectories a lot;\n> perhaps I don't understand it correctly, but the so-called file\n> move/rename detection in git doesn't seem to be implemented\n> everywhere. If you extract the history of a subdirectory that was\n> previously renamed, you'll get only the history since it was renamed.\n> (Which might be fine with you.  You can certainly still merge it back\n> in afterwards.)\n\nDid not know this; thanks -- will keep an eye out.\n\n> Also, git-subtree might have bugs yet.  Watch out for that :)\n\n:)\n\n> git-subtree is certainly easier to use than filter-branch, IMHO.  Plus\n> it helps you merge things back in afterwards, which filter-branch\n> doesn't particularly.\n\nAll in all it does look like it will fulfill my purposes.  I will\nprobably use it myself a bit before telling people to use it of\ncourse.\n\nThanks,\n\nSitaram\n"}]}