{"thread":{"id":"1590","subject":"Multi-head fetches, pulls, and a King Ghidorah","startedAt":"2005-08-19T07:45:07Z","lastAt":"2005-08-20T11:02:03Z","messageCount":5,"participants":["Junio C Hamano","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"7550","messageId":"7vy86y1ibg.fsf@assigned-by-dhcp.cox.net","threadId":"1590","inReplyTo":null,"subject":"Multi-head fetches, pulls, and a King Ghidorah","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-19T07:45:07Z","receivedAt":"2005-08-19T07:45:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The current \"pu\" branch has most of the necessary plumbing for\nmulti-head fetching, pulling and creating Octopus merges based\non multiple heads.  I have made \"git pull\" multi-head aware but\ndid not make it multi-head capable.\n\nI have been trying out Tony Luck's excellent topic-branch\nworkflow (see Documentation/using-topic-branches.txt), and the\nhead of the \"pu\" branch is an Octopus on top of the \"master\" and\nmy two private topic branches [*1*] made with \"git octopus\".\n\nI have been thinking about how the multi-head fetch and merge\nshould interact with each other.  It appears to me that coming\nup with a reasonably semantics for \"git pull\" that would fit\neverybody is very difficult, because there are at least two\ndifferent valid workflows that involve multi-head pulling, and\nthey need to do different things.  I'll talk about \"git pull\"\nthat can deal with more than one remote heads, which does not\nexist yet, in the next couple of paragraphs.\n\nSuppose I were the Emperor Penguin [*2*], and Jeff asks \"Please\npull from my netdev-2.6.git repository, sis190 and e100\nbranches.\"  It is very reasonable to expect the following to\nfetch heads of those two branches and attempt to make an Octopus\n[*3*] on top of a copy of the Penguin head:\n\n    $ cat .git/branches/jgarzik\n    kernel.org:/pub/linux/kernel/git/jgarzik/\n    $ git checkout -b try-jeffs-update master\n    $ git pull jgarzik/netdev-2.6.git/ sis190 e100\n    $ git diff -p master..HEAD | git apply --stat --summary\n\nThe arguments to pull, especially the refspec parameters\n(\"sis190\" and \"e100\" in the above example) are given to the\nunderlying \"git fetch\", and because neither of them have a\ncolon, they are not stored in corresponding places under the\nlocal refs/heads/.  Instead, fetched heads would be fed directly\nto \"git octopus\" inside \"git pull\" (just like the current \"git\npull\" invokes \"git resolve\" internally after fetching just one\nhead).\n\nHowever, suppose then I were Joe Random, an individual netdev\ncontributor who is interested in these two netdev branches.\nUpon seeing the pull request, I might decide it is a good time\nto get changes from there, my upstream.\n\n    $ git pull jgarzik/netdev-2.6.git/ sis190:sis190 e100:e100\n\nBecause I am keeping track of copies of these two branches, I\nuse \"sis190:sis190 e100:e100\" to update my local heads.\n\nI might have some local changes in these two branches.  The old\n\"git fetch\" unconditionally overwrote local heads when told to,\nbut lately it acquired the \"reverse push\" semantics Johannes\nSchindelin proposed to make it safer.  The fetch process only\noverwrites local heads when the upstream change results in a\nfast-forward merge; practically, that happens only when I have\nnot worked on that branch since I pulled from the upstream the\nlast time.  So my refs/heads/sis190 and refs/heads/e100 may be\ncopies of Jeff's heads, or they may be the same heads as I had\nbefore starting the fetch.\n\nAt this point, it might be reasonable to expect that the above\n\"git pull\" command would behave as if I pulled (i.e. fetched and\nmerged) these heads separately, using traditional single-head\npull:\n\n    $ git checkout sis190\n    $ git pull jgarzik/netdev-2.6.git/ sis190:sis190\n    $ git checkout e100\n    $ git pull jgarzik/netdev-2.6.git/ e100:e100\n\nThat is, fetch and resolve them independently and individually.\n\nBack in the Emperor Penguin example, he _could_ also have been\ninterested in keeping copies of Jeff's branch heads, so he could\nhave written refspecs on the command line the same way as Joe\nRandom did.  I.e. instead of:\n\n    $ git pull jgarzik/netdev-2.6.git/ sis190 e100\n\nhe could have said:\n\n    $ git pull jgarzik/netdev-2.6.git/ sis190:sis190 e100:e100\n\nBoth \"Octopus\" and \"Multiple independent pull\" semantics are\nvalid to support different workflows, and there is no way to\ndifferenciate the two from the command line, without giving an\nextra flag and making the implementation more complicated.\n\nCurrently, I am inclined to leave the current \"not more than one\nremote head\" implementation, and possibly extend it to support\nthe \"Octopus\" semantics later, for three very simple reasons.\n\n (1) What the latter \"git pull\" is buying us compared to two\n     traditional single-head pulls is very little; that the\n     underlying \"git fetch\" _could_ obtain packs more\n     efficiently than two independent fetches.\n\n (2) The netdev example happened to involve multiple heads from\n     a single repository, but the pull request could as well\n     have been \"jgarzik/netdev-2.6.git#sis190 and\n     jgarzik/libata-dev.git#sil24\", in which case I wouldn't\n     have to be worrying about multi-head pull at all; the user\n     would just have used two independent traditional \"git\n     pull\" --- there is no other option.\n\n (3) Or course, because I am lazy ;-).  Seriously, multiple\n     independent merges is a nightmere when you start thinking\n     about what to do when you get a conflict and need to have\n     the user hand merge in the middle of the first one.\n\nSince \"git pull\" tentatively would not do multi-head natively,\nthe Emperor Penguin example needs to be done this way:\n\n    $ git checkout -b try-jeffs-update master\n    $ git fetch jgarzik/netdev-2.6.git/ sis190 e100\n    $ git octopus\n    $ git diff -p master..HEAD | git apply --stat --summary\n\nI think this should already work with the current \"pu\" branch\nhead.\n\n\n[Footnotes]\n\n*1* I should probably write a bit about how I do things in a\nseparate message as a how-to.\n\n*2* I am not a penguin.  Figuring out what kind of animal I am\nis left as an easter-egg hunt.  I have had the answer somewhere\nin the current git.git archive for some time ;-).\n\n*3* This one has only three heads, so that would be a Tripus,\nbut what X-pus counts is not heads but legs, so we should really\nbe calling this a King Ghidorah who has three heads.\n\n*4* What's currently in \"pu\" branch dies when a head cannot be\nfast forwarded, but I think that is simply a thinko.  It should\njust refuse to fast-forward and warn, leave the obtained remote\nhead somewhere other tools can find later, just as the original\n\"reverse push\" script by Johannes did.\n"},{"id":"7558","messageId":"Pine.LNX.4.63.0508191039460.11916@wgmdd8.biozentrum.uni-wuerzburg.de","threadId":"1590","inReplyTo":"7vy86y1ibg.fsf@assigned-by-dhcp.cox.net","subject":"Re: Multi-head fetches, pulls, and a King Ghidorah","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-08-19T09:14:54Z","receivedAt":"2005-08-19T09:14:54Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nIf I understand correctly, the multi-head fetch would not write any ref if \nused like this:\n\n\tgit fetch remote:repository/ head tail\n\nbut it would try a fast-forward when used like this:\n\n\tgit fetch remote:repository/ head:head tail:tail\n\nCorrect? If yes: This is fantastic! It obsoletes my dumb script.\n\nOn Fri, 19 Aug 2005, Junio C Hamano wrote:\n\n> However, suppose then I were Joe Random, an individual netdev\n> contributor who is interested in these two netdev branches.\n> Upon seeing the pull request, I might decide it is a good time\n> to get changes from there, my upstream.\n> \n>     $ git pull jgarzik/netdev-2.6.git/ sis190:sis190 e100:e100\n> \n> Because I am keeping track of copies of these two branches, I\n> use \"sis190:sis190 e100:e100\" to update my local heads.\n\nI propose a \"--separate\" flag to git pull. This would do exactly the same \nas a plain git pull, but for each fetched branch which could not be \nfast-forwarded\n\n\t- try to switch to the branch (dying if it is not the current,\n\t  and the working tree is dirty)\n\n\t- try a merge\n\n\t- if the merge fails, reset the branch to original state,\n\t  write out a temporary head and output a warning\n\nAfter that, it would switch back to the original branch and check that \nout.\n\nFor all failed merges, the user needs to \"git resolve\" (the exact command \nline could be output by \"git pull --separate\").\n\nCiao,\nDscho\n"},{"id":"7561","messageId":"7vslx6z0ja.fsf@assigned-by-dhcp.cox.net","threadId":"1590","inReplyTo":"Pine.LNX.4.63.0508191039460.11916@wgmdd8.biozentrum.uni-wuerzburg.de","subject":"Re: Multi-head fetches, pulls, and a King Ghidorah","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-19T10:25:13Z","receivedAt":"2005-08-19T10:25:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> Hi,\n>\n> If I understand correctly, the multi-head fetch would not write any ref if \n> used like this:\n>\n> \tgit fetch remote:repository/ head tail\n>\n> but it would try a fast-forward when used like this:\n>\n> \tgit fetch remote:repository/ head:head tail:tail\n>\n> Correct? If yes: This is fantastic! It obsoletes my dumb script.\n\nThat is the intent, but what is currently in \"pu\" is somewhat\nbroken.\n"},{"id":"7581","messageId":"7voe7tqpae.fsf@assigned-by-dhcp.cox.net","threadId":"1590","inReplyTo":"7vy86y1ibg.fsf@assigned-by-dhcp.cox.net","subject":"My GIT Day","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-20T03:07:37Z","receivedAt":"2005-08-20T03:07:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <junkio@cox.net> writes:\n\n> *1* I should probably write a bit about how I do things in a\n> separate message as a how-to.\n\nSo here it is.\n\nNote that the version of git on my $PATH is usually the one from\nthe proposed updates branch, so some of the commands I use in\nthe following text may not work for you unless you also have\nbuilt \"pu\" one yourself.\n\nI am planning to finish updating, testing and documenting what's\nin the current proposed updates branch, and have most of them\ngraduate to the master branch over the weekend.  I am aiming for\ndoing the 0.99.5 on Wednesday next week.\n\n------------\n\nI have the following heads all the time in my private\nrepository:\n\n    master\t- the one to be pushed to public \"master\" branch\n    pu\t\t- master plus various proposed changes\n    rc\t\t- master plus minimum release engineering\n\n    ko-master\t- a copy of public \"master\" branch head\n    ko-rc\t- a copy of public \"rc\" branch head\n\nMy GIT day always starts with this command:\n\n    $ git fetch ko\n\nI have this in .git/remotes/ko:\n\n    $ cat .git/remotes/ko\n    URL: master.kernel.org:/pub/scm/git/git.git/\n    Pull: master:ko-master rc:ko-rc\n    Push: master pu rc\n\nThe Pull: line gives me the default set of <refspec>s to give to\nthe \"git fetch\" command.  I slurp \"master\" and \"rc\" heads from\nthe public repository at master.kernel.org, and fast-forward my\nko-master and ko-rc branches with them.  I do not touch these\ntwo branches in any way other than this \"git fetch\" fast\nforwarding.\n\nI have a few \"topic branches\" in addition to the above; they\nchange from time to time.  Recently I've been looking at\nmulti-head fetches, and that work is done in \"mhf\" branch.\nThere also is a catch-all topic called \"misc\".  I started them\nlike this:\n\n    $ git branch mhf master\n    $ git branch misc master\n\nThe first thing I do during my GIT day is to process the patches\nI received via e-mail.  I store them one topic per file in my\nworking tree, like this:\n\n    $ ls +*.txt\n    +js-glossary.txt  +mc-mailinfo.txt\n\nDepending on the quality of the patch, seriousness of the bug\nthey fix, and the area of the code they touch, they either go\ndirectly to \"master\", \"misc\", or sent back to the sender, but\nthe last one, luckily for me, rarely happens:\n\n    $ git checkout master\n    $ git applymbox -q ./+js-glossary.txt .git/info/signoff\n\n    $ git checkout misc\n    $ git applymbox -q ./+mc-mailinfo.txt please\n\nThe last parameter to the applymbox command is the name of a\nfile that has my signoff message.  The latest applymbox in the\n\"pu\" branch has a bit more useful extension to do the same thing\nas what \"git commit\" does.\n\nAt this point, I may push out the \"master\" branch (and nothing\nelse), like this:\n\n    $ git push ko master\n\nThis pushes only \"master\", ignoring the default <refspec>s\ndefined in the .git/remotes/ko file you saw earlier.\n\nOnce I am done with the outside patches, I go back to where I\nleft off the previous day:\n\n    $ git checkout mhf\n\nAnd I check where my head is relative to the master:\n\n    $ git show-branches master mhf\n    ! [master] Yet another tweak\n     * [mhf] Make git-fetch-script a bit more chatty.\n    +  Yet another tweak\n    +  Another tweak in Makefile\n     + Make git-fetch-script a bit more chatty.\n     + Update git-ls-remote-script\n     + ...\n     + Start adding the $GIT_DIR/remotes/ support.\n    ++ [PATCH] Allow file removal when \"git commit --all\" is used.\n\nThe output from show-branches is a poor-man's gitk.  The named\nbranches are shown, and '+' sign in each column shows whether\nthe commit is contained in each branch, and the output stops\nwhere all branches converge, or you hit ^C ;-).\n\nIf the mhf branch is way behind, I may choose to first rebase\nit, to clean up my history:\n\n    $ git rebase master\n    $ git show-branches master mhf\n    ! [master] Yet another tweak\n     * [mhf] Make git-fetch-script a bit more chatty.\n     + Make git-fetch-script a bit more chatty.\n     + Update git-ls-remote-script\n     + ...\n     + Start adding the $GIT_DIR/remotes/ support.\n    ++ Yet another tweak\n\nI keep working in my topic branches.  I may make some other\nchanges in \"misc\" topic branch.  It's a simple cycle of:\n\n    $ edit-and-test\n    $ git commit -s -a -v\n\nEventually I get to a good point where it makes sense to push\nthings to the public repository.\n\n    $ git show-branches master mhf misc\n    ! [master] Yet another tweak\n     * [mhf] Make git-fetch-script a bit more chatty.\n      ! [misc] Add hooks to tools/git-applypatch.\n     +  Make git-fetch-script a bit more chatty.\n     +  Update git-ls-remote-script\n     +  ...\n     +  Start adding the $GIT_DIR/remotes/ support.\n    ++  Yet another tweak\n    ++  Another tweak in Makefile\n      + Add hooks to tools/git-applypatch.\n      + Add commit hook and make the verification customizable.\n    +++ [PATCH] Allow file removal when \"git commit --all\" is used.\n\nAs you may have noticed, my topic branches are private and not\npushed to the public repository.  Instead, I make a grand total\nmerge of them into \"pu\".  The proposed update branch is always\nrewound and made from the head of the master:\n\n    $ git checkout pu\n    $ git reset master\n\nThis checks out the head of \"pu\" branch, and then resets the\nindex file to match \"master\" and updates .git/refs/heads/pu.\n\nWhat it does _not_ do is to update my working tree to match the\nindex file.  Linus recommends to do \"git checkout -f\" at this\npoint, but I typically do this instead:\n\n    $ git diff -R -p | git apply\n\nNote.  This is an embarrassingly expensive way; the only thing\nit buys me over \"git checkout -f\" is that it removes the files\nthat were introduced in \"pu\" branch that did not exist in the\n\"master\" head.  I have to come up with a not so expensive way to\ndo this.\n\nThen before doing the grand total merge, check again where those\nheads are:\n\n    $ git show-branches master mhf misc pu \n    ! [master] Yet another tweak\n     ! [mhf] Make git-fetch-script a bit more chatty.\n      ! [misc] Add hooks to tools/git-applypatch.\n       * [pu] Yet another tweak\n     +   Make git-fetch-script a bit more chatty.\n     +   Update git-ls-remote-script\n     +   ...\n     +   Start adding the $GIT_DIR/remotes/ support.\n    ++ + Yet another tweak\n    ++ + Another tweak in Makefile\n      +  Add hooks to tools/git-applypatch.\n      +  Add commit hook and make the verification customizable.\n    ++++ [PATCH] Allow file removal when \"git commit --all\" is used.\n\nNotice that I did not rebase \"misc\" above, but that is OK.  What\nI want to do here is to make new \"pu\" an Octopus over \"master\",\nmerging in all my topic branches (currently \"mhf\" and \"misc\").\n\n    $ git fetch . mhf misc\n    Packing 0 objects\n    Unpacking 0 objects\n    * committish: a101f32...e5580\trefs/heads/mhf from .\n    * committish: 4426ac7...0c5bc\trefs/heads/misc from .\n\nThis fetches two heads from the current repository (!).  The\nonly reason I do it is that the tentative implementation of \"git\noctopus\" always reads from $GIT_DIR/FETCH_HEAD, and \"git fetch\"\nis the way to populate that file.\n\n    $ git octopus\n    Removing git-parse-remote\n    Removing git-parse-remote\n    Committed octopus merge fe1899156bffa4be6722b2ca0b74ff17b76523da\n     Makefile                        |    3 -\n     git-commit-script               |   75 +++++-------------\n     ...\n     tools/git-applypatch            |   87 ++++++++++++++++----\n     15 files changed, 551 insertions(+), 224 deletions(-)\n\nThis makes an Octopus out of the master and other two topic\nbranches.  I can make sure that resulting \"pu\" contains all the\nnecessary commits from the branches involved:\n\n    $ git show-branches master mhf misc pu\n    ! [master] Yet another tweak\n     ! [mhf] Make git-fetch-script a bit more chatty.\n      ! [misc] Add hooks to tools/git-applypatch.\n       * [pu] Octopus merge of the following: \n       + Octopus merge of the following:\n     + + Make git-fetch-script a bit more chatty.\n     + + Update git-ls-remote-script\n     + + ...\n     + + Start adding the $GIT_DIR/remotes/ support.\n    ++ + Yet another tweak\n    ++ + Another tweak in Makefile\n      ++ Add hooks to tools/git-applypatch.\n      ++ Add commit hook and make the verification customizable.\n    ++++ [PATCH] Allow file removal when \"git commit --all\" is used.\n\nAnd things are now ready to be pushed out.  First I sanity check\nthe differences between ko-master and master (the earlier \"git\nfetch ko\" was done only for this step):\n\n    $ git show-branches master ko-master\n\n... and then run \"git push\":\n\n    $ git push ko\n\nThis will push \"master\" and \"rc\" but would fail to push \"pu\",\nbecause that is rebased and not based on what is on the public\nrepository.  So I push once more, this time with --force, like\nthis:\n\n    $ git push --force ko pu\n\nThis pushes only \"pu\", ignoring the default <refspec>s defined\nin the .git/remotes/ko file you saw earlier.  After that, I go\nback to reading the mail and wait until the kernel.org mirror\nnetwork catches up.\n"},{"id":"7583","messageId":"Pine.LNX.4.63.0508201254040.32350@wgmdd8.biozentrum.uni-wuerzburg.de","threadId":"1590","inReplyTo":"7voe7tqpae.fsf@assigned-by-dhcp.cox.net","subject":"Re: My GIT Day","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-08-20T11:02:03Z","receivedAt":"2005-08-20T11:02:03Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 19 Aug 2005, Junio C Hamano wrote:\n\n>     $ git checkout pu\n>     $ git reset master\n> \n> This checks out the head of \"pu\" branch, and then resets the\n> index file to match \"master\" and updates .git/refs/heads/pu.\n> \n> What it does _not_ do is to update my working tree to match the\n> index file.  Linus recommends to do \"git checkout -f\" at this\n> point, but I typically do this instead:\n> \n>     $ git diff -R -p | git apply\n> \n> Note.  This is an embarrassingly expensive way; the only thing\n> it buys me over \"git checkout -f\" is that it removes the files\n> that were introduced in \"pu\" branch that did not exist in the\n> \"master\" head.  I have to come up with a not so expensive way to\n> do this.\n\nHow about this?\n\n  $ git reset master\n  $ rm -i $(git-diff-tree ORIG_HEAD HEAD | grep \"^.\\{97\\}D\" | cut -c 100-)\n  $ git checkout -f\n\nCiao,\nDscho\n"}]}