{"thread":{"id":"18580","subject":"How to merge by subtree while preserving history?","startedAt":"2009-03-26T22:59:51Z","lastAt":"2009-03-27T17:20:36Z","messageCount":4,"participants":["David Reitter","Miklos Vajna","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"109584","messageId":"A5C2B218-4E4F-4C05-959D-5B6E9C619DDB@gmail.com","threadId":"18580","inReplyTo":null,"subject":"How to merge by subtree while preserving history?","fromName":"David Reitter","fromEmail":"david.reitter@gmail.com","sentAt":"2009-03-26T22:59:51Z","receivedAt":"2009-03-26T22:59:51Z","isPatch":false,"sender":{"key":"david.reitter@gmail.com","avatar":"https://gravatar.com/avatar/0b9f43c735737567aec3272fea5afa461ffa049f180103c1aab629aa6c1207f1?d=mp&s=160"},"body":"I have two separately developed projects (foo, bar) which I'd like to  \nmerge; the contents of foo should, initially, go in a subdirectory of  \nbar.\n\nI'm aware of two methods:  moving (renaming) everything within foo  \ninto foo-dir, and then just pulling foo into bar.\n\nThis works beautifully, except that the big rename causes havoc w.r.t.  \nto the files histories, i.e. git-log needs a \"--follow\" argument now,  \nand \"diff-tree\" can't track changes when given the new file name.  No  \ngood.\n\nI've also tried the method described in [1], but it seems that all  \nhistory is lost here (the text could point this out..)\nI've tried to \"git pull -s subtree foo master\" directly as well, but  \nthen it put foo into strange places (and lost the history).\n\n\nSo, I'm at a loss.  Suggestions much appreciated.\n\n\n\n\n[1] http://www.kernel.org/pub/software/scm/git/docs/howto/using-merge-subtree.html\n"},{"id":"109619","messageId":"20090327073847.GE27459@genesis.frugalware.org","threadId":"18580","inReplyTo":"A5C2B218-4E4F-4C05-959D-5B6E9C619DDB@gmail.com","subject":"Re: How to merge by subtree while preserving history?","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2009-03-27T07:38:47Z","receivedAt":"2009-03-27T07:38:47Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Thu, Mar 26, 2009 at 06:59:51PM -0400, David Reitter <david.reitter@gmail.com> wrote:\n> I have two separately developed projects (foo, bar) which I'd like to  \n> merge; the contents of foo should, initially, go in a subdirectory of  \n> bar.\n> \n> I'm aware of two methods:  moving (renaming) everything within foo  \n> into foo-dir, and then just pulling foo into bar.\n\nThe result of the two methods are the same.\n\n> This works beautifully, except that the big rename causes havoc w.r.t.  \n> to the files histories, i.e. git-log needs a \"--follow\" argument now,  \n> and \"diff-tree\" can't track changes when given the new file name.  No  \n> good.\n> \n> I've also tried the method described in [1], but it seems that all  \n> history is lost here (the text could point this out..)\n\nOf course it is not lost. :)\n\nExample:\n\ncommit f8c62880ef22b74ea6df47bb349ff0743d2a93f9\nMerge: f474c52... 52b8ea9...\nAuthor: Junio C Hamano <gitster@pobox.com>\nDate:   Sun Mar 1 22:20:52 2009 -0800\n\n    Merge git://git.kernel.org/pub/scm/gitk/gitk\n\nNow do a 'git log f474c52..52b8ea9' and you'll see the merged commits.\n\nBut you are right about that 'git log -- path' will find the merge\ncommits only (which is right, as the tree objects are not modified when\nmerging, just the resulting tree has the original tree in a\nsubdirectory).\n\nIf this is a one-time operation then I would just use git filter-branch\nto move the code to a subdir.\n"},{"id":"109664","messageId":"BA69E81F-C93F-4AD8-8486-A27B0A284D9A@gmail.com","threadId":"18580","inReplyTo":"20090327073847.GE27459@genesis.frugalware.org","subject":"Re: How to merge by subtree while preserving history?","fromName":"David Reitter","fromEmail":"david.reitter@gmail.com","sentAt":"2009-03-27T16:56:43Z","receivedAt":"2009-03-27T16:56:43Z","isPatch":false,"sender":{"key":"david.reitter@gmail.com","avatar":"https://gravatar.com/avatar/0b9f43c735737567aec3272fea5afa461ffa049f180103c1aab629aa6c1207f1?d=mp&s=160"},"body":"On Mar 27, 2009, at 3:38 AM, Miklos Vajna wrote:\n\n> Now do a 'git log f474c52..52b8ea9' and you'll see the merged commits.\n\nSure :)\nNeedless to say, this is not practical and doesn't support people's  \nworkflow.\n\nFor simple renames,  \"git log --follow\" helps, but as soon as you want  \nto do a \"diff\" in one of the listed revisions, filtering for just this  \none file, then history becomes invisible again.   Concretely, this  \nbreaks the common workflow with C-x C-v l, then \"d\" in Emacs.\n\nI'm aware of the content-tracking vs. file-tracking discussion; it's  \nall fine, except that file names are meaningful meta-data for some  \ncontent, at least in some projects.  Is there a command that gives me  \nthe diff  for a revision pair, restricted to what happened to content  \nin a given file in the current tree?\n\n\n> But you are right about that 'git log -- path' will find the merge\n> commits only (which is right, as the tree objects are not modified  \n> when\n> merging, just the resulting tree has the original tree in a\n> subdirectory).\n>\n> If this is a one-time operation then I would just use git filter- \n> branch\n> to move the code to a subdir.\n\n\nFor the record:\n\nIn the meantime, I managed to move the original file in the CVS  \nrepository (by just moving all the \",v\" files and getting rid of  \nCVSROOT/history, which doesn't seem needed).  The I re-ran cvsimport,  \nmitigating a bunch of problems with \"cvsps\".  For the record, cvsps /  \ncvsimport could not handle the case where my repository named \"foo\"  \nhad a subdirectory also called \"foo\", in which I moved all the  \nstuff.   I had to rename the directory to \"bar\".   I also had to  \ndelete cvsps's cache file with the -x argument (or delete it from the  \nsurprising location ~/.cvsps).\n\nThen, I merged with \"git pull\", noting the rev ID before the merge.\n\nNext, I used \"git filter-branch\" to rename the directory again from  \nBAR to FOO as follows:\n\ngit filter-branch --index-filter \\\n         'git ls-files -s | sed \"s-BAR/-FOO/-\" |\n                 GIT_INDEX_FILE=$GIT_INDEX_FILE.new \\\n                         git update-index --index-info &&\n          mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' <last-rev-before- \nmerge>..\n\nFinally, I had to \"git gc\" to prune a 200MB worth of objects (it told  \nme I had 500k objects overall).\n\n\n--\nhttp://aquamacs.org -- Aquamacs: Emacs on Mac OS X\n"},{"id":"109666","messageId":"7veiwievbv.fsf@gitster.siamese.dyndns.org","threadId":"18580","inReplyTo":"BA69E81F-C93F-4AD8-8486-A27B0A284D9A@gmail.com","subject":"Re: How to merge by subtree while preserving history?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-03-27T17:20:36Z","receivedAt":"2009-03-27T17:20:36Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Reitter <david.reitter@gmail.com> writes:\n\n> ...  Is there a command that gives me\n> the diff  for a revision pair, restricted to what happened to content\n> in a given file in the current tree?\n\nYou can get a half of it from blame (and I presume the other half by\nrunning the procedure in reverse).\n\n\"git blame\" has an obscure switch -S that lets you lie about the ancestry\nby allowing you to install a graft (this is primarily used by the annotate\noperation of git-cvsserver).\n\nSuppose you have revisions A and B, and a lot of code in a file F in the\noriginal revision A migrated to many other places in a later revision B\nover time.  You want to see where each and every line in F from A ended up\nin B.\n\nTo compute this, you pretend as if the history originates at B (i.e. B is\nthe root commit), and A is a direct descendant of it, and blame each and\nevery line of F in A, with a very agressive setting.  E.g.\n\n\t{\n\t\techo $(git rev-parse A) $(git rev-parse B)\n                echo $(git rev-parse B)\n\t} >tmp-graft\n        git blame -C -C -C -w -S tmp-graft A -- F\n\nI'll leave it as an exercise to the readers how to compute \"where did each\nand every line in G in B came from A?\"\n\nNote that in order for this to work, it needs a fix to \"blame -S\" that I\nposted about 10 days ago: aa9ea77 (blame: read custom grafts given by -S\nbefore calling setup_revisions(), 2009-03-18); the fix is sitting in 'pu',\nbecause as far as I know nobody has cared about the breakage other than I,\nat least until now.\n\nI've attached a script that uses this trick to compute \"How much of what\nLinus originally wrote still survives.\"  People who attended GitTogether'08\nmay have seen the result.\n\n---\n\n#!/bin/sh\n# How much of the very original version from Linus survive?\n\n_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'\n_x40=\"$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40\"\n\ninitial=$(git rev-parse --verify e83c5163316f89bfbde7d9ab23ca2e25604af290) &&\nthis=$(git rev-parse --verify ${1-HEAD}^0) || exit\n\ntmp=\"/var/tmp/Linus.$$\"\ntrap 'rm -f \"$tmp\".*' 0\n\n# We blame each file in the initial revision pretending as if it is a\n# direct descendant of the given version, and also pretend that the\n# latter is a root commit.  This way, lines in the initial revision\n# that survived to the other version can be identified (they will be\n# attributed to the other version).\ngraft=\"$tmp.graft\" &&\n{\n\techo \"$initial $this\"\n\techo \"$this\"\n} >\"$graft\" || exit\n\nopts='-C -C -C -w'\n\ngit ls-tree -r \"$initial\" |\nwhile read mode type sha1 name\ndo\n\tgit blame $opts --porcelain -S \"$graft\" \"$initial\" -- \"$name\" |\n\tsed -ne \"s/^\\($_x40\\) .*/\\1/p\" |\n\tsort |\n\tuniq -c | {\n\t\t# There are only two commits in the fake history, so\n\t\t# there won't be at most two output from the above.\n\t\tread cnt1 commit1\n\t\tread cnt2 commit2\n\t\tif test -z \"$commit2\"\n\t\tthen\n\t\t\tcnt2=0\n\t\tfi\n\t\tif test \"$initial\" != \"$commit1\"\n\t\tthen\n\t\t\tcnt_surviving=$cnt1\n\t\telse\n\t\t\tcnt_surviving=$cnt2\n\t\tfi\n\t\tcnt_total=$(( $cnt1 + $cnt2 ))\n\t\techo \"$cnt_surviving $cnt_total\t$name\"\n\t}\ndone | {\n\ttotal=0\n\tsurviving=0\n\twhile read s t n\n\tdo\n\t\ttotal=$(( $total + $t )) surviving=$(( $surviving + $s ))\n\t\tprintf \"%6d / %-6d\t%s\\n\" $s $t $n\n\tdone\n\tprintf \"%6d / %-6d\t%s\\n\" $surviving $total Total\n}\n"}]}