{"thread":{"id":"2372","subject":"[RFC] Applying a graft to a tree and \"rippling\" the changes through the history","startedAt":"2005-11-06T22:38:18Z","lastAt":"2005-11-18T08:49:34Z","messageCount":5,"participants":["Ryan Anderson","Randal L. Schwartz","Junio C Hamano","Matthias Urlichs"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"11224","messageId":"436E85DA.1080904@michonline.com","threadId":"2372","inReplyTo":null,"subject":"[RFC] Applying a graft to a tree and \"rippling\" the changes through the history","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2005-11-06T22:38:18Z","receivedAt":"2005-11-06T22:38:18Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"\nI've written a tool that will take a single commit, add it as a parent\nof another commit, and recreate the history above that second commit in\na fully compatible manner.\n\nThis is mostly useful for creating a fully merged-up repository of the\nLinux Historical tree, and the current working tree.\n\nI run this with /graft-ripple.pl linux-history.tmp/ linus origin\n\nWhere \"origin\" is the branch the historical repository is on, and\n\"linus\" is the branch the current repository is on.\n\nNote: This does not end up fixing up HEAD or any branches, it just pulls\nall the objects together and recreates the full history.\n\nGPLv2, but I'll redo with a proper patch, signed-off-by, command line\noptions and help and docs if anyone else feels this is useful as a\ngeneral tool.\n\n========= cut here =============\n\n#!/usr/bin/perl\n\nuse strict;\nuse warnings;\n\nuse Data::Dumper;\nuse IPC::Open2;\n\n\nsub git_commit_tree {\n        my ($tree,$comments,@parents) = @_;\n\n        my @cparents;\n        foreach my $p (@parents) {\n                push @cparents,\"-p\",$p;\n        }\n\n        my $pid = open2(*Reader, *Writer,\n\t\t\"git-commit-tree\",$tree,@cparents);\n        print Writer $comments;\n        close(Writer);\n        my $commit = <Reader>;\n\n        waitpid $pid, 0;\n        close(Reader);\n\n        chomp $commit;\n\n        return $commit;\n}\n\nchdir($ARGV[0]);\nopen(GRL,\"-|\",\"git-rev-list\",\"--parents\",$ARGV[1])\n\tor die \"Failed to run git-rev-list: \" . $!;\n\nmy %csets;\nmy @revs;\nwhile(<GRL>) {\n\tchomp;\n\tmy ($commit,@parents) = split /\\s+/;\n\t$csets{$commit}{parents} = \\@parents;\n\tpush @revs, $commit;\n\n\topen(GCF,\"-|\",\"git-cat-file\",\"commit\",$commit)\n\t\tor die \"Failed to open git-cat-file: \" . $!;\n\n\tmy $in_comments = 0;\n\twhile(<GCF>) {\n\t\tchomp;\n\t\tif ($in_comments) {\n\t\t\t$csets{$commit}{comments} .= $_ . \"\\n\";\n\n\t\t} elsif (m/^tree (.+)$/) {\n\t\t\t$csets{$commit}{tree} = $1;\n\t\t\t#printf(\"tree = %s\\n\",$1);\n\n\t\t} elsif (m/^parent (.+)$/) {\n\t\t\t# Do nothing, we already got\n\t\t\t# the parents from rev-list.\n\n\t\t} elsif (m/^(author|committer) (.*) <(.*)> (.*)$/) {\n\t\t\t#printf(\"%s = %s <%s> at %s\\n\",$1, $2,$3,$4);\n\t\t\t@{$csets{$commit}{$1}}{qw(name email datetime)}\n\t\t\t\t= ($2,$3,$4);\n\n\t\t} elsif (length == 0) {\n\t\t\t$in_comments = 1;\n\t\t\t$csets{$commit}{comments} = \"\";\n\t\t\tnext;\n\t\t}\n\t}\n\tclose(GCF);\n\n}\nclose(GRL);\n\n@revs = reverse @revs;\npush @{$csets{$revs[0]}{parents}},$ARGV[2];\n\nmy %newcsets;\nforeach my $old (@revs) {\n\tprintf(\"Processing commit %s\\n\",$old);\n        $ENV{GIT_AUTHOR_EMAIL} = $csets{$old}{author}{email};\n        $ENV{GIT_AUTHOR_NAME} = $csets{$old}{author}{name};\n        $ENV{GIT_AUTHOR_DATE} = $csets{$old}{author}{datetime};\n        $ENV{GIT_COMMITTER_DATE} = $csets{$old}{committer}{datetime};\n        $ENV{GIT_COMMITTER_EMAIL} = $csets{$old}{committer}{email};\n        $ENV{GIT_COMMITTER_NAME} = $csets{$old}{committer}{name};\n\n\tmy @parents = @{$csets{$old}{parents}};\n\tforeach my $p (@{$csets{$old}{parents}}) {\n\t\tif (exists $newcsets{$p}) {\n\t\t\tpush @parents, $newcsets{$p}\n\t\t\t\tif exists $newcsets{$p};\n\t\t\tprintf(\"Found new csetid %s for %s\\n\",\n\t\t\t\t$newcsets{$p},$p);\n\t\t}\n\t}\n        my $commit = git_commit_tree($csets{$old}{tree},\n\t\t$csets{$old}{comments},@parents);\n\t$newcsets{$old} = $commit;\n        printf(\"Commit for version %s is %s\\n\",$old,$newcsets{$old});\n}\n\n"},{"id":"11226","messageId":"867jblpfcq.fsf@blue.stonehenge.com","threadId":"2372","inReplyTo":"436E85DA.1080904@michonline.com","subject":"Re: [RFC] Applying a graft to a tree and \"rippling\" the changes through the history","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2005-11-06T22:43:17Z","receivedAt":"2005-11-06T22:43:17Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Ryan\" == Ryan Anderson <ryan@michonline.com> writes:\n\nRyan> chdir($ARGV[0]);\n\nThat's dangerous without an \"or-die\".  Being in the wrong directory\nbefore you do a lot of edits is a good way to bust your disk. :)\n\nRyan> \tmy ($commit,@parents) = split /\\s+/;\n\nsplit with no args splits $_ on whitespace, tossing leading whitespace,\njust in case they ever put whitespace indentation ahead.\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\nSee PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"11237","messageId":"7vll01ut6j.fsf@assigned-by-dhcp.cox.net","threadId":"2372","inReplyTo":"436E85DA.1080904@michonline.com","subject":"Re: [RFC] Applying a graft to a tree and \"rippling\" the changes through the history","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-11-07T01:45:40Z","receivedAt":"2005-11-07T01:45:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ryan Anderson <ryan@michonline.com> writes:\n\n> I've written a tool that will take a single commit, add it as a parent\n> of another commit, and recreate the history above that second commit in\n> a fully compatible manner.\n\nI think the procedure is reproducible, which is a very nice\nproperty to have for a tool like this, but I am not sure what\nyou mean by \"in a fully compatible manner\".  What are you\ncompatible with?\n\nAlso another rhetorical, tongue-in-cheek question.  What is your\nplan to ripple the graft through to update signed tags?  ;-)\n"},{"id":"11239","messageId":"436EBA69.6000900@michonline.com","threadId":"2372","inReplyTo":"7vll01ut6j.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Applying a graft to a tree and \"rippling\" the changes through the history","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2005-11-07T02:22:33Z","receivedAt":"2005-11-07T02:22:33Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"Junio C Hamano wrote:\n> Ryan Anderson <ryan@michonline.com> writes:\n> \n> \n>>I've written a tool that will take a single commit, add it as a parent\n>>of another commit, and recreate the history above that second commit in\n>>a fully compatible manner.\n> \n> \n> I think the procedure is reproducible, which is a very nice\n> property to have for a tool like this, but I am not sure what\n> you mean by \"in a fully compatible manner\".  What are you\n> compatible with?\n\nWell, what I meant was, \"It creates a history that is purely a superset\nof the old history, so merges should work cleanly from the pre-graft\nsubhistory to the fully merged history.\"\n\nBut clearly I was too ... terse.\n\nIOW, this should work perfectly, assuming neither tree has been pulled\ninto since the history was merged into historical-graft tree:\n\n$ cd linux-head\n$ git branch -b ryan-hacking HEAD\n$ quilt push -a\n$ git commit -a -m \"Apply quilt tree\"\n\n$ cd ../linux-historical-graft/\n$ git pull ../linux-historical-graft/\n\n> Also another rhetorical, tongue-in-cheek question.  What is your\n> plan to ripple the graft through to update signed tags?  ;-)\n\n:) Well, since I can't resist answering your rhetorical question:\n\nThey signed a specific DAG.  I'm providing a richer, more complete DAG\nthat is a pure-superset of the one they signed.  It is not, however,\nequivalent, so their signature is not related to the superset DAG I have\ncreated.  In practice, however, I don't expect that any tag-signers\nwould state that there is a meaningful difference between the two DAGs,\nfrom the perspective of their signature.\n\nFYI - I don't think merging the trees like this is a good idea, from the\nperspective of something like gitk - gitk took long enough to startup\nand display something on my merged tree here that I gave up and killed\nit off.\n\n"},{"id":"12206","messageId":"pan.2005.11.18.08.49.30.32345@smurf.noris.de","threadId":"2372","inReplyTo":"7vll01ut6j.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Applying a graft to a tree and \"rippling\" the changes through the history","fromName":"Matthias Urlichs","fromEmail":"smurf@smurf.noris.de","sentAt":"2005-11-18T08:49:34Z","receivedAt":"2005-11-18T08:49:34Z","isPatch":false,"sender":{"key":"matthias@urlichs.de","avatar":"https://gravatar.com/avatar/2708905af227313eba6f2b2ae0f7d0259b5ac5d71baef58fe5a13c699ce0bbf0?d=mp&s=160"},"body":"Hi, Junio C Hamano wrote:\n\n> Ryan Anderson <ryan@michonline.com> writes:\n> \n>> I've written a tool that will take a single commit, add it as a parent\n>> of another commit, and recreate the history above that second commit in\n>> a fully compatible manner.\n> \nYou're not the only one. My tool is different, however, in that it accepts\na list of old=>new commits. I have used it successfully to re-graft my\nchanges from a CVS->GIT tree to the corresponding CVS->SVN->GIT tree\n(which cannot be identical, because (a) SVN's timestamps are not accurate\nenough and (b) SVN's CVS import is more accurate in reproducing CVS\narchives than cvsps can be).\n\n> Also another rhetorical, tongue-in-cheek question.  What is your\n> plan to ripple the graft through to update signed tags?  ;-)\n\nI'd suggest adding the capability of grafting something onto a tag...\n\n\n#!/usr/bin/python\n\n# This is a simple script which clones a git subtree to another.\n\nimport sys,re,optparse,os,subprocess\n\nparser = optparse.OptionParser(\"corresponding_file [ old_commit ]\", conflict_handler=\"resolve\", description=\"\"\"\\\nTransfer a list of commits from one repository to another.\n\nThe first argument is a list of SHA1 entries of the form\n\told new\nIt lists which commits are \"the same\".\n\nold_commit and those of its parents which are not listed in the\ncorresponding_file are copied, i.e. new commit objects are created.\nTheir SHA1s are added to the file so that you may repeat the process\nwith other commits, or run it incrementally.\n\n\"\"\")\nparser.add_option(\"-h\",\"--help\",\"-?\", action=\"help\",\n                    help=\"Print this help message and exit\")\nparser.add_option(\"-v\", \"--verbose\", dest=\"verbose\", action=\"store_true\",\n                    help=\"Report progress\")\n\n(options, args) = parser.parse_args()\n\nif len(args) < 1 or len(args) > 3:\n\tparser.error(\"requires one to three arguments\")\n\n\ndef end(p):\n\ttry:\n\t\tretcode = p.wait()\n\texcept OSError,e:\n\t\tprint >>sys.stderr, \"git-rev-list failed:\", e\n\telse:\n\t\tif retcode < 0:\n\t\t\tprint >>sys.stderr, \"git-rev-list was terminated by signal\", -retcode\n\t\t\tsys.exit(1)\n\t\telif retcode > 0:\n\t\t\tprint >>sys.stderr, \"git-rev-list exited with non-zero exit code\", retcode\n\t\t\tsys.exit(1)\n\n\nre_cmt = re.compile(r'\\s*#.*')\ncorr = {}\nfor l in open(args[0]):\n\tl = re_cmt.sub(\"\",l).strip()\n\tif l == \"\": continue\n\ttry:\n\t\ta,b = l.split()\n\texcept ValueError:\n\t\tcontinue\n\tif len(a) != 40: continue\n\tif len(b) != 40: continue\n\tcorr[a]=b\n\ncorrf=open(args[0],\"a\")\n\nif len(args) >= 2:\n\tsrctag = args[1]\nelse:\n\tsrctag = \"HEAD\"\n\nsrcrepo = os.path.curdir\n\ncommits=[]\ncmd = [\"git-rev-list\",srctag]\nfor k in corr.iterkeys():\n\tcmd.append(\"^\"+k)\nif options.verbose:\n\tprint cmd\n\np=subprocess.Popen(cmd, stdout=subprocess.PIPE)\nfor l in p.stdout:\n\tl = l.strip()\n\tcommits.append(l)\nend(p)\n\nwhile len(commits):\n\tc = commits.pop()\n\tif c in corr: continue\n\tif options.verbose:\n\t\tprint \"Processing:\",c\n\n\tp=subprocess.Popen([\"git-cat-file\",\"commit\",c], stdout=subprocess.PIPE)\n\tqf=os.tempnam()\n\ttry:\n\t\tq=open(qf,\"w\")\n\t\tnx=False\n\t\tfor l in p.stdout:\n\t\t\tif nx:\n\t\t\t\tq.write(l)\n\t\t\t\tcontinue\n\t\t\tl = l.strip()\n\t\t\tif l == \"\":\n\t\t\t\tnx=True\n\t\t\t\tq.write(\"\\n\")\n\t\t\t\tcontinue\n\t\t\ta,b = l.split(\" \",1)\n\t\t\tif a == \"parent\":\n\t\t\t\tb = corr[b]\n\t\t\tprint >>q,a,b\n\t\tq.close()\n\t\tq=subprocess.Popen([\"git-hash-object\",\"-w\",\"-t\",\"commit\",qf], stdout=subprocess.PIPE)\n\t\td = q.stdout.read().strip()\n\t\tend(q)\n\tfinally:\n\t\tos.unlink(qf)\n\tend(p)\n\tcorr[c] = d\n\tprint >>corrf, c,d\n\tif options.verbose:\n\t\tprint c,d,l\n\n# OK, everything is done.\ncorrf.close()\nprint d\n\n-- \nMatthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de\nDisclaimer: The quote was selected randomly. Really. | http://smurf.noris.de\n - -\nIt's not what you know or what you do, it's who you know.\n"}]}