{"thread":{"id":"35398","subject":"[PATCH] subtree: add squash handling for split and push","startedAt":"2013-11-23T20:18:56Z","lastAt":"2013-12-10T22:46:38Z","messageCount":8,"participants":["Pierre Penninckx","Matthew Ogilvie","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"231002","messageId":"C3EF61D4-5274-4A10-9594-4E631D75C3F4@gmail.com","threadId":"35398","inReplyTo":null,"subject":"[PATCH] subtree: add squash handling for split and push","fromName":"Pierre Penninckx","fromEmail":"ibizapeanut@gmail.com","sentAt":"2013-11-23T20:18:56Z","receivedAt":"2013-11-23T20:18:56Z","isPatch":true,"sender":{"key":"ibizapeanut@gmail.com","avatar":"https://gravatar.com/avatar/1f166bda099816b4474d218fe080220f5a4c6f11b85c67165d7cb2904cd5fe86?d=mp&s=160"},"body":">From 2763be1fe68d07af60945762178b8494228eb45f Mon Sep 17 00:00:00 2001\nFrom: Pierre Penninckx <ibizapeanut@gmail.com>\nDate: Sat, 23 Nov 2013 20:03:20 +0100\nSubject: [PATCH] subtree: add squash handling for split and push\n\nThe documentation of subtree says that the --squash option can be used\nfor add, merge, split and push subtree commands but only add and merge\nis implemented.\n\ncmd_push() simply forwards the --squash argument to subtree split. All\nthe job is done by cmd_split().\n\ncmd_split() first lets split do it's job: finding which commits need to\nbe extracted. Now we remember which commit is the parent of the first\nextracted commit. When this step is done, cmd_split() generates a squash\nof the new commits, starting from the aforementioned parent to the last\nextracted commit. This new commit's sha1 is then used for the rest of\nthe script.\n\nTests verify that `git subtree split/push --squash` makes only one\ncommit where `git subtree split/push` without squash makes two.\n---\n contrib/subtree/git-subtree.sh     | 20 ++++++++++++++++++-\n contrib/subtree/t/t7900-subtree.sh | 40 ++++++++++++++++++++++++++++++++++++++\n 2 files changed, 59 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh\nindex 7d7af03..76eb136 100755\n--- a/contrib/subtree/git-subtree.sh\n+++ b/contrib/subtree/git-subtree.sh\n@@ -183,6 +183,7 @@ cache_set()\n \tnewrev=\"$2\"\n \tif [ \"$oldrev\" != \"latest_old\" \\\n \t     -a \"$oldrev\" != \"latest_new\" \\\n+\t     -a \"$oldrev\" != \"firstnewparents\" \\\n \t     -a -e \"$cachedir/$oldrev\" ]; then\n \t\tdie \"cache for $oldrev already exists!\"\n \tfi\n@@ -603,6 +604,10 @@ cmd_split()\n \t\tdebug \"  parents: $parents\"\n \t\tnewparents=$(cache_get $parents)\n \t\tdebug \"  newparents: $newparents\"\n+\t\tif [ -z \"$(cache_get firstnewparents)\" ]; then\n+\t\t\tcache_set firstnewparents $newparents\n+\t\t\tdebug \"  firstnewparents: $(cache_get firstnewparents)\"\n+\t\tfi\n\n \t\ttree=$(subtree_for_commit $rev \"$dir\")\n \t\tdebug \"  tree is: $tree\"\n@@ -625,11 +630,18 @@ cmd_split()\n \t\tcache_set latest_new $newrev\n \t\tcache_set latest_old $rev\n \tdone || exit $?\n+\n \tlatest_new=$(cache_get latest_new)\n \tif [ -z \"$latest_new\" ]; then\n \t\tdie \"No new revisions were found\"\n \tfi\n\n+\tif [ -n \"$squash\" ]; then\n+\t\tfrom=$(cache_get firstnewparents)\n+\t\tlatest_new=$(new_squash_commit \"$from\" \"$from\" \"$latest_new\") || exit $?\n+\t\tdebug \"New squash commit: $latest_new\"\n+\tfi\n+\n \tif [ -n \"$rejoin\" ]; then\n \t\tdebug \"Merging split branch into HEAD...\"\n \t\tlatest_old=$(cache_get latest_old)\n@@ -711,11 +723,17 @@ cmd_push()\n \tif [ $# -ne 2 ]; then\n \t    die \"You must provide <repository> <refspec>\"\n \tfi\n+\n+\tsquash_cmd=\n+\tif [ -n \"$squash\" ]; then\n+\t    squash_cmd=\"--squash\"\n+\tfi\n+\n \tif [ -e \"$dir\" ]; then\n \t    repository=$1\n \t    refspec=$2\n \t    echo \"git push using: \" $repository $refspec\n-\t    localrev=$(git subtree split --prefix=\"$prefix\") || die\n+\t    localrev=$(git subtree split --prefix=\"$prefix\" $squash_cmd) || die\n \t    git push $repository $localrev:refs/heads/$refspec\n \telse\n \t    die \"'$dir' must already exist. Try 'git subtree add'.\"\ndiff --git a/contrib/subtree/t/t7900-subtree.sh b/contrib/subtree/t/t7900-subtree.sh\nindex 66ce4b0..04eea94 100755\n--- a/contrib/subtree/t/t7900-subtree.sh\n+++ b/contrib/subtree/t/t7900-subtree.sh\n@@ -31,6 +31,16 @@ check_equal()\n \tfi\n }\n\n+check_not_equal()\n+{\n+        check_equal \"$1\" \"$2\"\n+        if [ $? -eq 0 ]; then\n+                return 1\n+        else\n+                return 0\n+        fi\n+}\n+\n fixnl()\n {\n \tt=\"\"\n@@ -161,6 +171,36 @@ test_expect_success 'fetch new subproj history' '\n         git branch sub2 FETCH_HEAD\n '\n\n+test_expect_success 'check that split and split --squash produce different commits' '\n+        split=$(git subtree split --prefix=subdir) &&\n+        split_squash=$(git subtree split --prefix=subdir --squash) &&\n+        check_not_equal \"$split\" \"$split_squash\"\n+'\n+\n+test_expect_success 'check that split and split --squash produce same diff' '\n+        split=$(git subtree split --prefix=subdir) &&\n+        split_squash=$(git subtree split --prefix=subdir --squash) &&\n+        split_diff=$(git diff sub1 $split) &&\n+        split_squash_diff=$(git diff sub1 $split_squash) &&\n+        check_equal \"$split_diff\" \"$split_squash_diff\"\n+'\n+\n+test_expect_success 'check that push introduces two commits in subproj' '\n+        git subtree push --prefix=subdir subproj mainline &&\n+        cd subproj &&\n+        check_equal \"$(git rev-parse mainline~2)\" \"$(git rev-parse sub1)\" &&\n+        git branch -D mainline &&\n+        cd ..\n+'\n+\n+test_expect_success 'check that push --squash introduces only one commit in subproj' '\n+        git subtree push --prefix=subdir subproj mainline --squash &&\n+        cd subproj &&\n+        check_equal \"$(git rev-parse mainline^)\" \"$(git rev-parse sub1)\" &&\n+        git branch -D mainline &&\n+        cd ..\n+'\n+\n test_expect_success 'check if --message works for merge' '\n         git subtree merge --prefix=subdir -m \"Merged changes from subproject\" sub2 &&\n         check_equal ''\"$(last_commit_message)\"'' \"Merged changes from subproject\" &&\n"},{"id":"231250","messageId":"20131128182309.GA3655@comcast.net","threadId":"35398","inReplyTo":"C3EF61D4-5274-4A10-9594-4E631D75C3F4@gmail.com","subject":"Re: [PATCH] subtree: add squash handling for split and push","fromName":"Matthew Ogilvie","fromEmail":"mmogilvi_git@miniinfo.net","sentAt":"2013-11-28T18:23:09Z","receivedAt":"2013-11-28T18:23:09Z","isPatch":true,"sender":{"key":"mmogilvi_git@miniinfo.net","avatar":null},"body":"On Sat, Nov 23, 2013 at 09:18:56PM +0100, Pierre Penninckx wrote:\n> The documentation of subtree says that the --squash option can be used\n> for add, merge, split and push subtree commands but only add and merge\n> is implemented.\n\nClarification: The current documentation (correctly) doesn't\nactually claim to support \"split --squash\", but it does erroneously\nclaim to support \"push --squash\".\n\n> cmd_split() first lets split do it's job: finding which commits need to\n> be extracted. Now we remember which commit is the parent of the first\n> extracted commit. When this step is done, cmd_split() generates a squash\n> of the new commits, starting from the aforementioned parent to the last\n> extracted commit. This new commit's sha1 is then used for the rest of\n> the script.\n\nI've been planning to implement something similar to this patch,\nbut the semantics I am aiming at are slightly different.\n\nIt looks like your patch is basically squashing the new subtree commits\ntogether, throwing out those commits completely, and only keeping\nthe squashed commit in the split --branch.  \n\nI intend to implement slightly different semantics, where\n--squash only affects --rejoin, not the printed commit nor\nthe split-off --branch.  This is intended to provide a better,\nthird option for --rejoin'ing a subtree with a lot of history,\nwhile preserving history in the split-off branch:\n\n1. (existing/slow) Don't ever use --rejoin at all?  You can use\n   \"merge --squash\" to merge in unrelated changes to the\n   split-off project, but every \"split\" still gets slower\n   and slower as each \"split\" needs to re-sift-through all\n   the same history the previous \"split\"s have sifted\n   through. \n   \n2. (existing/huge mass of duplicated history) Use \"split --rejoin\"\n   occasionally.  This pulls in the entire history of the\n   subtree branch (since the last --rejoin or non-squash merge,\n   or everything if neither has been done), which is difficult\n   to ignore when looking at global history of the full project,\n   especially if it is many pages of commits.  But subsequent\n   splits can stop history traversal at the known-common point,\n   and will run MUCH faster.\n   \n3. (new/better) Use \"split --rejoin --squash\" (or some other\n   invocation to be defined).  The subtree branch is generated\n   exactly like normal, including fine-grained history.  But\n   instead of merging the subtree branch directly, --rejoin\n   will squash all the changes to that branch, and merge in\n   just the squash (referencing the unsquashed split\n   branch tip in the commit message, but not the\n   parent).  Subsequent splits can run very fast, while the\n   \"--rejoin\" only generated two commits instead of the \n   potentially thousands of (mostly) duplicates it would pull\n   in without the \"--squash\".\n\nI have this third option half-coded already, but I still need\nto finish it.\n\nI'm fairly sure I can make this work without new adverse effects,\nbut if someone sees something I'm missing, let me know.\n\nDoes anyone have any suggestions about the UI?  Do we need to also\nsupport Pierre Penninckx's \"split --squash\" semantics somehow?  If\nso, what command line options would allow for distinguishing the\ntwo cases?\n\n--\nMatthew Ogilvie   [mmogilvi_git@miniinfo.net]\n"},{"id":"231267","messageId":"1B2468D5-F0B7-4D11-8710-A3A7C8D66215@gmail.com","threadId":"35398","inReplyTo":"20131128182309.GA3655@comcast.net","subject":"Re: [PATCH] subtree: add squash handling for split and push","fromName":"Pierre Penninckx","fromEmail":"ibizapeanut@gmail.com","sentAt":"2013-11-28T22:58:23Z","receivedAt":"2013-11-28T22:58:23Z","isPatch":true,"sender":{"key":"ibizapeanut@gmail.com","avatar":"https://gravatar.com/avatar/1f166bda099816b4474d218fe080220f5a4c6f11b85c67165d7cb2904cd5fe86?d=mp&s=160"},"body":"Hi Matthew,\n\n> Clarification: The current documentation (correctly) doesn't\n> actually claim to support \"split --squash\", but it does erroneously\n> claim to support \"push --squash ».\n\nYes indeed. ;)\n\n> It looks like your patch is basically squashing the new subtree commits\n> together, throwing out those commits completely, and only keeping\n> the squashed commit in the split —branch.\n\nExactly.\n\n> 3. (new/better) Use \"split --rejoin --squash\" (or some other\n>   invocation to be defined).  The subtree branch is generated\n>   exactly like normal, including fine-grained history.  But\n>   instead of merging the subtree branch directly, --rejoin\n>   will squash all the changes to that branch, and merge in\n>   just the squash (referencing the unsquashed split\n>   branch tip in the commit message, but not the\n>   parent).  Subsequent splits can run very fast, while the\n>   \"--rejoin\" only generated two commits instead of the \n>   potentially thousands of (mostly) duplicates it would pull\n>   in without the \"--squash ».\n\nIsn’t this similar to \"my\" way? I mean I too generate the fine-grained history and make a squash afterwards, no?\nI also don’t get why would your solution generate any duplicates. Would mine generate some?\nI suppose the two answers are linked.\n\n> I have this third option half-coded already, but I still need\n> to finish it.\n\nI’m eager to test it!\n\n> Does anyone have any suggestions about the UI?  Do we need to also\n> support Pierre Penninckx's \"split --squash\" semantics somehow?  If\n> so, what command line options would allow for distinguishing the\n> two cases?\n\nMaybe `split --rejoin-squash` since it’s really a third way?\nI intended to use `push --squash` to send a squash of the commits to hide the actual tinkering. So if your way allows to do it, I vote to stick with yours.\n\nRegards,\nPierre Penninckx"},{"id":"231731","messageId":"1386440485-3092-1-git-send-email-mmogilvi_git@miniinfo.net","threadId":"35398","inReplyTo":"1B2468D5-F0B7-4D11-8710-A3A7C8D66215@gmail.com","subject":"[PATCH 1/4] subtree: support split --rejoin --squash","fromName":"Matthew Ogilvie","fromEmail":"mmogilvi_git@miniinfo.net","sentAt":"2013-12-07T18:21:22Z","receivedAt":"2013-12-07T18:21:22Z","isPatch":true,"sender":{"key":"mmogilvi_git@miniinfo.net","avatar":null},"body":"Allow using --squash with \"git subtree split --rejoin\".  It\nwill still split off (and save to --branch) the complete\nsubtree history, but the merge done for the \"--rejoin\" will\nbe merging a squashed representation of the new subtree\ncommits, instead of the commits themselves (similar to\nhow \"git subtree merge --squash\" works).\n\nSigned-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>\n---\n\nI can think of a couple of possible objections to this patch.\nAre these (or any others) worth fixing?\n\n1. Perhaps someone want the saved subtree (--branch) to have\n   a squashed representation as well, as an option?  Maybe we\n   need two different --squash options?  Something\n   like \"--rejoin-squash\"?\n2. It could definitely use some automated tests.  In fact,\n   pre-existing --squash functionality is hardly tested at\n   all, either.\n      See patch 4 comments for a script I use to help with\n   mostly-manual testing.\n\n\n\n contrib/subtree/git-subtree.sh  | 60 +++++++++++++++++++++++++++++++----------\n contrib/subtree/git-subtree.txt | 27 ++++++++++++-------\n 2 files changed, 63 insertions(+), 24 deletions(-)\n\ndiff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh\nindex 7d7af03..998a9c5 100755\n--- a/contrib/subtree/git-subtree.sh\n+++ b/contrib/subtree/git-subtree.sh\n@@ -20,14 +20,13 @@ q             quiet\n d             show debug messages\n P,prefix=     the name of the subdir to split out\n m,message=    use the given message as the commit message for the merge commit\n+squash        merge subtree changes as a single commit\n  options for 'split'\n annotate=     add a prefix to commit message of new commits\n b,branch=     create a new branch from the split subtree\n ignore-joins  ignore prior --rejoin commits\n onto=         try connecting new tree to an existing one\n rejoin        merge the new branch back into HEAD\n- options for 'add', 'merge', 'pull' and 'push'\n-squash        merge subtree changes as a single commit\n \"\n eval \"$(echo \"$OPTS_SPEC\" | git rev-parse --parseopt -- \"$@\" || echo exit $?)\"\n \n@@ -229,13 +228,19 @@ find_latest_squash()\n \tsq=\n \tmain=\n \tsub=\n+\tpar1=\n+\tpar2=\n \tgit log --grep=\"^git-subtree-dir: $dir/*\\$\" \\\n-\t\t--pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |\n-\twhile read a b junk; do\n-\t\tdebug \"$a $b $junk\"\n+\t\t--pretty=format:'START %H %P%n%s%n%n%b%nEND%n' HEAD |\n+\twhile read a b c d junk; do\n+\t\tdebug \"$a $b $c $d $junk\"\n \t\tdebug \"{{$sq/$main/$sub}}\"\n \t\tcase \"$a\" in\n-\t\t\tSTART) sq=\"$b\" ;;\n+\t\t\tSTART)\n+\t\t\t\tsq=\"$b\"\n+\t\t\t\tpar1=\"$c\"\n+\t\t\t\tpar2=\"$d\"\n+\t\t\t\t;;\n \t\t\tgit-subtree-mainline:) main=\"$b\" ;;\n \t\t\tgit-subtree-split:) sub=\"$b\" ;;\n \t\t\tEND)\n@@ -243,7 +248,8 @@ find_latest_squash()\n \t\t\t\t\tif [ -n \"$main\" ]; then\n \t\t\t\t\t\t# a rejoin commit?\n \t\t\t\t\t\t# Pretend its sub was a squash.\n-\t\t\t\t\t\tsq=\"$sub\"\n+\t\t\t\t\t\tassert [ \"$main\" = \"$par1\" ]\n+\t\t\t\t\t\tsq=\"$par2\"\n \t\t\t\t\tfi\n \t\t\t\t\tdebug \"Squash found: $sq $sub\"\n \t\t\t\t\techo \"$sq\" \"$sub\"\n@@ -252,6 +258,8 @@ find_latest_squash()\n \t\t\t\tsq=\n \t\t\t\tmain=\n \t\t\t\tsub=\n+\t\t\t\tpar1=\n+\t\t\t\tpar2=\n \t\t\t\t;;\n \t\tesac\n \tdone\n@@ -565,6 +573,13 @@ cmd_split()\n \tdebug \"Splitting $dir...\"\n \tcache_setup || exit $?\n \t\n+\tif [ -n \"$rejoin\" ]; then\n+\t\tensure_clean\n+\t\tif [ -n \"$squash\" ]; then\n+\t\t\tfirst_split=\"$(find_latest_squash \"$dir\")\"\n+\t\tfi\n+\tfi\n+\n \tif [ -n \"$onto\" ]; then\n \t\tdebug \"Reading history for --onto=$onto...\"\n \t\tgit rev-list $onto |\n@@ -630,13 +645,6 @@ cmd_split()\n \t\tdie \"No new revisions were found\"\n \tfi\n \t\n-\tif [ -n \"$rejoin\" ]; then\n-\t\tdebug \"Merging split branch into HEAD...\"\n-\t\tlatest_old=$(cache_get latest_old)\n-\t\tgit merge -s ours \\\n-\t\t\t-m \"$(rejoin_msg $dir $latest_old $latest_new)\" \\\n-\t\t\t$latest_new >&2 || exit $?\n-\tfi\n \tif [ -n \"$branch\" ]; then\n \t\tif rev_exists \"refs/heads/$branch\"; then\n \t\t\tif ! rev_is_descendant_of_branch $latest_new $branch; then\n@@ -649,6 +657,30 @@ cmd_split()\n \t\tgit update-ref -m 'subtree split' \"refs/heads/$branch\" $latest_new || exit $?\n \t\tsay \"$action branch '$branch'\"\n \tfi\n+\tif [ -n \"$rejoin\" ]; then\n+\t\tdebug \"Merging split branch into HEAD...\"\n+\t\tlatest_old=$(cache_get latest_old)\n+\t\tnew=$latest_new\n+\n+\t\tif [ -n \"$squash\" ]; then\n+\t\t\tdebug \"Squashing split branch.\"\n+\n+\t\t\tset $first_split\n+\t\t\told=$1\n+\t\t\tsub=$2\n+\t\t\tif [ \"$sub\" = \"$latest_new\" ]; then\n+\t\t\t\tsay \"Subtree is already at commit $latest_new.\"\n+\t\t\t\texit 0\n+\t\t\tfi\n+\t\t\tnew=$(new_squash_commit \"$old\" \"$sub\" \"$latest_new\") \\\n+\t\t\t\t|| exit $?\n+\t\t\tdebug \"New squash commit: $new\"\n+\t\tfi\n+\n+\t\tgit merge -s ours -m \\\n+\t\t\t\"$(rejoin_msg $dir $latest_old $latest_new)\" \\\n+\t\t\t$new >&2 || exit $?\n+\tfi\n \techo $latest_new\n \texit 0\n }\ndiff --git a/contrib/subtree/git-subtree.txt b/contrib/subtree/git-subtree.txt\nindex e0957ee..92e7a4d 100644\n--- a/contrib/subtree/git-subtree.txt\n+++ b/contrib/subtree/git-subtree.txt\n@@ -140,18 +140,20 @@ OPTIONS\n \twant to manipulate.  This option is mandatory\n \tfor all commands.\n \n+\n+OPTIONS FOR add, merge, pull, rejoin\n+----------------------------------\n -m <message>::\n --message=<message>::\n-\tThis option is only valid for add, merge and pull (unsure).\n-\tSpecify <message> as the commit message for the merge commit.\n+\tThis option is only valid for add, merge, pull, and\n+\tsplit '--rejoin'.\n \n+\tSpecify <message> as the commit message for the merge commit.\n \n-OPTIONS FOR add, merge, push, pull\n-----------------------------------\n --squash::\n-\tThis option is only valid for add, merge, push and pull\n-\tcommands.\n-\n+\tThis option is only valid for add, merge, pull, and\n+\tsplit '--rejoin'.\n+\n \tInstead of merging the entire history from the subtree\n \tproject, produce only a single commit that contains all\n \tthe differences you want to merge, and then merge that\n@@ -180,6 +182,10 @@ OPTIONS FOR add, merge, push, pull\n \tlocal repository remain intact and can be later split\n \tand send upstream to the subproject.\n \n+\tUsing '--squash' with split '--rejoin' only squashes\n+\tthe merge back to the mainline, not the synthetic subtree\n+\thistory.\n+\n \n OPTIONS FOR split\n -----------------\n@@ -251,9 +257,10 @@ OPTIONS FOR split\n \tshowing an extra copy of every new commit that was\n \tcreated (the original, and the synthetic one).\n \t\n-\tIf you do all your merges with '--squash', don't use\n-\t'--rejoin' when you split, because you don't want the\n-\tsubproject's history to be part of your project anyway.\n+\tFortunately, you can use '--squash' with '--rejoin'\n+\tto simplify a sequence of synthetic commits as a\n+\tsingle squashed commit in the mainline.  The subtree\n+\twill still have full history.\n \n \n EXAMPLE 1. Add command\n-- \n1.8.3.2\n"},{"id":"231732","messageId":"1386440485-3092-2-git-send-email-mmogilvi_git@miniinfo.net","threadId":"35398","inReplyTo":"1386440485-3092-1-git-send-email-mmogilvi_git@miniinfo.net","subject":"[PATCH 2/4] subtree: allow --squash and --message with push","fromName":"Matthew Ogilvie","fromEmail":"mmogilvi_git@miniinfo.net","sentAt":"2013-12-07T18:21:23Z","receivedAt":"2013-12-07T18:21:23Z","isPatch":true,"sender":{"key":"mmogilvi_git@miniinfo.net","avatar":null},"body":"Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>\n---\n contrib/subtree/git-subtree.sh  | 8 +++++++-\n contrib/subtree/git-subtree.txt | 9 ---------\n 2 files changed, 7 insertions(+), 10 deletions(-)\n\ndiff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh\nindex 998a9c5..56d915f 100755\n--- a/contrib/subtree/git-subtree.sh\n+++ b/contrib/subtree/git-subtree.sh\n@@ -743,11 +743,17 @@ cmd_push()\n \tif [ $# -ne 2 ]; then\n \t    die \"You must provide <repository> <refspec>\"\n \tfi\n+\n+\topts=\n+\tif [ -n \"$squash\" ]; then\n+\t\topts=\"-squash\"\n+\tfi\n+\n \tif [ -e \"$dir\" ]; then\n \t    repository=$1\n \t    refspec=$2\n \t    echo \"git push using: \" $repository $refspec\n-\t    localrev=$(git subtree split --prefix=\"$prefix\") || die\n+\t    localrev=$(git subtree split --prefix=\"$prefix\" $opts --message=\"$message\") || die\n \t    git push $repository $localrev:refs/heads/$refspec\n \telse\n \t    die \"'$dir' must already exist. Try 'git subtree add'.\"\ndiff --git a/contrib/subtree/git-subtree.txt b/contrib/subtree/git-subtree.txt\nindex 92e7a4d..03092bc 100644\n--- a/contrib/subtree/git-subtree.txt\n+++ b/contrib/subtree/git-subtree.txt\n@@ -140,20 +140,11 @@ OPTIONS\n \twant to manipulate.  This option is mandatory\n \tfor all commands.\n \n-\n-OPTIONS FOR add, merge, pull, rejoin\n-----------------------------------\n -m <message>::\n --message=<message>::\n-\tThis option is only valid for add, merge, pull, and\n-\tsplit '--rejoin'.\n-\n \tSpecify <message> as the commit message for the merge commit.\n \n --squash::\n-\tThis option is only valid for add, merge, pull, and\n-\tsplit '--rejoin'.\n-\n \tInstead of merging the entire history from the subtree\n \tproject, produce only a single commit that contains all\n \tthe differences you want to merge, and then merge that\n-- \n1.8.3.2\n"},{"id":"231733","messageId":"1386440485-3092-3-git-send-email-mmogilvi_git@miniinfo.net","threadId":"35398","inReplyTo":"1386440485-3092-1-git-send-email-mmogilvi_git@miniinfo.net","subject":"[PATCH 3/4] subtree: add --edit option","fromName":"Matthew Ogilvie","fromEmail":"mmogilvi_git@miniinfo.net","sentAt":"2013-12-07T18:21:24Z","receivedAt":"2013-12-07T18:21:24Z","isPatch":true,"sender":{"key":"mmogilvi_git@miniinfo.net","avatar":null},"body":"Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>\n---\n contrib/subtree/git-subtree.sh  | 37 +++++++++++++++++++++++++++++--------\n contrib/subtree/git-subtree.txt |  4 ++++\n 2 files changed, 33 insertions(+), 8 deletions(-)\n\ndiff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh\nindex 56d915f..ac82b4d 100755\n--- a/contrib/subtree/git-subtree.sh\n+++ b/contrib/subtree/git-subtree.sh\n@@ -21,6 +21,7 @@ d             show debug messages\n P,prefix=     the name of the subdir to split out\n m,message=    use the given message as the commit message for the merge commit\n squash        merge subtree changes as a single commit\n+edit          allow user to edit squash commit message interactively\n  options for 'split'\n annotate=     add a prefix to commit message of new commits\n b,branch=     create a new branch from the split subtree\n@@ -45,6 +46,7 @@ ignore_joins=\n annotate=\n squash=\n message=\n+edit=\n \n debug()\n {\n@@ -91,6 +93,7 @@ while [ $# -gt 0 ]; do\n \t\t--ignore-joins) ignore_joins=1 ;;\n \t\t--no-ignore-joins) ignore_joins= ;;\n \t\t--squash) squash=1 ;;\n+\t\t--edit) edit=1 ;;\n \t\t--no-squash) squash= ;;\n \t\t--) break ;;\n \t\t*) die \"Unexpected option: $opt\" ;;\n@@ -434,13 +437,12 @@ new_squash_commit()\n \told=\"$1\"\n \toldsub=\"$2\"\n \tnewsub=\"$3\"\n+\tmsg_file=\"$4\"\n \ttree=$(toptree_for_commit $newsub) || exit $?\n \tif [ -n \"$old\" ]; then\n-\t\tsquash_msg \"$dir\" \"$oldsub\" \"$newsub\" | \n-\t\t\tgit commit-tree \"$tree\" -p \"$old\" || exit $?\n+\t\tgit commit-tree \"$tree\" -p \"$old\" -F \"$msg_file\" || exit $?\n \telse\n-\t\tsquash_msg \"$dir\" \"\" \"$newsub\" |\n-\t\t\tgit commit-tree \"$tree\" || exit $?\n+\t\tgit commit-tree \"$tree\" -F \"$msg_file\" || exit $?\n \tfi\n }\n \n@@ -556,7 +558,13 @@ cmd_add_commit()\n \tfi\n \t\n \tif [ -n \"$squash\" ]; then\n-\t\trev=$(new_squash_commit \"\" \"\" \"$rev\") || exit $?\n+\t\tmsg_file=\"$GIT_DIR/COMMIT_EDITMSG\"\n+\t\tsquash_msg \"$dir\" \"\" \"$rev\" >\"$msg_file\"\n+\t\tif [ -n \"$edit\" ]; then\n+\t\t\tgit_editor \"$msg_file\"\n+\t\tfi\n+\t\trev=$(new_squash_commit \"\" \"\" \"$rev\" \"$msg_file\") || exit $?\n+\t\trm -f \"$msg_file\"\n \t\tcommit=$(add_squashed_msg \"$rev\" \"$dir\" |\n \t\t\t git commit-tree $tree $headp -p \"$rev\") || exit $?\n \telse\n@@ -672,8 +680,14 @@ cmd_split()\n \t\t\t\tsay \"Subtree is already at commit $latest_new.\"\n \t\t\t\texit 0\n \t\t\tfi\n-\t\t\tnew=$(new_squash_commit \"$old\" \"$sub\" \"$latest_new\") \\\n-\t\t\t\t|| exit $?\n+\t\t\tmsg_file=\"$GIT_DIR/COMMIT_EDITMSG\"\n+\t\t\tsquash_msg \"$dir\" \"$sub\" \"$latest_new\" >\"$msg_file\"\n+\t\t\tif [ -n \"$edit\" ]; then\n+\t\t\t\tgit_editor \"$msg_file\"\n+\t\t\tfi\n+\t\t\tnew=$(new_squash_commit \"$old\" \"$sub\" \"$latest_new\" \\\n+\t\t\t\t\t\t\"$msg_file\") || exit $?\n+\t\t\trm -f \"$msg_file\"\n \t\t\tdebug \"New squash commit: $new\"\n \t\tfi\n \n@@ -708,7 +722,13 @@ cmd_merge()\n \t\t\tsay \"Subtree is already at commit $rev.\"\n \t\t\texit 0\n \t\tfi\n-\t\tnew=$(new_squash_commit \"$old\" \"$sub\" \"$rev\") || exit $?\n+\t\tmsg_file=\"$GIT_DIR/COMMIT_EDITMSG\"\n+\t\tsquash_msg \"$dir\" \"$sub\" \"$rev\" >\"$msg_file\"\n+\t\tif [ -n \"$edit\" ]; then\n+\t\t\tgit_editor \"$msg_file\"\n+\t\tfi\n+\t\tnew=$(new_squash_commit \"$old\" \"$sub\" \"$rev\" \"$msg_file\") || exit $?\n+\t\trm -f \"$msg_file\"\n \t\tdebug \"New squash commit: $new\"\n \t\trev=\"$new\"\n \tfi\n@@ -748,6 +768,7 @@ cmd_push()\n \tif [ -n \"$squash\" ]; then\n \t\topts=\"-squash\"\n \tfi\n+\t# Can't easily pass on --edit because of stdout capture redirection\n \n \tif [ -e \"$dir\" ]; then\n \t    repository=$1\ndiff --git a/contrib/subtree/git-subtree.txt b/contrib/subtree/git-subtree.txt\nindex 03092bc..16525d4 100644\n--- a/contrib/subtree/git-subtree.txt\n+++ b/contrib/subtree/git-subtree.txt\n@@ -177,6 +177,10 @@ OPTIONS\n \tthe merge back to the mainline, not the synthetic subtree\n \thistory.\n \n+--edit::\n+\tWhen used with '--squash', bring up an editor on the squash\n+\tcommit message, to allow customizing it.\n+\n \n OPTIONS FOR split\n -----------------\n-- \n1.8.3.2\n"},{"id":"231734","messageId":"1386440485-3092-4-git-send-email-mmogilvi_git@miniinfo.net","threadId":"35398","inReplyTo":"1386440485-3092-1-git-send-email-mmogilvi_git@miniinfo.net","subject":"[PATCH/BAD 4/4] subtree: poor bugfix for split new commits with parents before previous split","fromName":"Matthew Ogilvie","fromEmail":"mmogilvi_git@miniinfo.net","sentAt":"2013-12-07T18:21:25Z","receivedAt":"2013-12-07T18:21:25Z","isPatch":true,"sender":{"key":"mmogilvi_git@miniinfo.net","avatar":null},"body":"Bug description: Unless you use --ignore-joins, \"git subtree split\"'s\noptimization to avoid re-scanning all of history can trim too much.\nAny new merged branches that have parents before the previous \"split\"\nwill not be re-attached properly in the split-off subtree.\nIn the extreme case (if all the changes occurred on such\nbranches), I've seen the old subtree head might not be an\nancestor of the new head at all.\n\nThis \"fix\" is only for illustration.  It probably should not be\nincluded as-is; it probably scales worse than using --ignore-joins\nfor anything much beyond trivial cases.\n\nI'm not sure it is possible to speed it up much (while remaining\ncorrect) without switching to a better language than shell script.\n\nI'm not sure how to explain the constraint clearly (or even\nprecisely define the \"minimal\" constraint).  One attempt:\nExclude from \"unrevs\" any rejoin commit X for which ANY new commit\n(not just revs itself) has some other common ancestor (merge base)\nbesides X.  (\"New commit\" is one that is not an ancestor of\nsome previous rejoin commit.)\n\nIt would probably be fairly straightforward to adapt\ngraph traversal algorithms to do this efficiently, but as near\nas I can tell, none of the existing options in git-rev-list or\ngit-merge-base really does anything useful for optimizing this\nin shell script...\n\nThe simplest traversal technique to fix this might\nbe if there was a way to make history traversal only stop\nat SPECIFIC unrevs, instead of any ancestor of any unrevs.\n\nOther workarounds besides this patch:\n   * Use --ignore-joins and wait for the really slow processing...\n   * Plan ahead and delay: After using git subtree split --rejoin,\n     leave the rejoin merge commit off on a side branch until such time\n     that all other pre-split branches have been merged.  Then\n     \"merge the merge\" at that later time.  Only do rejoins fairly\n     rarely, based on the schedule of merging other branches.\n   * Ignore the problem: Allow the occasional falsely-disconnected\n     branch roots to be generated by split.  Of course, this\n     means --ignore-joins would no longer generate a consistent\n     subtree history, it is hard to examine what actually changed\n     in the false roots, etc.\n   * Perhaps manually use temporary grafts or similar to hide rejoins\n     that would throw it off when doing later splits.\n\n[Intentionally not signed off: Poor performance.]\n---\n\nTesting: Below I include a script I've been using to help with\nmostly-manual testing.  I regularly modify it based on\nwhat I'm testing at the moment.  It basically\ncreates and manipulates a subtree out of git's own \"contrib\"\ndirectory.\n\nIt may be convenient to use this on a copy of git's\nrepository instead of the copy in which you are messing with\ngit-subtree (avoiding changing code out from under the script).\nIt may also be convenient to symlink git-subtree to the top-level\ndirectory, rather than copy it, so you don't need to keep\nre-copying it.\n\n------CUT------\n#!/bin/sh\n\ndie()\n{ echo \"$@\" 1>&2\n  exit 1\n}\n\nGIT=../git-sandbox/bin-wrappers/git\n#GIT=git\n\nDBG=\n#DBG=-d\n\nEDIT=\n#EDIT=--edit\n\ngit branch -D br-contrib\n\ngit checkout 73bbc0796b4ce65bfb1a12b47a0edc27845ecf50 || die \"checkout\"\n\n\"$GIT\" subtree split -P contrib \\\n         --branch br-contrib --squash --rejoin $EDIT || die \"subtree 1\"\n\ngit tag -f step1\n\ngit merge -m 'MY MERGE1' 68a65f5fe54c2b21bfe16ef3a0b48956ecf5658a ||\n        die \"merge 1\"\n\n\"$GIT\" subtree split -P contrib \\\n         --branch br-contrib --squash --rejoin || die \"subtree 2\"\n\ngit tag -f step2\n\ngit merge -m 'MY MERGE2' 15f7221686eac053902b906c278680b485c865ce || \\\n        die \"merge 2\"\n\n\"$GIT\" subtree split -P contrib \\\n         --branch br-contrib --squash --rejoin $EDIT || die \"subtree 3\"\n\n#####\nif [ -n \"$EDIT\" ]; then\n    exit 0\nfi\n\ngit tag -f step3\n\ngit merge -m 'MY MERGE3' 26145c9c73ed51bbd8261949d31899d6507519d5 || \\\n        die \"merge 3\"\n\n\"$GIT\" subtree split -P contrib \\\n         $DBG --branch br-contrib --squash --rejoin || die \"subtree 4\"\n\ngit tag -f step4\n\ngit merge -m 'MY MERGE4' 583736c0bcf09adaa5621b142d8e43c22354041b || \\\n        die \"merge 4\"\n\n\"$GIT\" subtree split -P contrib \\\n         --branch br-contrib --squash --rejoin || die \"subtree 5\"\n\ngit tag -f step5\n------CUT------\n\n\n\n contrib/subtree/git-subtree.sh | 61 +++++++++++++++++++++++++++++++++++++-----\n 1 file changed, 55 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh\nindex ac82b4d..6ff4362 100755\n--- a/contrib/subtree/git-subtree.sh\n+++ b/contrib/subtree/git-subtree.sh\n@@ -36,6 +36,8 @@ PATH=$PATH:$(git --exec-path)\n \n require_work_tree\n \n+nl='\n+'\n quiet=\n branch=\n debug=\n@@ -224,6 +226,13 @@ try_remove_previous()\n \tfi\n }\n \n+try_remove()\n+{\n+\tif rev_exists \"$1\"; then\n+\t\techo \"$1\"\n+\tfi\n+}\n+\n find_latest_squash()\n {\n \tdebug \"Looking for latest squash ($dir)...\"\n@@ -293,8 +302,8 @@ find_existing_splits()\n \t\t\t\t\tdebug \"  Prior: $main -> $sub\"\n \t\t\t\t\tcache_set $main $sub\n \t\t\t\t\tcache_set $sub $sub\n-\t\t\t\t\ttry_remove_previous \"$main\"\n-\t\t\t\t\ttry_remove_previous \"$sub\"\n+\t\t\t\t\ttry_remove \"$main\"\n+\t\t\t\t\ttry_remove \"$sub\"\n \t\t\t\tfi\n \t\t\t\tmain=\n \t\t\t\tsub=\n@@ -303,6 +312,40 @@ find_existing_splits()\n \tdone\n }\n \n+filter_out_needed_existing()\n+{\n+\trevs=\"$1\"\n+\texisting=\"$2\"\n+\tbase=\n+\tgit rev-list --merges --parents $revs --not $existing | \\\n+\t\tsed -e 's/^[^ ]* //' -e \"y/ /\\\\$nl/\" | sort | uniq | \\\n+\t{\n+\t\tdebug \"filter_out_needed_existing\"\n+\t\tdebug \"  revs: $revs\"\n+\t\tdebug \"  existing: $existing\"\n+\t\twhile read needed; do\n+\t\t\tdebug \"loop: $needed\"\n+\t\t\tnextexisting=\n+\t\t\tfor exist in $existing ; do\n+\t\t\t\ttmp=\"$(git merge-base $exist $needed)\"\n+\t\t\t\tif [ x\"$tmp\" = x\"$exist\" -o -z \"$tmp\" ]; then\n+\t\t\t\t\tnextexisting=\"$nextexisting $exist\"\n+\t\t\t\tfi\n+\t\t\tdone\n+\t\t\texisting=$nextexisting\n+\t\tdone\n+\t\techo \"$existing\"\n+\t}\n+}\n+\n+existing_to_unrevs()\n+{\n+\texisting=\"$1\"\n+\tfor exist in $existing ; do\n+\t\ttry_remove_previous $exist\n+\tdone\n+}\n+\n copy_commit()\n {\n \t# We're going to set some environment vars here, so\n@@ -599,10 +642,16 @@ cmd_split()\n \t\tdone\n \tfi\n \t\n-\tif [ -n \"$ignore_joins\" ]; then\n-\t\tunrevs=\n-\telse\n-\t\tunrevs=\"$(find_existing_splits \"$dir\" \"$revs\")\"\n+\tunrevs=\n+\tif [ -z \"$ignore_joins\" ]; then\n+\t\texisting=\"$(find_existing_splits \"$dir\" \"$revs\")\"\n+\t\tif [ -n \"$existing\" ]; then\n+\t\t\tdebug \"existing: $existing\"\n+\t\t\texisting=\"$(filter_out_needed_existing \"$revs\" \"$existing\")\"\n+\t\t\tunrevs=\"$(existing_to_unrevs \"$existing\")\"\n+\t\t\tdebug \"base: $base\"\n+\t\t\tdebug \"unrevs: $unrevs\"\n+\t\tfi\n \tfi\n \t\n \t# We can't restrict rev-list to only $dir here, because some of our\n-- \n1.8.3.2\n"},{"id":"231876","messageId":"xmqqk3fctjch.fsf@gitster.dls.corp.google.com","threadId":"35398","inReplyTo":"1386440485-3092-1-git-send-email-mmogilvi_git@miniinfo.net","subject":"Re: [PATCH 1/4] subtree: support split --rejoin --squash","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-12-10T22:46:38Z","receivedAt":"2013-12-10T22:46:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthew Ogilvie <mmogilvi_git@miniinfo.net> writes:\n\n> Allow using --squash with \"git subtree split --rejoin\".  It\n> will still split off (and save to --branch) the complete\n> subtree history, but the merge done for the \"--rejoin\" will\n> be merging a squashed representation of the new subtree\n> commits, instead of the commits themselves (similar to\n> how \"git subtree merge --squash\" works).\n>\n> Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>\n> ---\n>\n> I can think of a couple of possible objections to this patch.\n> Are these (or any others) worth fixing?\n>\n> 1. Perhaps someone want the saved subtree (--branch) to have\n>    a squashed representation as well, as an option?  Maybe we\n>    need two different --squash options?  Something\n>    like \"--rejoin-squash\"?\n> 2. It could definitely use some automated tests.  In fact,\n>    pre-existing --squash functionality is hardly tested at\n>    all, either.\n>       See patch 4 comments for a script I use to help with\n>    mostly-manual testing.\n\nAs I am totally uninterested in \"git subtree\" (sorry!), I'll queue\nthese three patches as-is so that others who are interested and\nmotivated to work on polishing it can take a look at them more\neasily.\n\nThanks.\n"}]}