{"thread":{"id":"31722","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","startedAt":"2012-10-03T19:47:47Z","lastAt":"2012-10-19T22:24:47Z","messageCount":17,"participants":["Andrew Wong","Alexander Kostikov","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"200456","messageId":"CAGAhT3kofdaQEye9QHnvFhAAzoQqZtR7d5UzbxU+zEdkAHVfuQ@mail.gmail.com","threadId":"31722","inReplyTo":null,"subject":"Rebase doesn't restore branch pointer back on out of memory","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2012-10-03T19:47:47Z","receivedAt":"2012-10-03T19:47:47Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"Hi,\n\nI'd like to report a bug in git (observed on git version 1.7.11.msysgit.1).\nWhen you do a rebase and it fails due to out of memory exception,\nrebased branch pointer is changed but commits are not rebased. That\nmakes commits that you are rebasing unreachable (except via reflog):\n\n» git lg\n* 4c60761 - (origin/master, origin/HEAD, master) ...\n\n» git rebase master sql_script\nFirst, rewinding head to replay your work on top of it...\nfatal: Out of memory? mmap failed: No error\n\n» git lg\n* 4c60761 - (HEAD, origin/master, origin/HEAD, sql_script, master) ...\n\n» git reflog sql_script\n4c60761 sql_script@{0}: rebase finished: refs/heads/sql_script onto\n4c60761303fccbb0860b28e8094ad17ae8b01d07\n13555ed sql_script@{1}: branch: Reset to sql_script@{1}\n\nExpected behaviour:\n- restore branch to pre-rebase location on out of memory exception\n- not to fall with out of memory in the first place. But for our\nrepository that could be fixed only after either:\n--- a) msysgit would have x64 binary (currently it's not available)\n--- b) rebase -m option could be used by default somehow (currently\nit's not possible so specify default -m)\n\n--\nAlexander Kostikov\n"},{"id":"200453","messageId":"506CB3B5.808@gmail.com","threadId":"31722","inReplyTo":"CAGAhT3kofdaQEye9QHnvFhAAzoQqZtR7d5UzbxU+zEdkAHVfuQ@mail.gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Andrew Wong","fromEmail":"andrew.kw.w.lists@gmail.com","sentAt":"2012-10-03T21:52:53Z","receivedAt":"2012-10-03T21:52:53Z","isPatch":false,"sender":{"key":"andrew.kw.w.lists@gmail.com","avatar":null},"body":"On 10/03/2012 03:47 PM, Alexander Kostikov wrote:\n> Expected behaviour:\n> - restore branch to pre-rebase location on out of memory exception\n> - not to fall with out of memory in the first place. But for our\n> repository that could be fixed only after either:\n> --- a) msysgit would have x64 binary (currently it's not available)\n> --- b) rebase -m option could be used by default somehow (currently\n> it's not possible so specify default -m)\nThere are already some logic in \"rebase\" that will handles failures. And \nin the case of failures, the behavior is that \"rebase\" will just stop \nand not modify the branch. That allows you can go back to the pre-rebase \nstate by \"rebase --abort\".\n\nIn your case, it's possible that \"rebase\" is failing at unexpected \nplaces, and the error wasn't caught. I tried a few simple cases by \nforcing some commands to fail during a rebase, but I couldn't reproduce \nthe behavior that you're having. It might help if we can figure out \nwhich part of \"rebase\" or \"git\" is failing (or running out of memory).\n\nAnd since you're using msysgit, I guess another possible source of the \nproblem is be that msysgit is not catching the error properly, or not \nrelying the error back to git properly.\n"},{"id":"200494","messageId":"506DA7AE.50005@gmail.com","threadId":"31722","inReplyTo":"CAGAhT3mVn-W5P-n_YeafZ_7bntkJGArJ3o6+dA5GO_H44=KHFg@mail.gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Andrew Wong","fromEmail":"andrew.kw.w.lists@gmail.com","sentAt":"2012-10-04T15:13:50Z","receivedAt":"2012-10-04T15:13:50Z","isPatch":false,"sender":{"key":"andrew.kw.w.lists@gmail.com","avatar":null},"body":"On 10/03/2012 06:35 PM, Alexander Kostikov wrote:\n>> That allows you can go back to the pre-rebase state by\n>> \"rebase --abort\".\n> rebase --abort command were not available. I guess rebase file was not created.\nI meant \"rebase --abort\" would be available *if* the error was caught by \n\"rebase\". But in your case, \"rebase\" is probably dying somewhere and the \nerror was not caught, causing \"rebase\" to think that everything \ncompleted successfully, and go ahead to update the branch.\n\n> Is there a way to include some log verbose mode to detect where\n> exactly error happens?\nThere isn't any built-in to git itself. But one way to get more info is \nrunning the rebase command this way:\n     env SHELLOPTS=\"verbose\" git rebase <your arguments>\n\nThat should print out every shell command that rebase executes. Having \nthe last page of that output should give us enough context as to where \nit's failing.\n\nJust a wild guess: rebase is probably failing at the \"format-patch\" command.\nIt'd also be interesting to see if \"rebase -i\" will also workaround the \nissue. But like you said, there's no way set \"-i\" or \"-m\" as the default.\n"},{"id":"200550","messageId":"CAGAhT3k=T0SGngMQkbXHqNfh-=LUb71C7CSrWXP2wsAgc8Tb8A@mail.gmail.com","threadId":"31722","inReplyTo":"506DA7AE.50005@gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2012-10-04T21:09:03Z","receivedAt":"2012-10-04T21:09:03Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"> Having the\n> last page of that output should give us enough context as to where it's\n> failing.\nFull script is uploaded to\nhttps://dl.dropbox.com/u/10828740/rebase.log Here is the last page:\n\n-----------------------------------[code]\nif test -s \"$dotest\"/rewritten; then\n    git notes copy --for-rewrite=rebase < \"$dotest\"/rewritten\n    if test -x \"$GIT_DIR\"/hooks/post-rewrite; then\n        \"$GIT_DIR\"/hooks/post-rewrite rebase < \"$dotest\"/rewritten\n    fi\nfi\n\nrm -fr \"$dotest\"\ngit gc --auto\ngit rev-parse HEAD\n\nret=$?\ntest 0 != $ret -a -d \"$state_dir\" && write_basic_state\nexit $ret\n-----------------------------------[/code]\n\n\n> It'd also be interesting to see if \"rebase -i\" will also workaround the\n> issue.\n\nrebase -i fails with different error:\n\n» git rebase -i master rebase_debug\nfatal: Out of memory, malloc failed (tried to allocate 458753 bytes)\n\nDo you need verbose log for it as well?\n\n-- Alexander\n\n\nOn Thu, Oct 4, 2012 at 8:13 AM, Andrew Wong <andrew.kw.w.lists@gmail.com> wrote:\n> On 10/03/2012 06:35 PM, Alexander Kostikov wrote:\n>>>\n>>> That allows you can go back to the pre-rebase state by\n>>> \"rebase --abort\".\n>>\n>> rebase --abort command were not available. I guess rebase file was not\n>> created.\n>\n> I meant \"rebase --abort\" would be available *if* the error was caught by\n> \"rebase\". But in your case, \"rebase\" is probably dying somewhere and the\n> error was not caught, causing \"rebase\" to think that everything completed\n> successfully, and go ahead to update the branch.\n>\n>\n>> Is there a way to include some log verbose mode to detect where\n>> exactly error happens?\n>\n> There isn't any built-in to git itself. But one way to get more info is\n> running the rebase command this way:\n>     env SHELLOPTS=\"verbose\" git rebase <your arguments>\n>\n> That should print out every shell command that rebase executes. Having the\n> last page of that output should give us enough context as to where it's\n> failing.\n>\n> Just a wild guess: rebase is probably failing at the \"format-patch\" command.\n> It'd also be interesting to see if \"rebase -i\" will also workaround the\n> issue. But like you said, there's no way set \"-i\" or \"-m\" as the default.\n\n\n\n-- \nAlexander Kostikov\n"},{"id":"200533","messageId":"CAGAhT3nXeTCNyfywEtcpwaB2CDfOw8+eUK-oyXL-jA2TeajkOg@mail.gmail.com","threadId":"31722","inReplyTo":"CAGAhT3k=T0SGngMQkbXHqNfh-=LUb71C7CSrWXP2wsAgc8Tb8A@mail.gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2012-10-04T21:39:02Z","receivedAt":"2012-10-04T21:39:02Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"> rebase -i fails with different error:\nAlso in case of rebase -i the branch pointer is not changed. Thus\nnothing to fix there.\n\n-- Alexander\n\nOn Thu, Oct 4, 2012 at 2:09 PM, Alexander Kostikov\n<alex.kostikov@gmail.com> wrote:\n>> Having the\n>> last page of that output should give us enough context as to where it's\n>> failing.\n> Full script is uploaded to\n> https://dl.dropbox.com/u/10828740/rebase.log Here is the last page:\n>\n> -----------------------------------[code]\n> if test -s \"$dotest\"/rewritten; then\n>     git notes copy --for-rewrite=rebase < \"$dotest\"/rewritten\n>     if test -x \"$GIT_DIR\"/hooks/post-rewrite; then\n>         \"$GIT_DIR\"/hooks/post-rewrite rebase < \"$dotest\"/rewritten\n>     fi\n> fi\n>\n> rm -fr \"$dotest\"\n> git gc --auto\n> git rev-parse HEAD\n>\n> ret=$?\n> test 0 != $ret -a -d \"$state_dir\" && write_basic_state\n> exit $ret\n> -----------------------------------[/code]\n>\n>\n>> It'd also be interesting to see if \"rebase -i\" will also workaround the\n>> issue.\n>\n> rebase -i fails with different error:\n>\n> » git rebase -i master rebase_debug\n> fatal: Out of memory, malloc failed (tried to allocate 458753 bytes)\n>\n> Do you need verbose log for it as well?\n>\n> -- Alexander\n>\n>\n> On Thu, Oct 4, 2012 at 8:13 AM, Andrew Wong <andrew.kw.w.lists@gmail.com> wrote:\n>> On 10/03/2012 06:35 PM, Alexander Kostikov wrote:\n>>>>\n>>>> That allows you can go back to the pre-rebase state by\n>>>> \"rebase --abort\".\n>>>\n>>> rebase --abort command were not available. I guess rebase file was not\n>>> created.\n>>\n>> I meant \"rebase --abort\" would be available *if* the error was caught by\n>> \"rebase\". But in your case, \"rebase\" is probably dying somewhere and the\n>> error was not caught, causing \"rebase\" to think that everything completed\n>> successfully, and go ahead to update the branch.\n>>\n>>\n>>> Is there a way to include some log verbose mode to detect where\n>>> exactly error happens?\n>>\n>> There isn't any built-in to git itself. But one way to get more info is\n>> running the rebase command this way:\n>>     env SHELLOPTS=\"verbose\" git rebase <your arguments>\n>>\n>> That should print out every shell command that rebase executes. Having the\n>> last page of that output should give us enough context as to where it's\n>> failing.\n>>\n>> Just a wild guess: rebase is probably failing at the \"format-patch\" command.\n>> It'd also be interesting to see if \"rebase -i\" will also workaround the\n>> issue. But like you said, there's no way set \"-i\" or \"-m\" as the default.\n>\n>\n>\n> --\n> Alexander Kostikov\n\n\n\n-- \nAlexander Kostikov\n"},{"id":"200573","messageId":"506E1327.1070602@gmail.com","threadId":"31722","inReplyTo":"CAGAhT3k=T0SGngMQkbXHqNfh-=LUb71C7CSrWXP2wsAgc8Tb8A@mail.gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-04T22:52:23Z","receivedAt":"2012-10-04T22:52:23Z","isPatch":false,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"On 10/04/2012 05:09 PM, Alexander Kostikov wrote:\n> Full script is uploaded to\n> https://dl.dropbox.com/u/10828740/rebase.log  Here is the last page:\nJudging from that log, I'm pretty sure \"rebase\" is failing at \n\"format-patch\". I was able to reproduce the issue you're having: \n\"rebase\" finished and modified the branch even though it actually failed.\n\n\"rebase\" is not catching that error. I'll try to come up with a patch to \nfix it later tonight, so that \"rebase\" will fail correctly. And when it \ndoes, you'll be able to do \"rebase --abort\" to go back to your original \nstate.\n"},{"id":"200575","messageId":"CAGAhT3kihpANJ2h=MQ+mpD2ZsZzpbfxDKPRvrdOsEpz53w556g@mail.gmail.com","threadId":"31722","inReplyTo":"506E1327.1070602@gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2012-10-04T23:59:20Z","receivedAt":"2012-10-04T23:59:20Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"Thanks, Andrew!\nI'm looking forward for the patch.\n\nOn Thu, Oct 4, 2012 at 3:52 PM, Andrew Wong <andrew.kw.w@gmail.com> wrote:\n> On 10/04/2012 05:09 PM, Alexander Kostikov wrote:\n>>\n>> Full script is uploaded to\n>> https://dl.dropbox.com/u/10828740/rebase.log  Here is the last page:\n>\n> Judging from that log, I'm pretty sure \"rebase\" is failing at\n> \"format-patch\". I was able to reproduce the issue you're having: \"rebase\"\n> finished and modified the branch even though it actually failed.\n>\n> \"rebase\" is not catching that error. I'll try to come up with a patch to fix\n> it later tonight, so that \"rebase\" will fail correctly. And when it does,\n> you'll be able to do \"rebase --abort\" to go back to your original state.\n>\n\n\n\n-- \nAlexander Kostikov\n"},{"id":"200590","messageId":"1349412790-6087-1-git-send-email-andrew.kw.w@gmail.com","threadId":"31722","inReplyTo":"506E1327.1070602@gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-05T04:53:09Z","receivedAt":"2012-10-05T04:53:09Z","isPatch":false,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"'format-patch' is failing due to out of memory, and the error not being caught.\nSo 'rebase' thinks 'am' has completed successfully and continue on with\ncleanup. i.e. move_to_original_branch\nSo the user loses commits from the original head, and have to rely on reflog to\nreturn to the original head.\n\nSince the exit status of 'format-patch' is not available, we have to use ||\nwith 'format-patch' to handle the error.  Also, when 'format-patch' fails, the\nstate_dir does not necessarily exist, so I'm putting the 'format-patch-failed'\nfile inside GIT_DIR. Is there a better location to put such a file?\n\nThe way I handle the error feels a bit bruteforced.  Any suggestions on a\nbetter way to handle the error?\n\nI also thought about separating 'format-patch' and 'am' into two separate\ncommands, and use an intermediate file to store the output of 'format-patch'.\nBut the intermediate file could get very big, so it didn't seem like a good\nidea.\n\nAndrew Wong (1):\n  rebase: Handle cases where format-patch fails\n\n git-rebase--am.sh | 37 +++++++++++++++++++++++++++++++++++--\n 1 file changed, 35 insertions(+), 2 deletions(-)\n\n-- \n1.8.0.rc0.18.gf84667d\n"},{"id":"200592","messageId":"1349412790-6087-2-git-send-email-andrew.kw.w@gmail.com","threadId":"31722","inReplyTo":"1349412790-6087-1-git-send-email-andrew.kw.w@gmail.com","subject":"[RFC] rebase: Handle cases where format-patch fails","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-05T04:53:10Z","receivedAt":"2012-10-05T04:53:10Z","isPatch":false,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"'format-patch' could fail due to reasons such as out of memory. Such\nfailures are not detected or handled, which causes rebase to incorrectly\nthink that it completed successfully and continue with cleanup. i.e.\ncalling move_to_original_branch\n\nSince only the exit status of the last command in the pipeline is\navailable, we rely on || to detect whether 'format-patch' has failed.\n\nAlso print messages to help user with how to recover from such failures.\n\nSigned-off-by: Andrew Wong <andrew.kw.w@gmail.com>\n---\n git-rebase--am.sh | 37 +++++++++++++++++++++++++++++++++++--\n 1 file changed, 35 insertions(+), 2 deletions(-)\n\ndiff --git a/git-rebase--am.sh b/git-rebase--am.sh\nindex 392ebc9..8dae804 100644\n--- a/git-rebase--am.sh\n+++ b/git-rebase--am.sh\n@@ -26,10 +26,43 @@ then\n \t# makes this easy\n \tgit cherry-pick --allow-empty \"$revisions\"\n else\n-\tgit format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n+\t( git format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n \t\t--src-prefix=a/ --dst-prefix=b/ \\\n-\t\t--no-renames $root_flag \"$revisions\" |\n+\t\t--no-renames $root_flag \"$revisions\" ||\n+\t\techo $? > \"$GIT_DIR\"/format-patch-failed ) |\n \tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\"\n+\tret=$?\n+\tif test -f \"$GIT_DIR\"/format-patch-failed\n+\tthen\n+\t\tret=1\n+\t\trm -f \"$GIT_DIR\"/format-patch-failed\n+\t\tif test -d \"$state_dir\"\n+\t\tthen\n+\t\t\techo\n+\t\t\techo \"'git format-patch' seems to have failed in the middle of 'git am'.\"\n+\t\t\techo \"If you continue rebasing, you will likely be losing some commits.\"\n+\t\t\techo \"It is recommended that you abort rebasing by running:\"\n+\t\t\techo\n+\t\t\techo \"    git rebase --abort\"\n+\t\t\techo\n+\t\telse\n+\t\t\techo\n+\t\t\techo \"'git format-patch' seems to have failed before 'git am' started.\"\n+\t\t\techo \"It is impossible to continue or abort rebasing.\"\n+\t\t\techo \"You have to use the following to return to your original head:\"\n+\t\t\techo\n+\t\t\tcase \"$head_name\" in\n+\t\t\trefs/*)\n+\t\t\t\techo \"    git checkout $head_name\"\n+\t\t\t\t;;\n+\t\t\t*)\n+\t\t\t\techo \"    git checkout $orig_head\"\n+\t\t\t\t;;\n+\t\t\tesac\n+\t\t\techo\n+\t\tfi\n+\tfi\n+\ttest 0 != $ret && false\n fi && move_to_original_branch\n \n ret=$?\n-- \n1.8.0.rc0.18.gf84667d\n"},{"id":"200637","messageId":"7vipaou0zw.fsf@alter.siamese.dyndns.org","threadId":"31722","inReplyTo":"1349412790-6087-2-git-send-email-andrew.kw.w@gmail.com","subject":"Re: [RFC] rebase: Handle cases where format-patch fails","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-05T20:17:55Z","receivedAt":"2012-10-05T20:17:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Andrew Wong <andrew.kw.w@gmail.com> writes:\n\n> 'format-patch' could fail due to reasons such as out of memory. Such\n> failures are not detected or handled, which causes rebase to incorrectly\n> think that it completed successfully and continue with cleanup. i.e.\n> calling move_to_original_branch\n>\n> Since only the exit status of the last command in the pipeline is\n> available, we rely on || to detect whether 'format-patch' has failed.\n>\n> Also print messages to help user with how to recover from such failures.\n>\n> Signed-off-by: Andrew Wong <andrew.kw.w@gmail.com>\n> ---\n>  git-rebase--am.sh | 37 +++++++++++++++++++++++++++++++++++--\n>  1 file changed, 35 insertions(+), 2 deletions(-)\n>\n> diff --git a/git-rebase--am.sh b/git-rebase--am.sh\n> index 392ebc9..8dae804 100644\n> --- a/git-rebase--am.sh\n> +++ b/git-rebase--am.sh\n> @@ -26,10 +26,43 @@ then\n>  \t# makes this easy\n>  \tgit cherry-pick --allow-empty \"$revisions\"\n>  else\n> -\tgit format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n> +\t( git format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n>  \t\t--src-prefix=a/ --dst-prefix=b/ \\\n> -\t\t--no-renames $root_flag \"$revisions\" |\n> +\t\t--no-renames $root_flag \"$revisions\" ||\n> +\t\techo $? > \"$GIT_DIR\"/format-patch-failed ) |\n\nPlease make sure there is no marker-file that was leftover from\nprevious invocation or whatever reason, e.g.\n\n\trm -f \"$GIT_DIR/format-patch-failed\"\n        (\n\t\tgit format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n\t\t\t--src-prefix=a/ --dst-prefix=b/ \\\n\t\t\t--no-renames $root_flag \"$revisions\" ||\n\t\techo $? >\"$GIT_DIR\"/format-patch-failed\n\t) |\n  \tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\"\n\nBut when format-patch dies for whatever reason, it is likely that\nthe partial output will cause \"am\" to barf on the last part of it\n(either \"missing patch text\" if it stops in the middle of commit log\nmessage, or \"corrupt patch\" if it stops in the middle of a hunk).\nIt may make sense to make this all-or-none, i.e. when format-patch\nfails, you do not even start \"am\", something like...\n\n\trm -f \"$GIT_DIR/patch-input\"\n        if ! git format-patch -k --stdout >\"$GIT_DIR/patch-input\" \\\n\t        --full-index --ignore-if-in-upstream \\\n\t\t--src-prefix=a/ --dst-prefix=b/ \\\n\t\t--no-renames $root_flag \"$revisions\"\n\tthen\n\t\t... format-patch barfed, here is how to deal with it...\n\telse\n        \tgit am <\"$GIT_DIR/patch-input\" $git_am_opt ...\n\tfi\n\trm -f \"$GIT_DIR/patch-input\"\n\nbut I wonder what the performance implication would be for normal cases.\n\n> +\tret=$?\n> +\tif test -f \"$GIT_DIR\"/format-patch-failed\n> +\tthen\n> +\t\tret=1\n> +\t\trm -f \"$GIT_DIR\"/format-patch-failed\n> +\t\tif test -d \"$state_dir\"\n> +\t\tthen\n> +\t\t\techo\n> +\t\t\techo \"'git format-patch' seems to have failed in the middle of 'git am'.\"\n> +\t\t\techo \"If you continue rebasing, you will likely be losing some commits.\"\n> +\t\t\techo \"It is recommended that you abort rebasing by running:\"\n> +\t\t\techo\n> +\t\t\techo \"    git rebase --abort\"\n> +\t\t\techo\n> +\t\telse\n> +\t\t\techo\n> +\t\t\techo \"'git format-patch' seems to have failed before 'git am' started.\"\n> +\t\t\techo \"It is impossible to continue or abort rebasing.\"\n> +\t\t\techo \"You have to use the following to return to your original head:\"\n> +\t\t\techo\n> +\t\t\tcase \"$head_name\" in\n> +\t\t\trefs/*)\n> +\t\t\t\techo \"    git checkout $head_name\"\n> +\t\t\t\t;;\n> +\t\t\t*)\n> +\t\t\t\techo \"    git checkout $orig_head\"\n> +\t\t\t\t;;\n> +\t\t\tesac\n> +\t\t\techo\n> +\t\tfi\n> +\tfi\n> +\ttest 0 != $ret && false\n>  fi && move_to_original_branch\n>  \n>  ret=$?\n"},{"id":"200779","messageId":"1349724988-14625-1-git-send-email-andrew.kw.w@gmail.com","threadId":"31722","inReplyTo":"7vipaou0zw.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC] rebase: Handle cases where format-patch fails","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-08T19:36:27Z","receivedAt":"2012-10-08T19:36:27Z","isPatch":false,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"Here's an alternate method for handling 'format-patch' failing.  We remove the\npipe between 'format-patch' and 'am' by storing the patch in an intermediate\nfile. This means we can gurantee that 'am' is always invokved with the complete\ninput.\n\nI did some timing on rebasing 500 commits from the git repo.  The patch file\nhad a size of 6.9MB, but the overall timings between the pipe approach and\nintermediate file approach are approximately the same.  I did the tests on a\nLinux machine, so I don't know what the impact would be on other platforms, or\nin repos with larger files and perhaps binary files.\n\nAndrew Wong (1):\n  rebase: Handle cases where format-patch fails\n\n git-rebase--am.sh | 28 +++++++++++++++++++++++++---\n 1 file changed, 25 insertions(+), 3 deletions(-)\n\n-- \n1.8.0.rc0.19.ga19ab82.dirty\n"},{"id":"200780","messageId":"1349724988-14625-2-git-send-email-andrew.kw.w@gmail.com","threadId":"31722","inReplyTo":"1349724988-14625-1-git-send-email-andrew.kw.w@gmail.com","subject":"[RFC] rebase: Handle cases where format-patch fails","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-08T19:36:28Z","receivedAt":"2012-10-08T19:36:28Z","isPatch":false,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"'format-patch' could fail due to reasons such as out of memory. Such\nfailures are not detected or handled, which causes rebase to incorrectly\nthink that it completed successfully and continue with cleanup. i.e.\ncalling move_to_original_branch\n\nInstead of using a pipe, we separate 'format-patch' and 'am' by using an\nintermediate file. This gurantees that we can invoke 'am' with the\ncomplete input, or not invoking 'am' at all if 'format-patch' failed.\n\nAlso print messages to help user with how to recover from such failures.\n\nSigned-off-by: Andrew Wong <andrew.kw.w@gmail.com>\n---\n git-rebase--am.sh | 28 +++++++++++++++++++++++++---\n 1 file changed, 25 insertions(+), 3 deletions(-)\n\ndiff --git a/git-rebase--am.sh b/git-rebase--am.sh\nindex 392ebc9..a955b38 100644\n--- a/git-rebase--am.sh\n+++ b/git-rebase--am.sh\n@@ -26,10 +26,32 @@ then\n \t# makes this easy\n \tgit cherry-pick --allow-empty \"$revisions\"\n else\n-\tgit format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n+\trm -f \"$GIT_DIR/format-patch\"\n+\tif ! git format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n \t\t--src-prefix=a/ --dst-prefix=b/ \\\n-\t\t--no-renames $root_flag \"$revisions\" |\n-\tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\"\n+\t\t--no-renames $root_flag \"$revisions\" > \"$GIT_DIR/format-patch\" && ret=$?\n+\tthen\n+\t\trm \"$GIT_DIR/format-patch\"\n+\t\techo\n+\t\techo \"'git format-patch' seems to have failed.\"\n+\t\techo \"It is impossible to continue or abort rebasing.\"\n+\t\techo \"You have to use the following to return to your original head:\"\n+\t\techo\n+\t\tcase \"$head_name\" in\n+\t\trefs/*)\n+\t\t\techo \"    git checkout $head_name\"\n+\t\t\t;;\n+\t\t*)\n+\t\t\techo \"    git checkout $orig_head\"\n+\t\t\t;;\n+\t\tesac\n+\t\techo\n+\t\texit $ret\n+\tfi\n+\n+\tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\" < \"$GIT_DIR/format-patch\" || ret=$?\n+\trm -f \"$GIT_DIR/format-patch\"\n+\ttest 0 != ret && ( exit $ret )\n fi && move_to_original_branch\n \n ret=$?\n-- \n1.8.0.rc0.19.ga19ab82.dirty\n"},{"id":"200794","messageId":"7vtxu4io7o.fsf@alter.siamese.dyndns.org","threadId":"31722","inReplyTo":"1349724988-14625-2-git-send-email-andrew.kw.w@gmail.com","subject":"Re: [RFC] rebase: Handle cases where format-patch fails","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-08T22:38:35Z","receivedAt":"2012-10-08T22:38:35Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Andrew Wong <andrew.kw.w@gmail.com> writes:\n\n> 'format-patch' could fail due to reasons such as out of memory. Such\n> failures are not detected or handled, which causes rebase to incorrectly\n> think that it completed successfully and continue with cleanup. i.e.\n> calling move_to_original_branch\n>\n> Instead of using a pipe, we separate 'format-patch' and 'am' by using an\n> intermediate file. This gurantees that we can invoke 'am' with the\n> complete input, or not invoking 'am' at all if 'format-patch' failed.\n>\n> Also print messages to help user with how to recover from such failures.\n>\n> Signed-off-by: Andrew Wong <andrew.kw.w@gmail.com>\n> ---\n>  git-rebase--am.sh | 28 +++++++++++++++++++++++++---\n>  1 file changed, 25 insertions(+), 3 deletions(-)\n>\n> diff --git a/git-rebase--am.sh b/git-rebase--am.sh\n> index 392ebc9..a955b38 100644\n> --- a/git-rebase--am.sh\n> +++ b/git-rebase--am.sh\n> @@ -26,10 +26,32 @@ then\n>  \t# makes this easy\n>  \tgit cherry-pick --allow-empty \"$revisions\"\n>  else\n> -\tgit format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n> +\trm -f \"$GIT_DIR/format-patch\"\n> +\tif ! git format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n>  \t\t--src-prefix=a/ --dst-prefix=b/ \\\n> -\t\t--no-renames $root_flag \"$revisions\" |\n> -\tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\"\n> +\t\t--no-renames $root_flag \"$revisions\" > \"$GIT_DIR/format-patch\" && ret=$?\n> +\tthen\n\nIs it just me?  I find this construct\n\n\tif ! cmd && ret=$?\n        then\n\nvery hard to wrap my mind around.  Why not\n\n\tgit format-patch ... just as before ... \\\n          ... >\"$GIT_DIR/formatted-patches\" || {\n\t\t# error handling or advices come here...\n                rm -f \"$GIT_DIR/formatted-patches\"\n\t\texit 1\n\t}\n\n\tgit am ... just as before ... \"$GIT_DIR/formatted-patches\" || {\n\t\t# possibly another error handling or advices come here...\n\t\trm -f \"$GIT_DIR/formatted-patches\"\n\t\texit 1\n\t}\n\nwithout changing anything else?\n\n> +\t\trm \"$GIT_DIR/format-patch\"\n> +\t\techo\n> +\t\techo \"'git format-patch' seems to have failed.\"\n> +\t\techo \"It is impossible to continue or abort rebasing.\"\n> +\t\techo \"You have to use the following to return to your original head:\"\n> +\t\techo\n> +\t\tcase \"$head_name\" in\n> +\t\trefs/*)\n> +\t\t\techo \"    git checkout $head_name\"\n> +\t\t\t;;\n> +\t\t*)\n> +\t\t\techo \"    git checkout $orig_head\"\n> +\t\t\t;;\n> +\t\tesac\n\nYou _know_ format-patch failed, not just \"seems to have\", at this\npoint, no?  Why is it impossible to abort?\n\nWhat have we done before reaching to this point?  We know we are\ndoing the basic \"git rebase\", without any funny \"-m/-i/-p\" business,\nso the only thing we have done are (1) detached HEAD at the new\nonto, (2) set ORIG_HEAD to point at the original tip of the branch\nbeing rebased (or the commit we were sitting at, if we are rebasing\na detached history), and (3) head_name has the refname of the\noriginal branch (or detached HEAD) and branch_name has the name of\nthe branch (or HEAD).\n\nShouldn't we be just rewinding what we have done so far and error\nthe whole thing out instead?  Perhaps the first \"# error handling or\nadvises come here...\" part may simply be\n\n\tcase \"$head_name\" in\n\trefs/heads/*)\n\t\tgit checkout \"$head_name\"\n                ;;\n\t*)\n\t\tgit checkout \"$orig_head\"\n                ;;\n\tesac\n\tcat >&2 <<-\\EOF\n\tError was found while preparing the patches ($revisions) to\n        replay on the rewound head. You cannot rebase this history.\n        EOF\n\nor something like that.  The format-patch output (and its error) may\nbe of interest in getting help going forward.\n"},{"id":"200971","messageId":"1349927643-7195-1-git-send-email-andrew.kw.w@gmail.com","threadId":"31722","inReplyTo":"7vtxu4io7o.fsf@alter.siamese.dyndns.org","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-11T03:54:02Z","receivedAt":"2012-10-11T03:54:02Z","isPatch":false,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"For the 'format-patch' part, originally I was going to do something like:\n\n\tgit format-patch ... || {\n\t\t...\n\t}\n\nBut later I thought it's better to use a consistent style as the following\n'am' part.\n\nFor the 'am' part, if we kept the following line at the end of the if-block:\n\n\tfi && move_to_original_branch\n\nthen, before exiting the if-block, we would have to do something like:\n\n\ttest 0 != $ret && false\n\nwhich seems a bit ugly to me. So I removed the use of '&&', and rearrange the\n'write_basic_state' and 'move_to_original_branch' to make the logic flow a bit\nbetter and easier to read.\n\nAndrew Wong (1):\n  rebase: Handle cases where format-patch fails\n\n git-rebase--am.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++------\n 1 file changed, 45 insertions(+), 6 deletions(-)\n\n-- \n1.8.0.rc0.19.gc58a63a.dirty\n"},{"id":"200972","messageId":"1349927643-7195-2-git-send-email-andrew.kw.w@gmail.com","threadId":"31722","inReplyTo":"1349927643-7195-1-git-send-email-andrew.kw.w@gmail.com","subject":"[PATCH] rebase: Handle cases where format-patch fails","fromName":"Andrew Wong","fromEmail":"andrew.kw.w@gmail.com","sentAt":"2012-10-11T03:54:03Z","receivedAt":"2012-10-11T03:54:03Z","isPatch":true,"sender":{"key":"andrew.kw.w@gmail.com","avatar":"https://avatars.githubusercontent.com/u/489311?v=4"},"body":"'format-patch' could fail due to reasons such as out of memory. Such\nfailures are not detected or handled, which causes rebase to incorrectly\nthink that it completed successfully and continue with cleanup. i.e.\ncalling move_to_original_branch\n\nInstead of using a pipe, we separate 'format-patch' and 'am' by using an\nintermediate file. This gurantees that we can invoke 'am' with the\ncomplete input, or not invoking 'am' at all if 'format-patch' failed.\n\nAlso remove the use of '&&' at the end of the if-block, and rearrange\nthe 'write_basic_state' and 'move_to_original_branch' to make the logic\nflow a bit better and easier to read.\n\nSigned-off-by: Andrew Wong <andrew.kw.w@gmail.com>\n---\n git-rebase--am.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++------\n 1 file changed, 45 insertions(+), 6 deletions(-)\n\ndiff --git a/git-rebase--am.sh b/git-rebase--am.sh\nindex 392ebc9..85b594e 100644\n--- a/git-rebase--am.sh\n+++ b/git-rebase--am.sh\n@@ -18,6 +18,7 @@ esac\n \n test -n \"$rebase_root\" && root_flag=--root\n \n+ret=0\n if test -n \"$keep_empty\"\n then\n \t# we have to do this the hard way.  git format-patch completely squashes\n@@ -25,13 +26,51 @@ then\n \t# itself well to recording empty patches.  fortunately, cherry-pick\n \t# makes this easy\n \tgit cherry-pick --allow-empty \"$revisions\"\n+\tret=$?\n else\n+\trm -f \"$GIT_DIR/format-patch\"\n+\n \tgit format-patch -k --stdout --full-index --ignore-if-in-upstream \\\n \t\t--src-prefix=a/ --dst-prefix=b/ \\\n-\t\t--no-renames $root_flag \"$revisions\" |\n-\tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\"\n-fi && move_to_original_branch\n+\t\t--no-renames $root_flag \"$revisions\" > \"$GIT_DIR/format-patch\"\n+\tret=$?\n+\n+\tif test 0 != $ret\n+\tthen\n+\t\trm -f \"$GIT_DIR/format-patch\"\n+\n+\t\tcase \"$head_name\" in\n+\t\trefs/heads/*)\n+\t\t\tgit checkout -q \"$head_name\"\n+\t\t\t;;\n+\t\t*)\n+\t\t\tgit checkout -q \"$orig_head\"\n+\t\t\t;;\n+\t\tesac\n+\n+\t\tcat >&2 <<-EOF\n+\n+\t\tgit encountered an error while preparing the patches to replay\n+\t\tthese revisions:\n+\n+\t\t    $revisions\n+\n+\t\tAs a result, git cannot rebase these revisions.\n+\t\tEOF\n+\n+\t\texit $?\n+\tfi\n+\n+\tgit am $git_am_opt --rebasing --resolvemsg=\"$resolvemsg\" < \"$GIT_DIR/format-patch\"\n+\tret=$?\n+\n+\trm -f \"$GIT_DIR/format-patch\"\n+fi\n+\n+if test 0 != $ret\n+then\n+\ttest -d \"$state_dir\" && write_basic_state\n+\texit $ret\n+fi\n \n-ret=$?\n-test 0 != $ret -a -d \"$state_dir\" && write_basic_state\n-exit $ret\n+move_to_original_branch\n-- \n1.8.0.rc0.19.gc58a63a.dirty\n"},{"id":"201578","messageId":"CAGAhT3nNdPxtDKtVCnPAa4OWOhGygzoq6DqHVEckQ60XWAAKZA@mail.gmail.com","threadId":"31722","inReplyTo":"1349927643-7195-1-git-send-email-andrew.kw.w@gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2012-10-19T21:49:11Z","receivedAt":"2012-10-19T21:49:11Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"Sorry to bother but I was wondering what would be the release version\nthat would have this patch.\n\n-- Alexander\n\n\nOn Wed, Oct 10, 2012 at 8:54 PM, Andrew Wong <andrew.kw.w@gmail.com> wrote:\n>\n> For the 'format-patch' part, originally I was going to do something like:\n>\n>         git format-patch ... || {\n>                 ...\n>         }\n>\n> But later I thought it's better to use a consistent style as the following\n> 'am' part.\n>\n> For the 'am' part, if we kept the following line at the end of the if-block:\n>\n>         fi && move_to_original_branch\n>\n> then, before exiting the if-block, we would have to do something like:\n>\n>         test 0 != $ret && false\n>\n> which seems a bit ugly to me. So I removed the use of '&&', and rearrange the\n> 'write_basic_state' and 'move_to_original_branch' to make the logic flow a bit\n> better and easier to read.\n>\n> Andrew Wong (1):\n>   rebase: Handle cases where format-patch fails\n>\n>  git-rebase--am.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++------\n>  1 file changed, 45 insertions(+), 6 deletions(-)\n>\n> --\n> 1.8.0.rc0.19.gc58a63a.dirty\n>\n\n\n\n--\nAlexander Kostikov\n"},{"id":"201581","messageId":"7va9vihzgw.fsf@alter.siamese.dyndns.org","threadId":"31722","inReplyTo":"CAGAhT3nNdPxtDKtVCnPAa4OWOhGygzoq6DqHVEckQ60XWAAKZA@mail.gmail.com","subject":"Re: Rebase doesn't restore branch pointer back on out of memory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-19T22:24:47Z","receivedAt":"2012-10-19T22:24:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alexander Kostikov <alex.kostikov@gmail.com> writes:\n\n> Sorry to bother but I was wondering what would be the release version\n> that would have this patch.\n\nThat depends on how well the people who are interested in this\nchange test it to smoke out potential issues (if any) in it.\n\nIt currently is on the 'pu' branch.\n"}]}