{"thread":{"id":"54491","subject":"[bug] Stashes lost after out-of-memory situation","startedAt":"2020-10-23T15:02:32Z","lastAt":"2020-10-25T15:42:58Z","messageCount":4,"participants":["Marek Mrva","René Scharfe","Thomas Braun"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"408240","messageId":"65a3061a-47ef-9ca6-2468-5449cfc5b37c@eof-studios.com","threadId":"54491","inReplyTo":null,"subject":"[bug] Stashes lost after out-of-memory situation","fromName":"Marek Mrva","fromEmail":"mrva@eof-studios.com","sentAt":"2020-10-23T14:53:01Z","receivedAt":"2020-10-23T15:02:32Z","isPatch":false,"sender":{"key":"mrva@eof-studios.com","avatar":null},"body":"Hello, hopefully this is the correct mailing list - apologies if it is not.\n\nAfter issuing \"git stash pop\" while being low on memory, the following \nwas printed to the console:\n\n       0 [main] git 2061 fhandler_disk_file::fixup_mmap_after_fork: \nrequested 0x6FFFC1550000 != 0x0 mem alloc base 0x0, state 0x10000,\nsize 17545957736448, Win32 error 1455\n   36836 [main] git 2061 C:\\cygwin64\\bin\\git.exe: *** fatal error in \nforked process - recreate_mmaps_after_fork_failed\n   37523 [main] git 2061 cygwin_exception::open_stackdumpfile: Dumping \nstack trace to git.exe.stackdump\n       0 [main] git 2056 dofork: child -1 - forked process 12100 died \nunexpectedly, retry 0, exit code 0x100, errno 11\nerror: cannot fork() for status: Resource temporarily unavailable\nDropped refs/stash@{0} (06d44ccc5ed2ac93b370100f481147ae4f0065db)\nerror: cannot fork() for rev-parse: Resource temporarily unavailable\n\nAfterwards, the result of \"git stash list\" is empty, even though there \nused to be more than 10+ stashes saved.\n\nObviously while being low on memory, one should not expect commands to \nrun properly. Losing all the *other* stashes could hopefully be somehow \navoided, if possible. It is worth mentioning this happened in a cygwin \nenvironment on Windows.\n\nAny help would be greatly appreciated! :)\n\n\nWith best regards,\nMarek Mrva\n\n"},{"id":"408296","messageId":"618d66a8-e2c1-241c-5200-2298bfe24ac0@web.de","threadId":"54491","inReplyTo":"65a3061a-47ef-9ca6-2468-5449cfc5b37c@eof-studios.com","subject":"Re: [bug] Stashes lost after out-of-memory situation","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-10-24T17:06:48Z","receivedAt":"2020-10-24T17:11:55Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 23.10.20 um 16:53 schrieb Marek Mrva:\n> Hello, hopefully this is the correct mailing list - apologies if it is not.\n\nYou came to the right place.\n\n> After issuing \"git stash pop\" while being low on memory, the following was printed to the console:\n>\n>       0 [main] git 2061 fhandler_disk_file::fixup_mmap_after_fork: requested 0x6FFFC1550000 != 0x0 mem alloc base 0x0, state 0x10000,\n> size 17545957736448, Win32 error 1455\n>   36836 [main] git 2061 C:\\cygwin64\\bin\\git.exe: *** fatal error in forked process - recreate_mmaps_after_fork_failed\n>   37523 [main] git 2061 cygwin_exception::open_stackdumpfile: Dumping stack trace to git.exe.stackdump\n>       0 [main] git 2056 dofork: child -1 - forked process 12100 died unexpectedly, retry 0, exit code 0x100, errno 11\n> error: cannot fork() for status: Resource temporarily unavailable\n> Dropped refs/stash@{0} (06d44ccc5ed2ac93b370100f481147ae4f0065db)\n> error: cannot fork() for rev-parse: Resource temporarily unavailable\n>\n> Afterwards, the result of \"git stash list\" is empty, even though there used to be more than 10+ stashes saved.\n\nHow horrible!\n\n> Obviously while being low on memory, one should not expect commands\n> to run properly. Losing all the *other* stashes could hopefully be\n> somehow avoided, if possible.\n\nOf course.\n\n> It is worth mentioning this happened in a cygwin environment on Windows.\n>\n> Any help would be greatly appreciated! :)\n\nBefore any repair attempt please make a backup of the whole repository.\n\nYou may be able to recover the lost stacks using git fsck, which can\nshow dangling commits, i.e. commits that are no longer referenced.\nThey will be cleaned up eventually by git gc, so avoid running that\nuntil you recovered as many of them as possible.\n\nThe manpage of git stash says:\n\n  Recovering stash entries that were cleared/dropped erroneously::\n\n  If you mistakenly drop or clear stash entries, they cannot be recovered\n  through the normal safety mechanisms.  However, you can try the\n  following incantation to get a list of stash entries that are still in\n  your repository, but not reachable any more:\n\n  ----------------------------------------------------------------\n  git fsck --unreachable |\n  grep commit | cut -d\\  -f3 |\n  xargs git log --merges --no-walk --grep=WIP\n  ----------------------------------------------------------------\n\nYou are basically looking for merges with two parents and your stash\nmessage as commit message.  See the manpage for some more details.\n\nYou could use them to rebuild .git/logs/refs/stash manually, or to\napply them with git cherry-pick -m1.\n\nGood luck!\n\nSo why did this happen?  Looks like stash calls rev-parse to see if a\nstash pop removed the last stash and in that case proceeds to delete the\nstash ref and its reflog.  A failure of rev-parse is interpreted as no\nstashes being left.  This can be triggered by other reasons (like OOM),\nso this is dangerously fragile.  Let's make that check more precise.\n\n-- >8 --\nSubject: [PATCH] stash: simplify reflog emptiness check\n\nCalling rev-parse to check if the drop subcommand removed the last stash\nand treating its failure as confirmation is fragile, as the command can\nfail for other reasons, e.g. because the system is out of memory.\nDirectly check if the reflog is empty instead, which is more robust.\n\nReported-by: Marek Mrva <mrva@eof-studios.com>\nSigned-off-by: René Scharfe <l.s.r@web.de>\n---\n builtin/stash.c | 27 +++++++++++++--------------\n 1 file changed, 13 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/stash.c b/builtin/stash.c\nindex 3f811f3050..24ddb1bffa 100644\n--- a/builtin/stash.c\n+++ b/builtin/stash.c\n@@ -534,11 +534,22 @@ static int apply_stash(int argc, const char **argv, const char *prefix)\n \treturn ret;\n }\n\n+static int reject_reflog_ent(struct object_id *ooid, struct object_id *noid,\n+\t\t\t     const char *email, timestamp_t timestamp, int tz,\n+\t\t\t     const char *message, void *cb_data)\n+{\n+\treturn 1;\n+}\n+\n+static int reflog_is_empty(const char *refname)\n+{\n+\treturn !for_each_reflog_ent(refname, reject_reflog_ent, NULL);\n+}\n+\n static int do_drop_stash(struct stash_info *info, int quiet)\n {\n \tint ret;\n \tstruct child_process cp_reflog = CHILD_PROCESS_INIT;\n-\tstruct child_process cp = CHILD_PROCESS_INIT;\n\n \t/*\n \t * reflog does not provide a simple function for deleting refs. One will\n@@ -559,19 +570,7 @@ static int do_drop_stash(struct stash_info *info, int quiet)\n \t\t\t     info->revision.buf);\n \t}\n\n-\t/*\n-\t * This could easily be replaced by get_oid, but currently it will throw\n-\t * a fatal error when a reflog is empty, which we can not recover from.\n-\t */\n-\tcp.git_cmd = 1;\n-\t/* Even though --quiet is specified, rev-parse still outputs the hash */\n-\tcp.no_stdout = 1;\n-\tstrvec_pushl(&cp.args, \"rev-parse\", \"--verify\", \"--quiet\", NULL);\n-\tstrvec_pushf(&cp.args, \"%s@{0}\", ref_stash);\n-\tret = run_command(&cp);\n-\n-\t/* do_clear_stash if we just dropped the last stash entry */\n-\tif (ret)\n+\tif (reflog_is_empty(ref_stash))\n \t\tdo_clear_stash();\n\n \treturn 0;\n--\n2.28.0\n"},{"id":"408345","messageId":"5a3db65b-1877-c5be-8077-2926637fba6e@virtuell-zuhause.de","threadId":"54491","inReplyTo":"618d66a8-e2c1-241c-5200-2298bfe24ac0@web.de","subject":"Re: [bug] Stashes lost after out-of-memory situation","fromName":"Thomas Braun","fromEmail":"thomas.braun@virtuell-zuhause.de","sentAt":"2020-10-25T12:27:48Z","receivedAt":"2020-10-25T12:31:07Z","isPatch":false,"sender":{"key":"thomas.braun@virtuell-zuhause.de","avatar":"https://avatars.githubusercontent.com/u/1185677?v=4"},"body":"On 10/24/2020 7:06 PM, René Scharfe wrote:\n\n[...]\n\n> Looks like stash calls rev-parse to see if a\n> stash pop removed the last stash and in that case proceeds to delete the\n> stash ref and its reflog\n\nI was a bit suprised to learn that removing the last stash entry also\nremoves it from the reflog.\n\nWouldn't it be more convenient if it would be kept in the reflog even\nafter popping?\n\nSo that in cases like\n\ngit init\necho 1 > test\ngit add test\ngit commit -m \"one\" test\necho 2 > test\ngit stash\ngit checkout .\ngit stash pop\ngit checkout .\ngit reflog -p\n\nmy once stashed change would still be in the reflog?\n"},{"id":"408350","messageId":"22f34328-1e87-aa4b-3893-564dff1fb893@web.de","threadId":"54491","inReplyTo":"5a3db65b-1877-c5be-8077-2926637fba6e@virtuell-zuhause.de","subject":"Re: [bug] Stashes lost after out-of-memory situation","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-10-25T15:42:20Z","receivedAt":"2020-10-25T15:42:58Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 25.10.20 um 13:27 schrieb Thomas Braun:\n> On 10/24/2020 7:06 PM, René Scharfe wrote:\n>\n> [...]\n>\n>> Looks like stash calls rev-parse to see if a\n>> stash pop removed the last stash and in that case proceeds to delete the\n>> stash ref and its reflog\n>\n> I was a bit suprised to learn that removing the last stash entry also\n> removes it from the reflog.\n>\n> Wouldn't it be more convenient if it would be kept in the reflog even\n> after popping?\n\nThe log of the stash ref is separate from the normal reflog.  Popping\na stash doesn't remove anything from the latter.\n\nRené\n\n"}]}