{"thread":{"id":"53342","subject":"Git Stash brake splitIndex","startedAt":"2020-04-28T13:19:12Z","lastAt":"2020-05-04T10:26:22Z","messageCount":4,"participants":["Son Luong Ngoc","Christian Couder","Alban Gruin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"396458","messageId":"EED2CFF1-5BEF-429D-AB99-AD148A867614@gmail.com","threadId":"53342","inReplyTo":null,"subject":"Git Stash brake splitIndex","fromName":"Son Luong Ngoc","fromEmail":"sluongng@gmail.com","sentAt":"2020-04-28T13:19:06Z","receivedAt":"2020-04-28T13:19:12Z","isPatch":false,"sender":{"key":"sluongng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/26684313?v=4"},"body":"Hi folks,\n\nI am on git version 2.26.2.526.g744177e7f7 (latest next)\nWhen you do a git stash while using splitIndex, it seems like the index will get corrupted\n\nUsing configs:\ncore.splitindex=true\nsplitindex.maxpercentchange=50\nsplitindex.sharedindexexpire=now\n\nReproduce steps:\n\nmkdir repo\ncd repo && git init\necho a > a\ncommit -a -m 'add a'\necho b > b\ngit add b\ngit stash\nSaved working directory and index state WIP on master: 1955b62 add a\nfatal: .git/sharedindex.8ddd8dad6ccb4858f27d4ff20f4d8bf6654441e0: index file open failed: No such file or directory\n\nSome traces:\nmaster ~/test/git/repo> GIT_TRACE2=1 GIT_TRACE2_NESTING=5 git stash\n15:18:15.442295 common-main.c:48 version 2.26.2.526.g744177e7f7\n15:18:15.442914 common-main.c:49 start git stash\n15:18:15.443284 repository.c:134 worktree /Users/sluongngoc/test/git/repo\n15:18:15.443491 git.c:442 cmd_name stash (stash)\n15:18:15.448579 run-command.c:735 child_start[0] git update-index --ignore-skip-worktree-entries -z --add --remove --stdin\n15:18:15.455972 common-main.c:48 version 2.26.2.526.g744177e7f7\n15:18:15.456514 common-main.c:49 start /Users/sluongngoc/libexec/git-core/git update-index --ignore-skip-worktree-entries -z --add --remove --stdin\n15:18:15.456788 repository.c:134 worktree /Users/sluongngoc/test/git/repo\n15:18:15.456927 git.c:442 cmd_name update-index (stash/update-index)\n15:18:15.458444 git.c:672 exit elapsed:0.004021 code:0\n15:18:15.458457 trace2/tr2_tgt_normal.c:123 atexit elapsed:0.004039 code:0\n15:18:15.458774 run-command.c:990 child_exit[0] pid:1813 code:0 elapsed:0.010169\nSaved working directory and index state WIP on master: 1955b62 add a\n15:18:15.461082 run-command.c:735 child_start[1] git reset --hard -q --no-recurse-submodules\n15:18:15.467260 common-main.c:48 version 2.26.2.526.g744177e7f7\n15:18:15.467553 common-main.c:49 start /Users/sluongngoc/libexec/git-core/git reset --hard -q --no-recurse-submodules\n15:18:15.467931 repository.c:134 worktree /Users/sluongngoc/test/git/repo\n15:18:15.468071 git.c:442 cmd_name reset (stash/reset)\n15:18:15.468555 usage.c:64 error .git/sharedindex.8ddd8dad6ccb4858f27d4ff20f4d8bf6654441e0: index file open failed: No such file or directory\nfatal: .git/sharedindex.8ddd8dad6ccb4858f27d4ff20f4d8bf6654441e0: index file open failed: No such file or directory\n15:18:15.468587 usage.c:68 exit elapsed:0.002714 code:128\n15:18:15.468595 trace2/tr2_tgt_normal.c:123 atexit elapsed:0.002726 code:128\n15:18:15.468889 run-command.c:990 child_exit[1] pid:1814 code:128 elapsed:0.007797\n15:18:15.468930 git.c:672 exit elapsed:0.028400 code:1\n15:18:15.468947 trace2/tr2_tgt_normal.c:123 atexit elapsed:0.028418 code:1\nexit 1\n\nCheers,\nSon Luong."},{"id":"396461","messageId":"CAP8UFD32D4WnSbpB1erMTidFpinCNwzqHG_fsyHqRmWgZpsSjA@mail.gmail.com","threadId":"53342","inReplyTo":"EED2CFF1-5BEF-429D-AB99-AD148A867614@gmail.com","subject":"Re: Git Stash brake splitIndex","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2020-04-28T13:57:41Z","receivedAt":"2020-04-28T13:57:55Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Hi,\n\nOn Tue, Apr 28, 2020 at 3:21 PM Son Luong Ngoc wrote:\n\n> I am on git version 2.26.2.526.g744177e7f7 (latest next)\n> When you do a git stash while using splitIndex, it seems like the index will get corrupted\n>\n> Using configs:\n> core.splitindex=true\n> splitindex.maxpercentchange=50\n> splitindex.sharedindexexpire=now\n>\n> Reproduce steps:\n>\n> mkdir repo\n> cd repo && git init\n> echo a > a\n> commit -a -m 'add a'\n\nIt looks like it should be `git commit -a -m 'add a'`\n\nWhen I try to reproduce the steps using git version 2.26.2.526.g744177e7f7:\n\nmkdir repo\ncd repo && git init\ngit config core.splitindex true\ngit config splitindex.maxpercentchange 50\ngit config splitindex.sharedindexexpire now\necho a > a\ngit commit -a -m 'add a'\n\nI get a segfault then:\n\nProgram received signal SIGSEGV, Segmentation fault.\n0x00005555556ccf89 in ewah_each_bit (self=0x0, callback=0x5555557af648\n, payload=0x555555b47480 )\n at ewah/ewah_bitmap.c:252\n252 while (pointer < self->buffer_size) {\n(gdb) bt\n#0 0x00005555556ccf89 in ewah_each_bit (self=0x0,\ncallback=0x5555557af648 , payload=0x555555b47480\n)\n at ewah/ewah_bitmap.c:252\n#1 0x00005555557af90c in merge_base_index (istate=0x555555b47480\n) at split-index.c:162\n#2 0x0000555555748b9f in read_index_from (istate=0x555555b47480\n, path=0x555555b4cd80 \".git/index\",\n gitdir=0x555555b4ac20 \".git\") at read-cache.c:2335\n#3 0x000055555576faf3 in repo_read_index (repo=0x555555b33f20\n) at repository.c:271\n#4 0x000055555559b91b in prepare_to_commit (index_file=0x555555b4d760\n\"/tmp/git/repo/.git/index.lock\", prefix=0x0,\n current_head=0x0, s=0x555555b04420 ,\nauthor_ident=0x7fffffffd810) at builtin/commit.c:927\n#5 0x000055555559d6a4 in cmd_commit (argc=0, argv=0x7fffffffdcf0,\nprefix=0x0) at builtin/commit.c:1595\n#6 0x0000555555570fda in run_builtin (p=0x555555af1218\n, argc=4, argv=0x7fffffffdcf0) at git.c:447\n#7 0x000055555557134b in handle_builtin (argc=4, argv=0x7fffffffdcf0)\nat git.c:672\n#8 0x0000555555571610 in run_argv (argcp=0x7fffffffdb9c,\nargv=0x7fffffffdb90) at git.c:739\n#9 0x0000555555571aba in cmd_main (argc=4, argv=0x7fffffffdcf0) at git.c:870\n#10 0x0000555555641bc4 in main (argc=5, argv=0x7fffffffdce8) at common-main.c:52\n\nIt looks like merge_base_index() in split-index.c is calling\newah_each_bit(si->replace_bitmap, replace_entry, istate) when\nsi->replace_bitmap is NULL.\n"},{"id":"396463","messageId":"B48606E4-1ACB-45AE-9C24-4C285C989736@gmail.com","threadId":"53342","inReplyTo":"CAP8UFD32D4WnSbpB1erMTidFpinCNwzqHG_fsyHqRmWgZpsSjA@mail.gmail.com","subject":"Re: Git Stash brake splitIndex","fromName":"Son Luong Ngoc","fromEmail":"sluongng@gmail.com","sentAt":"2020-04-28T14:17:02Z","receivedAt":"2020-04-28T14:17:06Z","isPatch":false,"sender":{"key":"sluongng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/26684313?v=4"},"body":"Hi\n\n> On Apr 28, 2020, at 15:57, Christian Couder wrote:\n> \n> It looks like it should be `git commit -a -m 'add a'`\nI tried to reproduce with `splitIndex.sharedIndexExpire=1.day.ago` and everything works.\n\nIt seems like the config `splitIndex.sharedIndexExpire=now` cause the sharedindex to be deleted too early?\n\nCheers,\nSon Luong.\n\n"},{"id":"396893","messageId":"dcfcdd29-976b-5c8f-190b-ac14dc0abc21@gmail.com","threadId":"53342","inReplyTo":"EED2CFF1-5BEF-429D-AB99-AD148A867614@gmail.com","subject":"Re: Git Stash brake splitIndex","fromName":"Alban Gruin","fromEmail":"alban.gruin@gmail.com","sentAt":"2020-05-04T10:26:08Z","receivedAt":"2020-05-04T10:26:22Z","isPatch":false,"sender":{"key":"alban.gruin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6310153?v=4"},"body":"Hi,\n\nLe 28/04/2020 à 15:19, Son Luong Ngoc a écrit :\n> Hi folks,\n> \n> I am on git version 2.26.2.526.g744177e7f7 (latest next)\n> When you do a git stash while using splitIndex, it seems like the index will get corrupted\n> \n> Using configs:\n> core.splitindex=true\n> splitindex.maxpercentchange=50\n> splitindex.sharedindexexpire=now\n> \n> Reproduce steps:\n> \n> mkdir repo\n> cd repo && git init\n> echo a > a\n> commit -a -m 'add a'\n> echo b > b\n> git add b\n> git stash\n> Saved working directory and index state WIP on master: 1955b62 add a\n> fatal: .git/sharedindex.8ddd8dad6ccb4858f27d4ff20f4d8bf6654441e0: index file open failed: No such file or directory\n> \n> Some traces:\n> master ~/test/git/repo> GIT_TRACE2=1 GIT_TRACE2_NESTING=5 git stash\n> 15:18:15.442295 common-main.c:48 version 2.26.2.526.g744177e7f7\n> 15:18:15.442914 common-main.c:49 start git stash\n> 15:18:15.443284 repository.c:134 worktree /Users/sluongngoc/test/git/repo\n> 15:18:15.443491 git.c:442 cmd_name stash (stash)\n> 15:18:15.448579 run-command.c:735 child_start[0] git update-index --ignore-skip-worktree-entries -z --add --remove --stdin\n> 15:18:15.455972 common-main.c:48 version 2.26.2.526.g744177e7f7\n> 15:18:15.456514 common-main.c:49 start /Users/sluongngoc/libexec/git-core/git update-index --ignore-skip-worktree-entries -z --add --remove --stdin\n> 15:18:15.456788 repository.c:134 worktree /Users/sluongngoc/test/git/repo\n> 15:18:15.456927 git.c:442 cmd_name update-index (stash/update-index)\n> 15:18:15.458444 git.c:672 exit elapsed:0.004021 code:0\n> 15:18:15.458457 trace2/tr2_tgt_normal.c:123 atexit elapsed:0.004039 code:0\n> 15:18:15.458774 run-command.c:990 child_exit[0] pid:1813 code:0 elapsed:0.010169\n> Saved working directory and index state WIP on master: 1955b62 add a\n> 15:18:15.461082 run-command.c:735 child_start[1] git reset --hard -q --no-recurse-submodules\n> 15:18:15.467260 common-main.c:48 version 2.26.2.526.g744177e7f7\n> 15:18:15.467553 common-main.c:49 start /Users/sluongngoc/libexec/git-core/git reset --hard -q --no-recurse-submodules\n> 15:18:15.467931 repository.c:134 worktree /Users/sluongngoc/test/git/repo\n> 15:18:15.468071 git.c:442 cmd_name reset (stash/reset)\n> 15:18:15.468555 usage.c:64 error .git/sharedindex.8ddd8dad6ccb4858f27d4ff20f4d8bf6654441e0: index file open failed: No such file or directory\n> fatal: .git/sharedindex.8ddd8dad6ccb4858f27d4ff20f4d8bf6654441e0: index file open failed: No such file or directory\n> 15:18:15.468587 usage.c:68 exit elapsed:0.002714 code:128\n> 15:18:15.468595 trace2/tr2_tgt_normal.c:123 atexit elapsed:0.002726 code:128\n> 15:18:15.468889 run-command.c:990 child_exit[1] pid:1814 code:128 elapsed:0.007797\n> 15:18:15.468930 git.c:672 exit elapsed:0.028400 code:1\n> 15:18:15.468947 trace2/tr2_tgt_normal.c:123 atexit elapsed:0.028418 code:1\n> exit 1\n> \n\nThank you for the bug report.\n\nLooking at git-stash.c, it uses a second index, which is sometimes a\ncopy of the first. At least, it's the case in stash_working_tree(),\nused by `git stash': in do_create_stash(), &info->i_tree is created to\nsave the content of the index, then save_untracked_files() is skipped as\nwe did not call stash with the `--include-untracked' switch, then\nstash_working_tree() is called as we are not in patch mode. Here, the\nindex did not change. stash_working_tree() then unpacks &info->i_tree\nin a second index.\n\nAfter this, `git update-index' is called, which modifies the second\nindex. This is not a problem in the non-split-index case, because both\nindex are stand alone; they can be modified without interfering with the\nother. But not in the split-index case -- as the second is a copy of\nthe first one, and they both point to the same shared index file.\nModifying the second will delete this file (as the expiration is set to\n\"now\") and create another one. So, anything using the first index will\ntry to open a file that no longer exists, and crash.\n\nHere, `git reset' crashes (in do_create_stash(), after\nstash_working_tree() returned), but adding a `discard_cache();\nread_cache();' after running `update-index' (in stash_working_tree())\nmakes it crash at `read_cache();'.\n\nYour configuration is a bit extreme, but I worry that it may happen with\n`splitindex.sharedindexexpire' set to a higher value, albeit much less\nfrequently.\n\nThis double-index thing came from the shell script, perhaps it's about\ntime to remove it. I worked on a series to remove it this week-end, and\nit fixes this crash. I still have to write the commit messages, then I\nwill send it to the list.\n\n> Cheers,\n> Son Luong.\n> \n\nCheers,\nAlban\n\n"}]}