{"thread":{"id":"51006","subject":"Bug: fatal: Unable to create '.../.git/index.lock': File exists.","startedAt":"2019-04-29T11:02:55Z","lastAt":"2019-05-03T10:22:53Z","messageCount":15,"participants":["Aleksey Midenkov","Duy Nguyen","Johannes Schindelin","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"374635","messageId":"CAF8BazA-VYFns7o9F7gXfFZCspbM0yQKi+LQ+BnkpGH+EjPC9A@mail.gmail.com","threadId":"51006","inReplyTo":"CAF8BazDu_GqoCPBQ-gEJ+q8n1aWSjf_TOV7bDE5VCQkDgBjyfQ@mail.gmail.com","subject":"Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Aleksey Midenkov","fromEmail":"midenok@gmail.com","sentAt":"2019-04-29T11:02:39Z","receivedAt":"2019-04-29T11:02:55Z","isPatch":false,"sender":{"key":"midenok@gmail.com","avatar":null},"body":"Reproduce:\n```\ncat << EOF >> /tmp/check.sh\n#!/bin/sh\ngit log HEAD~..HEAD | cat\n# sleep 1\nEOF\nchmod +x /tmp/check.sh\ngit rebase -p -x /tmp/check.sh base\n```\nIf the `base` is far away enough it fails with \"fatal: Unable to\ncreate '.../.git/index.lock': File exists.\" at an arbitrary commit.\n\nAbort current rebase, uncomment 'sleep 1' and repeat rebase. Now it\ndoesn't fail.\n\nVersion:\ngit version 2.20.1\n\nuname --all\nLinux lian 4.18.0-15-generic #16-Ubuntu SMP Thu Feb 7 10:56:39 UTC\n2019 x86_64 x86_64 x86_64 GNU/Linux\n\n\n-- \nAll the best,\n\nAleksey Midenkov\n@midenok\n"},{"id":"374636","messageId":"CACsJy8DSW2f3v1KpU-QrAz-EeLwG4mVm9ToDdA2=kXSmtsEAYw@mail.gmail.com","threadId":"51006","inReplyTo":"CAF8BazA-VYFns7o9F7gXfFZCspbM0yQKi+LQ+BnkpGH+EjPC9A@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-04-29T11:34:59Z","receivedAt":"2019-04-29T11:35:28Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Apr 29, 2019 at 6:03 PM Aleksey Midenkov wrote:\n>\n> Reproduce:\n> ```\n> cat << EOF >> /tmp/check.sh\n> #!/bin/sh\n> git log HEAD~..HEAD | cat\n> # sleep 1\n> EOF\n> chmod +x /tmp/check.sh\n> git rebase -p -x /tmp/check.sh base\n> ```\n> If the `base` is far away enough it fails with \"fatal: Unable to\n> create '.../.git/index.lock': File exists.\" at an arbitrary commit.\n\nI gave it about 2000 commits (from v2.20.1 to master on git.git) to\nrebase. No luck.\n\nThis --preserve-merges is being completely replaced in latest git\nversion though. Is it possible for you to try master branch of git.git\nand see if the problem is still there?\n\n>\n> Abort current rebase, uncomment 'sleep 1' and repeat rebase. Now it\n> doesn't fail.\n>\n> Version:\n> git version 2.20.1\n>\n> uname --all\n> Linux lian 4.18.0-15-generic #16-Ubuntu SMP Thu Feb 7 10:56:39 UTC\n> 2019 x86_64 x86_64 x86_64 GNU/Linux\n>\n>\n> --\n> All the best,\n>\n> Aleksey Midenkov\n> @midenok\n\n\n\n-- \nDuy\n"},{"id":"374666","messageId":"nycvar.QRO.7.76.6.1904291709350.45@tvgsbejvaqbjf.bet","threadId":"51006","inReplyTo":"CAF8BazA-VYFns7o9F7gXfFZCspbM0yQKi+LQ+BnkpGH+EjPC9A@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-04-29T21:10:54Z","receivedAt":"2019-04-29T21:10:54Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Aleksey,\n\nOn Mon, 29 Apr 2019, Aleksey Midenkov wrote:\n\n> git rebase -p -x /tmp/check.sh base\n> ```\n> If the `base` is far away enough it fails with \"fatal: Unable to\n> create '.../.git/index.lock': File exists.\" at an arbitrary commit.\n\nDoes it work if you pass `-r` instead of `-p`? The latter will be\ndeprecated in favor of the former in the upcoming Git v2.22.\n\nCiao,\nJohannes\n"},{"id":"374699","messageId":"CAF8BazBShg9F2uCuVQ_PM6196kOUNWOA1T9APkCXCoey7as2mQ@mail.gmail.com","threadId":"51006","inReplyTo":"CACsJy8DSW2f3v1KpU-QrAz-EeLwG4mVm9ToDdA2=kXSmtsEAYw@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Aleksey Midenkov","fromEmail":"midenok@gmail.com","sentAt":"2019-04-30T11:19:11Z","receivedAt":"2019-04-30T11:19:26Z","isPatch":false,"sender":{"key":"midenok@gmail.com","avatar":null},"body":"On Mon, Apr 29, 2019 at 2:35 PM Duy Nguyen wrote:\n>\n> On Mon, Apr 29, 2019 at 6:03 PM Aleksey Midenkov wrote:\n> >\n> > Reproduce:\n> > ```\n> > cat << EOF >> /tmp/check.sh\n> > #!/bin/sh\n> > git log HEAD~..HEAD | cat\n> > # sleep 1\n> > EOF\n> > chmod +x /tmp/check.sh\n> > git rebase -p -x /tmp/check.sh base\n> > ```\n> > If the `base` is far away enough it fails with \"fatal: Unable to\n> > create '.../.git/index.lock': File exists.\" at an arbitrary commit.\n>\n> I gave it about 2000 commits (from v2.20.1 to master on git.git) to\n> rebase. No luck.\n\nPlease, try on this repo: git@github.com:tempesta-tech/mariadb\n\n```\ngit checkout 62a082f573\ngit rebase -p -x /tmp/check.sh ca7fbcea6c4\n```\n\nOn Tue, Apr 30, 2019 at 12:10 AM Johannes Schindelin\n wrote:\n...\n>\n> Does it work if you pass `-r` instead of `-p`? The latter will be\n> deprecated in favor of the former in the upcoming Git v2.22.\n\nIt also fails with `-r` but less frequently.\n\n-- \nAll the best,\n\nAleksey Midenkov\n@midenok\n"},{"id":"374703","messageId":"20190430174110.GA16729@sigill.intra.peff.net","threadId":"51006","inReplyTo":"CAF8BazBShg9F2uCuVQ_PM6196kOUNWOA1T9APkCXCoey7as2mQ@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-04-30T17:41:10Z","receivedAt":"2019-04-30T17:41:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 30, 2019 at 02:19:11PM +0300, Aleksey Midenkov wrote:\n\n> > I gave it about 2000 commits (from v2.20.1 to master on git.git) to\n> > rebase. No luck.\n> \n> Please, try on this repo: git@github.com:tempesta-tech/mariadb\n> \n> ```\n> git checkout 62a082f573\n> git rebase -p -x /tmp/check.sh ca7fbcea6c4\n> ```\n\nIt doesn't reproduce for me.\n\nUsually when we see racy contention on index.lock, the culprit turns out\nto be another unrelated git process refreshing the index. Do you have\nanything else running which might be using \"git status\" (e.g., magit in\nemacs, vim git integration, etc)?\n\n-Peff\n"},{"id":"374738","messageId":"CAF8BazBBP53uhh+oOroFuVCEL-FaqJheSYX5Q5_NQxGRt=g_xA@mail.gmail.com","threadId":"51006","inReplyTo":"20190430174110.GA16729@sigill.intra.peff.net","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Aleksey Midenkov","fromEmail":"midenok@gmail.com","sentAt":"2019-05-01T07:15:19Z","receivedAt":"2019-05-01T07:15:34Z","isPatch":false,"sender":{"key":"midenok@gmail.com","avatar":null},"body":"On Tue, Apr 30, 2019 at 8:41 PM Jeff King wrote:\n>\n> On Tue, Apr 30, 2019 at 02:19:11PM +0300, Aleksey Midenkov wrote:\n>\n> > > I gave it about 2000 commits (from v2.20.1 to master on git.git) to\n> > > rebase. No luck.\n> >\n> > Please, try on this repo: git@github.com:tempesta-tech/mariadb\n> >\n> > ```\n> > git checkout 62a082f573\n> > git rebase -p -x /tmp/check.sh ca7fbcea6c4\n> > ```\n>\n> It doesn't reproduce for me.\n>\n> Usually when we see racy contention on index.lock, the culprit turns out\n> to be another unrelated git process refreshing the index. Do you have\n> anything else running which might be using \"git status\" (e.g., magit in\n> emacs, vim git integration, etc)?\n>\n\nkdevelop which is git-aware. But if git fails on concurrent operation\nthis is still not good. I would expect it to wait until lock releases\nfor some time.\n\nI confirm, without kdevelop running it doesn't reproduce.\n\n> -Peff\n\n\n\n-- \nAll the best,\n\nAleksey Midenkov\n@midenok\n"},{"id":"374775","messageId":"20190501183638.GF4109@sigill.intra.peff.net","threadId":"51006","inReplyTo":"CAF8BazBBP53uhh+oOroFuVCEL-FaqJheSYX5Q5_NQxGRt=g_xA@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-01T18:36:38Z","receivedAt":"2019-05-01T18:36:41Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 01, 2019 at 10:15:19AM +0300, Aleksey Midenkov wrote:\n\n> > Usually when we see racy contention on index.lock, the culprit turns out\n> > to be another unrelated git process refreshing the index. Do you have\n> > anything else running which might be using \"git status\" (e.g., magit in\n> > emacs, vim git integration, etc)?\n> \n> kdevelop which is git-aware. But if git fails on concurrent operation\n> this is still not good. I would expect it to wait until lock releases\n> for some time.\n\nHistorically Git does not wait for locks because whoever is holding the\nlock is likely to invalidate the changes we're proposing to make by\ntaking the lock in the first place. We've softened on that a bit in\nrecent years (e.g., ref updates now retry with a timeout to accommodate\nthings like reflog pruning), but I don't think the index code does.\n\nIf the other entity holding the lock is just updating the stat\ninformation in the index, that's probably OK. If it's actually\nmanipulating the index, I think we'd have to give more thought about\nwhether that's safe.\n\nAssuming that kdevelop is just running \"git status\" in the background,\nthough, there's an easier solution. If it uses \"git --no-optional-locks\nstatus\" instead, that will instruct it not to take the index lock at\nall.\n\n-Peff\n"},{"id":"374817","messageId":"CAF8BazAK_s89XY8-AAsSSbgOFgP03CLRZ50bLGPsc89bfnN7kQ@mail.gmail.com","threadId":"51006","inReplyTo":"20190501183638.GF4109@sigill.intra.peff.net","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Aleksey Midenkov","fromEmail":"midenok@gmail.com","sentAt":"2019-05-02T13:45:36Z","receivedAt":"2019-05-02T13:45:51Z","isPatch":false,"sender":{"key":"midenok@gmail.com","avatar":null},"body":"On Wed, May 1, 2019 at 9:36 PM Jeff King wrote:\n>\n> On Wed, May 01, 2019 at 10:15:19AM +0300, Aleksey Midenkov wrote:\n>\n> > > Usually when we see racy contention on index.lock, the culprit turns out\n> > > to be another unrelated git process refreshing the index. Do you have\n> > > anything else running which might be using \"git status\" (e.g., magit in\n> > > emacs, vim git integration, etc)?\n> >\n> > kdevelop which is git-aware. But if git fails on concurrent operation\n> > this is still not good. I would expect it to wait until lock releases\n> > for some time.\n>\n> Historically Git does not wait for locks because whoever is holding the\n> lock is likely to invalidate the changes we're proposing to make by\n> taking the lock in the first place. We've softened on that a bit in\n> recent years (e.g., ref updates now retry with a timeout to accommodate\n> things like reflog pruning), but I don't think the index code does.\n>\n> If the other entity holding the lock is just updating the stat\n> information in the index, that's probably OK. If it's actually\n> manipulating the index, I think we'd have to give more thought about\n> whether that's safe.\n>\n> Assuming that kdevelop is just running \"git status\" in the background,\n> though, there's an easier solution. If it uses \"git --no-optional-locks\n> status\" instead, that will instruct it not to take the index lock at\n> all.\n\nAnd can we disable optional locks at git configuration level? Because\nchanging source code of each application that is not aware of this\noption is not an easier solution.\n\n>\n> -Peff\n\n\n\n-- \nAll the best,\n\nAleksey Midenkov\n@midenok\n"},{"id":"374832","messageId":"20190502150701.GA14906@sigill.intra.peff.net","threadId":"51006","inReplyTo":"CAF8BazAK_s89XY8-AAsSSbgOFgP03CLRZ50bLGPsc89bfnN7kQ@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-02T15:07:01Z","receivedAt":"2019-05-02T15:07:04Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 02, 2019 at 04:45:36PM +0300, Aleksey Midenkov wrote:\n\n> > Assuming that kdevelop is just running \"git status\" in the background,\n> > though, there's an easier solution. If it uses \"git --no-optional-locks\n> > status\" instead, that will instruct it not to take the index lock at\n> > all.\n> \n> And can we disable optional locks at git configuration level? Because\n> changing source code of each application that is not aware of this\n> option is not an easier solution.\n\nSince the decision of whether to use the locks is dependent on the\noperation being performed, it's an environment variable and not a config\noption. You should be able to do:\n\n GIT_OPTIONAL_LOCKS=0 kdevelop\n\nand any commands run by kdevelop will avoid taking locks when they can\n(but for now, the only command which does this is git-status anyway).\n\n-Peff\n"},{"id":"374835","messageId":"CACsJy8Dimn9+ogDNEgy3xmLunyX_pStBq=g-1jrf74LsOW1xrA@mail.gmail.com","threadId":"51006","inReplyTo":"20190502150701.GA14906@sigill.intra.peff.net","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-05-02T16:38:51Z","receivedAt":"2019-05-02T16:39:20Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, May 2, 2019 at 10:07 PM Jeff King wrote:\n>\n> On Thu, May 02, 2019 at 04:45:36PM +0300, Aleksey Midenkov wrote:\n>\n> > > Assuming that kdevelop is just running \"git status\" in the background,\n> > > though, there's an easier solution. If it uses \"git --no-optional-locks\n> > > status\" instead, that will instruct it not to take the index lock at\n> > > all.\n> >\n> > And can we disable optional locks at git configuration level? Because\n> > changing source code of each application that is not aware of this\n> > option is not an easier solution.\n>\n> Since the decision of whether to use the locks is dependent on the\n> operation being performed, it's an environment variable and not a config\n> option.\n\nAnd there's also tradeoff for doing it. If git-status will not take\nlocks, it cannot update the index to save refresh information and\nreuse the next time. git-status may become more and more expensive\nover time (*). Setting a config variable for this does not sound like\na good idea at all. The same for setting GIT_OPTIONAL_LOCKS=0 in\n~/.bashrc to \"fix\" the problem once and for all.\n\nI might take a stab at the \"wait and try to hold the lock again, doing\nnecessary verification after if needed\" idea. It sounds like the right\nway to go and we haven't had problems with refs doing the same thing\n(have we?).\n\n(*) not entirely true since other commands can also refresh and save.\nBut in the ideal world when optional locks are used for all optional\nupdates, it's true.\n\n> You should be able to do:\n>\n> GIT_OPTIONAL_LOCKS=0 kdevelop\n>\n> and any commands run by kdevelop will avoid taking locks when they can\n> (but for now, the only command which does this is git-status anyway).\n-- \nDuy\n"},{"id":"374837","messageId":"20190502165802.GA19341@sigill.intra.peff.net","threadId":"51006","inReplyTo":"CACsJy8Dimn9+ogDNEgy3xmLunyX_pStBq=g-1jrf74LsOW1xrA@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-02T16:58:03Z","receivedAt":"2019-05-02T16:58:08Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 02, 2019 at 11:38:51PM +0700, Duy Nguyen wrote:\n\n> > Since the decision of whether to use the locks is dependent on the\n> > operation being performed, it's an environment variable and not a config\n> > option.\n> \n> And there's also tradeoff for doing it. If git-status will not take\n> locks, it cannot update the index to save refresh information and\n> reuse the next time. git-status may become more and more expensive\n> over time (*). Setting a config variable for this does not sound like\n> a good idea at all. The same for setting GIT_OPTIONAL_LOCKS=0 in\n> ~/.bashrc to \"fix\" the problem once and for all.\n\nRight. I suspect in the long run it might not be _too_ bad to run with\nsuch a setting, because any non-read operations would eventually refresh\nthe index (and as you note even many read-only operations like porcelain\ngit-diff unconditionally refresh for now).\n\nBut I agree it's not really the direction we want to go.\n\n> I might take a stab at the \"wait and try to hold the lock again, doing\n> necessary verification after if needed\" idea. It sounds like the right\n> way to go and we haven't had problems with refs doing the same thing\n> (have we?).\n\nNo, but it's a bit easier with refs because the locking is just\natomically checking the lease. I.e., after taking the lock we still say\n\"we expected the ref to be at oid XYZ, is it still there?\". What's the\nequivalent for an index operation?\n\nI think it is more common with the index to take the lock, then while\nholding it read it in fresh (possibly dumping old results), manipulate\nthe result, and then write it out. For callers which make sure to\nget a fresh view _after_ taking the lock, they should be OK if taking\nthe lock is delayed.\n\nI guess arguably any callers that aren't that careful are already\nbroken, since it is a race; any delay-and-retry _could_ have happened as\n\"we were too slow to see the initial lock\".\n\n-Peff\n"},{"id":"374839","messageId":"CACsJy8D7bx46bix_LmGr=xcwsrA=LehXmLmnONLz2w3q6f80vw@mail.gmail.com","threadId":"51006","inReplyTo":"20190502165802.GA19341@sigill.intra.peff.net","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-05-02T17:24:29Z","receivedAt":"2019-05-02T17:24:58Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, May 2, 2019 at 11:58 PM Jeff King wrote:\n> > I might take a stab at the \"wait and try to hold the lock again, doing\n> > necessary verification after if needed\" idea. It sounds like the right\n> > way to go and we haven't had problems with refs doing the same thing\n> > (have we?).\n>\n> No, but it's a bit easier with refs because the locking is just\n> atomically checking the lease. I.e., after taking the lock we still say\n> \"we expected the ref to be at oid XYZ, is it still there?\". What's the\n> equivalent for an index operation?\n\nThat's something for me to find out :)\n\n> I think it is more common with the index to take the lock, then while\n> holding it read it in fresh (possibly dumping old results), manipulate\n> the result, and then write it out. For callers which make sure to\n> get a fresh view _after_ taking the lock, they should be OK if taking\n> the lock is delayed.\n>\n> I guess arguably any callers that aren't that careful are already\n> broken, since it is a race; any delay-and-retry _could_ have happened as\n> \"we were too slow to see the initial lock\".\n\nI have a feeling that most operations read the index unlocked,\nmanipulate and only lock before writing things out. So yeah it's\nprobably already racy.\n\nWe could use the trailing SHA-1 to determine if the index has not\nchanged since last time, but then refresh-only updates would be\nconsidered valuable while it's not. Full index comparison is way too\nexpensive (at least with giant repos) to even consider. I think I\nstart to see why nobody has done this...\n-- \nDuy\n"},{"id":"374853","messageId":"CACsJy8AWfARPKczV7nGoBz35cBGsuNBtfh5pCFHvjBSB1_HHWg@mail.gmail.com","threadId":"51006","inReplyTo":"20190502165802.GA19341@sigill.intra.peff.net","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-05-03T05:42:16Z","receivedAt":"2019-05-03T05:42:45Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, May 2, 2019 at 11:58 PM Jeff King wrote:\n> > I might take a stab at the \"wait and try to hold the lock again, doing\n> > necessary verification after if needed\" idea. It sounds like the right\n> > way to go and we haven't had problems with refs doing the same thing\n> > (have we?).\n>\n> No, but it's a bit easier with refs because the locking is just\n> atomically checking the lease. I.e., after taking the lock we still say\n> \"we expected the ref to be at oid XYZ, is it still there?\". What's the\n> equivalent for an index operation?\n\nWe could add a second hash that only covers the valuable parts (e.g.\npaths, stage index, SHA-1 and probably some ce_flags). This should be\nenough to verify if the index is still the same as before (stat info\ndoes not count, the same for other cache data like untracked cache,\ncache-tree...).\n\nThis will add some overhead of course because we hash more, especially\non large index files. So it will be optional. The hash is stored in an\nextension. If you run things in parallel and want this, enable it. If\nthe extension is not present in the first place, we don't attempt to\nwait and retry or anything.\n-- \nDuy\n"},{"id":"374868","messageId":"nycvar.QRO.7.76.6.1905031146490.45@tvgsbejvaqbjf.bet","threadId":"51006","inReplyTo":"CACsJy8D7bx46bix_LmGr=xcwsrA=LehXmLmnONLz2w3q6f80vw@mail.gmail.com","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-05-03T09:47:23Z","receivedAt":"2019-05-03T09:47:29Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Duy,\n\nOn Fri, 3 May 2019, Duy Nguyen wrote:\n\n> I have a feeling that most operations read the index unlocked,\n> manipulate and only lock before writing things out. So yeah it's\n> probably already racy.\n\nIIRC there is a check for that, so it is not actually racy ;-)\n\nCiao,\nJohannes\n"},{"id":"374872","messageId":"CACsJy8BcBk6Fsfy7o1aBTQAPeH-w+rBwLZrzw4jJ-OM_jhkxMQ@mail.gmail.com","threadId":"51006","inReplyTo":"nycvar.QRO.7.76.6.1905031146490.45@tvgsbejvaqbjf.bet","subject":"Re: Bug: fatal: Unable to create '.../.git/index.lock': File exists.","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-05-03T10:22:24Z","receivedAt":"2019-05-03T10:22:53Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, May 3, 2019 at 4:47 PM Johannes Schindelin\n wrote:\n>\n> Hi Duy,\n>\n> On Fri, 3 May 2019, Duy Nguyen wrote:\n>\n> > I have a feeling that most operations read the index unlocked,\n> > manipulate and only lock before writing things out. So yeah it's\n> > probably already racy.\n>\n> IIRC there is a check for that, so it is not actually racy ;-)\n\nYeah the update_if_able(), used exclusively for refreshing index. My\nfeeling was wrong actually. Looking around a bit, I think we do take\nthe lock, re-read the index, do things, then write.\n\nThere may be racy spots still. Looking quickly through some\nwrite_locked_index callsites, difftool.c and checkout-index.c may\nleave a gap between loading the index and locking it. Or\nrefresh_and_write_cache() and a couple others in am.c do not look\nexactly race-free.\n\nWe probably should provide an API where locking requires re-reading\nthe index. The version without re-reading has a big fat warning about\ndanger and stuff.\n\nIn any case, i'm getting off topic. I'll stop here.\n-- \nDuy\n"}]}