{"thread":{"id":"26142","subject":"False positives in git diff-index","startedAt":"2010-12-27T08:49:40Z","lastAt":"2011-01-06T12:12:34Z","messageCount":12,"participants":["Alexander Gladysh","Zenaan Harkness","Jakub Narebski","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"158615","messageId":"AANLkTimLW+J_rmRsqUQJO-9Gzn7aK0ZHkd1-s=Wg4Vbi@mail.gmail.com","threadId":"26142","inReplyTo":null,"subject":"False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2010-12-27T08:49:40Z","receivedAt":"2010-12-27T08:49:40Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"Hi, list!\n\nI wrote about the related issue earlier:\n\nhttp://lists-archives.org/git/731516-false-positives-from-git-diff-index-when-used-with-git-dir.html\n\nNow I've got a case when I can reproduce this problem each time I try to.\n\nUnfortunately I can not share it or create a minimal example — the\ncase is triggered by a custom complicated automated build process on a\nprivate repository.\n\nAnyway, I'm ready to debug this issue if someone will guide me.\n\nWorkflow:\n\n<...change files in /path/dir1/...>\n(cd /path && git add </path/dir1/>)\n(cd /path && git commit -m <message1>)\n\n... repeat change-add-commit several times for various directories\n(can be the same directory or not) ...\n\n<...generate file /path/dirN/foo...>\n# Accidentally the file is generated the same as it was\n\n(cd /path && git add </path/dirN/>)\n(cd /path && git status) # Refresh index\n(cd /path && git diff-index --exit-code --quiet HEAD -- /path/dirN) #\nIncorrectly reports that there are some changes\n(cd /path && git commit -m <messageN>) # fails, saying that there is\nnothing to commit\n\nIf I insert sleep 10 between git status and git diff-index, the\nproblem goes away.\n\nAny help?\nAlexander.\n"},{"id":"158877","messageId":"AANLkTinDSCPz-oukxzn24hj94d9WpzZ8_64TBHeNTmoG@mail.gmail.com","threadId":"26142","inReplyTo":"AANLkTimLW+J_rmRsqUQJO-9Gzn7aK0ZHkd1-s=Wg4Vbi@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2011-01-04T09:45:33Z","receivedAt":"2011-01-04T09:45:33Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"Nobody is interested?\n\nIs there a way I can get some help with this issue?\n\nThanks,\nAlexander.\n\nOn Mon, Dec 27, 2010 at 11:49, Alexander Gladysh <agladysh@gmail.com> wrote:\n> Hi, list!\n>\n> I wrote about the related issue earlier:\n>\n> http://lists-archives.org/git/731516-false-positives-from-git-diff-index-when-used-with-git-dir.html\n>\n> Now I've got a case when I can reproduce this problem each time I try to.\n>\n> Unfortunately I can not share it or create a minimal example — the\n> case is triggered by a custom complicated automated build process on a\n> private repository.\n>\n> Anyway, I'm ready to debug this issue if someone will guide me.\n>\n> Workflow:\n>\n> <...change files in /path/dir1/...>\n> (cd /path && git add </path/dir1/>)\n> (cd /path && git commit -m <message1>)\n>\n> ... repeat change-add-commit several times for various directories\n> (can be the same directory or not) ...\n>\n> <...generate file /path/dirN/foo...>\n> # Accidentally the file is generated the same as it was\n>\n> (cd /path && git add </path/dirN/>)\n> (cd /path && git status) # Refresh index\n> (cd /path && git diff-index --exit-code --quiet HEAD -- /path/dirN) #\n> Incorrectly reports that there are some changes\n> (cd /path && git commit -m <messageN>) # fails, saying that there is\n> nothing to commit\n>\n> If I insert sleep 10 between git status and git diff-index, the\n> problem goes away.\n>\n> Any help?\n> Alexander.\n>\n"},{"id":"158879","messageId":"AANLkTi=Po7zA1YG-VdN6cZEV+ZF3GYNM9W9CLVXFaE5Z@mail.gmail.com","threadId":"26142","inReplyTo":"AANLkTinDSCPz-oukxzn24hj94d9WpzZ8_64TBHeNTmoG@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Zenaan Harkness","fromEmail":"zen@freedbms.net","sentAt":"2011-01-04T11:47:51Z","receivedAt":"2011-01-04T11:47:51Z","isPatch":false,"sender":{"key":"zen@freedbms.net","avatar":null},"body":"On Tue, Jan 4, 2011 at 20:45, Alexander Gladysh <agladysh@gmail.com> wrote:\n> Nobody is interested?\n\nYour problem set appears that you have a rather gnarly corner case\nissue, arising from your custom build processes. Although git really\nis amazing, I believe you may well be pushing git to its technological\nlimits.\n\nSo your problem could be quite hard to debug, whilst being distinctly\ndifficult to ascertain the root causes.\n\nIt also appears that your custom complicated build process is likely\nprotecting, or at least integral to, your high value corporate process\nassets.\n\nSo _in this case_ you would be remiss to not find a suitable\nconsultant to provide professional and discreet assistance - perhaps\nGitHub.com, as GitHub’s Tender provides both public and _private_\nsupport issue posting, and customized and private training if you and/\nor your colleagues require; you might contact GitHub direct (\nhttps://github.com/contact ) as their Support page does not link\ndirectly to support contract information; oh, and GitHub supports a\nlot of community projects too: their support for our community ought\nbe supported.\n\n<disclaimer> I am _not_ affiliated with GitHub, I do work full time\nwith a human rights association in Australia.\n\nGood luck\nZenaan\n\n\n> Is there a way I can get some help with this issue?\n>\n> Thanks,\n> Alexander.\n>\n> On Mon, Dec 27, 2010 at 11:49, Alexander Gladysh <agladysh@gmail.com> wrote:\n>> Hi, list!\n>>\n>> I wrote about the related issue earlier:\n>>\n>> http://lists-archives.org/git/731516-false-positives-from-git-diff-index-when-used-with-git-dir.html\n>>\n>> Now I've got a case when I can reproduce this problem each time I try to.\n>>\n>> Unfortunately I can not share it or create a minimal example — the\n>> case is triggered by a custom complicated automated build process on a\n>> private repository.\n...\n"},{"id":"158881","messageId":"AANLkTinfbyve-k8xBzDb1sTcXhJGvL_B+auuA8BQSUy2@mail.gmail.com","threadId":"26142","inReplyTo":"AANLkTi=Po7zA1YG-VdN6cZEV+ZF3GYNM9W9CLVXFaE5Z@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2011-01-04T12:01:56Z","receivedAt":"2011-01-04T12:01:56Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"On Tue, Jan 4, 2011 at 14:47, Zenaan Harkness <zen@freedbms.net> wrote:\n> On Tue, Jan 4, 2011 at 20:45, Alexander Gladysh <agladysh@gmail.com> wrote:\n>> Nobody is interested?\n\n> Your problem set appears that you have a rather gnarly corner case\n> issue, arising from your custom build processes. Although git really\n> is amazing, I believe you may well be pushing git to its technological\n> limits.\n\nCommitting few megabytes of data several times per second is\ntechnological limits? I do not believe so.\n\n> So your problem could be quite hard to debug, whilst being distinctly\n> difficult to ascertain the root causes.\n\n> It also appears that your custom complicated build process is likely\n> protecting, or at least integral to, your high value corporate process\n> assets.\n\n> So _in this case_ you would be remiss to not find a suitable\n> consultant to provide professional and discreet assistance - perhaps\n> GitHub.com, as GitHub’s Tender provides both public and _private_\n> support issue posting, and customized and private training if you and/\n> or your colleagues require; you might contact GitHub direct (\n> https://github.com/contact ) as their Support page does not link\n> directly to support contract information; oh, and GitHub supports a\n> lot of community projects too: their support for our community ought\n> be supported.\n\n> <disclaimer> I am _not_ affiliated with GitHub, I do work full time\n> with a human rights association in Australia.\n\nThank you for your opinion.\n\nI view this particular situation as follows:\n\n1. I found a reproducible case for a hard to catch bug in Git. (This\nis a bug in Git, not in my build process.) This bug in its\nintermittent form annoyed me for quite some time — several months at\nleast — and is likely to annoy other users. (I'm not *that* unique!)\n\n2. I can live happily with sleep(0.2) in my deployment code (while\nthis is not very satisfying, it is acceptable — certainly cheaper than\na paid consultant).\n\n3. I'm willing to help Git developers with catching this bug for\nmutual benefit — I will get rid of annoying issue and make my\ndeployment code more robust. Git will, well, be a bit more robust as\nwell.\n\n4. The sole reason I'm pinging back on this bug report is that I'm\nafraid to accidentally lose the data snapshot (or something in\nenvironment) that makes the issue reproducible.\n\n5. If no one is interested, well, that's opensource :-) No hard feelings.\n\nAlexander.\n"},{"id":"158889","messageId":"m339p8dap4.fsf@localhost.localdomain","threadId":"26142","inReplyTo":"AANLkTinfbyve-k8xBzDb1sTcXhJGvL_B+auuA8BQSUy2@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-01-04T14:08:19Z","receivedAt":"2011-01-04T14:08:19Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Alexander Gladysh <agladysh@gmail.com> writes:\n\n> On Tue, Jan 4, 2011 at 14:47, Zenaan Harkness <zen@freedbms.net> wrote:\n> > On Tue, Jan 4, 2011 at 20:45, Alexander Gladysh <agladysh@gmail.com> wrote:\n\n> > > Nobody is interested?\n> \n> > Your problem set appears that you have a rather gnarly corner case\n> > issue, arising from your custom build processes. Although git really\n> > is amazing, I believe you may well be pushing git to its technological\n> > limits.\n> \n> Committing few megabytes of data several times per second is\n> technological limits? I do not believe so.\n\nWell, at least it is not what version control system is about; git is\ndesigned towards manual and not automatic commits, and version control\nof source code.\n\n> > So your problem could be quite hard to debug, whilst being distinctly\n> > difficult to ascertain the root causes.\n\n> 1. I found a reproducible case for a hard to catch bug in Git. (This\n> is a bug in Git, not in my build process.) This bug in its\n> intermittent form annoyed me for quite some time — several months at\n> least — and is likely to annoy other users. (I'm not *that* unique!)\n\nBut it is reproductible to you: from what I understand you didn't find\nsome minimal example to reproduce this issue without need for access\nyour proprietary build process.\n\nAG> Unfortunately I can not share it or create a minimal example ? the\nAG> case is triggered by a custom complicated automated build process on a\nAG> private repository.\n\n> 3. I'm willing to help Git developers with catching this bug for\n> mutual benefit — I will get rid of annoying issue and make my\n> deployment code more robust. Git will, well, be a bit more robust as\n> well.\n\nTo debug it, if you cannot do it yourself, you would have to find git\ndeveloper who is both knowledgeable about fairly deep part of git\ncode, and can work with remote debugging with you at remote.\n\n\nP.S. Somewhere in the depths of git maling list archive (it didn't\nunfortunately made it to \"Interfaces, Frontends and tools\" page on git\nwiki) there is tool/script for anonymizing git repository, to allow\ndebugging of bugs which occurs in some repositories that cannot be\nmade public.  Perhaps something similar could be done for your build\nprocess (you need to reproduce only stat + git part)?\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"158891","messageId":"AANLkTikD5znEPTqR-UeLsusV6sj80vO0dOPuK9QDN6LW@mail.gmail.com","threadId":"26142","inReplyTo":"m339p8dap4.fsf@localhost.localdomain","subject":"Re: False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2011-01-04T14:46:54Z","receivedAt":"2011-01-04T14:46:54Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"On Tue, Jan 4, 2011 at 14:08, Jakub Narebski <jnareb@gmail.com> wrote:\n> Alexander Gladysh <agladysh@gmail.com> writes:\n>> On Tue, Jan 4, 2011 at 14:47, Zenaan Harkness <zen@freedbms.net> wrote:\n>> > On Tue, Jan 4, 2011 at 20:45, Alexander Gladysh <agladysh@gmail.com> wrote:\n\n>> > So your problem could be quite hard to debug, whilst being distinctly\n>> > difficult to ascertain the root causes.\n\n>> 1. I found a reproducible case for a hard to catch bug in Git. (This\n>> is a bug in Git, not in my build process.) This bug in its\n>> intermittent form annoyed me for quite some time — several months at\n>> least — and is likely to annoy other users. (I'm not *that* unique!)\n\n> But it is reproductible to you: from what I understand you didn't find\n> some minimal example to reproduce this issue without need for access\n> your proprietary build process.\n\n> AG> Unfortunately I can not share it or create a minimal example ? the\n> AG> case is triggered by a custom complicated automated build process on a\n> AG> private repository.\n\nYes, that is true. Still, much, much better than intermittent.\n\n>> 3. I'm willing to help Git developers with catching this bug for\n>> mutual benefit — I will get rid of annoying issue and make my\n>> deployment code more robust. Git will, well, be a bit more robust as\n>> well.\n\n> To debug it, if you cannot do it yourself, you would have to find git\n> developer who is both knowledgeable about fairly deep part of git\n> code, and can work with remote debugging with you at remote.\n\nI understand that. But is the second part of requirement is such a\nlarge problem?\n\nAnyway, as I said, if no one will step up, no problem.\n\n> P.S. Somewhere in the depths of git maling list archive (it didn't\n> unfortunately made it to \"Interfaces, Frontends and tools\" page on git\n> wiki) there is tool/script for anonymizing git repository, to allow\n> debugging of bugs which occurs in some repositories that cannot be\n> made public.  Perhaps something similar could be done for your build\n> process (you need to reproduce only stat + git part)?\n\nI remember, somebody advised me to use this tool, when I reported some\nbug some time (maybe a year) ago.\n\nBut, I'm afraid, I do not know how to separate my deployment tool\nlogic (which reproduces the bug) from the repository data. If I did\nknow, I'd come up with a minimal example already. Nothing trivial\n\"along the lines\", that I tried so far, does reproduce it.\n\nAlexander.\n"},{"id":"158943","messageId":"20110105054825.GC5884@sigill.intra.peff.net","threadId":"26142","inReplyTo":"AANLkTinDSCPz-oukxzn24hj94d9WpzZ8_64TBHeNTmoG@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-01-05T05:48:26Z","receivedAt":"2011-01-05T05:48:26Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jan 04, 2011 at 12:45:33PM +0300, Alexander Gladysh wrote:\n\n> > Anyway, I'm ready to debug this issue if someone will guide me.\n> >\n> > Workflow:\n> >\n> > <...change files in /path/dir1/...>\n> > (cd /path && git add </path/dir1/>)\n> > (cd /path && git commit -m <message1>)\n> >\n> > ... repeat change-add-commit several times for various directories\n> > (can be the same directory or not) ...\n> >\n> > <...generate file /path/dirN/foo...>\n> > # Accidentally the file is generated the same as it was\n> >\n> > (cd /path && git add </path/dirN/>)\n> > (cd /path && git status) # Refresh index\n> > (cd /path && git diff-index --exit-code --quiet HEAD -- /path/dirN) #\n> > Incorrectly reports that there are some changes\n> > (cd /path && git commit -m <messageN>) # fails, saying that there is\n> > nothing to commit\n> >\n> > If I insert sleep 10 between git status and git diff-index, the\n> > problem goes away.\n\nIf adding a sleep makes it work, that sounds like a race condition in\ngit. But from the description of your workflow, it should be easy to\nmake a minimal example:\n\n-- >8 --\n#!/bin/sh\n\nrandom() {\n  perl -e 'print int(rand(5))+1, \"\\n\"'\n}\n\nrm -rf repo\nmkdir repo && cd repo && git init\n\nfor i in 1 2 3 4 5; do\n  mkdir dir$i\n  echo initial >dir$i/file\ndone\ngit add .\ngit commit -m initial\n\nwhile true; do\n  for i in 1 2 3 4 5; do\n    random >dir$i/file\n    git add dir$i\n    git update-index --refresh\n    if ! git diff-index --exit-code --quiet HEAD -- dir$i; then\n      if ! git commit -m foo; then\n        echo breakage\n        exit 1\n      fi\n    else\n      echo not bothering to commit\n    fi\n  done\ndone\n-- 8< --\n\nBasically, we generate random data which has a 20% chance of\nbeing the same as what's there. When it is, we should get \"not bothering\nto commit\", but in your error case, we would try to commit (and get \"no\nchanges\").\n\nBut using that script, I can't replicate your problem. Can you try\nrunning it on the same box you're having trouble with? That might at\nleast tell us if it's your environment or something more complex going\non.\n\n-Peff\n"},{"id":"158944","messageId":"AANLkTimQCMr+emw=rXdBKKnD_W9G981zCkboKgiDWxPF@mail.gmail.com","threadId":"26142","inReplyTo":"20110105054825.GC5884@sigill.intra.peff.net","subject":"Re: False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2011-01-05T06:07:35Z","receivedAt":"2011-01-05T06:07:35Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"On Wed, Jan 5, 2011 at 05:48, Jeff King <peff@peff.net> wrote:\n> On Tue, Jan 04, 2011 at 12:45:33PM +0300, Alexander Gladysh wrote:\n>> > Anyway, I'm ready to debug this issue if someone will guide me.\n\n> If adding a sleep makes it work, that sounds like a race condition in\n> git. But from the description of your workflow, it should be easy to\n> make a minimal example:\n\n> -- 8< --\n\n> Basically, we generate random data which has a 20% chance of\n> being the same as what's there. When it is, we should get \"not bothering\n> to commit\", but in your error case, we would try to commit (and get \"no\n> changes\").\n\n> But using that script, I can't replicate your problem. Can you try\n> running it on the same box you're having trouble with? That might at\n> least tell us if it's your environment or something more complex going\n> on.\n\nThank you. I tried it, and, unfortunately, it does not reproduce the problem.\n\nAlexander.\n"},{"id":"158945","messageId":"20110105061515.GA12163@sigill.intra.peff.net","threadId":"26142","inReplyTo":"AANLkTimQCMr+emw=rXdBKKnD_W9G981zCkboKgiDWxPF@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-01-05T06:15:15Z","receivedAt":"2011-01-05T06:15:15Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jan 05, 2011 at 06:07:35AM +0000, Alexander Gladysh wrote:\n\n> > Basically, we generate random data which has a 20% chance of\n> > being the same as what's there. When it is, we should get \"not bothering\n> > to commit\", but in your error case, we would try to commit (and get \"no\n> > changes\").\n> \n> > But using that script, I can't replicate your problem. Can you try\n> > running it on the same box you're having trouble with? That might at\n> > least tell us if it's your environment or something more complex going\n> > on.\n> \n> Thank you. I tried it, and, unfortunately, it does not reproduce the\n> problem.\n\nOh well, thanks for trying.\n\nGoing back to your original reproduction recipe, can you change the\n\"diff-index\" line to actually report on what it thinks is different?\nThat is, drop the \"--quiet\" and have it actually produce a patch?\n\nIt would be interesting to see what is different, and how that compares\nwith the \"git status\" you run just prior to it (and whether it matches\nthe file you \"git add\"ed just above).\n\nYou haven't told us much about your build process. Are you absolutely\nsure that there couldn't be another process on the system manipulating\nthe files between the various runs?\n\nAre you running on top of any special filesystem that might not meet the\nconsistency guarantees we expect (though in that case, I would assume my\ntrivial script would have reproduced).\n\n-Peff\n"},{"id":"158951","messageId":"AANLkTimFbodDe=DMj5kkR749sG8bDTxyjDCtOEvmDkRd@mail.gmail.com","threadId":"26142","inReplyTo":"20110105061515.GA12163@sigill.intra.peff.net","subject":"Re: False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2011-01-05T07:46:19Z","receivedAt":"2011-01-05T07:46:19Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"On Wed, Jan 5, 2011 at 06:15, Jeff King <peff@peff.net> wrote:\n> On Wed, Jan 05, 2011 at 06:07:35AM +0000, Alexander Gladysh wrote:\n\n>> > But using that script, I can't replicate your problem. Can you try\n>> > running it on the same box you're having trouble with? That might at\n>> > least tell us if it's your environment or something more complex going\n>> > on.\n\n>> Thank you. I tried it, and, unfortunately, it does not reproduce the\n>> problem.\n\n> Oh well, thanks for trying.\n\n> Going back to your original reproduction recipe, can you change the\n> \"diff-index\" line to actually report on what it thinks is different?\n> That is, drop the \"--quiet\" and have it actually produce a patch?\n\n----> Rebuilding manifest...\nMaking manifest for .\nGenerating index.html for .\n:100644 100644 483a7292436daecc9bea0ab265ee19d587b14298\n0000000000000000000000000000000000000000\nM\tcluster/localhost-ag/rocks/index.html\n:100644 100644 fcb9ff896fd1a1bd15663fa9be19b250789d4a25\n0000000000000000000000000000000000000000\nM\tcluster/localhost-ag/rocks/manifest\n\nThese are the two files, which are overridden with identical content.\n(See below, looks like I realized who to blame.)\n\nIf I read this correctly, Git tells me that the files are deleted. No?\n\nAnyway, I checked, looks like that files are overridden (by\nfopen(\"name\", \"w\")), never explicitly deleted. If it is important, I\nwill checkout strace.\n\nContunuing with the script:\n\n----> Comitting changed manifest...\n2edcbfabc11f9bbab4fc8c059490cba9ae196d27\n# On branch ag/git-debugging\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working directory)\n#\n#\ttypechange: cluster/localhost-ag/versions/versions-current.lua\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\nSuddenly: no changes.\n\n> It would be interesting to see what is different, and how that compares\n> with the \"git status\" you run just prior to it (and whether it matches\n> the file you \"git add\"ed just above).\n\nGit status before:\n\n$ git status\n# On branch ag/git-debugging\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working directory)\n#\n#\ttypechange: cluster/localhost-ag/versions/versions-current.lua\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n> You haven't told us much about your build process. Are you absolutely\n> sure that there couldn't be another process on the system manipulating\n> the files between the various runs?\n\nNo other process. But see below.\n\n> Are you running on top of any special filesystem that might not meet the\n> consistency guarantees we expect (though in that case, I would assume my\n> trivial script would have reproduced).\n\nAnd here I have to say \"Oops\".\n\nMy apologies, I should have realized this before: my project is\nmounted on VMWare's HGFS.\n\n(That is: VMWare Fusion Ubuntu Guest -> HGFS -> OS X 10.6 Host files.)\n\nThe problem is not reproduced if I copy the project to the native fs\nin the guest machine.\n\nBut the problem is also not reproduced if I execute your script on the HGFS.\n\nSo, does that mean that HGFS violates consistency guarantees?\n\nAlexander.\n"},{"id":"158952","messageId":"20110105080802.GA15997@sigill.intra.peff.net","threadId":"26142","inReplyTo":"AANLkTimFbodDe=DMj5kkR749sG8bDTxyjDCtOEvmDkRd@mail.gmail.com","subject":"Re: False positives in git diff-index","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-01-05T08:08:02Z","receivedAt":"2011-01-05T08:08:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jan 05, 2011 at 07:46:19AM +0000, Alexander Gladysh wrote:\n\n> ----> Rebuilding manifest...\n> Making manifest for .\n> Generating index.html for .\n> :100644 100644 483a7292436daecc9bea0ab265ee19d587b14298\n> 0000000000000000000000000000000000000000\n> M\tcluster/localhost-ag/rocks/index.html\n> :100644 100644 fcb9ff896fd1a1bd15663fa9be19b250789d4a25\n> 0000000000000000000000000000000000000000\n> M\tcluster/localhost-ag/rocks/manifest\n> \n> These are the two files, which are overridden with identical content.\n> (See below, looks like I realized who to blame.)\n> \n> If I read this correctly, Git tells me that the files are deleted. No?\n\nNo, it just means that the files are stat-dirty with respect to the\nindex. For example:\n\n  $ git init\n  $ touch file && git add file && git commit -m one\n  $ touch file\n  $ git diff-files\n  :100644 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0000000000000000000000000000000000000000 M      file\n  $ git update-index --refresh\n  $ git diff-files\n  <no output>\n\nBut in your case, the stat information should be up to date, since you\njust ran update-index. But see below.\n\n> > Are you running on top of any special filesystem that might not meet the\n> > consistency guarantees we expect (though in that case, I would assume my\n> > trivial script would have reproduced).\n> \n> And here I have to say \"Oops\".\n> \n> My apologies, I should have realized this before: my project is\n> mounted on VMWare's HGFS.\n> \n> (That is: VMWare Fusion Ubuntu Guest -> HGFS -> OS X 10.6 Host files.)\n> \n> The problem is not reproduced if I copy the project to the native fs\n> in the guest machine.\n> \n> But the problem is also not reproduced if I execute your script on the HGFS.\n> \n> So, does that mean that HGFS violates consistency guarantees?\n\nHmm. That could be the problem.  It may not violate traditional\nconsistency guarantees, but I wonder if it is returning slightly\ndifferent stat information between the program runs. That would mean\n\"git status\" does an index refresh and puts some stat information in the\nindex, but the followup \"git diff-index\" might see different stat\ninformation.\n\nThat's just a theory, though. You might try the patch below:\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 4f2e890..1b415a3 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -283,6 +283,8 @@ int ie_match_stat(const struct index_state *istate,\n \t\treturn DATA_CHANGED | TYPE_CHANGED | MODE_CHANGED;\n \n \tchanged = ce_match_stat_basic(ce, st);\n+\tif (changed)\n+\t\tfprintf(stderr, \"changed (%u): %s\\n\", changed, ce->name);\n \n \t/*\n \t * Within 1 second of this sequence:\n\nThe number in parentheses is the bitwise-or of the things git found that\ncaused the stat information to be stale (the actual flags are the\n*_CHANGED defines in cache.h, but I was too lazy to write a\npretty-printer). If you can get the output from diff-files for the error\ncase, we can at least see why git thinks the cache is stale.\n\n-Peff\n"},{"id":"159025","messageId":"AANLkTikJY2u2XKDpqwBYLcTwXz1Je6nW1-cffyaw=-fR@mail.gmail.com","threadId":"26142","inReplyTo":"20110105080802.GA15997@sigill.intra.peff.net","subject":"Re: False positives in git diff-index","fromName":"Alexander Gladysh","fromEmail":"agladysh@gmail.com","sentAt":"2011-01-06T12:12:34Z","receivedAt":"2011-01-06T12:12:34Z","isPatch":false,"sender":{"key":"agladysh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/38239?v=4"},"body":"Hi, Jeff,\n\nApologies for delay. Apparently, after all, the problem does not\nreproduce each time (like 33% of the time, or there is some another\nfactor that I did not realized yet). And the build process is quite\nlengthy, so I was not able to gather stats fast enough. But here they\nare, below:\n\nOn Wed, Jan 5, 2011 at 08:08, Jeff King <peff@peff.net> wrote:\n> On Wed, Jan 05, 2011 at 07:46:19AM +0000, Alexander Gladysh wrote:\n\n>> ----> Rebuilding manifest...\n>> Making manifest for .\n>> Generating index.html for .\n>> :100644 100644 483a7292436daecc9bea0ab265ee19d587b14298\n>> 0000000000000000000000000000000000000000\n>> M     cluster/localhost-ag/rocks/index.html\n>> :100644 100644 fcb9ff896fd1a1bd15663fa9be19b250789d4a25\n>> 0000000000000000000000000000000000000000\n>> M     cluster/localhost-ag/rocks/manifest\n\n>> So, does that mean that HGFS violates consistency guarantees?\n>\n> Hmm. That could be the problem.  It may not violate traditional\n> consistency guarantees, but I wonder if it is returning slightly\n> different stat information between the program runs. That would mean\n> \"git status\" does an index refresh and puts some stat information in the\n> index, but the followup \"git diff-index\" might see different stat\n> information.\n>\n> That's just a theory, though. You might try the patch below:\n\n<...>\n\n\n----> Rebuilding manifest...\nMaking manifest for .\nGenerating index.html for .\nchanged (3): cluster/localhost-ag/rocks/index.html\nchanged (3): cluster/localhost-ag/rocks/manifest\nchanged (3): cluster/localhost-ag/rocks/index.html\nchanged (3): cluster/localhost-ag/rocks/manifest\nchanged (115): cluster/localhost-ag/versions/versions-current.lua\nchanged (115): cluster/localhost-ag/versions/versions-current.lua\nchanged (3): cluster/localhost-ag/rocks/index.html\nchanged (3): cluster/localhost-ag/rocks/manifest\n:100644 100644 483a7292436daecc9bea0ab265ee19d587b14298\n0000000000000000000000000000000000000000\nM\tcluster/localhost-ag/rocks/index.html\n:100644 100644 fcb9ff896fd1a1bd15663fa9be19b250789d4a25\n0000000000000000000000000000000000000000\nM\tcluster/localhost-ag/rocks/manifest\n----> Comitting changed manifest...\nchanged (3): cluster/localhost-ag/rocks/index.html\nchanged (3): cluster/localhost-ag/rocks/index.html\nchanged (3): cluster/localhost-ag/rocks/manifest\nchanged (3): cluster/localhost-ag/rocks/manifest\nchanged (115): cluster/localhost-ag/versions/versions-current.lua\nchanged (115): cluster/localhost-ag/versions/versions-current.lua\n9760438c65f7b0293459e622153f235434436ad6\nchanged (115): cluster/localhost-ag/versions/versions-current.lua\n# On branch ag/git-debugging\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working directory)\n#\n#\ttypechange: cluster/localhost-ag/versions/versions-current.lua\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\nHope this makes some sense,\nAlexander.\n"}]}