{"thread":{"id":"18977","subject":"Git update delay","startedAt":"2009-04-20T21:37:12Z","lastAt":"2009-04-20T21:37:12Z","messageCount":1,"participants":["Todd Charron"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"111788","messageId":"200904201737.13101.tcharron@novator.com","threadId":"18977","inReplyTo":null,"subject":"Git update delay","fromName":"Todd Charron","fromEmail":"tcharron@novator.com","sentAt":"2009-04-20T21:37:12Z","receivedAt":"2009-04-20T21:37:12Z","isPatch":false,"sender":{"key":"tcharron@novator.com","avatar":null},"body":"Hi all,\n\nI'm trying to track down a potential git problem we've been having at my \noffice.  Recently, we deployed a branch of code using git.  We deployed this \ncode to a virtual machine running openvz with simfs as  the filesystem (the \nunderlying filesystem on the host machine is ext3 with the noatime option \non).  Kernel is 2.6.18-028stab053-vzserver-hammer. git version 1.6.0.5.\n\nA git checkout of the branch was done and completed.   Upon returning to the \nbash prompt an rsync was done to sync the files out to other servers.  \nHowever, what went out, was not the version of code that was deployed, but \nonly some of the changesets that should have been there.\n\nThe machines also rsync via a cron every night and the evening sync sent out \nthe correct code with no human intervention or rechecking out of code, so it \nappears as if once the git pull was completed (returned to the prompt) the \nchanges had not been fully applied to the filesystem.  The sync then sync'd \nout an incomplete version of the code, which would have later finished \nupdating and corrected itself on the nightly sync.   \n\nUnfortunately, due to the second sync I can no longer prove this as all the \ncode is no up to date. However, we have recently seen other instances that \nindicate a similar delay in the updates taking place on the file system.\n\nExample 1:\n\nuser@dev:~/fol3$ git add modules/fol/local/MgntAuth.pm\nuser@dev:~/fol3$ git commit -m \"fix auth closure bug\"\nCreated commit 55ba2c9: fix auth closure bug\n 1 files changed, 32 insertions(+), 29 deletions(-)\nuser@dev:~/fol3$ git push\nPassword:\nPassword:\nCounting objects: 11, done.\nCompressing objects: 100% (6/6), done.\nWriting objects: 100% (6/6), 859 bytes, done.\nTotal 6 (delta 5), reused 0 (delta 0)\nTo ssh://cvs00t/clients/fol.git/central_git_repo/\n   ba2e848..55ba2c9  ticket_128766 -> ticket_128766\nuser@dev:~/fol3$ git checkout qa_mar2009_1\nM       modules/fol/local/FOL3_catalog_indices.pm\nM       modules/fol/local/FOL3_indices.pm\nM       www/fol/mgnt/Mgnt_indices_custom.html\nSwitched to branch \"qa_mar2009_1\"\nuser@dev:~/fol3$ git pull origin ticket_128766\nPassword:\nFrom ssh://cvs00t/clients/fol.git/central_git_repo\n * branch            ticket_128766 -> FETCH_HEAD\nerror: Entry 'modules/fol/local/MgntAuth.pm' not uptodate. Cannot merge.\nfatal: merging of trees e2b6fe42a504049333ef73b887554cf6afc8aedc and \n7c06608c2a7351522f0f357eacd982168b2179da failed\nMerge with strategy recursive failed.\nuser@dev:~/fol3$ git status\n# On branch qa_mar2009_1\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#\n#       modified:   modules/fol/local/FOL3_catalog_indices.pm\n#       modified:   modules/fol/local/FOL3_indices.pm\n#       modified:   www/fol/mgnt/Mgnt_indices_custom.html\n#\n# Untracked files:\n#   (use \"git add <file>...\" to include in what will be committed)\n#\n#       bin/fol/#test.txt#\n#       modules/fol/local/#FOL3_indices.pm#\n#       update-cache-scripts\n#       www/fol/mgnt/Mgnt_index_custom.html.new\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\nuser@dev:~/fol3$ git pull origin ticket_128766\nPassword:\nFrom ssh://cvs00t/clients/fol.git/central_git_repo\n * branch            ticket_128766 -> FETCH_HEAD\nMerge made by recursive.\n modules/fol/local/MgntAuth.pm |   61 +++++++++++++++++++++-------------------\n 1 files changed, 32 insertions(+), 29 deletions(-)\nuser@dev:~/fol3$ git push\nPassword:\nCounting objects: 13, done.\nCompressing objects: 100% (5/5), done.\nWriting objects: 100% (5/5), 524 bytes, done.\nTotal 5 (delta 4), reused 0 (delta 0)\nTo ssh://cvs00t/clients/fol.git/central_git_repo/\n   94cd7dc..08f276d  qa_mar2009_1 -> qa_mar2009_1\n\nHere, the only thing that changes between the first pull which fails and the \nsecond which succeeds is time.  No other user is pushing or pulling changes \nat this time.  The developer just walked away, came back and tried it again \nand it worked...\n\nExample 2:\n\nuser@dev:~/fol3$ git commit -m \"remove warning\" \nCreated commit b1dd989: remove warning \n 1 files changed, 0 insertions(+), 1 deletions(-) \nuser@dev:~/fol3$ git push \nPassword: \nPassword: \nCounting objects: 11, done. \nCompressing objects: 100% (6/6), done. \nWriting objects: 100% (6/6), 484 bytes, done. \nTotal 6 (delta 5), reused 0 (delta 0) \nTo ssh://cvs00t/clients/fol.git/central_git_repo/ \n  55ba2c9..b1dd989  ticket_128766 -> ticket_128766 \nuser@dev:~/fol3$ git checkout qa_mar2009_1 \ngit pull origin modules/fol/local/Order_system.pm \nSwitched to branch \"qa_mar2009_1\" \nuser@dev:~/fol3$ git pull origin ticket_128766 \nPassword: \nFrom ssh://cvs00t/clients/fol.git/central_git_repo \n * branch            ticket_128766 -> FETCH_HEAD \nfatal: unable to create '.git/index.lock': File exists \nMerge with strategy recursive failed. \n\nHere we see an index.lock file still existing when it should not.  Again, \nwaiting a minute or two and trying again resolved the issue.  These were all \ndone on a local filesystem (no-NFS) with one user in isolation.\n\nThese instances seem to happen at random and we have been unable to \nconsistently reproduce them.  In all cases, just waiting a minute or two \nbefore running the commands again seems to resolve the issue.\n\nI can't seem to determine whether this is a git issue or a filesystem issue \nand if so, what is the cause.  Can anyone think of any reason why this would \nhappen?\n\nThanks!\n\nTodd\n"}]}