{"thread":{"id":"25164","subject":"git performance after directory copy","startedAt":"2010-09-20T09:20:44Z","lastAt":"2010-09-20T13:57:26Z","messageCount":8,"participants":["Gaer, A.","Michael J Gruber","Matthieu Moy","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"151097","messageId":"C5A8FDEFF7647F4C9CB927D7DEB307730F33B7C9@ahr075s.basler.corp","threadId":"25164","inReplyTo":null,"subject":"git performance after directory copy","fromName":"Gaer, A.","fromEmail":"andreas.gaer@baslerweb.com","sentAt":"2010-09-20T09:20:44Z","receivedAt":"2010-09-20T09:20:44Z","isPatch":false,"sender":{"key":"andreas.gaer@baslerweb.com","avatar":null},"body":"Hello all,\n\nwhile moving a project directory around I stumbled over an interesting\nphenomenon. On a copied source directory \"git status\" seems to be about\n3 times slower than on the original directory. Only after a \"git reset\"\nboth copies behave the same again. Is this connected to the timestamps\nof files & directories? Actually I would like to move the project\ndirectories of several software developers to a new partition and\nforcing them all to \"git reset\" in all of their repos is a little bit\nannoying. Any suggestions how to \"repair\" the repos less intrusive?\n\nHere's how I measured. The trees reside on an ext3 FS. I have lots of\nfree RAM, so after the first operation all further \"git status\" seem to\nrun from FS cache in RAM.\n\n$ git clone <path to your preferred kernel>\n$ cd kernel\n$ time git status # several times!\n...\n$ time git status\n# On branch master\nnothing to commit (working directory clean)\n\nreal    0m0.691s\nuser    0m0.256s\nsys     0m0.356s\n\n$ cd ..\n$ rsync -a kernel/ kernel2/\n$ cd kernel2\n$ time git status # several times!\n...\n$ time git status\n# On branch master\nnothing to commit (working directory clean)\n\nreal    0m2.705s\nuser    0m1.724s\nsys     0m0.816s\n\n$ git reset\n$ time git status\n# On branch master\nnothing to commit (working directory clean)\n\nreal    0m0.704s\nuser    0m0.296s\nsys     0m0.348s\n\nRegards,\n Andreas.\n"},{"id":"151102","messageId":"4C972D49.9040306@drmicha.warpmail.net","threadId":"25164","inReplyTo":"C5A8FDEFF7647F4C9CB927D7DEB307730F33B7C9@ahr075s.basler.corp","subject":"Re: git performance after directory copy","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2010-09-20T09:45:45Z","receivedAt":"2010-09-20T09:45:45Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Gaer, A. venit, vidit, dixit 20.09.2010 11:20:\n> Hello all,\n> \n> while moving a project directory around I stumbled over an interesting\n> phenomenon. On a copied source directory \"git status\" seems to be about\n> 3 times slower than on the original directory. Only after a \"git reset\"\n> both copies behave the same again. Is this connected to the timestamps\n> of files & directories? Actually I would like to move the project\n> directories of several software developers to a new partition and\n> forcing them all to \"git reset\" in all of their repos is a little bit\n> annoying. Any suggestions how to \"repair\" the repos less intrusive?\n> \n\nSince you clone from A to B, then copy from B several times, why don't\nyou clone from A several times instead?\n\nIf it's really about moving across file-system boundaries, then I don't\nthink there's a way around: you need to refresh the index with the\nchanged inodes information.\n\nMichael\n"},{"id":"151103","messageId":"vpq62y0hh63.fsf@bauges.imag.fr","threadId":"25164","inReplyTo":"C5A8FDEFF7647F4C9CB927D7DEB307730F33B7C9@ahr075s.basler.corp","subject":"Re: git performance after directory copy","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2010-09-20T09:56:36Z","receivedAt":"2010-09-20T09:56:36Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"\"Gaer, A.\" <Andreas.Gaer@baslerweb.com> writes:\n\n> On a copied source directory \"git status\" seems to be about\n> 3 times slower than on the original directory.\n\nIt is expected that the first \"git status\" be slower. It will most\nlikely have to actually re-diff the files and update the index\nstat-cache.\n\nBut I'm surprised that the next \"git status\" are still slow. Other\npeople may get a better explanation, but this very much looks like a\nbug.\n\nAre you sure you don't have any permission problem, like a read-only\n.git/index?\n\nTo investigate a bit, you can play with diff.autorefreshindex. When\nset to false, \"git diff\" will tell you about the files which are\nidentical, but do not have the same stat information (and hence, that\nGit has to re-diff). For example:\n\n$ git config diff.autorefreshindex false\n$ git diff\n\n# no output: bar.txt exists, but is up to date.\n\n$ touch bar.txt\n$ git diff     \ndiff --git a/bar.txt b/bar.txt\n\n# No actual diff, but the file appears since its stat information is\n# different.\n\n$ git diff\ndiff --git a/bar.txt b/bar.txt\n\n# Further \"git diff\" behave the same.\n\n$ git status\n...\n$ git diff  \n\n# git status did update the stat-cache, hence, no output from git diff\n# anymore.\n\nAnother diagnosis tool would be \"strace -e open git status,lstat64\".\nFor example:\n\n$ touch bar.txt \n$ strace -e open,lstat64 git status |& grep bar.txt\nlstat64(\"bar.txt\", {st_mode=S_IFREG|0644, st_size=30, ...}) = 0\nopen(\"bar.txt\", O_RDONLY|O_LARGEFILE)   = 3\n$ strace -e open,lstat64 git status |& grep bar.txt\nlstat64(\"bar.txt\", {st_mode=S_IFREG|0644, st_size=30, ...}) = 0\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"151106","messageId":"4C973D5F.4080807@drmicha.warpmail.net","threadId":"25164","inReplyTo":"vpq62y0hh63.fsf@bauges.imag.fr","subject":"Re: git performance after directory copy","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2010-09-20T10:54:23Z","receivedAt":"2010-09-20T10:54:23Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Matthieu Moy venit, vidit, dixit 20.09.2010 11:56:\n> \"Gaer, A.\" <Andreas.Gaer@baslerweb.com> writes:\n> \n>> On a copied source directory \"git status\" seems to be about\n>> 3 times slower than on the original directory.\n> \n> It is expected that the first \"git status\" be slower. It will most\n> likely have to actually re-diff the files and update the index\n> stat-cache.\n> \n> But I'm surprised that the next \"git status\" are still slow. Other\n> people may get a better explanation, but this very much looks like a\n> bug.\n\nIn the OP's case, the inode/dev info differs (after copying, or after\nmoving across FS boundaries). I don't think \"git status\" updates these.\nI noticed (and reported) something like this a while ago when, after a\nreboot, I would get warnings about crossing FS boundaries. Only a \"rm\n.git/index && git reset --hard\" was able to refresh the index properly.\nThis case seems to be similar, although a simple \"git reset --hard\"\nappears to suffice.\n\nMichael\n"},{"id":"151110","messageId":"C5A8FDEFF7647F4C9CB927D7DEB307730F33B85B@ahr075s.basler.corp","threadId":"25164","inReplyTo":"vpq62y0hh63.fsf@bauges.imag.fr","subject":"AW: git performance after directory copy","fromName":"Gaer, A.","fromEmail":"andreas.gaer@baslerweb.com","sentAt":"2010-09-20T11:37:23Z","receivedAt":"2010-09-20T11:37:23Z","isPatch":false,"sender":{"key":"andreas.gaer@baslerweb.com","avatar":null},"body":"Hello all,\n\nthanks Matthieu for your tip. \"git diff\" without \"autorefreshindex\"\nshows that all files have different stat info. Calling \"git update-index\n--refresh\" seems to be the right cure.\n\nWhat I find a little bit confusing is that \"git status\" does not warn\nabout such a situation, or even \"repair\" it. As I said, I wanted to move\nthe project directories of several developers to a new partition without\ntoo much interference and the first try didn't succeed because people\ncomplained about \"bad\" \"git status\" performance (people get used to git\nrocket-fast performance very soon ;-).\n\nMaybe this was introduced in the 1.7 release: \"git status\" is not \"git\ncommit --dry-run\" anymore. \"git commit --dry-run\" does behave as you\nexpect: first call takes a little bit longer, subsequent calls are fast\nagain.\n\nBTW, I tested on a system with git version 1.7.1 installed, but release\nnotes do not suggest any changes in that respect in 1.7.2 or 1.7.3.\n\nRegards,\n Andreas.\n"},{"id":"151112","messageId":"4C974A00.8030905@viscovery.net","threadId":"25164","inReplyTo":"vpq62y0hh63.fsf@bauges.imag.fr","subject":"Re: git performance after directory copy","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2010-09-20T11:48:16Z","receivedAt":"2010-09-20T11:48:16Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 9/20/2010 11:56, schrieb Matthieu Moy:\n> But I'm surprised that the next \"git status\" are still slow. Other\n> people may get a better explanation, but this very much looks like a\n> bug.\n\nMost likely, Andreas works with 1.7.1. From the release notes of 1.7.1.1:\n\n * \"git status\" stopped refreshing the index by mistake in 1.7.1.\n\n-- Hannes\n"},{"id":"151113","messageId":"C5A8FDEFF7647F4C9CB927D7DEB307730F33B865@ahr075s.basler.corp","threadId":"25164","inReplyTo":"4C974A00.8030905@viscovery.net","subject":"AW: git performance after directory copy","fromName":"Gaer, A.","fromEmail":"andreas.gaer@baslerweb.com","sentAt":"2010-09-20T11:53:15Z","receivedAt":"2010-09-20T11:53:15Z","isPatch":false,"sender":{"key":"andreas.gaer@baslerweb.com","avatar":null},"body":"Hello Johannes,\n\nyou are right. Sorry. I totally overlooked release notes for 1.7.1.1. Thanks for all your help!\n\nRegards,\n Andreas.\n \n\n-----Ursprüngliche Nachricht-----\nVon: Johannes Sixt [mailto:j.sixt@viscovery.net] \nGesendet: Montag, 20. September 2010 13:48\nAn: Matthieu Moy\nCc: Gaer, A.; git@vger.kernel.org\nBetreff: Re: git performance after directory copy\n\nAm 9/20/2010 11:56, schrieb Matthieu Moy:\n> But I'm surprised that the next \"git status\" are still slow. Other\n> people may get a better explanation, but this very much looks like a\n> bug.\n\nMost likely, Andreas works with 1.7.1. From the release notes of 1.7.1.1:\n\n * \"git status\" stopped refreshing the index by mistake in 1.7.1.\n\n-- Hannes\n"},{"id":"151118","messageId":"vpqeico8qm1.fsf@bauges.imag.fr","threadId":"25164","inReplyTo":"4C974A00.8030905@viscovery.net","subject":"Re: git performance after directory copy","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2010-09-20T13:57:26Z","receivedAt":"2010-09-20T13:57:26Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Johannes Sixt <j.sixt@viscovery.net> writes:\n\n> Am 9/20/2010 11:56, schrieb Matthieu Moy:\n>> But I'm surprised that the next \"git status\" are still slow. Other\n>> people may get a better explanation, but this very much looks like a\n>> bug.\n>\n> Most likely, Andreas works with 1.7.1. From the release notes of 1.7.1.1:\n>\n>  * \"git status\" stopped refreshing the index by mistake in 1.7.1.\n\nNice catch, and this explains why I couldn't reproduce with latest\nGit.\n\nMore precisely, it was fixed here:\n\nb2f6fd9 t7508: add a test for \"git status\" in a read-only repository\n4bb6644 git status: refresh the index if possible\n4c926b3 t7508: add test for \"git status\" refreshing the index\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"}]}