{"thread":{"id":"25824","subject":"Inexplicably deteriorating performance of Git repositories on Windows","startedAt":"2010-11-23T19:08:41Z","lastAt":"2010-11-28T22:18:05Z","messageCount":21,"participants":["Dun Peal","Wilbert van Dolleweerd","Stephen Bash","Martin Langhoff","Ferry Huberts","Andreas Ericsson","Nguyen Thai Ngoc Duy","Tay Ray Chuan","Joshua Jensen","A Large Angry SCM","Johannes Sixt","Robin Rosenberg"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"156432","messageId":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","threadId":"25824","inReplyTo":null,"subject":"Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-23T19:08:41Z","receivedAt":"2010-11-23T19:08:41Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"Hey,\n\nWe have a bunch of Windows users, unfortunately, and they're using the\nlatest msysGit release (Git-1.7.3.1-preview20101002).\n\nAn interesting issue we've noticed is that the Time To Complete of\ntheir common operations start deteriorating inexplicably, and\nseverely, some time after the clone.\n\nFor instance, immediately after a clone, `git status` takes about\n5-6s. Which is slow compared to Linux (consistent 1-2s), but still\nusable (it's a BIG repo).\n\nHowever, after a reboot (of all things), `git status` latency\nskyrockets to 14-15s, making the repo unusable.\n\nAny idea what's going on? We just recently switched from SVN, and\nthose users are getting really frustrated. BTW, the only real\nalternative I'm aware of, Cygwin's git, is even slower.\n\nThanks, D\n"},{"id":"156434","messageId":"AANLkTikXkWvHrc7=FjePfX5WyyNF1U=KH2DBCU+CcVu6@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Wilbert van Dolleweerd","fromEmail":"wilbert@arentheym.com","sentAt":"2010-11-23T19:12:58Z","receivedAt":"2010-11-23T19:12:58Z","isPatch":false,"sender":{"key":"wilbert@arentheym.com","avatar":"https://gravatar.com/avatar/781148004f6b87d06fb292344721b83d353d1ba510c59b4dce9ac3feba78150e?d=mp&s=160"},"body":"> We have a bunch of Windows users, unfortunately, and they're using the\n> latest msysGit release (Git-1.7.3.1-preview20101002).\n>\n> An interesting issue we've noticed is that the Time To Complete of\n> their common operations start deteriorating inexplicably, and\n> severely, some time after the clone.\n>\n> For instance, immediately after a clone, `git status` takes about\n> 5-6s. Which is slow compared to Linux (consistent 1-2s), but still\n> usable (it's a BIG repo).\n>\n> However, after a reboot (of all things), `git status` latency\n> skyrockets to 14-15s, making the repo unusable.\n>\n> Any idea what's going on?  We just recently switched from SVN, and\n> those users are getting really frustrated. BTW, the only real\n> alternative I'm aware of, Cygwin's git, is even slower.\n\nHow big is your repository? We're using some fairly big repositories\nover here but I haven't seen this behavior with the latest version of\nmsysgit.\n\nKind regards,\n\nWilbert van Dolleweerd\nBlog: http://walkingthestack.wordpress.com/\nTwitter: http://www.twitter.com/wvandolleweerd\n"},{"id":"156440","messageId":"AANLkTimky3Ojc5w3PcCoJOs=NMfMpgUWUDcx+ry6h1dF@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTikXkWvHrc7=FjePfX5WyyNF1U=KH2DBCU+CcVu6@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-23T19:59:17Z","receivedAt":"2010-11-23T19:59:17Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Tue, Nov 23, 2010 at 7:12 PM, Wilbert van Dolleweerd\n wrote:\n> How big is your repository? We're using some fairly big repositories\n> over here but I haven't seen this behavior with the latest version of\n> msysgit.\n\nThe working copy totals about 4GB. The .git directory, tightly packed, is 1GB.\n\n.D\n"},{"id":"156442","messageId":"AANLkTik=s7dxFRTRZdjJD3pLtzRLPT_HwMCR5VWCNKCg@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTimky3Ojc5w3PcCoJOs=NMfMpgUWUDcx+ry6h1dF@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Wilbert van Dolleweerd","fromEmail":"wilbert@arentheym.com","sentAt":"2010-11-23T20:10:40Z","receivedAt":"2010-11-23T20:10:40Z","isPatch":false,"sender":{"key":"wilbert@arentheym.com","avatar":"https://gravatar.com/avatar/781148004f6b87d06fb292344721b83d353d1ba510c59b4dce9ac3feba78150e?d=mp&s=160"},"body":"2010/11/23 Dun Peal :\n\n> On Tue, Nov 23, 2010 at 7:12 PM, Wilbert van Dolleweerd\n> wrote:\n>> How big is your repository? We're using some fairly big repositories\n>> over here but I haven't seen this behavior with the latest version of\n>> msysgit.\n>\n> The working copy totals about 4GB. The .git directory, tightly packed, is 1GB.\n\nI'll see what size our largest repository is at my company first thing\nin the morning.\n\nkind regards,\n\nWilbert van Dolleweerd\nBlog: http://walkingthestack.wordpress.com/\nTwitter: http://www.twitter.com/wvandolleweerd\n"},{"id":"156445","messageId":"32148197.47150.1290543936407.JavaMail.root@mail.hq.genarts.com","threadId":"25824","inReplyTo":"AANLkTimky3Ojc5w3PcCoJOs=NMfMpgUWUDcx+ry6h1dF@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2010-11-23T20:25:36Z","receivedAt":"2010-11-23T20:25:36Z","isPatch":false,"sender":{"key":"bash@genarts.com","avatar":null},"body":"----- Original Message -----\n> From: \"Dun Peal\" \n> To: \"Wilbert van Dolleweerd\" \n> Cc: \"Git ML\" \n> Sent: Tuesday, November 23, 2010 2:59:17 PM\n> Subject: Re: Inexplicably deteriorating performance of Git repositories on Windows\n>\n> On Tue, Nov 23, 2010 at 7:12 PM, Wilbert van Dolleweerd\n> wrote:\n> > How big is your repository? We're using some fairly big repositories\n> > over here but I haven't seen this behavior with the latest version\n> > of msysgit.\n> \n> The working copy totals about 4GB. The .git directory, tightly packed,\n> is 1GB.\n\nWe're working with about a 1.5GB repository, and while I haven't seen an specific msysgit slow downs, I did run into build issues due to Windows anti-virus programs (on-access scans, new files scans, etc). I had to add my development directory to the anti-virus exception list to speed things back up.\n\nThat being said, I do most of my development on Mac and Linux, and msysgit is noticeably slower across the board for me...\n\nThanks,\nStephen\n"},{"id":"156449","messageId":"AANLkTimAxK66U1NRO7=R9Qb4r_rspNHWkkyq1L-z5--A@mail.gmail.com","threadId":"25824","inReplyTo":"32148197.47150.1290543936407.JavaMail.root@mail.hq.genarts.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-23T21:07:24Z","receivedAt":"2010-11-23T21:07:24Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Tue, Nov 23, 2010 at 8:25 PM, Stephen Bash wrote:\n> ----- Original Message -----\n> We're working with about a 1.5GB repository, and while I haven't seen an specific msysgit slow downs, I did run into build issues due to Windows anti-virus programs (on-access scans, new files scans, etc). I had to add my development directory to the anti-virus exception list to speed things back up.\n\nYeah, the performance numbers I mentioned are *after* excluding our AV\nsoftware from that directory.\n\n> That being said, I do most of my development on Mac and Linux, and msysgit is noticeably slower across the board for me...\n\nYup, as mentioned, even under the best case Windows scenario (freshly\ncloned repo) I'm still seeing `git status` latencies that are x2-3\ntimes those of Linux machines.\n\nI don't hope to get Windows' git to be as fast as on Linux; at this\npoint, just making it fast enough to be usable would be an\nachievement: I can't tell developers who just switched from a fast SVN\nsetup to wait for 15s for `git status`, an operation they perform\ndozens of times per day (other operations, like stash, also take\ninvolve long, unworkable waits).\n\n.D\n"},{"id":"156450","messageId":"AANLkTik2+e+9M73RdPT5xha0=QGxD1cVfmLih=bDtCTU@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2010-11-23T21:13:55Z","receivedAt":"2010-11-23T21:13:55Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Tue, Nov 23, 2010 at 2:08 PM, Dun Peal wrote:\n> However, after a reboot (of all things), `git status` latency\n> skyrockets to 14-15s, making the repo unusable.\n\nA reboot clears your cache. Reboot and time the first and second run\nof git status.\n\nThe first one should be slow (due to cold cache), the second one\nshould be roughly as fast as right after a clone.\n\nIf it's not, more debugging may be neeed. But always take your timings\non a well seeded ('hot') cache.\n\ncheers,\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- School Server Architect\n - ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"156451","messageId":"AANLkTin2W4TG9BX41fkGCMTkrDdFeqpotyFFYXqNOSv-@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTik2+e+9M73RdPT5xha0=QGxD1cVfmLih=bDtCTU@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-23T21:17:59Z","receivedAt":"2010-11-23T21:17:59Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Tue, Nov 23, 2010 at 9:13 PM, Martin Langhoff\n wrote:\n> A reboot clears your cache. Reboot and time the first and second run\n> of git status.\n\nThanks, after all this benchmarking I'm well aware of cold and warm\ncaches. The 14-15s timing is unfortunately stable across multiple\nruns, after the cache was warm (without it, we got times much longer\nthan that).\n\n.D\n"},{"id":"156452","messageId":"4CEC36F3.5020908@hupie.com","threadId":"25824","inReplyTo":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Ferry Huberts","fromEmail":"mailings@hupie.com","sentAt":"2010-11-23T21:49:39Z","receivedAt":"2010-11-23T21:49:39Z","isPatch":false,"sender":{"key":"mailings@hupie.com","avatar":"https://gravatar.com/avatar/ca355376c0713475e17ae413a49f6b98bcbc54cd8364cab7113c53107fb839bb?d=mp&s=160"},"body":"Are your users using the 'show my status in the prompt' feature?\nIf so, then disable all but showing the current branch, it makes a whole\nlot of difference :-)\n\nOn 11/23/2010 08:08 PM, Dun Peal wrote:\n> Hey,\n> \n> We have a bunch of Windows users, unfortunately, and they're using the\n> latest msysGit release (Git-1.7.3.1-preview20101002).\n> \n> An interesting issue we've noticed is that the Time To Complete of\n> their common operations start deteriorating inexplicably, and\n> severely, some time after the clone.\n> \n> For instance, immediately after a clone, `git status` takes about\n> 5-6s. Which is slow compared to Linux (consistent 1-2s), but still\n> usable (it's a BIG repo).\n> \n> However, after a reboot (of all things), `git status` latency\n> skyrockets to 14-15s, making the repo unusable.\n> \n> Any idea what's going on? We just recently switched from SVN, and\n> those users are getting really frustrated. BTW, the only real\n> alternative I'm aware of, Cygwin's git, is even slower.\n> \n> Thanks, D\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n\ngrtz\n\n-- \nFerry Huberts\n"},{"id":"156455","messageId":"AANLkTi=UPSaG=sF=zH7eyxWqyPOFSbx0b+-yeZqMdP+1@mail.gmail.com","threadId":"25824","inReplyTo":"4CEC36F3.5020908@hupie.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-23T23:23:46Z","receivedAt":"2010-11-23T23:23:46Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Tue, Nov 23, 2010 at 9:49 PM, Ferry Huberts wrote:\n> Are your users using the 'show my status in the prompt' feature?\n> If so, then disable all but showing the current branch, it makes a whole\n> lot of difference :-)\n\nThanks, we're not :-) .D\n"},{"id":"156479","messageId":"4CECF837.1080404@op5.se","threadId":"25824","inReplyTo":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2010-11-24T11:34:15Z","receivedAt":"2010-11-24T11:34:15Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"On 11/23/2010 08:08 PM, Dun Peal wrote:\n> Hey,\n> \n> We have a bunch of Windows users, unfortunately, and they're using the\n> latest msysGit release (Git-1.7.3.1-preview20101002).\n> \n> An interesting issue we've noticed is that the Time To Complete of\n> their common operations start deteriorating inexplicably, and\n> severely, some time after the clone.\n> \n> For instance, immediately after a clone, `git status` takes about\n> 5-6s. Which is slow compared to Linux (consistent 1-2s), but still\n> usable (it's a BIG repo).\n> \n\nHow many refs (tags and branches) do you have?\nAre the refs packed or loose?\nIf they are loose, does packing them resolve the issue?\nAre you using network-mounted or local storage?\nWhat does the .git/config file look like for a user where git status\nis excruciatingly slow?\nDoes copying the config file from a windows user to a linux user make\ntimings somewhat consistent between various systems?\nDo older version of git perform as poorly?\nHow is the repository laid out (ie, are there any directories with\na ton of files in, or are they spread across multiple directories)?\nHow many .gitignore files are you using, and what do they look like?\n\n> However, after a reboot (of all things), `git status` latency\n> skyrockets to 14-15s, making the repo unusable.\n> \n\nThat's just plain weird, and is almost certainly a system issue.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"156481","messageId":"AANLkTinTisPh-x-JQBMkz-b=RoMJ6PUQ8Hp1VV_uZ=V+@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-11-24T13:32:16Z","receivedAt":"2010-11-24T13:32:16Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Nov 24, 2010 at 2:08 AM, Dun Peal wrote:\n> Hey,\n>\n> We have a bunch of Windows users, unfortunately, and they're using the\n> latest msysGit release (Git-1.7.3.1-preview20101002).\n>\n> An interesting issue we've noticed is that the Time To Complete of\n> their common operations start deteriorating inexplicably, and\n> severely, some time after the clone.\n>\n> For instance, immediately after a clone, `git status` takes about\n> 5-6s. Which is slow compared to Linux (consistent 1-2s), but still\n> usable (it's a BIG repo).\n>\n> However, after a reboot (of all things), `git status` latency\n> skyrockets to 14-15s, making the repo unusable.\n\nDoes assume-unchanged bit (see \"git update-index\") help? I'm not\nsuggesting to use it but it would help determine if the slowdown is\nworktree-related.\n-- \nDuy\n"},{"id":"156483","messageId":"AANLkTim-1uKTVacr1N=9bhZ+=ngggrJS=GD-YNjkSuBR@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTimky3Ojc5w3PcCoJOs=NMfMpgUWUDcx+ry6h1dF@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Tay Ray Chuan","fromEmail":"rctay89@gmail.com","sentAt":"2010-11-24T14:16:29Z","receivedAt":"2010-11-24T14:16:29Z","isPatch":false,"sender":{"key":"rctay89@gmail.com","avatar":"https://avatars.githubusercontent.com/u/61553?v=4"},"body":"On Wed, Nov 24, 2010 at 3:59 AM, Dun Peal wrote:\n> On Tue, Nov 23, 2010 at 7:12 PM, Wilbert van Dolleweerd\n> wrote:\n>> How big is your repository? We're using some fairly big repositories\n>> over here but I haven't seen this behavior with the latest version of\n>> msysgit.\n>\n> The working copy totals about 4GB. The .git directory, tightly packed, is 1GB.\n\nWhat does the structure of your working tree look like? I think the\ndepth might be affecting performance.\n\n-- \nCheers,\nRay Chuan\n"},{"id":"156488","messageId":"4CED488A.2070507@workspacewhiz.com","threadId":"25824","inReplyTo":"AANLkTim-1uKTVacr1N=9bhZ+=ngggrJS=GD-YNjkSuBR@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-11-24T17:16:58Z","receivedAt":"2010-11-24T17:16:58Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"----- Original Message -----\nFrom: Tay Ray Chuan\nDate: 11/24/2010 7:16 AM\n> On Wed, Nov 24, 2010 at 3:59 AM, Dun Peal wrote:\n>> On Tue, Nov 23, 2010 at 7:12 PM, Wilbert van Dolleweerd\n>> wrote:\n>> The working copy totals about 4GB. The .git directory, tightly \n>> packed, is 1GB.\n> What does the structure of your working tree look like? I think the\n> depth might be affecting performance\nWhenever I want to know exactly what is going on with disk access, I \ndownload Process Monitor from http://sysinternals.com/.\n\nIn order to just show disk access, I filter entries that begin with TCP, \nUDP, and Reg out.\n\nJosh\n"},{"id":"156496","messageId":"AANLkTim9hvekD0VmYdM2YfyGCqPe71yHv9PxsvnbNnR0@mail.gmail.com","threadId":"25824","inReplyTo":"4CECF837.1080404@op5.se","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-24T20:10:40Z","receivedAt":"2010-11-24T20:10:40Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Wed, Nov 24, 2010 at 11:34 AM, Andreas Ericsson wrote:\n> How many refs (tags and branches) do you have?\n\nWe have about 15 branches, but thousands of tags (we tag each release,\nand we make those often). However, I'm not sure how that could make\n`git status` slow, since it shouldn't care about tags at all?\n\n> Are the refs packed or loose?\n\nPacked.\n\n> If they are loose, does packing them resolve the issue?\n> Are you using network-mounted or local storage?\n\nLocal storage on all workstations. One machine with those performance\nissues actually utilizes an incredibly fast, cutting edge SSD. It's\ntwice as fast as the other slow machine, but 8s for `git status` is\nstill slow.\n\n> What does the .git/config file look like for a user where git status\n> is excruciatingly slow?\n\nIt's the normal .git/config you get by default on Git 1.7.x when\ncloning from a remote. I can paste it if you want, but doubt it will\ninterest anyone: we didn't modify it.\n\n> Does copying the config file from a windows user to a linux user make\n> timings somewhat consistent between various systems?\n\nNo.\n\n> Do older version of git perform as poorly?\n\nYes.\n\n> How is the repository laid out (ie, are there any directories with\n> a ton of files in, or are they spread across multiple directories)?\n\nGenerally spread across multiple directories.\n\n> How many .gitignore files are you using, and what do they look like?\n\nWe're using 25 of those,\n\n>> However, after a reboot (of all things), `git status` latency\n>> skyrockets to 14-15s, making the repo unusable.\n> That's just plain weird, and is almost certainly a system issue.\n\nYes, it's weird. I'm not sure it's a \"system\" issue since we're seeing\nit across a diverse set of Windows systems: Vista, Windows 7, pretty\nsure I saw XP as well, different hardware and software setups\n(different programs installed and running).\n\n.D\n"},{"id":"156499","messageId":"AANLkTi=DQBYSMhu38asSGxKBktuzeSem5wq_48s64w0F@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTinTisPh-x-JQBMkz-b=RoMJ6PUQ8Hp1VV_uZ=V+@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-24T20:22:11Z","receivedAt":"2010-11-24T20:22:11Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Wed, Nov 24, 2010 at 1:32 PM, Nguyen Thai Ngoc Duy wrote:\n> Does assume-unchanged bit (see \"git update-index\") help? I'm not\n> suggesting to use it but it would help determine if the slowdown is\n> worktree-related.\n\nI will try that. Another possibility, perhaps a bit cleaner and more\nappropriate as a steady-state solution, is to use sparse checkouts.\n\n.D\n"},{"id":"156501","messageId":"AANLkTin4Y35M+tD8sQmd6C_VikBdx_W2GQjrNANAeW3O@mail.gmail.com","threadId":"25824","inReplyTo":"AANLkTim-1uKTVacr1N=9bhZ+=ngggrJS=GD-YNjkSuBR@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-24T20:48:52Z","receivedAt":"2010-11-24T20:48:52Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Wed, Nov 24, 2010 at 2:16 PM, Tay Ray Chuan wrote:\n> What does the structure of your working tree look like? I think the\n> depth might be affecting performance.\n\nI don't think there's anything special about it; it's a big tree, so\nnaturally we have some deep (15-level) directory branches. Any\nparticular reason you suspect the depth?\n\n.D\n"},{"id":"156503","messageId":"AANLkTi=X724OJgUvG0Ggu3OwxyaJprr9CLL+t+x=MbTO@mail.gmail.com","threadId":"25824","inReplyTo":"4CED488A.2070507@workspacewhiz.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Dun Peal","fromEmail":"dunpealer@gmail.com","sentAt":"2010-11-24T21:00:57Z","receivedAt":"2010-11-24T21:00:57Z","isPatch":false,"sender":{"key":"dunpealer@gmail.com","avatar":"https://gravatar.com/avatar/42f3e6a1166eb44e33f24c20ccbe809fa8591413f366dfe3755c8a8d4935ace9?d=mp&s=160"},"body":"On Wed, Nov 24, 2010 at 5:16 PM, Joshua Jensen\n wrote:\n> Whenever I want to know exactly what is going on with disk access, I\n> download Process Monitor from http://sysinternals.com/.\n>\n> In order to just show disk access, I filter entries that begin with TCP,\n> UDP, and Reg out.\n>\n> Josh\n\nThanks, we tried that and we don't see a whole lot of disk activity on\nthe \"fast\" machines.\n\nOne emerging theory is that the \"slow\" Windows machines differ from\nthe \"fast\" ones by how their disk cache works.\n\nSo `git status` on a large tree heavily depends on caching. Without\nit, it would be slow; with it, it's much faster.\n\nWe verified that part since when we reboot a fast Windows machine, the\nfirst run of `git status` is slow (~30s) but the next one is much\nfaster (~5s).\n\nWe see a similar phenomenon on Linux: the first run is always\nsignificantly slower than the others.\n\nOn slow Windows machines, this difference is much less pronounced.\n\nOn a typical \"slow\" machine, if you clone the repo, the first run of\n`git status` on it would already be fast (5s). But then your reboot,\nand the first run is slow, but then it only gets up to 14s. And you\ncan't get back the 5s latency unless you re-clone the repo and status\nthe fresh clone.\n\nSo my theory is that there's a cache that on the \"fast\" machines\naggressively caches the entire tree on a regular `git status` run. On\nsuch a machine, it's enough to run `git status` once, and after that\ninitial cold run, the rest will be warm... until you reboot the\nmachine, rinse, repeat.\n\nOn a slow machine, however, cache isn't so aggressive. It might be\nwrite-oriented. So when you write out a whole new working tree, that\ntree gets cached as it is written. And for the remainder of the\nlifetime of that cache, you get the fully-cached performance you see\non the \"fast\" machines. But then you reboot the machine, and lose the\ncache. And since the caching process isn't aggressive, any number of\n`git status` runs won't get you back to the fully cached state. You\nwill only get that on a newly written working copy.\n\nWhat do you think?\n\n.D\n"},{"id":"156504","messageId":"4CED8127.8060505@gmail.com","threadId":"25824","inReplyTo":"AANLkTi=X724OJgUvG0Ggu3OwxyaJprr9CLL+t+x=MbTO@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2010-11-24T21:18:31Z","receivedAt":"2010-11-24T21:18:31Z","isPatch":false,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"On 11/24/2010 04:00 PM, Dun Peal wrote:\n> On Wed, Nov 24, 2010 at 5:16 PM, Joshua Jensen\n> wrote:\n>> Whenever I want to know exactly what is going on with disk access, I\n>> download Process Monitor from http://sysinternals.com/.\n>>\n>> In order to just show disk access, I filter entries that begin with TCP,\n>> UDP, and Reg out.\n>>\n>> Josh\n>\n> Thanks, we tried that and we don't see a whole lot of disk activity on\n> the \"fast\" machines.\n>\n> One emerging theory is that the \"slow\" Windows machines differ from\n> the \"fast\" ones by how their disk cache works.\n>\n> So `git status` on a large tree heavily depends on caching. Without\n> it, it would be slow; with it, it's much faster.\n>\n> We verified that part since when we reboot a fast Windows machine, the\n> first run of `git status` is slow (~30s) but the next one is much\n> faster (~5s).\n>\n> We see a similar phenomenon on Linux: the first run is always\n> significantly slower than the others.\n>\n> On slow Windows machines, this difference is much less pronounced.\n>\n> On a typical \"slow\" machine, if you clone the repo, the first run of\n> `git status` on it would already be fast (5s). But then your reboot,\n> and the first run is slow, but then it only gets up to 14s. And you\n> can't get back the 5s latency unless you re-clone the repo and status\n> the fresh clone.\n>\n> So my theory is that there's a cache that on the \"fast\" machines\n> aggressively caches the entire tree on a regular `git status` run. On\n> such a machine, it's enough to run `git status` once, and after that\n> initial cold run, the rest will be warm... until you reboot the\n> machine, rinse, repeat.\n>\n> On a slow machine, however, cache isn't so aggressive. It might be\n> write-oriented. So when you write out a whole new working tree, that\n> tree gets cached as it is written. And for the remainder of the\n> lifetime of that cache, you get the fully-cached performance you see\n> on the \"fast\" machines. But then you reboot the machine, and lose the\n> cache. And since the caching process isn't aggressive, any number of\n> `git status` runs won't get you back to the fully cached state. You\n> will only get that on a newly written working copy.\n>\n> What do you think?\n\nHow much memory do the fast and slow machines have? How much memory will \nwindows use for disk caching? Is it possible that your normal work flow \nbetween status' are forcing the caches to pruned due to memory pressure?\n"},{"id":"156508","messageId":"201011242306.17834.j6t@kdbg.org","threadId":"25824","inReplyTo":"AANLkTi=X724OJgUvG0Ggu3OwxyaJprr9CLL+t+x=MbTO@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2010-11-24T22:06:17Z","receivedAt":"2010-11-24T22:06:17Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"On Mittwoch, 24. November 2010, Dun Peal wrote:\n> So my theory is that there's a cache that on the \"fast\" machines\n> aggressively caches the entire tree on a regular `git status` run. On\n> such a machine, it's enough to run `git status` once, and after that\n> initial cold run, the rest will be warm... until you reboot the\n> machine, rinse, repeat.\n>\n> On a slow machine, however, cache isn't so aggressive. It might be\n> write-oriented. So when you write out a whole new working tree, that\n> tree gets cached as it is written. And for the remainder of the\n> lifetime of that cache, you get the fully-cached performance you see\n> on the \"fast\" machines. But then you reboot the machine, and lose the\n> cache. And since the caching process isn't aggressive, any number of\n> `git status` runs won't get you back to the fully cached state. You\n> will only get that on a newly written working copy.\n\nYou can test this theory on a slow machine: You have cloned a repository, \nrebooted, and you are now at 14s per 'git status'. Now:\n\n $ git rm -r .\n $ git reset --hard\n\nerases the worktree and writes it out again (do this only on a clean \ncheckout!). Are you now at 5s per 'git status'?\n\n-- Hannes\n"},{"id":"156779","messageId":"897B500F-7EDE-4A9D-9196-39075A8046B3@dewire.com","threadId":"25824","inReplyTo":"AANLkTimTh7ka21inpovM=qqdWs6j2OcPXVsFh_CMiZ7N@mail.gmail.com","subject":"Re: Inexplicably deteriorating performance of Git repositories on Windows","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2010-11-28T22:18:05Z","receivedAt":"2010-11-28T22:18:05Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"\n23 nov 2010 kl. 20:08 skrev Dun Peal:\n\n> Hey,\n> \n> We have a bunch of Windows users, unfortunately, and they're using the\n> latest msysGit release (Git-1.7.3.1-preview20101002).\n> \n> An interesting issue we've noticed is that the Time To Complete of\n> their common operations start deteriorating inexplicably, and\n> severely, some time after the clone.\n> \n> For instance, immediately after a clone, `git status` takes about\n> 5-6s. Which is slow compared to Linux (consistent 1-2s), but still\n> usable (it's a BIG repo).\n> \n> However, after a reboot (of all things), `git status` latency\n> skyrockets to 14-15s, making the repo unusable.\n> \n> Any idea what's going on? We just recently switched from SVN, and\n> those users are getting really frustrated. BTW, the only real\n> alternative I'm aware of, Cygwin's git, is even slower.\n> \n\nIs the file system badly fragmented? The worst kind is when the MFT is fragmented,\nwhich may give you very bad performance and the defrag that comes with Windows does\nnot fix MFT fragmentation.\n\nIs NTFS compression enabled? I doubt its helpfulness with Git.\n\n-- robin\n"}]}