{"thread":{"id":"1862","subject":"slow \"stg push\" on NFS","startedAt":"2005-09-19T19:01:14Z","lastAt":"2005-09-20T17:56:05Z","messageCount":3,"participants":["Chuck Lever","Catalin Marinas"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"8889","messageId":"432F0AFA.3050401@citi.umich.edu","threadId":"1862","inReplyTo":null,"subject":"slow \"stg push\" on NFS","fromName":"Chuck Lever","fromEmail":"cel@citi.umich.edu","sentAt":"2005-09-19T19:01:14Z","receivedAt":"2005-09-19T19:01:14Z","isPatch":false,"sender":{"key":"cel@citi.umich.edu","avatar":null},"body":"hi catalin-\n\nstill looking at this problem.\n\nthanks to the several recent patches that add fast-forward support, CPU \nutilization seems to no longer be a problem in general.  what appears to \nbe happening is that each push operation deletes and recreates \neverything in the working directory.\n\nfor shared file systems, creating and deleting files will always be \nsynchronous.  anything we can do to eliminate the need for deleting and \nrecreating files that are not changed by a commit would be great.\n\n(i've also noticed that an \"stg pop / stg push\" sequence causes 'make' \nto rebuild everything -- possibly due to the same reason?)\n\nam i all wet here?\n\n\nbegin:vcard\nfn:Chuck Lever\nn:Lever;Charles\norg:Network Appliance, Incorporated;Linux NFS Client Development\nadr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA\nemail;internet:cel@citi.umich.edu\ntitle:Member of Technical Staff\ntel;work:+1 734 763 4415\ntel;fax:+1 734 763 4434\ntel;home:+1 734 668 1089\nx-mozilla-html:FALSE\nurl:http://www.monkey.org/~cel/\nversion:2.1\nend:vcard\n\n"},{"id":"8980","messageId":"1127201053.7019.13.camel@localhost.localdomain","threadId":"1862","inReplyTo":"432F0AFA.3050401@citi.umich.edu","subject":"Re: slow \"stg push\" on NFS","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-20T07:24:13Z","receivedAt":"2005-09-20T07:24:13Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"On Mon, 2005-09-19 at 15:01 -0400, Chuck Lever wrote:\n> thanks to the several recent patches that add fast-forward support, CPU \n> utilization seems to no longer be a problem in general.\n\nThat's a good improvement but this doesn't work when the base of the\nstack is changed.\n\n> what appears to \n> be happening is that each push operation deletes and recreates \n> everything in the working directory.\n\nThe push operation only does a 'git-read-tree -u -m' which updates the\nfilesystem. I haven't looked at the git-read-tree code but I suspect it\nonly updates the files modified as a result of the merge operation. As\nBlaisorblade reported, if there are big changes between the new HEAD and\nthe old bottom of the patch, like file removals and additions, it will\nindeed take more time since git-read-tree cannot deal with them and\ngitmergeonefile.py will be invoked for each file.\n\n> for shared file systems, creating and deleting files will always be \n> synchronous.  anything we can do to eliminate the need for deleting and \n> recreating files that are not changed by a commit would be great.\n\nI did some profiling with pushing 15 patches with 'git-read-tree\n-m' (without -u for updating the working directory) over NFS and only\nupdating the working files after pushing all the patches. It didn't make\nmuch difference and decided that it is not worth fully implementing it.\nThat's mainly because my patches were relatively small (few files\nchanged) and the base of the stack was already checked out, hence just\nthe patch files were being updated in the working directory.\n\n> (i've also noticed that an \"stg pop / stg push\" sequence causes 'make' \n> to rebuild everything -- possibly due to the same reason?)\n\nThe same as 'git checkout branch'. I haven't checked whether the\nmodification time is preserved.\n\n-- \nCatalin\n"},{"id":"9003","messageId":"43304D35.3030601@citi.umich.edu","threadId":"1862","inReplyTo":"1127201053.7019.13.camel@localhost.localdomain","subject":"Re: slow \"stg push\" on NFS","fromName":"Chuck Lever","fromEmail":"cel@citi.umich.edu","sentAt":"2005-09-20T17:56:05Z","receivedAt":"2005-09-20T17:56:05Z","isPatch":false,"sender":{"key":"cel@citi.umich.edu","avatar":null},"body":"Catalin Marinas wrote:\n> On Mon, 2005-09-19 at 15:01 -0400, Chuck Lever wrote:\n>>what appears to \n>>be happening is that each push operation deletes and recreates \n>>everything in the working directory.\n> \n> \n> The push operation only does a 'git-read-tree -u -m' which updates the\n> filesystem. I haven't looked at the git-read-tree code but I suspect it\n> only updates the files modified as a result of the merge operation. As\n> Blaisorblade reported, if there are big changes between the new HEAD and\n> the old bottom of the patch, like file removals and additions, it will\n> indeed take more time since git-read-tree cannot deal with them and\n> gitmergeonefile.py will be invoked for each file.\n\nja, indeed, gitmergeonefile.py is doing some heavy lifting in my case. \nhowever, i think the bogosity stems from something my new cache API is \ndoing.  i can't reproduce this behavior with stock GIT.\n\n\nbegin:vcard\nfn:Chuck Lever\nn:Lever;Charles\norg:Network Appliance, Incorporated;Linux NFS Client Development\nadr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA\nemail;internet:cel@citi.umich.edu\ntitle:Member of Technical Staff\ntel;work:+1 734 763 4415\ntel;fax:+1 734 763 4434\ntel;home:+1 734 668 1089\nx-mozilla-html:FALSE\nurl:http://www.monkey.org/~cel/\nversion:2.1\nend:vcard\n\n"}]}