threads / discuss / 1862

slow "stg push" on NFS

Subject: slow "stg push" on NFS

## tl;dr

3 messages between Sep 19, 2005 and Sep 20, 2005.

replies: 2people: 2as markdown or json

Chuck Lever· Sep 19, 2005, 19:01 UTC · lore

hi catalin-

still looking at this problem.

thanks to the several recent patches that add fast-forward support, CPU utilization seems to no longer be a problem in general. what appears to be happening is that each push operation deletes and recreates everything in the working directory.

for shared file systems, creating and deleting files will always be synchronous. anything we can do to eliminate the need for deleting and recreating files that are not changed by a commit would be great.

(i've also noticed that an "stg pop / stg push" sequence causes 'make' to rebuild everything -- possibly due to the same reason?)

am i all wet here?

begin:vcard fn:Chuck Lever n:Lever;Charles org:Network Appliance, Incorporated;Linux NFS Client Development adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA email;internet:cel@citi.umich.edu title:Member of Technical Staff tel;work:+1 734 763 4415 tel;fax:+1 734 763 4434 tel;home:+1 734 668 1089 x-mozilla-html:FALSE url:http://www.monkey.org/~cel/ version:2.1 end:vcard

Catalin Marinas· Sep 20, 2005, 07:24 UTC · re: Chuck Lever · lore

Re: slow "stg push" on NFS

On Mon, 2005-09-19 at 15:01 -0400, Chuck Lever wrote:

> thanks to the several recent patches that add fast-forward support, CPU 
> utilization seems to no longer be a problem in general.

That's a good improvement but this doesn't work when the base of the stack is changed.

> what appears to 
> be happening is that each push operation deletes and recreates 
> everything in the working directory.

The push operation only does a 'git-read-tree -u -m' which updates the filesystem. I haven't looked at the git-read-tree code but I suspect it only updates the files modified as a result of the merge operation. As Blaisorblade reported, if there are big changes between the new HEAD and the old bottom of the patch, like file removals and additions, it will indeed take more time since git-read-tree cannot deal with them and gitmergeonefile.py will be invoked for each file.

> for shared file systems, creating and deleting files will always be 
> synchronous.  anything we can do to eliminate the need for deleting and 
> recreating files that are not changed by a commit would be great.

I did some profiling with pushing 15 patches with 'git-read-tree -m' (without -u for updating the working directory) over NFS and only updating the working files after pushing all the patches. It didn't make much difference and decided that it is not worth fully implementing it. That's mainly because my patches were relatively small (few files changed) and the base of the stack was already checked out, hence just the patch files were being updated in the working directory.

> (i've also noticed that an "stg pop / stg push" sequence causes 'make' 
> to rebuild everything -- possibly due to the same reason?)

The same as 'git checkout branch'. I haven't checked whether the modification time is preserved.

-- 
Catalin

Chuck Lever· Sep 20, 2005, 17:56 UTC · re: Catalin Marinas · lore

Re: slow "stg push" on NFS

Catalin Marinas wrote:

Show 13 quoted lines

> On Mon, 2005-09-19 at 15:01 -0400, Chuck Lever wrote:
>>what appears to 
>>be happening is that each push operation deletes and recreates 
>>everything in the working directory.
> 
> 
> The push operation only does a 'git-read-tree -u -m' which updates the
> filesystem. I haven't looked at the git-read-tree code but I suspect it
> only updates the files modified as a result of the merge operation. As
> Blaisorblade reported, if there are big changes between the new HEAD and
> the old bottom of the patch, like file removals and additions, it will
> indeed take more time since git-read-tree cannot deal with them and
> gitmergeonefile.py will be invoked for each file.

ja, indeed, gitmergeonefile.py is doing some heavy lifting in my case. however, i think the bogosity stems from something my new cache API is doing. i can't reproduce this behavior with stock GIT.