threads / discuss / 21526

performance issue: git clone compression

Subject: performance issue: git clone compression

## tl;dr

4 messages between Nov 8, 2009 and Nov 8, 2009.

replies: 3people: 2as markdown or json

Tim Webster· Nov 8, 2009, 01:09 UTC · lore
git clone gitosis@gitserver:code.git

After remote counting objects, remote compressing objects transfer begins... This compression operation results in excess of swapage on the remote server.

--------------------

I need to avoiding the clone compression step for servers with limited memory. I am doing the following to avoid this object compression. Is there a better way? -------------------

scp -r gitserver:/srv/gitosis/repositories/code.git  . git clone /tmp/junk/code.git --no-hardlinks code # IS there a better way of converting the bare git repository to working git repository?

edit .git/config url = gitosis@gitserver:code.git

Nicolas Pitre· Nov 8, 2009, 02:17 UTC · re: Tim Webster · lore

Re: performance issue: git clone compression

On Sat, 7 Nov 2009, Tim Webster wrote:
Show 9 quoted lines
> git clone gitosis@gitserver:code.git
> 
> After remote counting objects, remote compressing objects
> transfer begins...
> This compression operation results in excess of swapage on the remote server.
> 
> --------------------
> 
> I need to avoiding the clone compression step for servers with limited memory.

Make sure the remote repository is fully packed. To do so, just go into the remote repository and run 'git gc'.

Nicolas
Tim Webster· Nov 8, 2009, 02:45 UTC · re: Nicolas Pitre · lore

Re: performance issue: git clone compression

On servers with limit memory it is not piratical to run git gc. I can run git gc on a bare mirror and rsync back to the server.

I would prefer to for go compress packing entirely for the large git repositories containing media files.

Can I disable compression by file attribute via .gitattributes?

I am trying to use git for all repositories to avoid a mix of version control methods. For media repositories I am using

[core]
    repositoryformatversion = 0
    filemode = false
    bare = false
    logallrefupdates = true
    compression = 0
    loosecompression = 0
[pack]
    # disable delta-based packing
    depth = 1
    # disable compression
    compression = 0

There is no problem with git pull and push Just git clone swaps the server to death

Show 6 quoted lines
> Make sure the remote repository is fully packed.  To do so, just go into
> the remote repository and run 'git gc'.
>
>
> Nicolas
>
Nicolas Pitre· Nov 8, 2009, 03:04 UTC · re: Tim Webster · lore

Re: performance issue: git clone compression

On Sat, 7 Nov 2009, Tim Webster wrote:
> On servers with limit memory it is not piratical to run git gc.
> I can run git gc on a bare mirror and rsync back to the server.

That is fine. As long as you end up with a single pack, then a clone would basically be a sinple file transfer.

Also you might want to investigate the following config variable for a low-memory server:

	core.packedGitWindowSize
	core.packedGitLimit
	pack.windowMemory
	pack.deltaCacheSize
	pack.deltaCacheLimit
Nicolas

← back to recent threads