Re: git gc / git repack not removing unused objects?

From: Nicolas Pitre <nico@fluxnic.net>
Date: Feb 8, 2010, 02:11 UTC
Message-ID: <alpine.LFD.2.00.1002071937000.1681@xanadu.home>
In-Reply-To: <cccedfc61002071540l7019d5d0mc4666dbd81189064@mail.gmail.com>

On Sun, 7 Feb 2010, Jon Nelson wrote:

Show 13 quoted lines

> On Sat, Feb 6, 2010 at 7:16 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> >On Sat, 6 Feb 2010, Jon Nelson wrote:
> >>         packsizelimit = 256m
> >
> > Why are you using this?
> 
> I didn't want my pack files to be too huge. I've bumped that up to 2G.
> 
> >> pack.packsizelimit=2M
> 
> My ~/.gitconfig normally uses 2M for easy rsyncing. In this repo I
> thought the value was overridden by the project's config (which was
> specifying 256m and now specifies 2048m).

Why do you synchronize with rsync? Why not a simple 'git fetch' which is likely to be much faster in any case?

> Suboptimal or not, it still doesn't explain why the repo grows with each repack.

That's strange indeed.  Except that ...

Show 22 quoted lines

> turnip:/mnt/backups.git # git count-objects -v ; git repack -ad ; git
> count-objects -v
> count: 0
> size: 0
> in-pack: 2408195
> packs: 675
> size-pack: 174924053
> prune-packable: 0
> garbage: 0
> 
> Counting objects: 266309, done.
> Compressing objects: 100% (207820/207820), done.
> Writing objects: 100% (266309/266309), done.
> Total 266309 (delta 47751), reused 227730 (delta 45117)
> 
> count: 0
> size: 0
> in-pack: 2674504
> packs: 686
> size-pack: 196517094
> prune-packable: 0
> garbage: 0

You previously had 2408195 packed objects and after the repack this is 
2674504.  The difference is 266309, which incidentally is the number of 
objects 'git repack' counted.

If that 266309 object count is stable between repack attempts (and it should), then something is failing to clean up the old packs. If I remember right, you were playing with modifications to git-repack.sh lately? It is well possible that you broke it.

I tried to simulate your setup but I'm regretting it now. My test repo expanded to 10x its original size with 900 packs. Git is also painfully slow now while I'm trying to repack it properly back into a single pack. However I wasn't able to reproduce your repository growth. Please retest with a vanilla Git version.

I think you should really consider using an unlimited pack size. Not only you'll save quite a lot of disk space, but Git will be much faster too. And Git itself is far more efficient than rsync at keeping Git repositories in sync, regardless of pack sizes (unless you have too many packs of course).

Nicolas