Re: Why repository grows after "git gc"? / Purpose of *.keep files?

From: Shawn O. Pearce <spearce@spearce.org>
Date: May 12, 2008, 23:49 UTC
Message-ID: <20080512234906.GX29038@spearce.org>
In-Reply-To: <e1dab3980805121017u4c244d25s76b39cf015f6c5c5@mail.gmail.com>

David Tweed <david.tweed@gmail.com> wrote:

Show 8 quoted lines

> On Mon, May 12, 2008 at 4:52 PM, Teemu Likonen <tlikonen@iki.fi> wrote:
> > Teemu Likonen wrote (2008-05-12 15:29 +0300):
> > Probably a crazy idea: What if "gc --aggressive" first removed *.keep
> > files and after packing and garbage-collecting and whatever it does it
> > would add a .keep file for the newly created pack?
> 
> My understanding is that the repacking with -a redoes the computation
> to repack ALL the objects in every pack and loose objects,

No. -a means repack all objects in all packs which do not have a .keep on them. Without -a we only repack loose objects.

> whereas
> what would be preferred is to try to delta new objects (loose and
> packed) against the existing .keep pack (extending it with the new
> objects) but not trying to re-deltify objects in the .keep pack.

We cannot do that. Deltas in pack A may not reference base objects in pack B. This is a simplification rule that prevents us from needing to worry about damaging a pack when we repack and delete another pack.

Show 5 quoted lines

> This
> is because .keep files are primarily for those who are cloning onto a
> machine that isn't powerful (maybe even a laptop/palmtop) but who are
> cloning from a powerful server, so that you wouldn't necessarily want
> to apply your strategy unconditionally.

Yes, sort of.  We use .keep for two reasons:

  - As a "lock file" to prevent a pack that was just created by a
    git-fetch or git-recieve-pack from being deleted by a concurrent
    git-repack before the objects it contains are linked into the
    refs space and thus considered reachable;

  - As a way to avoid _huge_ packs (say >1G) that would take a lot
    of disk IO just to copy with 100% delta reuse from an old pack
    to a new pack each time the user runs git-gc.

I think git-clone marking a 150M linux-2.6 pack with .keep is wrong; most users working with the linux-2.6 sources have sufficient hardware to deal with the disk IO required to copy that with 100% delta reuse. But I have a repository at day-job with a 600M pack, that's starting to head into the realm where git-gc while running on battery on a laptop would prefer to have that .keep.

-- 
Shawn.