Re: Why repository grows after "git gc"? / Purpose of *.keep files?

From: Paolo Bonzini <bonzini@gnu.org>
Date: May 13, 2008, 05:08 UTC
Message-ID: <48292243.3050307@gnu.org>
In-Reply-To: <20080513000925.GA29038@spearce.org>

Shawn O. Pearce wrote:

Show 14 quoted lines

> Junio C Hamano <gitster@pobox.com> wrote:
>> "Shawn O. Pearce" <spearce@spearce.org> writes:
>>> I think git-clone marking a 150M linux-2.6 pack with .keep is wrong;
>>> most users working with the linux-2.6 sources have sufficient
>>> hardware to deal with the disk IO required to copy that with 100%
>>> delta reuse.  But I have a repository at day-job with a 600M pack,
>>> that's starting to head into the realm where git-gc while running
>>> on battery on a laptop would prefer to have that .keep.
>> Perhaps clone can decide to keep the .keep file depending on the size of
>> the pack then?
> 
> Yea, I think that's the better thing to do here.  I'm not sure where
> the cut-off is, maybe its <512M delete the .keep once the refs are
> inplace and the objects are ensured to be reachable.

I think separate cutoffs should be in place for file size and number of objects. Very tight packs probably require hours to repack as efficiently.

By the way, another scenario where I used pack files is when I can only distribute via http because of firewalls. I make a clone of the original repository and mark the pack as keep; then I push to the distribution site, gc, and mark the pack as keep; then I have every day a cron job that does git-gc. This way I know that the user will only have to download the third pack. I think I'll modify the cron job to mark as keep the packs that exceed 2 megabytes or something like that.

Thinking about both use cases, the best would be to have options (common to git-clone, git-remote add, git-gc at least; and available via config keys too) like

   --keep-packs[=THRES1,THRES2,...]

where:

- one threshold would be enough to mark a pack as keep
- thresholds could be in the form "\d+[kmg]?b" for file size, 
"\d+[kmg]?" for number of objects.
- if no threshold is given, the default could be --keep-packs=100k,512MB 
or whatever is in the config.
- to mark all packs, use --keep-packs=0

Paolo