From: Paolo Bonzini <bonzini@gnu.org>
Date: Tue, 13 May 2008 05:08:19 GMT
Subject: Re: Why repository grows after "git gc"? / Purpose of *.keep files?
Message-ID: <48292243.3050307@gnu.org>
In-Reply-To: <20080513000925.GA29038@spearce.org>

Shawn O. Pearce wrote:
> Junio C Hamano <gitster@pobox.com> wrote:
>> "Shawn O. Pearce" <spearce@spearce.org> writes:
>>> I think git-clone marking a 150M linux-2.6 pack with .keep is wrong;
>>> most users working with the linux-2.6 sources have sufficient
>>> hardware to deal with the disk IO required to copy that with 100%
>>> delta reuse.  But I have a repository at day-job with a 600M pack,
>>> that's starting to head into the realm where git-gc while running
>>> on battery on a laptop would prefer to have that .keep.
>> Perhaps clone can decide to keep the .keep file depending on the size of
>> the pack then?
> 
> Yea, I think that's the better thing to do here.  I'm not sure where
> the cut-off is, maybe its <512M delete the .keep once the refs are
> inplace and the objects are ensured to be reachable.

I think separate cutoffs should be in place for file size and number of 
objects.  Very tight packs probably require hours to repack as efficiently.

By the way, another scenario where I used pack files is when I can only 
distribute via http because of firewalls.  I make a clone of the 
original repository and mark the pack as keep; then I push to the 
distribution site, gc, and mark the pack as keep; then I have every day 
a cron job that does git-gc.  This way I know that the user will only 
have to download the third pack.  I think I'll modify the cron job to 
mark as keep the packs that exceed 2 megabytes or something like that.

Thinking about both use cases, the best would be to have options (common 
to git-clone, git-remote add, git-gc at least; and available via config 
keys too) like

   --keep-packs[=THRES1,THRES2,...]

where:

- one threshold would be enough to mark a pack as keep
- thresholds could be in the form "\d+[kmg]?b" for file size, 
"\d+[kmg]?" for number of objects.
- if no threshold is given, the default could be --keep-packs=100k,512MB 
or whatever is in the config.
- to mark all packs, use --keep-packs=0


Paolo