Re: fetching packs and storing them as packs

From: Junio C Hamano <junkio@cox.net>
Date: Oct 27, 2006, 18:56 UTC
Message-ID: <7vy7r1egfl.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <20061027143854.GC20017@pasky.or.cz>

Petr Baudis <pasky@suse.cz> writes:

Show 14 quoted lines

> Dear diary, on Fri, Oct 27, 2006 at 04:27:05PM CEST, I got a letter
> where Nicolas Pitre <nico@cam.org> said that...
>> On Thu, 26 Oct 2006, Shawn Pearce wrote:
>> > OK so the repository won't get corrupted but the repack would be
>> > forced to abort.
>> 
>> Maybe this is the best way out?  Abort git-repack with "a fetch is in 
>> progress -- retry later".  No one will really suffer if the repack has 
>> to wait for the next scheduled cron job, especially if the fetch doesn't 
>> explode packs into loose objects anymore.
>
> I don't really like this that much. Big projects can have 10 commits per
> hour on average, and they also take potentially long time to repack, so
> you might get to never really repack them.

One question about that statistics is if the frequency of 10 commits per hour is 10 pushes into the central repository per hour or 10 commits distributed all over the world in dozens of developers' repositories.

Even if the number is 10 pushes into the central repository per hour, I do not see it as a big problem in practice from the workflow point of view. Even people sticking to their CVS workflow to have a central repository model are gaining big time from being able to keep working disconnected by switching to git using the shared repository mode, and it should not be a big deal if the central repository master shuts down pushes into the repository for N minutes a day for scheduled repacking. So it could be that a more practical way out is to say "'repack -a -d' and 'prune' are to be run when things are quiescent".

A cron job for the scheduled repack/prune can set a flag (repository wide lockfile or something) to ask new push/fetch to wait and come back later, and we could set up a pre-* hooks for push/fetch to notice it. While push/fetch processes that have already been started can still interfere, as long as they cause repack/prune to fail the "deletion" part, eventually outstanding push/fetch will die out and the cron job will have that quiescent window.