Re: Keeping unreachable objects in a separate pack instead of loose?

From: Jeff King <peff@peff.net>
Date: Jun 12, 2012, 19:23 UTC
Message-ID: <20120612192318.GC16911@sigill.intra.peff.net>
In-Reply-To: <alpine.LFD.2.02.1206121507120.23555@xanadu.home>

On Tue, Jun 12, 2012 at 03:09:25PM -0400, Nicolas Pitre wrote:

Show 10 quoted lines

> > Jeff King <peff@peff.net> writes:
> > 
> > > We could close it in both cases by tweaking the mtime of the file
> > > containing the object when we decide not to write because the object
> > > already exists.
> > 
> > Though there is always the window between the existence check and the
> > mtime update where pruning can hit you.
> 
> This is a tiny window compared to 2 weeks.

I don't think the race window is actually 2 weeks long. If you have this sequence:

  1. object X becomes unreferenced

  2. 1 week later, you create a new ref that mentions X

  3. >2 weeks later, you run "git prune --expire=2.weeks.ago"

we will not consider the object for pruning in step 3, because it is reachable. The race is more like:

  1. object X becomes unreferenced

  2. >2 weeks later, you run "git prune --expire=2.weeks.ago"

  3. git-prune reads the list of refs

  4. simultaneous to the git-prune, you reference X

  5. git-prune removes X

  6. your reference is now broken

So the race window depends on the time it takes "git prune" to run.

I wonder if git-prune could do a double-check of the refs. Something like:

  1. calculate reachability on all refs

  2. read list of objects to prune, and make a list of unreachable ones

  3. calculate reachability again (which should be very cheap, because
     you can stop when you get to an object you have already seen)

  4. Drop any objects found in (3) from the list in (2), and delete
     items from your list

But I think that still has a race where objects are created before step 2, but are not actually referenced until after step 3. I think doing it safely may actually require a repo-wide prune lock.

-Peff