Re: git pull & git gc
- From
Jeff King <peff@peff.net>
- Date
- Mar 18, 2015, 21:04 UTC
- Message-ID
- <20150318210458.GA3376@peff.net>
- In-Reply-To
- <20150318145815.GN1369@serenity.lan>
On Wed, Mar 18, 2015 at 02:58:15PM +0000, John Keeping wrote:
Show 20 quoted lines
> On Wed, Mar 18, 2015 at 09:41:59PM +0700, Duy Nguyen wrote: > > On Wed, Mar 18, 2015 at 9:33 PM, Duy Nguyen <pclouds@gmail.com> wrote: > > > If not, I made some mistake in analyzing this and we'll start again. > > > > I did make one mistake, the first "gc" should have reduced the number > > of loose objects to zero. Why didn't it.? I'll come back to this > > tomorrow if nobody finds out first :) > > Most likely they are not referenced by anything but are younger than 2 > weeks. > > I saw a similar issue with automatic gc triggering after every operation > when I did something equivalent to: > > git add <lots of files> > git commit > git reset --hard HEAD^ > > which creates a log of unreachable objects which are not old enough to > be pruned.
Yes, this is almost certainly the problem. Though to be pedantic, the command above will still have a reflog entry, so the objects will be reachable (and packed). But there are other variants that don't leave the objects reachable from even reflogs.
I don't know if there is an easy way around this. Auto-gc's object count is making the assumption that running the gc will reduce the number of objects, but obviously it does not always do so. We could do a more thorough check and find the number of actual packable and prune-able objects. The "prune-able" part of that is easy; just omit objects from the count that are newer than 2 weeks. But "packable" is expensive. You would have to compute reachability by walking from the tips. That can take tens of seconds on a large repo.
You could perhaps cut off the walk early when you hit a packed commit (this does not strictly imply that all of the related objects are packed, but it would be good enough for a heuristic). But even that is probably too expensive for "gc --auto".
-Peff
PS Note that in git v2.2.0 and up, prune will leave not only "recent" unreachable objects, but older objects which are reachable from those recent ones (so that we keep or prune whole chunks of history, rather than dropping part and leaving the rest broken). Technically this exacerbates the problem (we keep more objects), though I doubt it makes much difference in practice (most chunks of history were created at similar times, so the mtimes of the whole chunk will be close together).