Re: [PATCH] add `ignore_missing_links` mode to revwalk

From: Jeff King <peff@peff.net>
Date: Apr 1, 2014, 07:54 UTC
Message-ID: <20140401075452.GB22591@sigill.intra.peff.net>
In-Reply-To: <5339E2BD.3090303@fb.com>

On Mon, Mar 31, 2014 at 02:48:45PM -0700, Siddharth Agarwal wrote:

Show 16 quoted lines

> On 03/28/2014 03:00 AM, Jeff King wrote:
> >From: Vicent Marti <tanoku@gmail.com>
> >
> >When pack-objects is computing the reachability bitmap to serve a
> >fetch request, it can erroneously die() if some of the UNINTERESTING
> >objects are not present. Upload-pack throws away HAVE lines from the
> >client for objects we do not have, but we may have a tip object
> >without all of its ancestors (e.g., if the tip is no longer reachable
> >and was new enough to survive a `git prune`, but some of its
> >reachable objects did get pruned).
> 
> Thanks for this patch. It looks pretty sensible.
> 
> Unfortunately, I can't provide feedback on running it in production
> because we've decided to set aside experimenting with bitmaps for a
> bit. I hope to get back to it in a couple of months.

Bummer. Thanks for taking a look at it.

I do think this patch is definitely fixing a bug, and needs to be pursued. We've been running with bitmaps in production on GitHub since last summer, but have never run into this situation. However, I think it is largely caused by our pruning parameters:

  1. We tend not to prune very often, and instead keep unreachable
     objects around as a safety mechanism.

  2. When we do prune, we use a very tight cutoff, rather than the
     default 2-week period. So the window of opportunity is much smaller
     for a repo to prune an object but not its descendant (typically
     either we keep both, or they both get pruned).

So if you do come back to it later, the fix should have filtered through to "master" by then. :)

-Peff