Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 9, 2016, 18:06 UTC
Message-ID: <xmqqvaz9akbk.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <20160809174528.2ydgkhd7ayclat3t@sigill.intra.peff.net>

Jeff King <peff@peff.net> writes:

Show 8 quoted lines

> I ran the repack again with stock git (no MRU patch), and the number of
> objects in the delta phase jumped to 3087880, around 56,000 more than
> with the MRU patch. So that's probably where the magic "3%" is coming
> from.  By looking at the smaller packs first, we are more likely to find
> copies of objects which _aren't_ deltas, and then consider them for
> delta compression. And that compression tends to find a better match
> than reusing what was in the big pack, and we end up with a slightly
> smaller output pack.

Yeah, that is a very plausible explanation.

Show 5 quoted lines

> So why are the deltas we find so much better than what is in the big
> pack?
>
> There's something rather subtle going on, and it has to do with the fact
> that the existing pack was _not_ made by a stock version of git.

;-)

Show 5 quoted lines

> I may have mentioned before that I have some patches which restrict the
> delta selection process. The goal is that if you have several "islands"
> of refs (in my case, refs corresponding to particular forks of a
> repository), it's desirable that you don't make a delta against a base
> that is not reachable from all of the same islands.

That also explains it. It is expected (small) price to pay to ensure the islands are standalone.

> I also suspect that one of the tricks I tried, simply reversing the pack
> order (so the big pack is at the front, but the order is stable) would
> work very well for this case. But as I mentioned before, I prefer the
> MRU strategy because it's less susceptible to pathological cases.

Yup, excellent.  Thanks for working on this.