Re: [PATCH 0/7] speeding up cat-file by reordering object access

From: Derrick Stolee <stolee@gmail.com>
Date: Aug 15, 2018, 14:05 UTC
Message-ID: <fc30f98e-0aa0-89d3-ee32-3d86cc5b342a@gmail.com>
In-Reply-To: <20180810230729.GA19090@sigill.intra.peff.net>

On 8/10/2018 7:07 PM, Jeff King wrote:

Show 10 quoted lines

> The general idea is that accessing objects in packfile order is way
> kinder to the delta base cache, and thus way more efficient. See patches
> 4 and 7 in particular for discussion and numbers.
>
> I'm primarily interested in cat-file, so this series is focused there.
> But there may be other callers of for_each_packed_object() who could
> benefit. Most of the existing ones just care about getting the oid, so
> they're better off as-is. It's possible the call in is_promisor_object()
> could benefit, since it calls parse_object() on each entry it visits. I
> didn't experiment with it.

I like this series, and the follow-up. I could not find any problems with it.

One thing that I realized while reading it is that the multi-pack-index is not integrated into the for_each_packed_object method. I was already going to work on some cleanups in that area [1][2].

When using the new flag with the multi-pack-index, I expect that we will want to load the pack-files that are covered by the multi-pack-index (simply, the 'packs' array) and use the same mechanism to traverse them in order. The only "strange" thing about this is that we would see duplicate objects when traversing the pack-files directly but not when traversing the multi-pack-index (since it de-duplicates when indexing).

I hope to have a series working on top of this series by end-of-week.

Thanks,

-Stolee

[1] https://public-inbox.org/git/CAPig+cTU--KrGcv4C_CwBZEuec4dgm_tJqL=CFWKT6vxxR016w@mail.gmail.com/

     Re: [PATCH v4 04/23] multi-pack-index: add 'write' verb

     (Recommends more user-friendly usage reporting in 'git 
multi-pack-index')

[2] https://public-inbox.org/git/20180814222846.GG142615@aiede.svl.corp.google.com/

     [PATCH] partial-clone: render design doc using asciidoc

     (The commit-graph and multi-pack-index docs are not in the 
Makefile, either.)