Re: [PATCH 4/7] pack-bitmap: iterate object sources when opening bitmaps
- From
Justin Tobler <jltobler@gmail.com>
- Date
- Jul 9, 2026, 21:08 UTC
- Message-ID
- <alADU8qRZcPB0Zcv@denethor>
- In-Reply-To
- <20260709-pks-odb-for-each-object-filter-v1-4-82fe014b12b3@pks.im>
On 26/07/09 10:35AM, Patrick Steinhardt wrote:
Show 7 quoted lines
> When opening a bitmap for a repository we perform two steps: > > - We first look for a multi-pack index bitmap in any of the object > sources connected to the repository. > > - We then look for a packfile bitmap in any of the packfiles of any of > the object sources.
So IIUC, we generally stop searching for a bitmap once we find one.
Show 9 quoted lines
> Both of these steps thus iterate through object sources themselves, one > via `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This > layout makes it hard to introduce a way to open the bitmap of one > specific object source, which is functionality that we'll require in a > subsequent commit. > > Reverse the loop so that we instead loop through all sources in the > outer loop, and then for each source we try to load its bitmap via > either the multi-pack index or via a packfile.
Conceptually, I think this is a lot easier to follow too which is nice.
Show 10 quoted lines
> Note that this changes the precedence of bitmaps in one specific edge > case: when an earlier object source only has a packfile bitmap, but a > later source has a multi-pack index bitmap, we now pick the packfile > bitmap of the earlier source. Previously, a multi-pack index bitmap from > any source would have taken precedence over all packfile bitmaps. Given > that object sources are ordered such that the local source comes first, > this arguably is an improvement, as we now prefer local bitmaps over > bitmaps in alternates. Furthermore, we already warn about repositories > that have multiple bitmaps, so this setup is broken and thus arguably > not worth worrying about too much.
I agree that the change in bitmap precedent is probably not a big deal. Having multiple bitmaps in a repository is already something we warn against so I think this should be fine.
Show 35 quoted lines
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> pack-bitmap.c | 65 ++++++++++++++++++++++++++---------------------------------
> 1 file changed, 29 insertions(+), 36 deletions(-)
>
> diff --git a/pack-bitmap.c b/pack-bitmap.c
> index eda38a5433..0e3e18a557 100644
> --- a/pack-bitmap.c
> +++ b/pack-bitmap.c
> @@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,
> return 0;
> }
>
> -static int open_pack_bitmap(struct repository *r,
> - struct bitmap_index *bitmap_git)
> +static int open_bitmap_for_source(struct odb_source_packed *source,
> + struct bitmap_index *bitmap_git)
> {
> - struct packed_git *p;
> + struct multi_pack_index *midx = get_multi_pack_index(source);
> + struct packfile_list_entry *e;
> int ret = -1;
>
> - repo_for_each_pack(r, p) {
> - if (open_pack_bitmap_1(bitmap_git, p) == 0) {
> - ret = 0;
> - /*
> - * The only reason to keep looking is to report
> - * duplicates.
> - */
> - if (!trace2_is_enabled())
> - break;
> - }
> + if (midx && !open_midx_bitmap_1(bitmap_git, midx))
> + ret = 0;Ok, open_midx_bitmap_1() returns 0 if it find a MIDX and -1 otherwise. Probably just a matter of preference, but I think writing out like below is a little bit easier on the eyes:
if (midx)
ret = open_midx_bitmap_1(bitmap_git, midx);it might just be that I find the return values a bit confusing though. Maybe we could instead use `found` like a bit later in this patch.
Show 8 quoted lines
> +
> + for (e = packfile_store_get_packs(source); e; e = e->next) {
> + /*
> + * When tracing is enabled we want to keep looking to report
> + * duplicates even if we have already found a bitmap.
> + */
> + if (!ret && !trace2_is_enabled())
> + break;So if have already found a bitmap from the MIDX and tracing is not enabled, we don't continue searching for bitmaps in this source.
Show 53 quoted lines
> +
> + if (open_pack_bitmap_1(bitmap_git, e->pack))
> + continue;
> + ret = 0;
> }
>
> return ret;
> }
>
> -static int open_midx_bitmap(struct repository *r,
> - struct bitmap_index *bitmap_git)
> +static int open_bitmap(struct repository *r,
> + struct bitmap_index *bitmap_git)
> {
> struct odb_source *source;
> - int ret = -1;
> + int found = 0;
>
> assert(!bitmap_git->map);
>
> odb_prepare_alternates(r->objects);
> for (source = r->objects->sources; source; source = source->next) {
> struct odb_source_files *files = odb_source_files_downcast(source);
> - struct multi_pack_index *midx = get_multi_pack_index(files->packed);
> - if (midx && !open_midx_bitmap_1(bitmap_git, midx))
> - ret = 0;
> - }
> - return ret;
> -}
> -
> -static int open_bitmap(struct repository *r,
> - struct bitmap_index *bitmap_git)
> -{
> - int found;
>
> - assert(!bitmap_git->map);
> + found |= !open_bitmap_for_source(files->packed, bitmap_git);
>
> - found = !open_midx_bitmap(r, bitmap_git);
> -
> - /*
> - * these will all be skipped if we opened a midx bitmap; but run it
> - * anyway if tracing is enabled to report the duplicates
> - */
> - if (!found || trace2_is_enabled())
> - found |= !open_pack_bitmap(r, bitmap_git);
> + /*
> + * The only reason to keep looking after having found a bitmap
> + * is to report duplicates.
> + */
> + if (found && !trace2_is_enabled())
> + break;
> + }Ok, we only advance to the next source if tracing is enabled to print warnings for multiple bitmaps. Makes sense.
Overall I quite like the direction of this patch.
-Justin